When a system fails correctly

This week, I ran into a rather peculiar problem on one of my automated systems.

A migration had just been carried out. The deployment was confirmed, the connections were working and the first tests were conclusive.

At first glance, everything was fine.

Yet some tasks were no longer being carried out.

No crash, no abrupt interruption, no particularly alarming error message.

The system seemed to be working.

But it was no longer doing everything that was expected of it.

A perfectly executed failure #

Going back through the execution logs, I eventually understood what was happening.

Part of the permissions needed to operate in production hadn’t been properly checked after the migration.

When an operation required those permissions, the system hit an error.

It then applied exactly the behaviour it was designed for: record the incident, make several attempts, then abandon the operation.

From a software point of view, nothing particularly abnormal.

Error handling worked. So did the retry mechanisms. The system didn’t block and kept running.

It failed correctly.

And that is precisely what made the problem hard to spot.

The failure was indeed recorded somewhere, but it wasn’t visible enough at the level where the result was expected.

Running is not accomplishing #

This experience reminded me of a fundamental distinction in systems engineering.

We often check that a component works, that a service responds, that a connection is established or that a process runs.

These checks are necessary.

But they don’t necessarily demonstrate that the system fulfils its function.

Take a production line.

The controllers communicate, the sensors respond, the actuators perform their movements and the cycles follow one another.

Yet the parts produced don’t meet the expected tolerances.

The equipment works. The process runs.

But the result isn’t compliant.

The same phenomenon can occur in a software architecture, a medical device or an automated system.

Components working correctly doesn’t guarantee the objective is met.

And the more complex systems become, the more this distinction matters.

What we measure determines what we see #

We now have many tools to monitor our systems.

We measure their availability, their response times, their resource consumption, their errors or their performance.

But these indicators mainly describe their technical behaviour.

They don’t always tell us whether the expected work was actually accomplished.

In my case, the system was able to report an authorisation error.

It even knew perfectly well what to do when one occurred.

On the other hand, it had no mechanism explicit enough to signal that an expected task had, in the end, not been carried out.

A level of verification was missing.

Not on the execution of the process, but on its result.

That’s an important nuance.

An execution log lets you understand what happened.

A result check lets you verify whether what was supposed to happen actually did.

Both are necessary, but they don’t answer the same question.

Success has to be demonstrable #

When we design a system, we generally define what it must do.

But do we always define, with the same precision, how to verify that it has actually done it?

A task may have been triggered without being executed.

An operation may have been executed without producing a result.

A result may have been produced without being compliant with the need.

And a system can handle each of these situations perfectly without the user having a clear view of it.

That raises a design question.

Success shouldn’t be inferred solely from the absence of errors.

It should be observable from verifiable elements, directly tied to the expected function.

This doesn’t mean measuring everything or multiplying checks endlessly.

It’s rather about identifying the results that truly matter and planning how to observe their absence, their presence and, where possible, their compliance.

A question of technical responsibility #

In my case, correcting the configuration was relatively simple once the problem was identified.

But the real lesson wasn’t in that fix.

It was in the way I had validated the system after its migration.

I had checked several conditions necessary for it to work.

Not enough the actual completion of all the expected functions.

That’s a difference easy to overlook when you know your architecture well and the first checks are reassuring.

And it is also one of the reasons I place so much importance on a systems view.

It isn’t enough to understand how each element works.

You also have to understand what their assembly is meant to accomplish, and how to verify it.

Because in the end, a system can work exactly as it was designed and not fulfil the mission it was built for.

And that isn’t necessarily a fault.

It is sometimes a problem in the design of the validation itself.

When you check that a system is working, are you checking its execution or its result?