Robust Yet Fragile
Before calling something reliable, ask what kind of failure it was built to survive.
A system can get very good at handling familiar trouble while one less familiar failure can still hit everything at once.
Reliable against which failure?
Systems usually get better at handling the failures they see often because those are the ones people study, add controls for and practise recovering from.
That same work can put too much on one shared thing. Standardising puts more teams on the same component. Optimising removes spare room. One control can then become the reason many parts fail together.
This is Robust Yet Fragile: a system can handle the trouble it knows well and still be badly exposed to a different problem, especially one that reaches many parts at once.
As a system is tuned for the problems it sees every day, it can lose its backups, alternatives and spare room. Day-to-day life improves, while more parts become vulnerable to the same break.
Reliability is always reliability against something. Saying that a system is robust without naming the disturbance leaves out the part that matters.
One login system can lock everyone out
Imagine a company replacing several inconsistent access systems with one identity provider. Support tickets fall, people get access faster, offboarding becomes safer and the security team finally has one place to enforce policy.
When the identity provider fails, customers can't sign in, support can't reach the tools used to help them, and the on-call engineer discovers that the emergency console uses the same login. People wait while the engineer tries to recover a system whose recovery path depends on the failed system.
Centralisation solved many ordinary access failures. It also turned one provider into a failure that could reach almost everyone at once. The common incidents disappeared, so the design looked more reliable, but the remaining shared failure could now affect almost everyone.
How this shows up elsewhere
Biology. A field planted with one high-performing crop can be easier to manage and better at resisting the pests it was bred for. The same similarity means a new disease can hit almost every plant in the same way. The useful part of the comparison is simple: variety can stop one problem spreading everywhere. Software is different because a team can replace a component far faster than a crop can change.
Finance. A portfolio can hold many investments and still depend on one hidden condition, such as cheap credit or buyers being easy to find. In normal times it looks diverse because the prices do not move together. In a crisis, the shared dependency shows itself and everyone tries to sell through the same narrow door. A long list of investments is not much protection when they all fail for the same reason.
Software and organisational design. Shared platforms, standard workflows and central teams remove duplicated work and make the common path more reliable. They also give many teams the same dependency, so one deployment tool, approval queue or vendor can block everyone at once. Local variation creates its own errors too. Where can we tolerate a shared failure, and where do we need a boundary that stops it spreading?
Where to look for it
Use this model when consolidating vendors, standardising a platform, removing redundancy, centralising a decision or celebrating a large improvement in reliability.
Ask:
- Which disturbances did we optimise against?
- Which parts now fail for the same reason?
- What appears redundant only because the normal conditions are narrow?
- Can recovery happen without the component that has failed?
- Where does the failure stop?
For each failure you care about, write down who it reaches, how quickly you will notice, how recovery works and whether the recovery path depends on the thing that just failed.
Where the model stops helping
Optimisation and standardisation still matter. Diversity, spare capacity and independent recovery paths cost money and can introduce more everyday failure. Preparing for every imaginable shock would make most systems expensive and difficult to operate.
Use the model when a failure would be widespread, hard to detect or difficult to reverse, and keep an independent recovery path there. Standardise the parts where a shared failure is tolerable.
Requisite Variety asks whether the people or system in charge have enough different responses for the situations they may face. Robust Yet Fragile asks which problems the system handles well and whether that success has made many parts vulnerable to the same new failure. Normalization of Deviance is different: it happens when people keep accepting risky exceptions because nothing bad happened last time. A system can follow every rule and still be fragile because too many parts depend on one thing.