Failing fast
A Socratic walk-through of failing fast — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why should a program that notices something wrong stop immediately rather than carry on carefully?
Robustness sounds like persistence. A program that hits a problem and keeps going — substituting a default, skipping the bad record, retrying quietly — seems obviously better than one that throws up its hands and dies. Users do not thank you for a crash.
Yet a long tradition of engineering advice says the opposite: when a program discovers that something it believed is false, it should stop at once and loudly. Not degrade, not compensate. Stop. That is worth interrogating, because on its face it trades a small problem for a total one.
Reasoning it through
REASONING #Begin by separating two things that both get called "something wrong".
The first is an expected bad situation: malformed input, a missing file, a timeout, a user typing letters into a number field. These are not surprises. The program's specification covers them, and handling them — rejecting the request with a clear message, retrying the call — is ordinary correct behaviour, not perseverance.
The second is different in kind: the program discovers that an assumption it relies on is false. A quantity that must be positive is negative. A lookup that cannot miss has missed. An invariant the code was written around no longer holds. Nobody planned for this because nobody believed it could happen. That is not an error condition; it is evidence of a bug.
Now ask what "carrying on carefully" means in that second case. The code must do something with a state it does not understand. Whatever it chooses — a default, a clamp, a skip — is a guess about a situation whose cause is unknown. And here is the decisive question: if you do not know why the assumption failed, on what basis do you believe the rest of the computation is still valid?
You do not. So the compensating action does not restore correctness; it conceals the moment of incorrectness while the program keeps writing. Which leads to the real argument, and it is about distance rather than severity.
Think about the gap between where a fault is introduced and where it becomes visible. Stop immediately and that gap is nearly zero: the stack trace points at the line, the state is still the state that violated the assumption, and diagnosis is short. Carry on and the gap grows with every subsequent operation. The failure eventually surfaces somewhere downstream — a report that does not balance, a customer email with a blank name — hours or weeks later, in code that is entirely innocent. The evidence has been overwritten by the very carrying-on that was meant to be careful. Debugging cost rises steeply with that distance, which is the practical reason for the rule.
There is a second cost, worse than debugging. A program that continues past a violated invariant does not merely risk producing a wrong answer; it risks persisting one. A wrong number that reaches a database, a message queue, or another service is no longer a bug in a running process — it is corrupted data outside your control, and cleaning it up is a project rather than a fix. Stopping is what bounds the damage to memory that is about to be discarded anyway.
So what makes stopping acceptable, given that a crash really is bad for users? The answer is that fail-fast is only half a design. Its partner is fast, clean restart. Erlang's culture states this pair explicitly: let a process crash, and let a supervisor restart it from a known-good state, because a fresh start from a known state is more trustworthy than a repair from an unknown one. Crash-only software, argued for by Candea and Fox, makes the same move — if the only way to stop is to crash, then the recovery path is the one you exercise constantly instead of the one that is untested when it finally matters.
That reframes the trade. Failing fast is not "an outage instead of a glitch". It is a small, contained, recoverable failure now, in place of an unbounded and possibly silent one later — and it is only sound when the containment exists.
The analogy
THE ANALOGY #Think of a smoke alarm rather than a fire door. A cook waving a tea towel at a smoking pan is compensating: the alarm stays quiet, the meal continues, and nobody learns whether the smoke was toast or wiring. An alarm that sounds the instant it detects smoke is maximally annoying and maximally early, and its whole value is that the interruption happens while the cause is still a few feet away and easy to find.
an alarm merely announces, whereas a program that fails fast also stops doing the work — so the analogy understates both the cost (real requests are dropped) and the benefit (nothing further is written while the state is suspect).
Clarifying the model
THE MODEL #Three qualifications, because "fail fast" applied everywhere is its own bug.
First, the rule is about violated internal assumptions, not about the system boundary. At the edge — a flaky network, an unavailable dependency, a user's typo — degradation, retries with backoff, and circuit breakers are correct, and crashing would be an over-reaction to something entirely expected. The instinct to distinguish is: could this happen even if every line of my code were right?
Second, failing fast without supervision is just failing. The pattern presupposes something that restarts the unit, a bounded blast radius, and enough logged context to diagnose after the fact. A monolith with no restart strategy that crashes on assertion is not following this advice; it has adopted half of it.
Third, fail-fast is not the same as fail-safe, and confusing them is dangerous in physical systems. A brake that fails safe engages; a controller that fails fast stops controlling. Where stopping is itself the hazard — flight surfaces, medical devices, industrial control — the design must define a safe state to fail into, and getting there may require exactly the careful continuation this rule warns against elsewhere.
A picture of it
THE PICTURE #How to readThe diamond is the only judgement that matters, and it has three exits rather than two: an expected condition is handled and is not a failure at all; a violated invariant halts; guessing carries on. Follow the right-hand branch and notice it reaches a data store — once suspect values are persisted, the loop back to the top is an investigation rather than a fix. The left branch is short because that is the entire claim: halt, restart from a known state, and diagnose while the cause is still in view.
What became clearer
WHAT CLEARED #Stopping immediately looks like fragility and is actually damage control. When a program discovers that something it believed is false, every further instruction is executed on unknown state, so continuing does not preserve correctness — it only moves the visible failure away from its cause and gives it time to escape into storage. Failing fast pays a small, contained, immediate cost to keep the distance between fault and symptom near zero. It is sound advice exactly when a supervisor can restart the failed unit from a known state, and exactly where stopping is not itself the hazard.
Where to go next
ONWARD #- How assertions and preconditions decide, at design time, which conditions deserve a halt.
- Where circuit breakers and bulkheads sit relative to this rule, since both are ways of failing fast at a boundary rather than inside a component.
Key terms
TERMS #| Term | What it means |
|---|---|
| Invariant | a condition the code assumes to hold at a given point; its violation indicates a defect rather than an input problem. |
| Error propagation distance | the gap in code and time between where a fault is introduced and where it becomes observable. |
| Crash-only software | a design in which stopping is always a crash and starting is always recovery, so the recovery path is continuously exercised. |
| Supervision tree | a hierarchy of processes whose job is to restart failed children from a known state, the partner mechanism that makes failing fast affordable. |
Every term the collection defines is gathered in the glossary.