Vertical and horizontal scaling

Why does a point arrive where buying a bigger machine stops being an option at any price?

When a system is short of capacity there are two moves: make the machine bigger, or add more machines. The first is enormously easier — no code changes, no rethinking of where data lives, no new failure modes. You pay money and the problem goes away.

So why does every large system eventually stop doing the easy thing? Not because engineers prefer the hard one, but because at some point the easy one stops being available — and it stops before you run out of money. What makes a bigger machine cease to exist rather than merely cease to be affordable?

Begin with what "bigger" means: more cores, more memory, faster storage, more bandwidth. Is there a largest one? For a while, effectively no — you move from eight cores to sixteen to sixty-four and the parts exist. But parts come from a few manufacturers in a few designs, so at any moment there is a largest server you can buy. That number climbs over the years; today it sits in the low thousands of gigabytes of memory and the low hundreds of cores for a mainstream two-socket machine. Above that you are not shopping, you are commissioning.

That is the crude ceiling. Notice a second thing well below it: price does not rise in step with capacity. Doubling from small to medium costs roughly double; near the top each doubling costs several times the previous one, because the largest parts are made in small volumes and sell to buyers with no alternative. The ceiling is not a wall you hit at speed but a slope that steepens until climbing is absurd, so the practical limit arrives well below the physical one.

The third constraint is not about money at all. What does a bigger machine do for a workload not limited by the resource you added? Suppose the bottleneck is a single lock every request must acquire in turn: doubling the cores doubles the threads waiting for it and does nothing to the rate at which it is handed over. This is Amdahl's 1967 observation — speed-up from parallelism is bounded by the fraction of work that must happen serially. If a tenth is serial, no hardware gets you past a tenfold improvement; not expensive, impossible. Gunther's universal scalability law adds a coordination term, which is worse than a ceiling: past some point, adding workers makes throughput fall.

The fourth decides the matter in practice. A machine, however large, fails as a unit — its power supply, its rack, its data centre. The largest one you can buy has an availability set by one of everything, and no money converts one machine into two. Redundancy cannot be bought inside a box.

Put those together and the shape changes: vertical scaling does not end because big machines run out, but because three curves converge — cost turns superlinear, returns turn sublinear, and availability stays flat at one machine whatever you spend.

What does horizontal scaling do differently? It buys capacity at roughly linear cost — the hundredth machine costs what the first did — and redundancy as a side effect. It pays for both with the thing vertical scaling gave you free: a single address space where everything sees everything. Once state lives on more than one machine you have permanently acquired partitioning, replication, consensus, and network partitions. Those are not implementation details; they are the entire reason horizontal scaling is hard.

An honest caveat. For fifteen years the default advice was to go horizontal early, and much unnecessary distribution followed. Single machines have grown very large — a workload that genuinely required a cluster in 2010 often fits on one modern server — so "scale up until it hurts" is an increasingly respectable default. Vertical scaling is not primitive; it simply has a horizon, and it is worth knowing which of the four constraints reaches you first.

Three refinements.

First, the two are not alternatives chosen between once. Nearly every real system is both: horizontally scaled at the tier that splits easily (stateless request handling) and vertically scaled at the tier that does not (a primary database). The interesting engineering is about which components sit on which side of that line.

Second, "stateless" is what makes horizontal scaling cheap, and it does not mean the system holds no state — it means each machine holds none another would need. That is why scaling web servers is a purchase and scaling databases is a project. The state did not disappear; it moved somewhere that is still, usually, one machine.

Third, a caution about Amdahl's law. It is often quoted as if it forbade large-scale parallelism, which it does not: Gustafson pointed out in 1988 that people use bigger machines to solve bigger problems, and a larger problem's serial fraction is frequently smaller. Both laws are correct and answer different questions — Amdahl asks how much faster a fixed problem gets, Gustafson how much more problem fits in fixed time. Which applies depends on whether your workload grows with your hardware, and for most systems serving more users, it does.

• How sharding splits state across machines, and which queries stop being answerable once it has.
• The universal scalability law's second term: the point at which adding a machine reduces total throughput.

Every term the collection defines is gathered in the glossary.