A rate limiter, a circuit breaker, an auto-scaler, and a surge-pricing engine are the same thing. Each measures a signal. Each compares it to a threshold. Each changes system behaviour when the signal moves.
Most engineers learn these tools one at a time. They are not separate tools. T11 — Feedback Loops is the thread that ties them together. If you have watched one of them misbehave, you already know how the others break.
This article traces T11 across the layers where it appears in the series. The mechanism stays constant. The signal and the actuator change.
Layer 1 — Rate limiting: the loop in its simplest form
Rate limiting is a feedback loop with one signal and one action. The system counts requests per unit time. It compares the count to a threshold. It rejects excess requests until the count drops.
The signal is arrival rate. The actuator is admission. The setpoint is the configured limit. Nothing else.
The output of the loop — rejections — feeds back into the input. Rejected clients slow down or retry later. The measured rate falls. The system stops rejecting. This is negative feedback creating stability.
Every T11 instance in the series starts here. Everything else adds state, memory, or a wider signal.
Layer 2 — Circuit breakers: feedback with state
A circuit breaker sits between a caller and a dependency. It watches call outcomes. It counts failures. It changes its own state when the failure signal crosses a threshold.
The mechanism is a three-state machine. CLOSED passes requests through to the dependency. OPEN rejects every request immediately without calling. HALF-OPEN admits a single probe request to test recovery.
The transitions are driven by the feedback signal. Enough failures in CLOSED move the breaker to OPEN. A reset timer moves it from OPEN to HALF-OPEN. A successful probe returns it to CLOSED. A failed probe sends it back to OPEN.
The signal is failure rate. The actuator is admission to the dependency. The setpoint is the failure threshold. This is the rate-limiter loop with one added dimension: memory of recent outcomes.
The circuit breaker exists to prevent FM2 — Cascading Failures. A slow dependency holds caller threads on every request. Held threads fill the caller's thread pool. Its own callers then starve. The failure climbs the dependency graph until the system stalls.
Fast-failing at the breaker keeps the cascade local. Rejected requests do not hold threads. Thread pools stay drained. The rest of the system keeps running with a degraded feature, not a full outage.
Layer 3 — Load balancer health checks: feedback that routes
A load balancer probes each backend on a schedule. It marks a backend healthy or unhealthy based on the probe result. It sends traffic only to healthy backends.
The signal is probe outcome. The actuator is routing. The setpoint is the health threshold — often N consecutive successes to mark healthy, M consecutive failures to mark unhealthy.
This is the same loop again. The measured output — backend errors — feeds back into the input by removing that backend from the pool. Traffic shifts to healthy peers. The unhealthy backend either recovers or stays out of rotation.
The added dimension here is spatial. The circuit breaker gates one dependency edge. The load balancer gates a fan-out across many backends. Same loop, wider actuator.
Layer 4 — Auto-scaling: feedback that provisions
An auto-scaler watches a load signal — CPU, request rate, queue depth. It compares the signal to a threshold. It launches or terminates instances to move the signal back to the target.
The signal is resource utilisation. The actuator is fleet size. The setpoint is the target utilisation.
This is where positive feedback risk appears. If the scale-up delay is longer than the load rise time, the loop cannot catch up. Load rises. The scaler launches instances. The instances take three minutes to boot. Load rises further during boot. More instances launch. The loop overshoots on both sides.
Every T11 loop has a delay. Every delay determines whether the loop stabilises or oscillates.
Layer 5 — Surge pricing: feedback that steers demand
A ride-hailing platform measures unmet demand in a geographic area. It raises the price until demand falls or supply rises. When the imbalance clears, the price returns to baseline.
The signal is the ratio of ride requests to available drivers. The actuator is price. The setpoint is a demand-supply ratio the platform considers healthy.
Surge pricing runs two feedback loops on the same signal. Higher price suppresses demand — negative feedback. Higher price attracts drivers from adjacent areas — negative feedback again. Both loops pull the ratio back toward the setpoint.
The dimension that scales up here is scope. The loop is not inside one service. It is a market. The signal comes from users. The actuator changes user behaviour. The loop is closed through the outside world.
Layer 6 — Recommendation engines: feedback that trains itself
A recommendation engine ranks items for a user. The user clicks some items and skips others. Those clicks feed back into the training data. The next model version learns from the recorded behaviour.
The signal is engagement. The actuator is ranking. The setpoint is whatever the objective function encodes — clicks, watch time, purchases.
This loop closes over hours or days rather than seconds. The delay is long. The gain is uncertain. The signal itself is shaped by the ranking the loop produced last time. Yesterday's recommendations decide today's training data.
That is a positive feedback loop hiding inside a negative one. Popular items get shown. Shown items get clicked. Clicked items become training signal for the next round. The loop amplifies whatever it starts with.
Where feedback loops fail: gain, delay, and the wrong target
Every T11 instance can fail in the same three ways.
Gain too high. The loop overreacts to noise. A rate limiter that trips on a one-second spike drops legitimate traffic. A circuit breaker that opens after two failures rejects real requests during a garbage-collection pause. An auto-scaler with an aggressive threshold launches instances for a transient burst.
Delay too long. The loop reacts to conditions that no longer exist. An auto-scaler that takes three minutes to boot instances chases load that has already changed. A health check with a slow interval keeps sending traffic to a backend that is already down.
Wrong target. The signal being measured is not the objective. This is Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. A recommendation engine optimised for clicks maximises clicks — including clicks on misleading titles. The loop is still working. It is working on the wrong thing.
Correct calibration is not one-time. It is an ongoing operation. Every T11 loop needs its gain, its delay, and its target re-examined as the system's behaviour shifts.
FM11 — The silent loop
A feedback loop that acts without emitting a signal is worse than no loop.
A circuit breaker opens. From the user's view, a feature stops working. From the engineer's view, the upstream call now returns fast errors instead of slow timeouts. There is no latency spike to alert on. The change is invisible unless the breaker state is exported.
The same applies to every other loop. An auto-scaler that scales in without recording the decision leaves engineers debugging a phantom capacity problem. A rate limiter that rejects without logging the rejections looks like a client bug. A recommendation loop that shifts its objective silently reshapes the product without anyone noticing.
Every state change in every T11 loop must emit a metric. This is FM11 — Observability Blindness. The loop must publish its own behaviour, or its operators cannot see the system.
What repeats across the layers
Five properties describe every T11 loop in the series.
Signal — what the loop measures. Threshold — the value that triggers action. Actuator — what the loop changes. Delay — the time between measurement and effect. Gain — how strongly the actuator responds to a change in the signal.
The rate limiter, the circuit breaker, the load balancer, the auto-scaler, the surge-pricing engine, and the recommendation ranker each fill in the same five slots differently. Once you see the shape, you stop learning them one at a time. You start reading any new control mechanism as another instance of the same object.
The signal that this applies to your system
Your service's error rate rises. Its P99 latency drops at the same moment. Errors are now returning faster than successful responses used to.
That combination is a feedback loop taking over upstream. A circuit breaker has opened, a rate limiter has engaged, or a load balancer has shed the slow backend. Fast failure has replaced slow timeout. Read the loop's state metrics next, not the latency histogram.
The question this article did not answer
Every T11 loop in isolation has five knobs. A real system runs many loops at once. Loops interact.
An auto-scaler that reacts to latency and a circuit breaker that reacts to failures can chase each other. The breaker opens, latency drops because failed calls return fast, the scaler scales in, the remaining instances saturate, the breaker opens wider. Two negative loops combine into one positive one.
How do you reason about a system that runs many feedback loops on overlapping signals? That is where T11 stops being a pattern and starts being an architecture.
The full framework treatment — compression blocks, three-level exercises, and the complete AT/FM mapping — is in the Reference Book, Chapter 3 (Twelve Threads, T11). Free chapter available at computingseries.com/books/ref.