How Healthy Systems Say “No” to Stay Alive

Systems don’t fail because they get traffic.
They fail because they accept too much traffic.

The Core Problem: Unbounded Demand

What Most Systems Do ❌

• Accept all requests

• Queue indefinitely

• Let latency explode

• Die under their own queues

Queues are not backpressure. Queues are deferred failure.

What Is Backpressure?

Backpressure is the ability of a system to slow down or stop upstream traffic when it is overloaded.

It answers:

“I am unhealthy — please send me less.”

What Is Load Shedding?

Load shedding is intentionally dropping requests to protect the system.

It answers:

“I cannot handle this request — fail fast.”

Why This Matters (Tail Latency Link)

When load > capacity:

• Queues grow

• Context switches explode

• GC pressure rises

• P99 goes to infinity

Protecting P99 requires rejecting work.

🧩 Failure Mode — No Backpressure

❌ Naive Server

app.get("/data", async (req, res) => {
  const result = await db.fetch();
  res.send(result);
});

What Happens

• DB slows

• Requests pile up

• Memory explodes

• Process crashes

🧩 Solution 1 — Backpressure via Concurrency Limits

✅ Semaphore-Based Control

import pLimit from "p-limit";

const limit = pLimit(100); // max 100 concurrent

app.get("/data", async (req, res) => {
  try {
    const result = await limit(() => db.fetch());
    res.send(result);
  } catch {
    res.status(503).send("Overloaded");
  }
});

Why This Works

✔ Caps resource usage
✔ Prevents queue explosion
✔ Stabilizes latency

🧩 Solution 2 — Load Shedding (Fail Fast)

✅ Reject Excess Traffic

let inFlight = 0;
const MAX = 200;

app.get("/data", async (req, res) => {
  if (inFlight > MAX) {
    return res.status(429).send("Too Busy");
  }

  inFlight++;
  try {
    res.send(await db.fetch());
  } finally {
    inFlight--;
  }
});

Key Insight

A fast failure is better than a slow success.

🧩 Solution 3 — Adaptive Load Shedding

Shed Based on Latency

let shed = false;

setInterval(() => {
  shed = metrics.p99 > 500;
}, 1000);

app.get("/data", async (req, res) => {
  if (shed) return res.status(503).send("Degraded");
  res.send(await db.fetch());
});

✔ Protects P99
✔ Self-healing
✔ Production-grade

🧩 Solution 4 — Priority-Based Load Shedding ⭐

Idea

Not all requests are equal.

✅ Code

app.get("/data", async (req, res) => {
  if (overloaded && req.headers["x-priority"] !== "high") {
    return res.status(503).send("Shed");
  }
  res.send(await db.fetch());
});

✔ Premium users survive
✔ Internal traffic survives
✔ Graceful degradation

🧩 Solution 5 — Queue with Backpressure (Correct Way)

Queues must be bounded.

❌ Bad Queue

• Infinite size

• Hides overload

✅ Good Queue

const queue = [];
const MAX_QUEUE = 1000;

function enqueue(req) {
  if (queue.length >= MAX_QUEUE) {
    throw new Error("Queue Full");
  }
  queue.push(req);
}

🧠 Backpressure vs Load Shedding

Aspect	Backpressure	Load Shedding
Goal	Slow traffic	Drop traffic
User impact	Delay	Failure
Latency	Controlled	Minimal
Complexity	Medium	Low

Healthy systems use both.