Bulkhead Pattern

Overview

Suppose an order server calls a payment service and a product catalog service, allowing at most two external calls at once. One day the payment service stops responding. Two payment calls occupy both slots indefinitely. A later catalog call cannot obtain a slot even though its service is healthy, so it ends with catalog: rejected by shared. A payment outage has stopped the catalog feature too.

The Bulkhead pattern partitions these resources by dependency. Its name comes from the watertight compartments dividing a ship's hull. With two payment slots and two separate catalog slots, a stalled payment service fills only its own allocation. Catalog calls use their own slots and finish normally; only payment calls are rejected when payment capacity is full.

The resource divided in these examples is a concurrency slot. One slot represents one call in progress. A semaphore counts the remaining slots: acquiring one decrements the counter, and completing the call returns one. The semaphore and its limit allocated to one dependency form a bulkhead. Calling code goes through the bulkhead assigned to its target dependency.

The policy must also specify what happens when every slot is occupied. Immediate rejection lets the caller promptly choose another response; this is failing fast. A bounded queue lets a few calls wait for a slot, then rejects further calls when the queue fills too.

Other designs partition connection pools, thread pools, message queues or service instances instead of slots, and may partition by customer, or tenant, instead of dependency. Either way, one dependency or customer can exhaust only its own allocation.¹

Structure

The diagram shows the types in the extended example. Program creates separate payment and catalog instances of Bulkhead. Each Bulkhead uses SemaphoreSlim to count execution slots and admitted to count running and queued calls together. When both capacities are full, it returns Rejected without waiting.

History

Microsoft's Azure Architecture Center traces the name to compartments in a ship's hull: if the hull is breached, only the damaged compartment floods, preventing the ship from sinking. It defines the pattern as isolating application elements into pools so the others keep working when one fails, and notes that it is also called cell-based architecture.²

Michael Nygard's Release It!, a book about designs that help systems survive production, was first published by Pragmatic Bookshelf in 2007. In the publisher's first-edition table of contents, Chapter 5, Stability Patterns, has 5.3 Bulkheads after 5.1 Use Timeouts and 5.2 Circuit Breaker.³ The second edition, published in January 2018, lists Timeouts, Circuit Breaker and Bulkheads under Stability Patterns in the same order.⁴

Netflix's Hystrix documentation describes using bulkheads to isolate dependencies. It gives each dependency a separate thread pool to bound its resource use, and also discusses using semaphores instead to limit concurrent calls per dependency.⁵

Polly v7 offered this pattern through Policy.Bulkhead(maxParallelization, maxQueuingActions). Its v7-to-v8 migration guide says that the rate limiter strategy replaces the old policy, with ConcurrencyLimiter as its counterpart. The guide also says that AddConcurrencyLimiter(permitLimit, queueLimit) requires the Polly.RateLimiting package.⁶ The strategy is a thin layer over APIs in System.Threading.RateLimiting. That package's ConcurrencyLimiter is a RateLimiter implementation managing concurrent resource access.⁷⁸

Why use it?

Bulkheads prevent one slow dependency from causing a cascading failure in other features. Here catalog calls succeed while payment is stalled. Azure lists containing failures, keeping some functions available during an outage, and giving important consumers dedicated resources for different service levels as benefits.⁹

Immediate rejection when slots are full keeps callers from waiting on unresponsive work while holding threads or connections. They can promptly decide whether to return an alternative response or try later.

A bulkhead does not end a slow call already occupying a slot. A payment call retains its slot until the service responds or the call is cancelled. To bound when the slot returns, use a Timeout pattern. Hystrix likewise notes that semaphore isolation can shed load but cannot make slow calls exit through a timeout.¹⁰ A Circuit Breaker pattern handles withholding calls from a persistently failing dependency. Azure recommends considering bulkheads alongside retries, circuit breakers and throttling.¹¹

Partitions cannot borrow one another's capacity. Catalog traffic can exhaust catalog slots while payment slots sit idle. Finer partitions leave more unused capacity and more settings to configure and monitor. Azure says the pattern may not fit when less efficient resource use is unacceptable or the extra complexity is unnecessary.¹²

A semaphore inside one process partitions only that process's calls. With several server instances, each has its own limits, and process-wide resources such as CPU and memory remain shared. Isolating those resources requires deploying services into separate virtual machines, containers or processes.¹¹

Slot capacity and queues

Each partition needs two numbers: the maximum running calls and the calls allowed to wait when all slots are occupied. A zero-length queue rejects immediately. A queue absorbs brief bursts, but queued calls still consume memory and the caller's response time. An unbounded queue lets calls accumulate indefinitely before a stalled dependency, defeating the resource partition.

Measure the dependency's usual concurrency and response-time distribution to choose capacity. During operation, record occupied slots and rejections per partition. Azure likewise recommends monitoring performance and SLAs for each partition.¹¹ The same document recommends using existing platform controls, such as API-management usage limits or container resource limits, rather than recreating them in application code.¹³ Without a custom limiter, Polly's rate limiter strategy defaults to a concurrency limiter with PermitLimit 1000, QueueLimit 0. A rejected execution throws RateLimiterRejectedException.¹⁴

Return rejection separately from dependency failure. A full partition is a local decision by the calling server; a failed dependency is a problem in the remote service. The extended example returns Rejected or Failed so callers can handle them differently.

Example

A TaskCompletionSource gate simulates the stalled payment service. It remains incomplete until the program calls SetResult, reproducing an unresponsive service without waiting for real time. Run the same call sequence twice: once with a shared partition and once with a separate partition per dependency. Wait(0) immediately returns false if no slot is available.¹⁵

using System;
using System.Threading;
using System.Threading.Tasks;

var shared = new Bulkhead("shared", 2);
await Scenario("one shared bulkhead", payments: shared, catalog: shared);
await Scenario("separate bulkheads", payments: new Bulkhead("payments", 2), catalog: new Bulkhead("catalog", 2));

static async Task Scenario(string title, Bulkhead payments, Bulkhead catalog)
{
    Console.WriteLine($"== {title} ==");
    // Stalled payments: calls remain incomplete until their gate opens.
    var paymentGate = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
    var pay1 = payments.RunAsync(() => paymentGate.Task);
    var pay2 = payments.RunAsync(() => paymentGate.Task);
    Console.WriteLine("payment 3: " + await payments.RunAsync(() => paymentGate.Task));
    Console.WriteLine("catalog: " + await catalog.RunAsync(() => Task.FromResult("ok")));
    paymentGate.SetResult("ok");
    Console.WriteLine("payment 1: " + await pay1);
    Console.WriteLine("payment 2: " + await pay2);
}

sealed class Bulkhead(string name, int slots)
{
    private readonly SemaphoreSlim semaphore = new(slots, slots);

    public async Task<string> RunAsync(Func<Task<string>> call)
    {
        // Reject immediately if no slot is available. Without acquisition, there is nothing to release.
        if (!semaphore.Wait(0)) return "rejected by " + name;
        try
        {
            return await call();
        }
        finally
        {
            semaphore.Release();
        }
    }
}

== one shared bulkhead ==
payment 3: rejected by shared
catalog: rejected by shared
payment 1: ok
payment 2: ok
== separate bulkheads ==
payment 3: rejected by payments
catalog: ok
payment 1: ok
payment 2: ok

A shared partition rejects even the catalog call. With separate partitions, only the third payment call is rejected by the payment bulkhead.

Extended example

Give payments two execution slots and one queue position, and the catalog two slots with no queue. Of four payment calls, two run, the third queues, and the fourth immediately receives Rejected. Then make pay-2 fail with an exception. Its finally returns the slot, allowing pay-3 to run. SemaphoreSlim leaves pairing Wait with Release to the programmer. If failed calls never return their slots, capacity disappears one slot at a time until the whole partition only rejects.¹⁶

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

var payments = new Bulkhead("payments", maxConcurrent: 2, maxQueue: 1);
var catalog = new Bulkhead("catalog", maxConcurrent: 2, maxQueue: 0);

// Fake payment gateway: each call has a gate the program opens or faults.
var gates = new Dictionary<string, TaskCompletionSource<string>>();
foreach (var id in new[] { "pay-1", "pay-2", "pay-3", "pay-4", "pay-5" })
    gates[id] = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
Task<string> Pay(string id) => gates[id].Task;

var pay1 = payments.RunAsync(() => Pay("pay-1"));
var pay2 = payments.RunAsync(() => Pay("pay-2"));
var pay3 = payments.RunAsync(() => Pay("pay-3")); // No slot: wait in the queue.
var pay4 = payments.RunAsync(() => Pay("pay-4")); // The queue is full too: reject immediately.
Console.WriteLine("after 4 payment calls: " + payments.Describe());
Console.WriteLine("pay-4: " + Describe(await pay4));
Console.WriteLine("catalog-1: " + Describe(await catalog.RunAsync(() => Task.FromResult("item list"))));

gates["pay-2"].SetException(new InvalidOperationException("gateway reset"));
Console.WriteLine("pay-2: " + Describe(await pay2));
gates["pay-3"].SetResult("charged pay-3"); // Run in the slot returned by pay-2.
Console.WriteLine("pay-3: " + Describe(await pay3));
gates["pay-1"].SetResult("charged pay-1");
Console.WriteLine("pay-1: " + Describe(await pay1));
Console.WriteLine("after all calls: " + payments.Describe());

gates["pay-5"].SetResult("charged pay-5");
Console.WriteLine("pay-5: " + Describe(await payments.RunAsync(() => Pay("pay-5"))));

static string Describe(BulkheadResult<string> result) => result switch
{
    BulkheadResult<string>.Ok ok => "ok: " + ok.Value,
    BulkheadResult<string>.Rejected r => $"rejected: {r.Bulkhead} full (running {r.Running}, queued {r.Queued})",
    BulkheadResult<string>.Failed f => "failed: " + f.Error.Message,
    _ => "unknown result",
};

abstract record BulkheadResult<T>
{
    public sealed record Ok(T Value) : BulkheadResult<T>;
    public sealed record Rejected(string Bulkhead, int Running, int Queued) : BulkheadResult<T>;
    public sealed record Failed(Exception Error) : BulkheadResult<T>;
}

sealed class Bulkhead(string name, int maxConcurrent, int maxQueue)
{
    private readonly SemaphoreSlim slots = new(maxConcurrent, maxConcurrent);
    private int admitted; // Running plus queued calls

    public int Running => maxConcurrent - slots.CurrentCount;
    public int Queued => Volatile.Read(ref admitted) - Running;
    public string Describe() => $"{name} running={Running} queued={Queued}";

    public async Task<BulkheadResult<T>> RunAsync<T>(Func<Task<T>> call)
    {
        // If execution slots and queue are full, return rejection without waiting.
        if (Interlocked.Increment(ref admitted) > maxConcurrent + maxQueue)
        {
            Interlocked.Decrement(ref admitted);
            return new BulkheadResult<T>.Rejected(name, Running, Queued);
        }
        var acquired = false;
        try
        {
            // Wait if no slot is available. The finally below restores counters even if waiting throws.
            await slots.WaitAsync();
            acquired = true;
            return new BulkheadResult<T>.Ok(await call());
        }
        catch (Exception ex)
        {
            // Dependency boundary: convert exceptions to failure results, retaining their type and stack.
            return new BulkheadResult<T>.Failed(ex);
        }
        finally
        {
            // Always decrement admitted; release a slot only when actually acquired.
            Interlocked.Decrement(ref admitted);
            if (acquired) slots.Release();
        }
    }
}

after 4 payment calls: payments running=2 queued=1
pay-4: rejected: payments full (running 2, queued 1)
catalog-1: ok: item list
pay-2: failed: gateway reset
pay-3: ok: charged pay-3
pay-1: ok: charged pay-1
after all calls: payments running=0 queued=0
pay-5: ok: charged pay-5

There is no limit on queue waiting time here, so pay-3 waits until another call releases a slot. Production code should give WaitAsync a timeout or CancellationToken and reject when the time expires. A call that never acquired a slot must not call Release; it must only decrement admitted. This example's RunAsync puts the wait inside its try as well. Its finally always decrements admitted and returns a slot only if acquired is true, preventing counter leaks even if waiting throws on cancellation. When adding a timeout, assign the WaitAsync(timeout) return value, a bool, to acquired and return Rejected when false. Running calls have no timeout either: if the payment service never responds, pay-1 never releases its slot. SemaphoreSlim guarantees no admission order for waiting calls,¹⁷ so ordering requires a separate queue. Running and Queued are calculated by reading two values separately, so they are approximate during concurrent traffic. This partition applies only within one process and exports no metrics for rejections or occupied slots.

TypeScript example

This repeats the first C# scenario in TypeScript. Within one JavaScript execution environment, a number counting available slots limits concurrency. An async function runs synchronously until its first await, so the slot check and decrement happen when run is called. The gate stays incomplete until open resolves its Promise.

type CallResult =
  | { ok: true; value: string }
  | { ok: false; error: "BULKHEAD_FULL"; bulkhead: string };

// A small semaphore counting free slots; report success immediately without waiting.
class Semaphore {
  #available: number;
  constructor(slots: number) { this.#available = slots; }
  tryAcquire(): boolean {
    if (this.#available === 0) return false;
    this.#available -= 1;
    return true;
  }
  release(): void { this.#available += 1; }
}

class Bulkhead {
  readonly #slots: Semaphore;
  constructor(readonly name: string, slots: number) { this.#slots = new Semaphore(slots); }
  async run(call: () => Promise<string>): Promise<CallResult> {
    // Reject if acquisition fails. Never release a slot that was not acquired.
    if (!this.#slots.tryAcquire()) return { ok: false, error: "BULKHEAD_FULL", bulkhead: this.name };
    try {
      return { ok: true, value: await call() };
    } finally {
      this.#slots.release();
    }
  }
}

// A gate pending until explicitly opened, simulating a stalled payment service.
function gate(): { promise: Promise<string>; open: (value: string) => void } {
  let open: (value: string) => void = () => {};
  const promise = new Promise<string>((resolve) => { open = resolve; });
  return { promise, open };
}

function describe(result: CallResult): string {
  return result.ok ? result.value : "rejected by " + result.bulkhead;
}

async function scenario(title: string, payments: Bulkhead, catalog: Bulkhead): Promise<void> {
  console.log(`== ${title} ==`);
  const paymentGate = gate();
  const pay1 = payments.run(() => paymentGate.promise);
  const pay2 = payments.run(() => paymentGate.promise);
  console.log("payment 3: " + describe(await payments.run(() => paymentGate.promise)));
  console.log("catalog: " + describe(await catalog.run(() => Promise.resolve("ok"))));
  paymentGate.open("ok");
  console.log("payment 1: " + describe(await pay1));
  console.log("payment 2: " + describe(await pay2));
}

async function main(): Promise<void> {
  const shared = new Bulkhead("shared", 2);
  await scenario("one shared bulkhead", shared, shared);
  await scenario("separate bulkheads", new Bulkhead("payments", 2), new Bulkhead("catalog", 2));
}

void main();

== one shared bulkhead ==
payment 3: rejected by shared
catalog: rejected by shared
payment 1: ok
payment 2: ok
== separate bulkheads ==
payment 3: rejected by payments
catalog: ok
payment 1: ok
payment 2: ok

A call that fails to acquire a slot returns before entering try and has no slot to release. This semaphore has no queue; a slot never returns if its occupying call never finishes.

See also

• Programming Patterns

• Timeout Pattern

• Circuit Breaker Pattern

• Retry Pattern

• Rate Limiter Pattern

• Cache-Aside

• Result Type

Each C# example is a standalone .NET 10 console program.

Compile the TypeScript examples to JavaScript targeting ES2022 or later, then run them in Node.js.