Timeout Pattern

Overview

Suppose an order screen asks an inventory service for stock. If the service never responds, await inventory.GetStockAsync("A-1") never returns. The request keeps its connection and memory, and a synchronous call also ties up its processing thread. Client defaults determine how long it waits: .NET's HttpClient.Timeout defaults to 100 seconds,¹ while gRPC sets no deadline by default, so a client may wait indefinitely.² As these calls accumulate, a delay in one inventory service becomes a delay across the order screen.

The Timeout pattern gives each call a maximum waiting time. Once it expires, the caller stops waiting and treats the call as a failure. It has three roles. The caller decides how long to wait and handles an expired call as timed out. A cancellation signal tells the operation that its time is up. In .NET, a CancellationTokenSource requests cancellation after the specified interval, and its CancellationToken is passed to the operation. In JavaScript, AbortController and AbortSignal serve this purpose. The operation observes the signal, cleans up its work, and stops.

Without the third role, only half the pattern works: the caller stops waiting, but an operation that receives no signal runs to completion and keeps using resources. .NET uses cooperative cancellation: the requester sends a signal, and the operation receiving it decides when and how to stop.³

Structure

caller schedules cancellation with CancelAfter(50 ms) and passes the same source's token to GetStockAsync. After 50 ms the token is cancelled, and SlowInventory leaves its 1000 ms delay and completes with an OperationCanceledException. The caller checks that its own timeout caused the cancellation before handling it as timed out.

History

Michael Nygard examined how production software breaks down, and designs that prevent those failures, in Release It! Pragmatic Bookshelf published the first edition in 2007.⁴ Chapter 5, Stability Patterns, begins with 5.1 Use Timeouts, followed by 5.2 Circuit Breaker and 5.3 Bulkheads.⁵ The second edition, published in 2018, also opens its Stability Patterns section with Timeouts.⁶

The second edition's chapter on stability antipatterns treats integration points, any connection to another system through sockets or remote calls, as risks. When a remote server cannot keep up with connection requests, half-open connections remain in a queue. The thread opening a connection blocks inside the operating-system kernel, and connection timeouts commonly run into minutes. A read() call reading a response often defaults to an indefinite wait as well, requiring an explicit socket timeout to escape a blocked call. A blocked thread cannot handle other requests, so the book concludes that slow responses are much worse than no response.⁷

.NET material also distinguishes stopping the wait from stopping the operation. In a 2012 article, Stephen Toub asked whether an asynchronous operation without cancellation support can be cancelled, and answered no. His approach waits for either the operation or a cancellation request, whichever arrives first. He explicitly notes that it ends only the wait; the operation itself may keep running for a while.⁸ Available since .NET 6, Task.WaitAsync(TimeSpan) has the same character. Its documentation says that the returned Task completes when the original Task finishes or the timeout expires, with a TimeoutException in the latter case. It does not say that the original Task stops.⁹

Why use it?

A timeout bounds how long a slow dependency can retain the caller's resources. If the operation responds to cancellation and cleans up, it releases connections and memory, and threads for synchronous calls. The caller can then choose whether to return a default, report an error, or try again later. Timeout counts also signal trouble in a dependency. In Microsoft's standard HTTP resilience handler, the circuit breaker blocks execution when direct failures or timeouts become too frequent.¹⁰ This decision is covered in Circuit Breaker Pattern.

If an operation receives no cancellation signal, or ignores one it receives, a timeout leaves the operation running after its caller has moved on. Polly's timeout strategy documentation likewise warns that a callback that ignores the token continues executing after cancellation is requested.¹¹ Such abandoned work keeps using connections and memory. Nobody may be left to dispose an object it eventually returns, or an exception raised in the meantime may go unobserved.⁸

A timeout leaves the outcome unknown. When the caller gives up, the server may still be processing the request or may already have completed it. Before resending a request with an external effect, such as a payment or order creation, establish whether it is idempotent. Idempotency means that sending the same request repeatedly has the same effect as sending it once. Conditions for combining timeouts with retries are covered in Retry Pattern.

Which stages of a call a configured timeout covers depends on the implementation. An AWS Builders' Library article notes that some implementations leave remote-call stages such as DNS lookup or the TLS handshake outside the timeout.¹²

Choosing the limit and where to apply it

Choose the value from the dependency's latency distribution. The AWS article starts with an acceptable false-timeout rate, for example 0.1%, and uses the corresponding downstream latency percentile. Here that is p99.9, the time within which 99.9% of requests complete. A false timeout cuts off a call that would have succeeded with a little more time. The article warns that a long timeout loses value because resources stay occupied while waiting. An overly short timeout increases retries, backend traffic and latency; even a small latency increase can cause every request to retry and turn into a full outage.¹²

When a call has several stages, independent timeouts at each stage are insufficient. Giving an outer request two seconds and each of three inner calls two seconds allows inner work to continue after the outer caller has given up. Instead, set one deadline for the entire request and pass it downstream. A timeout describes how long; a deadline describes until when. gRPC defines a deadline as the point after which the client will no longer wait for a response. Calls exceeding it fail with DEADLINE_EXCEEDED. The deadline received by a server is converted into a timeout with elapsed time deducted; some language implementations propagate it automatically to downstream calls.²

Within a .NET process, passing the same CancellationToken to downstream calls propagates the deadline. Tokens cannot cross process boundaries, so a call to another service must carry the remaining time in the request, as with a gRPC deadline. A downstream call that also wants its own limit creates a linked token. It combines several source tokens and is cancelled when any of them is cancelled.¹³ The effective limit is then the shorter of the remaining deadline and the call's own limit. The HttpClient documentation likewise says that combining the client-wide Timeout with a per-request CancellationTokenSource applies only the shorter of the two.¹

Retries need two timeouts: an attempt timeout covering a single attempt, and a total timeout covering every retry and the waits between them. With only attempt timeouts, total duration can grow with the number of attempts and exceed the caller's expectation. With only a total timeout, one slow attempt can consume the budget and leave no chance to retry. From outermost to innermost, the standard Microsoft.Extensions.Http.Resilience handler stacks rate limiting, a total timeout of 30 seconds by default, retries, a circuit breaker, and an attempt timeout of 10 seconds by default. The total timeout wraps retries; the attempt timeout sits within retries and limits one attempt.¹⁰ Independent retries at several layers multiply the load. In the AWS article's example, one customer request traverses five service layers and ends with a database query. With three attempts at each layer, independent retries amplify database load by 243 times.¹⁴

Example

Apply a 50 ms timeout to a fake inventory service that takes 1000 ms. The operation passes its token to Task.Delay so cancellation ends the delay promptly. It records its state and calls ThrowIfCancellationRequested to tell the caller that it ended through cancellation. Microsoft recommends this method for acknowledging a cancellation request to the caller.¹⁵ ConfigureAwaitOptions.SuppressThrowing (available since .NET 8) suppresses the exception from a cancelled await, giving the operation a place to check the token itself and clean up.¹⁶ This option works only with a resultless Task. Using it with Task<T> throws an ArgumentOutOfRangeException.¹⁷

using System;
using System.Threading;
using System.Threading.Tasks;

var inventory = new SlowInventory(TimeSpan.FromMilliseconds(1000));

// The caller sets its waiting limit. This source's token is cancelled after 50 ms.
using var timeout = new CancellationTokenSource();
timeout.CancelAfter(TimeSpan.FromMilliseconds(50));
try
{
    int quantity = await inventory.GetStockAsync("A-1", timeout.Token);
    Console.WriteLine($"stock: {quantity}");
}
catch (OperationCanceledException) when (timeout.IsCancellationRequested)
{
    Console.WriteLine("timed out");
}
Console.WriteLine($"work: {inventory.State}");

// Simulate an inventory service taking 1000 ms that observes cancellation.
sealed class SlowInventory(TimeSpan delay)
{
    public string State { get; private set; } = "not started";

    public async Task<int> GetStockAsync(string sku, CancellationToken ct)
    {
        State = "running";
        // Cancellation ends Delay promptly. SuppressThrowing lets us check the token directly.
        await Task.Delay(delay, ct).ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
        if (ct.IsCancellationRequested)
        {
            // Clean up open connections or locks here, then report cancellation.
            State = "stopped by cancellation";
            ct.ThrowIfCancellationRequested();
        }
        State = "completed";
        return 7;
    }
}

timed out
work: stopped by cancellation

The caller uses the when (timeout.IsCancellationRequested) filter to handle only cancellation caused by its own timeout as timed out. The operation stops when it receives the signal instead of waiting the full 1000 ms.

Extended example

Compare three cases. First, WaitAsync ends only the wait. The operation receives CancellationToken.None so no signal reaches it. Second, CreateLinkedTokenSource combines an upstream token and a per-call time limit, so whichever comes first stops the operation. The upstream token is cancelled when the requesting user cancels or disconnects. Third, the whole request has a 500 ms deadline, with three calls sharing its one token. CallAsync receives the upstream and deadline tokens separately and returns a CallStatus describing how the call ended. User cancellation is UpstreamCanceled, expiry of the shared deadline is DeadlineExceeded, and expiry of this call's own limit is TimedOut. These cases are separated because counting user cancellations as dependency failures can make a circuit breaker block a healthy dependency. Deadline expiry also depends on how much budget earlier calls used, so whether it counts toward the breaker requires a separate decision.

using System;
using System.Threading;
using System.Threading.Tasks;

TimeSpan Ms(int value) => TimeSpan.FromMilliseconds(value);

// 1) WaitAsync: only the caller stops waiting. No cancellation reaches the operation.
var abandoned = new FakeService("stock", Ms(1000));
Task<int> running = abandoned.GetAsync(CancellationToken.None);
try
{
    await running.WaitAsync(Ms(50));
}
catch (TimeoutException)
{
    Console.WriteLine($"WaitAsync: caller stopped waiting, work: {abandoned.State}");
}
int late = await running;
Console.WriteLine($"WaitAsync: abandoned work finished later with {late}, work: {abandoned.State}");

// 2) Linked tokens: upstream cancellation or the per-call limit stops work, whichever comes first.
using (var request = new CancellationTokenSource())
{
    Console.WriteLine(await CallAsync(new FakeService("stock", Ms(1000)), request.Token, CancellationToken.None, Ms(50)));
}
using (var request = new CancellationTokenSource())
{
    request.CancelAfter(Ms(50)); // For example, the requesting client disconnected.
    Console.WriteLine(await CallAsync(new FakeService("stock", Ms(1000)), request.Token, CancellationToken.None, Ms(5000)));
}

// 3) Shared deadline: give the request 500 ms and pass one deadline token to every downstream call.
using (var request = new CancellationTokenSource())
using (var deadline = new CancellationTokenSource(Ms(500)))
{
    Console.WriteLine(await CallAsync(new FakeService("price", Ms(30)), request.Token, deadline.Token, Ms(5000)));
    Console.WriteLine(await CallAsync(new FakeService("stock", Ms(10000)), request.Token, deadline.Token, Ms(5000)));
    Console.WriteLine(await CallAsync(new FakeService("ship", Ms(30)), request.Token, deadline.Token, Ms(5000)));
}

// Apply a per-call limit and return a value describing how the call ended.
// Separate upstream and deadline tokens let us distinguish their causes.
static async Task<CallResult> CallAsync(FakeService service, CancellationToken upstream, CancellationToken deadline, TimeSpan limit)
{
    // Do not start if cancellation or deadline expiry has already occurred.
    if (upstream.IsCancellationRequested || deadline.IsCancellationRequested)
        return new CallResult(service.Name, CallStatus.NotStarted, service.State);

    using var linked = CancellationTokenSource.CreateLinkedTokenSource(upstream, deadline);
    linked.CancelAfter(limit);
    try
    {
        await service.GetAsync(linked.Token);
        return new CallResult(service.Name, CallStatus.Ok, service.State);
    }
    catch (OperationCanceledException) when (upstream.IsCancellationRequested)
    {
        // The user cancelled; this is not a dependency failure.
        return new CallResult(service.Name, CallStatus.UpstreamCanceled, service.State);
    }
    catch (OperationCanceledException) when (deadline.IsCancellationRequested)
    {
        // The whole request's deadline expired.
        return new CallResult(service.Name, CallStatus.DeadlineExceeded, service.State);
    }
    catch (OperationCanceledException) when (linked.IsCancellationRequested)
    {
        // This call's own limit expired.
        return new CallResult(service.Name, CallStatus.TimedOut, service.State);
    }
}

enum CallStatus { Ok, TimedOut, DeadlineExceeded, UpstreamCanceled, NotStarted }

sealed record CallResult(string Name, CallStatus Status, string Work)
{
    public override string ToString() => $"{Name}: {Status}, work: {Work}";
}

// Simulate a slow dependency that observes cancellation during its delay.
sealed class FakeService(string name, TimeSpan delay)
{
    public string Name => name;
    public string State { get; private set; } = "not started";

    public async Task<int> GetAsync(CancellationToken ct)
    {
        State = "running";
        // Cancellation ends Delay promptly. SuppressThrowing lets us check the token directly.
        await Task.Delay(delay, ct).ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
        if (ct.IsCancellationRequested)
        {
            // Clean up open connections or locks here, then report cancellation.
            State = "stopped by cancellation";
            ct.ThrowIfCancellationRequested();
        }
        State = "completed";
        return 7;
    }
}

WaitAsync: caller stopped waiting, work: running
WaitAsync: abandoned work finished later with 7, work: completed
stock: TimedOut, work: stopped by cancellation
stock: UpstreamCanceled, work: stopped by cancellation
price: Ok, work: completed
stock: DeadlineExceeded, work: stopped by cancellation
ship: NotStarted, work: not started

The first two lines show that after WaitAsync raises a TimeoutException, the operation remains running and eventually completes. A real service would keep using connections and memory during that time. Under the shared deadline, the stock call stops because the remaining deadline expires, rather than its own 5000 ms limit, and reports DeadlineExceeded. The later ship call never starts because the deadline has already passed. Cancellation here works only where the operation checks its token. A library call that accepts no token, or a computation loop that never checks one, cannot stop partway through. Choose APIs that accept tokens or check the token within the loop. A token also stops only work within the same process. A server may finish a request that has already gone over the network, so when its outcome is unknown, check its state or establish idempotency before retrying.

TypeScript example

In Node.js and browsers, AbortSignal carries the cancellation signal. AbortSignal.timeout(ms) returns a signal that aborts after the specified time. Its reason is a TimeoutError DOMException. The browser specification measures this limit in active time: counting pauses while a worker is suspended or a page stays in the bfcache.¹⁸ By contrast, calling AbortController.abort() without a reason uses AbortError as the reason.¹⁹ AbortSignal.any combines signals and aborts when any one aborts, serving the same purpose as linked tokens in C#. Node.js added AbortSignal.timeout in v17.3.0 and v16.14.0, and AbortSignal.any in v20.3.0 and v18.17.0.²⁰ In the second case, Promise.race ends only the wait, like WaitAsync.

// Simulate a dependency taking delayMs that observes AbortSignal.
class FakeService {
  state = "not started";
  constructor(readonly name: string, private readonly delayMs: number) {}

  get(signal: AbortSignal): Promise<number> {
    return new Promise((resolve, reject) => {
      if (signal.aborted) {
        reject(signal.reason);
        return;
      }
      this.state = "running";
      const onAbort = () => {
        // Clearing the timer is what actually stops the work.
        clearTimeout(timer);
        this.state = "stopped by abort";
        reject(signal.reason);
      };
      const timer = setTimeout(() => {
        signal.removeEventListener("abort", onAbort);
        this.state = "completed";
        resolve(7);
      }, this.delayMs);
      signal.addEventListener("abort", onAbort, { once: true });
    });
  }
}

// Translate the abort reason: TimeoutError for a timeout, AbortError for abort().
function reasonOf(e: unknown): string {
  if (e instanceof DOMException && e.name === "TimeoutError") return "timed out";
  if (e instanceof DOMException && e.name === "AbortError") return "canceled by caller";
  return "failed";
}

async function call(service: FakeService, signal: AbortSignal): Promise<string> {
  try {
    await service.get(signal);
    return `${service.name}: ok, work: ${service.state}`;
  } catch (e) {
    return `${service.name}: ${reasonOf(e)}, work: ${service.state}`;
  }
}

async function main(): Promise<void> {
  // 1) AbortSignal.timeout: after 50 ms the signal aborts and the operation stops.
  console.log(await call(new FakeService("stock", 1000), AbortSignal.timeout(50)));

  // 2) Stop only the wait: without a signal, the operation keeps running.
  const abandoned = new FakeService("stock", 1000);
  const running = abandoned.get(new AbortController().signal);
  const timer = new Promise<"timeout">((resolve) => setTimeout(() => resolve("timeout"), 50));
  const first = await Promise.race([running, timer]);
  console.log(`race: ${first}, work: ${abandoned.state}`);
  console.log(`race: abandoned work finished later with ${await running}, work: ${abandoned.state}`);

  // 3) Combine signals: caller cancellation or timeout stops work, whichever arrives first.
  const caller = new AbortController();
  setTimeout(() => caller.abort(), 50);
  const combined = AbortSignal.any([caller.signal, AbortSignal.timeout(5000)]);
  console.log(await call(new FakeService("stock", 1000), combined));
}

main();

stock: timed out, work: stopped by abort
race: timeout, work: running
race: abandoned work finished later with 7, work: completed
stock: canceled by caller, work: stopped by abort

FakeService handles the abort event by clearing the timer and rejecting the promise. Without that handling, the operation runs to completion even after its signal aborts. APIs such as fetch that accept a signal option can receive it directly, as in fetch(url, { signal: AbortSignal.timeout(5000) }).²¹

See also

• Programming Patterns

• Retry Pattern

• Circuit Breaker Pattern

• Bulkhead Pattern

• Result Type

• Cache-Aside

Each C# example is a standalone .NET 10 console program.

Compile the TypeScript examples to JavaScript targeting ES2022 or later, then run them in Node.js.