Backpressure
Follow receiver feedback from TCP through staged servers to asynchronous stream demand, with source diagrams and examples.
· Last updated: 2026-11-07 · 11 min read
Overview
If a file reader emits 100 items a second while the next stage finishes only 60, 40 items pile up every single second. Making the buffer bigger buys you a few extra seconds, nothing more. As long as the consumer stays slower, any finite buffer eventually overflows.
Backpressure closes the loop by making the sender pace its output to match what the receiver can actually take. Data flows downstream from producer to consumer, while capacity signals travel upstream in the opposite direction, whether expressed as a TCP receive window in bytes, a suspended put on a bounded queue, or an explicit request(n) count on a stream.
The receiver advertises how much it can accept
Network protocols solved this long before asynchronous stream libraries existed. In the 1981 TCP specification (RFC 793), the receiver advertises a receive window telling the sender how many bytes it may transmit right now, forcing a fast sender to hold back even when its own outbound buffer is full.1
Yet the kernel's TCP stack accepting bytes does not mean your application has finished processing the order or transcoding the video! If user-space code eagerly drains the socket into an unbounded in-memory queue, the OS happily re-opens the TCP receive window to invite more network traffic while uncompleted jobs quietly blow up application memory, so capacity signals have to reach all the way from the slowest internal worker back to the socket.
Networking also separates flow control from congestion control: flow control protects a specific receiver's buffer via the receive window, whereas congestion control protects the shared network path between hosts, a distinction the current TCP standard (RFC 9293) maintains through separate send and congestion windows.2
In an asynchronous pipeline, where does waiting occur?
In ordinary single-threaded code, calling a function naturally blocks the caller until the callee returns. Put an asynchronous queue or background Task between producer and consumer, however, and the producer can keep minting work long before the consumer finishes, immediately raising the question of where pending items live and how many are allowed to wait.
When a producer awaits every queue.put(), a full queue halts the producer in its tracks. Spawn a brand-new detached Task for each put instead, and the queue itself may stay tiny while thousands of pending put Tasks pile up outside the gate.

In the diagram above, the top arrows carry data downstream while the bottom arrows carry waiting or demand signals upstream. That upstream feedback is what keeps the pipeline stable.
A server uses queues between stages as control points
Server architecture brought those queues center stage in Welsh, Culler, and Brewer's 2001 SEDA (Staged Event-Driven Architecture) paper, which broke a server into explicit stages connected by bounded event queues so each stage could watch its own queue length and tune its thread pool or batching on the fly.3

Events pass through the incoming queue to a handler, which sends events to other stages. The controller adjusts resource use and scheduling in this stage. (Original figure).
Crucially, §3.3 of the SEDA paper notes that stage queues can reject incoming items once full, forcing upstream code either to block or to shed load by dropping events. Dropping a queue between two components accomplishes nothing by itself; the application has to enforce a bound at that queue to slow upstream intake or reject excess requests.
Splitting a pipeline into stages makes each stage easier to tune in isolation, at the cost of having to propagate slowdowns backward from a bottleneck stage to the front door: the exact socket flow-control problem replaying inside a single process.
Demand passed between libraries
In April 2015, the Reactive Streams initiative tagged version 1.0.0 of a minimal interface specification designed so different asynchronous JVM stream libraries could exchange demand and cancellation signals without blocking threads.4

The opening sentence of the official introduction summarizes the participants' goal, while the specification defines the contract for each signal.5 (Original figure).
Its mechanics are strict: a Publisher emits elements to a Subscriber, which calls Subscription.request(n) to grant permission for n more items; Publisher rule 1.1 forbids total emitted items from ever exceeding total requested demand, and Subscription rule 3.8 makes request(n) additive.6
Call request(2), receive one item, and one credit remains; call request(3) next and remaining demand climbs to four. Notice that n is a cumulative item count, not a rate per second, and if the source runs out of data or closes early, it is free to emit fewer items than requested.
When remaining demand hits zero, the Publisher must stay silent. Every new request call must pass a strictly positive integer (request(0) is a spec error, not a pause button), though terminal signals (onComplete and onError) are not data elements and may travel downstream even at zero demand.7
Java 9 adopted those exact four interfaces, Publisher, Subscriber, Subscription, and Processor, under java.util.concurrent.Flow, meaning libraries only interoperate cleanly when they honor the demand, cancellation, and completion rules alongside onNext.8
Ways to express waiting
Mechanism | How a downstream constraint reaches upstream | Scope to examine |
|---|---|---|
Synchronous call or blocking write | Wait to send the next item until the call returns | The waiting thread and its resources |
Bounded asynchronous queue | A full queue suspends the producer's | Whether the producer awaits each completion |
Pull | Produce the next value when the consumer asks for it | Internal prefetching and other queues |
Demand-based stream | Deliver elements within remaining demand | Delivered element count and work still in progress |
- Mechanism
Synchronous call or blocking write
- How a downstream constraint reaches upstream
Wait to send the next item until the call returns
- Scope to examine
The waiting thread and its resources
- Mechanism
Bounded asynchronous queue
- How a downstream constraint reaches upstream
A full queue suspends the producer's
put- Scope to examine
Whether the producer awaits each completion
- Mechanism
Pull
- How a downstream constraint reaches upstream
Produce the next value when the consumer asks for it
- Scope to examine
Internal prefetching and other queues
- Mechanism
Demand-based stream
- How a downstream constraint reaches upstream
Deliver elements within remaining demand
- Scope to examine
Delivered element count and work still in progress
How waiting looks in code depends on the concurrency model: a blocking queue parks the OS thread, whereas an async runtime suspends only the coroutine and frees the thread to run other tasks, which is why Reactive Streams mandates non-blocking control signals end to end.9
One subtle trap is confusing receiving an element with finishing its work. If a Subscriber kicks off an async database write inside onNext and immediately calls request(1) before that write completes, it obeys the letter of the Reactive Streams count while still unleashing thousands of concurrent database queries.
Flow control with a two-slot queue
The Python 3.11+ snippet below wires a producer and consumer through an asyncio.Queue(maxsize=2): because the producer awaits every queue.put(), filling both slots suspends the producer coroutine and hands the event loop over to the consumer.10
import asyncio
async def produce(queue):
for item in range(6):
await queue.put(item)
await queue.put(None)
async def consume(queue):
values = []
while True:
item = await queue.get()
if item is None:
return values
await asyncio.sleep(0.01)
values.append(item)
async def main():
queue = asyncio.Queue(maxsize=2)
async with asyncio.TaskGroup() as group:
group.create_task(produce(queue))
consumer = group.create_task(consume(queue))
assert consumer.result() == list(range(6))
print(consumer.result()) # [0, 1, 2, 3, 4, 5]
if __name__ == "__main__":
asyncio.run(main())Six numbers (0 through 5) flow through the queue followed by None as a stop sentinel. Once the consumer reads None and returns, the enclosing TaskGroup waits for both tasks to finish cleanly, so this pipeline needs no separate queue.join() or task_done() bookkeeping.
If the consumer raises an unhandled exception, TaskGroup automatically cancels the producer too; cancelling CancelledError inside a single child behaves differently from a normal exception, so shutting down the whole pipeline cleanly means cancelling at the parent scope.11
Here a shared in-process queue communicates capacity by suspending put(), whereas Reactive Streams communicates capacity through explicit request(n) calls, two different surface contracts solving the same coordination problem.
What if a full queue should reject immediately instead of waiting? Using put_nowait on a one-slot queue raises QueueFull on the second item right away, leaving first as the only item read at the end.12
import asyncio
queue = asyncio.Queue(maxsize=1)
queue.put_nowait("first")
try:
queue.put_nowait("second")
except asyncio.QueueFull:
print("second: not accepted")
print(queue.get_nowait()) # firstSwitch that same one-slot queue to await queue.put() and the second insertion pauses instead: in the script below, put_started fires as soon as the second put begins, yet second_put_done stays false for as long as the first item occupies the slot.
import asyncio
async def main():
queue = asyncio.Queue(maxsize=1)
await queue.put("first")
put_started = asyncio.Event()
async def put_second():
put_started.set()
await queue.put("second")
async with asyncio.TaskGroup() as group:
pending = group.create_task(put_second())
await put_started.wait()
print(pending.done()) # False
first = await queue.get()
await pending
print(first, await queue.get()) # first second
if __name__ == "__main__":
asyncio.run(main())As soon as the first get() removes first, the slot opens and the second put completes, even though the consumer merely stored first in a local variable without doing any real work on it yet, showing the exact line between waiting for a buffer slot and waiting for downstream processing.
What happens when the buffer fills?
Mechanism | What it directly sets | What else to decide |
|---|---|---|
Buffer bound | How much data can wait | Whether to wait, reject, or drop when full |
Rate limiter | How much is allowed over a period | What to do when actual consumer speed falls |
Concurrency limit | How many jobs can run at once | Where and how many requests can wait to run |
Drop or reject admission | Which items are discarded or refused | How to tell the caller about loss and retries |
- Mechanism
Buffer bound
- What it directly sets
How much data can wait
- What else to decide
Whether to wait, reject, or drop when full
- Mechanism
Rate limiter
- What it directly sets
How much is allowed over a period
- What else to decide
What to do when actual consumer speed falls
- Mechanism
Concurrency limit
- What it directly sets
How many jobs can run at once
- What else to decide
Where and how many requests can wait to run
- Mechanism
Drop or reject admission
- What it directly sets
Which items are discarded or refused
- What else to decide
How to tell the caller about loss and retries
Remember what maxsize=2 actually bounds: only the two items sitting inside the queue. One item already popped by the consumer, another item held ready by the producer, and the values list accumulating outputs all live outside those two slots, so bounding queue length is never identical to bounding total process memory in bytes.
Some inputs, hardware sensors, UDP telemetry, mouse movement, cannot be told to slow down at the source. At that boundary you have to buffer briefly, coalesce/sample readings, or drop frames, a reality the Reactive Streams specification explicitly acknowledges for unpaceable sources.13
If sustained arrival rate exceeds sustained processing rate, every finite buffer eventually fills. Temporary buffering only buys time during a burst; surviving sustained load requires slowing the source, dropping work, or scaling downstream throughput.
TCP advertises byte windows, SEDA bounds inter-stage queues, and Reactive Streams counts element credits: three different units for the exact same upstream feedback loop.
See also
Footnotes
- RFC 793 (1981). RFC 793 §2.6. ↩
- RFC 9293 (2022). RFC 9293 §§3.8.2 and 3.8.6. ↩
- Welsh, Culler, Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services (2001). SEDA paper §3; publication record. ↩
- Reactive Streams 1.0.0. 1.0.0 record; official announcement. ↩
- Reactive Streams 1.0.4. Reactive Streams specification. ↩
- Reactive Streams 1.0.4. Reactive Streams 1.0.4 rules 1.1 and 3.8. ↩
- Reactive Streams 1.0.4. Specification rules 1.2 and 3.9. ↩
- Oracle. Java SE 9 API. Java 9 Flow; Subscription contract. ↩
- Reactive Streams. Problem and scope in the official introduction. ↩
- Python 3.12. asyncio.Queue. ↩
- Python 3.12. TaskGroup failure and cancellation rules. ↩
- Python 3.12. put_nowait contract. ↩
- Reactive Streams 1.0.4. Subscriber-controlled queue bounds. ↩