Compare revisions

…

Backpressure

Body changes between the two selected stored revisions. Code is not executed.

17 changed sections

− Removed+ Added3 unchanged content blocks omitted.

Content blocks 1-15 → 1-21

This change is too large for word-level comparison. The whole changed section is marked as removed and added.

Suppose a file reader produces 100 items per second while the next stage processes 60. For as long as that difference persists, another 40 items accumulate each second. A larger buffer buys time, but it eventually fills too. Backpressure is a way for the receiving side's capacity to influence production or transmission upstream. Data travels from producer to consumer, while information about how much more can be accepted travels in the other direction. The problem is how processing stages with finite resources coordinate their rates. Telling the sender what the receiver can accept This problem predates asynchronous stream libraries. The 1981 TCP specification, RFC 793, describes a receiver advertising a window: the number of bytes it is currently prepared to receive. A sender considers that capacity even when it has more data to send. This is an early protocol example of communicating downstream constraints, rather than evidence that the term backpressure originated with TCP. RFC 793 section 2.6. TCP counts bytes; an application queue may count items, and a stream API may count requested elements. They address related problems at different layers. Available TCP receive-buffer space does not mean that order processing or file conversion has finished, so an application can still accumulate work in an unbounded queue above TCP. Flow control and congestion control have different subjects. Flow control considers the receiver's capacity; congestion control considers conditions along the network path. The current TCP base specification, RFC 9293, distinguishes the send window from the congestion window. RFC 9293 sections 3.8.2 and 3.8.6. Where waiting goes in an asynchronous pipeline Sequential function calls on one thread naturally couple progress: the next call waits for the current one to return. Moving work into a queue or separate task lets the producer advance without waiting for the consumer to finish. We then have to decide where the intervening backlog lives and how large it may grow. Asynchronous execution and bounded resource use are separate design decisions. A queue that keeps accepting items when its consumer slows down separates the two executions, but it does not coordinate their rates. A capacity limit needs a feedback path so that waiting affects upstream production. [Image] The upper arrows show data flow; the lower path shows waiting or demand information travelling upstream. This is an explanatory diagram, rather than a library class structure or a network packet format. If a producer waits for each queue put to complete, a full queue constrains the producer. If it keeps creating separate tasks to perform those puts, the queue can stay small while waiting tasks accumulate outside it. We need to trace which stage actually stops further production. Demand becomes a shared library contract Reactive Streams defines interfaces and rules through which asynchronous stream implementations can exchange these signals. The commit referenced by its 1.0.0 tag dates to April 2015, and the official announcement describes the goal of standardizing asynchronous streams with non-blocking backpressure. The scope extends from managing one queue to preserving flow-control rules when different components are connected. 1.0.0 record, official announcement. [Image]

Suppose a file reader produces 100 items per second while the next stage processes 60. As long as that difference persists, 40 more items accumulate each second. A larger buffer can hold them for a while, but it too will fill if the consumer continues at the same rate. With backpressure, the sender paces production or transmission according to the receiver's available capacity. Data flows downstream from producer to consumer, while signals indicating how much more can be accepted travel upstream. TCP receive windows, waiting on an asynchronous queue, and stream demand address this problem in different units and execution models. The receiver advertises how much it can accept In RFC 793, the 1981 TCP specification, a receiver advertises a window stating how many bytes it can currently accept. The sender takes that limit into account even if it has more data ready. Protocols communicated receiver capacity to senders before asynchronous stream libraries appeared.[Footnote 1] When the receiving TCP stack accepts data, the application may still have an order to process or a file to convert. If the application keeps reading from the socket into an unbounded queue, work can accumulate internally even as it frees the TCP receive buffer. The producer and consumer beyond the socket need their own connection between downstream capacity and upstream waiting. Networking also distinguishes flow control from congestion control. A receiver advertises its own capacity; the sender's congestion control governs what the network path can carry. RFC 9293, the current TCP base specification, distinguishes the send window from the congestion window.[Footnote 2] In an asynchronous pipeline, where does waiting occur? When one thread calls functions in sequence, it waits for a call to finish before making the next one. Move work to a queue or a separate Task, and a producer can create more work without waiting for the consumer to finish. The system then needs a place and a limit for data waiting between them. If the producer awaits every queue put, a full queue prevents it from adding the next item until a slot opens. If instead it keeps creating Tasks to perform each put, the queue can remain small while Tasks waiting outside it accumulate. [Image] The upper arrows show data flow, and the lower path shows waiting or demand signals traveling upstream. Whether through queue waiting or explicit demand, that feedback governs the producer's next action. A server uses queues between stages as control points In their 2001 SEDA paper, Welsh, Culler, and Brewer divided a server into processing stages and made the queues between them explicit. Each stage could inspect its backlog and adjust its thread count or processing batch.[Footnote 3] [Image] (Original source) Events pass through the incoming queue to a handler, which sends events to other stages. The controller adjusts resource use and scheduling in this stage. In §3.3 the authors explain that a queue may reject new items, allowing an application to wait at a full queue or drop events. Placing a queue between stages does not set bounds automatically; the application uses that control point to slow production or shed load. Separate stages are easier to tune individually, but a slow stage also needs a way to affect the stages ahead of it. The problem seen in socket flow control reappears inside the server. Demand passed between libraries The Reactive Streams participants defined interfaces so asynchronous stream libraries could exchange demand and cancellation under common rules. The commit tagged 1.0.0 dates from April 2015, and the official announcement states their goal of standardizing asynchronous stream processing with non-blocking backpressure.[Footnote 4] [Image]

Content blocks 17-24 → 23-31

This change is too large for word-level comparison. The whole changed section is marked as removed and added.

The first sentence of the official introduction was captured on October 28, 2026. It states the initiative's goal; the individual protocol rules are specified in the document linked below. Specification and MIT-0 license. A Publisher emits elements, a Subscriber receives them, and a Subscription connects demand and cancellation. The Subscriber uses request(n) to add the number of elements it is prepared to receive. Publisher rule 1.1 limits cumulative delivery to cumulative demand, while Subscription rule 3.8 makes each new request additive. Reactive Streams 1.0.4 rules 1.1 and 3.8. After request(2) and delivery of one element, one unit of demand remains. Calling request(3) then raises the remaining demand to four. These are illustrative counts, not a limit of four elements per second. Demand permits delivery; it does not require a source to produce that many elements if it has no data or the stream ends. Java 9's java.util.concurrent.Flow provides Publisher, Subscriber, Subscription, and Processor interfaces. Its documentation describes them as corresponding to Reactive Streams. A shared contract matters because agreeing only on a method that sends data leaves demand, cancellation, and termination disconnected. Java 9 Flow, Subscription demand and cancellation. Different ways to express waiting Mechanism | How downstream constraints travel upstream | Relevant scope Synchronous calls or blocking writes | The sender waits for the call to return | Waiting may occupy a thread Bounded asynchronous queue | A full queue suspends the producer's put | The producer must await completion Pull | Production proceeds when the consumer asks for the next value | Prefetching and internal queues need their own bounds Demand-based streams | Elements are delivered within remaining demand | Element count differs from processing completion and bytes The common connection is that a downstream constraint changes upstream behavior. A blocking implementation may wait on a thread; an asynchronous one may postpone resuming a task while allowing other work to run. Reactive Streams requires non-blocking signaling so that the control path between asynchronous stages does not hold threads waiting for one another. Reactive Streams problem and scope. Waiting can therefore be represented as demand or a condition for resumption. A separate contract determines when business processing is complete. Treating receipt of an element as completion of its database write would obscure how much work is still running.

The opening sentence of the official introduction summarizes the participants' goal, while the specification defines the contract for each signal.[Footnote 5] A Publisher sends elements and a Subscriber receives them. The Subscriber calls request(n) on its Subscription to say how many more it can accept. Publisher rule 1.1 prevents cumulative delivery from exceeding cumulative demand, and Subscription rule 3.8 adds each new request to the remaining demand.[Footnote 6] Suppose the Subscriber calls request(2) and receives one element. One unit of demand remains; a subsequent request(3) brings the remainder to four. This count permits delivery of elements. It is not a rate of four per second, and a producer with no data, or one that ends the stream, can send fewer than requested. While remaining demand is zero, the Publisher sends no additional data elements. New requests use a positive number; request(0) is invalid, not a pause command. Completion and error are separate signals from data elements, so they may be delivered even when no demand remains.[Footnote 7] Java 9 also provides Publisher, Subscriber, Subscription, and Processor through java.util.concurrent.Flow. Its documentation describes these interfaces as corresponding to Reactive Streams. Library authors must match the contracts for demand, cancellation, and termination as well as the method for sending data if their components are to connect.[Footnote 8] Ways to express waiting Mechanism | How a downstream constraint reaches upstream | Scope to examine Synchronous call or blocking write | Wait to send the next item until the call returns | The waiting thread and its resources Bounded asynchronous queue | A full queue suspends the producer's put | Whether the producer awaits each completion Pull | Produce the next value when the consumer asks for it | Internal prefetching and other queues Demand-based stream | Deliver elements within remaining demand | Delivered element count and work still in progress In a blocking implementation, a thread may wait. In an asynchronous implementation, the runtime can run other work while a coroutine is suspended. Reactive Streams requires non-blocking exchange of control signals when connecting asynchronous components, so a thread need not be held waiting on that exchange.[Footnote 9] Counting demand also requires a choice about when an item is received versus when its work is done. Suppose a Subscriber starts a new database task for every element it receives and immediately replenishes demand. It can obey the element-delivery rules while the number of database tasks in progress keeps growing. A stage that limits resource use must decide when completed work permits another request.

Content blocks 26 → 33

~~This~~ ~~standalone~~ ~~example~~ ~~requires~~ Python 3.11 or later. A producer puts ~~items~~ into a queue with maxsize=2 ~~and~~ ~~awaits~~ each put. When the queue ~~is full~~, await queue.put() suspends ~~that~~ coroutine ~~while~~ the event loop can run other work, ~~including~~ the consumer. ~~asyncio.Queue.~~

The following runs in Python 3.11 or later. The producer puts values into a queue with maxsize=2, awaiting each put. When the queue fills, await queue.put() suspends the producer coroutine, and the event loop can run other work such as the consumer.[Footnote 10]

Content blocks 28-34 → 35-48

This change is too large for word-level comparison. The whole changed section is marked as removed and added.

Here None is a termination marker excluded from the input values. The single consumer processes the preceding items in order before receiving it, and the parent waits for both tasks when leaving the TaskGroup. The flow does not need the separate unfinished-item accounting provided by join and task_done, so they are omitted. If the consumer fails with an ordinary exception, TaskGroup also cancels the producer. Cancelling the individual consumer is treated differently from this failure rule, so stopping both tasks requires a path that ends their shared lifetime, such as cancellation of the parent. This example demonstrates queue waiting rather than the complete Reactive Streams protocol. TaskGroup failure and cancellation rules. Why buffers and rate limits need separate names Mechanism | What it directly controls | Relationship to backpressure Buffer bound | How much data may wait | A full buffer must influence upstream behavior to change the flow Rate limiter | How much is allowed in a period | A fixed allowance may differ from current consumer capacity Concurrency limit | How many jobs run at once | An unbounded waiting queue still permits accumulation Dropping or rejecting | What is discarded or not accepted | A separate policy is needed where production cannot be slowed The queue's two slots bound waiting items. An item already being processed, the producer's next item, and queues in other stages exist outside those slots. This differs from bounding total process memory in bytes. The example has one producer and one consumer so that the path remains visible. Some inputs, such as sensor readings or mouse movements, originate at a rate we cannot control. A stream boundary may buffer, coalesce, or drop them, but preserving every original event then needs a separate guarantee. Reactive Streams likewise discusses buffering or dropping to respect demand bounds when the original source cannot be slowed. Subscriber-controlled queue bounds. TCP, queues, and streams express backpressure through different mechanisms for communicating capacity between stages. Their history connects byte-level flow control, backlog between asynchronous stages, and shared demand protocols. Reading an implementation means following both the data path and the path through which constraints return upstream.

The input consists of six values, 0 through 5, and None marks termination. The single consumer processes the earlier values in order, then receives the marker and exits. The parent waits for both tasks on leaving the TaskGroup, so this flow does not use join and task_done to count completed items separately. If the consumer fails with an ordinary exception, TaskGroup also cancels the producer. To stop both tasks together, use a path that ends their shared lifetime, such as cancelling the parent task. Cancellation of an individual consumer with CancelledError is handled differently from an ordinary child failure.[Footnote 11] This example passes queue waiting through one process. Reactive Streams passes demand through its separate interfaces and signal rules. Each API has its own termination contract for the same underlying coordination problem. Compare immediate rejection when the queue is full. This Python 3 example uses put_nowait: instead of waiting to admit the second item, it receives QueueFull and reports that the item was not accepted. The final read returns first, the item already in the queue.[Footnote 12] import asyncio queue = asyncio.Queue(maxsize=1) queue.put_nowait("first") try: queue.put_nowait("second") except asyncio.QueueFull: print("second: not accepted") print(queue.get_nowait()) # first Using await put with the same one-slot capacity waits instead. Run this example on Python 3.11 or later. The put_started event establishes when the second insertion starts; while the first item remains in the queue, the second put has not completed. import asyncio async def main(): queue = asyncio.Queue(maxsize=1) await queue.put("first") put_started = asyncio.Event() async def put_second(): put_started.set() await queue.put("second") async with asyncio.TaskGroup() as group: pending = group.create_task(put_second()) await put_started.wait() print(pending.done()) # False first = await queue.get() await pending print(first, await queue.get()) # first second if __name__ == "__main__": asyncio.run(main()) The first get frees a slot for the producer's second item. All we have done with the first item is save it in a variable; no separate business processing has started. This shows where waiting for a queue slot differs from waiting for the consumer to finish its work. What happens when the buffer fills? Mechanism | What it directly sets | What else to decide Buffer bound | How much data can wait | Whether to wait, reject, or drop when full Rate limiter | How much is allowed over a period | What to do when actual consumer speed falls Concurrency limit | How many jobs can run at once | Where and how many requests can wait to run Drop or reject admission | Which items are discarded or refused | How to tell the caller about loss and retries The queue's two slots bound the number of waiting items. An item that the consumer has already taken for processing and the next item held by the producer sit outside those slots. The consumer's values list is another store for results, holding only six values here. A bound on item count therefore has a different scope from a bound on total process memory in bytes. Some sources, such as sensor measurements or mouse movements, cannot slow the rate at which input originates. At the stream boundary, an application can briefly buffer, combine values, or drop some of them. The Reactive Streams specification likewise says buffering or dropping may be needed to respect demand bounds when the original source cannot be slowed.[Footnote 13] If input continues to exceed processing capacity, finite storage eventually fills. Adding a place to retain data longer and increasing sustainable throughput address different limits. Preserving every input requires slowing production enough or securing enough processing capacity. In TCP the receiver advertises a window, in SEDA the application controls stage queues, and in reactive streams a Subscriber requests a count of elements. Following both the forward flow of data and the upstream path of constraints makes clear which producer is waiting and why.

Footnote 1

(No content)

RFC 793 (1981). [RFC 793 §2.6](https://www.rfc-editor.org/rfc/rfc793.html#section-2.6).

Footnote 2

(No content)

RFC 9293 (2022). [RFC 9293 §§3.8.2 and 3.8.6](https://www.rfc-editor.org/rfc/rfc9293.html#section-3.8.6).

Footnote 3

(No content)

Welsh, Culler, Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services (2001). [SEDA paper §3](https://www.sosp.org/2001/papers/welsh.pdf#page=4); [publication record](https://doi.org/10.1145/502059.502057).

Footnote 4

(No content)

Reactive Streams 1.0.0. [1.0.0 record](https://github.com/reactive-streams/reactive-streams-jvm/releases/tag/v1.0.0); [official announcement](https://www.reactive-streams.org/announce-1.0.0).

Footnote 5

(No content)

Reactive Streams 1.0.4. [Reactive Streams specification](https://github.com/reactive-streams/reactive-streams-jvm/tree/v1.0.4).

Footnote 6

(No content)

Reactive Streams 1.0.4. [Reactive Streams 1.0.4 rules 1.1 and 3.8](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#specification).

Footnote 7

(No content)

Reactive Streams 1.0.4. [Specification rules 1.2 and 3.9](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#specification).

Footnote 8

(No content)

Oracle. Java SE 9 API. [Java 9 Flow](https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/Flow.html); [Subscription contract](https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/Flow.Subscription.html).

Footnote 9

(No content)

Reactive Streams. [Problem and scope in the official introduction](https://www.reactive-streams.org/).

Footnote 10

(No content)

Python 3.12. [asyncio.Queue](https://docs.python.org/3.12/library/asyncio-queue.html#asyncio.Queue).

Footnote 11

(No content)

Python 3.12. [TaskGroup failure and cancellation rules](https://docs.python.org/3.12/library/asyncio-task.html#task-groups).

Footnote 12

(No content)

Python 3.12. [put_nowait contract](https://docs.python.org/3.12/library/asyncio-queue.html#asyncio.Queue.put_nowait).

Footnote 13

(No content)

Reactive Streams 1.0.4. [Subscriber-controlled queue bounds](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#subscriber-controlled-queue-bounds).