Jane Street's Async library is a great example of backpressure (or as they call it, pushback) done right. Specifically, their Pipe module[0] is what implements this concept very nicely. In essence, a Pipe is a stage in a pipeline. You shove values into one end, and it'll pop out the other end. Then, you can build on a pipe by map-ing over it, filter-ing over it, doing whatever you want over it. Each time you add another operations like this, you're extending a pipeline with another stage, all of which can proceed concurrently.
A Pipe is an unbounded FIFO. But, you can give it a size budget. What that size budget does is determine when writes to a Pipe will block. So if a Pipe has a size budget of 0, then any write will return a Deferred.t (i.e., a promise) that will become determined only when the value has been sent downstream to the next stage of the pipe. If the size budget is 1 on the other hand, then the Deferred.t of the first write will become determined immediately, allowing computation to proceed. If on the second write the first value hasn't been sent downstream, then it'll block. Once there's only one value waiting in the first stage of the Pipe, the second Deferred will become determined.
Another nice module in the library that addresses the connection pooling issue at the end of the blog post (and really that's all it amounts to) is the Throttle module[1]. Here's you can create a Throttle object with however many connections you like, say 5, to however many servers you like, say 5 or possibly less with some redundancy in there. Whenever you want a connection to one of these servers, you go through the Throttle object to get the connection, do the work, and release it automatically when your operation completes (or throws an exception). If there are more than 5 active connections, then you block until one of those gets relinquished. If you want to fail instead of block, you can query the Throttle object to check the number of jobs running, and if there aren't any free jobs, fail.
It's a really nice library. I think that you can't compile Async to JavaScript using js_of_ocaml just yet, but it'll probably happen sometime in the near future.
A Pipe is an unbounded FIFO. But, you can give it a size budget. What that size budget does is determine when writes to a Pipe will block. So if a Pipe has a size budget of 0, then any write will return a Deferred.t (i.e., a promise) that will become determined only when the value has been sent downstream to the next stage of the pipe. If the size budget is 1 on the other hand, then the Deferred.t of the first write will become determined immediately, allowing computation to proceed. If on the second write the first value hasn't been sent downstream, then it'll block. Once there's only one value waiting in the first stage of the Pipe, the second Deferred will become determined.
Another nice module in the library that addresses the connection pooling issue at the end of the blog post (and really that's all it amounts to) is the Throttle module[1]. Here's you can create a Throttle object with however many connections you like, say 5, to however many servers you like, say 5 or possibly less with some redundancy in there. Whenever you want a connection to one of these servers, you go through the Throttle object to get the connection, do the work, and release it automatically when your operation completes (or throws an exception). If there are more than 5 active connections, then you block until one of those gets relinquished. If you want to fail instead of block, you can query the Throttle object to check the number of jobs running, and if there aren't any free jobs, fail.
It's a really nice library. I think that you can't compile Async to JavaScript using js_of_ocaml just yet, but it'll probably happen sometime in the near future.
By the way, this is all OCaml.
[0]: https://ocaml.janestreet.com/ocaml-core/111.28.00/doc/async_...
[1]: https://ocaml.janestreet.com/ocaml-core/111.28.00/doc/async_...