One thing I'd like to add to the discussion is the difference between edge-triggered and level-triggered notifications. Quoting from the Linux Programming Interface (Kerrisk, 2010),
> Level-triggered notification: A file descriptor is considered to be ready if it is possible to perform an I/O system call without blocking.
> Edge-triggered notification: Notification is provided if there is I/O activity (e.g., new input) on a file descriptor since it was last monitored.
The two different types can affect the way the program is structured. For example, the same book says that for level-triggered notification, generally when a file descriptor is ready, only one read/write operation is performed, and then go back to the main loop. Whereas for edge-triggered, usually I/O is performed as much as possible, so we don't miss any more opportunities.
In practice, you usually want your file descriptors to be non-blocking, for many reasons (like writing a large enough buffer can nevertheless block even when the file descriptor is initially considered ready for writing), so even for level-triggered notifications you can read/write in a loop.
Personally I believe edge-triggered notifications can make program design slightly simpler, though I'm not exactly uncertain how much simpler. I'd appreciate if my comment would invite a more detailed and nuanced discussion about those two.
> In practice, you usually want your file descriptors to be non-blocking, for many reasons
I think I know what you're saying, but in practice this is exactly backwards.
Usually you're doing I/O for some practical reason and want to do simple, well-defined, typically sequential processing on the results. Which is to say blocking is your friend and you shouldn't be mucking with parallel I/O paradigms if you can at all avoid it.
> Usually you're doing I/O for some practical reason and want to do simple, well-defined, typically sequential processing on the results. Which is to say blocking is your friend and you shouldn't be mucking with parallel I/O paradigms if you can at all avoid it.
In principle yes, though it becomes annoying when your use-case evolves or you have to miss out on otherwise obvious optimization opportunities because they'd seriously complicate your program.
E.g., Your program is mostly sequential but in one step, you'd be able to do a bunch of requests in parallel.
I think paradigms like async/await are a step ahead to give you the best of both worlds here: You can write your programs as if your requests block, but it still uses async IO behind the scenes - and you can drop the pretense of blocking when it makes sense at any time.
> async/await are a step ahead to give you the best of both worlds here: You can write your programs as if your requests block
Are you thinking of “green”/M:N threading (as found e.g. in Go)?
Async/await (as found e.g. in Python) is precisely what hinders the style you describe: If your brand new I/O routine is “async colored” to take advantage of non-blocking syscalls, you can’t easily call it from your regular “sync colored” code without “spilling the color” all around, i.e. considerable refactoring.
I'm a fan of the user space threading for IO programs that handle more than one stream. It gives you the same context you are used to in blocking programs without the mostly needless thread context switching.
My context is that most of my recent years work was in high performance systems that handle thousands of streams concurrently and only have a limited amount of cores and a rather limited amount of processing for each io request have so the context switching cost becomes a high percentage compared to actual work done.
Yes. The default of blocking I/O is optimized for simple programs where I/O is not the main thing. That default is perfect for those programs. Another default behavior, killing the program on SIGPIPE is also optimized for those programs.
But I'm specifically talking about those that need sophisticated strategies to deal with multiplexed I/O (which is a topic of this article you're commenting on).
> Another default behavior, killing the program on SIGPIPE is also optimized for those programs
I've never understood this choice myself. It has always seemed unhelpful given that the read/write returns an error anyhow. Why is it a helpful default?
Exactly. Also it generally relies on every process everywhere to properly handle an error return from write(). If you launch a big pipe from the shell and just one stage goofs this up and keeps writing after error, the whole thing will stall until you Ctrl-C or otherwise manually kill the process group, which will wreck whatever result you were trying to get from the (already completed successfully!) "head" or "cut" or whatever.
Basically it's a very sane robustness choice and one of the great ideas of classic unix. It's just surprising the first time you stumble over it.
I had written a program a while back that needed to load all of the customers outgoing emails from multiple large .pst files. I got a pretty big performance gain using a thread pool to do the io for all the files concurrently and block until they all finished with thread join() calls. (As opposed to loading them 1 after another)
The actual runtime difference for me was 40 mins -> 10 mins
You may have missed the "multi threaded vs single threaded" section where they talk about a very similar pattern:
"The way it works is simple, it uses blocking IO, but each blocking call is made in its own thread. Now depending on the implementation, it either takes a callback, or uses a polling model, like returning a Future."
It is very relevant. Both async io and multithreaded io are ways to extract parallelism. Which one is more appropriate depends on the characteristics of the problem.
You also have EONESHOT which means that the event monitor is automatically removed after it's triggered.
It depends on the design of your concurrency model.
For event based systems, you may prefer level-triggered notifications. That's because you want to trigger the callback any time data is available.
For fiber/green-thread based systems, you may prefer edge-triggered notifications, and often typically one-shot (e.g. EONESHOT). That's because someone called `wait_until_readable` and when that function returns, they are done. If they want to wait again, they will call the function again.
* Event driven with callbacks (`on data`, `on error`, and so on).
* Stackless coroutines (async/await).
* Stackful coroutines (no direct semantic changes to code required).
Personally, I like stackful coroutines because the concurrency model has minimal effect on the structure of the code, i.e. it's possible to retrofit existing code without change.
My understanding is the same - in C#, async and await are syntactic sugar for callbacks. The code is massively transformed into a state machine for the events.
I think stackful coroutines will always have their fans but the industry is heading towards stackless. The reality is that blocking I/O is a dangerous illusion. Any time you leave your address space you are fundamentally engaged in an asynchronous exchange. APIs that hide this from people -- even in the name of "ease of use" -- are simply inviting all sorts of unexpected behavior or worse, deadlocks. Stackful coroutines hide their blocking nature from the caller and this makes it very difficult to reason about their behavior.
You can get deadlocks in stackless coroutines too. The reason node.js won't deadlock is that it is single threaded.
I find stackful coroutines (ie, Go, Rust, Python Async, etc.) quite useful, they don't behave differently to normal functions and in most languages also don't require callbacks (you can exchange channels, generators, etc.).
Additionally, such actually threaded implementations of coroutines can take advantage of multiple CPU cores more easily, goroutines can swap quickly in and out of CPU threads. Any waiting action will suspend the goroutine until the IO is available. Unlike in Node.js it won't have to wait until the current synchronous action is done since it can be scheduled in another goroutine at any time (and goroutines automatically yield regularly or you put in manual yields).
There is nothing wrong with hiding the nature of the async under the hood of a sync function as long as the abstraction is clean, which in most cases it quite easily can be. For some cases you'll obviously need locks of course, but a simply RWMutex/RWLock can prevent deadlocks easily for 99% of situations where you will need it.
> There is nothing wrong with hiding the nature of the async under the hood of a sync function as long as the abstraction is clean, which in most cases it quite easily can be.
This is the question and I would disagree. Hiding the asynchronous nature of any given function is a dangerous illusion. Languages that promote it through stackful coroutines ultimately lead to programs that are difficult to reason about and often don't perform well because developers lack any real control over concurrency and are completely at the mercy of an opaque scheduler. (And frankly, even languages like Go which get this 'right' ... don't. Eventually even people who like Go realize Go channels are bad[1].)
The answer here I think is going to be higher kinded types a la Rust. Asynchronous is a "fundamental property of time and space" -- ignore it at your own risk. But if the type system can elegantly capture the difference between present-values and values-to-come then you can realize the best of both words: code that reads like a single-thread-of-control but is actually heavily asynchronous. This is why 'await' style stackless coroutines and Futures prove to be so popular.
Though I agree more research is needed here. I was disappointed to see the Rust developers (who apparently convinced people to pay them to research this stuff) converge so quickly on stackless coroutines.
Nobody forces you to use channels in Go, I use them rather rarely, in 9 out of 10 cases it's to pass around a stop handler for some activity loop. That's where they are quite useful.
I think of myself of an older generation so my approach to concurrency is to use locks and fancy data structures and fancy architecture to avoid race conditions.
I would suggest looking into how easy it is to write an HTTP handler in Go. That involves plenty of go routines. Each connection is a go routine. Each HTTP request also gets it's own (in HTTP/2 the former and later may not be the same).
I don't have to think about the blocking nature of an operation to, for a recent example, fetch and parse a remote webpage, I simply do it. The Go runtime will take care of scheduling the goroutines such that if a new request comes in while I'm a tight loop the HTTP server can continue to handle queries.
In JS land however, I can't write tight for loops without potentially blocking up the entire server.
You mentioned the solution yourself, hide the async nature of the code. My HTTP code doesn't read like async code until you hit global resources. That's what I essentially want.
I do not want to think about async until I need it and when I need it I should be able to pretend it's sync without cognitive overhead; that enables me to efficiently reason about the steps a routine takes before every resource a request uses is released again.
As the author of the linked post I would love to correct a misconception here.
Go is fantastic about asynchronous programming and the fact that it hides it is one of Go's strengths. Channels are/were overused when I wrote that post but that's completely independent of Go's excellent async support.
My all time favorite blog post on the matter is http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... which is required reading about why a language hiding a program's async nature is actually the best thing it can do, and Rust's decision to not implement green threads at the language level was wrong, in my opinion.
> This is the question and I would disagree. Hiding the asynchronous nature of any given function is a dangerous illusion.
I understand where you are coming from. However, in my experience, it's "turtles all the way down". Pre-emptive multi-tasking, NUMA, multiple CPUs, CISC, all add their level of asynchronicity to your program execution whether you are aware of it or not.
The right answer in this case is "hiding the nature of async under the hood". Yet, we've seen in a few cases recently, e.g. the recent CPU bugs, where this isn't entirely possible.
> ultimately lead to programs that are difficult to reason about and often don't perform well because developers lack any real control over concurrency
My experience has been the complete opposite. It's hard to make complex parallel programs and reason about execution, but concurrency primitives like fibers and reactors are a negative overhead abstraction which makes code simpler and easier to understand and therefore allows us to do more complex things with the same cognitive potential.
Rust allows you to implement bot stackless/stackful coroutines. But the popular library(futures) is stackless and there are proposals to make stackless coroutines convenient(generators RFC).
It illustrates decisively that a language that doesn't hide async behavior balkanizes libraries into (at least) two camps for everyone's detriment.
Fixing this and doing it correctly requires language support, yes (the language's mutexes must work right in the appropriate contexts) but it is much better for the ecosystem to hide the details of async.
Making the programmer deal with async nature (even with await, yield, etc) adds additional unnecessary details for the programmer to worry about and makes the program harder to reason about.
As for where the industry is headed, Go shows what it's like getting this right.
It seems to me that a simple solution to this is to default to using async for everything. That way there is no "colours", everything is a single colour (async) and everybody is happy.
On Linux, nonblocking IO has not always been great, so under the covers Go often translates calls to actually schedule blocking work on a threadpool somewhere, but from the perspective of the programmer, everything is implicitly using await/yield and promises.
What color is your function is one the most inadequate articles on concurrency ever written. Even managed to claim shared memory multithreading as superior to all other concurrency models. It's sad that it's popular, shows how little people understand about the subject.
The article isn't about concurrency in general though. It's explicitly about the question of how coroutines are called and implemented. The article is specifically about one facet of concurrent programming that arises when you use shared memory multitasking.
I would imagine articles that focus on a small subpiece of concurrency are inadequate articles about concurrency in general, yes.
But it is. It tries to argue that other concurrency models are bad, because they do not look like shared memory multithreading. Even though other concurrency models actually aim for something different than shared memory multithreading to at least get away from its flaws.
Pretty much throughout the whole article. Callbacks, promises/futures, async/await for example all assume different concurrency model from shared memory multithreading and hence are programmed differently.
The article is explicitly saying that green threads are better than callbacks, promises/futures, and async/await, yes. It is definitely making a comparison between those two styles of implementations, and it is saying that (what you call) shared memory multithreading is better than callbacks.
I want to point out though that the article is not claiming OS 1:1 threading is better than async/await/callbacks. OS-level 1:1 threading is too heavy, and it's obvious that many programmers reach for callbacks to handle high concurrency evented I/O. The article is claiming that languages that allow a blocking style of programming via green threads (to still get the same high concurrency) allow a more natural way to program.
Your point was the article was one of "the most inadequate articles on concurrency ever written." I agree. It wasn't trying to be a complete article on concurrency. It says nothing about the actor model.
Then you said it "Even managed to claim shared memory multithreading as superior to all other concurrency models." The article doesn't talk about all other concurrency models. It only talks about the programming styles that "shared memory multithreading" is better than.
> The article is claiming that languages that allow a blocking style of programming via green threads (to still get the same high concurrency) allow a more natural way to program.
I would say this is a subjective conclusion at best. We might ask instead which model facilitates collaboration. That is in a large, very concurrent program composed by many developers which model is going to lead to more robust code. The interesting thing about M:N programming is that large programs in 1:1 languages (Java, C#) tend to converge on to the M:N model. Many large concurrent Java programs therefore end up involving several carefully monitored threadpools with well-defined contracts that describe how work gets scheduled and distributed between the threadpools. But this is not an argument for M:N languages precisely because the "problem of scheduling" often tends to be very domain specific. Other programs have lots of success with the Reactive model because they're never truly blocking and waiting for information that hasn't arrived. Some find success with Actors. It's worth considering then that different concurrency problems call for very different solutions and saying one model is better than another is erroneous.
There are different concurrency models. Shared memory multithreading that Go uses is one. Green threads, goroutines, 1:1 threads, M:N threads are all part of that model. It has synchronous APIs, concurrent memory access and lots of problems. Callbacks, promises/futures, async/await all deliberately chose a different model, that doesn't have those problems and instead are programmed asynchronously. Their whole purpose is to enable programs that only explicitly give up control. So you can't claim they are the problem if they don't allow implicit blocking, as this is the reason they exist and the way they avoid all of the problems shared memory multithreading has.
> It illustrates decisively that a language that doesn't hide async behavior balkanizes libraries into (at least) two camps for everyone's detriment.
That article simply begs the question. Or rather, I think the author has missed the point. Advocates of Futures-based APIs for example don't regard the balkanization as a bad thing. That's the point -- asynchronous functions really are different, they really do accept and return different types of values in the form of callbacks and futures.
Threading and shared memory don't solve anything, btw. These are implementation details. This is not about implementation so much as it's about expressing contracts between modules. Go channels are, frankly, rather useless but still, to my point, they don't try to hide asynchronous behavior. Go could've gone whole hog and provided yield and subsumption but I think they wisely realized that concurrency at scale requires shared nothing message passing. Or rather the essential insight of CSP (and Actors, and PPP and so many other models) is that reifying events and the flow of events is a good thing.
(Though I've heard it proposed that this is just because the limited human mind is "event oriented". Super intelligent martians and AIs would use Dataflow[1] languages to talk about the illusion of time. They might see the world as a giant spreadsheet.)
> Making the programmer deal with async nature (even with await, yield, etc) adds additional unnecessary details for the programmer to worry about and makes the program harder to reason about.
Again, I think it's kind of wacky to think that the async nature is "unnecessary" or accidental for the programmer. The solution the programmer is trying to model is asynchronous. The whole point is to allow the end User to do stuff while you talk to the database or process multiple trades at the same time. You can't wish this asynchronous nature of the problem away because its asynchronous nature, rather than being an unnecessary detail, is the problem. A lot of very smart people have spent a lot of time trying to "solve" concurrency by making it invisible. All failed. The issues around concurrency are a problem that must be addressed head on.
Types are the abstraction that let us be very precise for describing how a system can change. Using the type system to bring the problems of concurrency to the forefront so that they can reasoned about and enforced by the compiler is absolutely absolutely a good thing. People don't like this though because concurrent APIs are "viral". One asynchronous function now requires all callers to either block or become asynchronous themselves. But this was always the case anyways. As Morpheus would say, "How would this be different from any other day?"
> asynchronous functions really are different, they really do accept and return different types of values in the form of callbacks and futures.
I'd argue that all code at some level is asynchronous. Once you start considering how the CPU executes code, and how OSes schedule threads/processes, it's not all that different. Yet, for some reason, we call that code synchronous.
The stackful coroutine route is ideal on paper, but in practice it tends to perform poorly if you retrofit existing code -- because quite often, that existing code depends on blocking i/o or relatively long computations; e.g. opening a file, reading data from a database, or waiting for a TCP connection on the other side of the world to yield data.
It is, of course, possible to wrap all of them in a stackful coroutine interface - but that is no longer "retrofitting existing code without change". My impression is that the cumulative experience is to go either with event driven (in C / C++ / Python) or Stackless (in Python / JavaScript) or threaded (in Java), but stackful coroutines very rarely deliver on their promise.
There are two kinds of stackful coroutines - fibers and green threads.
The difference is that fibers stay on the thread they were created on, while green threads can transit between OS threads. There have also been projects to cooperatively schedule OS level threads to minimise latency, but that's a bit different.
I don't know that many decent green thread implementations [off the top of my head], but an interesting one is yahns: https://yhbt.net/yahns/design_notes.txt - to me, green threads have all the benefits of coroutines but all of the downsides of multi-threaded programming.
Haskell does async IO by default using green threads. GHC allocates Thread State Objects which completely represent a (green) thread on the heap.
A capabillity which represents a cpu core contains a queue of TSO's. If a thread blocks on something it can also move to a different queue which is used for thread synchronization, blocking on computations from other threads, io etc.
However, he is an advocate for stackless coroutines. They are a good idea in some situations but they still have a semantic overhead unlike stackful coroutines. They may have slightly better performance but at the expense of semantic overhead (async/await, explicit futures/concurrency).
I like to think that I'm not advocating for a specific design choice, but for a design choice which makes it possible to write higher level abstractions without the need to expose the underlying concurrency models.
If the video doesn't answer all your questions, I'd be happy to answer specific questions.
> On the IO completing, the OS will suspend your thread, and execute your callback.
On Windows, you have to put your thread to sleep to receive any callbacks [1]. If OS would suspend your thread at random point to execute a callback, that could lead to hard-to-detect/debug deadlocks and race conditions.
It's a little bad for newcomers to define these as one being better than the other. In reality it's a trade-off. And multiplexed/async are not really "I/O types" but ways to handle the two I/O types. In reality everything is either blocking or non-blocking and you always need to consider which option is better for your current usecase. Even the most fancy of web frameworks won't take that out of the equation.
I think there are some misleading points in a hardware perspective at intro.
> That is, gone are the days of serial communication. In that sense, all communications between the CPU and peripherals is therefore asynchronous at a hardware level.
At a hardware level, 99.99% of modern peripherals are based on serial communication (e.g. USB, PCIe, I2C...), and these serial comminications are of course fully asynchronous. They implement concurrent communication at a higher level.
> Think of the simple case, where the CPU would ask the peripheral to read something, and it would then go in an infinite loop,
Many peripherals are logic blocks that doesn’t infinite loop internally. Infinite loop is a kind of software techniques and they don’t need to implement looping to do their job.
In both of your arguments you are disagreeing because you are not talking the same thing that the author is talking about. You are talking about what the hardware does behind the scenes. He is talking about what is going on in the software on the surface. “serial communication” should maybe be rephrased as “serialized communication and execution” in the application process.
Erlang iolists are an example. It means you can generate HTML from templates as a deeply nested list of strings, and write that to a socket efficiently, without concatenating the strings.
The point isn't the requirement, the point is the possibility. You can supply a vector with just one element if you like. But if the data is already in multiple buffers, you avoid either a copy or the overhead of additional syscalls. Typical use cases are when you have headers and payload of some protocol in different buffers, possibly of multiple layers of protocol headers.
I feel like one critical misunderstanding people have is left out in between the hardware and software discussion, which is that the kernel provides certain abstractions, and everything else is built on top of that. Whether or not a program is blocking or non-blocking or multiplexed or whathaveyou is an effect depends on the layer of abstraction you are looking at. It is perfectly possible to have a programming language that is blocking that, if you strace it, is using the non-blocking kernel calls. It is perfectly possible to have an "async" layer under the hood and implement a "non-blocking" layer on top of it, and then on top of that you could easily (even accidentally!) implement a blocking layer of abstraction. It's a lot more a continuum than people realize.
I also got the sense during the height of the Node craze that some of the people who were very excited about it had the impression that Node had some sort of unique access to a "non-blocking" kernel layer or something (of course, they just weren't thinking about the kernel layer at all) that no other languages were using. In reality, if you set something like a Go program to a single OS thread for execution, you could mechanically translate a Go program into something that executes in principle identically to a Node program at the assembler level, at least in terms of what events come in and what sort of high-level code gets executed in response (vast differences in the literal instruction stream, of course). The converted Go would have a lot more "event handlers" than you'd expect because Go programs also break at function calls and a few other places (as the compiler is doing it it's easy to have lots of "event break points" that you'd never write by hand), but the compiler essentially converts the "blocking code" that uses lots of threads into "non-blocking event-based code". The difference between the two is not at the execution layer; it's at the layer you're programming. In fact pretty much everything nowadays is "non-blocking event-based code" at the execution layer, because the high-powered kernel functions that make that efficient implement event-based code, so anything else you find at a programming-language-level must be converting that abstraction into the language's abstraction. Even using kernel threads still turns into non-blocking event-based code under the hood, if you dig far enough in.
(A more clear example of how critical the layer of abstraction you are looking at is IMHO is the distinction between immutable and mutable. You can implement an immutable abstraction layer on top of mutable storage, which given that our hardware is based on mutable storage, all immutability in programming languages necessarily comes from. You can implement a mutable abstraction on top of an immutable layer with a worst-case penalty of O(log n), as in the worst case you represent memory as a immutable tree of bytes and then mutate as you would normally. Whether or not something is "mutable" or "immutable" critically depends on layer of abstraction you are looking at; it does not have a non-contingent answer, or, if it does, since our hardware is mutable the answer is that immutability does not exist at all. But while true in a sense, that answer is not as useful as one that is contingent on the abstraction layer being examined.)
> Level-triggered notification: A file descriptor is considered to be ready if it is possible to perform an I/O system call without blocking.
> Edge-triggered notification: Notification is provided if there is I/O activity (e.g., new input) on a file descriptor since it was last monitored.
The two different types can affect the way the program is structured. For example, the same book says that for level-triggered notification, generally when a file descriptor is ready, only one read/write operation is performed, and then go back to the main loop. Whereas for edge-triggered, usually I/O is performed as much as possible, so we don't miss any more opportunities.
In practice, you usually want your file descriptors to be non-blocking, for many reasons (like writing a large enough buffer can nevertheless block even when the file descriptor is initially considered ready for writing), so even for level-triggered notifications you can read/write in a loop.
Personally I believe edge-triggered notifications can make program design slightly simpler, though I'm not exactly uncertain how much simpler. I'd appreciate if my comment would invite a more detailed and nuanced discussion about those two.