What color is your function?
One of my software blog posts that I always go back to is the now famous What Color is Your Function?. After multiple read-throughs and reading through the comments both on the post and in Hacker News, I still don’t understand what the arguments is supposed to be besides “I don’t want to have to think through a robust type system therefore async is bad.” Having spent the last few years working in both green-threaded (Go) and async-await languages (JavaScript, Python, Rust) and styling myself as an amateur language designer I feel I have a good enough background take on this problem and prove that async-await is better than green-threads once and for all.
Background
For anyone not familiar with the basic problem, I highly recommended reading the linked blog post above, but the basic gist is this: using blocking preemptive OS-threads for all concurrency can be heavy-weight and slow, so cooperative multitasking is the new hotness being added programming languages. Cooperative multitasking yields at every point where the code would block (IO, sleeps, …) and allows a user space scheduler to handle allocating CPU time to run code, without incurring the cost of full context switches. There are two primary systems languages use to implement cooperative multitasking, green threads and async-await (with async-await implemented directly in monads with do notation in Haskell). Classic callback style JavaScript technically fits into this category as well, but I’ll omit talking about it here as async-await is strictly better.
Async-Await
Async-Await draws from the functional programming concept of monads (when implemented correctly), which are far too out of scope to talk about here. The key idea of this approach is that any function that would block gets the async keyword before it, and, depending on your language’s terminology, returns a future or a promise. This future can then be combined with other futures, returned directly, or be awaited on, which yields control of the current function until the future is resolved.
E.g.
1
2
3
4
async function make_http_call() {
let result = await fetch('http://example.com');
return await response.text();
}
This forces you to mark which functions block and do not block, in the terminology of “What Color is Your Function” this forces you to specify some functions as blue vs red. If you want to wait for the result of a blocking function call you must yield control with await, and if you want a task to run in the background you must submit it to the runtime executor, as futures are not executed unless something is awaiting on them. There are exceptions, namely JavaScript which always schedules all promises immediately and doesn’t use true monads due to poor language design, but JavaScript not being well thought out is not what’s being debated here.
Green Threads
Green threads (or goroutines, or fibers) are used like classic OS threads but they automatically yield on all blocking operations. This allows you to not worry about which function calls make such operations and which do not, because the compiler will take care of it for you.
E.g.
1
2
3
4
function make_http_call() {
let result = fetch('http://example.com');
return response.text();
}
If you want to wait for the result of a blocking function call you just call that function normally, if you want a task to run in the background you spawn a new lightweight thread for it to run on. Go does this with the go keyword.
Why Async-Await is Superior
Functionally the end result of the two approaches are the same and both approaches can be implemented in terms of each other. A language could actually translate green-threads into an async-await style by automatically by adding async to every function definition and await every function call and optimizing away any cases where it’s unnecessary. One of the most common misconceptions about async-await based concurrency is that you’re marking the functions that block, but in reality, you’re marking the functions that don’t block. Any function that doesn’t have async in its type signature is guaranteed not to block.
Similarly you can implement promises as channels in golang using the <- operator as await.
1
2
aCh, bCh, cCh := longRunningTask(), longRunningTask(), longRunningTask()
a, b, c := <-aCh, <-bCh, <-cCh
Yes, having to specify async or not in the function’s type signature absolutely introduces cognitive load, but so does specifying if arguments are constants or mutable or event their types and the return values - all of these are done for the guarantees enforced by the compiler, they allow you to reason about the behavior of your program. An example of something I’ve seen time and time again is problems introduced by adding blocking operations into code in a hot loop. This isn’t unnecessary a bug in any code, but rather an extreme inefficiency by say doing a remote lookup by id sequentially for every element in a list rather rather than a single lookup with the list of ids. Having async in the type signature makes it explicit that you’re you introducing IO to a place it did not previously exist, making it easy to catch in code reviews, and optimize before it becomes a problem.
This extra cognitive load has a side effect that it naturally leads to pushing IO up the call stack. Rather than doing IO deep in nested functions, tends to change the style more to high level IO with deep non-blocking functions doing transformations. This pattern (sometimes called functional core, imperative shell) is very common in more functional languages, and often results in much cleaner code.
Similarly, its common to see code in languages like go that doesn’t use concurrency where it could because it’s not obvious that the underlying calls perform IO. In async-await based languages, you can just about add linting rules to prevent IO in loops, or sequential IO for values that don’t depend on each other.
1
2
3
4
5
6
7
8
9
10
let pages = [
"http://example.com/1",
"http://example.com/2",
"http://example.com/3",
"http://example.com/4",
];
let results = [];
for (page of pages) {
results.push(await fetch(page)); // Lint error, await in a for loop.
}
1
2
let a = await fetch("http://example.com/1");
let b = await fetch("http://example.com/2"); // Lint error, IO not dependent on other IO
One of the key advantages of async-await is that it gives you the future to combine with other futures.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
let pages = [
"http://example.com/1",
"http://example.com/2",
"http://example.com/3",
"http://example.com/4",
];
let futures = [];
for (page of pages) {
futures.push(fetch(page));
}
let results = await Promise.all(futures);
// alternatively
let results = await Promise.all(pages.map(page => fetch(page)));
1
2
3
4
let a, b = await Promise.all([
fetch("http://example.com/1"),
fetch("http://example.com/2")
]);
This allows you to turn your sequential code into more of a dependency map going from blocking location to blocking location minimizing the number of non-concurrency tasks with Promise.all (or Future::join or asyncio.gather depending on your language). While languages like go have similar mechanisms, they do not match the convenience of being able to await a list of futures, and for an arbitrary list you can be forced to build a giant pile of channels to try and handle successes vs failures or deal with trying to build your own result types.
I’m not actually saying async-await is good for all cases all the time, it’s a form of “Effect Typing” for encoding expected behavior into the type system, but not all programs require it for the same reason not all programs require strict typing, Python for example is excellent for getting out of your way when you need to write short scripts, but if you’re dealing with a large system with many moving parts that extra layer of safety allows you to reasonably build software with confidence.
Common Arguments
Every article I see on this topic always has the same comments, so I’m going to take a moment to debunk the most common ones.
Stack Traces
One common argument against async-await type concurrency is that because the future can be passed around, we end up with bad stack traces when there’s an error. This is true of languages like JavaScript, where if there’s an exception in async code, you end up with super shallow stack traces missing all relevant context.
1
2
3
4
ReferenceError: foo is not defined
at Timeout.setTimeout [as _onTimeout] (/example/test_async.js:4:5)
at listOnTimeout (timers.js:324:15)
at processTimers (timers.js:268:5)
This is mainly just a JavaScript problem. Rust and Python both implement good stack traces.
Caller vs Callee
“It’s the caller’s responsibility to determine if it should be async.” - Someone on the internet
Agreed, that’s actually a point in the favor of async-await. With Go you don’t have the flexibility of promises to either block on them or not, you always block, you just have the option to spawn another thread and have that block. Async-Await doesn’t automatically somehow make it an async background task. Maybe a better way to phrase async-await would be blocking-block as in:
1
2
3
4
blocking function make_http_call() {
let result = block fetch('http://example.com');
return block response.text();
}
It’s really all just labels to encode the fact that some blocking operation is happening and that you may want to be able to do other things concurrently.
Sequential code
“You remember sequential code, that’s the code you can read.” -gar1t
I talked about this briefly above with linting, but sequential code isn’t necessarily that useful, what we actually want is a way to map dependencies between lines of code:
1
2
3
a = 1 + 2
b = a + 3 // dependent on a
c = 4 + 5 // not dependent on a or b
The top to bottom nature of code as text can give us an illusion that this order matters, but c could run before a and b and the result would be identical. While this isn’t super important for math, when blocking operations that can take milliseconds to seconds or longer are happening it’s important to be able to reason about what code is dependent on other code and what can be run concurrently.
I don’t want to deal with category theory, it’s an implementation detail, Go solves this with “good, pragmatic engineering”
These arguments generally confuse more familiar styles of programming with “better” styles. You don’t need to know that monads are monoids in the category of endofunctors, only that you can now code effects directly into the type system that you can rely on. It’s an extra tool in the tool box of writing reliable software. C was the end-all be-all of writing software, developments in programming language theory are real even if it produces about as many duds as good ideas.
Function colors as a way of labeling functions in the type system
Async-Await is only one type of labeling we can do in this space, there are many examples of monads in Haskell but I’d like to point out a few interesting use-cases here.
So if we consider the primary goal of async-await based concurrency to be marking functions that don’t block, we can view this as a way of restricting what affects a function can have. I’m not aware of any language that does this, but one such example that would be good in the embedded systems space is memory allocation. We could introduce a label like async for functions called alloc like:
1
2
3
alloc function make_thing() {
return new Thing();
}
Only alloc functions could call other alloc functions and you know if a function is not marked as alloc that it does not allocate memory.
Similarly in high performance computing it can be helpful to know if code branches or loops, you could build a language where you cannot use for (or recursion) or if unless you explicitly mark the function.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
looping branching function low_pef(list) {
for (a of list) {
if (a % 2 == 0) {
return a;
}
}
return 0;
}
function high_perf(list) {
list[0] += 2;
list[1] += 2;
list[2] += 2;
}
These restrictions allow you to reason about what your program does (and perhaps more importantly want it does not do). While it may be super tedious to mark every possible property of a function
1
2
3
async looping branching alloc logging state_transforming error maybe function my_function() {
// insert code here
}
there is real value when it can prevent your program from behaving in an unexpected way.