Thinking About Async Rust

February 9, 2025

I recently noticed that I've been writing asynchronous code pretty frequently at work. Actually, nowadays it's much more common for me to work with async rust than just "vanilla" rust. The distinction sounds a little weird when you think about it–both are still just Rust–but the mental model for writing async code is a sort of "extension" of vanilla Rust; All the normal lifetime and borrowing rules apply, and you get plenty of complaints from the compiler if you do something wrong but from within an async context.

I've had a lot of time to think about my mental model for async Rust, and I've talked to other engineers about how they view it. I'm not typically one for analogies, but I certainly think they're extremely helpful when starting to think about a foreign concept, as long as you later leave them behind like the training wheels on your bike and understand what actually happens. Let's do that with async Rust! The approach in this post is straight-forward: we'll think of the async environment in Rust as a restaurant with clients, chefs, tables and dishes, and we'll go back and forth between the analogy and reality, exploring a little more in depth how Rust approaches async, and how other languages like Go approach the same concept.

The Key Players in Rust's Async World

Before diving into our restaurant analogy, let's identify what we're actually trying to model here:

Rust's async landscape:
- Rust std
  - Futures
  - Wakers
- Runtime (tokio, async-std, smol)
  - Executor
  - Reactor

I'm going to explain each of these components and how they interact. If you're anything like me, just having a list doesn't do much for understanding, so let's map these to our restaurant metaphor.

Futures: The Recipe Cards

If you check the standard library, a Future is defined as:

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

It's a trait with a single required method, poll. This method gets called to advance the computation. The method returns either Poll::Ready(value) if the future has completed, or Poll::Pending if it needs to wait for something.

Futures are lazy and inert. They don't do anything on their own - they just sit there until someone polls them. This is a key aspect of Rust's approach to async.

In our restaurant metaphor, futures are like recipe cards. A recipe describes the steps to prepare a dish but doesn't actually cook anything until a chef picks it up and follows it.

When we write something like:

async fn cook_dish(dish: &str, cooking_time: u64) -> String {
    println!("👨‍🍳 Chef started cooking: {}", dish);
    time::sleep(Duration::from_secs(cooking_time)).await;
    println!("👨‍🍳 Chef finished cooking: {}", dish);
    format!("Dish: {}", dish)
}

Calling this function doesn't actually do any cooking. It just returns a future - basically a promise that the dish will be cooked when someone follows this recipe. The actual work happens when this future is awaited somewhere.

Just as a recipe might require certain equipment or ingredients to be ready before proceeding to the next step, futures can be composed of other futures. This brings us to an important distinction:

  • Leaf futures represent primitive operations, usually I/O-related, like tokio::net::TcpStream::connect("127.0.0.1:8000"). These are like fundamental cooking techniques - boiling water, frying an egg.

  • Non-leaf futures combine multiple steps, like when you write an async fn that awaits multiple operations. These are like full recipes that combine several techniques.

Tasks: The Chefs in Action

A future on its own doesn't do any work - it needs someone to execute it. In Rust's async world, a task is a future that has been submitted to an executor for execution.

The Tokio docs put it clearly: "In the Tokio execution model, futures are lazy. When a future is created, no work is performed. In order for the work defined by the future to happen, the future must be submitted to an executor. A future that is submitted to an executor is called a 'task'."

In our restaurant, the distinction is:

  • Futures are recipe cards - just instructions
  • Tasks are chefs following those recipes - actually doing the work

When we write code like:

let table1 = task::spawn(serve_table(1));

We're taking the serve_table(1) recipe and assigning a chef to execute it. The spawn function creates a new task, which can be scheduled and executed independently.

The Executor: Our Restaurant Manager

The executor is responsible for polling futures until they complete. It decides which tasks get CPU time and when.

The executor keeps track of a collection of futures. It picks a future, polls it, and gives it a way to signal when it's ready to make progress (a Waker). The future runs until either it completes or it hits an .await point where it can't proceed yet. At that point, the executor can move on to poll another future.

In our restaurant, the executor is the manager who assigns work to chefs and coordinates the kitchen. The manager decides which chef works on which dish and when, ensuring that the kitchen runs efficiently.

The Reactor: Kitchen Notification System

While the executor is actively polling futures, the reactor is passively waiting for I/O events to occur. It's watching for things like "is this network request complete?" or "has this timer expired?".

The reactor maintains a collection of Wakers and notifies the appropriate one when an event happens. When the reactor receives a notification that an event has occurred, it finds the corresponding waker and calls wake() on it, which tells the executor that the associated future is ready to make progress.

In our restaurant, the reactor is the notification system - the bells, timers, and buzzers that let chefs know when something is ready. Think of the bell that rings when an order is up or the timer that goes off when something is done baking.

Importantly, the executor and reactor don't communicate directly. They coordinate through wakers.

Wakers: The Pager System

A Waker is a handle for notifying that a task should be polled again. It's how the reactor tells the executor "hey, this future you were waiting on can make progress now!"

When a future returns Poll::Pending, it typically holds onto the waker it received. Later, when some event occurs that would let the future make progress, that waker gets called.

In our restaurant, wakers are like pagers given to chefs. When ingredients arrive or equipment becomes available, the chef gets paged to come back to a dish they had to set aside.

Two Ways to Cook: Intertask vs. Intratask Concurrency

Now that we have the basic components, let's talk about how concurrency actually works in this environment.

Intertask Concurrency: Multiple Chefs

When you want completely independent operations to happen concurrently, you use intertask concurrency by spawning multiple tasks. This is like having multiple chefs working independently on different orders:

let table1 = task::spawn(serve_table(1));
let table2 = task::spawn(serve_table(2));

Each task gets its own "thread" of execution. With a multi-threaded runtime like Tokio's default, these tasks can genuinely run in parallel across different CPU cores.

Intratask Concurrency: One Chef, Multiple Dishes

Sometimes you want concurrency within a single task. This happens when you use combinators like join to run multiple futures concurrently but within the same task context:

let dish1_future = cook_dish(&order[0], 3);
let dish2_future = cook_dish(&order[1], 4);
let (dish1, dish2) = join(dish1_future, dish2_future).await;

This is like having one chef cooking multiple dishes at once - working on one while another simmers. It's more resource-efficient since you're only using one task, but all these futures are dependent on the same task making progress.

One interesting thing about this approach is that it creates what some Rust engineers call a "perfectly sized stack" - the exact amount of memory needed for all the operations is allocated upfront as part of the task's state machine. This means no dynamic allocations are needed for each concurrent operation, making it extremely efficient.

Serving Strategies: Join vs. Select

Rust provides two main patterns for handling multiple futures:

Join: The Complete Table Service

The join pattern runs multiple futures concurrently and waits for all of them to complete. It's like a waiter ensuring that all dishes for a table are ready before serving:

let (dish1, dish2) = join(dish1_future, dish2_future).await;

This says: "Cook these dishes concurrently, but don't proceed until both are done." Only when all futures complete will the code continue, with the results of all futures in a tuple.

Under the hood, when the executor polls the joined future, it polls each of the child futures. If any return Pending, the joined future also returns Pending. Only when all child futures return Ready will the joined future return Ready with all the results.

This is perfect for when you need all parts of an operation to complete before proceeding - like ensuring all API calls finish before rendering a UI, or all database queries complete before generating a report.

Select: First Come, First Served

While join waits for all futures, select races them and takes the first one to finish:

let coffee = async { 
    time::sleep(Duration::from_secs(2)).await;
    "Coffee"
};

let cocktail = async {
    time::sleep(Duration::from_secs(3)).await;
    "Cocktail"
};

pin_mut!(coffee);
pin_mut!(cocktail);
let winner = select(coffee, cocktail).await;

This is like a bartender serving whichever drink gets prepared first. As soon as one future completes, the select returns its result (along with the still-pending future, which you can continue to poll if needed).

The select pattern is ideal for:

  • Implementing timeouts (race an operation against a timer)
  • Fetching data from multiple redundant sources (use whichever responds first)
  • Handling user input that could come from multiple sources

You might have noticed that we need to pin the futures before using select. This brings us to an important concept in async Rust.

Pinning: Keeping Things in Place

Pinning is about ensuring that a value doesn't move in memory after being polled. This is crucial because the state machines generated by async functions can contain self-references - references to data within themselves. If the future moved in memory after being polled, these internal references would become invalid.

Looking at the Future trait again:

fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;

The self: Pin<&mut Self> parameter means the future must be pinned before polling.

In our restaurant, pinning is like designating a fixed workstation for a chef preparing a complex dish. Once they've set up their station with specific tools and ingredients in particular places, you can't suddenly move them to a different counter without disrupting everything.

For most code, the compiler handles pinning for you. But for advanced patterns or when implementing futures directly, you need to work with pinning explicitly.

State Machines: The Hidden Compiler Magic

When you write an async fn, the Rust compiler transforms it into a state machine where each .await point becomes a different state. This is similar to a numbered recipe:

┌───────────────────────────────┐
│ Recipe: Pasta Carbonara       │
│                               │
│ 1. Boil water and cook pasta  │
│ 2. Fry pancetta               │
│ 3. Mix eggs and cheese        │
│ 4. Combine all ingredients    │
│ 5. Garnish and serve          │
└───────────────────────────────┘

The chef (or executor) follows these steps in order but might pause after step 1 while waiting for pasta to cook. When they return, they pick up exactly where they left off.

An async function like:

async fn prepare_carbonara() -> Dish {
    let pasta = boil_pasta().await;
    let pancetta = fry_pancetta().await;
    let sauce = mix_eggs_and_cheese().await;
    let combined = combine(pasta, pancetta, sauce).await;
    garnish_and_serve(combined).await
}

Each .await point represents a potential pause where the task might yield control to let other tasks run.

The Visualization of Futures

To better understand how futures work together, let's visualize them in a hierarchical structure:

┌─────────────────────────────────┐
│        Top-level future         │
│                                 │
│  ┌──────────┐      ┌──────────┐ │
│  │  Child   │      │  Child   │ │
│  │ Future A │      │ Future B │ │
│  └──────────┘      └──────────┘ │
└─────────────────────────────────┘

When the executor polls the top-level future, it polls all of its child futures until it reaches leaf futures (which often represent I/O resources). The top-level future won't complete until all child futures complete. If any future returns Poll::Pending, that result propagates up the chain immediately.

In Our Restaurant: Order Completion

In our restaurant, this is like the completion of a table's entire order:

┌────────────────────────────────────┐
│        Table 1's entire order      │
│                                    │
│ ┌─────────┐  ┌─────────┐ ┌───────┐ │
│ │Appetizer│  │ Main    │ │Dessert│ │
│ └─────────┘  └─────────┘ └───────┘ │
└────────────────────────────────────┘

The table's order isn't complete until all three courses are ready. If any course isn't ready yet, the entire order is considered pending.

The Async Communication Flow

Let's put it all together and see how these components interact:

┌──────────┐          ┌──────────┐
│          │  poll    │          │
│ Executor │─────────►│  Future  │
│          │          │          │
└──────────┘          └──────────┘
     ▲                      │
     │                      │ 
     │ notify               │ register
     │                      │
     │                      ▼
┌──────────┐          ┌──────────┐
│          │  event   │          │
│  Waker   │◄─────────│  Reactor │
│          │          │          │
└──────────┘          └──────────┘

The cycle goes:

  1. Executor polls a future
  2. Future either completes or returns Pending and registers a waker
  3. Reactor watches for events
  4. When an event occurs, reactor wakes the appropriate future
  5. Executor notices the wakeup and polls the future again

This design allows for efficient handling of thousands of concurrent operations with minimal resource usage.

Comparing tokio::join vs tokio::spawn

To better understand the difference between intratask and intertask concurrency, let's visualize them:

Intratask Concurrency (join)     Intertask Concurrency (spawn)
┌──────────────────────┐         ┌──────────────────┐
│     Single Task      │         │       Task 1     │
│  ┌───────┐ ┌───────┐ │         └──────────────────┘
│  │Future1│ │Future2│ │                  │
│  └───────┘ └───────┘ │                  ▼
└──────────────────────┘         ┌──────────────────┐
                                 │       Task 2     │
                                 └──────────────────┘

In a practical example:

// Intratask concurrency with join
async fn intratask_example() {
    // One chef preparing multiple dishes
    let (dish1, dish2) = tokio::join!(
        cook_dish("Pasta", 3),
        cook_dish("Salad", 2)
    );
    // Both operations run concurrently within this task
    // Control returns here only when both are complete
}

// Intertask concurrency with spawn
async fn intertask_example() {
    // Different chefs preparing different dishes
    let handle1 = tokio::spawn(cook_dish("Pasta", 3));
    let handle2 = tokio::spawn(cook_dish("Salad", 2));
    
    // These operations run as independent tasks
    // We need to await the handles to get their results
    let dish1 = handle1.await.unwrap();
    let dish2 = handle2.await.unwrap();
}

The key differences:

  • join runs futures within the same task (one chef multitasking)
  • spawn creates separate tasks that can run independently (multiple chefs)
  • spawn has more overhead but allows truly parallel execution on multiple threads

The Hidden Danger: Blocking Functions in Async Code

One thing that catches many Rust developers by surprise is that regular blocking functions and async functions look identical to the caller except for the async keyword and .await syntax. This can lead to some insidious bugs.

For example, if you accidentally use this:

// BAD: Blocks the worker thread
std::thread::sleep(Duration::from_secs(10));

Instead of this:

// GOOD: Yields control during sleep
tokio::time::sleep(Duration::from_secs(10)).await;

You're blocking an entire worker thread that could be running other tasks! It might not be immediately obvious that anything is wrong, especially in a multi-threaded runtime, but you'll likely see performance degradation.

In our restaurant analogy, this is like having a chef who refuses to step aside when waiting for water to boil, preventing other chefs from using the stove. A better chef puts the water on, sets a timer, and works on other dishes in the meantime.

The most problematic case is the standard library's Mutex. If you hold this mutex across an .await point:

// DANGER ZONE
let mut data = mutex.lock().unwrap();
something_async().await;  // Other tasks can't take the lock while we wait!
*data = new_value;

You could easily deadlock if another task on the same thread tries to take the same lock while your task is waiting on the future. This is why async runtimes usually provide their own async mutex implementations.

Go's Approach: Goroutines

To provide some contrast, let's look at how Go handles concurrency with goroutines, which are quite different from Rust's futures.

Goroutines are lightweight threads managed by the Go runtime. Unlike OS threads, which might require megabytes of stack space, goroutines start with just a few kilobytes and can grow or shrink as needed. This makes them much more resource-efficient than traditional threads.

The Go runtime includes a scheduler that multiplexes goroutines onto a smaller number of OS threads. This scheduler operates on an m:n scheduling principle, where many goroutines (m) are mapped onto a smaller number of OS threads (n).

Here's a simple example in Go:

func printMessage() {
    fmt.Println("Hello from goroutine")
}

func main() {
    go printMessage()  // Start a new goroutine
    fmt.Println("Hello from main function")
    time.Sleep(time.Second)  // Wait for the goroutine to finish
}

The go keyword starts a new goroutine, and the Go runtime takes care of scheduling it efficiently.

Key Differences from Rust's Approach

  1. Implicit vs. Explicit Yielding: In Go, the runtime can preempt goroutines during function calls or at certain points, allowing for transparent scheduling without explicit yield points. In Rust, you must explicitly .await to yield control.

  2. Memory Model: Go's goroutines are stackful coroutines with a dynamic stack. Rust's futures are stackless coroutines that store their state in a state machine on the heap.

  3. Programming Model: In Go, you write code as if it were synchronous, and the runtime handles the asynchronous parts. In Rust, async functions are explicitly marked, making the distinction between synchronous and asynchronous code clear in the type system.

  4. Error Handling: Go uses multiple return values for error handling, while Rust uses the Result type, which can be combined with async using the ? operator.

Go's approach makes concurrent programming simpler, but at the cost of some control and transparency. Rust's approach is more explicit and gives you more control, but requires more careful management of async/await points.

Wrapping Up

Hopefully this restaurant analogy helps clarify how Rust's async system works. To summarize:

  • Futures are recipes waiting to be executed
  • Tasks are chefs doing the work
  • Executors are managers coordinating the kitchen
  • Reactors are notification systems for when things are ready
  • Wakers are pagers notifying chefs
  • Join ensures all dishes are ready before serving
  • Select takes whatever is ready first
  • Await points are where chefs can switch to other tasks

With these concepts in mind, you should be better equipped to write effective async Rust code. While the analogy is helpful for building an initial mental model, the real power comes from understanding the actual mechanisms at work. The more you work with async Rust, the more intuitive these concepts will become, and eventually you won't need the restaurant analogy at all.