How Tokio Works: The Async Runtime Explained from the Ground Up

A deep dive into Tokio's internals: from what #[tokio::main] actually expands to, to how tasks, wakers, the IO driver, and the work-stealing scheduler all fit together.

My name is Max and I'm the founder of Rustify.

I've been writing Rust professionally for 2 years and I've helped dozens of engineers make the switch to Rust professionally.

If you want to go further, here's how I can help:

Fullstack Bootcamp: structured program with real projects, async support, and private community access.
Blockchain Bootcamp: structured program with real projects, async support, and private community access.
1:1 Mentorship: personalized sessions to get you hired faster, with projects tailored to you and mock interviews.

And if you want more Rust content, I post regularly on my YouTube channel:

You've written #[tokio::main] dozens of times. But do you actually know what it does?

Most Rust developers treat Tokio as a black box. They write async fn, sprinkle .await everywhere, and hope for the best. When something goes wrong: a deadlock, unexpected latency, a task that never completes: they have no mental model to debug from.

This article fixes that. We'll go from #[tokio::main] all the way down to epoll and back up. By the end, you'll know exactly why .await works, why blocking in async code is dangerous, and how Tokio juggles thousands of tasks on a handful of OS threads.

1. What `#[tokio::main]` Actually Expands To

Every Tokio program starts with this:

#[tokio::main]
async fn main() {
    do_stuff().await;
}

It looks like magic. It isn't. The macro expands to this:

fn main() {
    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap()
        .block_on(async {
            do_stuff().await;
        })
}

Three things happen here:

A Runtime is built. This creates worker threads, an IO driver, a timer wheel: the whole machine.
block_on is called. This is the bridge between the synchronous world (fn main) and the async world. It blocks the calling OS thread until the future completes.
Your async code runs inside. Everything after this point is managed by Tokio.

block_on is not magic either. It's just a loop:

while future not complete:
    poll the future
    if pending: wait for IO events
    if event: wake relevant tasks
    repeat

That loop is the runtime. Everything else is infrastructure to make it efficient.

2. Futures: State Machines in Disguise

Before understanding Tokio, you need to understand what async fn actually compiles to.

When you write:

async fn fetch_user(id: u64) -> User {
    let row = db.query(id).await;     // yield point 1
    let profile = api.get(id).await;  // yield point 2
    User::from(row, profile)
}

The compiler doesn't create a thread. It creates a state machine enum:

enum FetchUserFuture {
    State0 { id: u64 },                    // before first await
    State1 { id: u64, row: DbRow },        // waiting on api.get()
    Done,
}

Each .await is a yield point: a place where the future says "I can't continue right now, come back later." The compiler stores everything needed to resume (local variables, where we left off) in that enum.

Nothing runs until someone polls it. A future sitting in memory does nothing. Tokio is the entity that polls futures.

Future created ──► just sits there, doing nothing
                        │
                   Tokio polls it
                        │
              ┌─────────▼──────────┐
              │   Poll::Ready(v)   │  ← done, value available
              │   Poll::Pending    │  ← not ready, come back later
              └────────────────────┘

3. The Full Architecture in One Diagram

Here's the complete picture before we zoom into each part:

┌─────────────────────────────────────────────────────────────────┐
│                        YOUR PROGRAM                             │
│                                                                 │
│   #[tokio::main]  →  Runtime::block_on(main_future)            │
└──────────────────────────────┬──────────────────────────────────┘
                               │ creates
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                       TOKIO RUNTIME                             │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │                      SCHEDULER                            │  │
│  │                                                           │  │
│  │  ┌────────────┐   ┌────────────┐   ┌────────────┐        │  │
│  │  │  Worker 1  │   │  Worker 2  │   │  Worker 3  │        │  │
│  │  │            │   │            │   │            │        │  │
│  │  │[task][task]│   │  [task]    │   │[task][task]│        │  │
│  │  │ local queue│   │ local queue│   │ local queue│        │  │
│  │  └─────┬──────┘   └─────┬──────┘   └─────┬──────┘        │  │
│  │        │  steal ◄───────┘  steal ◄────────┘               │  │
│  └────────┼──────────────────────────────────────────────────┘  │
│           │                                                     │
│           │  futures return Poll::Pending → park here          │
│           ▼                                                     │
│  ┌──────────────────────────────┐  ┌──────────────────────────┐  │
│  │  IO DRIVER  (1 thread)       │  │  TIMER DRIVER            │  │
│  │                              │  │                          │  │
│  │  Linux: epoll                │  │  hierarchical timer wheel│  │
│  │  macOS: kqueue               │  │                          │  │
│  │  Windows: IOCP               │  │  sleep(5s) registered    │  │
│  │                              │  │  → wheel ticks           │  │
│  │  socket ready → calls Waker  │  │  → deadline hit          │  │
│  │  → task re-queued            │  │  → calls Waker           │  │
│  └──────────────────────────────┘  └──────────────────────────┘  │
│       same mechanism: both call Waker to re-queue tasks          │
│  (in practice both run on the same OS thread inside the runtime) │
│                                                                  │
│  ┌───────────────────────────────────────────────────────────┐   │
│  │            BLOCKING THREAD POOL  (separate)               │   │
│  │                                                           │   │
│  │   tokio::task::spawn_blocking(|| std::fs::read(...))      │   │
│  │   → dedicated OS thread, allowed to block                 │   │
│  │   → async workers never stall                             │   │
│  └───────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Four distinct layers:

Scheduler + Workers: poll futures, run tasks
IO Driver: watches OS events (epoll/kqueue), wakes sleeping tasks
Timer Driver: timer wheel that fires Wakers when sleep/timeout/interval expire
Blocking pool: isolated OS threads for blocking code

Mental model for the IO Driver: think of it as a waiting room:

SCHEDULER = work floor        IO DRIVER = waiting room
 
┌─────────────────────┐      ┌─────────────────────┐
│     SCHEDULER       │      │     IO DRIVER        │
│                     │      │                      │
│  Worker 1 : Task B  │      │  Task A sleeps here  │
│  Worker 2 : Task C  │      │  Task D sleeps here  │
│  Worker 3 : Task E  │      │  Task F sleeps here  │
│                     │      │                      │
│  → doing work       │      │  → waiting for data  │
└─────────────────────┘      └─────────────────────┘

A task enters the waiting room when it hits socket.read().await and the data isn't there yet. It stays there until the OS says "data arrived"; then the IO Driver pushes it back to the Scheduler and a worker picks it up.

Without the waiting room, every worker would block waiting for network data and no new requests could be handled. With it, workers stay free and handle other tasks while the IO Driver watches for incoming data.

Concrete example: one line of reqwest:

let resp = reqwest::get("https://api.com/data").await;

You never touch sockets directly. But here's what happens under the hood:

Step 1: your task calls reqwest::get().await
 
Scheduler → Worker 1 → polls your task
reqwest internally:
  1. opens a TCP socket to api.com
  2. sends "GET /data HTTP/1.1..."
  3. waits for response → Poll::Pending
 
IO Driver registers: fd#5 (api.com socket) → Waker of your task
Worker 1: FREE → picks up other tasks

Step 2: api.com responds (100ms later)
 
bytes arrive on fd#5
OS notifies IO Driver: "fd#5 has data"
IO Driver: waker.wake()
Scheduler: puts your task back in the queue

Step 3: your task resumes
 
Worker 2 → polls your task
reqwest reads bytes → parses HTTP response
→ Poll::Ready(resp) ✓
 
let resp = ...  // you have your response

What you see: one line with .await. What happens: socket opened, IO Driver watches, task sleeps, OS notifies, task resumes. All transparent.

In practice you never write TcpStream::read() directly; reqwest, sqlx, axum all do it for you under the hood.

4. Tasks and the Waker: How Sleeping Works

When you call tokio::spawn, you create a task. A task is a future wrapped in metadata Tokio needs to manage it:

Task
├── Future (the state machine)
├── Waker (handle to re-queue this task)
└── State: NOTIFIED | RUNNING | SLEEPING

Here's the full lifecycle:

tokio::spawn(my_future)
        │
        ▼
   Task created → push to worker run queue
        │
        ▼
   Worker polls future
        │
        ├─── Poll::Ready(v) ──► task complete ✓
        │
        └─── Poll::Pending
                  │
                  │  future registers its Waker
                  │  with whatever it's waiting on
                  ▼
             Task state = SLEEPING
             (no CPU used, no thread blocked)
                  │
             (time passes)
             (IO event fires / timer fires)
                  │
                  ▼
             IO Driver calls waker.wake()
                  │
                  ▼
             Task state = NOTIFIED
             pushed to run queue
                  │
                  ▼
             Worker polls again...

The Waker is the key mechanism. It's a lightweight handle: essentially a pointer to a function that re-queues the task. When the OS says "your socket has data", the IO driver finds the Waker registered for that socket, calls it, and the task wakes up on a worker thread.

Zero spinning. Zero wasted CPU. Tasks only run when there's actually something to do.

5. Work-Stealing Scheduler: Why Tokio is Fast

Tokio's multi-thread runtime spawns one worker thread per CPU core by default. Each worker has its own local run queue. But a simple per-thread queue creates a problem: one thread might be flooded while others sit idle.

Work-stealing solves this:

Thread 1 queue:  [A][B][C][D][E]   ← overloaded
Thread 2 queue:  []                ← idle
Thread 3 queue:  [F]
 
Thread 2 sees it's idle → steals half of Thread 1's queue:
 
Thread 1 queue:  [A][B]
Thread 2 queue:  [C][D]            ← stolen tasks
Thread 3 queue:  [F]

Automatic load balancing with no central coordinator. No lock contention. Threads steal from the back while the owner pops from the front.

This is why Tokio scales well under mixed workloads: CPU-bound tasks don't starve IO-bound tasks, because idle threads steal work.

6. The Critical Rule: Never Block an Async Thread

This is the most important practical rule from all of the above, and the most commonly broken one. Blocking inside an async worker is the single most frequent production issue I see in code reviews with my clients.

Worker threads are shared. If you block one, every task queued on it waits. Not pauses: completely stuck.

Worker Thread 1:
  running Task A → calls std::thread::sleep(5s)
  
  ENTIRE THREAD BLOCKED FOR 5 SECONDS
  
  Task B, C, D on this thread: stuck. Not running.

What "blocking" means:

std::thread::sleep: blocks
std::fs::read_to_string: blocks (synchronous file IO)
A heavy CPU loop (1M iterations): blocks
std::sync::Mutex::lock that waits: blocks

What to do instead:

Situation	Wrong	Right
Sleep	`std::thread::sleep`	`tokio::time::sleep`
File IO	`std::fs::read`	`tokio::fs::read`
CPU work	inline	`tokio::task::spawn_blocking`
Sync mutex with long hold	`std::sync::Mutex`	`tokio::sync::Mutex`

spawn_blocking moves the work to the dedicated blocking thread pool, which is allowed to block. The async worker stays free.

// Wrong: blocks the async worker
let content = std::fs::read_to_string("file.txt").unwrap();
 
// Right: moves blocking IO to the blocking pool
let content = tokio::task::spawn_blocking(|| {
    std::fs::read_to_string("file.txt")
}).await.unwrap().unwrap();

7. `tokio::select!`: Racing Futures

select! runs multiple futures concurrently and completes when the first one finishes. The others are dropped (cancelled).

tokio::select! {
    result = some_operation() => {
        println!("operation done: {:?}", result);
    }
    _ = tokio::time::sleep(Duration::from_secs(5)) => {
        println!("timeout");
    }
}

Internally, select! polls all branches on each iteration. First branch to return Poll::Ready wins. The rest are dropped immediately.

Iteration 1:
  poll operation()  → Pending
  poll sleep(5s)    → Pending
  → wait for any Waker to fire
 
IO event fires on operation():
  poll operation()  → Ready(result)  ← WINNER
  drop sleep future
  execute first branch

This is the canonical pattern for timeouts and cancellation in Tokio code.

8. Cancellation: Drop is Cancel

In Tokio, there's no special cancel API. Dropping a future cancels it.

When a task is dropped:

Its state machine is destroyed
In-progress work is abandoned at the current .await point
No cleanup runs automatically

Task: [State2 { connection, buffer }]
         │
    dropped here
         │
         ▼
    State machine destroyed.
    connection dropped (triggers Drop impl if any).
    buffer freed.
    Any pending DB write: NOT committed.

This is why cancellation safety matters. If your future holds a half-written database transaction and gets dropped at an .await, the transaction is abandoned. Not rolled back: just abandoned.

Use tokio_util::sync::CancellationToken for graceful cancellation that gives tasks a chance to clean up:

let token = CancellationToken::new();
let child = token.child_token();
 
tokio::spawn(async move {
    tokio::select! {
        _ = child.cancelled() => {
            // clean up here
        }
        _ = do_work() => {}
    }
});
 
token.cancel(); // signals the task to stop cleanly

9. Common Footguns and How to Avoid Them

┌─────────────────────────────────────────────────────────────────┐
│                        FOOTGUN MAP                              │
├──────────────────────────┬──────────────────────────────────────┤
│ Holding std::sync::Mutex │ async fn holds lock across .await    │
│ across .await            │ → other tasks that need the lock     │
│                          │   can never acquire it               │
│                          │ Fix: tokio::sync::Mutex              │
├──────────────────────────┼──────────────────────────────────────┤
│ Non-Send future in       │ Rc, RefCell, raw ptr across .await   │
│ multi-thread runtime     │ → compile error: Future not Send     │
│                          │ Fix: Arc, use Send-safe types        │
├──────────────────────────┼──────────────────────────────────────┤
│ spawn inside             │ runtime.block_on(async {             │
│ block_on                 │   tokio::spawn(...)  ← panics        │
│                          │ }): no runtime context for spawn    │
│                          │ Fix: use tokio::main or spawn inside │
│                          │ an actual runtime handle             │
├──────────────────────────┼──────────────────────────────────────┤
│ Assuming spawn = serial  │ Two spawned tasks run concurrently   │
│                          │ on multi-thread runtime: order not  │
│                          │ guaranteed                           │
│                          │ Fix: use channels or join! if order  │
│                          │ matters                              │
└──────────────────────────┴──────────────────────────────────────┘

Quick Reference

The mental model:

async fn = state machine, nothing runs until polled
tokio::spawn = create a task, schedule it on a worker
.await = yield point, park this task until ready
IO Driver = calls Waker when OS socket/event is ready
Timer Driver = calls Waker when sleep/timeout/interval expires
Worker threads = never block these

Key APIs:

tokio::spawn(fut)              → concurrent task
tokio::task::spawn_blocking    → blocking code on separate pool
tokio::time::sleep             → async sleep (non-blocking)
tokio::sync::Mutex             → mutex safe across .await
tokio::select!                 → race futures, first wins
CancellationToken              → graceful task cancellation

Red flags in your code:

std::thread::sleep inside async fn
std::fs::* inside async fn
std::sync::Mutex held across .await
Rc or RefCell across .await in multi-thread runtime
CPU-heavy loop without spawn_blocking

Most Rust developers who use tokio fall into one of two positions.

The first position: tokio mostly works, and when something goes wrong, they add logs, restart things, hope it resolves. They've never fully debugged a hung task from first principles. The runtime stays a black box.

The second position: they've internalized the executor model. When a task never completes, they know where to look. When latency spikes unexpectedly, they have a mental model to work from. When someone asks "why can't we just block here?", they can explain exactly what that does to the thread pool.

That second position is where the interesting infrastructure conversations happen. It's also reachable in weeks, not years, with the right structure and someone to debug alongside.

If you want to get there with a real production codebase, that's what 1:1 mentorship is for.

Book a session →

If you want to go deeper on async Rust, concurrent architecture patterns, or how to structure a real production Tokio app, that's exactly what the Rustify Bootcamp covers end-to-end:

Fullstack Bootcamp: structured program with real projects, async support, and private community access.
Blockchain Bootcamp: structured program with real projects, async support, and private community access.
1:1 Mentorship: personalized sessions to get you hired faster, with projects tailored to you and mock interviews.

Student Success Stories

Hear from engineers who built real Rust projects with Rustify

Arik Dutta

Technical Lead · Low-code & Python → Rust

Tiago Afonso

Fullstack Developer

Ugo Tiberto

Rust Engineer · Fullstack Developer

Watch all student stories on YouTube