New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: document fairness guarantees and current behavior #6145
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -172,6 +172,128 @@ | |
//! [`Builder::enable_io`]: crate::runtime::Builder::enable_io | ||
//! [`Builder::enable_time`]: crate::runtime::Builder::enable_time | ||
//! [`Builder::enable_all`]: crate::runtime::Builder::enable_all | ||
//! | ||
//! # Detailed runtime behavior | ||
//! | ||
//! This section gives more details into how the Tokio runtime will schedule | ||
//! tasks for execution. | ||
//! | ||
//! At its most basic level, a runtime has a collection of tasks that need to be | ||
//! polled. It will repeatedly remove a task from that collection and poll it. | ||
//! When the collection is empty, the thread will go to sleep until a task is | ||
//! added to the collection. | ||
//! | ||
//! However, the above is not sufficient to guarantee a well-behaved runtime. | ||
//! For example, the runtime might have a single task that is always ready to be | ||
//! polled, and poll that task every time. This is a problem because it starves | ||
//! other tasks by not polling them. To solve this, Tokio provides the following | ||
//! fairness guarantee: | ||
//! | ||
//! > Under the following two assumptions: | ||
//! > | ||
//! > * There is some number `MAX_TASKS` such that the total number of tasks on | ||
//! > the runtime never exceeds `MAX_TASKS`. | ||
//! > * There is some number `MAX_POLL` such that calling [`poll`] on any task | ||
//! > spawned on the runtime returns within `MAX_POLL` time units. | ||
Darksonn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
//! > | ||
//! > Then, it is guaranteed that tasks are scheduled fairly, that is, there is | ||
//! > some number `MAX_DELAY` such that when a task becomes ready, it will be | ||
//! > polled by the runtime within `MAX_DELAY` time units. | ||
//! | ||
//! Other than the above fairness guarantee, there is no guarantee about the | ||
//! order in which tasks are polled. There is also no guarantee that the runtime | ||
//! is equally fair to all tasks. For example, if the runtime has two tasks A | ||
//! and B that are both ready, then the runtime may poll A five times before it | ||
//! polls B. This is the case even if A yields using [`yield_now`]. All that is | ||
//! guaranteed is that it will poll B eventually. | ||
//! | ||
//! Normally, tasks are polled only if they have notified the runtime by waking | ||
//! their waker. However, this is not guaranteed, and Tokio may poll tasks that | ||
//! have not notified the runtime under some circumstances. This is called a | ||
//! spurious wakeup. | ||
//! | ||
//! ## IO and timers | ||
//! | ||
//! Beyond just polling tasks, the runtime must also manage IO resources and | ||
//! timers. It does this by periodically checking whether there are any IO | ||
//! resources or timers that are ready, and notifying the relevant task so that | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does "notifying" here mean waking the task? I feel that some clarification is needed because in the previous section we talk about a task notifying the runtime (via the waker) and here we talk about the runtime notifying the task. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've wished every so often that we had a "glossary" of terms and ensured we stuck with the same terms across the docs... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed this to "wake" everywhere. |
||
//! it will be polled. | ||
//! | ||
//! These checks are performed periodically between polling tasks. Under the | ||
//! same assumptions as the previous fairness guarantee, Tokio guarantees that | ||
//! it will notify tasks with an IO or timer event within some maximum number of | ||
//! time units. | ||
//! | ||
//! ## Current behavior (current thread runtime) | ||
Darksonn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
//! | ||
//! This section describes how the [current thread runtime] behaves today. This | ||
//! behavior may change in future versions of Tokio. | ||
//! | ||
//! The current thread runtime maintains two FIFO queues of tasks that are ready | ||
//! to be polled: the global queue and the local queue. The runtime will prefer | ||
//! to choose the next task to poll from the local queue, and will only pick a | ||
//! task from the global queue if the local queue is empty, or if it has picked | ||
//! a task from the local queue 31 times in a row. The number 31 can be | ||
//! changed using the [`global_queue_interval`] setting. | ||
Darksonn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
//! | ||
//! The runtime will check for new IO or timer events whenever there are no | ||
//! tasks ready to be polled, or when it has polled 61 tasks in a row. The | ||
//! number 61 may be changed using the [`event_interval`] setting. | ||
//! | ||
//! When a task is woken from within a task running on the runtime, then the | ||
//! woken task is added directly to the local queue. Otherwise, the task is | ||
//! added to the global queue. The current thread runtime does not use [the lifo | ||
//! slot optimization]. | ||
//! | ||
//! ## Current behavior (multi thread runtime) | ||
//! | ||
//! This section describes how the [multi thread runtime] behaves today. This | ||
//! behavior may change in future versions of Tokio. | ||
//! | ||
//! A multi thread runtime has a fixed number of worker threads, which are all | ||
//! created on startup. The multi thread runtime maintains one global queue, and | ||
//! a local queue for each worker thread. The local queue of a worker thread can | ||
//! fit at most 256 tasks. If more than 256 tasks are added to the local queue, | ||
//! then half of them are moved to the global queue to make space. | ||
//! | ||
//! The runtime will prefer to choose the next task to poll from the local | ||
//! queue, and will only pick a task from the global queue if the local queue is | ||
//! empty, or if it has picked a task from the local queue 61 times in a row. | ||
Darksonn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
//! The number 61 can be changed using the [`global_queue_interval`] setting. If | ||
//! both the local queue and global queue is empty, then the worker thread will | ||
//! attempt to steal tasks from the local queue of another worker thread. | ||
//! Stealing is done by moving half of the tasks in one local queue to another | ||
//! local queue. | ||
//! | ||
//! The runtime will check for new IO or timer events whenever there are no | ||
//! tasks ready to be polled, or when it has polled 61 tasks in a row. The | ||
//! number 61 may be changed using the [`event_interval`] setting. | ||
//! | ||
//! The multi thread runtime uses [the lifo slot optimization]: whenever a task | ||
//! wakes up another task, the other task is added to the worker thread's lifo | ||
//! slot instead of being added to a queue. If there was already a task in the | ||
//! lifo slot when this happened, then the lifo slot is replaced, and the task | ||
//! that used to be in the lifo slot is placed in the thread's local queue. | ||
//! When the runtime finishes polling a task, it will poll the task in the lifo | ||
//! slot immediately, if any. When the lifo slot is used, the [coop budget] is | ||
//! not reset. Furthermore, if a worker thread uses the lifo slot three times in | ||
//! a row, it is temporarily disabled until the worker thread has polled a task | ||
//! that didn't come from the lifo slot. The lifo slot can be disabled using the | ||
//! [`disable_lifo_slot`] setting. The lifo slot is separate from the local | ||
//! queue, so other worker threads cannot steal the task in the lifo slot. | ||
//! | ||
//! When a task is notified from a thread that is not a worker thread, then the | ||
Darksonn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
//! task is placed in the global queue. | ||
//! | ||
//! [`poll`]: std::future::Future::poll | ||
//! [`yield_now`]: crate::task::yield_now | ||
//! [current thread runtime]: crate::runtime::Builder::new_current_thread | ||
//! [multi thread runtime]: crate::runtime::Builder::new_multi_thread | ||
//! [`global_queue_interval`]: crate::runtime::Builder::global_queue_interval | ||
//! [`event_interval`]: crate::runtime::Builder::event_interval | ||
//! [`disable_lifo_slot`]: crate::runtime::Builder::disable_lifo_slot | ||
//! [the lifo slot optimization]: crate::runtime::Builder::disable_lifo_slot | ||
//! [coop budget]: crate::task#cooperative-scheduling | ||
|
||
// At the top due to macros | ||
#[cfg(test)] | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe:
"polling" is pretty specific to the future implementation detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed it to use "schedule" everywhere.