Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rt(threaded): cap LIFO slot polls #5712

Merged
merged 11 commits into from
May 23, 2023
Merged

rt(threaded): cap LIFO slot polls #5712

merged 11 commits into from
May 23, 2023

Commits on May 23, 2023

  1. rt(threaded): cap LIFO slot polls 

    As an optimization to improve locality, the multi-threaded scheduler
maintains a single slot (LIFO slot). When a task is scheduled, it goes
into the LIFO slot. The scheduler will run tasks in the LIFO slot first,
before checking the local queue.

In ping-ping style workloads where task A notifies task B, which
notifies task A again, this can cause starvation as these two tasks will
repeatedly schedule the other in the LIFO slot. #5686, a first
attempt at solving this problem, consumes a unit of budget each time a
task is scheduled from the LIFO slot. However, at the time of this
commit, the scheduler allocates 128 units of budget for each chunk of
work. This is quite high in situation where tasks do not perform many
async operations, yet have meaningful poll times (even 5-10 microsecond
poll times can have outsized impact on the scheduler).

In an ideal world, the scheduler would adapt to the workload it is
executing. However, as a stopgap, this commit limits the number of times
the LIFO slot is prioritized per scheduler tick.
    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    8b60f58 View commit details
    Browse the repository at this point in the history

  2. remove consume_budget when scheduling from LIFO

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    4ff6296 View commit details
    Browse the repository at this point in the history

  3. rm unused method

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    0c05de8 View commit details
    Browse the repository at this point in the history

  4. move LIFO enabled check to schedule_local

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    7722aee View commit details
    Browse the repository at this point in the history

  5. add counters + include in CI

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    b6d37ea View commit details
    Browse the repository at this point in the history

  6. reset lifo_enabled after stealing core`

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    61dbf13 View commit details
    Browse the repository at this point in the history

  7. add debug assertions

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    91e849f View commit details
    Browse the repository at this point in the history

  8. fmt

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    57d550a View commit details
    Browse the repository at this point in the history

  9. another assertion

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    5cc3716 View commit details
    Browse the repository at this point in the history

  10. enable debug assertions in loom tests

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    e4ae2f8 View commit details
    Browse the repository at this point in the history

  11. tweak

    @carllerche
    carllerche committed May 23, 2023
    Configuration menu
    Copy the full SHA
    7701edc View commit details
    Browse the repository at this point in the history