Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

On-demand log-spacemap flush; zpool condense command #16747

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

On-demand log-spacemap flush; zpool condense command #16747

wants to merge 6 commits into from

Commits on Nov 13, 2024

  1. spa_stats: kstats for unflushed log spacemaps 

    Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    e967723 View commit details
    Browse the repository at this point in the history

  2. log_spacemap: extend pool flushall to have "request" and "export" modes 

    Normally, log spacemaps are flushed out to the metaslabs when the pool
is exported. For large logs, this can lead to export taking an
inordinate amount of time.

This commit adds a "mode" parameter for the log spacemap "flushall"
operation, and functions for starting and stopping it in a particular
mode. The existing behaviour of flushing everything is now the "export"
mode.

Then, we add a new "request" mode, that can be triggered externally.
This mode differs in that it only flushes spacemaps that were dirtied
before the current transaction, stopping when the only dirty ones
remaining, if any, are newer.

This commit only adds the behaviours and sets up the entry points; the
next commit will add something to call them.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    aa2bcb0 View commit details
    Browse the repository at this point in the history

  3. zpool: add condense verb and wire up to log-spacemap flush 

    The idea is to have a single command that could signal to any background
cleanup task that it should do its work faster, or care less about not
getting in the way of user IO, or whatever.

This adds the the `zpool condense` command, the `ZFS_IOC_POOL_CONDENSE`
ioctl and counters so userspace can get progress. Because the target
could be anything, there's no particular unit, just a total number of
items to condense and count of how many done.

It also adds a `log-spacemap` condense target, which calls the "request
log flush" function.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    8205d5a View commit details
    Browse the repository at this point in the history

  4. condense: add support for waiting for condense to complete 

    Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    963af68 View commit details
    Browse the repository at this point in the history

  5. ztest: periodically start/stop log spacemap flush 

    Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    94b1364 View commit details
    Browse the repository at this point in the history

  6. zts: add test for log spacemap flushall + zpool condense 

    Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
    @robn
    robn committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    0aa7b7f View commit details
    Browse the repository at this point in the history