[6.18] Track btrfs patches

[kakra]

Export patch series: https://github.com/kakra/linux/pull/40.patch

btrfs: tiered allocation hints and queue-based read balancing

Special Thanks to @Forza-tng for extensive testing, feedback, and maintaining the documentation guide:
👉 Btrfs Allocator Hints and Read Policies Guide by Forza-tng

---

This PR introduces a set of patches to improve Btrfs performance and flexibility in heterogeneous storage environments (mixed SSD/HDD, tiered storage, bcache).

1. Allocator Hints (Data Placement)

Allows preferring specific devices for data or metadata allocations. This works by storing a hint in the persistent device item on-disk.

• Essential for tiered setups: Force metadata onto SSDs (for speed) while keeping bulk data on HDDs.
• Graceful removal: Mark devices to accept no new allocations, allowing to drain them naturally or via balance without racing against new writes.

2. Read Balancing Policies

Extends Btrfs RAID1 read balancing with queue-aware read policies. The standard PID-based policy is often insufficient for mixed-device pools, high-IOPS workloads, or devices with unstable latency.

• pid: (Default) Static hashing by process ID. Good for simple setups.
• round-robin: Distributes reads equally. Good for aggregate throughput on identical disks.
• queue: (Old recommendation) Routes requests to the device with the fewest in-flight requests.
  • Adapts instantly to current device load.
  • Breaks equal-depth ties by mirror age, so idle mirrors are rotated more fairly instead of implicitly preferring the first candidate.
  • Works well for healthy devices and general mixed-load systems.
• queue-adaptive: (Recommended for latency-sensitive desktops or mixed/aging devices, still experimental) Keeps the queue policy as the primary selector, but adds a peer-relative latency guard.
  • Temporarily avoids mirror candidates whose recent read latency is much worse than their current peer.
  • Helps with transient SSD/NVMe latency spikes, slow HDD areas, weak sectors, and mixed pools where one mirror is temporarily much slower.
  • Still allows a slower mirror to absorb overflow when the faster peer is sufficiently busier.
  • Can reduce desktop and gaming stalls caused by latency-sick mirror members.
• devid: Pin reads to a specific device ID (mostly for testing).

On fully healthy and evenly performing devices, queue-adaptive should stay close to queue. Its benefit is expected to be most visible under mixed load, mixed devices, or devices with temporary or localized latency spikes.

(Note: Older global latency-based policies were dropped in favor of the queue-based design. queue-adaptive uses latency only as a peer-relative guard on top of queue, not as the primary selector.)

3. Decoupling from Experimental Status

Important: Upstream Kernels (6.13+) use CONFIG_BTRFS_EXPERIMENTAL to gate various unstable work-in-progress features. To allow using the allocator hints and read policies without enabling potentially unstable upstream code, these features have been moved out of the experimental gate.

Recommendation: Remove the line CONFIG_BTRFS_EXPERIMENTAL from your .config before running make oldconfig. The build system will then prompt you specifically for the new options (CONFIG_BTRFS_ALLOCATOR_HINTS, CONFIG_BTRFS_READ_POLICIES), allowing you to enable them safely without turning on other experimental Btrfs features.

---

Quickstart Guide

Setting Allocator Hints

1. Enable CONFIG_BTRFS_ALLOCATOR_HINTS in kernel config.
2. Run btrfs device usage /mnt/path to identify your device IDs.
3. Set the hint via sysfs (this persists on-disk, no udev rules needed!):
   echo <TYPE> | sudo tee /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/type

Available Types:

• 0: Prefer data (Default for HDDs).
• 1: Prefer metadata (Recommended for SSDs/NVMe).
• 2: Metadata only (Use with caution).
• 3: Data only (Use with caution).
• 4: None preferred (Avoids new allocations, useful to drain a drive).
• Added: 5: None (Strictly prevents ANY new allocation, useful for parallel device remove).

After changing hints, a rebalance of metadata/data is required to move existing extents to their preferred location.

Enabling Read Policy

1. Enable CONFIG_BTRFS_READ_POLICIES in kernel config.
2. Set boot parameter: btrfs.read_policy=queue-adaptive or btrfs.read_policy=queue
3. Or switch at runtime:
   • echo queue-adaptive | sudo tee /sys/fs/btrfs/<UUID>/read_policy
   • echo queue | sudo tee /sys/fs/btrfs/<UUID>/read_policy

---

Diagnostic Statistics

Adds per-device read statistics to /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/read_stats.

ios %lu wait %llu avg %llu age %llu ignored %llu health avg %llu ago %llu score %llu vetoed %llu overflow %llu

• ios: Total read I/O count.
• wait: Total accumulated read wait time in nanoseconds.
• avg: Cumulative average read latency in nanoseconds.
• age: Fairness counter. Increments while the device is a read candidate but not selected. Resets to 0 when selected.
• ignored: Total count of times this device was a candidate but skipped.
• health avg: Recent windowed average read latency in nanoseconds, used by queue-adaptive.
• ago: Seconds since health avg was last successfully refreshed.
• score: Ratio between cumulative average latency and recent health latency. Values above 100 mean the recent window is faster than the cumulative average; values below 100 mean it is slower.
• vetoed: Number of times queue-adaptive rejected this device because its recent latency looked much worse than its peer.
• overflow: Number of times queue-adaptive still selected this slower-looking device because the healthier peer was already sufficiently busier.

---

Benchmark Results

The following benchmarks (based on kernel 6.12 LTS) compare the new policies against the defaults. Tests were performed on a mixed HDD RAID10 array with bcache, comparing an idle system vs. a system under heavy background load (defrag).

queue proved to be the superior all-rounder, effectively isolating foreground workloads from background noise. queue-adaptive hasn't been benchmarked yet. The following are the results from the old version:

Scenario: No Background Load

Policy	RandRead 4k QD1 (Lat)	RandRead 4k QD32 (IOPS)	SeqRead 1M (BW)
pid	65 IOPS	537 IOPS	261 MiB/s
round-robin	241 IOPS	1180 IOPS	231 MiB/s
latency-rr*	702 IOPS	2477 IOPS	240 MiB/s
queue	1181 IOPS	3647 IOPS	272 MiB/s

Scenario: Heavy Background Load (Defrag)

Policy	RandRead 4k QD1 (Lat)	RandRead 4k QD32 (IOPS)	SeqRead 1M (BW)
pid	~0 IOPS (Stalled)	505 IOPS	121 MiB/s
round-robin	38 IOPS	717 IOPS	126 MiB/s
latency-rr*	585 IOPS	1562 IOPS	235 MiB/s
queue	967 IOPS	2437 IOPS	247 MiB/s

(latency-rr was an experimental hybrid policy used during testing, superseded by queue due to better performance and simplicity, queue-adaptive not yet included in these results)

---

Changes in this version (Kernel 6.18 Port)

• Ported to Linux 6.18.
• Refactored Kconfig: Features are individually selectable and moved out of "Experimental".
• Robustness: Fixed potential NULL pointer dereferences in stats tracking during mount/unmount and race conditions in round-robin calculation.
• Simplified: Dropped older global/EMA-based latency policies. The new queue-adaptive policy uses latency only as a peer-relative guard on top of queue.

---

Status

• Intended for advanced users and maintained out-of-tree.
• Not expected to be accepted upstream in current form.
• Allocator hints are experimental but used in production by maintainers/testers.
• Use btrfs filesystem usage -T after enabling hints to confirm metadata placement.

---

Upstream outlook

This patchset is not currently an upstream submission. It serves as a practical implementation and testbed for tiered Btrfs allocation and read policy ideas. Some concepts may later be replaced by upstream-native mechanisms.

---

FAQ: Why is this not upstream?

1. Allocator Hints

The allocator hint patches (originally developed by Goffredo Baroncelli, now maintained here) have been discussed on the mailing list but were not merged for design reasons:

• Free Space Calculation (df): Btrfs calculates available space assuming any chunk can be allocated on any device (respecting RAID profiles). Restricting allocations via hints makes this calculation unreliable. Tools might report free space while Btrfs returns ENOSPC (No space left on device) because the allowed devices for a specific chunk type are full, even if other devices are empty.
• Maintenance: The original author ceased updates for newer kernels; this repository bridges that gap.

Compatibility Note: This patch reuses the existing (unused) type field in the device item on disk. It does not change the on-disk format version. Unpatched kernels simply ignore the value, ensuring data remains accessible (though allocation preferences will be lost until booted with a patched kernel).

2. Read Policies (queue and queue-adaptive)

The new queue and queue-adaptive policies are an experimental addition in this patch set. It is unlikely to be accepted upstream in its current form due to Layer Violation:

• The filesystem layer (Btrfs) directly accesses internal block-layer statistics (in-flight queue depth) to make routing decisions. The Linux kernel generally enforces strict separation between these subsystems.
• However, benchmarks show this cross-layer optimization yields significant performance gains in mixed setups, justifying its inclusion here.

[CHN-beta]

I tried this patch with CachyOS kernel 6.18.0. After recompiling and rebooting, /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/type did not exist. I can confirm that the patch is applied, and the corresponding kernel options are set.

$ zcat /proc/config.gz | grep -i btrfs
CONFIG_BTRFS_FS=m
CONFIG_BTRFS_FS_POSIX_ACL=y
# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
# CONFIG_BTRFS_DEBUG is not set
# CONFIG_BTRFS_ASSERT is not set
CONFIG_BTRFS_ALLOCATOR_HINTS=y
# CONFIG_BTRFS_PER_DEVICE_IO_STATS is not set
CONFIG_BTRFS_READ_POLICIES=y
CONFIG_BTRFS_EXPERIMENTAL=y
$ ls /sys/fs/btrfs/f2dfd4a4-276d-4451-999e-a39457f032b5/devinfo/2
error_stats  fsid  in_fs_metadata  missing  replace_target  scrub_speed_max  writeable

Is there anything I am missing?

[Forza-tng]

I tried this patch with CachyOS kernel 6.18.0. After recompiling and rebooting, /sys/fs/btrfs/<UUID>/devinfo/<DEVID>/type did not exist. I can confirm that the patch is applied, and the corresponding kernel options are set.

$ zcat /proc/config.gz | grep -i btrfs
CONFIG_BTRFS_FS=m
CONFIG_BTRFS_FS_POSIX_ACL=y
# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
# CONFIG_BTRFS_DEBUG is not set
# CONFIG_BTRFS_ASSERT is not set
CONFIG_BTRFS_ALLOCATOR_HINTS=y
# CONFIG_BTRFS_PER_DEVICE_IO_STATS is not set
CONFIG_BTRFS_READ_POLICIES=y
CONFIG_BTRFS_EXPERIMENTAL=y
$ ls /sys/fs/btrfs/f2dfd4a4-276d-4451-999e-a39457f032b5/devinfo/2
error_stats  fsid  in_fs_metadata  missing  replace_target  scrub_speed_max  writeable

Is there anything I am missing?

Thanks for the reporty. I can confirm the same.

[kakra]

Thanks for the report. Will fix...

[kakra]

@CHN-beta @Forza-tng Thanks for reporting and confirming. This was actually a bug I introduced when I made allocator hints configurable via make menuconfig: I used the wrong definition in the C code, the allocator hints actually never compiled and were not active (it thus also slipped through my compile tests which showed two syntax issues now). I never verified if the type fields still existed, I'll keep this in mind for the future. Sorry.

Important: This means, whoever used the 6.18 patches until now, never had allocator hints enabled since. Please use the new patches, then verify that devinfo/*/type exists (it will still have your original type value). If it exists, it means the patch is working now. But you'll need to check if your btrfs moved meta data to the slow devices:

# btrfs filesystem usage -T {BTRFS-MOUNT-PATH}
...
                  Data    Metadata System
Id Path           RAID1   RAID1    RAID1    Unallocated Total     Slack
-- -------------- ------- -------- -------- ----------- --------- -------
 1 /dev/bcache2   2.51TiB        -        -     1.12TiB   3.63TiB 3.50KiB
 2 /dev/bcache0   2.52TiB        -        -     1.12TiB   3.63TiB 3.50KiB
 4 /dev/nvme0n1p2       - 86.00GiB 32.00MiB    41.97GiB 128.00GiB       -
 6 /dev/nvme1n1p2       - 86.00GiB 32.00MiB    41.97GiB 128.00GiB       -
 7 /dev/bcache3   2.52TiB        -        -     1.12TiB   3.64TiB       -
 8 /dev/bcache1   2.50TiB        -        -     1.14TiB   3.64TiB       -
-- -------------- ------- -------- -------- ----------- --------- -------
   Total          5.03TiB 86.00GiB 32.00MiB     4.57TiB  14.79TiB 7.00KiB
   Used           4.92TiB 30.35GiB  1.05MiB

If it lists devices with unexpected meta data, take note of the affected device IDs, then run a meta data balance filtered for device ID (separate each ID by spaces):

for ID in {SLOW_DEV_IDs}; do btrfs balance start -mdevid=$ID --enqueue {BTRFS-MOUNT-PATH}; done

E.g., run for ID in 1 7; do ... if your meta data ended up unwanted on device ID 1 and 7. Balance will then rewrite all meta chunks on device 1 and 7, effectively re-allocating it from the fast dedicated devices, without touching existing meta data chunks on other devices. It should be a fast and safe operation.

Thanks.

[CHN-beta]

I never verified if the type fields still existed, I'll keep this in mind for the future. Sorry.

Please don't worry about it. Everyone makes mistakes sometimes, and this one didn't actually cause any damage. Thank you for your contribution!

[kakra]

Updated the branch to improve the style of some if statements.

[kakra]

Added a Github workflow to automatically compile and build-check the patches.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[redjard]

Hope this isn't inappropriate to ask here, what's the status (or chances) of getting these patches (especially queue and allocator hints) upstreamed into the mainline kernel?

[kakra]

Hope this isn't inappropriate to ask here, what's the status (or chances) of getting these patches (especially queue and allocator hints) upstreamed into the mainline kernel?

I happily answer that:

Chances are near zero. This is not a quality or functional problem. It's more an organisational issue.

• Allocator hints currently confuse free space calculation because the current approach to calculate df does not expect that we may allocate from another device or cannot allocate from some devices. Admins are expected to use btrfs cmdline tooling instead. Currently, upstream is working on a different approach to allocation hints, our implementation will stay compatible until then and cause no conflicts, but at some point in the future you may need to migrate and we drop our patches. I will send proper instructions when the time comes. Don't expect this to happen any time soon - read: not in the next 5-10 years. :-D
• Our implementation of queue depth balancing breaks the rules of separation of concerns in the Linux kernel: file systems should not peek into underlying storage details. Thus, it won't be accepted until I find a better solution. One solution could be to provide an API from the lower levels, or try something similar to what mdraid does. For the type of systems that this patch targets, it's not a real issue. Crossing boundaries of concerns, however, may be an issue if you run filesystems which have to process 100k IOPS or more - but these have very different scaling characteristics and probably don't benefit from this patch anyways.
• Our implementation of sysfs to persisted file system data is currently not accepted in the kernel because it bypasses btrfs transactions (thus, it can become lost under certain situations). I will probably fix this with the next LTS cycle. But even if we fixed this, chances aren't good due to the above points. We will see as time passes.

I hope this helps. Let me know if I should improve the initial FAQ above.

[voidpointertonull]

Allocator hints currently confuse free space calculation because the current approach to calculate df does not expect that we may allocate from another device or cannot allocate from some devices.

I've wondered about this problem for a while, so guess I'll use this "revive" of topic to ask: Is this actually a problem for preferred location too? I was under the impression that in that case there's no lie about free space, because for example devices preferring metadata storage could be also used for data.

Also, if free space calculation problems being made worse is the only blocker, then isn't that feasible to work around by metadata only devices not contributing to total space at all? While that would only cover a subset of what's offered here, I believe that to be the most important part.

I'm also generally confused about free space calculation concerns raising the bar of acceptance so high while free space fragmentation can still lead to the dreaded ENOSPC issue, so it's not like a perfect (or even good) system would be degraded, and the location hints being opt-in also prevent most of the surprise even if something does get worse (which I still doubt with only the preferred flags being in use).

Don't expect this to happen any time soon - read: not in the next 5-10 years. :-D

That's long enough to "solve" the issue for a lot of people by just waiting until technology advances to the point of being capable of hiding bad design decisions, which then leads to the problem no longer being considered worthy of treatment.

Quite like how dirty page cache flushing still sucks, it's just less apparent with faster and generally better storage devices.

[Forza-tng]

Allocator hints currently confuse free space calculation because the current approach to calculate df does not expect that we may allocate from another device or cannot allocate from some devices.

I've wondered about this problem for a while, so guess I'll use this "revive" of topic to ask: Is this actually a problem for preferred location too? I was under the impression that in that case there's no lie about free space, because for example devices preferring metadata storage could be also used for data.

Also, if free space calculation problems being made worse is the only blocker, then isn't that feasible to work around by metadata only devices not contributing to total space at all? While that would only cover a subset of what's offered here, I believe that to be the most important part.

I'm also generally confused about free space calculation concerns raising the bar of acceptance so high while free space fragmentation can still lead to the dreaded ENOSPC issue, so it's not like a perfect (or even good) system would be degraded, and the location hints being opt-in also prevent most of the surprise even if something does get worse (which I still doubt with only the preferred flags being in use).

Don't expect this to happen any time soon - read: not in the next 5-10 years. :-D

That's long enough to "solve" the issue for a lot of people by just waiting until technology advances to the point of being capable of hiding bad design decisions, which then leads to the problem no longer being considered worthy of treatment.

Quite like how dirty page cache flushing still sucks, it's just less apparent with faster and generally better storage devices.

for free space calc, I think we could come up with a simple rule.

• preferred allocations : no change today
• only data: no special treatment
• only metadata: exclude from free space calculation
• no allocation: exclude from free space calculation
  *...

[kakra]

That's long enough to "solve" the issue for a lot of people by just waiting until technology advances to the point of being capable of hiding bad design decisions, which then leads to the problem no longer being considered worthy of treatment.

I think file system are generally a slow moving target... Btrfs took maybe 10 years to be usable without major corruption issues or other annoyances. I took another (overlapping) 10 years to fix most performance issues. We are probably already in the face of adding features.

I can think of some, and I'm working on these ideas:

For example, I'm thinking of a feature to select faster devices for allocating smaller extents. The size classes that have been added to the chunk allocation should be able to provide such an allocation strategy. It will likely also mess with free space calculation, tho. But it will also partially implement hot data placement, and we can maybe get rid of bcache. Meta data placement already solves part of it so I could already exclude meta data from bcache. Let's be honest: bcache is nice but it is far from providing the actually latency and throughput that could be expected. Especially small seeks it doesn't handle very well, exactly what it promises to be good at. True, it's still way faster than seeking purely on spinning disks, but it's also far from getting near native SSD speeds.

Next on the plan would be to provide some maintenance process to migrate small extents to faster devices, and big extents to slower devices.

I'm not sure, tho, how to properly decide or persist "data hotness" without generating a big write overhead penalty.

Given that, most of this is experimental. Finally getting to an implementation which solves hot/cold tier scenarios would be the final upstream candidate. Until then, we shouldn't push half baked features into upstream. The current state of the patches only partially solves a bigger problem, and may not be the optimal solution, so it probably shouldn't go into upstream anyways.

Quite like how dirty page cache flushing still sucks, it's just less apparent with faster and generally better storage devices.

Yeah, but given current pricing, this puts back pressure on improved implementations for slow or mixed devices. I always believed that you should have cold data on slow devices, and hot or small data on fast devices, but in one single file system, without the headache of manually managing that. Bcache mostly solves this but it is a distinct layer from btrfs, it doesn't know about btrfs, or how CoW relocates data and makes orphaned blocks obsolete in the cache, or that mirrored stripes contain identical data, so with btrfs, it can mostly make bad decisions except your data sits there unmodified. So I believe my idea for small/big placement may already make such a big improvement that we would get rid of bcache as an intermediate layer (consuming not particularly few CPU cycles). With this patch series using the queue read balancing, we can at least prefer exactly those devices which already have cached blocks in bcache - resulting in less usage penalty (duplicated data) in bcache itself.

Also, a note on page cache: As far as I understood, the page cache cannot identify the same (shared) extent from different on two different btrfs subvolumes as the same data - so it will be read twice, and will be kept twice in memory. That's one other potential optimization to solve. I wonder if that's why zfs implemented its own RAM cache.

I'm not sure what you mean by dirty page cache flushing: in general, or for btrfs specifically? In "my" other patch series (#45), I incorporated cache and memory management changes (le9). Well, it can behave quite well in many scenarios, even exceptionally well sometimes, but it can also get a lot worse in others. So I understand why such changes haven't gone upstream. IOW, just improving some scenarios can make it a lot worse for others, not only slightly, but a lot of. So it doesn't always seem easy, even if some solutions look obvious. Looking at the results of using the le9 patches, this pretty much proves that page cache isn't so easy to properly change or improve.

And then there's one other elephant it the room: Linux tries to be a kernel for any workload (servers, databases, desktops, gaming, big data) - it can only be tuned up to some level. I think we'd really need specialized versions that go deeper than just tuning some knobs on the scheduler or memory manager.

[kakra]

for free space calc, I think we could come up with a simple rule

Yep, this would probably solve most of the headaches with it. It seems simple and straight forward but nonetheless, it hasn't been implemented yet. So there's probably some deeper cause, why. I think one blocker would be btrfs file systems that used mixed metadata/data. I'd say btrfs should probably not run on very small devices anyways, and that mode of operation should be ripped out of btrfs.

It's one of those "bad ideas" that slipped into the code base without enough thought - similar to what I outlined above why the current ideas of data placement should not yet land in btrfs.

[kakra]

This update changes Btrfs RAID1 read balancing for the queue read policy and introduces a new queue-adaptive read policy.

The queue policy now breaks equal-depth ties by mirror age instead of implicitly preferring the first candidate. On mostly idle mirrors this rotates reads more fairly across available copies.

The new queue-adaptive policy keeps the normal in-flight queue selection as its primary signal, but adds a latency guard. If one mirror currently looks much slower than its peer, it can be temporarily skipped unless the faster peer is already sufficiently busier.

In practice this can help avoid devices with temporary or localized read latency spikes, for example:

• SSDs/NVMes currently busy with internal maintenance such as FTL work or discard/TRIM side effects
• HDD areas or sectors that respond unusually slowly
• weak or failing media where read recovery causes large latency spikes
• mixed pools where one mirror is temporarily much worse than another

The policy does not diagnose disk health directly and does not replace SMART, scrub, or error handling. It only reacts to observed read latency and may route reads away from a slow mirror while it is slow.

Expected effects:

• fewer desktop and gaming stalls caused by latency-sick mirror members
• better tail latency under mixed or degraded device behavior
• potentially better throughput under load when avoiding slow mirrors
• slightly different idle read distribution due to the fairness tie-break

On fully healthy and evenly performing devices, behavior should remain close to the normal queue policy. The age tie-break may distribute idle reads more evenly, which can slightly change benchmark results in either direction depending on the workload.

@Forza-tng If you want to test or compare this update against previous benchmarks, please benchmark it in two steps:

1. Compare the previous version of this patchset against the new version with the queue tie-breaker. This isolates the effect of the new mirror age tie-break in the regular queue policy.
2. Then compare queue against queue-adaptive. This isolates the latency guard from the fairness change.

I could not produce clean benchmark numbers myself because one of my disks started failing, most likely after the recent European heatwave. The disk now reads many sectors much more slowly, sometimes stalling for up to a minute, but eventually still recovers the reads.

I lowered SCT-ERC so very slow sectors are reported as unreadable earlier, allowing Btrfs to recover and rewrite them from another mirror. The disk is still broken and needs to be replaced.

This failure case is what made me revisit the idea of a latency-adaptive queue variant, implement it, and test it under real desktop workloads. In my setup it also avoids latency contention between Btrfs metadata devices and bcache using the same underlying NVMes, resulting in a faster and smoother system.

New diagnostics:

ios %lu wait %llu avg %llu age %llu ignored %llu health avg %llu ago %llu score %llu vetoed %llu overflow %llu

• ios: Total read I/O count.
• wait: Total accumulated read wait time in nanoseconds.
• avg: Cumulative average read latency in nanoseconds.
• age: Fairness counter. Increments while the device is a read candidate but not selected. Resets to 0 when selected.
• ignored: Total count of times this device was a candidate but skipped.
• health avg: Recent windowed average read latency in nanoseconds, used by queue-adaptive.
• ago: Seconds since health avg was last successfully refreshed.
• score: Ratio between cumulative average latency and recent health latency. Values above 100 mean the recent window is faster than the cumulative average; values below 100 mean it is slower.
• vetoed: Number of times queue-adaptive rejected this device because its recent latency looked much worse than its peer.
• overflow: Number of times queue-adaptive still selected this slower-looking device because the healthier peer was already sufficiently busier.

[kakra]

Update: several important fixes since the last review round

I've gone through another review pass on the RAID1 read-policy series (queue, queue-adaptive) and folded in a number of fixes, plus some housekeeping. Everything below has been squashed into the commit that originally introduced the affected code, so the series is still one logical step per commit and bisects cleanly.

Correctness fixes:

• Fixed a NULL-pointer dereference on degraded arrays: queue and queue-adaptive could dereference a missing mirror's block device when a RAID1 array is running with a dropped/missing device. Both policies now skip mirrors without a bdev, matching the existing tolerance in find_live_mirror().
• Fixed a race condition in the read-latency health tracker: concurrent readers hitting the same device could race on updating its windowed latency average. A cmpxchg()-based single-writer-per-refresh-interval scheme now makes this safe, and simplified the update logic to a single pass while at it.
• Corrected a misleading comment on the health_score sysfs field (the direction was backwards - a higher score means the device is currently healthier than its own history, not worse).

Behavioral improvement:

• The queue-adaptive latency guard (how much in-flight lead a "slow" mirror needs before it's still trusted) is now scaled by the actual latency ratio between the slow and healthy candidate instead of a fixed margin of 2. This favors tail latency more on arrays with a large speed gap between mirrors (e.g. HDD mirrored against SSD) while keeping the old minimum-of-2 behavior when the gap is small or latency data isn't available yet.

Housekeeping (no functional change):

• queue's own age-based tie-break (added a couple of commits back to fix first-device bias) is now shared with queue-adaptive via a small helper, so both policies stay consistent.
• Relocated a helper function to sit directly above its only caller, eliminating a forward declaration.
• Added explanatory #endif /* CONFIG_FOO */ comments throughout to make future rebases across kernel versions easier to review.

All four CONFIG_BTRFS_READ_POLICIES x CONFIG_BTRFS_PER_DEVICE_IO_STATS combinations build cleanly with W=1, and each commit in the series builds and bisects on its own.

[Forza-tng]

@Forza-tng If you want to test or compare this update against previous benchmarks, please benchmark it in two steps:

1. Compare the previous version of this patchset against the new version with the queue tie-breaker. This isolates the effect of the new mirror age tie-break in the regular queue policy.
2. Then compare queue against queue-adaptive. This isolates the latency guard from the fairness change.

Hi, I am on vacation and won't be able to do any testing on large arrays before i'm back at work. I shouldbe able to do local testing on a plain HDD RAID1 mirror though.

Thanks for your excellent work! We really ought to push it for inclusion in the kernel, and see if we can fix any remaining complaints they have.

Update: I'm using dm-cache for my HDD arrays, so I am not able to do plain HDD test.

These results are with the v2 queue policy on a 4 HDD RAID1 on dm-cache:

> fio --name=randread --ioengine=io_uring --iodepth=16 --rw=randread --bs=4k --direct=1 --size=4096M --numjobs=4 --runtime=240 --group_reporting
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16
...
fio-3.41
Starting 4 processes
Jobs: 4 (f=4): [r(4)][100.0%][r=943MiB/s][r=241k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=26388: Tue Jul 14 22:07:45 2026
  read: IOPS=227k, BW=885MiB/s (928MB/s)(16.0GiB/18511msec)
    slat (usec): min=5, max=126866, avg= 9.93, stdev=75.96
    clat (nsec): min=541, max=138034k, avg=271287.93, stdev=718228.32
     lat (usec): min=43, max=138041, avg=281.22, stdev=722.65
    clat percentiles (usec):
     |  1.00th=[  119],  5.00th=[  151], 10.00th=[  182], 20.00th=[  217],
     | 30.00th=[  235], 40.00th=[  247], 50.00th=[  258], 60.00th=[  269],
     | 70.00th=[  277], 80.00th=[  289], 90.00th=[  306], 95.00th=[  326],
     | 99.00th=[  502], 99.50th=[ 1037], 99.90th=[ 3884], 99.95th=[ 5604],
     | 99.99th=[21365]
   bw (  KiB/s): min=325128, max=1013968, per=100.00%, avg=907315.32, stdev=31838.94, samples=145
   iops        : min=81282, max=253492, avg=226828.83, stdev=7959.73, samples=145
  lat (nsec)   : 750=0.01%
  lat (usec)   : 50=0.01%, 100=0.07%, 250=43.18%, 500=55.75%, 750=0.22%
  lat (usec)   : 1000=0.27%
  lat (msec)   : 2=0.31%, 4=0.11%, 10=0.07%, 20=0.01%, 50=0.01%
  lat (msec)   : 100=0.01%, 250=0.01%
  cpu          : usr=6.15%, sys=54.31%, ctx=1540889, majf=0, minf=27
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4194304,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
   READ: bw=885MiB/s (928MB/s), 885MiB/s-885MiB/s (928MB/s-928MB/s), io=16.0GiB (17.2GB), run=18511-18511msec

In this scenario, the queue-adaptive didn't add much. But I expect we are on the limit of what SSD cache on the sata bus can achieve.

> fio --name=randread --ioengine=io_uring --iodepth=16 --rw=randread --bs=4k --direct=1 --size=4096M --numjobs=4 --runtime=240 --group_reporting
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16
...
fio-3.41
Starting 4 processes
Jobs: 4 (f=4): [r(4)][100.0%][r=917MiB/s][r=235k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=32194: Tue Jul 14 22:17:19 2026
  read: IOPS=236k, BW=922MiB/s (967MB/s)(16.0GiB/17767msec)
    slat (usec): min=5, max=15735, avg= 9.46, stdev=17.45
    clat (nsec): min=411, max=30152k, avg=260030.43, stdev=254423.09
     lat (usec): min=54, max=30166, avg=269.49, stdev=255.56
    clat percentiles (usec):
     |  1.00th=[  130],  5.00th=[  172], 10.00th=[  194], 20.00th=[  221],
     | 30.00th=[  237], 40.00th=[  247], 50.00th=[  258], 60.00th=[  269],
     | 70.00th=[  277], 80.00th=[  289], 90.00th=[  306], 95.00th=[  322],
     | 99.00th=[  363], 99.50th=[  400], 99.90th=[ 1860], 99.95th=[ 3621],
     | 99.99th=[16712]
   bw (  KiB/s): min=851216, max=985360, per=100.00%, avg=945961.37, stdev=7226.99, samples=140
   iops        : min=212804, max=246340, avg=236490.34, stdev=1806.75, samples=140
  lat (nsec)   : 500=0.01%, 750=0.01%
  lat (usec)   : 10=0.01%, 50=0.01%, 100=0.05%, 250=42.43%, 500=57.21%
  lat (usec)   : 750=0.08%, 1000=0.05%
  lat (msec)   : 2=0.09%, 4=0.05%, 10=0.03%, 20=0.01%, 50=0.01%
  cpu          : usr=6.59%, sys=56.52%, ctx=1497838, majf=0, minf=34
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4194304,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
   READ: bw=922MiB/s (967MB/s), 922MiB/s-922MiB/s (967MB/s-967MB/s), io=16.0GiB (17.2GB), run=17767-17767msec

[kakra]

Thanks for your excellent work!

I tested this on my relatively weak an old work PC doing a heavy background emerge world upgrade (Gentoo). I had almost no full desktop stalls, just sometimes single programs did still stall for a few seconds. But the difference is huge: no multiple stalls in a row, no longer did the complete desktop stall. Analyzing the stats, this is most likely a result from the system intelligently avoiding busy meta data devices in btrfs (because I share bcache and btrfs meta data on the same devices which seemingly can cause multi second latency stalls when the SSD/NVMe starts internal house keeping like FTL or discard). Now, such busy devices are detected early and avoided for reading, by reducing their IO request share fairly.

With the v2 fixes to the queue-adaptive policy, my broken/latency sick HDD is also avoided most of the time without completely starving it. Bandwidth while loading heavy games improved from below 200 MB/s to over 500 MB/s for games which properly can use multiple parallel IO requests (only few do, Star Wars Outlaws is one such example that can make use of multiple disks in parallel).

[kakra]

Thanks for adding the dm-cache benchmarks. Looking more closely at the latency distribution, I think queue-adaptive actually added quite a lot here, even though the throughput improvement is relatively modest.

Compared with queue, queue-adaptive improved:

• bandwidth from 885 to 922 MiB/s: about +4%
• average completion latency from 271 to 260 us: about -4%
• p99 from 502 to 363 us: about -28%
• p99.5 from 1037 to 400 us: about -61%
• p99.9 from 3884 to 1860 us: about -52%
• maximum completion latency from 138 to 30 ms: about -78%
• completion-latency standard deviation from 718 to 254 us: about -65%

The median and the p80/p90 latencies stayed essentially unchanged, while the upper tail became much shorter and less variable. That is very close to the intended purpose of queue-adaptive: it primarily tries to reduce latency spikes rather than maximize aggregate bandwidth. The slightly worse low percentiles are an acceptable tradeoff in that context.

This may be related to the effect I observed on my own system, although the storage layout is different. In my case, bcache activity competes with Btrfs metadata reads on the same SSDs, causing transient multi-second latency spikes. With dm-cache, cache misses, promotion/writeback, cache metadata activity, discard, SSD internal housekeeping, or contention between cache mappings can similarly make one mirror path temporarily much slower than the others. The regular queue policy only sees the current queue depth, whereas queue-adaptive can detect that the path is currently latency-sick and reduce its share of new reads.

There are some caveats: these runs completed after about 18 seconds because the specified data size was exhausted, despite --runtime=240, and the adaptive run followed the regular queue run, so cache warming may account for part of the difference. However, cache warming alone would more likely shift a larger part of the distribution. Here, p50 through p90 barely changed while p99 and above improved strongly, which is at least consistent with the latency guard cutting off transient outliers.

If you repeat the test later, alternating the order (queue, queue-adaptive, queue-adaptive, queue) and capturing devinfo/*/read_stats before and after each run would be especially useful. In particular, increases in vetoed with comparatively few overflow events would help connect the improved fio tail latencies directly to the adaptive policy.

[kakra]

We really ought to push it for inclusion in the kernel, and see if we can fix any remaining complaints they have.

That would be nice but the patches won't go in as-is due to layer violations (direct access to bdev), performance considerations for high core count systems (IO stats calculation across all CPU cores), administration concerns (semantic change for df calculation) and follow-up work for allocation hints that is being worked on (our implementation clearly has limitations). But I think we could probably draw some conclusions and insights from this which could feed back into new ideas to improve the existing kernel code. For now, we'd have to at least wait for the read policy infrastructure to settle and officially get out of "btrfs experimental".

If the btrfs devs would consider an official internal btrfs API to query bdev IO stats in an efficient way, our chances would become much better. Currently, I'm importing unrelated headers from a different storage layer, which we should probably not do.

[Forza-tng]

We really ought to push it for inclusion in the kernel, and see if we can fix any remaining complaints they have.

That would be nice but the patches won't go in as-is due to layer violations (direct access to bdev), performance considerations for high core count systems (IO stats calculation across all CPU cores), administration concerns (semantic change for df calculation) and follow-up work for allocation hints that is being worked on (our implementation clearly has limitations). But I think we could probably draw some conclusions and insights from this which could feed back into new ideas to improve the existing kernel code. For now, we'd have to at least wait for the read policy infrastructure to settle and officially get out of "btrfs experimental".

If the btrfs devs would consider an official internal btrfs API to query bdev IO stats in an efficient way, our chances would become much better. Currently, I'm importing unrelated headers from a different storage layer, which we should probably not do.

That's what I am thinking too; if we can define the gaps that prevent adoption, we can start working on finding solutions.

As you say, a generic way to get queue and latency information/count btrfs requests might help. We also have the df issue, as well as dev replace and possibly remove, add and corner cases to solve. The high core count overhead is perhaps something that can be optimised at a later time? It is after all an opt-in option.

If we approach the mailing list constructively with a RFC/POC, perhaps we can gain some support?

[MarkRose]

Did a review of the series with Claude and found one crash bug plus one minor robustness note.

Bug: NULL-deref reading read_stats on a bdev-less (missing) device

btrfs_devinfo_read_stats_show() in fs/btrfs/sysfs.c reads block-layer stats without a NULL check on device->bdev:

u64 read_wait = part_stat_read(device->bdev, nsecs[READ]);
unsigned long read_ios = part_stat_read(device->bdev, ios[READ]);

part_stat_read() dereferences device->bdev->bd_stats, so this oopses when bdev == NULL. The path is reachable: btrfs_sysfs_add_device() gates only the disk symlink on device->bdev, while the devid_kobj and its attribute group (including read_stats) are created unconditionally. So a missing device on a degraded-mounted array still exposes .../devinfo/<devid>/read_stats, and a plain cat of it dereferences NULL.

This predates the queue-adaptive work (it came in with the original per-device read_stats attribute), but this series extends the same function with additional reads, and "add read latency health stats" (patch 15) specifically added a if (!device->bdev) continue; guard to the sibling btrfs_update_read_health() for exactly this hazard — the sysfs show path right next to it stays unguarded, which is inconsistent.

Suggested fix — early-return before touching part_stat_read(); the age/ignored/vetoed/overflow atomics are safe to read without a bdev:

if (!device->bdev)
        return sysfs_emit(buf,
                "ios 0 wait 0 avg 0 age 0 ignored 0 health avg 0 ago 0 score 0 vetoed 0 overflow 0\n");

Minor: torn reads of the health fields on 32-bit

health_avg_ns, health_check_ios, and health_check_wait are plain u64 accessed with READ_ONCE/WRITE_ONCE. On 32-bit architectures those don't guarantee a non-torn 64-bit access, so a reader can observe a half-updated value. It's advisory data only (worst case a single bad veto or a bogus read_stats line, no crash), but it's inconsistent with last_io_age, which uses atomic64_t precisely for this. Not urgent — just noting it.

The rest of the integration looked good: enum ↔ btrfs_read_policy_name[] ordering matches, sysfs_match_string() avoids any queue/queue-adaptive prefix collision, the find_live_mirror() switch handles the new policy (default still resets to pid), the avg[BTRFS_RAID1_MAX_MIRRORS] array is correctly bounded, and the patch 17 best_healthy_inflight - best_inflight subtraction can't underflow (min-over-all ≤ min-over-subset). The selection path itself (part_in_flight, btrfs_read_earliest, btrfs_read_queue_adaptive) already guards !device->bdev, so the sysfs show is the only spot that misses it.

[kakra]

Bug: NULL-deref reading read_stats on a bdev-less (missing) device

I'm pretty sure this has been fixed with the latest version of the patch set. Which commit sha did you test? I've run two AIs over the code for review (Claude Sonnet with review skills which partially used Opus, and Codex Sol), and they found multiple bdev issues, partially old ones existing since the 6.12 version. I'll recheck, plus some more improvements which I already planned.

Minor: torn reads of the health fields on 32-bit

Yes, this has been mentioned in my review, too. But I deliberately chose not to target 32 bit systems given that kernel upstream is working on removing support for building in Intel 32 bit targets. This issue will be deferred until the next LTS rebase, and then reconsidered.

[kakra]

Bug: NULL-deref reading read_stats on a bdev-less (missing) device

I'm pretty sure this has been fixed with the latest version of the patch set. Which commit sha did you test? I've run two AIs over the code for review (Claude Sonnet with review skills which partially used Opus, and Codex Sol), and they found multiple bdev issues, partially old ones existing since the 6.12 version. I'll recheck, plus some more improvements which I already planned.

Okay, I verified it: It's an pre-existing bug in the patch set and thus had not been reviewed by AI (I asked to explicitly only review the changes).

Minor: torn reads of the health fields on 32-bit

Yes, this has been mentioned in my review, too. But I deliberately chose not to target 32 bit systems given that kernel upstream is working on removing support for building in Intel 32 bit targets. This issue will be deferred until the next LTS rebase, and then reconsidered.

This has a simple fix which I'll commit.

Thanks for reporting.

[MarkRose]

Okay, I verified it: It's an pre-existing bug in the patch set and thus had not been reviewed by AI (I asked to explicitly only review the changes).

I had asked Opus to review the patch set in its entirety. Thank you for maintaining these patches. I am using the allocator hints on multiple machines.

[kakra]

@MarkRose Thanks for the careful review, especially catching the read_stats crash - that's a real one, and a nasty way to hit it: any missing mirror on a degraded array keeps its devid_kobj (there's already a missing attribute for that state), so cat .../read_stats on it hits part_stat_read() on a NULL bdev immediately.

Both are fixed now, pushed as two separate fixup commits on top of the branch (not yet squashed into the original patches - that'll happen in a follow-up force-push once the rebase is done):

• a9ebcbc – guards both part_stat_read() calls behind device->bdev, falling back to 0 for read_ios/read_wait when it's missing. The rest of the fields (age, ignored, health avg/ago, vetoed, overflow) never depended on bdev, so they keep reporting whatever history the device still has instead of being zeroed out too.
• cfbba57 – switched health_avg_ns, health_check_ios, health_check_wait to atomic64_t, matching the other per-device counters in the struct. You're right that it's advisory data only, but there's no reason to leave the torn-read window open on 32-bit when the fix is this cheap and the pattern is already established next to it.

Will land in the affected patches' v2/v3 changelog notes once I fold these in.