This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
If the dm-integrity device has the SB_FLAG_DIRTY_BITMAP flag set and the
user activates the device in the 'R' mode, a crash in dm_integrity_resume
happens because the function attempts to read the journal containing the
bitmap.
This patch makes dm-integrity skip any writes to the device in
dm_integrity_resume if the device is activated in the 'R' mode.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: 468dfca38b ("dm integrity: add a bitmap mode")
Cc: stable@vger.kernel.org
The demotion counter is incremented per incoming write bio before the
invalidation begins, causing the demotion count to exceed the actual
number of cached blocks when multiple bios target the same cached
block. Additionally, the counter is incremented unconditionally
regardless of invalidation failure.
Reproduce steps:
1. Create a cache device consisting of 512 cache entries
modprobe brd rd_size=262144
dmsetup create cmeta --table "0 8192 linear /dev/ram0 0"
dmsetup create cdata --table "0 65536 linear /dev/ram0 8192"
dmsetup create corig --table "0 65536 linear /dev/ram0 262144"
dd if=/dev/zero of=/dev/mapper/cmeta bs=4k count=1 oflag=direct
dmsetup create cache --table "0 65536 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 writethrough smq 0"
2. Populate the cache, and record the number of cached blocks
fio --name=populate --filename=/dev/mapper/cache --rw=randwrite --bs=4k \
--direct=1 --ioengine=libaio --iodepth=32 --io_size=2048m
nr_cached=$(dmsetup status cache | awk '{split($7, a, "/"); print a[1]}')
3. Reload the cache into passthrough mode
dmsetup suspend cache
dmsetup reload cache --table "0 65536 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 passthrough smq 0"
dmsetup resume cache
4. Write to the passthrough cache with multiple jobs to trigger
multiple bios hitting the same cached block.
fio --filename=/dev/mapper/cache --name=test --rw=write --bs=4k \
--direct=1 --ioengine=libaio --iodepth=32 --numjobs=4
5. Check if demoted matches cached block count. These numbers should
match but may differ due to overcounting per bio.
nr_demoted=$(dmsetup status cache | awk '{print $12}')
echo "$nr_cached, $nr_demoted"
Fix by moving the demotion counter increment into invalidate_complete(),
gated on the success flag.
Reported-by: Ben Marzinski <bmarzins@redhat.com>
Fixes: b29d4986d0 ("dm cache: significant rework to leverage dm-bio-prison-v2")
Cc: stable@vger.kernel.org
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Reviewed-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
When integrity_metadata handles a discard, it fills a buffer with
DISCARD_FILLER and writes it over the tags, max_blocks blocks at a
time. If the kmalloc fails, the buffer is the on-stack array
checksums_onstack and max_size is set to HASH_MAX_DIGESTSIZE. So if the
tag size is larger than HASH_MAX_DIGESTSIZE, max_blocks is zero, bi_size
is never decremented and the loop never terminates.
Fix this by using sizeof(checksums_onstack) as max_size. The array has
MAX_TAG_SIZE bytes since commit b93b6643e9 ("dm integrity: fix a
crash with unusually large tag size"), so max_blocks is at least 1.
Fixes: 84597a44a9 ("dm integrity: add optional discard support")
Cc: stable@vger.kernel.org
Reviewed-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Signed-off-by: Ben Cressey <ben@cressey.dev>
Assisted-by: Claude:unspecified
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Since commit 68c5c42567 ("dm-integrity: replace forgeable discard
filler with a keyed sector marker"), integrity_metadata computes a
checksum for every discarded block into the "checksums" buffer.
integrity_sector_checksum always writes the whole digest. So if the tag
size is smaller than the digest size, the checksum of the last block
that fits into the buffer is written past the end of it. For example,
with hmac(sha256) and tag size 16, a 4MiB discard writes 16 bytes past
the kmalloc'ed page.
Fix this by subtracting extra_space from the buffer size when computing
max_blocks, like we do for writes.
Fixes: 68c5c42567 ("dm-integrity: replace forgeable discard filler with a keyed sector marker")
Reviewed-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Signed-off-by: Ben Cressey <ben@cressey.dev>
Assisted-by: Claude:unspecified
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm-integrity direct, bitmap and inline internal-hash modes compute
integrity tags from the pages carried by the write bio. The lower data
write also uses those pages, so the tag and the data write depend on the
same memory contents staying unchanged while writeback is in flight.
Without stable writes, a buffered writer can modify a writeback folio
after dm-integrity has submitted the data bio and before the lower device
has consumed the data. After a crash, this can leave data from the later
contents with a tag calculated from the earlier contents, causing
permanent checksum failures on read.
Set BLK_FEAT_STABLE_WRITES for internal-hash D, B and I modes so
filesystems wait for writeback folios to become stable before modifying
them again. Journal mode is left unchanged because it copies data into
the journal before computing and persisting the tag.
Tested using dm-delay over a virtio-blk test disk, dm-integrity
internal_hash:crc32c and no-journal ext4. The D and B reproducers both
failed with checksum errors before this change and completed with
READ_RC=0 and zero mismatches after it.
Fixes: 7eada909bf ("dm: add integrity target")
Cc: stable@vger.kernel.org
Reported-by: Sun Yangkai <sunyangkai@fygo.io>
Link: https://github.com/chencheng-fnnas/reproducer/blob/main/dm-integrity-writeback-race.py
Signed-off-by: Chen Cheng <chencheng@fnnas.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm cache used a rw_semaphore for background_work_lock. Write locks on
rw_semaphores have strict owner semantics, but there was no guarantee
that the process that locked background_work_lock was the same process
that unlocked it. This can be easily seen using a kernel compiled with
CONFIG_DEBUG_RWSEMS. Given a dm cache device <cache>, run: 'dmsetup
suspend <cache> && dmsetup resume <cache>'. This will trigger a kernel
warning:
DEBUG_RWSEMS_WARN_ON((rwsem_owner(sem) != current) && !rwsem_test_oflags(sem, RWSEM_NONSPINNABLE))
triggered by cache_resume(). To fix this, switch from a rw_semaphore to
a spinlock and a wait queue. dm cache already has a wait queue and
associated counter, migration_wait and nr_allocated_migrations, that was
getting woken up when background work was getting completed, but wasn't
actually used by anything. This is replaced by the background_work queue
and counter.
Fixes: b29d4986d0 ("dm cache: significant rework to leverage dm-bio-prison-v2")
Cc: stable@vger.kernel.org
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Reviewed-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
crypt_dec_pending reads io->error before calling atomic_dec_and_test.
Another context, for example crypt_endio called from an interrupt, may
set io->error and drop its reference between the read and the
decrement. crypt_dec_pending then drops the last reference and completes
the bio with the stale status - so a read that failed and was never
decrypted, or a write that failed, is reported as successful.
The read was placed before the decrement by commit b35f8caa08 ("dm
crypt: wait for endio to complete before destruction"), because that
commit freed dm_crypt_io before calling bio_endio. This is no longer the
case, dm_crypt_io lives in the per-bio data now.
Read io->error after atomic_dec_and_test instead. atomic_dec_and_test is
fully ordered, so no additional barrier is needed.
Fixes: b35f8caa08 ("dm crypt: wait for endio to complete before destruction")
Cc: stable@vger.kernel.org
Reviewed-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Signed-off-by: Ben Cressey <ben@cressey.dev>
Assisted-by: Claude:unspecified
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
Pull device mapper updates from Mikulas Patocka:
- minor cleanups found by Claude Opus 4.6
- small cleanups in dm core, dm-cache, dm-switch, dm-inlinecrypt,
dm-vdo
- improve validation of metadata in dm-pcache
- fix resume-vs-remove ioctl race condition
- fix race condition when issuing table load ioctls concurrently
- fix dm-raid1 and dm-io, so that they work with unaligned bio vectors
- dm-integrity: use keyed markers as discard fillers
- improve metadata validation in dm-array
- fix dm-stats crash on memory allocation failure
- fix dm-dust, so that it works if it is not the first target in a
table
- dm-era: fix superblock refcount leak on snapshot failure
* tag 'for-7.3/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (46 commits)
dm-era: fix shadowed superblock leak on take-snap failure
dm dust: make badblock messages target-relative
dm-stats: fix a crash if allocation of per-cpu data fails
dm array: reject an array block whose value size is not the caller's
dm array: validate array block headers on read
dm-integrity: replace forgeable discard filler with a keyed sector marker
dm vdo indexer: embed geometry in parent structures
dm vdo indexer: simplify sub-index parameter calculations
dm-pcache: remove unused 'cache' parameter from cache_key_gc()
docs: device-mapper: dm-inlinecrypt: fix 'bellow' spelling
dm-pcache: remove unused miss_read_end_work_fn declaration
dm-io: report non-retryable errors separatedly
dm-io: clone the source bio instead of copying its biovec
dm: fix race when loading and unloading a table
dm: fix resume-vs-remove race
dm-pcache: remove unused 'allocated' variable in cache_data_alloc()
dm-pcache: replace tabs with spaces in comments to fix ASCII diagram alignment
dm-pcache: fix use-after-free and invalid seg operations in kset_replay()
dm-pcache: fix implicit u8 truncation of gc_percent in message handler
dm raid1: reserve space for NUL-terminator in build_constructor_string()
...
Pull x86 cpuid updates from Borislav Petkov:
- Get rid of static_cpu_has() - one less API to care about testing CPU
features
- Unify the handling of CPU core types (performance, efficient, etc) by
mapping the vendor-specific types to Linux ones
- Continuation of the work of Ahmed Darwish to centralize CPUID leaf
representation
* tag 'x86_cpu_for_v7.3_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/CPU: Rename struct cpuid_read_output to struct cpuid_output
x86/cpu/scattered: Sort it properly
x86/cpu: Use parsed CPUID(0x1)
x86/lib: Add CPUID(0x1) family and model calculation
x86/cpu: Use parsed CPUID(0x0)
x86/cpu/transmeta: Rescan CPUID(0x1) after modifying capabilities
x86/topology: Add TOPO_CPU_TYPE_LOW_POWER
x86/topology: Name the AMD core-type values
x86/topo: Map vendor CPU types to generic Linux such types
x86/bugs: Don't use cpu-type matching in cpu_vuln_blacklist
x86/cpu: Hide and rename static_cpu_has()
raid1_takeover() sets conf->array_frozen = 1 on the newly-allocated
r1conf and nothing ever clears it, so every I/O to the array stalls
permanently once _wait_barrier() sees it stuck at 1.
This used to be harmless: level_store() called mddev_resume() right
after pers->run(), which called raid1_quiesce(mddev, 0) and cleared
array_frozen back to 0 regardless of what raid1_takeover() set. Commit
b39f35ebe8 ("md: don't quiesce in mddev_suspend()") removed that
quiesce(mddev, 0) call, so the pre-set now sticks.
setup_conf() already zero-initializes the new r1conf via kzalloc, so
just don't set array_frozen here.
Same class of bug as commit 892da88d1c ("md/raid10: fix a
'conf->barrier' leakage in raid10_takeover()"), also triggered by
b39f35ebe8.
Fixes: b39f35ebe8 ("md: don't quiesce in mddev_suspend()")
Link: https://issues.redhat.com/browse/RHEL-191802
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Link: https://patch.msgid.link/20260803180240.1177104-1-bjohnsto@redhat.com
Signed-off-by: Yu Kuai <yukuai@fygo.io>
blk_stack_limits() uses min_not_zero() when stacking discard limits.
Thus an array containing devices with different discard capabilities can
expose discard support as long as at least one member has a non-zero
discard limit.
raid0 and raid10 use md_submit_discard_bio() to submit a discard bio to
each member covered by the request. The helper currently also submits
bios to members whose max_discard_sectors is zero. The block layer
completes these bios with BLK_STS_NOTSUPP, and bio chaining propagates
that status to the original discard request.
Discard is optional, so skip members which do not support it. Members that
do support discard continue to receive their portion of the request.
Signed-off-by: Wale Zhang <wale.zhang.ftd@gmail.com>
Link: https://patch.msgid.link/20260731074729.1885314-1-wale.zhang.ftd@gmail.com
Signed-off-by: Yu Kuai <yukuai@fygo.io>
When sync_request() reports a skipped region (*skipped == 1),
md_do_sync()'s main loop advances the cursor and takes an early
continue:
j += sectors;
...
if (last_check + window > io_sectors || j == max_sectors)
continue;
If the personality returns a small span per call (raid10 recovery
returns only 128 sectors), syncing a large, mostly clean array iterates
this branch an enormous number of times without ever yielding the CPU.
On a non-preemptive kernel the resync thread then trips the soft-lockup
watchdog:
watchdog: BUG: soft lockup - CPU#149 stuck for 313s! [mdX_resync]
md_bitmap_start_sync+0x6f/0xe0
raid10_sync_request+0x2c9/0x1530 [raid10]
md_do_sync+0x810/0x1030
md_thread+0xa7/0x150
Add a cond_resched(). This does not reduce the wasted iterations; the
excessive iteration count is a raid10 problem addressed separately.
Signed-off-by: Yunye Zhao <yunye.zhao@linux.alibaba.com>
Link: https://patch.msgid.link/20260723135535.101995-3-yunye.zhao@linux.alibaba.com
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Commit fe6a19d40c ("md/md-bitmap: merge md_bitmap_start_sync() into
bitmap_operations") converted still_degraded from int to bool, but
inverted the assignment in the loop that checks whether the array will
still be degraded after the current device is recovered:
"still_degraded = 1" became "still_degraded = false".
As a result, recovering a device while another mirror is still missing
calls md_bitmap_start_sync() with degraded == false, which clears bitmap
bits that the still-missing device needs. When that device is re-added,
its bitmap-based recovery finds the bits already cleared and skips every
region written while the array was degraded, so it is marked In_sync
while holding stale data: silent corruption.
Reproducer (raid10 near=2, 4 disks, internal bitmap):
- fail and remove one disk of each mirror pair
- write to the degraded array
- re-add both disks and let recovery finish
- "check" reports mismatch_cnt=262272 after 256 MiB of degraded
writes and file contents differ; the second disk's "recovery"
completes in milliseconds because everything is skipped
The same conversion in raid1 got it right (still_degraded = true).
Restore the correct value.
Fixes: fe6a19d40c ("md/md-bitmap: merge md_bitmap_start_sync() into bitmap_operations")
Cc: stable@vger.kernel.org
Signed-off-by: Yunye Zhao <yunye.zhao@linux.alibaba.com>
Reviewed-by: Mykola Marzhan <mykola@meshstor.io>
Reviewed-by: Paul Menzel <pmenzel@molgen.mpg.de>
Reviewed-by: Yu Kuai <yukuai@fygo.io>
Link: https://patch.msgid.link/20260723135535.101995-2-yunye.zhao@linux.alibaba.com
Signed-off-by: Yu Kuai <yukuai@fygo.io>
RAID5 maps array sectors through different geometries before and after the
reshape position. During llbitmap reshape, md core cannot account one bio
against both geometries as a single bitmap range, because the old and new
bitmap mappings can cover different chunks.
Split bios that cross reshape_position before md_account_bio(), so the
bitmap only sees ranges that belong to one side of the reshape boundary.
mddev_bio_split_at_reshape_offset() uses bio_submit_split_bioset(), which
submits the remainder immediately and returns the front split bio. If that
front bio later has to wait for reshape, md_handle_request() must not retry
the original bio pointer, because after the split that pointer is the
already-submitted remainder. Track whether the split happened, clear the
temporary BLK_STS_RESOURCE status after the internal clone completion, and
resubmit the front bio directly after the reshape wait. Keep the old
return-false retry path for unsplit bios, where md_handle_request() still
owns the same bio.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-30-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap reshape keeps one live bitmap and cannot safely make an
existing bitmap bit cover a smaller data range.
The llbitmap chunksize itself will not shrink when mddev->chunk_sectors
stays the same or grows. However, shrinking mddev->chunk_sectors shrinks
sectors_per_chunk used by raid5_bitmap_sector_map(). That can shrink the
effective data range covered by each bit across the old and new RAID5
geometry. Reject that reshape while llbitmap is active.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-28-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap reshape keeps one live bitmap and cannot safely make an
existing bitmap bit cover a smaller data range.
The llbitmap chunksize itself will not shrink when mddev->chunk_sectors
stays the same or grows. However, shrinking mddev->chunk_sectors can
shrink the effective data range covered by each bit for the RAID10
reshape geometry. Reject that reshape while llbitmap is active.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-24-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap_state_machine() can be called with an end bit beyond
llbitmap->chunks. In particular, llbitmap_cond_end_sync() passes
sector >> chunkshift, and sector can reach the tracked boundary
exactly.
Clamp the state-machine range to llbitmap->chunks so it cannot walk
past the tracked bitmap.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-23-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Merge checkpointed old llbitmap state forward as reshape_position advances
and record the checkpoint remap through reshape_mark().
Normal write accounting can run while the reshape thread checkpoints a new
reshape position. llbitmap_reshape_mark() reads old state bytes, merges them
into destination bits, and writes the result back. If llbitmap_start_write()
or llbitmap_start_discard() updates the same state bytes at the same time,
the two read/modify/write paths can overwrite each other and lose the state
from one side.
Serialize only this state-byte race with a rwlock. Normal I/O takes the read
side around llbitmap_state_machine(), after page active references are raised,
so concurrent normal I/O updates still run in parallel. Reshape checkpointing
takes the write side only while merging the checkpointed range, avoiding page
suspension and avoiding a sleeping mutex in the I/O accounting path.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-22-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Teach llbitmap to choose old versus new geometry during reshape and to
encode exact bitmap ranges for the active geometry. This is the mapping
groundwork for checkpoint remapping.
Range preparation now distinguishes writes from discards. Normal writes
must cover every touched bitmap chunk, while discards may only mark fully
covered chunks unwritten. Without this distinction, a discard that starts
or ends inside a chunk can make live data look unwritten after the range
has been mapped and floored.
Reproduce that with a RAID1 llbitmap using 128-sector chunks. A discard
starting halfway into chunk 8 with a 128-sector length changed clean bits
from 16352 to 16350 and unwritten bits from 0 to 2, even though no chunk
was fully discarded. With discard-specific range encoding, both counts
stay unchanged for the same test.
Range preparation also clamps the pre-map range in the same coordinate
space as the incoming IO. RAID5 receives array-sector offsets but tracks
llbitmap sync size in component sectors, so steady-state RAID5 must use
bitmap_array_sectors() before mapping and keep the existing sync-size
clamp after mapping.
Reproduce that with a 4-disk RAID5 llbitmap created --assume-clean. A
write below dev_sectors changed dirty bits from 0 to 512, but a write at
seek=2094080 left the count at 512. With the array-sector pre-map limit,
writing at seek=component_size + 65536 increased dirty bits from 512 to
1024.
Link: https://lore.kernel.org/all/20260726185916.2223460-1-mykola@meshstor.io/
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-20-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Commit the staged llbitmap geometry when reshape finishes.
When assembling a stopped reshape, md_run() creates the bitmap before
publishing mddev->pers. llbitmap_read_sb() can therefore only initialize
the reshape fields from the old on-disk sync size. Refresh the staged
reshape geometry again from llbitmap_load(), after mddev->pers is
available, and expand the in-memory page controls before replaying bitmap
state.
Reproduce on the old kernel by creating a RAID10 llbitmap with four
active disks and two spares, growing it to six disks, then stopping and
assembling while reshape is still running. The llbitmap chunk count was
32704 before grow, 49056 during reshape, then rolled back to 32704 after
reassemble.
The fixed kernel kept the target geometry across the same stop/reassemble
flow: 65440 chunks before grow, 98160 during reshape, and 98160 after
reassemble.
Link: https://lore.kernel.org/all/20260726185916.2223460-1-mykola@meshstor.io/
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-18-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Allocate one llbitmap page-control object at a time and free each
object through the same model.
Let llbitmap_read_page() return a zeroed page without reading disk when
the page index is beyond the current bitmap size, so page-control
allocation no longer needs a separate read_existing flag.
This keeps the llbitmap page-control lifetime self-consistent and
prepares the page-cache code for later in-place growth.
Reviewed-by: Su Yue <glass.su@suse.com>
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-15-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Add bitmap mapping and reshape hooks needed by llbitmap reshape
support without teaching md core to account a single bio against
multiple bitmap ranges.
This also adds the old/new bitmap geometry helpers used by
personalities to describe reshape mapping to llbitmap.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-13-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Add mddev_bio_split_at_reshape_offset() so personalities can share
reshape-offset bio splitting instead of open-coding the same boundary
handling in multiple places.
The helper first applies the optional max_sectors limit. If reshape is
running and the bio crosses reshape_position, it further limits the front
bio to the current reshape boundary so callers can account and submit one
side of the reshape at a time.
Snapshot reshape_position with READ_ONCE(). RAID5 and RAID10 update this
field as reshape progresses, while the I/O path only needs one consistent
decision point for the current bio. Using an explicit single load avoids a
plain lockless access and prevents the compiler from refetching a different
boundary while deciding whether and where to split.
When a split is needed, bio_submit_split_bioset() submits the remainder and
returns the front bio. Callers must therefore continue processing the
returned bio, not the original pointer.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-12-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
mkfs.ext4 can submit zero-sector flush/FUA bios. These bios are WRITE
bios for md_write_start() purposes, but they do not cover any data sector
and must not dirty bitmap bits.
md bitmap accounting currently passes such bios to bitmap start_write().
For llbitmap this reaches llbitmap_start_write() with sectors == 0,
which underflows the end chunk calculation.
Personality bitmap mapping can also turn a non-empty bio into an empty
bitmap range when the requested sectors are outside the active bitmap
geometry. Treat both cases as not started, so the completion path will not
call end_write() for an empty range.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-11-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap_destroy() deletes pending_timer before flushing
md_llbitmap_io_wq. However, daemon_work can still be queued or running
after the timer has been deleted, and the daemon path can arm
pending_timer again when it finds dirty chunks that are not ready to
flush yet.
If that happens during teardown, pending_timer can remain armed after
llbitmap is freed and later dereference freed memory.
Add a BITMAP_SHUTDOWN bit to llbitmap->flags, set it before deleting
the timer, and make the timer and daemon paths stop queueing or rearming
work once teardown starts. Cancel daemon_work before flushing the shared
workqueue so no already queued daemon instance can race with the free.
Use timer_shutdown_sync() so a daemon instance that passed the shutdown
check before teardown cannot rearm the timer afterward.
BITMAP_SHUTDOWN is a runtime-only state. Mask it out when reading and
updating the llbitmap superblock so the shutdown state is never loaded
from disk or persisted to disk.
Fixes: 5ab829f197 ("md/md-llbitmap: introduce new lockless bitmap")
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-10-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap_create() publishes mddev->bitmap before reading the bitmap
superblock. This is needed because llbitmap_read_sb() can initialize a
new bitmap and flush it through helpers that use mddev->bitmap.
If llbitmap_read_sb() fails, the old cleanup dropped bitmap_info.mutex
and freed llbitmap before clearing mddev->bitmap. Readers such as
/proc/mdstat rely on bitmap_info.mutex to keep the bitmap pointer stable
while collecting bitmap stats, so they could observe the stale pointer
after the failed create path released the mutex.
Clear mddev->bitmap while still holding bitmap_info.mutex, then free the
failed llbitmap after dropping the mutex. This makes mutex-protected
readers see either a live bitmap or no bitmap.
Fixes: 5ab829f197 ("md/md-llbitmap: introduce new lockless bitmap")
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-9-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
md_clone_bio() always allocates the clone from mddev->io_clone_set, even
when queue I/O stats are disabled. In that case it does not call
bio_start_io_acct(), but it also left md_io_clone->start_time untouched.
The clone private data comes from a mempool and can contain data from a
previous user. md_end_clone_io() checks start_time to decide whether it
needs to call bio_end_io_acct(), so a stale non-zero value can make the
completion path end accounting that was never started for this bio.
Set start_time to 0 in the no-stats branch. This keeps the end path tied
to whether bio_start_io_acct() actually ran.
Fixes: c687297b88 ("md: also clone new io if io accounting is disabled")
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-8-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
__md_stop() destroyed the bitmap before calling mddev_detach(). That made
mddev_detach() skip bitmap_ops->wait_behind_writes(), because the bitmap
was already disconnected from mddev.
This was still safe for the legacy bitmap because bitmap_destroy() waits
for behind writes itself. llbitmap keeps that wait in its
->wait_behind_writes() operation instead, while ->destroy() tears down the
llbitmap storage. With the old ordering, RAID1 behind-write completions
could still run after llbitmap storage had been freed.
Call mddev_detach() before md_bitmap_destroy() so the common detach path
can wait for behind writes while the bitmap is still alive. Only destroy
the bitmap after those users are gone.
Fixes: 5ab829f197 ("md/md-llbitmap: introduce new lockless bitmap")
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-7-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
raid5_bitmap_sector_map() aligns the array range to full RAID5 stripe
widths before converting it to component sectors. That width is
chunk_sectors multiplied by the number of data disks, and it is not
always a power of two.
Reproduce with a 4-disk RAID5, 1024-sector chunks, and three data disks.
The full-stripe width is 3072 sectors. For a one-sector write at array
sector 3072, correct rounding gives array range [3072, 6144), which maps
to component range [1024, 2048). The old round_down()/round_up() logic
instead gives [1024, 4096), which maps to [0, 1024).
Use sector_div() based arithmetic so the rounded range is aligned to the
actual RAID5 stripe width.
The deterministic mapper test now reports the fixed component range as
[1024, 2048), while the old mask-based range was [0, 1024).
Fixes: 9c89f60447 ("md/raid5: implement pers->bitmap_sector()")
Reported-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://lore.kernel.org/all/20260726185916.2223460-1-mykola@meshstor.io/
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-6-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
Sashiko reported that RAID5 can accept a reshape chunk size that becomes
zero sectors. chunk_size_store() stores the sysfs byte value as n >> 9, so
writing a value below 512 bytes sets mddev->new_chunk_sectors to zero.
RAID5 then accepted that pending reshape geometry and raid5_start_reshape()
installed it into conf->chunk_sectors, letting reshape code divide by zero.
Reject zero-sector chunks both in check_reshape(), where normal sysfs
requests are validated, and in raid5_start_reshape(), so assembly/resume
paths also cannot install zero chunk geometry.
Test script: in QEMU, create a plain three-disk RAID5 array with 64K
chunks, write/read back a small pattern, write 1 to
/sys/block/md0/md/chunk_size, add a fourth disk, and run mdadm --grow
--raid-devices=4 --backup-file=... . The script scans dmesg for divide
error/Oops/KASAN signatures.
Bad kernel, eb29914412c3:
echo 1 > /sys/block/md0/md/chunk_size
mdadm --grow /dev/md0 --raid-devices=4 --backup-file=/root/md0-grow.bak
Oops: divide error: 0000 [#1] SMP KASAN NOPTI
RIP: raid5_get_active_stripe+0x863/0xc10
Call Trace:
raid5_sync_request
md_do_sync
md_thread
Kernel panic - not syncing: Fatal exception
Fixed kernel: echo 1 > /sys/block/md0/md/chunk_size bash: echo: write
error: Invalid argument chunk_write_rc=1 grow_rc=skipped RESULT:
REJECTED_ZERO_CHUNK_NO_OOPS
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-5-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap_cond_end_sync() is called with the sync thread's current sector.
That value is an exclusive progress boundary: sectors below it have
completed, but the llbitmap chunk containing it can still be in progress.
The old code converted that sector directly to the last bit passed to
BitmapActionEndsync. If resync had only advanced part-way into a large
llbitmap chunk, the in-progress chunk was marked synced and flushed before
the rest of the chunk was repaired. A later bitmap-assisted RAID1 resync
could then skip the remainder of that chunk and leave stale mirror data
behind.
This can be reproduced without editing bitmap metadata by creating a large
RAID1 with a lockless bitmap so llbitmap naturally selects a 524288-sector
chunk (with the default 128 KiB bitmap area, an array just over 16 TiB is
enough), making one mirror stale through the normal degraded write/re-add
path, and throttling resync so the daemon checkpoint runs while resync is
still inside the first chunk. On the bad kernel, bit 0 is ended early and a
stale sector later in the same chunk is skipped. With this fix, bit 0
remains Syncing until resync reaches the next chunk boundary.
Round the exclusive progress sector down to the nearest llbitmap chunk
boundary and end only chunks strictly below that boundary. Also honor the
force argument so callers that need an immediate checkpoint are not
suppressed by daemon_sleep.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-4-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap allocates its in-memory page cache and page-control structures from
paths that can already be holding MD reconfiguration or bitmap state locks.
For example, component_size_store() takes mddev_lock(), update_size() calls
the personality resize method, and llbitmap_resize() can grow the page cache
through llbitmap_prepare_resize().
Using GFP_KERNEL in those paths allows direct reclaim to enter filesystem or
block I/O while MD resize state is locked. That can recurse back into the
same array and wait on state that cannot make progress until the resize path
finishes.
Use GFP_NOIO for the llbitmap object, cached bitmap pages, page controls,
page-control arrays, and percpu_ref initialization. Leave the explicit
metadata zeroout path unchanged because it is intentional bitmap I/O rather
than reclaim-driven allocation.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-3-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
llbitmap_flush() sets LLPageFlush on each bitmap page before it queues the
daemon worker. The flag tells md_llbitmap_daemon_fn() to ignore the normal
barrier_idle expiry check and clean the page immediately.
The daemon only tested LLPageFlush. Once a page had been flushed explicitly,
the flag stayed set, so later dirty bits on that page also bypassed
barrier_idle and were cleaned the next time the daemon ran. That can make a
new write look clean much earlier than the configured idle window.
Consume LLPageFlush in md_llbitmap_daemon_fn() with test_and_clear_bit() and
use the returned value for the current expiry check. The explicit flush still
forces the current daemon pass, while later writes on the same page wait for
barrier_idle again.
This can be reproduced through normal sysfs operations:
1. Create a small RAID1 with --bitmap=lockless and --assume-clean.
2. Set llbitmap/daemon_sleep=1 and llbitmap/barrier_idle=10.
3. Toggle md/array_state from active to readonly and back to active to call
llbitmap_flush() without destroying the in-memory bitmap.
4. Write one sector and read llbitmap/bits immediately, after 2 seconds,
and after 12 seconds.
On the bad kernel the dirty bit is already clean after 2 seconds. With this
change it remains dirty until the barrier_idle window expires.
Tested-by: Mykola Marzhan <mykola@meshstor.io>
Link: https://patch.msgid.link/20260802195038.164272-2-yukuai@kernel.org
Signed-off-by: Yu Kuai <yukuai@fygo.io>
metadata_take_snap() bumps the live superblock refcount and then
dm_tm_shadow_block() allocates a new block for the metadata snapshot.
If the subsequent dm_sm_inc_block() of writeset_tree_root or
era_array_root fails, the function only unlocks the clone and
returns. The newly allocated shadow block is never returned to the
metadata space map, so each failed take-snap permanently leaks one
metadata block.
Free the clone with dm_sm_dec_block() on those error paths, matching
the final step of metadata_drop_snap().
Fixes: eec40579d8 ("dm: add era target")
Cc: stable@vger.kernel.org
Signed-off-by: liyouhong <liyouhong@kylinos.cn>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm-dust currently treats addbadblock, removebadblock and queryblock
arguments as block numbers on the underlying device. That is surprising
for a device-mapper target: a dm-dust table with a non-zero backing
offset can add bad blocks that are outside the mapped target, and a
badblock added for logical block 0 is missed because the I/O path checks
the remapped backing-device block instead.
Interpret badblock message arguments as blocks relative to the start of
the dm-dust target instead. Bound the arguments by the target length and
perform badblock lookup using target-relative sectors before remapping
the bio to the underlying device.
This intentionally changes the non-zero backing-offset behavior to make
the badblock control interface match the mapped dm-dust device, rather
than the underlying device.
Assisted-by: Codex:gpt-5.5-cyber-preview
Signed-off-by: Samuel Moelius <sam.moelius@trailofbits.com>
Tested-by: Bryan Gurney <bgurney@redhat.com>
Reviewed-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
If "dm_kvzalloc(percpu_alloc_size, cpu_to_node(cpu))" fails, the code
jumps to the "out" label and calls dm_stat_free. dm_stat_free does
"for_each_possible_cpu(cpu) { dm_kvfree(s->stat_percpu[cpu][0].histogram,
s->histogram_alloc_size);", which crashes with NULL pointer dereference
if s->stat_percpu[cpu] is NULL.
This commit fixes the bug by testing s->stat_percpu[cpu] for NULL before
using it.
Reported-by: Junzhe Yu <junzheyu1@gmail.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: fd2ed4d252 ("dm: add statistics support")
Cc: stable@vger.kernel.org
Kees Cook