mm/vma: correctly unaccount on mmap_prepare() failure

__mmap_setup() accounts memory for relevant mappings via:

security_vm_enough_memory_mm()
  -> __vm_enough_memory()
    -> vm_acct_memory()

If __mmap_setup() fails, this indicates that this accounting did not take
place, and thus it's appropriate for __mmap_region() to jump to
abort_munmap.

However if call_mmap_prepare() fails, it also jumps there and any accounted
memory is not correctly unaccounted.

Fix this by handling each error separately.

Link: https://lore.kernel.org/20260902-fix-unaccount-mmap_prepare-v1-1-ea070189fdfb@kernel.org
Fixes: c84bf6dd2b ("mm: introduce new .mmap_prepare() file callback")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This commit is contained in:
Lorenzo Stoakes (ARM)
2026-09-08 23:28:06 -07:00
committed by Andrew Morton
parent e14a345480
commit 6cc27d8219
+4 -2
View File
@@ -2859,10 +2859,12 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
map.check_ksm_early = can_set_ksm_flags_early(&map);
error = __mmap_setup(&map, &desc, uf);
if (!error && have_mmap_prepare)
error = call_mmap_prepare(&map, &desc);
if (error)
goto abort_munmap;
if (have_mmap_prepare)
error = call_mmap_prepare(&map, &desc);
if (error)
goto unacct_error;
if (map.check_ksm_early)
update_ksm_flags(&map);