From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (unknown [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id 90215802A5 for ; Wed, 14 Oct 2026 17:12:50 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69EHBZfR001241; Wed, 14 Oct 2026 20:11:36 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69EHBZfR001241 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="MHrFguZ9" Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69EHBIZT001189 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Wed, 14 Oct 2026 20:11:18 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69EHBIZT001189 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4a1813fea7dso66036085e9.2 for ; Wed, 14 Oct 2026 10:11:18 -0700 (PDT) X-Gm-Message-State: AFq9FYKOD40fK9n2hUGEK+eepsR7kmkVJ8L0JfOrmtMBjqGjM5Ga00m4 wdkH40szS0sgyeslHy7EhEkfR8cALIDJkSpId9cfgJD7Lj7c68b0U/cXfw80OSamF4fyZc7m7UZ Iodu4K6b9ueZstRbwYXHdxZKWILG+f1gEYS9pNG5MFHbhj8E2LVQjgw== X-Gm-Gg: AYBFou09Lu87BlLNbALrr0tbsQS1uaJrVraFY6d+72eKtbUx5u1j20TQG+lCfYM3bYJ J/icizVIupmjSp491ch9PDxSTtljUqePiN0z5+ZIwGlHhMC/0r1aoswdQ9OIhWGgbGWljD1xGDk byRLZfk++7YMgG4SmULIRXnYkZDWJjqO254Quz3QHzfBuR4tuTXSxfbFrW/e3fJ9lYu7E3vVAZd Y2OrSNIg8mLDurcNcGKPsUeJ67878nSQcLYJsDab3QKa96hjEMG7eDUhegKvOIaRdfDjnLJ7fST QtaWxiFQrDoFFJ4bwN0kYLhi8M7V9z+PP7RmeewXhRRsL4ndvr1EDoqyf+c3KB3+DRDyGOVcEsY HxCzxDM//iCXwtInB7A== X-Received: by 2002:a05:600c:c3cc:10b0:4a1:a7f8:c12b with SMTP id 5b1f17b1804b1-4a1a7f8c219mr107198395e9.19.1791997878280; Wed, 14 Oct 2026 10:11:18 -0700 (PDT) X-Received: by 2002:a05:600c:c3cc:10b0:4a1:a7f8:c12b with SMTP id 5b1f17b1804b1-4a1a7f8c219mr107197945e9.19.1791997877715; Wed, 14 Oct 2026 10:11:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1791997877; cv=none; d=google.com; s=arc-20260327; b=MiLWC5xTJAv/nCxvf/+2q3Srnhr3zriJM1bwXshNjDHuTxWaLGvpMmRb8OQaE6ESjl dHb7nhSIwYCRsanwZYsgpBgwJe+quHJNGR9BjlyD/XDiP/ySKwu7QXdNMvelj21eWFIW J4SrXMXcEuqkoc+q/eH7jm6FvEtsUAaokzxWKdX32bCyCQAWJczhLfnx3Zw6rQHBx/Jz T+Xx8BvnV3imjnMGd6HxRpU2xZulN6Nope6xIjOpHZtdzf7Yh10GgNcdxge/r5X4+sqF 7jCGcHwy84OUtvhXX5RPyY9QddugemrV2G1x7gfPTPf0lAXhi6XPhbSm3kRsO1trmlPc yo8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:dkim-signature; bh=feya81klbkbfF3o7ovac+LUs53LqeTJpBrCbtKsBaeQ=; fh=aQfw1i5a4NQPWqzJLDKkrMTvdRoUn/cswL6yDWdmKfA=; b=Abc/oqWUsALsCUp4aozycpRxK6LWVqfd+SBb4YrpST5hpdjur8CQYLklNY+34+NvQs NMYgzmi7GskDwvkFY2sWgVf7sz3ajRZMFzqskVui6UiX6EtKNGnbzjUrO6Cb0HAXNnsN 5ufV+a5nHQ9xvZLhmsQ0vJ2FdJspFRTSWZT2P1HRQEoKqJyeRT4OpHp8pg8vtgJo/HdG v4LURTcFotgjKp40+NRmCp5yJc5BZZOmY4UUg5UPIURODI49Q7yY+Q+NUD8z8AohQohr N99xDJduj1Ze3OfQh4xuYi135UvM9JjY7pVm96HfLAW1ubORhxA9mEWLiJ28m8Ax3uys V7Hg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=MHrFguZ9; spf=pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=khorenko@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from relay.virtuozzo.com (relay.virtuozzo.com. [130.117.225.111]) by mx.google.com with ESMTPS id ffacd0b85a97d-48de0bfdda3si12954177f8f.339.2026.10.14.10.11.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Oct 2026 10:11:17 -0700 (PDT) Received-SPF: pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) client-ip=130.117.225.111; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=MHrFguZ9; spf=pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=khorenko@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=Message-Id:Content-Type:MIME-Version:Subject:Date :From; bh=feya81klbkbfF3o7ovac+LUs53LqeTJpBrCbtKsBaeQ=; b=MHrFguZ9aeDvdBKk4My MzBsvtqbtNxpOhdemuCgsbnZc5BX/JBtt+5gDgzxy+NoE70BNICQ9w4kFQ10HTJxCOjgIieGMeiB4 VRk/8J9ZFHhxFutvlRQ9OgS2WeE89wD0SdyGRBLpcyP/ZwYFO1Dh5g8ItqT5hNPdB68riM91drxT4 P9AtaTgHXRuymRlwat/0mOfv+7OPrdupcAimIS4F6+nRxn9tn40W9iBnfrtHWlxDupunZO4sS3zsK 2CPBuyIxuFXMPuU0Y9ovAgQS+8Nyd0yOaGddQ+9dBMmFtEoZDzdOfW6iLzQKQPFkY0CW52Tz6Ick+ kmuMK0v1FaZVkYw==; Received: from ch-demo-asa.virtuozzo.com ([130.117.225.8] helo=f0.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1xH2S8-0081J5-0q; Wed, 14 Oct 2026 19:11:16 +0200 From: Konstantin Khorenko Date: Wed, 14 Oct 2026 19:11:09 +0200 MIME-Version: 1.0 Message-Id: <20261014-b4-prep-khorenko-vstor-149847-2-doexec-pinned-memory-v1-6-36f1120f9b57@virtuozzo.com> References: <20261014-b4-prep-khorenko-vstor-149847-2-doexec-pinned-memory-v1-0-36f1120f9b57@virtuozzo.com> In-Reply-To: <20261014-b4-prep-khorenko-vstor-149847-2-doexec-pinned-memory-v1-0-36f1120f9b57@virtuozzo.com> To: Vasileios Almpanis , Andrey Drobyshev , Pavel Tikhomirov X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6871; i=khorenko@virtuozzo.com; h=from:subject:message-id; bh=ENFF+Blj5Q1EBNzVmovKFa7+HNElxTfTQl2v/WTo1UY=; b=owEBbQKS/ZANAwAKAVGWCkf5YVwqAcsmYgBqz7ey/mMUtFaDAmpqcbxKLAw0tcepuyDAUArL8 5ml7chmXhOJAjMEAAEKAB0WIQRWD23IPcXT0GO9CCxRlgpH+WFcKgUCas+3sgAKCRBRlgpH+WFc KjkND/4/M0O+pFVxEQYxU8s2CiTxUMGk/qiWRj9+gJ+1i6YS9Czmp3NC9wUP3g44sVhufdMxHtJ hOqg/WPKdz34MygkJOO22zwVTs/nfjr2UPTTF0mZu1rrR6/L6DF2iXCxxmTG5toKuyjOUQag23z WhuwOQDrIxctF92rBwbu3I1Fc53qxZr3o5XL21hyZ64Zzokvf53BtoZ5PTZX8Re8/eQskCsqgRw JJL8eOsrMjM4my+rDMAIEnct/IEgDtFgsITeTSt7LjJJ4z5WhIx7eO415zAq6PNQHNewBd5AXEf wSi+H2u2QVedT5tLTz1LzH7Xl+qPTyrswAFhEKo6N9CeI4LIjX89/gVnlXyZIH4bwlthkvPHAtx hjif4qGhxngypNoszy0hBcQ/hVn156w3tEep+XwVSGBDABK7E2tfH7Akc4CpKlI+6QK8AhGkjXl x49MukYj5g1EwRBO5VcV/azdhX7S/rOXdwhkCzNiBY3ruAVhraXlnI3vKS1euPcLVbkv2t23Y6M FJ8xX4P/5KfLsSa3udmJzETDPAHz2kKCjqvotXqYdzxtyx5cOiSgVL5PXcWRE8xZoPkMcSmYYBY Fv9a1eUFHUF0mZDHeV8L1JIKiX4t+dYNR0U1C0MwoJB2oJ1tIdHlgFgLmQkI+ZC93KoGlc5DioT wnlsAeldYYduB1w== X-Developer-Key: i=khorenko@virtuozzo.com; a=openpgp; fpr=560F6DC83DC5D3D063BD082C51960A47F9615C2A X-OZ-Fwd: true Cc: devel@openvz.org Subject: [Devel] [PATCH hci-8.1 6/8] oracle/mm: zap the preserved ranges of the old mm after copying them X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org A preserved page keeps PageAnonExclusive in both the old and the new mm after the copy. exec_mmap() only drops a reference to the old mm, and anything that holds mm_users keeps it mapped there, such as a /proc/pid/mem reader stuck in copy_to_user() or process_vm_readv(). Code that trusts PageAnonExclusive to mean a single mapping then goes wrong. UFFDIO_MOVE in either mm calls folio_move_anon_rmap() on a folio the other mm still maps, and once the destination anon_vma is freed the next rmap walk reads freed memory. try_to_migrate_one() hits VM_BUG_ON_PAGE() for a writable PTE of a page that the other mapping made shared, and a writable set_pte_at() in the new mm trips PAGE_TABLE_CHECK. A write through the old mm reuses the page and lands in the new image. Zap the VM_EXEC_KEEP ranges of the old mm right after the copy, under its write lock, so that the copies are the only mappings left. The pages are not freed, the new mm holds them; a later access through the old mm faults in fresh memory there. The unmap work moves from the teardown of the old mm, which ran it with several threads, into the zap, so zap_vma_mt() spreads an anonymous VMA over several threads as well, each with an mmu_gather of its own as concurrent MADV_DONTNEED calls have; the teardown finds these VMAs empty and no longer treats VM_EXEC_KEEP specially. Fixes: 37d9f55655ea ("oracle/mm: do not write protect COW mappings when preserving across exec") https://virtuozzo.atlassian.net/browse/VSTOR-150231 Feature: oracle/mm: MADV_DOEXEC madvise() flag Assisted-by: LLM Signed-off-by: Konstantin Khorenko --- fs/exec.c | 10 +++++++- include/linux/mm.h | 1 + mm/memory.c | 74 +++++++++++++++++++++++++++++++++++++++++------------- 3 files changed, 67 insertions(+), 18 deletions(-) diff --git a/fs/exec.c b/fs/exec.c index 8677347963e20..32567010a2f27 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -997,7 +997,15 @@ static int vma_dup_some(struct mm_struct *old_mm, struct mm_struct *new_mm) } mmap_write_unlock(new_mm); - return ret; + if (ret) + return ret; + + /* Whoever keeps old_mm alive past the exec must not reach the pages through it. */ + vma_iter_set(&vmi, 0); + for_each_vma(vmi, vma) + if (vma->vm_flags & VM_EXEC_KEEP) + zap_vma_mt(vma); + return 0; } /* diff --git a/include/linux/mm.h b/include/linux/mm.h index 75b8bcd9f4428..9ebf3c6dcb978 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2462,6 +2462,7 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); +void zap_vma_mt(struct vm_area_struct *vma); void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, unsigned long size, struct zap_details *details); static inline void zap_vma_pages(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index 1c128162f5cda..2566f02fb7d5e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2043,30 +2043,20 @@ static void unmap_page_range_mt(struct mmu_gather *tlb, BUG_ON(addr >= end); - /* - * Assumptions made below about VM_EXEC_KEEP VMAs only hold true - * if the entire address space is being unmapped. - */ if (!tlb->fullmm) { unmap_page_range(tlb, vma, addr, end, details); return; } /* - * Pages in VMAs being preserved will have an additional reference - * due to the mappings having been copied prior to unmapping the full - * address space. Since lock contention due to freeing pages will not - * be an issue, use additional threads to unmap them. - * Otherwise for sufficiently large, private anon VMAs, using - * multiple threads can result in severe lock contention the swap - * subsystem if a signficant portion is swapped out. Since the number - * of swapped pages is tracked per MM and not per VMA, err on the - * side of caution and assume the number applies to the VMA and revert - * to unmapping single threaded if necessary. + * For sufficiently large, private anon VMAs, using multiple threads + * can result in severe lock contention in the swap subsystem if a + * significant portion is swapped out. Since the number of swapped + * pages is tracked per MM and not per VMA, err on the side of caution + * and assume the number applies to the VMA and revert to unmapping + * single threaded if necessary. */ - if (vma->vm_flags & VM_EXEC_KEEP) { - job.max_threads = 16; - } else if (!(vma->vm_flags & VM_SHARED) && total_sz >= min_chunk_sz * 3) { + if (!(vma->vm_flags & VM_SHARED) && total_sz >= min_chunk_sz * 3) { unsigned long swapped_sz; swapped_sz = get_mm_counter(tlb->mm, MM_SWAPENTS) << PAGE_SHIFT; @@ -2079,6 +2069,17 @@ static void unmap_page_range_mt(struct mmu_gather *tlb, padata_do_multithreaded(&job); } +static int zap_vma_chunk(unsigned long addr, unsigned long end, void *arg) +{ + struct vm_area_struct *vma = arg; + struct mmu_gather tlb; + + tlb_gather_mmu(&tlb, vma->vm_mm); + unmap_page_range(&tlb, vma, addr, end, NULL); + tlb_finish_mmu(&tlb); + return 0; +} + #endif /* CONFIG_PADATA */ static void unmap_single_vma(struct mmu_gather *tlb, @@ -2214,6 +2215,45 @@ void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, hugetlb_zap_end(vma, details); } +/** + * zap_vma_mt - remove all ptes of a vma, with several threads if it is large + * @vma: vm_area_struct holding ptes to be zapped + * + * The caller holds the mmap lock for writing. An anonymous vma is zapped in + * PMD-aligned parts, each with an mmu_gather of its own, by several threads + * when it is large; other vmas, and all of them without CONFIG_PADATA, are + * zapped by the caller alone. + */ +void zap_vma_mt(struct vm_area_struct *vma) +{ +#ifdef CONFIG_PADATA + struct padata_mt_job job = { + .thread_fn = zap_vma_chunk, + .fn_arg = vma, + .start = vma->vm_start, + .size = vma->vm_end - vma->vm_start, + .align = PMD_SIZE, + .min_chunk = max(1ul << 27, PMD_SIZE), + .max_threads = 16, + }; + struct mmu_notifier_range range; + + mmap_assert_write_locked(vma->vm_mm); + if (vma_is_anonymous(vma)) { + lru_add_drain(); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, + vma->vm_start, vma->vm_end); + update_hiwater_rss(vma->vm_mm); + mmu_notifier_invalidate_range_start(&range); + padata_do_multithreaded(&job); + mmu_notifier_invalidate_range_end(&range); + return; + } +#endif + zap_page_range_single(vma, vma->vm_start, vma->vm_end - vma->vm_start, + NULL); +} + /** * zap_vma_ptes - remove ptes mapping the vma * @vma: vm_area_struct holding ptes to be zapped -- 2.43.0 _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel