From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (unknown [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id 2AFF5802A5 for ; Fri, 9 Oct 2026 14:01:56 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 699E0ZYp008350; Fri, 9 Oct 2026 17:00:38 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 699E0ZYp008350 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="Z8TDihX0" Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 699E0N4g008329 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Fri, 9 Oct 2026 17:00:24 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 699E0N4g008329 Received: by mail-pg1-f199.google.com with SMTP id 41be03b00d2f7-cc4216aee8fso6910065a12.1 for ; Fri, 09 Oct 2026 07:00:24 -0700 (PDT) X-Gm-Message-State: AFuF++m9OduMIFjKXOxIuFVxGZ7Xq587OuPZW5bB19kIAurHMaehRKPq NUklzEaB4n22Yo6/OQhz3PVPbTT374bbbltPwNQb4fsvZf1DC1s+Qmf8+zM4Vr2aHvUXQbdyfn1 1uKXHbdw5bP8IK7ddqoobnq2i2pQGD9JZ4I7IC+u2eQWdpiga8ty16A== X-Gm-Gg: AYBFou2t1CrhOD2lr1IWSf4/UUZA2YS94oiAZNfU/wvPvbK5uxpE9JfsvB2CrTmmW3n h+n1uyQwbI9c2+/Q2zIQJqWpJAdJyqSorI1RvTWFe1sVApHRG4Pq9Rakpa77WHi8f/CjqzEOdJf rhHHT8KzOheWerOfrUCPcUlyHQE3WljBALg1vFbzFr17bvZiofbbnQHjuIs9WXGywmqNHeMiZ6g tO4ZW+8nC5/7bfwF4IyO/px0gnRIFGcGbHAo3VFAAP+4VFj3xv7ExaaW1ZOvJFspMfIZxP7S7vN /z6yMn1uO9dJloTLE9bzCoXv58xxATdGyq+nJhQZIQOtQOmLZX96Dzp7qHZ3a1tEo5YL8EZou1o SDPoraYMC86Jx0hHfTqrze7hIJZQawwsijByKHLIqvmfm3BMV46lcasyo83BMyMMxpbcvjLPvLY ZP2Wv9rf8QkuI/+RfGgKY8m3kfmxL/dcI= X-Received: by 2002:a05:6a21:7a46:b0:3e1:4215:f519 with SMTP id adf61e73a8af0-3e16bebc03emr1698240637.55.1791554421449; Fri, 09 Oct 2026 07:00:21 -0700 (PDT) X-Received: by 2002:a05:6a21:7a46:b0:3e1:4215:f519 with SMTP id adf61e73a8af0-3e16bebc03emr1698181637.55.1791554420424; Fri, 09 Oct 2026 07:00:20 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1791554420; cv=pass; d=google.com; s=arc-20260327; b=pHDWuKgaZnA53E1ciDQQuE9IEYWayynStNXMSEvHLsdkIHlbmdcn1PnCRb2BeP9p8+ cQlpYIlmhHcR2kgbEQDo1TcnITtLWvKt7b8Nn4ikYYhDIJst6NsbNwRX6xaYA0PONgGl fOeAuJLx1pDSvkDu/HPte+n/kn0dB2VPaSF0gUNGnNdkUdIVgZ01V/0COKIBzCpAvsJF ntRJhYFN3fJDthL1gVj1K5IcPSkPMQXLhvkEeK3vYwEDB17+JCwYHTl06Jj8/GqINOxK 3fVwTLGUcQAm/hLVYgt1y4NXhyvynSbfsx5yQZW0K6Lv8ltfScQCc/J6QoUhMlzCdhGX lADQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=mime-version:cc:to:in-reply-to:references:message-id :content-transfer-encoding:subject:date:from:dkim-signature; bh=4AUbFb1cvuMT3AyeIkKbRnCVmYYIcyPOkYDb/E2i5UM=; fh=QMK9nbJ5AYrM+hiPA/q5PkyEnWxrtaWE6qYGnw4J4RA=; b=cno7wZ+m8QX1OiWO7oaWp85Oqd2XRt1YmXqY5Ha1ov/Xk7gO909/qicGyea1G80f5m 8e0iIlK6+0lqiZ9E9U6ImOUejoxzUfb4kD0f7hERlRm6oKu40ZLiKCSb9DjzGcVzVvBq ADjNr19QJwRSzD5kqsqZE2TUbOL0jpQKnpinKUmUKVtT23WjzNY/ruOiKggpbKjGzX0D BJj8Cj8sx76IYaTJdWFFw1aNkK2Xg75/eUFcq8YQMmzQs8E1fIkvDNA2Kjf8kwFQ8qSZ PwvAU1FqoUxmFdrtKxZRB3faclgdDlV04k3+661AI7i+1bXz0wZ7SDsclByboTFXu9zY qfiQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector2 header.b=Z8TDihX0; arc=pass (i=1 spf=pass spfdomain=virtuozzo.com dkim=pass dkdomain=virtuozzo.com dmarc=pass fromdomain=virtuozzo.com); spf=pass (google.com: domain of andrey.drobyshev@virtuozzo.com designates 2a01:111:f403:c201::1 as permitted sender) smtp.mailfrom=andrey.drobyshev@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazlp170100001.outbound.protection.outlook.com. [2a01:111:f403:c201::1]) by mx.google.com with ESMTPS id 41be03b00d2f7-cd3da079de5si3637768a12.252.2026.10.09.07.00.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Oct 2026 07:00:20 -0700 (PDT) Received-SPF: pass (google.com: domain of andrey.drobyshev@virtuozzo.com designates 2a01:111:f403:c201::1 as permitted sender) client-ip=2a01:111:f403:c201::1; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector2 header.b=Z8TDihX0; arc=pass (i=1 spf=pass spfdomain=virtuozzo.com dkim=pass dkdomain=virtuozzo.com dmarc=pass fromdomain=virtuozzo.com); spf=pass (google.com: domain of andrey.drobyshev@virtuozzo.com designates 2a01:111:f403:c201::1 as permitted sender) smtp.mailfrom=andrey.drobyshev@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JZQ5r3DdoHfd7gs33AoGYD7Glbf2QWQOjC8t5LBaecBvrYgbbbB7kdJX8hhfodpEdeH6rZzdc7PjJRn+KIb5kKoSrf60UBXccOLjAvp73LRukvgglK2viKREjlhp8DG3mu+A6Nk27LCs4tC2rxyiNaq71q0ilr+ZeppafeOiKml6N1ny9wWOtBon1ds8SGAZVM054Xqr6yUD/jBQjHeKCMlSnmdz+m6S0IDBQmR+Crclewk8JD0dQ7nFAOlHLHF27ZSI8NdfICvWqioIdM+kSt1fwQV78fTgnLgupQElaIVoNatGv7MDVoV/xMZ/KLyQ+p+Q92YRrd/2eKmV3XKXGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4AUbFb1cvuMT3AyeIkKbRnCVmYYIcyPOkYDb/E2i5UM=; b=D/WDkO8ML2UycCEds5YqvU8WAluIGn5tdqHVwXcNIMPYvzYffoCOULZHBML0kFAwjnd8Z6yXQACMQVL8CUUv4U4NaHEdRiBhPh9b6mnT+odRi6nAe3m/Btr22U0UVpwL3kgg6T3tVyxtQ9xVKrlGF25MxIbZZAqcUe0TItrt0U8BLMKUi29qoReA2kmm9/2wdCfNaizcLBCrofCo2YxUOhArefNcCoeS+hTNkfiEhVHZW3ZVSChmY5Kws353Pplth3R1zKv0bRoKVw7qXtFvwzyS25LBIzRSkruOGAomAIL0Rmnjtr7XeZNaX0tbuGkR2RIIaVoz1jmjYqEGxJClvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4AUbFb1cvuMT3AyeIkKbRnCVmYYIcyPOkYDb/E2i5UM=; b=Z8TDihX0NPS9PagxSkXSo/EoVHnWo5ihfAXHIrJaU2fzETlOf82Cw9E4CtgVfKui376DHclauMafb4BgF4Wuy3nGMQAAV6YznU4bVZv4fRvmSkmgOGJ/tliKyJIcz5vTO4w4W7WusKYuiWaA+78/VcSNwa2j16l5H0ipgpyhNLG997f6LGMbGDxgB4Smi7jhk0yYemx1fesIxf9uGmAndXFiQsFb3mlBWYhOJtDDVJqlOoKeXAPB9KAd2UcAfsczY/AXGzvRNE2zatJpkH3nS1wK10L4En7TOCz/M7ovhBuBTtb32KiyXfWMYNYEmyXXSkoxpYzaJnhneKGYxBmLwg== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by PR3PR08MB5802.eurprd08.prod.outlook.com (2603:10a6:102:8a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.496.15; Fri, 9 Oct 2026 14:00:16 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0496.010; Fri, 9 Oct 2026 14:00:16 +0000 From: Andrey Drobyshev Date: Fri, 09 Oct 2026 17:00:04 +0300 Message-Id: <20261009-vfio_v1-v1-1-1901cd6c3c74@virtuozzo.com> References: <20261009-vfio_v1-v1-0-1901cd6c3c74@virtuozzo.com> In-Reply-To: <20261009-vfio_v1-v1-0-1901cd6c3c74@virtuozzo.com> To: Konstantin Khorenko X-Mailer: b4 0.16.0 X-ClientProxiedBy: VI6PEPF0000022A.AUTP296.PROD.OUTLOOK.COM (2603:10a6:808:1::8e6) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|PR3PR08MB5802:EE_ X-MS-Office365-Filtering-Correlation-Id: acbd1706-09ad-4b24-475d-08df260da4b0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|52116014|366016|1800799024|376014|23010399003|6133799003|22082099003|18002099003|10067099003|38350700014|56012099006; X-Microsoft-Antispam-Message-Info: IMwogC1bvNxjnlBWAFio4FSF53jxjxLCYi0ds4J+fg0WkAovu3gYSYVEgbkpjoedXxLFLnAcyPrnP/FfZDnfi4BbfhO0vOqLuh5/wRSocKjrhU1pUimNYdBxrzNK5ePmJfRfnbxQodNiQGI+v1XeNtvkGIV73PGBe6FLzrDndKQF6ml/zXniZqz5tumF7Hsf7AOPr2pE5CmdcSwEZDi06kR68vVmAr/KIdgpXn2HYyDDlpVjT5kEpBxY5vpsDmIzEbco5xRGQOLRPIDgn60EuLeyH9eEoJfFd2OZLYvEjG1Vi86iGZf62Az03CAGV6LzEdNpoH9yyaXSbVcZWRvGY6GToP3PgkpT0v4krrTO7zTJsBElGDc0+0JPwx3bT0v7AxHotKW6zWzMe4QGnudDkWOUdJzFSzsbrkQDeI8OcAo43ecAVluh6bec1CjO2mjaqN2T5H95VLS1EApfW47RsjlcZpxXBwQhMi2b3Bp7cig3jGpZ4O8mZrl5I66w6Jy0O8IMFk4fSxxmca/qPGc3pO7BHUdNtFD7sj03XpK/92l1pA+IsNyabQ6guN5YCrZRO9bh+xJq0BlrBMv+Wb5jwfj7OfQrFsTF1RH0FgO706Qhe9i3xK7yDUeBdtLGp3NKaBkTX4ksQtwJVufQsPm3RRgfBJwdjxPPnd/iLbpzrblTU8wywVRabIEWFYPrpQxCyjBDYf9a5lARQaq4tP+Y1ckXbc/LzvOHg/bY6bv1dFw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI0PR08MB10656.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(52116014)(366016)(1800799024)(376014)(23010399003)(6133799003)(22082099003)(18002099003)(10067099003)(38350700014)(56012099006); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ak01eGRiRWMweTdWRHErbldvaFQ5WFhmWFZqdGJXdzRNWklyNjhMY29UaVN5?= =?utf-8?B?NS9tc2RndFhwdDk0UWUyYTh1eVI2aGM1RjR4blYvWXkvNGNITFNORTFSYW9w?= =?utf-8?B?ZzVvMkxIQWhqREJ6V1ZQLzFxUDBxRkgyZWxJWnNoYnh6UjE0V3llelZ0SXRK?= =?utf-8?B?VCtXY0dQSmRiOGN5RjB2a21rYndGa1ZEQmlVTUEwUjZiSUI3Q2N0bm16a0Nn?= =?utf-8?B?UEYreTdmSlNYK1hJVC9oNmt6SU9sQ2NIdjhieGxPeWt4eEpDY2dVaVlBRTli?= =?utf-8?B?dVlKU1NCL1orOUN0QXh0U1doQzFBSXdkd0EySE1ldWY1S0ZKaUpma3k3NlZD?= =?utf-8?B?aEErV2c4bFgrRWVjUi9BRFJ0cy9YVWpqZWczSWpSTk1TWlZ3UEZuc3QrZm81?= =?utf-8?B?OFVHb2x6cVhMeVlBbjFrbnJ4Q2NPeXY3cDV2ZVhNekhqNGtPdHlQa1Y0d1Ax?= =?utf-8?B?MlEvK2pOR0k0RjB4aUpTbi9sU1JaMC9DblFmWllQOXdiV1V0ZFkrNmR0RWlp?= =?utf-8?B?a05FRzNVSjc3dkVrWnhmNzBXaGVFdXB2UE0wUFV3Q01IQk1LakVwdWY0VTVx?= =?utf-8?B?VC9tZHBjUUhTZFdVYmc0QllOTkZ6S1NRdXZVNVJZa0xkRDlwYWQ1Nk43eDMr?= =?utf-8?B?WkptNjRrV1ZPNWExOXJMMFlCZFNuREtkN3ZpcFp2VjVzQ3lxUFpvNG9jQ3lK?= =?utf-8?B?clFQS0pFa0J0RURVbmpXdFE1SjVzQWlFeXM4NFMxdzViUWNGYTJGNDFmUTF1?= =?utf-8?B?WDMvMGN6L2NaUFVlZ3NKM1BId0J4M3Q4MzYyUnBTSk5OUk9oU3pDMVJkbXBE?= =?utf-8?B?YVA3bGlUeXp0NzkyT0p0RmlIU0lLZ0V0NVhPTzQ1UzdqRVdTM2wweHh1RHF0?= =?utf-8?B?U05XdURYRjVPZUk5ZytLYlUyeUswd2pQVHhkUGJhOWtjNVlzRjhiK3VoTlRh?= =?utf-8?B?T1pPOGVFMk9Id2plUVRzRi8zQ0k5SURzMmt4QlJaZ1JGUDJQMUdBT1l2UUNI?= =?utf-8?B?MDRSb3l3QTVoZVh6M2pEcmM4QTZxdzMyZFF5VkdVTEFlRUt5TUZCNGV0YWk1?= =?utf-8?B?bUREdExpZjRZckQ1Zy8zUGoxYVRMSCtrMkVQWEVMeEQ0VndKeUROa210OHRB?= =?utf-8?B?YVdzYUdEVHRiUk0wQXJmbjBFWHJnNVBKTjRoTFMzb2tZQzZsUXRhaCtkU3NZ?= =?utf-8?B?akhIVHI0U01CQTIxald0N21kZzN0Y0hZYnYrRklia0hzVEUxenBoUUw3SWxB?= =?utf-8?B?akRnQW9ueFZJQlVGVnFuQWN0MFNCNVVFQzFNUmZLYVorSVB4bjZid05KT0tP?= =?utf-8?B?Z3FYcXcxam5ORFJieHlkMUplckJGNVdtMjFiaGswaXQ0SGdMUGhwcmlQcndC?= =?utf-8?B?WFU5N09SblhzT2RmYVAvQ3RCYnUwTUgySFh5UW5KMTVqRTMycXFEeDFIcUJX?= =?utf-8?B?NDZOTmdGS25USnFLendEaEcvenRja3BPN1UyQmtXUGd0VmhCakZIMU9xeCts?= =?utf-8?B?c3pVMnI2NjMwSnk3UnBXU1R3L3lyczRwWkR5a3IydzMxTXBhWlRZRVFmdEFa?= =?utf-8?B?SWdaK2tRWWlibFVTd0J1RGtUNFRKRlNQVVJ1TS9sdGcxNktENURFMHVHc2VX?= =?utf-8?B?dVNQY1FsWWJ2MkttQmoyM0c2ME4ySk9NM1Q1clF2VXByRHkydnc5ZVl6QnA0?= =?utf-8?B?UUVUam1hN0pUaHMzMFIxYUUzdzlqWC8zbjZkZjErSXRxc0RhM25GU2ZycTRF?= =?utf-8?B?OE9mSU44ZkZnNC9sUzRSOWNPamo2alcrS1g4NnF0aStwNXlBWkdnZVdGOUVZ?= =?utf-8?B?Z3VCcll0cDIxWWNkRHZjQUJOUThTNzYxNDU5empqZnVkbk13S2FobFZFdFNG?= =?utf-8?B?TGJIWXRobFRkNjBXays3dkx2WDgzN2o2dThKTnZTeDNzODdmRk9jeVVHMk15?= =?utf-8?B?cUhLUktwemEzM1dmZXpyekxDS2xZR3hRNFREZ1J3SSt5cXpZSU1CZUtla1hr?= =?utf-8?B?KzFDMFBxTE5lM3dRajVnZDZVM09KSUNmbzRsVkxLWVhXTC9FVXdCc2JrcnUw?= =?utf-8?B?RTBmd09TOU8vUS9DQmxoeXMwQ1NWMkN0NFBWbzVYWnMzcmEyam1seU1ZN1o2?= =?utf-8?B?YnZ0YnZpQVVaTTV0RW1KTHRjcHdRWVlmN0dWNVRTWmZjaGVSRm9KcjFwMFAz?= =?utf-8?B?WXBlZGRpQlZzeVVtMHVTN2ZZZzkvVXp5NWlSbTdQMmt5UnFQWDc5a1QwV3gr?= =?utf-8?B?WUF2ZjlVNDN6N1pNU3NuS3ZGZ05QNkRDR0Y4cUh0dGhmdnF0WnFHbnc1MTBY?= =?utf-8?B?UUF2QUIxWk5HVkJ0RWNSWWgrSFBQeEJBOVNyeEZsTUxLT2dpRUlrN2pVVit3?= =?utf-8?Q?iuF/ooW5iE8BduL4=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: acbd1706-09ad-4b24-475d-08df260da4b0 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2026 14:00:15.0792 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GQS9z1DFFWRyzUDeN8aTaap1Fdc2m6+IZzL4AM7FtqtsTCl8HpaAlr7TH4HDmM2IbYnJuHL3lJsGCC4UlOyKbMzhMOT5RSx8K9F+BmOBHmg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5802 X-OZ-Fwd: true Cc: devel@openvz.org Subject: [Devel] [PATCH RFC PATCH hci-8.1 1/4] mm: support moving anonymous page tables between address spaces X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org Teach move_page_tables and move_huge_pmd to use the destination mm for page-table operations and transfer RSS, swap-list and huge-page table accounting. Restrict cross-mm moves to same-address anonymous DOEXEC VMAs. Add the parallel exec move wrapper, joining all workers before returning on failure. Existing DOEXEC callers still use the copy path in this commit. https://virtuozzo.atlassian.net/browse/VSTOR-103860 Signed-off-by: Andrey Drobyshev --- include/linux/huge_mm.h | 3 +- mm/huge_memory.c | 28 +++++++--- mm/internal.h | 2 + mm/mremap.c | 140 +++++++++++++++++++++++++++++++++++++++++++----- 4 files changed, 152 insertions(+), 21 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index b94c2e8ee918..e2ec0348bebd 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -31,7 +31,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr); int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, pud_t *pud, unsigned long addr); -bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, +bool move_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma, + unsigned long old_addr, unsigned long new_addr, pmd_t *old_pmd, pmd_t *new_pmd); int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, pgprot_t newprot, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 90868f69ee33..9954dac5e53e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2219,12 +2219,15 @@ static pmd_t clear_uffd_wp_pmd(pmd_t pmd) return pmd; } -bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, +bool move_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma, + unsigned long old_addr, unsigned long new_addr, pmd_t *old_pmd, pmd_t *new_pmd) { spinlock_t *old_ptl, *new_ptl; pmd_t pmd; struct mm_struct *mm = vma->vm_mm; + struct mm_struct *new_mm = new_vma->vm_mm; + bool cross_mm = mm != new_mm; bool force_flush = false; /* @@ -2243,7 +2246,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, */ old_ptl = __pmd_trans_huge_lock(old_pmd, vma); if (old_ptl) { - new_ptl = pmd_lockptr(mm, new_pmd); + new_ptl = pmd_lockptr(new_mm, new_pmd); if (new_ptl != old_ptl) spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); pmd = pmdp_huge_get_and_clear(mm, old_addr, old_pmd); @@ -2251,15 +2254,28 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, force_flush = true; VM_BUG_ON(!pmd_none(*new_pmd)); - if (pmd_move_must_withdraw(new_ptl, old_ptl, vma)) { + if (cross_mm || pmd_move_must_withdraw(new_ptl, old_ptl, vma)) { pgtable_t pgtable; pgtable = pgtable_trans_huge_withdraw(mm, old_pmd); - pgtable_trans_huge_deposit(mm, new_pmd, pgtable); + pgtable_trans_huge_deposit(new_mm, new_pmd, pgtable); + if (cross_mm) { + mm_dec_nr_ptes(mm); + mm_inc_nr_ptes(new_mm); + } + } + if (cross_mm) { + if (is_huge_zero_pmd(pmd)) { + /* The source retains its per-mm zero-page reference. */ + mm_get_huge_zero_folio(new_mm); + } else { + add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR); + add_mm_counter(new_mm, MM_ANONPAGES, HPAGE_PMD_NR); + } } pmd = move_soft_dirty_pmd(pmd); - if (vma_has_uffd_without_event_remap(vma)) + if (cross_mm || vma_has_uffd_without_event_remap(vma)) pmd = clear_uffd_wp_pmd(pmd); - set_pmd_at(mm, new_addr, new_pmd, pmd); + set_pmd_at(new_mm, new_addr, new_pmd, pmd); if (force_flush) flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE); if (new_ptl != old_ptl) diff --git a/mm/internal.h b/mm/internal.h index d57e67a83d70..b0fed4393315 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1514,6 +1514,8 @@ extern struct list_lru shadow_nodes; } while (0) /* mremap.c */ +int move_page_range_exec(struct vm_area_struct *new_vma, + struct vm_area_struct *old_vma); unsigned long move_page_tables(struct vm_area_struct *vma, unsigned long old_addr, struct vm_area_struct *new_vma, unsigned long new_addr, unsigned long len, diff --git a/mm/mremap.c b/mm/mremap.c index 1e6d40a240c9..6f64ff7a8aaa 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -133,13 +134,59 @@ static pte_t move_soft_dirty_pte(pte_t pte) return pte; } +/* Transfer RSS without duplicating page references, rmaps or swap entries. */ +static void move_pte_account(struct vm_area_struct *vma, unsigned long addr, + struct mm_struct *new_mm, pte_t pte) +{ + struct mm_struct *mm = vma->vm_mm; + struct folio *folio; + swp_entry_t entry; + int counter; + + if (pte_present(pte)) { + folio = vm_normal_folio(vma, addr, pte); + if (!folio) { +#ifdef CONFIG_KSM + if (is_ksm_zero_pte(pte)) { + atomic_long_dec(&mm->ksm_zero_pages); + atomic_long_inc(&new_mm->ksm_zero_pages); + } +#endif + return; + } + counter = mm_counter(folio); + } else { + entry = pte_to_swp_entry(pte); + if (!non_swap_entry(entry)) { + /* swapoff must also visit the destination mm. */ + if (list_empty(&new_mm->mmlist)) { + spin_lock(&mmlist_lock); + if (list_empty(&new_mm->mmlist)) + list_add(&new_mm->mmlist, &mm->mmlist); + spin_unlock(&mmlist_lock); + } + counter = MM_SWAPENTS; + } else if (is_migration_entry(entry) || + is_device_private_entry(entry) || + is_device_exclusive_entry(entry)) { + counter = mm_counter(pfn_swap_entry_folio(entry)); + } else { + return; + } + } + dec_mm_counter(mm, counter); + inc_mm_counter(new_mm, counter); +} + static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, unsigned long old_addr, unsigned long old_end, struct vm_area_struct *new_vma, pmd_t *new_pmd, unsigned long new_addr, bool need_rmap_locks) { - bool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma); struct mm_struct *mm = vma->vm_mm; + struct mm_struct *new_mm = new_vma->vm_mm; + bool cross_mm = mm != new_mm; + bool need_clear_uffd_wp = cross_mm || vma_has_uffd_without_event_remap(vma); pte_t *old_pte, *new_pte, pte; pmd_t dummy_pmdval; spinlock_t *old_ptl, *new_ptl; @@ -184,7 +231,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, * mmap_lock, so this new_pte page is stable, so there is no need to get * pmdval and do pmd_same() check. */ - new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + new_pte = pte_offset_map_rw_nolock(new_mm, new_pmd, new_addr, &dummy_pmdval, &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); @@ -202,6 +249,8 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, continue; pte = ptep_get_and_clear(mm, old_addr, old_pte); + if (cross_mm) + move_pte_account(vma, old_addr, new_mm, pte); /* * If we are remapping a valid PTE, make sure * to flush TLB before we drop the PTL for the @@ -219,7 +268,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, pte = move_soft_dirty_pte(pte); if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) - pte_clear(mm, new_addr, new_pte); + pte_clear(new_mm, new_addr, new_pte); else { if (need_clear_uffd_wp) { if (pte_present(pte)) @@ -227,7 +276,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, else if (is_swap_pte(pte)) pte = pte_swp_clear_uffd_wp(pte); } - set_pte_at(mm, new_addr, new_pte, pte); + set_pte_at(new_mm, new_addr, new_pte, pte); } } @@ -495,6 +544,7 @@ static __always_inline unsigned long get_extent(enum pgt_entry entry, * pgt_entry. Returns true if the move was successful, else false. */ static bool move_pgt_entry(enum pgt_entry entry, struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, unsigned long new_addr, void *old_entry, void *new_entry, bool need_rmap_locks) { @@ -515,7 +565,7 @@ static bool move_pgt_entry(enum pgt_entry entry, struct vm_area_struct *vma, break; case HPAGE_PMD: moved = IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - move_huge_pmd(vma, old_addr, new_addr, old_entry, + move_huge_pmd(vma, new_vma, old_addr, new_addr, old_entry, new_entry); break; case HPAGE_PUD: @@ -596,10 +646,19 @@ unsigned long move_page_tables(struct vm_area_struct *vma, struct mmu_notifier_range range; pmd_t *old_pmd, *new_pmd; pud_t *old_pud, *new_pud; + bool cross_mm = vma->vm_mm != new_vma->vm_mm; if (!len) return 0; + /* Cross-mm moves are only used for same-address anonymous DOEXEC VMAs. */ + if (cross_mm) { + if (WARN_ON_ONCE(!vma_is_anonymous(vma) || + !(vma->vm_flags & VM_EXEC_KEEP) || + old_addr != new_addr || !need_rmap_locks)) + return 0; + } + old_end = old_addr + len; if (is_vm_hugetlb_page(vma)) @@ -610,7 +669,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma, * If possible, realign addresses to PMD boundary for faster copy. * Only realign if the mremap copying hits a PMD boundary. */ - if (len >= PMD_SIZE - (old_addr & ~PMD_MASK)) + if (!cross_mm && len >= PMD_SIZE - (old_addr & ~PMD_MASK)) try_realign_addr(&old_addr, vma, &new_addr, new_vma, PMD_MASK, for_stack); @@ -630,19 +689,23 @@ unsigned long move_page_tables(struct vm_area_struct *vma, old_pud = get_old_pud(vma->vm_mm, old_addr); if (!old_pud) continue; - new_pud = alloc_new_pud(vma->vm_mm, vma, new_addr); + new_pud = alloc_new_pud(new_vma->vm_mm, new_vma, new_addr); if (!new_pud) break; if (pud_trans_huge(*old_pud) || pud_devmap(*old_pud)) { + /* Anonymous PUD-sized THPs are not supported. */ + if (cross_mm) + break; if (extent == HPAGE_PUD_SIZE) { - move_pgt_entry(HPAGE_PUD, vma, old_addr, new_addr, + move_pgt_entry(HPAGE_PUD, vma, new_vma, old_addr, new_addr, old_pud, new_pud, need_rmap_locks); /* We ignore and continue on error? */ continue; } - } else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) { + } else if (!cross_mm && IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && + extent == PUD_SIZE) { - if (move_pgt_entry(NORMAL_PUD, vma, old_addr, new_addr, + if (move_pgt_entry(NORMAL_PUD, vma, new_vma, old_addr, new_addr, old_pud, new_pud, true)) continue; } @@ -651,24 +714,24 @@ unsigned long move_page_tables(struct vm_area_struct *vma, old_pmd = get_old_pmd(vma->vm_mm, old_addr); if (!old_pmd) continue; - new_pmd = alloc_new_pmd(vma->vm_mm, vma, new_addr); + new_pmd = alloc_new_pmd(new_vma->vm_mm, new_vma, new_addr); if (!new_pmd) break; again: if (is_swap_pmd(*old_pmd) || pmd_trans_huge(*old_pmd) || pmd_devmap(*old_pmd)) { if (extent == HPAGE_PMD_SIZE && - move_pgt_entry(HPAGE_PMD, vma, old_addr, new_addr, + move_pgt_entry(HPAGE_PMD, vma, new_vma, old_addr, new_addr, old_pmd, new_pmd, need_rmap_locks)) continue; split_huge_pmd(vma, old_pmd, old_addr); - } else if (IS_ENABLED(CONFIG_HAVE_MOVE_PMD) && + } else if (!cross_mm && IS_ENABLED(CONFIG_HAVE_MOVE_PMD) && extent == PMD_SIZE) { /* * If the extent is PMD-sized, try to speed the move by * moving at the PMD level if possible. */ - if (move_pgt_entry(NORMAL_PMD, vma, old_addr, new_addr, + if (move_pgt_entry(NORMAL_PMD, vma, new_vma, old_addr, new_addr, old_pmd, new_pmd, true)) continue; } @@ -693,6 +756,55 @@ unsigned long move_page_tables(struct vm_area_struct *vma, return len + old_addr - old_end; /* how much done */ } +struct move_page_range_args { + struct vm_area_struct *old_vma; + struct vm_area_struct *new_vma; +}; + +static int move_page_range_chunk(unsigned long addr, unsigned long end, void *arg) +{ + struct move_page_range_args *args = arg; + unsigned long len = end - addr; + + return move_page_tables(args->old_vma, addr, args->new_vma, addr, + len, true, false) == len ? 0 : -ENOMEM; +} + +/* + * The caller holds both mmap locks while workers transfer disjoint PMDs. + * A failure can leave mappings in both mms; there is no rollback. This is + * only usable after exec's point of no return, where errors are fatal. + * padata_do_multithreaded() joins all workers, including on error, before + * either mm may be torn down. + */ +int move_page_range_exec(struct vm_area_struct *new_vma, + struct vm_area_struct *old_vma) +{ + struct move_page_range_args args = { old_vma, new_vma }; +#ifdef CONFIG_PADATA + struct padata_mt_job job = { + .thread_fn = move_page_range_chunk, + .fn_arg = &args, + .start = old_vma->vm_start, + .size = old_vma->vm_end - old_vma->vm_start, + .align = PMD_SIZE, + .min_chunk = max(1ul << 27, PMD_SIZE), + .max_threads = 16, + }; +#endif + + mmap_assert_write_locked(old_vma->vm_mm); + mmap_assert_write_locked(new_vma->vm_mm); + if (WARN_ON_ONCE(!(old_vma->vm_flags & VM_EXEC_KEEP))) + return -EINVAL; + +#ifdef CONFIG_PADATA + return padata_do_multithreaded(&job); +#else + return move_page_range_chunk(old_vma->vm_start, old_vma->vm_end, &args); +#endif +} + static unsigned long move_vma(struct vm_area_struct *vma, unsigned long old_addr, unsigned long old_len, unsigned long new_len, unsigned long new_addr, -- 2.47.1 _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel