From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (mail.openvz.org [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id 5AC868017D for ; Wed, 28 Oct 2026 21:25:38 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69SLO1HH023783; Thu, 29 Oct 2026 00:24:02 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69SLO1HH023783 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="wJ8HG8fx" Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69SLNvk2023779 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Thu, 29 Oct 2026 00:23:58 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69SLNvk2023779 Received: by mail-pl1-f198.google.com with SMTP id d9443c01a7336-2ea446ea8c9so921775ad.0 for ; Wed, 28 Oct 2026 14:23:58 -0700 (PDT) X-Forwarded-Encrypted: i=3; AKwUvBzxhUNyN8ZCQ0WslhTFbH3j80XBBcuWHRF4RdC8I1foabkoCJmcx6MhFFlo/JEMdb6A28JaRg==@openvz.org X-Gm-Message-State: AFq9FYIjwfanateOPPbCjEu9dja1fmMzXjZWv8/XTVi1G/SVRr8+7DmG veJMeIs9o7gE5c8WCKvRL1SwY2OdC7RLo0BtGuIJ6BcyajDn6oucS0hrEGnGhJrM7ifPG7x6Sc1 iET2EjOwTIvRPq5yoemU4N7P76Kh9cwauTTiIaXsPkCiOMUNQR7WciQ== X-Gm-Gg: AYBFou3phndBN8ZSUAzdAmEZTz9k0PJ4GPd3eUDnK/DRiBPb1W4/0lBYL+fx9o1EkWc qwzq02zGW68yqHM44vqp+oR1HR7IaCM0wU965MkvpZ4VZ+I5XhG9vvGv7pn3qmeesMq8lz17N6B 8RX8+DKqvw5APiYM8NNTKIWN6gKE94r3/GjY6NB5TCyt2RrKyf38CgCEPMaF5t1M+eH3ndiTrKq g+YqdJLL0TSUzYyinKJisg7wENZZKQVvtXUMc42VppazbTJkCwU9tZZxXglefEvwDvG84bb6n+1 rSrjUW3kvnZWmb+Z+NbPMKk45FIqOijO9X4G733a8DDMHcJe2grUZRAz6gl3P1ojWaePNWyCbr8 I+JF7r79u0ec/OO1GWYvyWhqPBYhlnUA97N+B2qi8pShyySUNpmtnW3FdAQej1FzGSdmUYXXp63 UQjWg4m2hTluW7VbJ1KOmBqTDAhFa8WpPm0OQ= X-Received: by 2002:a17:903:38c3:b0:2df:b004:df28 with SMTP id d9443c01a7336-2eb1ab6dd70mr4493755ad.35.1793222637324; Wed, 28 Oct 2026 14:23:57 -0700 (PDT) X-Received: by 2002:a17:903:38c3:b0:2df:b004:df28 with SMTP id d9443c01a7336-2eb1ab6dd70mr4493675ad.35.1793222636778; Wed, 28 Oct 2026 14:23:56 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1793222636; cv=pass; d=google.com; s=arc-20260327; b=W0fnscylhYQqhfMjItpVoFmiHd9pmvPPERrP09NuK7prBH1F1FHebdcLEEuF7TJOBB cWbxB8G+Py6zb5Y0jQcke3VPn6YaUuyO+tAUnsgk0MDZid5Nyc3BYCz7RdYy7gW34ZGe yOxrYAgLvr9W2IzamMOvseAbrEKXdBE6YARDocS0dGhSIc34cf6YJit4hDNJ9Fo7IN+V LCPH09pdr/6qPbgLMaez2ebeqGqqQ+JXqUd5SxFOLnBGUVQP7A8Y9rrJ0At1plE7R5bm uejbkoOwG7D6crtk4lq0nKrqKrcLvGrnRszY0G3vYQJVOWqAp4JGbO7dglDIxMHSJFW9 sUuA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=mime-version:content-transfer-encoding:in-reply-to:from :content-language:references:cc:to:subject:user-agent:date :message-id:dkim-signature; bh=P5ecfsOZRdiKsgTcNqytrsmdzeDfwmjD/M43P+PDEGM=; fh=LbBPFp1Ac2EbMwN8gkdsP5Qlphxeu1M1fkqBpODMmTY=; b=M5l8umtUfv6FZLVkQvspAgJ8rY8dKZqavsGgWShsNiMA1lFtuu/qveVUPThZya/uyw XVdcGQKR/DCuym0IEY0miQD0qEzFNjDKn5NJgQI2JfLRe8bSTzIzQtzmD9KvvTho4pTc u+8unFIMQc1iTJMZkbRu4REFP/y13euS2W5J7DCFFdMihzgGeC9ZYifAsh1KD1aK4XId xcvUg/f4CIDAuk/47ENA7+PzXAXvLKEvwlNZXLpo4EQiF9kjfFRGDCmJAviQC/9RxDZr q017939VJziYDnCjgrnZO8/tppufPRtgianaaZGsODgA+CQREO6RFmTaMgJXwlFnRNdi Pp4w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector2 header.b=wJ8HG8fx; arc=pass (i=1 spf=pass spfdomain=virtuozzo.com dkim=pass dkdomain=virtuozzo.com dmarc=pass fromdomain=virtuozzo.com); spf=pass (google.com: domain of vladimir.riabchun@virtuozzo.com designates 2a01:111:f403:c20a::7 as permitted sender) smtp.mailfrom=vladimir.riabchun@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazlp170130007.outbound.protection.outlook.com. [2a01:111:f403:c20a::7]) by mx.google.com with ESMTPS id d9443c01a7336-2eae6a28b6fsi121833925ad.74.2026.10.28.14.23.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2026 14:23:56 -0700 (PDT) Received-SPF: pass (google.com: domain of vladimir.riabchun@virtuozzo.com designates 2a01:111:f403:c20a::7 as permitted sender) client-ip=2a01:111:f403:c20a::7; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector2 header.b=wJ8HG8fx; arc=pass (i=1 spf=pass spfdomain=virtuozzo.com dkim=pass dkdomain=virtuozzo.com dmarc=pass fromdomain=virtuozzo.com); spf=pass (google.com: domain of vladimir.riabchun@virtuozzo.com designates 2a01:111:f403:c20a::7 as permitted sender) smtp.mailfrom=vladimir.riabchun@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=p/Fu7eNnhazhLyO1oWOJM8R8020PTwrGePqcIUKG2tuFEbYs4uGuPREu3X+EFtKv6pDgVQy7gfnZLZMSAwTcZuQXHRXODNw1LJx8MgPjHCIgAhcyqQykzSqRvJrK0SevakCEq2YIivDkq9oZy1twhNPIJZe9uEfeYSXZOOgR/L/D4LYriun4RY0oqCNOUb6cE11Y7ZxDVMLouqIUShPd/ChyA0oJarRtpu3kshHIuRd1i3m3z9Ce37fAaFJ3EHrLHiUdISjhz/wNqn9Tokg+x352j6d2tsaQLhcOh/oSk6T3PhaT/4p+ZHT7C5AvEon80dWxXy/rpffwTyleDlKXOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P5ecfsOZRdiKsgTcNqytrsmdzeDfwmjD/M43P+PDEGM=; b=hn8+WGJm68Gb3/6Sm5J18B78Cr/YPrsdXomh7xQp7pTInSGFijQWleReU5yBq+vhZ0x4ZU2YPQcYWZsaPIPhZylZj4iK/uV098D/YgFtoFbSDZHubIhvzGUVeo3V+Ald+eDIZyZPl2I9y4ZOCHWXyzrE6lqMAHswGicadx+aDqSZKkoH0jKciO64eMvoNtJrOPEYG3cWJi2DNvMQ4LnUsW+5lJfwVX1wY+G1AdTfndfDJn9SoaiG253GK0BDCR3nKdFVw09T+D5B6udwFhvCXtlNZFx1axSk26yIgYSqvNTHLSLhWOdL4B/hq91aQqalpfuygL/qsvb4v9gOfSQZ8w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P5ecfsOZRdiKsgTcNqytrsmdzeDfwmjD/M43P+PDEGM=; b=wJ8HG8fx71d0pKrONZVKuAVkIi9gOus5B/rvgddEJdUf1f8p2n7mvkx/RNtkdidaC5b1TNBaNt1r5vYEoAsOEp9LxxsSEcgreW4zdPtDVByAzoay3iyLdJ3eAaURengqED7RuBrCgN18mD9UDV2reRAgA8AaJnuRlqY41/GbqV04/BebTyK+WwrqH7pT463x8+BP0b+ReaqysgvZpNX9hYUHVwIqX6dnFQxyv0PKwAu4aZXDCVMTDMdANtm9uqfK4AXoXM7VLJ4LUzmw7pOwdCy3DhtE0kXV2UuIAEXPe1exqZhy2StmQdttFMyCX/l8V3xXo1PosWklyUjODdOL5w== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from MI7PR08MB201474.eurprd08.prod.outlook.com (2603:10a6:290:112::17) by DU0PR08MB8448.eurprd08.prod.outlook.com (2603:10a6:10:404::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.8; Wed, 28 Oct 2026 21:23:52 +0000 Received: from MI7PR08MB201474.eurprd08.prod.outlook.com ([fe80::f35b:af88:bd82:6dfa]) by MI7PR08MB201474.eurprd08.prod.outlook.com ([fe80::f35b:af88:bd82:6dfa%3]) with mapi id 15.21.0546.021; Wed, 28 Oct 2026 21:23:52 +0000 Message-ID: <6977dd74-a5b1-4c6e-8de8-8b4df259a7c0@virtuozzo.com> Date: Wed, 28 Oct 2026 22:23:51 +0100 User-Agent: Mozilla Thunderbird To: Konstantin Khorenko , Pavel Tikhomirov References: <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-0-893494e8db76@virtuozzo.com> <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-2-893494e8db76@virtuozzo.com> Content-Language: en-US From: Vladimir Riabchun In-Reply-To: <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-2-893494e8db76@virtuozzo.com> X-ClientProxiedBy: FR0P281CA0084.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1e::23) To MI7PR08MB201474.eurprd08.prod.outlook.com (2603:10a6:290:112::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MI7PR08MB201474:EE_|DU0PR08MB8448:EE_ X-MS-Office365-Filtering-Correlation-Id: 344ca6db-71a8-4344-45b2-08df3539c3d3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|1800799024|23010399003|376014|261028144727099003|4143699003|56012099006|261009223027099003|261022075727099003|261022115727099003|261022105027099003|261021121427099003|261026140227099003|10067099003|261026121527099003|18002099003|22082099003|6133799003; X-Microsoft-Antispam-Message-Info: +r3NBaeFRcWJ0SAsQoNRlrWwdvG+lxEtxom0jEC1x2Cr0/ZWln/ZTABhO2M/bIKShQiyp+eJ+szlaxOOBlNx2/IIjDDodRkmz2B7s0Gbrj5xT4/Eb038TRIgjtMELZjnmqkNLCJ9c18yBAHWBLqPrTbDQlWrmu7/wZVXqVyPAoHQgWkA9LRF1kNVmEKQL9HHOhQ/ATtw/yveGc4SDxzBkkRjCzkEzJMWJqiGd940iL49+M89I/8i4NMSoXKiHKTSwp1I4CX5Af56WG3Caeae3uk3Kex1r4QB2zW51YX8OnOFSeKwZ7n/RviPaktx1uMlwc1bFZ+9RFYwkDF8nRcdlvgNxOxRolro/XSzV8MiGfPwsHhJ5ghGOiFaMXjSBxG0N5gcOXjI1rsXm+pu0ZmWBXRlo33fhfYZ765+iX6PyTzI+be+XDFN1wBax5bX/D1n0L3Y7HE3rxCGOh3XO2AN016BrW/xfOtPaskYQy4WjeNJKzFcZhd2OJZeJBfa9rRb0kr0Ov0/NecylZj5Vq197bes91BLS3N49Fq+l/dys+gd13oG2nK1uLwmbYMmAOKU4HCr7UzVnSst0Y957wkKU5E52EEDmES3AzHFXQSW/EY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MI7PR08MB201474.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(23010399003)(376014)(261028144727099003)(4143699003)(56012099006)(261009223027099003)(261022075727099003)(261022115727099003)(261022105027099003)(261021121427099003)(261026140227099003)(10067099003)(261026121527099003)(18002099003)(22082099003)(6133799003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WFJGMXhJVTF3N3ZlM1ZaeU5UNU4zbzNwOXNLTXNCMlQ5amtZWU52OTN6QmZD?= =?utf-8?B?K1pranpLM0lNdDlyaWg4VHB1eHFySldvTnNFZHJDbG9idUh3Z0ZRL0x1dURF?= =?utf-8?B?NGxzaENmQzB1TVFHTVV6cXFCS3VRVGNhSmtDUEZKOEtDeFZZejVxdysvK0hW?= =?utf-8?B?TitNeU45WTJzbmJCL044RjROdTVGaHhSVFFWTzFvYWJlRlpkTEkra21DK3pU?= =?utf-8?B?Mm1pb2Vnb1BLZkVDNGJzakJFd0RucU1KMVg2ZllSNW9tR1l4MGpycTNXalVH?= =?utf-8?B?UzEzQ2lOU0xSMm5kY0RET0kvRGNreTN3ZjdjajV6Sk5WWUI0Z1djaHpUS1JX?= =?utf-8?B?NHpzNWg5U2hPUFZxakhkUVlJQTdweDdpamdJQ1ZHMWFpQUhFc3cyRkZSMDJw?= =?utf-8?B?VTlra0Y1L0tzVW5kbUhHTkowUUxqS1ZwekhNTnowUVhibmoxc1BwOXlNZXdw?= =?utf-8?B?clUza3UvUW9VSi9jNWE4bGhxL0ZhWTRsK0VoNWFsM01VY2l4ZDEzUVlNamRr?= =?utf-8?B?T1N3WTRKVWExWFZsYWU3VEYwQ1ppeVRIZmt5aWdRZTRDc2h6emJ4N3RrcDAw?= =?utf-8?B?dWRMZnU1cDVMRHZLWVRCZmtBeXNmZDd2VXIxSFY4T1BzQjl4MzdkTk5TemNZ?= =?utf-8?B?ZmJzVW1sNHJNdjZXTnFRaVltejVNbE0xOTBvWU0wbXpsR0pzc041c1YvWWV2?= =?utf-8?B?dDU2TmNLSzlJM0dCNEZLR2RQQnIxRERxa3Z4dllaQjBPZlVSV1p1M0RPbjBY?= =?utf-8?B?YjRLM3FaQUZIdE1NbThTb3MzQWpMWktqWkZEVGxoWVhVWUtuT2IyYjYrb0RD?= =?utf-8?B?dHNRcmFzd3RjTFlDVlVKNDZYbnk5bFZBOWZJOXF4YWZrQ3lKSVJSVE9mdnZY?= =?utf-8?B?V0VpUERzZU1nNGEwdnNGN245SVdMT0ZkQTBWQTlma3huNStvVnJtcXdLa3pI?= =?utf-8?B?YWRyN2JGL0tiZXJFT0twRExhdW1GWncybkdLbDFHcVQwMHJqdGwwcVRuRjdE?= =?utf-8?B?UE5GenlNTytSUUVsQUEwSXNxbDFiUkJLelNpUFl6QXVCUmcyZWZQMkJYM3FF?= =?utf-8?B?ZWJ5eUhkbjZLRlcyR3JmTnpoaUlGOEo4Tm84Vjh3NE9hVHR4UTFCRisyd2J1?= =?utf-8?B?K3lNMjhxa0psalp3M0k2Sk5URUsxTGxKdHMzQU5DL0pVd0pOeW9ZSDZvTnJW?= =?utf-8?B?WkkzWnNQcm5tdlJoVVI0bjZNR2hXZUhGTWF3WDhaVlRNZkFGeWtPN25YRkda?= =?utf-8?B?TGl2OE5mQjZjaXRqVkJEc1ppakx1SjdkZUhwd2xWL3pCSFZYdUxUS3MvRTRD?= =?utf-8?B?cHRqbCtTY2k3WDdITWdGSmJRMDNQRUo4c2piVlpBMkl6UktXWTQxemFKditt?= =?utf-8?B?Y0QrZU5IYnhwQU5wOE1mWjg3MGxxRTZmNDZYN1BjSEhpc003emRQWXYrYTBm?= =?utf-8?B?Q3h3Mi9MRTlFZEhBb3Bmc1BXL2h2N2hBYVN3Z1hqb2Y3MGI1dnhnaEVwcmd4?= =?utf-8?B?T0JMNzlZQmpFVFRveGdUN2ZjZ1d2cGw5QmFJV0ZHYzI0eHFUUmtyQys3SEd2?= =?utf-8?B?TUdkcVdsTC96VUtYa3VaYjZFR295NzNXQU9YSk9HVWtIeFpSd1YzbDI5M0Iy?= =?utf-8?B?MGQvejA4dmEvUEVQY05UWWdScTBIeERJMjZZd3pCdFZ2M3F1QmZHcU5NbTRh?= =?utf-8?B?K3hWZDQ5andPaFNNSGtzVDI2akUvMHNoWHZLdC8rUi9DaUlCUGVBOGR1WXkr?= =?utf-8?B?L3FGV29YTDhYUTJ2ZGNlOE1ub0cyWWovY3luYThVcDNPNnN3OFg0eGtqVnhN?= =?utf-8?B?ME1PNGRVK3o4Z0JzeWEwYXBwend1dzJpWGlyc0hqY3gwRHdoMG9XL05IRnEz?= =?utf-8?B?UVJSdU9USnYrMHhKcFYyZk5XbkFyOFFoK016Tk9HMG8wTWNVOTlvVGRoalB2?= =?utf-8?B?S1ZWSTNBZHdvTGtOOVgvemk2eXFKdER4bDZXTWNTWXZNdERwTFFoMkpqdS9J?= =?utf-8?B?ZkJsaGJ4dTV6bFNSN1Nyb0Z6a1piSVlIL0IyWUdmM0ZBbys5V1VXSk50WjVi?= =?utf-8?B?aTdNeXFkZGVNMVRUa0NWWVN1bEQ1d3p3Mk8wNFpiYVh0UzlWOURlYUNkTEht?= =?utf-8?B?a05SemNuNGFYRlA1a0JqN3piams5czBybFpJdC9WVUtzSXhxWFhIZjB6MW90?= =?utf-8?B?d0RONk9WSysrc0lhOFlmRDY3Z2s0UWF2OTNTc1Vhc2w3aHBOb0dSamEzYXpz?= =?utf-8?B?bU54YXlHYXc4QXlPbHhib0tCMDdCeWNORnVHOWUrbGZ1R3lHY1JqTVRHek0r?= =?utf-8?B?Q3QwUlIralVURksyeEJkQTM0ZmhuYURFNnJkREJlTDF4MzRhcHBkUTBRQVh1?= =?utf-8?Q?D9VU3ULPpM7k4ABk=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 344ca6db-71a8-4344-45b2-08df3539c3d3 X-MS-Exchange-CrossTenant-AuthSource: MI7PR08MB201474.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2026 21:23:52.5248 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: z9PwkiXaR3x6BT5b74R3aJ12Foat6B15pKauMVUILO7ITZkNjKH+jRiQse660huZGGdh7lMQXTS+qPCX+JSg+E80KKwx5jjmXz42bipszW0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8448 X-OZ-Fwd: true Cc: devel@openvz.org Subject: Re: [Devel] [PATCH hci-8.1 v3 2/2] selftests/ve: check memory.numa_migrate keeps the folio order X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org On 10/28/26 18:11, Konstantin Khorenko wrote: > memcg_numa_migrate_new_folio() used to give every large folio a > PMD-sized destination. A smaller page cache folio then read back as > garbage, and its inode died in clear_inode() on eviction. > > The test writes a file on an XFS image in 64K chunks, which gives > order-4 folios, from a cgroup bound to one NUMA node. It migrates the > cgroup to a second node with memory.numa_migrate and compares the file > with the written pattern. > > Two checks make sure the test reaches the bug: > > - before the migration the page cache must hold a large folio below > the PMD size, read through /proc/self/pagemap and /proc/kpageflags; > - after it at least one such folio must sit on the target node in one > piece, its node read through move_pages(). > > A failed check is a failure, not a skip, so that a setup where the > test stops reaching the bug shows up. The chunks at 2 MB aligned > offsets are not written: a folio there reads right even from an > oversized destination. > > On an unfixed kernel the comparison fails and the test leaves the > image mounted, because an eviction would crash the node. > > Needs root, two NUMA nodes with memory (numa=fake=2 is enough), > cgroup2 with the memory and cpuset controllers, and mkfs.xfs; skips > otherwise. > > https://virtuozzo.atlassian.net/browse/VSTOR-151607 > > Feature: ve selftests > Signed-off-by: Konstantin Khorenko > --- > tools/testing/selftests/ve/Makefile | 1 + > tools/testing/selftests/ve/ve_numa_migrate_test.c | 494 ++++++++++++++++++++++ Please, update gitignore as well with test executable. > 2 files changed, 495 insertions(+) > > diff --git a/tools/testing/selftests/ve/Makefile b/tools/testing/selftests/ve/Makefile > index 51f9316beb99b..e96320a183844 100644 > --- a/tools/testing/selftests/ve/Makefile > +++ b/tools/testing/selftests/ve/Makefile > @@ -6,6 +6,7 @@ TEST_GEN_PROGS += ve_ns_owner_test > TEST_GEN_PROGS += ve_perms_test > TEST_GEN_PROGS += ve_mount_accounting_test > TEST_GEN_PROGS += ve_vtty_stress > +TEST_GEN_PROGS += ve_numa_migrate_test > > # CPUID faulting is x86-64 only. ARCH says "x86" for 32-bit too and lib.mk > # picks the clang target by its emptiness, so decide by the compiler. > diff --git a/tools/testing/selftests/ve/ve_numa_migrate_test.c b/tools/testing/selftests/ve/ve_numa_migrate_test.c > new file mode 100644 > index 0000000000000..2c6499195b9ba > --- /dev/null > +++ b/tools/testing/selftests/ve/ve_numa_migrate_test.c > @@ -0,0 +1,494 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * memory.numa_migrate must give a large page cache folio a destination of > + * the same order. An oversized destination inherits the index and the > + * reference count of the source, only the source pages are copied into it, > + * a read of the folio is served from the destination by file offset and > + * lands in its uninitialised part, and the inode dies in clear_inode() on > + * eviction. > + * > + * Write a file on an XFS image (XFS allocates large folios for 64K writes) > + * from a cgroup bound to one NUMA node, migrate the cgroup to another node > + * and check that the data reads back intact. > + * > + * Needs root, two NUMA nodes with memory (numa=fake=2 will do), cgroup2 > + * with the memory and cpuset controllers, and mkfs.xfs. The folio sizes > + * are read through /proc/self/pagemap and /proc/kpageflags before the > + * migration, and the node of every page through move_pages() after it: a > + * file without large folios below the PMD size, or a migration that > + * moved none of them in one piece, fails the test, as it would pass > + * without touching the bug. > + */ > +#define _GNU_SOURCE > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "../kselftest_harness.h" > +#include "ve_selftest.h" > + > +#define CG_NAME "ve_numa_migrate_test" > +#define DIR_TEMPLATE "/var/tmp/ve_numa_migrate.XXXXXX" > +/* mkfs.xfs refuses anything smaller; the image is sparse. */ > +#define IMG_SIZE (320UL << 20) > +#define FILE_SIZE (4UL << 20) > +#define CHUNK_SIZE (64UL << 10) > +#define PM_PFN_MASK ((1ULL << 55) - 1) > +#ifndef MADV_POPULATE_READ > +#define MADV_POPULATE_READ 22 > +#endif > +#define PM_PRESENT (1ULL << 63) > +/* The periodic memcg stats flush runs every two seconds. */ > +#define STAT_RETRIES 50 > +#define STAT_RETRY_US 100000 > + > +FIXTURE(numa_migrate) > +{ > + int cgv2_fd; > + int cg_fd; > + int node_from; > + int node_to; > + char dir[64]; > + char img[80]; > + char mnt[80]; > + int mounted; > + int leave_mounted; > + int data_fd; > +}; > + > +/* First two node ids of a sysfs node list such as "0-1" or "0,2-3". */ > +static int two_nodes_with_memory(int *from, int *to) > +{ > + char buf[256], *tok, *save; > + int found = 0; > + > + if (read_file_at(AT_FDCWD, "/sys/devices/system/node/has_memory", > + buf, sizeof(buf)) <= 0) > + return -1; > + > + for (tok = strtok_r(buf, ",\n", &save); tok && found < 2; > + tok = strtok_r(NULL, ",\n", &save)) { > + int a, b, n; > + > + if (sscanf(tok, "%d-%d", &a, &b) != 2) { > + if (sscanf(tok, "%d", &a) != 1) > + return -1; > + b = a; > + } > + for (n = a; n <= b && found < 2; n++) { > + if (found == 0) > + *from = n; > + else > + *to = n; > + found++; > + } > + } > + return found == 2 ? 0 : -1; > +} > + > +static int run_cmd(const char *fmt, ...) > +{ > + char cmd[256]; > + va_list ap; > + int status; > + > + va_start(ap, fmt); > + vsnprintf(cmd, sizeof(cmd), fmt, ap); > + va_end(ap); > + > + status = system(cmd); > + if (status < 0 || !WIFEXITED(status)) > + return -1; > + return WEXITSTATUS(status); > +} > + > +/* Bytes of file pages the cgroup has on @node, from memory.numa_stat. */ > +static int file_bytes_on_node(int cg_fd, int node, unsigned long long *out) > +{ > + static char buf[32768]; > + char key[16], *p; > + > + if (read_file_at(cg_fd, "memory.numa_stat", buf, sizeof(buf)) <= 0) > + return -1; > + p = strstr(buf, "\nfile "); > + if (!p) > + return -1; > + snprintf(key, sizeof(key), " N%d=", node); > + p = strstr(p, key); > + if (!p) > + return -1; > + *out = strtoull(p + strlen(key), NULL, 10); > + return 0; > +} > + > +static unsigned long page_size; > + > +/* A folio at a PMD-aligned index reads right from an oversized one. */ > +static bool chunk_is_hole(unsigned long off, unsigned long pmd_size) > +{ > + return !((off / CHUNK_SIZE * CHUNK_SIZE) % pmd_size); > +} > + > +#define SCAN_ORDERS 16 > + > +struct folio_scan { > + long large; /* pages in large folios */ > + long smallest; /* pages in the smallest large folio */ > + long on_node; /* large folios below @pmd_pages that sit on @node */ > + long hist[2][SCAN_ORDERS]; /* folios by order, on @node / elsewhere */ > +}; > + > +static void folio_scan_end_run(struct folio_scan *fs, long run, int run_node, > + int node, long pmd_pages) > +{ > + int order = 0; > + > + if (!run) > + return; > + while ((1L << order) < run && order < SCAN_ORDERS - 1) > + order++; > + fs->hist[run_node != node][order]++; > + if (run == 1) > + return; > + if (!fs->smallest || run < fs->smallest) > + fs->smallest = run; > + if (run < pmd_pages && run_node == node) > + fs->on_node++; > +} > + > +/* "64 x 64K, 3 x 4K" or "none" */ > +static const char *folio_hist_str(const long *hist, char *buf, size_t len) > +{ > + size_t pos = 0; > + int order; > + > + for (order = SCAN_ORDERS - 1; order >= 0; order--) { > + if (!hist[order]) > + continue; > + pos += snprintf(buf + pos, len - pos, "%s%ld x %luK", > + pos ? ", " : "", hist[order], > + (page_size << order) / 1024); > + if (pos >= len) > + break; > + } > + return pos ? buf : "none"; > +} > + > +/* > + * Walk the page cache of @fd through /proc/self/pagemap, /proc/kpageflags > + * and move_pages(2) in query mode, leaving the holes alone: faulting them > + * in would add folios to the file. Returns -1 when that is not possible. > + */ > +static int scan_file_folios(int fd, unsigned long size, int node, > + unsigned long pmd_size, struct folio_scan *fs) > +{ > + unsigned long nr = size / page_size, i, off; > + long pmd_pages = pmd_size / page_size; > + long run = 0; > + int run_node = -1, ret = -1; > + int pagemap, kpageflags, *status; > + unsigned char *map; > + void **pages; > + uint64_t *pm; > + > + memset(fs, 0, sizeof(*fs)); > + pagemap = open("/proc/self/pagemap", O_RDONLY); > + kpageflags = open("/proc/kpageflags", O_RDONLY); > + pm = malloc(nr * sizeof(*pm)); > + pages = malloc(nr * sizeof(*pages)); > + status = malloc(nr * sizeof(*status)); > + map = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0); > + if (pagemap < 0 || kpageflags < 0 || !pm || !pages || !status || > + map == MAP_FAILED) > + goto out; > + for (off = 0; off < size; off += CHUNK_SIZE) { > + if (!chunk_is_hole(off, pmd_size) && > + madvise(map + off, CHUNK_SIZE, MADV_POPULATE_READ)) > + goto out; > + } > + if (pread(pagemap, pm, nr * sizeof(*pm), > + (unsigned long)map / page_size * sizeof(*pm)) != > + (ssize_t)(nr * sizeof(*pm))) > + goto out; > + for (i = 0; i < nr; i++) > + pages[i] = map + i * page_size; > + if (syscall(__NR_move_pages, 0, nr, pages, NULL, status, 0)) > + goto out; > + for (i = 0; i < nr; i++) { > + uint64_t flags, pfn = pm[i] & PM_PFN_MASK; > + > + if (chunk_is_hole(i * page_size, pmd_size)) { > + folio_scan_end_run(fs, run, run_node, node, pmd_pages); > + run = 0; > + continue; > + } > + if (!(pm[i] & PM_PRESENT) || !pfn || status[i] < 0 || > + pread(kpageflags, &flags, sizeof(flags), > + pfn * sizeof(flags)) != sizeof(flags)) > + goto out; > + if (flags & (1ULL << KPF_COMPOUND_HEAD)) { > + folio_scan_end_run(fs, run, run_node, node, pmd_pages); > + fs->large++; > + run = 1; > + run_node = status[i]; > + } else if (flags & (1ULL << KPF_COMPOUND_TAIL)) { > + fs->large++; > + run++; > + } else { > + folio_scan_end_run(fs, run, run_node, node, pmd_pages); > + folio_scan_end_run(fs, 1, status[i], node, pmd_pages); > + run = 0; > + } > + } > + folio_scan_end_run(fs, run, run_node, node, pmd_pages); > + ret = 0; > +out: > + if (map != MAP_FAILED) > + munmap(map, size); > + free(status); > + free(pages); > + free(pm); > + if (pagemap >= 0) > + close(pagemap); > + if (kpageflags >= 0) > + close(kpageflags); > + return ret; > +} > + > +static void cleanup(FIXTURE_DATA(numa_migrate) *self) > +{ > + char path[PATH_MAX]; > + > + if (self->data_fd >= 0) { > + close(self->data_fd); > + self->data_fd = -1; > + } > + if (self->dir[0]) { > + snprintf(path, sizeof(path), "%s/data", self->mnt); > + unlink(path); > + if (self->mounted) > + umount(self->mnt); > + unlink(self->img); > + rmdir(self->mnt); > + rmdir(self->dir); > + self->dir[0] = '\0'; > + } > + if (self->cg_fd >= 0) { > + close(self->cg_fd); > + self->cg_fd = -1; > + } > + if (self->cgv2_fd >= 0) { > + /* The test task may still sit in the cgroup after a failure. */ > + write_u64_at(self->cgv2_fd, "cgroup.procs", getpid()); > + unlinkat(self->cgv2_fd, CG_NAME, AT_REMOVEDIR); > + close(self->cgv2_fd); > + self->cgv2_fd = -1; > + } > +} > + > +/* > + * Returns 0, or 1 with everything undone when the host cannot run the test. > + * The harness runs no teardown after a failed setup, so the undo is here. > + */ > +static int prepare(FIXTURE_DATA(numa_migrate) *self, const char **why) > +{ > + int fd; > + > + self->cgv2_fd = mount_cg2_fd(); > + if (self->cgv2_fd < 0) { > + *why = "cannot mount cgroup2"; > + goto fail; > + } > + if (write_file_at(self->cgv2_fd, "cgroup.subtree_control", > + "+memory +cpuset")) { > + *why = "cannot enable the memory and cpuset controllers"; > + goto fail; > + } > + if (mkdirat(self->cgv2_fd, CG_NAME, 0755) && errno != EEXIST) { > + *why = "cannot create the test cgroup"; > + goto fail; > + } > + self->cg_fd = openat(self->cgv2_fd, CG_NAME, O_PATH | O_DIRECTORY); > + if (self->cg_fd < 0) { > + *why = "cannot open the test cgroup"; > + goto fail; > + } > + if (faccessat(self->cg_fd, "memory.numa_migrate", W_OK, 0)) { > + *why = "no memory.numa_migrate"; > + goto fail; > + } > + if (write_u64_at(self->cg_fd, "cpuset.mems", self->node_from)) { > + *why = "cannot bind the cgroup to a node"; > + goto fail; > + } > + > + snprintf(self->dir, sizeof(self->dir), DIR_TEMPLATE); > + if (!mkdtemp(self->dir)) { > + self->dir[0] = '\0'; > + *why = "cannot create a temporary directory"; > + goto fail; > + } > + snprintf(self->img, sizeof(self->img), "%s/img", self->dir); > + snprintf(self->mnt, sizeof(self->mnt), "%s/mnt", self->dir); > + if (mkdir(self->mnt, 0755)) { > + *why = "cannot create the mount point"; > + goto fail; > + } > + fd = open(self->img, O_CREAT | O_RDWR, 0600); > + if (fd < 0) { > + *why = "cannot create the image file"; > + goto fail; > + } > + if (ftruncate(fd, IMG_SIZE)) { > + close(fd); > + *why = "cannot size the image file"; > + goto fail; > + } > + close(fd); > + if (run_cmd("mkfs.xfs -q -f %s", self->img)) { > + *why = "mkfs.xfs failed"; > + goto fail; > + } > + if (run_cmd("mount -o loop %s %s", self->img, self->mnt)) { > + *why = "cannot loop-mount the XFS image"; > + goto fail; > + } > + self->mounted = 1; > + return 0; > +fail: > + cleanup(self); > + return 1; > +} > + > +FIXTURE_SETUP(numa_migrate) > +{ > + const char *why; > + > + self->cgv2_fd = -1; > + self->cg_fd = -1; > + self->data_fd = -1; > + self->dir[0] = '\0'; > + page_size = sysconf(_SC_PAGESIZE); > + > + if (geteuid()) > + SKIP(return, "needs root"); > + if (two_nodes_with_memory(&self->node_from, &self->node_to)) > + SKIP(return, "needs two NUMA nodes with memory"); > + if (prepare(self, &why)) > + SKIP(return, "%s", why); > +} > + > +FIXTURE_TEARDOWN(numa_migrate) > +{ > + if (self->leave_mounted) { > + TH_LOG("leaving %s mounted, eviction would hit BUG_ON(nrpages)", > + self->mnt); > + self->dir[0] = '\0'; > + } > + cleanup(self); > +} > + > +TEST_F(numa_migrate, large_folio_order) > +{ > + unsigned long long on_to_before, on_to_after; > + unsigned char *buf, *check; > + char path[PATH_MAX], list[64]; > + unsigned long long pmd_size; > + struct folio_scan fs; > + char h0[128], h1[128]; > + long pmd_pages; > + unsigned long off; > + int fd, i; > + > + ASSERT_EQ(write_u64_at(self->cg_fd, "cgroup.procs", getpid()), 0); > + > + buf = malloc(CHUNK_SIZE); > + check = malloc(CHUNK_SIZE); > + ASSERT_NE(buf, NULL); > + ASSERT_NE(check, NULL); > + > + /* The old code allocated PMD-sized destinations: those were fine. */ > + if (read_u64_at(AT_FDCWD, > + "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", > + &pmd_size)) > + pmd_size = 2UL << 20; > + pmd_pages = pmd_size / page_size; > + > + snprintf(path, sizeof(path), "%s/data", self->mnt); > + fd = open(path, O_CREAT | O_RDWR | O_TRUNC, 0600); > + ASSERT_GE(fd, 0); > + self->data_fd = fd; > + for (off = 0; off < FILE_SIZE; off += CHUNK_SIZE) { > + if (chunk_is_hole(off, pmd_size)) > + continue; > + memset(buf, (off / CHUNK_SIZE) & 0xff, CHUNK_SIZE); > + ASSERT_EQ(pwrite(fd, buf, CHUNK_SIZE, off), (ssize_t)CHUNK_SIZE); > + } > + ASSERT_EQ(fsync(fd), 0); > + ASSERT_EQ(scan_file_folios(fd, FILE_SIZE, self->node_to, pmd_size, &fs), 0) > + TH_LOG("cannot read the page cache folio sizes"); > + TH_LOG("before: on node %d: %s, elsewhere: %s", self->node_to, > + folio_hist_str(fs.hist[0], h0, sizeof(h0)), > + folio_hist_str(fs.hist[1], h1, sizeof(h1))); > + ASSERT_GT(fs.large, 0) > + TH_LOG("no large folio in the page cache, the bug is out of reach"); > + ASSERT_LT(fs.smallest, pmd_pages) > + TH_LOG("only PMD-sized folios in the page cache, the bug is out of reach"); > + > + ASSERT_EQ(file_bytes_on_node(self->cg_fd, self->node_to, > + &on_to_before), 0); > + > + /* From here on a failure means eviction would crash the kernel. */ > + self->leave_mounted = 1; > + snprintf(list, sizeof(list), "%d,%d", self->node_from, self->node_to); > + ASSERT_EQ(write_file_at(self->cg_fd, "cpuset.mems", list), 0); > + ASSERT_EQ(write_u64_at(self->cg_fd, "memory.numa_migrate", > + self->node_to), 0); > + > + for (off = 0; off < FILE_SIZE; off += CHUNK_SIZE) { > + if (chunk_is_hole(off, pmd_size)) > + continue; > + memset(buf, (off / CHUNK_SIZE) & 0xff, CHUNK_SIZE); > + ASSERT_EQ(pread(fd, check, CHUNK_SIZE, off), (ssize_t)CHUNK_SIZE); > + ASSERT_EQ(memcmp(buf, check, CHUNK_SIZE), 0); > + } > + > + /* memory.numa_stat flushes per-cpu counters lazily, give it time. */ > + for (i = 0; i < STAT_RETRIES; i++) { > + ASSERT_EQ(file_bytes_on_node(self->cg_fd, self->node_to, > + &on_to_after), 0); > + if (on_to_after >= on_to_before + FILE_SIZE) > + break; > + usleep(STAT_RETRY_US); > + } > + TH_LOG("file bytes on node %d: %llu -> %llu", self->node_to, > + on_to_before, on_to_after); > + > + ASSERT_EQ(scan_file_folios(fd, FILE_SIZE, self->node_to, pmd_size, &fs), 0) > + TH_LOG("cannot read the page cache folio sizes"); > + TH_LOG("after: on node %d: %s, elsewhere: %s", self->node_to, > + folio_hist_str(fs.hist[0], h0, sizeof(h0)), > + folio_hist_str(fs.hist[1], h1, sizeof(h1))); > + ASSERT_GT(fs.on_node, 0) > + TH_LOG("no folio below the PMD size reached node %d in one piece", > + self->node_to); > + self->leave_mounted = 0; > + > + free(buf); > + free(check); > +} > + > +TEST_HARNESS_MAIN > -- Best regards, Riabchun Vladimir Linux Kernel Developer, Virtuozzo _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel