From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (mail.openvz.org [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id B54DE8045D for ; Wed, 28 Oct 2026 17:12:30 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69SHBEGt022263; Wed, 28 Oct 2026 20:11:15 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69SHBEGt022263 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="FztAK9Uk" Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 69SHB77V022250 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Wed, 28 Oct 2026 20:11:07 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 69SHB77V022250 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4a028616bebso8991035e9.1 for ; Wed, 28 Oct 2026 10:11:07 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzaic9CBEgUi0t3F0wjyyEah0y/qdg6ynFwcepzQfaRCrSVdOxPLAwz4xHZgVylk47AAaUrRQ==@openvz.org X-Gm-Message-State: AFq9FYK86Dni+B3eF32J5DCdaYmk3VaSTA57fFn6VY9KA3rWx36PyhsQ uGUO6JfkMY8Tr2XYAq2lGWXm7IwIAopdg7WDytd4I9FL/74vWnmpwt3xkFdY55uC72dOpXVeZqX DJsrd56YQsj24XlxXN9Cu+C2kZl8CBwZBXa32zOqmMWw1qVswY9lpig== X-Gm-Gg: AYBFou2MM0SPFeuHTPffdfMYIyXinRK5R+tejnNTLCF1X41l89wg3reWiiOc5J7ZEWH YyqSY+9ugrRe/TUlV5BLz57PZDoo5GSmtZuVUmxaHoMkzlWRvQCyn+tky8ZuLwev5Wfy0q/jP3a NY320GrvxlsooL6mQs/rcrpTTFi2byJAqytEMIlPWs7m3W2TVsw3waEfDHNw67lJT6odb2ONAlj Q/6hXAECDMIxSQ7obww0zLah/MnUwzlB1yGeJe7TVebmecskQdNXphbWrl8F48X9LMVJlUZzW2p q65eCltmGCUvwmdPj0y6iFbLTDik70YaMrqJZijqc/kwwB/5l3oi4rzqXTqWYfJIzxGPMlt4PWz RCUkWBQ6qzT7DCAjz8g== X-Received: by 2002:a05:600c:524c:b0:4a2:86b7:b76e with SMTP id 5b1f17b1804b1-4a9661ab517mr62752905e9.13.1793207466650; Wed, 28 Oct 2026 10:11:06 -0700 (PDT) X-Received: by 2002:a05:600c:524c:b0:4a2:86b7:b76e with SMTP id 5b1f17b1804b1-4a9661ab517mr62751915e9.13.1793207465803; Wed, 28 Oct 2026 10:11:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1793207465; cv=none; d=google.com; s=arc-20260327; b=sB/DzehaGn6EnNMdPHYQICZ7vcJPwKRHdxod4hZ0D3MMUI29VWwV0t47kDFbOhvFRo 3mEUlWigsRhtR7TJVKA56g2Q2R4m76K8SXgOpP+2A1GOouTbdTxXKF6kR/O+auGMZdiW oRpAuDWz692lC0jNBGCA1a/6uqhjWU3kMV4gg1d3j7NC+cGZyi/hejRZTqyONuWHfBJ9 4lFntFKGLeDeIkfHLxm396j/STwgrnoabE/Vvcy23jmU77Tv8TOxC8QJkvdQb25rCI5U KGnc4dgmPUc3YJAvFj04/N3A0nfeX9DWFVphGPrYRDDP8U6pCuOYDyPY086uKyZapCZS KVww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:dkim-signature; bh=khMdiLYSzKH4j3Wk3lEFAJekORG1o7TEny/mG5QFfME=; fh=XcqC0Cib/C4XsQyLUc3b6E9Ii37Wk0LS3yBNHj5szHI=; b=PoPjnWbguPqsZ/ZasrvBP/uEDVII9tgWzOTi2YKpswnhssd7mHOLSEyot0vK2K33iA 1/NIeXnwHEw4CzZOafmYOtJhckl459gfFrDQTejk5rlrMumoQncGbzfZ1MIFV2K1KLHt 5ZExiktxZgI4EBv1OQuFl6bP2zD7vQvlZNc+v7Jkyth3DC7QiiEs4iGjS2Kqg0iv5FKD +Ua3Z7N/vQyv85H+APVDezRDR8H/Nb0IggLhUwtqDPZxBPu921G71I5OMLzlog6DK0/C XTzMQ7LbvNkqNoIknJASUZ+DH7lm7/GZBZOITig7BMFZaGjC/NGJNEZfZ/6wcUn+uHJo CamQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=FztAK9Uk; spf=pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=khorenko@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from relay.virtuozzo.com (relay.virtuozzo.com. [130.117.225.111]) by mx.google.com with ESMTPS id ffacd0b85a97d-48e3eb9c5b4si25055774f8f.43.2026.10.28.10.11.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2026 10:11:05 -0700 (PDT) Received-SPF: pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) client-ip=130.117.225.111; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=FztAK9Uk; spf=pass (google.com: domain of khorenko@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=khorenko@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=Message-Id:Content-Type:MIME-Version:Subject:Date :From; bh=khMdiLYSzKH4j3Wk3lEFAJekORG1o7TEny/mG5QFfME=; b=FztAK9UkpzEgIDCcdwF QZWqo6AwPuuz5McMHxV1YgZFFtzjKrwX0DHIDpjPwRpaW6LwGXNIigWGgK7BJ9ri7O1d81jXjMqG2 mDl0paj97VEmdF4QqGVt5uNgXF7t0lxYaeeSXxXnjO46U5MeFDBD0/dOfo/9ahvvwewQj5NwC7odJ tl00D6z5pr3z51lhyQiz7VF1YZQ+HqibE7A1QSOUepw4tpPY47HvJG1etTpt7eMZNU2kfrnpQXtCI c3X16+oEyGfPtKBgRo7PP3tlkSM04H/0J4jMEGaHjQ58y41IdtwJotxmYgKczDH4+Iht7y0tDqomv k3ZzDIL9hOk41Kw==; Received: from ch-demo-asa.virtuozzo.com ([130.117.225.8] helo=f0.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1xM77G-00Dgbl-2h; Wed, 28 Oct 2026 18:11:04 +0100 From: Konstantin Khorenko Date: Wed, 28 Oct 2026 18:11:01 +0100 MIME-Version: 1.0 Message-Id: <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-2-893494e8db76@virtuozzo.com> References: <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-0-893494e8db76@virtuozzo.com> In-Reply-To: <20261028-b4-prep-khorenko-vstor-151607-numa-migrate-order-v3-0-893494e8db76@virtuozzo.com> To: Pavel Tikhomirov X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17104; i=khorenko@virtuozzo.com; h=from:subject:message-id; bh=MUqXnmetLpAPUyuWgRCz7Ogn6jFcwqlV7mq5QNbdQZo=; b=owEBbQKS/ZANAwAKAVGWCkf5YVwqAcsmYgBq4iyo56oH/dXOM3VXekzE0y3T9UtWonpHr3mA5 hUus25oVFeJAjMEAAEKAB0WIQRWD23IPcXT0GO9CCxRlgpH+WFcKgUCauIsqAAKCRBRlgpH+WFc KpccD/wKdY/WD6YofEdaSeoM1Knv/opqYTTg52GA5pRq8/CaM+TZvLqlhy4ZAA4ar13immJsugF RzdSo+ijhe8XyHpkbsVqr8yEElwnai5cm+srw3wAhMH9nMVG942TuWsDQwzbMAPV/K8csuouWu9 CZqa3Oq3jmiW+iD4D9TKSheoHdfHzF+lzhwRmTBLYKwEkm7HvTn/u+OdZvuD8rToIGPOjJeWcbP eEYVQGPT2lDG1Hk4+Oh0G1rZtS7fs4kbuf7nh8u1I36mKW4y3iURrvi7d6zkAdQgCAKEh6riYqO gX6SArmrOTAtJ3pZFclcYY+LOp/OHC1GerNqQZyiLDZgSXaW/lVyNIh4EuyBXZx2We6TvtRQ1av swq4HikLPhcIIgcPRgdKYD7vYcRSH/7IlRK0y3mUkP9JrWnw/Ba5Iymz3HqWxwEVmtbpvYTP8+l OoVf3Qma9VUvfXa2sNopuDYvEuP6NktL3MjsO8wNqp1N8i71hkHiO9ioTiEiBqjksplAPyd+Ubr eYVi/5ct/k/BCbNvcmNUrBLJfKqIRYSq3uWwjQOX9/zch4Eip3d16yCpivJ5Si89s1Y/fJozH+n eOOPlWFjY0W7CbVh2vUVMCZKWsXyo/aDk0j2aYQirGZkaiMZKNYrqFPb8yKFDWEEIisGQz0gh81 yzVJIqwi0tHwMyg== X-Developer-Key: i=khorenko@virtuozzo.com; a=openpgp; fpr=560F6DC83DC5D3D063BD082C51960A47F9615C2A X-OZ-Fwd: true Cc: devel@openvz.org Subject: [Devel] [PATCH hci-8.1 v3 2/2] selftests/ve: check memory.numa_migrate keeps the folio order X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org memcg_numa_migrate_new_folio() used to give every large folio a PMD-sized destination. A smaller page cache folio then read back as garbage, and its inode died in clear_inode() on eviction. The test writes a file on an XFS image in 64K chunks, which gives order-4 folios, from a cgroup bound to one NUMA node. It migrates the cgroup to a second node with memory.numa_migrate and compares the file with the written pattern. Two checks make sure the test reaches the bug: - before the migration the page cache must hold a large folio below the PMD size, read through /proc/self/pagemap and /proc/kpageflags; - after it at least one such folio must sit on the target node in one piece, its node read through move_pages(). A failed check is a failure, not a skip, so that a setup where the test stops reaching the bug shows up. The chunks at 2 MB aligned offsets are not written: a folio there reads right even from an oversized destination. On an unfixed kernel the comparison fails and the test leaves the image mounted, because an eviction would crash the node. Needs root, two NUMA nodes with memory (numa=fake=2 is enough), cgroup2 with the memory and cpuset controllers, and mkfs.xfs; skips otherwise. https://virtuozzo.atlassian.net/browse/VSTOR-151607 Feature: ve selftests Signed-off-by: Konstantin Khorenko --- tools/testing/selftests/ve/Makefile | 1 + tools/testing/selftests/ve/ve_numa_migrate_test.c | 494 ++++++++++++++++++++++ 2 files changed, 495 insertions(+) diff --git a/tools/testing/selftests/ve/Makefile b/tools/testing/selftests/ve/Makefile index 51f9316beb99b..e96320a183844 100644 --- a/tools/testing/selftests/ve/Makefile +++ b/tools/testing/selftests/ve/Makefile @@ -6,6 +6,7 @@ TEST_GEN_PROGS += ve_ns_owner_test TEST_GEN_PROGS += ve_perms_test TEST_GEN_PROGS += ve_mount_accounting_test TEST_GEN_PROGS += ve_vtty_stress +TEST_GEN_PROGS += ve_numa_migrate_test # CPUID faulting is x86-64 only. ARCH says "x86" for 32-bit too and lib.mk # picks the clang target by its emptiness, so decide by the compiler. diff --git a/tools/testing/selftests/ve/ve_numa_migrate_test.c b/tools/testing/selftests/ve/ve_numa_migrate_test.c new file mode 100644 index 0000000000000..2c6499195b9ba --- /dev/null +++ b/tools/testing/selftests/ve/ve_numa_migrate_test.c @@ -0,0 +1,494 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * memory.numa_migrate must give a large page cache folio a destination of + * the same order. An oversized destination inherits the index and the + * reference count of the source, only the source pages are copied into it, + * a read of the folio is served from the destination by file offset and + * lands in its uninitialised part, and the inode dies in clear_inode() on + * eviction. + * + * Write a file on an XFS image (XFS allocates large folios for 64K writes) + * from a cgroup bound to one NUMA node, migrate the cgroup to another node + * and check that the data reads back intact. + * + * Needs root, two NUMA nodes with memory (numa=fake=2 will do), cgroup2 + * with the memory and cpuset controllers, and mkfs.xfs. The folio sizes + * are read through /proc/self/pagemap and /proc/kpageflags before the + * migration, and the node of every page through move_pages() after it: a + * file without large folios below the PMD size, or a migration that + * moved none of them in one piece, fails the test, as it would pass + * without touching the bug. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../kselftest_harness.h" +#include "ve_selftest.h" + +#define CG_NAME "ve_numa_migrate_test" +#define DIR_TEMPLATE "/var/tmp/ve_numa_migrate.XXXXXX" +/* mkfs.xfs refuses anything smaller; the image is sparse. */ +#define IMG_SIZE (320UL << 20) +#define FILE_SIZE (4UL << 20) +#define CHUNK_SIZE (64UL << 10) +#define PM_PFN_MASK ((1ULL << 55) - 1) +#ifndef MADV_POPULATE_READ +#define MADV_POPULATE_READ 22 +#endif +#define PM_PRESENT (1ULL << 63) +/* The periodic memcg stats flush runs every two seconds. */ +#define STAT_RETRIES 50 +#define STAT_RETRY_US 100000 + +FIXTURE(numa_migrate) +{ + int cgv2_fd; + int cg_fd; + int node_from; + int node_to; + char dir[64]; + char img[80]; + char mnt[80]; + int mounted; + int leave_mounted; + int data_fd; +}; + +/* First two node ids of a sysfs node list such as "0-1" or "0,2-3". */ +static int two_nodes_with_memory(int *from, int *to) +{ + char buf[256], *tok, *save; + int found = 0; + + if (read_file_at(AT_FDCWD, "/sys/devices/system/node/has_memory", + buf, sizeof(buf)) <= 0) + return -1; + + for (tok = strtok_r(buf, ",\n", &save); tok && found < 2; + tok = strtok_r(NULL, ",\n", &save)) { + int a, b, n; + + if (sscanf(tok, "%d-%d", &a, &b) != 2) { + if (sscanf(tok, "%d", &a) != 1) + return -1; + b = a; + } + for (n = a; n <= b && found < 2; n++) { + if (found == 0) + *from = n; + else + *to = n; + found++; + } + } + return found == 2 ? 0 : -1; +} + +static int run_cmd(const char *fmt, ...) +{ + char cmd[256]; + va_list ap; + int status; + + va_start(ap, fmt); + vsnprintf(cmd, sizeof(cmd), fmt, ap); + va_end(ap); + + status = system(cmd); + if (status < 0 || !WIFEXITED(status)) + return -1; + return WEXITSTATUS(status); +} + +/* Bytes of file pages the cgroup has on @node, from memory.numa_stat. */ +static int file_bytes_on_node(int cg_fd, int node, unsigned long long *out) +{ + static char buf[32768]; + char key[16], *p; + + if (read_file_at(cg_fd, "memory.numa_stat", buf, sizeof(buf)) <= 0) + return -1; + p = strstr(buf, "\nfile "); + if (!p) + return -1; + snprintf(key, sizeof(key), " N%d=", node); + p = strstr(p, key); + if (!p) + return -1; + *out = strtoull(p + strlen(key), NULL, 10); + return 0; +} + +static unsigned long page_size; + +/* A folio at a PMD-aligned index reads right from an oversized one. */ +static bool chunk_is_hole(unsigned long off, unsigned long pmd_size) +{ + return !((off / CHUNK_SIZE * CHUNK_SIZE) % pmd_size); +} + +#define SCAN_ORDERS 16 + +struct folio_scan { + long large; /* pages in large folios */ + long smallest; /* pages in the smallest large folio */ + long on_node; /* large folios below @pmd_pages that sit on @node */ + long hist[2][SCAN_ORDERS]; /* folios by order, on @node / elsewhere */ +}; + +static void folio_scan_end_run(struct folio_scan *fs, long run, int run_node, + int node, long pmd_pages) +{ + int order = 0; + + if (!run) + return; + while ((1L << order) < run && order < SCAN_ORDERS - 1) + order++; + fs->hist[run_node != node][order]++; + if (run == 1) + return; + if (!fs->smallest || run < fs->smallest) + fs->smallest = run; + if (run < pmd_pages && run_node == node) + fs->on_node++; +} + +/* "64 x 64K, 3 x 4K" or "none" */ +static const char *folio_hist_str(const long *hist, char *buf, size_t len) +{ + size_t pos = 0; + int order; + + for (order = SCAN_ORDERS - 1; order >= 0; order--) { + if (!hist[order]) + continue; + pos += snprintf(buf + pos, len - pos, "%s%ld x %luK", + pos ? ", " : "", hist[order], + (page_size << order) / 1024); + if (pos >= len) + break; + } + return pos ? buf : "none"; +} + +/* + * Walk the page cache of @fd through /proc/self/pagemap, /proc/kpageflags + * and move_pages(2) in query mode, leaving the holes alone: faulting them + * in would add folios to the file. Returns -1 when that is not possible. + */ +static int scan_file_folios(int fd, unsigned long size, int node, + unsigned long pmd_size, struct folio_scan *fs) +{ + unsigned long nr = size / page_size, i, off; + long pmd_pages = pmd_size / page_size; + long run = 0; + int run_node = -1, ret = -1; + int pagemap, kpageflags, *status; + unsigned char *map; + void **pages; + uint64_t *pm; + + memset(fs, 0, sizeof(*fs)); + pagemap = open("/proc/self/pagemap", O_RDONLY); + kpageflags = open("/proc/kpageflags", O_RDONLY); + pm = malloc(nr * sizeof(*pm)); + pages = malloc(nr * sizeof(*pages)); + status = malloc(nr * sizeof(*status)); + map = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0); + if (pagemap < 0 || kpageflags < 0 || !pm || !pages || !status || + map == MAP_FAILED) + goto out; + for (off = 0; off < size; off += CHUNK_SIZE) { + if (!chunk_is_hole(off, pmd_size) && + madvise(map + off, CHUNK_SIZE, MADV_POPULATE_READ)) + goto out; + } + if (pread(pagemap, pm, nr * sizeof(*pm), + (unsigned long)map / page_size * sizeof(*pm)) != + (ssize_t)(nr * sizeof(*pm))) + goto out; + for (i = 0; i < nr; i++) + pages[i] = map + i * page_size; + if (syscall(__NR_move_pages, 0, nr, pages, NULL, status, 0)) + goto out; + for (i = 0; i < nr; i++) { + uint64_t flags, pfn = pm[i] & PM_PFN_MASK; + + if (chunk_is_hole(i * page_size, pmd_size)) { + folio_scan_end_run(fs, run, run_node, node, pmd_pages); + run = 0; + continue; + } + if (!(pm[i] & PM_PRESENT) || !pfn || status[i] < 0 || + pread(kpageflags, &flags, sizeof(flags), + pfn * sizeof(flags)) != sizeof(flags)) + goto out; + if (flags & (1ULL << KPF_COMPOUND_HEAD)) { + folio_scan_end_run(fs, run, run_node, node, pmd_pages); + fs->large++; + run = 1; + run_node = status[i]; + } else if (flags & (1ULL << KPF_COMPOUND_TAIL)) { + fs->large++; + run++; + } else { + folio_scan_end_run(fs, run, run_node, node, pmd_pages); + folio_scan_end_run(fs, 1, status[i], node, pmd_pages); + run = 0; + } + } + folio_scan_end_run(fs, run, run_node, node, pmd_pages); + ret = 0; +out: + if (map != MAP_FAILED) + munmap(map, size); + free(status); + free(pages); + free(pm); + if (pagemap >= 0) + close(pagemap); + if (kpageflags >= 0) + close(kpageflags); + return ret; +} + +static void cleanup(FIXTURE_DATA(numa_migrate) *self) +{ + char path[PATH_MAX]; + + if (self->data_fd >= 0) { + close(self->data_fd); + self->data_fd = -1; + } + if (self->dir[0]) { + snprintf(path, sizeof(path), "%s/data", self->mnt); + unlink(path); + if (self->mounted) + umount(self->mnt); + unlink(self->img); + rmdir(self->mnt); + rmdir(self->dir); + self->dir[0] = '\0'; + } + if (self->cg_fd >= 0) { + close(self->cg_fd); + self->cg_fd = -1; + } + if (self->cgv2_fd >= 0) { + /* The test task may still sit in the cgroup after a failure. */ + write_u64_at(self->cgv2_fd, "cgroup.procs", getpid()); + unlinkat(self->cgv2_fd, CG_NAME, AT_REMOVEDIR); + close(self->cgv2_fd); + self->cgv2_fd = -1; + } +} + +/* + * Returns 0, or 1 with everything undone when the host cannot run the test. + * The harness runs no teardown after a failed setup, so the undo is here. + */ +static int prepare(FIXTURE_DATA(numa_migrate) *self, const char **why) +{ + int fd; + + self->cgv2_fd = mount_cg2_fd(); + if (self->cgv2_fd < 0) { + *why = "cannot mount cgroup2"; + goto fail; + } + if (write_file_at(self->cgv2_fd, "cgroup.subtree_control", + "+memory +cpuset")) { + *why = "cannot enable the memory and cpuset controllers"; + goto fail; + } + if (mkdirat(self->cgv2_fd, CG_NAME, 0755) && errno != EEXIST) { + *why = "cannot create the test cgroup"; + goto fail; + } + self->cg_fd = openat(self->cgv2_fd, CG_NAME, O_PATH | O_DIRECTORY); + if (self->cg_fd < 0) { + *why = "cannot open the test cgroup"; + goto fail; + } + if (faccessat(self->cg_fd, "memory.numa_migrate", W_OK, 0)) { + *why = "no memory.numa_migrate"; + goto fail; + } + if (write_u64_at(self->cg_fd, "cpuset.mems", self->node_from)) { + *why = "cannot bind the cgroup to a node"; + goto fail; + } + + snprintf(self->dir, sizeof(self->dir), DIR_TEMPLATE); + if (!mkdtemp(self->dir)) { + self->dir[0] = '\0'; + *why = "cannot create a temporary directory"; + goto fail; + } + snprintf(self->img, sizeof(self->img), "%s/img", self->dir); + snprintf(self->mnt, sizeof(self->mnt), "%s/mnt", self->dir); + if (mkdir(self->mnt, 0755)) { + *why = "cannot create the mount point"; + goto fail; + } + fd = open(self->img, O_CREAT | O_RDWR, 0600); + if (fd < 0) { + *why = "cannot create the image file"; + goto fail; + } + if (ftruncate(fd, IMG_SIZE)) { + close(fd); + *why = "cannot size the image file"; + goto fail; + } + close(fd); + if (run_cmd("mkfs.xfs -q -f %s", self->img)) { + *why = "mkfs.xfs failed"; + goto fail; + } + if (run_cmd("mount -o loop %s %s", self->img, self->mnt)) { + *why = "cannot loop-mount the XFS image"; + goto fail; + } + self->mounted = 1; + return 0; +fail: + cleanup(self); + return 1; +} + +FIXTURE_SETUP(numa_migrate) +{ + const char *why; + + self->cgv2_fd = -1; + self->cg_fd = -1; + self->data_fd = -1; + self->dir[0] = '\0'; + page_size = sysconf(_SC_PAGESIZE); + + if (geteuid()) + SKIP(return, "needs root"); + if (two_nodes_with_memory(&self->node_from, &self->node_to)) + SKIP(return, "needs two NUMA nodes with memory"); + if (prepare(self, &why)) + SKIP(return, "%s", why); +} + +FIXTURE_TEARDOWN(numa_migrate) +{ + if (self->leave_mounted) { + TH_LOG("leaving %s mounted, eviction would hit BUG_ON(nrpages)", + self->mnt); + self->dir[0] = '\0'; + } + cleanup(self); +} + +TEST_F(numa_migrate, large_folio_order) +{ + unsigned long long on_to_before, on_to_after; + unsigned char *buf, *check; + char path[PATH_MAX], list[64]; + unsigned long long pmd_size; + struct folio_scan fs; + char h0[128], h1[128]; + long pmd_pages; + unsigned long off; + int fd, i; + + ASSERT_EQ(write_u64_at(self->cg_fd, "cgroup.procs", getpid()), 0); + + buf = malloc(CHUNK_SIZE); + check = malloc(CHUNK_SIZE); + ASSERT_NE(buf, NULL); + ASSERT_NE(check, NULL); + + /* The old code allocated PMD-sized destinations: those were fine. */ + if (read_u64_at(AT_FDCWD, + "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", + &pmd_size)) + pmd_size = 2UL << 20; + pmd_pages = pmd_size / page_size; + + snprintf(path, sizeof(path), "%s/data", self->mnt); + fd = open(path, O_CREAT | O_RDWR | O_TRUNC, 0600); + ASSERT_GE(fd, 0); + self->data_fd = fd; + for (off = 0; off < FILE_SIZE; off += CHUNK_SIZE) { + if (chunk_is_hole(off, pmd_size)) + continue; + memset(buf, (off / CHUNK_SIZE) & 0xff, CHUNK_SIZE); + ASSERT_EQ(pwrite(fd, buf, CHUNK_SIZE, off), (ssize_t)CHUNK_SIZE); + } + ASSERT_EQ(fsync(fd), 0); + ASSERT_EQ(scan_file_folios(fd, FILE_SIZE, self->node_to, pmd_size, &fs), 0) + TH_LOG("cannot read the page cache folio sizes"); + TH_LOG("before: on node %d: %s, elsewhere: %s", self->node_to, + folio_hist_str(fs.hist[0], h0, sizeof(h0)), + folio_hist_str(fs.hist[1], h1, sizeof(h1))); + ASSERT_GT(fs.large, 0) + TH_LOG("no large folio in the page cache, the bug is out of reach"); + ASSERT_LT(fs.smallest, pmd_pages) + TH_LOG("only PMD-sized folios in the page cache, the bug is out of reach"); + + ASSERT_EQ(file_bytes_on_node(self->cg_fd, self->node_to, + &on_to_before), 0); + + /* From here on a failure means eviction would crash the kernel. */ + self->leave_mounted = 1; + snprintf(list, sizeof(list), "%d,%d", self->node_from, self->node_to); + ASSERT_EQ(write_file_at(self->cg_fd, "cpuset.mems", list), 0); + ASSERT_EQ(write_u64_at(self->cg_fd, "memory.numa_migrate", + self->node_to), 0); + + for (off = 0; off < FILE_SIZE; off += CHUNK_SIZE) { + if (chunk_is_hole(off, pmd_size)) + continue; + memset(buf, (off / CHUNK_SIZE) & 0xff, CHUNK_SIZE); + ASSERT_EQ(pread(fd, check, CHUNK_SIZE, off), (ssize_t)CHUNK_SIZE); + ASSERT_EQ(memcmp(buf, check, CHUNK_SIZE), 0); + } + + /* memory.numa_stat flushes per-cpu counters lazily, give it time. */ + for (i = 0; i < STAT_RETRIES; i++) { + ASSERT_EQ(file_bytes_on_node(self->cg_fd, self->node_to, + &on_to_after), 0); + if (on_to_after >= on_to_before + FILE_SIZE) + break; + usleep(STAT_RETRY_US); + } + TH_LOG("file bytes on node %d: %llu -> %llu", self->node_to, + on_to_before, on_to_after); + + ASSERT_EQ(scan_file_folios(fd, FILE_SIZE, self->node_to, pmd_size, &fs), 0) + TH_LOG("cannot read the page cache folio sizes"); + TH_LOG("after: on node %d: %s, elsewhere: %s", self->node_to, + folio_hist_str(fs.hist[0], h0, sizeof(h0)), + folio_hist_str(fs.hist[1], h1, sizeof(h1))); + ASSERT_GT(fs.on_node, 0) + TH_LOG("no folio below the PMD size reached node %d in one piece", + self->node_to); + self->leave_mounted = 0; + + free(buf); + free(check); +} + +TEST_HARNESS_MAIN -- 2.43.0 _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel