* [Devel] [PATCH hci-8.1 v2 1/2] fs/aio: restore the AIO ring of the dumped size in a pseudosuper VE
2026-11-16 10:31 [Devel] [PATCH hci-8.1 v2 0/2] fs/aio: fix CT migration with AIO to a node with more CPUs Pavel Tikhomirov
@ 2026-11-16 10:31 ` Pavel Tikhomirov
2026-11-16 10:36 ` no-reply
2026-11-16 10:31 ` [Devel] [PATCH hci-8.1 v2 2/2] selftests/ve: check the AIO ring " Pavel Tikhomirov
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: Pavel Tikhomirov @ 2026-11-16 10:31 UTC (permalink / raw)
To: Konstantin Khorenko; +Cc: devel
io_setup() makes the ring at least 4 events per possible CPU of the
node. The ring is mapped into the task memory, and CRIU restores it with
io_setup() and moves the new mapping to the address of the old one, so
it needs a ring of the same size. On a node with more possible CPUs it
gets a bigger ring, and the restore fails. E.g. io_setup(1) makes a
ring of one page, 127 events, on 4 CPUs, and of two pages, 255 events,
on 16 CPUs.
Skip the CPU heuristic while the VE is pseudosuper, which it is almost
only for the restore. Keep the other adjustments, the doubling and the
rounding up to pages: they don't depend on the node, and the queue depth
CRIU estimates from the dumped ring already offsets them, so the restore
gets the ring of the dumped size.
A ring restored below the minimum of this node doesn't use the per-CPU
caches of free slots: it has too few slots to spare for them.
https://virtuozzo.atlassian.net/browse/VSTOR-154129
Feature: ve: CRIU support
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
---
fs/aio.c | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/fs/aio.c b/fs/aio.c
index 901cb622fe2c4..2ff1758507611 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -109,6 +109,7 @@ struct kioctx {
* counter at a time:
*/
unsigned req_batch;
+ bool no_percpu_reqs;
/*
* This is what userspace passed to io_setup(), it's not used for
* anything but counting against the global max_reqs quota.
@@ -745,6 +746,7 @@ static struct kioctx *ioctx_alloc(unsigned nr_events)
struct mm_struct *mm = current->mm;
struct kioctx *ctx;
struct ve_struct *ve = get_exec_env();
+ bool no_percpu_reqs = false;
int err = -ENOMEM;
/*
@@ -761,8 +763,16 @@ static struct kioctx *ioctx_alloc(unsigned nr_events)
* and unavailable, double nr_events so userspace sees what they
* expected: additionally, we move req_batch slots to/from percpu
* counters at a time, so make sure that isn't 0:
+ *
+ * CRIU restores a ring with io_setup() and needs the size it had on
+ * dump, which can be below the minimum of this node due to the CPU
+ * heuristic below, so skip the heuristic on restore. Such a ring
+ * can't spare slots for the percpu counters, so it doesn't use them.
*/
- nr_events = max(nr_events, num_possible_cpus() * 4);
+ if (ve_is_super(ve) || !ve->is_pseudosuper)
+ nr_events = max(nr_events, num_possible_cpus() * 4);
+ else if (nr_events < num_possible_cpus() * 4)
+ no_percpu_reqs = true;
nr_events *= 2;
/* Prevent overflows */
@@ -779,6 +789,7 @@ static struct kioctx *ioctx_alloc(unsigned nr_events)
return ERR_PTR(-ENOMEM);
ctx->max_reqs = max_reqs;
+ ctx->no_percpu_reqs = no_percpu_reqs;
ctx->ve = get_ve(ve);
spin_lock_init(&ctx->ctx_lock);
@@ -950,6 +961,11 @@ static void put_reqs_available(struct kioctx *ctx, unsigned nr)
struct kioctx_cpu *kcpu;
unsigned long flags;
+ if (ctx->no_percpu_reqs) {
+ atomic_add(nr, &ctx->reqs_available);
+ return;
+ }
+
local_irq_save(flags);
kcpu = this_cpu_ptr(ctx->cpu);
kcpu->reqs_available += nr;
@@ -968,6 +984,9 @@ static bool __get_reqs_available(struct kioctx *ctx)
bool ret = false;
unsigned long flags;
+ if (ctx->no_percpu_reqs)
+ return atomic_dec_if_positive(&ctx->reqs_available) >= 0;
+
local_irq_save(flags);
kcpu = this_cpu_ptr(ctx->cpu);
if (!kcpu->reqs_available) {
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 7+ messages in thread* [Devel] [PATCH hci-8.1 v2 2/2] selftests/ve: check the AIO ring size in a pseudosuper VE
2026-11-16 10:31 [Devel] [PATCH hci-8.1 v2 0/2] fs/aio: fix CT migration with AIO to a node with more CPUs Pavel Tikhomirov
2026-11-16 10:31 ` [Devel] [PATCH hci-8.1 v2 1/2] fs/aio: restore the AIO ring of the dumped size in a pseudosuper VE Pavel Tikhomirov
@ 2026-11-16 10:31 ` Pavel Tikhomirov
2026-11-16 10:38 ` no-reply
2026-11-16 16:55 ` [Devel] [PATCH hci-8.1 v2 0/2] fs/aio: fix CT migration with AIO to a node with more CPUs Konstantin Khorenko
2026-11-16 16:55 ` Konstantin Khorenko
3 siblings, 1 reply; 7+ messages in thread
From: Pavel Tikhomirov @ 2026-11-16 10:31 UTC (permalink / raw)
To: Konstantin Khorenko; +Cc: devel
Check that the host and a VE keep the CPU heuristic of the io_setup()
ring and a pseudosuper VE skips it, and that io_setup((nr - 2) / 2)
there gives back a ring of nr events for rings of 1 to 4 pages.
The rings differ only when the CPU heuristic makes a one-page ring
bigger, e.g. with 16 possible CPUs and 4K pages, else the test is
skipped. Before the fix the pseudosuper tests fail on 16 CPUs with a
ring of 255 events instead of 127.
https://virtuozzo.atlassian.net/browse/VSTOR-154129
Feature: ve: CRIU support
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
---
tools/testing/selftests/ve/.gitignore | 1 +
tools/testing/selftests/ve/Makefile | 1 +
tools/testing/selftests/ve/ve_aio_ring_test.c | 218 ++++++++++++++++++++++++++
3 files changed, 220 insertions(+)
diff --git a/tools/testing/selftests/ve/.gitignore b/tools/testing/selftests/ve/.gitignore
index 3d1179bd2c5f4..2ff027048a6ae 100644
--- a/tools/testing/selftests/ve/.gitignore
+++ b/tools/testing/selftests/ve/.gitignore
@@ -4,3 +4,4 @@ ve_mount_accounting_test
ve_cpuid_override_test
ve_vtty_stress
ve_numa_migrate_test
+ve_aio_ring_test
diff --git a/tools/testing/selftests/ve/Makefile b/tools/testing/selftests/ve/Makefile
index e96320a183844..124b4a918016c 100644
--- a/tools/testing/selftests/ve/Makefile
+++ b/tools/testing/selftests/ve/Makefile
@@ -7,6 +7,7 @@ TEST_GEN_PROGS += ve_perms_test
TEST_GEN_PROGS += ve_mount_accounting_test
TEST_GEN_PROGS += ve_vtty_stress
TEST_GEN_PROGS += ve_numa_migrate_test
+TEST_GEN_PROGS += ve_aio_ring_test
# CPUID faulting is x86-64 only. ARCH says "x86" for 32-bit too and lib.mk
# picks the clang target by its emptiness, so decide by the compiler.
diff --git a/tools/testing/selftests/ve/ve_aio_ring_test.c b/tools/testing/selftests/ve/ve_aio_ring_test.c
new file mode 100644
index 0000000000000..1922a05aceef4
--- /dev/null
+++ b/tools/testing/selftests/ve/ve_aio_ring_test.c
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ve_aio_ring selftest
+ *
+ * io_setup() makes the ring at least 4 events per possible CPU. CRIU
+ * restores a ring with io_setup() and needs the size it had on dump, so
+ * a VE in pseudosuper mode (restore) gets the ring of exactly the size
+ * asked for. The tests check that:
+ *
+ * - the host keeps the CPU heuristic
+ * - a VE keeps the CPU heuristic
+ * - a VE in pseudosuper mode skips the CPU heuristic
+ * - a VE in pseudosuper mode gets a ring of nr events from
+ * io_setup((nr - 2) / 2), for rings of 1 to 4 pages
+ *
+ * The rings differ only when the CPU heuristic makes a small ring bigger,
+ * e.g. with 16 possible CPUs and 4K pages, else the test is skipped.
+ */
+#define _GNU_SOURCE
+#include <sched.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <linux/aio_abi.h>
+
+#include "../kselftest_harness.h"
+#include "ve_selftest.h"
+
+/* The ring header mapped by io_setup(), a copy of the one in fs/aio.c. */
+struct aio_ring {
+ unsigned int id;
+ unsigned int nr; /* ring capacity in io_events */
+ unsigned int head;
+ unsigned int tail;
+ unsigned int magic;
+ unsigned int compat_features;
+ unsigned int incompat_features;
+ unsigned int header_length;
+ struct io_event io_events[];
+};
+
+#define PAGE_ALIGN(size, ps) (((size) + ((ps) - 1)) & ~((ps) - 1))
+
+/* The biggest ring, in pages, the pseudosuper restore test covers. */
+#define RING_PAGES_MAX 4
+
+struct aio_ring_arg {
+ unsigned int nr_events; /* in: io_setup() nr_events */
+ unsigned int ring_nr; /* out: ring capacity, struct aio_ring::nr */
+};
+
+FIXTURE(ve_aio_ring)
+{
+ int cgv2_fd;
+ int ctid;
+ struct namespace_ctx ns_ctx;
+ unsigned int cpus;
+ unsigned long page_size;
+ struct aio_ring_arg *arg; /* shared with the child in the VE */
+};
+
+/* The ring capacity aio_setup_ring() makes for @nr_events. */
+static unsigned int calculate_ring_nr(FIXTURE_DATA(ve_aio_ring) * self,
+ unsigned int nr_events)
+{
+ unsigned long size;
+
+ size = sizeof(struct aio_ring) + (nr_events + 2) * sizeof(struct io_event);
+ size = PAGE_ALIGN(size, self->page_size);
+ return (size - sizeof(struct aio_ring)) / sizeof(struct io_event);
+}
+
+/* The ring capacity io_setup(@nr_events) makes. */
+static unsigned int ring_nr(FIXTURE_DATA(ve_aio_ring) * self,
+ unsigned int nr_events)
+{
+ unsigned int cpu_events = self->cpus * 4;
+
+ if (nr_events < cpu_events)
+ nr_events = cpu_events;
+ return calculate_ring_nr(self, 2 * nr_events);
+}
+
+/* The ring capacity io_setup(@nr_events) makes without the CPU heuristic. */
+static unsigned int ring_nr_pseudosuper(FIXTURE_DATA(ve_aio_ring) * self,
+ unsigned int nr_events)
+{
+ return calculate_ring_nr(self, 2 * nr_events);
+}
+
+/* Create an AIO context of arg->nr_events, store its ring capacity. */
+static int aio_ring_nr(void *data)
+{
+ struct aio_ring_arg *arg = data;
+ aio_context_t ctx = 0;
+
+ if (syscall(__NR_io_setup, arg->nr_events, &ctx)) {
+ fprintf(stderr, "%s: io_setup(%u): %s\n", __func__,
+ arg->nr_events, strerror(errno));
+ return -1;
+ }
+ arg->ring_nr = ((struct aio_ring *)ctx)->nr;
+
+ if (syscall(__NR_io_destroy, ctx)) {
+ fprintf(stderr, "%s: io_destroy: %s\n", __func__,
+ strerror(errno));
+ return -1;
+ }
+ return 0;
+}
+
+/* The ring capacity io_setup(@nr_events) makes in the VE, or 0. */
+static unsigned int ve_ring_nr(FIXTURE_DATA(ve_aio_ring) * self,
+ unsigned int nr_events)
+{
+ self->arg->nr_events = nr_events;
+ self->arg->ring_nr = 0;
+ if (run_in_ve(self->cgv2_fd, self->ctid, &self->ns_ctx, CLONE_NEWVE,
+ aio_ring_nr, self->arg))
+ return 0;
+ return self->arg->ring_nr;
+}
+
+static int set_pseudosuper(FIXTURE_DATA(ve_aio_ring) * self, int value)
+{
+ char path[64];
+
+ snprintf(path, sizeof(path), "%d/ve.pseudosuper", self->ctid);
+ return write_u64_at(self->cgv2_fd, path, value);
+}
+
+FIXTURE_SETUP(ve_aio_ring)
+{
+ long cpus;
+ int ret;
+
+ self->ctid = -1;
+ self->arg = MAP_FAILED;
+
+ self->page_size = sysconf(_SC_PAGESIZE);
+ cpus = sysconf(_SC_NPROCESSORS_CONF);
+ ASSERT_GT(cpus, 0);
+ self->cpus = cpus;
+ if (ring_nr(self, 1) == ring_nr_pseudosuper(self, 1))
+ SKIP(return, "%u possible CPUs are too few for the CPU heuristic to matter",
+ self->cpus);
+
+ self->arg = mmap(NULL, sizeof(*self->arg), PROT_READ | PROT_WRITE,
+ MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+ ASSERT_NE(self->arg, MAP_FAILED);
+
+ self->cgv2_fd = mount_cg2_fd();
+ ASSERT_GE(self->cgv2_fd, 0);
+
+ ASSERT_EQ(write_file_at(self->cgv2_fd, "cgroup.subtree_control",
+ VE_CONTROLLERS), 0);
+
+ self->ctid = make_ve(self->cgv2_fd, CTID_MIN);
+ ASSERT_GE(self->ctid, 0);
+
+ ret = prepare_namespaces(&self->ns_ctx, CLONE_NEWVE,
+ self->cgv2_fd, self->ctid);
+ if (ret)
+ destroy_ve(self->cgv2_fd, self->ctid);
+ ASSERT_EQ(ret, 0);
+}
+
+FIXTURE_TEARDOWN(ve_aio_ring)
+{
+ if (self->ctid >= 0) {
+ set_pseudosuper(self, 0);
+ close_namespace_fds(&self->ns_ctx);
+ destroy_ve(self->cgv2_fd, self->ctid);
+ close(self->cgv2_fd);
+ }
+ if (self->arg != MAP_FAILED)
+ munmap(self->arg, sizeof(*self->arg));
+}
+
+TEST_F(ve_aio_ring, host_keeps_cpu_heuristic)
+{
+ self->arg->nr_events = 1;
+ EXPECT_EQ(aio_ring_nr(self->arg), 0);
+ EXPECT_EQ(self->arg->ring_nr, ring_nr(self, 1));
+}
+
+TEST_F(ve_aio_ring, ve_keeps_cpu_heuristic)
+{
+ EXPECT_EQ(ve_ring_nr(self, 1), ring_nr(self, 1));
+}
+
+TEST_F(ve_aio_ring, pseudosuper_ve_skips_cpu_heuristic)
+{
+ ASSERT_EQ(set_pseudosuper(self, 1), 0);
+ EXPECT_EQ(ve_ring_nr(self, 1), ring_nr_pseudosuper(self, 1));
+}
+
+/*
+ * A ring of nr events is restored with io_setup((nr - 2) / 2), check
+ * it gives back a ring of nr events.
+ */
+TEST_F(ve_aio_ring, pseudosuper_ve_restores_ring_size)
+{
+ unsigned int pages, nr;
+
+ ASSERT_EQ(set_pseudosuper(self, 1), 0);
+ for (pages = 1; pages <= RING_PAGES_MAX; pages++) {
+ nr = (pages * self->page_size - sizeof(struct aio_ring)) /
+ sizeof(struct io_event);
+ EXPECT_EQ(ve_ring_nr(self, (nr - 2) / 2), nr)
+ TH_LOG("ring of %u pages", pages);
+ }
+}
+
+TEST_HARNESS_MAIN
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 7+ messages in thread