* [PATCH hci-8.0/hci-8.1 1/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066
2026-09-24 12:13 [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066 Denis V. Lunev
@ 2026-09-24 12:13 ` Denis V. Lunev
2026-09-24 12:13 ` [PATCH hci-8.0/hci-8.1 2/2] iotests/migrate-bitmaps-postcopy-test: add a bitmap on a second node #VSTOR-147066 Denis V. Lunev
2026-09-24 13:58 ` [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066 Andrey Drobyshev
2 siblings, 0 replies; 4+ messages in thread
From: Denis V. Lunev @ 2026-09-24 12:13 UTC (permalink / raw)
To: svt-core; +Cc: andrey.drobyshev, den, Denis V. Lunev
From: Denis V. Lunev <den@openvz.org>
With dirty bitmaps on more than one node, the source switches nodes
during the postcopy bulk phase and sends the node name again. The
destination reads those chunks in the postcopy listen thread, which does
not hold the BQL, and resolves the name through bdrv_lookup_bs():
qemu-system-x86_64: ../block.c:6400: bdrv_lookup_bs: Assertion
`qemu_in_main_thread()' failed.
The bdrv_find_dirty_bitmap() call right after it needs the BQL or the
dirty bitmap lock as well. Taking the BQL there would deadlock: the
listen thread holds s->lock, and dirty_bitmap_mig_before_vm_start()
takes s->lock with the BQL held.
Every node and bitmap that a non-START chunk refers to was set up by a
START chunk, and those are loaded under the BQL during setup. Record the
node alias there, and resolve all other chunks against s->bitmaps
instead of the block graph.
Bitmaps on a single node never resend the node name after setup, which
is why migrate-bitmaps-postcopy-test does not catch this.
Fixes: f791bf7f93f2 ("assertions for block global state API")
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
migration/block-dirty-bitmap.c | 59 +++++++++++++++++++++++++++++++---
1 file changed, 55 insertions(+), 4 deletions(-)
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 9c8c308773d..d17eec412bb 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -136,6 +136,7 @@ typedef struct DBMSaveState {
} DBMSaveState;
typedef struct LoadBitmapState {
+ char node_alias[256];
BlockDriverState *bs;
BdrvDirtyBitmap *bitmap;
bool migrated;
@@ -861,6 +862,7 @@ static int dirty_bitmap_load_start(QEMUFile *f, DBMLoadState *s)
}
b = g_new(LoadBitmapState, 1);
+ g_strlcpy(b->node_alias, s->node_alias, sizeof(b->node_alias));
b->bs = s->bs;
b->bitmap = s->bitmap;
b->migrated = false;
@@ -1058,6 +1060,54 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoadState *s)
return 0;
}
+/*
+ * Only START chunks are guaranteed to be loaded under the BQL; the rest may
+ * come from the postcopy listen thread and must not touch the block graph.
+ */
+static BlockDriverState *dirty_bitmap_load_find_bs(DBMLoadState *s,
+ const char *device,
+ const char *node_name,
+ Error **errp)
+{
+ GSList *item;
+
+ if (s->flags & DIRTY_BITMAP_MIG_FLAG_START) {
+ return bdrv_lookup_bs(device, node_name, errp);
+ }
+
+ for (item = s->bitmaps; item; item = g_slist_next(item)) {
+ LoadBitmapState *b = item->data;
+
+ if (!strcmp(b->node_alias, s->node_alias)) {
+ return b->bs;
+ }
+ }
+
+ error_setg(errp, "Error: no migrated bitmaps on node alias '%s'",
+ s->node_alias);
+ return NULL;
+}
+
+static BdrvDirtyBitmap *dirty_bitmap_load_find_bitmap(DBMLoadState *s)
+{
+ GSList *item;
+
+ if (s->flags & DIRTY_BITMAP_MIG_FLAG_START) {
+ return bdrv_find_dirty_bitmap(s->bs, s->bitmap_name);
+ }
+
+ for (item = s->bitmaps; item; item = g_slist_next(item)) {
+ LoadBitmapState *b = item->data;
+
+ if (b->bs == s->bs &&
+ !strcmp(bdrv_dirty_bitmap_name(b->bitmap), s->bitmap_name)) {
+ return b->bitmap;
+ }
+ }
+
+ return NULL;
+}
+
static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s,
GHashTable *alias_map)
{
@@ -1086,11 +1136,12 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s,
s->bs = NULL;
} else {
bitmap_alias_map = amin->subtree;
- s->bs = bdrv_lookup_bs(NULL, amin->string, &local_err);
+ s->bs = dirty_bitmap_load_find_bs(s, NULL, amin->string,
+ &local_err);
}
} else {
- s->bs = bdrv_lookup_bs(s->node_alias, s->node_alias,
- &local_err);
+ s->bs = dirty_bitmap_load_find_bs(s, s->node_alias,
+ s->node_alias, &local_err);
}
if (!s->bs) {
error_report_err(local_err);
@@ -1141,7 +1192,7 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s,
if (!s->cancelled) {
g_strlcpy(s->bitmap_name, bitmap_name, sizeof(s->bitmap_name));
- s->bitmap = bdrv_find_dirty_bitmap(s->bs, s->bitmap_name);
+ s->bitmap = dirty_bitmap_load_find_bitmap(s);
/*
* bitmap may be NULL here, it wouldn't be an error if it is the
--
2.53.0
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH hci-8.0/hci-8.1 2/2] iotests/migrate-bitmaps-postcopy-test: add a bitmap on a second node #VSTOR-147066
2026-09-24 12:13 [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066 Denis V. Lunev
2026-09-24 12:13 ` [PATCH hci-8.0/hci-8.1 1/2] " Denis V. Lunev
@ 2026-09-24 12:13 ` Denis V. Lunev
2026-09-24 13:58 ` [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066 Andrey Drobyshev
2 siblings, 0 replies; 4+ messages in thread
From: Denis V. Lunev @ 2026-09-24 12:13 UTC (permalink / raw)
To: svt-core; +Cc: andrey.drobyshev, den, Denis V. Lunev
From: Denis V. Lunev <den@openvz.org>
All bitmaps in the test live on drive0, so the source never switches
nodes after setup and the destination never resolves a node name from
the postcopy listen thread. Add a second drive with one bitmap and check
that it arrives intact.
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
.../tests/migrate-bitmaps-postcopy-test | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
index c519e6db8c7..71982045d53 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
@@ -27,6 +27,8 @@ debug = False
disk_a = os.path.join(iotests.test_dir, 'disk_a')
disk_b = os.path.join(iotests.test_dir, 'disk_b')
+disk_a1 = os.path.join(iotests.test_dir, 'disk_a1')
+disk_b1 = os.path.join(iotests.test_dir, 'disk_b1')
size = '256G'
fifo = os.path.join(iotests.test_dir, 'mig_fifo')
@@ -97,16 +99,23 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
self.vm_b.shutdown()
os.remove(disk_a)
os.remove(disk_b)
+ os.remove(disk_a1)
+ os.remove(disk_b1)
os.remove(fifo)
def setUp(self):
os.mkfifo(fifo)
qemu_img('create', '-f', iotests.imgfmt, disk_a, size)
qemu_img('create', '-f', iotests.imgfmt, disk_b, size)
+ qemu_img('create', '-f', iotests.imgfmt, disk_a1, '1G')
+ qemu_img('create', '-f', iotests.imgfmt, disk_b1, '1G')
self.vm_a = iotests.VM(path_suffix='a').add_drive(disk_a,
'discard=unmap')
self.vm_b = iotests.VM(path_suffix='b').add_drive(disk_b,
'discard=unmap')
+ # A bitmap on a second node makes postcopy switch nodes after setup
+ self.vm_a.add_drive(disk_a1)
+ self.vm_b.add_drive(disk_b1)
self.vm_b.add_incoming("exec: cat '" + fifo + "'")
self.vm_a.launch()
self.vm_b.launch()
@@ -127,6 +136,13 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
node='drive0', name='bitmap0')
empty_sha256 = result['return']['sha256']
+ self.vm_a.cmd('block-dirty-bitmap-add', node='drive1',
+ name='bitmap0', granularity=granularity)
+ self.vm_a.hmp_qemu_io('drive1', 'write 0 64k')
+ result = self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256',
+ node='drive1', name='bitmap0')
+ self.drive1_sha256 = result['return']['sha256']
+
apply_discards(self.vm_a, discards1)
result = self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256',
@@ -199,6 +215,9 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
# check that bitmaps are migrated and persistence works
check_bitmaps(self.vm_b, nb_bitmaps)
+ result = self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256',
+ node='drive1', name='bitmap0')
+ self.assert_qmp(result, 'return/sha256', self.drive1_sha256)
self.vm_b.shutdown()
# recreate vm_b, so there is no incoming option, which prevents
# loading bitmaps from disk
--
2.53.0
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066
2026-09-24 12:13 [PATCH hci-8.0/hci-8.1 0/2] migration/block-dirty-bitmap: fix postcopy abort with bitmaps on several nodes #VSTOR-147066 Denis V. Lunev
2026-09-24 12:13 ` [PATCH hci-8.0/hci-8.1 1/2] " Denis V. Lunev
2026-09-24 12:13 ` [PATCH hci-8.0/hci-8.1 2/2] iotests/migrate-bitmaps-postcopy-test: add a bitmap on a second node #VSTOR-147066 Denis V. Lunev
@ 2026-09-24 13:58 ` Andrey Drobyshev
2 siblings, 0 replies; 4+ messages in thread
From: Andrey Drobyshev @ 2026-09-24 13:58 UTC (permalink / raw)
To: Denis V. Lunev, svt-core; +Cc: den
LGTM
Ack
On 9/24/26 3:13 PM, Denis V. Lunev wrote:
> Live migration of a VM with dirty bitmaps on more than one node aborts
> the destination once the bitmaps move post-copy:
>
> qemu-kvm: ../block.c:6477: bdrv_lookup_bs: Assertion
> `qemu_in_main_thread()' failed.
>
> libvirt triggers this on every such VM: a CBT backup leaves a persistent
> x-libvirt-cbt bitmap on each disk, and libvirt migrates bitmaps with
> dirty-bitmaps + migrate-start-postcopy. hci-upgrade hit it evacuating a
> three-disk VM, see VSTOR-147066 for the RCA.
>
> This is the series posted upstream, unchanged apart from the ticket tag:
> https://lore.kernel.org/qemu-devel/20260924115611.4098417-1-den@openvz.org/
>
> Both patches apply cleanly to hci-8.0 and hci-8.1. The VZ-only changes in
> migration/block-dirty-bitmap.c (PSBM-105022, VSTOR-98747) touch the save
> side only. The source-side runstate_set(-1) abort seen in the same
> ticket on hci-7.4 is already fixed here by 9867d4ddd04.
>
> Denis V. Lunev (2):
> migration/block-dirty-bitmap: fix postcopy abort with bitmaps on
> several nodes #VSTOR-147066
> iotests/migrate-bitmaps-postcopy-test: add a bitmap on a second node
> #VSTOR-147066
>
> migration/block-dirty-bitmap.c | 59 +++++++++++++++++--
> .../tests/migrate-bitmaps-postcopy-test | 19 ++++++
> 2 files changed, 74 insertions(+), 4 deletions(-)
>
>
> base-commit: e8a73d39faf578d58aa2c4b4b024226796c390e6
^ permalink raw reply [flat|nested] 4+ messages in thread