From: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
To: khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com
Cc: devel@openvz.org
Subject: Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
Date: Wed, 11 Nov 2026 16:34:34 +0100 [thread overview]
Message-ID: <92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com> (raw)
In-Reply-To: <010701a1c556541f-a3b013d6-70e1-45b6-bfc1-37c018cdf88f-000000@eu-central-1.amazonses.com>
On 11/10/26 13:02, no-reply@sashiko.virtuozzo.com wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] dm-qcow2: concurrent qio preparers can deadlock on qio_pool under memory pressure
> --
>
> commit 1b2b6ed6f3d2b43e71cc7424d0502aa9998c0d28
> Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
>
> dm-qcow2: add per-qio work infrastructure
>
> This commit adds a work_struct and a state field to struct qio. It uses
> them to queue the first pipeline step, qio preparation, as one work item
> per data request on tgt->wq. Previously this step ran serialised inside the
> single qcow2->worker.
>
> Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
>
>> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
>> --- a/drivers/md/dm-qcow2-map.c
>> +++ b/drivers/md/dm-qcow2-map.c
>
> [ ... ]
>
>> @@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>>
>> if (blk_rq_bytes(rq)) {
>> queue_list_id = QLIST_EMBEDDED;
>> - worker = &qcow2->worker;
>> + worker = &qio->worker;
>> } else {
>
> [ ... ]
>
>> spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>>
>> if (queue)
>> queue_work(tgt->wq, worker);
>
> [Severity: Medium]
> Can concurrent preparers deadlock on qio_pool under memory pressure?
>
> Before this patch, preparation ran serialised inside qcow2->worker, so
> only one preparer held a partial split list at a time. Now one work item
> is queued per data request on the unbound tgt->wq, which has the default
> max_active limit of 256.
>
> Each work item runs:
>
> do_qio_work()
> prepare_one_embedded_qio()
> qcow2_split_qio_to_list()
> split_and_chain_qio()
> qcow2_alloc_qio(qcow2->tgt->qio_pool, true)
> mempool_alloc(GFP_NOIO)
>
> The split qios stay on a local list and are only handed to QLIST_DEFERRED
> in do_qio_work() after the whole request has been split:
>
>> + prepare_one_embedded_qio(qio, &deferred_qios);
>> + if (list_empty(&deferred_qios))
>> + break;
>
> The pool is created in alloc_qcow2_target() as:
>
> tgt->qio_pool = mempool_create_kmalloc_pool(MIN_QIOS, sizeof(struct qio));
>
> with 512 elements, shared by all preparers and the rest of the pipeline.
> Up to 512 requests may be in flight (qrq_pool), and each request needs up
> to request_size/clu_size splits (about 20 for a 64K cluster and a 1280K
> request, and more for smaller clusters). Total demand can therefore exceed
> 512.
>
> If kmalloc keeps failing under memory pressure, can several preparers each
> hold part of the reserve while all of them sleep in mempool_alloc() waiting
> for more? The only wake-up source is a free of a qio from a completed
> request, and none of the blocked preparers' own splits can complete until
> splitting finishes.
>
> The later patches in the series do not appear to remove this. The final
> state (d9084b95) drops the per-qio work_struct and runs do_qio_work() from
> per-CPU qcow2_fast_work items on the same tgt->wq. That lowers the number
> of preparers from up to 256 to one per CPU, but each preparer still builds
> its full split list with blocking mempool_alloc() before releasing any of
> it, so the hold-and-wait remains possible, mainly on many-core machines or
> with small clusters.
>
> Would it make sense to throttle or reserve pool space for preparers?
>
Even before the series amount of splits could exceed 512 and
theoretically exhaust this pool and sleep. Anyway mempool_alloc also has
kmalloc retry logic and other mitigation mechanisms. Not only this issue
is pre-existing but qio's are also quite small and if the node is out
of 0-order allocations we are likely doomed anyway.
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
next prev parent reply other threads:[~2026-11-11 15:36 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
2026-11-10 11:55 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
2026-11-10 11:56 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
2026-11-10 11:59 ` no-reply
2026-11-10 13:50 ` Andrey Zhadchenko
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 14:36 ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
2026-11-10 12:02 ` no-reply
2026-11-11 15:34 ` Andrey Zhadchenko [this message]
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
2026-11-10 12:08 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 15:00 ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
2026-11-10 17:56 ` [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Konstantin Khorenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com \
--to=andrey.zhadchenko@virtuozzo.com \
--cc=alexander.ivanov@virtuozzo.com \
--cc=devel@openvz.org \
--cc=khorenko@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox