OpenVZ / Virtuozzo kernel development (devel@openvz.org)
 help / color / mirror / Atom feed
From: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
To: khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com
Cc: devel@openvz.org
Subject: Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
Date: Wed, 11 Nov 2026 16:34:34 +0100	[thread overview]
Message-ID: <92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com> (raw)
In-Reply-To: <010701a1c556541f-a3b013d6-70e1-45b6-bfc1-37c018cdf88f-000000@eu-central-1.amazonses.com>



On 11/10/26 13:02, no-reply@sashiko.virtuozzo.com wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] dm-qcow2: concurrent qio preparers can deadlock on qio_pool under memory pressure
> --
> 
> commit 1b2b6ed6f3d2b43e71cc7424d0502aa9998c0d28
> Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
> 
> dm-qcow2: add per-qio work infrastructure
> 
> This commit adds a work_struct and a state field to struct qio. It uses
> them to queue the first pipeline step, qio preparation, as one work item
> per data request on tgt->wq. Previously this step ran serialised inside the
> single qcow2->worker.
> 
> Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
> 
>> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
>> --- a/drivers/md/dm-qcow2-map.c
>> +++ b/drivers/md/dm-qcow2-map.c
> 
> [ ... ]
> 
>> @@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>>   
>>   	if (blk_rq_bytes(rq)) {
>>   		queue_list_id = QLIST_EMBEDDED;
>> -		worker = &qcow2->worker;
>> +		worker = &qio->worker;
>>   	} else {
> 
> [ ... ]
> 
>>   	spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>>   
>>   	if (queue)
>>   		queue_work(tgt->wq, worker);
> 
> [Severity: Medium]
> Can concurrent preparers deadlock on qio_pool under memory pressure?
> 
> Before this patch, preparation ran serialised inside qcow2->worker, so
> only one preparer held a partial split list at a time. Now one work item
> is queued per data request on the unbound tgt->wq, which has the default
> max_active limit of 256.
> 
> Each work item runs:
> 
> do_qio_work()
>    prepare_one_embedded_qio()
>      qcow2_split_qio_to_list()
>        split_and_chain_qio()
>          qcow2_alloc_qio(qcow2->tgt->qio_pool, true)
>            mempool_alloc(GFP_NOIO)
> 
> The split qios stay on a local list and are only handed to QLIST_DEFERRED
> in do_qio_work() after the whole request has been split:
> 
>> +		prepare_one_embedded_qio(qio, &deferred_qios);
>> +		if (list_empty(&deferred_qios))
>> +			break;
> 
> The pool is created in alloc_qcow2_target() as:
> 
> tgt->qio_pool = mempool_create_kmalloc_pool(MIN_QIOS, sizeof(struct qio));
> 
> with 512 elements, shared by all preparers and the rest of the pipeline.
> Up to 512 requests may be in flight (qrq_pool), and each request needs up
> to request_size/clu_size splits (about 20 for a 64K cluster and a 1280K
> request, and more for smaller clusters). Total demand can therefore exceed
> 512.
> 
> If kmalloc keeps failing under memory pressure, can several preparers each
> hold part of the reserve while all of them sleep in mempool_alloc() waiting
> for more? The only wake-up source is a free of a qio from a completed
> request, and none of the blocked preparers' own splits can complete until
> splitting finishes.
> 
> The later patches in the series do not appear to remove this. The final
> state (d9084b95) drops the per-qio work_struct and runs do_qio_work() from
> per-CPU qcow2_fast_work items on the same tgt->wq. That lowers the number
> of preparers from up to 256 to one per CPU, but each preparer still builds
> its full split list with blocking mempool_alloc() before releasing any of
> it, so the hold-and-wait remains possible, mainly on many-core machines or
> with small clusters.
> 
> Would it make sense to throttle or reserve pool space for preparers?
> 

Even before the series amount of splits could exceed 512 and 
theoretically exhaust this pool and sleep. Anyway mempool_alloc also has 
kmalloc retry logic and other mitigation mechanisms. Not only this issue 
  is pre-existing but qio's are also quite small and if the node is out 
of 0-order allocations we are likely doomed anyway.

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

  reply	other threads:[~2026-11-11 15:36 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
2026-11-10 11:55   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
2026-11-10 11:56   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
2026-11-10 11:59   ` no-reply
2026-11-10 13:50     ` Andrey Zhadchenko
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-11 14:36   ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
2026-11-10 12:02   ` no-reply
2026-11-11 15:34     ` Andrey Zhadchenko [this message]
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
2026-11-10 12:08   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
2026-11-10 12:06   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-11 15:00   ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT Andrey Zhadchenko
2026-11-10 12:06   ` no-reply
2026-11-10 17:56 ` [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Konstantin Khorenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com \
    --to=andrey.zhadchenko@virtuozzo.com \
    --cc=alexander.ivanov@virtuozzo.com \
    --cc=devel@openvz.org \
    --cc=khorenko@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox