All Virtuozzo development lists (kernel + QEMU)
 help / color / mirror / Atom feed
From: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
To: khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com
Cc: devel@openvz.org
Subject: Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
Date: Wed, 11 Nov 2026 16:34:34 +0100	[thread overview]
Message-ID: <92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com> (raw)
In-Reply-To: <010701a1c556541f-a3b013d6-70e1-45b6-bfc1-37c018cdf88f-000000@eu-central-1.amazonses.com>



On 11/10/26 13:02, no-reply@sashiko.virtuozzo.com wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] dm-qcow2: concurrent qio preparers can deadlock on qio_pool under memory pressure
> --
> 
> commit 1b2b6ed6f3d2b43e71cc7424d0502aa9998c0d28
> Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
> 
> dm-qcow2: add per-qio work infrastructure
> 
> This commit adds a work_struct and a state field to struct qio. It uses
> them to queue the first pipeline step, qio preparation, as one work item
> per data request on tgt->wq. Previously this step ran serialised inside the
> single qcow2->worker.
> 
> Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
> 
>> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
>> --- a/drivers/md/dm-qcow2-map.c
>> +++ b/drivers/md/dm-qcow2-map.c
> 
> [ ... ]
> 
>> @@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>>   
>>   	if (blk_rq_bytes(rq)) {
>>   		queue_list_id = QLIST_EMBEDDED;
>> -		worker = &qcow2->worker;
>> +		worker = &qio->worker;
>>   	} else {
> 
> [ ... ]
> 
>>   	spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>>   
>>   	if (queue)
>>   		queue_work(tgt->wq, worker);
> 
> [Severity: Medium]
> Can concurrent preparers deadlock on qio_pool under memory pressure?
> 
> Before this patch, preparation ran serialised inside qcow2->worker, so
> only one preparer held a partial split list at a time. Now one work item
> is queued per data request on the unbound tgt->wq, which has the default
> max_active limit of 256.
> 
> Each work item runs:
> 
> do_qio_work()
>    prepare_one_embedded_qio()
>      qcow2_split_qio_to_list()
>        split_and_chain_qio()
>          qcow2_alloc_qio(qcow2->tgt->qio_pool, true)
>            mempool_alloc(GFP_NOIO)
> 
> The split qios stay on a local list and are only handed to QLIST_DEFERRED
> in do_qio_work() after the whole request has been split:
> 
>> +		prepare_one_embedded_qio(qio, &deferred_qios);
>> +		if (list_empty(&deferred_qios))
>> +			break;
> 
> The pool is created in alloc_qcow2_target() as:
> 
> tgt->qio_pool = mempool_create_kmalloc_pool(MIN_QIOS, sizeof(struct qio));
> 
> with 512 elements, shared by all preparers and the rest of the pipeline.
> Up to 512 requests may be in flight (qrq_pool), and each request needs up
> to request_size/clu_size splits (about 20 for a 64K cluster and a 1280K
> request, and more for smaller clusters). Total demand can therefore exceed
> 512.
> 
> If kmalloc keeps failing under memory pressure, can several preparers each
> hold part of the reserve while all of them sleep in mempool_alloc() waiting
> for more? The only wake-up source is a free of a qio from a completed
> request, and none of the blocked preparers' own splits can complete until
> splitting finishes.
> 
> The later patches in the series do not appear to remove this. The final
> state (d9084b95) drops the per-qio work_struct and runs do_qio_work() from
> per-CPU qcow2_fast_work items on the same tgt->wq. That lowers the number
> of preparers from up to 256 to one per CPU, but each preparer still builds
> its full split list with blocking mempool_alloc() before releasing any of
> it, so the hold-and-wait remains possible, mainly on many-core machines or
> with small clusters.
> 
> Would it make sense to throttle or reserve pool space for preparers?
> 

Even before the series amount of splits could exceed 512 and 
theoretically exhaust this pool and sleep. Anyway mempool_alloc also has 
kmalloc retry logic and other mitigation mechanisms. Not only this issue 
  is pre-existing but qio's are also quite small and if the node is out 
of 0-order allocations we are likely doomed anyway.

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

  reply	other threads:[~2026-11-11 15:36 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
2026-11-10 11:55   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
2026-11-10 11:56   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
2026-11-10 11:59   ` no-reply
2026-11-10 13:50     ` Andrey Zhadchenko
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-11 14:36   ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
2026-11-10 12:02   ` no-reply
2026-11-11 15:34     ` Andrey Zhadchenko [this message]
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
2026-11-10 12:08   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
2026-11-10 12:06   ` no-reply
2026-11-10 17:56   ` Konstantin Khorenko
2026-11-11 15:00   ` Alexander Ivanov
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT Andrey Zhadchenko
2026-11-10 12:06   ` no-reply
2026-11-10 17:56 ` [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Konstantin Khorenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=92b5dce5-fbc5-4072-a893-f5affa82bf37@virtuozzo.com \
    --to=andrey.zhadchenko@virtuozzo.com \
    --cc=alexander.ivanov@virtuozzo.com \
    --cc=devel@openvz.org \
    --cc=khorenko@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.