* [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 11:55 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
` (6 subsequent siblings)
7 siblings, 2 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
parse_metadata() is called not only from the main kwork:
qcow2_llseek_hole() calls it in process context, in parallel with
the kwork. When both miss the same md page, both drop md_pages_lock
in submit_read_md_page() and try to insert a new md page. The one
that loses the insert gets -EEXIST from qcow2_alloc_and_insert_md_page()
and returns it as is: a guest I/O on the kwork side ends with EIO,
or lseek(SEEK_DATA/SEEK_HOLE) fails with -EEXIST.
Teach __handle_md_page() to handle -EEXIST by repeating the lookup:
the page inserted in parallel is found then, and the qio is either
served from it or postponed until its read completes.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Fixes: 412d5a4f7bf1 ("dm/dm-qcow2: add llseek_hole")
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: say in the message what fails without the patch: the race of
qcow2_llseek_hole() with main kwork; added Fixes: tag
drivers/md/dm-qcow2-map.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index 34d9d9810f601..21d1c8975af99 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -1594,20 +1594,23 @@ static int submit_read_md_page(struct qcow2 *qcow2, struct qio **qio,
/*
* This may be called with @qio == NULL, in case of we are
* interesting in searching cached in memory md only.
- * This is aimed to be called not only from main kwork
- * for L1/L2 pages, so all callers looking for L1/L2
- * must care about submit_read_md_page() may return EEXIST.
*/
static int __handle_md_page(struct qcow2 *qcow2, u64 page_id,
struct qio **qio, struct md_page **ret_md)
{
struct md_page *md;
+ int ret;
lockdep_assert_held(&qcow2->md_pages_lock);
+again:
md = qcow2_md_page_find_or_postpone(qcow2, page_id, qio);
if (!md) {
- if (qio && *qio)
- return submit_read_md_page(qcow2, qio, page_id);
+ if (qio && *qio) {
+ ret = submit_read_md_page(qcow2, qio, page_id);
+ if (unlikely(ret == -EEXIST))
+ goto again;
+ return ret;
+ }
return 0;
}
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
@ 2026-11-10 11:55 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: no-reply @ 2026-11-10 11:55 UTC (permalink / raw)
To: khorenko, Andrey Zhadchenko, alexander.ivanov; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] dm-qcow2: unbounded goto again in __handle_md_page() on injected -EEXIST
--
commit 949be3603b3c812058ed084d1aae429a3ec4e585
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
dm-qcow2: repeat md page lookup if it was inserted in parallel
This patch makes __handle_md_page() retry the md page lookup when
submit_read_md_page() returns -EEXIST. The -EEXIST means another context,
such as qcow2_llseek_hole() running in parallel with the kwork, inserted the
same md page first. The retry lets the qio be served from, or postponed on,
that page.
> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
> --- a/drivers/md/dm-qcow2-map.c
> +++ b/drivers/md/dm-qcow2-map.c
[ ... ]
> static int __handle_md_page(struct qcow2 *qcow2, u64 page_id,
> struct qio **qio, struct md_page **ret_md)
> {
> struct md_page *md;
> + int ret;
>
> lockdep_assert_held(&qcow2->md_pages_lock);
> +again:
> md = qcow2_md_page_find_or_postpone(qcow2, page_id, qio);
> if (!md) {
> - if (qio && *qio)
> - return submit_read_md_page(qcow2, qio, page_id);
> + if (qio && *qio) {
> + ret = submit_read_md_page(qcow2, qio, page_id);
> + if (unlikely(ret == -EEXIST))
> + goto again;
> + return ret;
> + }
> return 0;
> }
[Severity: Low]
Can this retry loop in __handle_md_page() spin forever?
The goto again assumes that -EEXIST from submit_read_md_page() always means
a page with this id is now in the rb-tree, so the next
qcow2_md_page_find_or_postpone() will find it.
That holds for the genuine source of -EEXIST, md_page_try_insert() in
dm-qcow2-target.c, which returns it only when the id is already in the tree.
However, qcow2_alloc_and_insert_md_page() is annotated:
ALLOW_ERROR_INJECTION(qcow2_alloc_and_insert_md_page, ERRNO);
so fail_function can make it return -EEXIST while no page exists. In that
case:
__handle_md_page()
qcow2_md_page_find_or_postpone() -> NULL
submit_read_md_page() -> -EEXIST (injected)
goto again
repeats with no retry cap and no cond_resched(). Before this patch, the
injected -EEXIST was returned as an ordinary error.
All callers of __handle_md_page() that pass a qio would be affected, so the
kwork or the llseek caller in process context could spin or trigger a soft
lockup. Would a bounded retry, or a cond_resched() in the loop, be
appropriate?
>
> *ret_md = md;
> return 1;
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=1
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
2026-11-10 11:55 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:23 +0300, Andrey Zhadchenko wrote:
> + if (qio && *qio) {
> + ret = submit_read_md_page(qcow2, qio, page_id);
> + if (unlikely(ret == -EEXIST))
> + goto again;
> + return ret;
> + }
Item 4 of the cover letter reply: answer Sashiko's remark on the
unbounded retry, or retry only once.
Why: Sashiko raised it on v1 and again on v2. With fail_function
injecting -EEXIST into qcow2_alloc_and_insert_md_page(), or NULL into
qcow2_md_page_find_or_postpone(), this loop never ends, and
cond_resched() cannot help under md_pages_lock. A real -EEXIST means
the page is in the tree, so one retry always finds it.
The code is otherwise fine, and the message and the Fixes tag are
right.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped()
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 11:56 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
` (5 subsequent siblings)
7 siblings, 2 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
perform_rw_mapped() reads md->status and md->wbd without
md_pages_lock, relying on dm-qcow2 sequential nature: L1/L2 pages
become dirty, go to writeback and drop md->wbd from main kwork only.
Expand md_pages_lock to also cover the check, so it can be called
from the per-qio fast path in parallel with main kwork.
The lock alone is not enough. parse_l2() does not delay a write to
a changed index of a DIRTY L2 page, as perform_rw_mapped() is going
to attach the write to md->wbd. Main kwork does parse, submit and the
DIRTY -> WRITEBACK switch in one do_qcow2_work() pass, so the page
is still DIRTY there. The fast path parses and submits under separate
lock holds, and main kwork may switch the page to WRITEBACK in
between:
main kwork fast worker
allocates cluster X,
L2 page DIRTY
parse_l2(): write to X, page DIRTY,
not delayed, unlock
submit_metadata_writeback():
DIRTY -> WRITEBACK
perform_rw_mapped(): no MD_DIRTY,
qio->data = NULL, submit
The write is then acked as soon as its data is written, before the
L2 entry mapping it is on disk. A FLUSH after it finds
QLIST_COMPLETED_WB empty and only calls vfs_fsync(), so it does not
wait for the L2 write in flight, and a crash loses a write that was
acked and flushed. If the L2 write fails instead, the entry is
reverted and X freed while the write already returned success.
So, when the page is under writeback and the index is changed, park
the qio on md->wait_list, as parse_l2() does. It is redispatched to
main kwork when the writeback ends and parsed again. Do not attach it
to the wbd under writeback: after a failed writeback complete_wbd()
reverts the indexes and frees the wbd.
This changes nothing for main kwork, where parse_l2() has already
delayed such writes in the same pass.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: renamed from "check MD_DIRTY and md->wbd under lock in
perform_rw_mapped()"; park a write on md->wait_list if its L2 page
went to writeback after parse_l2(), instead of acking it before its
L2 entry is on disk; commit message describes the race
drivers/md/dm-qcow2-map.c | 24 +++++++++++++++++++-----
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index 21d1c8975af99..49dc3e8119e76 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -2912,14 +2912,28 @@ static void perform_rw_mapped(struct qcow2_map *map, struct qio *qio)
* wait for md writeback completion.
*
* L1/L2 pages become set and unset dirty from main work only,
- * so lock is not needed for MD_DIRTY/changed_indexes visibility.
+ * but this may be called from per-qio fast path in parallel
+ * with main work submitting the writeback and with its completion
+ * dropping md->wbd.
+ *
+ * So md may have gone DIRTY -> WRITEBACK after parse_l2() decided
+ * the qio need not wait. Then the qio must wait for writeback end
+ * the way parse_l2() makes it. It can't be attached to wbd: after
+ * a failed writeback, complete_wbd() reverts the indexes and frees
+ * the wbd without looking at qios attached in the meantime.
*/
index_in_page = map->l2.index_in_page + !!(qcow2->ext_l2);
- if (rw == WRITE && (md->status & MD_DIRTY) &&
- test_bit(index_in_page, md->wbd->changed_indexes)) {
+ if (rw == WRITE) {
spin_lock_irqsave(&qcow2->md_pages_lock, flags);
- md->wbd->nr_submitted++;
- qio->data = md->wbd;
+ if (__delay_if_writeback(qcow2, md, index_in_page, &qio, false)) {
+ spin_unlock_irqrestore(&qcow2->md_pages_lock, flags);
+ return;
+ }
+ if ((md->status & MD_DIRTY) &&
+ test_bit(index_in_page, md->wbd->changed_indexes)) {
+ md->wbd->nr_submitted++;
+ qio->data = md->wbd;
+ }
spin_unlock_irqrestore(&qcow2->md_pages_lock, flags);
}
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped()
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
@ 2026-11-10 11:56 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: no-reply @ 2026-11-10 11:56 UTC (permalink / raw)
To: alexander.ivanov, Andrey Zhadchenko, khorenko; +Cc: devel
> perform_rw_mapped() reads md->status and md->wbd without
> md_pages_lock, relying on dm-qcow2 sequential nature: L1/L2 pages
> become dirty, go to writeback and drop md->wbd from main kwork only.
> Expand md_pages_lock to also cover the check, so it can be called
> from the per-qio fast path in parallel with main kwork.
>
> The lock alone is not enough. parse_l2() does not delay a write to
> a changed index of a DIRTY L2 page, as perform_rw_mapped() is going
> to attach the write to md->wbd. Main kwork does parse, submit and the
> DIRTY -> WRITEBACK switch in one do_qcow2_work() pass, so the page
> is still DIRTY there. The fast path parses and submits under separate
> lock holds, and main kwork may switch the page to WRITEBACK in
> between:
>
> main kwork fast worker
> [ ... ]
> delayed such writes in the same pass.
>
> https://virtuozzo.atlassian.net/browse/VSTOR-95494
> Feature: dm-qcow2: block device over QCOW2 files driver
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=2
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped()
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
2026-11-10 11:56 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:24 +0300, Andrey Zhadchenko wrote:
> + if (__delay_if_writeback(qcow2, md, index_in_page, &qio, false)) {
> + spin_unlock_irqrestore(&qcow2->md_pages_lock, flags);
> + return;
> + }
Item 1 of the cover letter reply: under this lock also check that the
L2 entry (and the ext_l2 word) is what parse_l2() saw and that the
index is not locked; if not, hand the qio back to main kwork.
Why: a failed L2 writeback can complete before the fast worker gets
here. complete_wbd() then reverts the entry and clears both
MD_WRITEBACK and md->wbd, so this code sees neither DIRTY nor
WRITEBACK. The write goes to X untracked and is acked, while X is no
longer mapped: the data is lost. In main kwork the write was attached
to the wbd in the same pass and got the error. The same recheck also
catches a discard that has already changed the entry (item 3).
> acked and flushed. If the L2 write fails instead, the entry is
> reverted and X freed while the write already returned success.
Item 1 of the cover letter reply: "X freed" becomes "X leaked".
Why: complete_wbd() clears FREE_ALLOCATED_CLU of the allocating qio,
so X stays allocated and unmapped. That wording came from my v1 reply,
sorry. Also, md->wbd is dropped in complete_wbd() from the fsync work,
not only by main kwork as the first paragraph says.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 1/7] dm-qcow2: repeat md page lookup if it was inserted in parallel Andrey Zhadchenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 2/7] dm-qcow2: wait for L2 writeback in perform_rw_mapped() Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 11:59 ` no-reply
` (2 more replies)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
` (4 subsequent siblings)
7 siblings, 3 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
dm-qcow2 heavily relies on main kwork do_qcow2_work() being
sequential. All metadata change happens there and cannot
run in parallel. Unfortunately that caps dm-qcow2 performance.
But if the request does not require metadata changes we should
be able to immediately dispatch it.
process_one_qio() and handle_metadata() already know what to
do: just add avoid_metadata_changes flag and return codes.
While at it, just return in handle_metadata() instead of
setting ret to 0 and jumping to if (ret < 0) label.
Also pass the flag to parse_metadata() and return to the slow path
before it looks up R1/R2 pages. They are needed only by COW writes
and cluster unmapping discards, which go to the slow path anyway.
Besides the useless lookup, R1 is unstable outside main kwork: an R1
page read in parallel with relocate_refcount_table() may stay cached
under a stale page id.
Also exclude compressed clusters as there are some complications
with wpc_readers.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: return to the slow path before R1/R2 lookup in parse_metadata();
QIO_NEEDS_MAIN_KWORK instead of magic 2; fixed the
"avoid_metadata_changes() flag" typo
drivers/md/dm-qcow2-map.c | 55 ++++++++++++++++++++++++++++++---------
1 file changed, 42 insertions(+), 13 deletions(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index 49dc3e8119e76..c495834a4d29f 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -2002,6 +2002,12 @@ static int qcow2_handle_r1r2_maps(struct qcow2 *qcow2, loff_t pos, struct qio **
return ret;
}
+/*
+ * Returned by parse_metadata() and handle_metadata() with
+ * @avoid_metadata_changes, when @qio must be processed by main kwork.
+ */
+#define QIO_NEEDS_MAIN_KWORK 2
+
/*
* This caches pages of allocated on disk md levels, which are
* required for submission @qio, and checks they are stable.
@@ -2009,9 +2015,12 @@ static int qcow2_handle_r1r2_maps(struct qcow2 *qcow2, loff_t pos, struct qio **
* Returns: negative in case of error, or 0 if success.
* Special case: return 0 with zeroed @qio means @qio was deferred
* till some event: reading of md page, end of writeback, etc.
+ * With @avoid_metadata_changes, return QIO_NEEDS_MAIN_KWORK if @qio
+ * needs R1/R2: outside main kwork R1 is unstable, see
+ * __handle_r1r2_maps().
*/
static int parse_metadata(struct qcow2 *qcow2, struct qio **qio,
- struct qcow2_map *map)
+ struct qcow2_map *map, bool avoid_metadata_changes)
{
bool write = op_is_write((*qio)->bi_op);
struct md_page *md;
@@ -2075,6 +2084,9 @@ static int parse_metadata(struct qcow2 *qcow2, struct qio **qio,
if (!qio_may_modify_image(qcow2, *qio))
return 0;
+ if (avoid_metadata_changes)
+ return QIO_NEEDS_MAIN_KWORK;
+
/* Now refcounters table/block */
ret = qcow2_handle_r1r2_maps(qcow2, pos, qio, &map->r1,
&map->r2, map->compressed);
@@ -3548,24 +3560,31 @@ static int prepare_cluster_discard(struct qcow2 *qcow2, struct qio **qio,
}
static int handle_metadata(struct qcow2 *qcow2, struct qio **qio,
- struct qcow2_map *map)
+ struct qcow2_map *map, bool avoid_metadata_changes)
{
bool write = op_is_write((*qio)->bi_op);
int ret;
- ret = parse_metadata(qcow2, qio, map);
+ ret = parse_metadata(qcow2, qio, map, avoid_metadata_changes);
if (ret < 0 || !*qio) /* Error or postponed */
goto check_err;
+ if (ret == QIO_NEEDS_MAIN_KWORK)
+ return QIO_NEEDS_MAIN_KWORK;
+
+ if (avoid_metadata_changes && map->compressed)
+ return QIO_NEEDS_MAIN_KWORK;
ret = 1;
if (unlikely(qcow2->backward_merge_in_process)) {
+ if (avoid_metadata_changes)
+ return QIO_NEEDS_MAIN_KWORK;
/* Keep in mind the below may replace *qio */
ret = prepare_backward_merge(qcow2, qio, map, write);
} else if (unlikely(fake_l1cow_qio(*qio)) &&
(!map->clu_is_cow || (map->level & L2_LEVEL))) {
/* Nothing to COW or L1 is mapped exactly once */
qio_endio(*qio);
- ret = 0;
+ return 0;
} else if (unlikely(op_is_discard((*qio)->bi_op))) {
/*
* Discard carries no data and is advisory. If it does not
@@ -3573,13 +3592,17 @@ static int handle_metadata(struct qcow2 *qcow2, struct qio **qio,
* is not present, end early.
*/
if (qio_discard_updates_metadata(qcow2, *qio, map)) {
+ if (avoid_metadata_changes)
+ return QIO_NEEDS_MAIN_KWORK;
ret = prepare_cluster_discard(qcow2, qio, map);
} else if (!map->data_clu_alloced || map->clu_is_cow) {
qio_endio(*qio);
- ret = 0;
+ return 0;
}
} else if (write &&
(!qio_is_fully_alloced(qcow2, *qio, map) || map->clu_is_cow)) {
+ if (avoid_metadata_changes)
+ return QIO_NEEDS_MAIN_KWORK;
if (map->clu_is_cow) {
/* COW to compressed or shared with snapshot cluster */
ret = prepare_l1l2_replace(qcow2, *qio, map);
@@ -3691,13 +3714,17 @@ static void process_read_qio(struct qcow2 *qcow2, struct qio *qio,
qio_endio(qio);
}
-static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
+static int process_one_qio(struct qcow2 *qcow2, struct qio *qio, bool avoid_metadata_changes)
{
struct qcow2_map map = { .qcow2 = qcow2, };
bool write;
+ int ret;
- if (!handle_metadata(qcow2, &qio, &map))
- return;
+ ret = handle_metadata(qcow2, &qio, &map, avoid_metadata_changes);
+ if (!ret)
+ return 0;
+ else if (ret == QIO_NEEDS_MAIN_KWORK)
+ return 1;
/*
* Merge machinery makes out of order merges for present
@@ -3709,7 +3736,7 @@ static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
if (unlikely(qcow2->backward_merge_in_process) &&
(fake_merge_qio(qio) || qcow2_file_is_writable(qcow2))) {
submit_top_delta_read(&map, qio);
- return;
+ return 0;
}
write = op_is_write(qio->bi_op);
@@ -3717,13 +3744,13 @@ static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
/* Process discard early. Don't do COW just to discard */
if (unlikely(op_is_discard(qio->bi_op))) {
issue_discard(&map, qio);
- return;
+ return 0;
}
if (unlikely(map.compressed)) {
/* Compressed qio never uses sub-clus */
submit_read_compressed(&map, qio, write);
- return;
+ return 0;
}
if (!write) {
@@ -3736,6 +3763,8 @@ static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
else
perform_rw_mapped(&map, qio);
}
+
+ return 0;
}
static struct bio_vec *create_bvec_from_rq(struct request *rq)
@@ -3838,7 +3867,7 @@ static void process_deferred_qios(struct qcow2 *qcow2, struct list_head *qios)
continue;
}
- process_one_qio(qcow2, qio);
+ process_one_qio(qcow2, qio, false);
}
}
@@ -4774,7 +4803,7 @@ static int qcow2_llseek_hole_qio(struct qio *qio, int whence, loff_t *result)
map.qcow2 = qio->qcow2;
qptr = qio;
- ret = parse_metadata(qio->qcow2, &qptr, &map);
+ ret = parse_metadata(qio->qcow2, &qptr, &map, false);
if (ret < 0)
break;
if (qptr == NULL) {
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
@ 2026-11-10 11:59 ` no-reply
2026-11-10 13:50 ` Andrey Zhadchenko
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 14:36 ` Alexander Ivanov
2 siblings, 1 reply; 26+ messages in thread
From: no-reply @ 2026-11-10 11:59 UTC (permalink / raw)
To: Andrey Zhadchenko, alexander.ivanov, khorenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] dm-qcow2: fast path uses unpinned L2 mapping between parse_metadata() and data submit
--
commit a3dc5e64e9aa8f73ed35df62194a86cf78276045
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
dm-qcow2: prepare process_one_qio() for fastpath
This patch adds an avoid_metadata_changes flag to process_one_qio(),
handle_metadata() and parse_metadata(). With the flag set, a qio that needs
metadata changes, R1/R2 lookups or a compressed cluster is handed back to the
main kwork through the new QIO_NEEDS_MAIN_KWORK return code. All callers in
this patch pass false.
Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
> --- a/drivers/md/dm-qcow2-map.c
> +++ b/drivers/md/dm-qcow2-map.c
[ ... ]
> @@ -2075,6 +2084,9 @@ static int parse_metadata(struct qcow2 *qcow2, struct qio **qio,
> if (!qio_may_modify_image(qcow2, *qio))
> return 0;
>
> + if (avoid_metadata_changes)
> + return QIO_NEEDS_MAIN_KWORK;
> +
> /* Now refcounters table/block */
> ret = qcow2_handle_r1r2_maps(qcow2, pos, qio, &map->r1,
> &map->r2, map->compressed);
[Severity: Medium]
In this patch alone the flag is only ever passed as false, so this path is
not reachable yet. It becomes live with the later patches that add per-qio
fast processing and per-cpu fast workers. Those call process_one_qio(...,
true) from outside the main kwork. Is the L2 mapping stable between
parse_metadata() and the data submit in that case?
parse_metadata() drops md_pages_lock at its unlock label before this point.
A qio that continues on the fast path then carries map->data_clu_pos and
map->l2.md with no lock, pin or wpc_readers reference:
process_one_qio()
handle_metadata()
parse_metadata() /* md_pages_lock dropped at unlock */
process_read_qio() or perform_rw_mapped()
map_and_submit_rw(qcow2, map->data_clu_pos, qio)
Meanwhile the main kwork can run the discard path:
prepare_cluster_discard()
prepare_l_entry_replace()
issue_discard()
qcow2_punch_hole(qcow2->file, pos, qio->bi_iter.bi_size)
issue_discard() punches the hole immediately. It does not wait for fast
path qios that already parsed the same L2 entry.
Can a fast-path read or write then go to a cluster that a concurrent
discard has already unmapped and freed, and which may have been reallocated
to another guest offset? That would return stale or foreign data on a read,
or write into a reused cluster.
As far as I can see, wpc_readers is taken only for compressed and shared
clusters in submit_read_whole_cow_clu() and submit_read_compressed(), and
this patch excludes compressed clusters from the fast path. The other
mechanisms only affect qios that parse later: delay_if_wpc_readers_locked()
and md_index_set_locked() do not cover a qio that has already parsed.
perform_rw_mapped() only revalidates the DIRTY and WRITEBACK state.
Fully allocated, non-COW, non-compressed clusters were already untracked
before this patch, so the new exposure is the added parse-to-submit window
in the per-cpu worker. That window can stretch if the worker is preempted.
The commit message says "if the request does not require metadata changes
we should be able to immediately dispatch it." What keeps the L2 entry and
data cluster stable for such a qio until its data I/O is submitted?
[ ... ]
> -static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
> +static int process_one_qio(struct qcow2 *qcow2, struct qio *qio, bool avoid_metadata_changes)
> {
> struct qcow2_map map = { .qcow2 = qcow2, };
> bool write;
> + int ret;
>
> - if (!handle_metadata(qcow2, &qio, &map))
> - return;
> + ret = handle_metadata(qcow2, &qio, &map, avoid_metadata_changes);
> + if (!ret)
> + return 0;
> + else if (ret == QIO_NEEDS_MAIN_KWORK)
> + return 1;
[Severity: Medium]
This is the other side of the same concern. When handle_metadata() returns 1
with avoid_metadata_changes set, the code below goes on to
process_read_qio() or perform_rw_mapped() without any protection against a
concurrent discard on the main kwork, as described above. Is there something
that closes this window?
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=3
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath
2026-11-10 11:59 ` no-reply
@ 2026-11-10 13:50 ` Andrey Zhadchenko
0 siblings, 0 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 13:50 UTC (permalink / raw)
To: alexander.ivanov, khorenko; +Cc: devel
This whole situation is pre-existing in general (the same scenario is
possible for base dm-qcow2). Theoretically this issue exists but
practically higly higly unlikely because on each re-allocation we first
do fallocate to punch zeroes which takes inode lock on xfs/ext4 like
discard does.
So not only there should be concurrent write/discard, there also should
be third write, and the write into discarding cluster should be paused
for astronomically big amount of time: after first query it should at
least outsleep two inode-wide exclusive IO locks and a lot of actions
in-between.
Despite the gap becoming bigger after this series, given the
pre-existing nature, I have not planned to change this behavior as this
is a separate patchset that implements per-md write counters or locking
for discards.
On 11/10/26 12:59, no-reply@sashiko.virtuozzo.com wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] dm-qcow2: fast path uses unpinned L2 mapping between parse_metadata() and data submit
> --
>
> commit a3dc5e64e9aa8f73ed35df62194a86cf78276045
> Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
>
> dm-qcow2: prepare process_one_qio() for fastpath
>
> This patch adds an avoid_metadata_changes flag to process_one_qio(),
> handle_metadata() and parse_metadata(). With the flag set, a qio that needs
> metadata changes, R1/R2 lookups or a compressed cluster is handed back to the
> main kwork through the new QIO_NEEDS_MAIN_KWORK return code. All callers in
> this patch pass false.
>
> Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
>
>> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
>> --- a/drivers/md/dm-qcow2-map.c
>> +++ b/drivers/md/dm-qcow2-map.c
>
> [ ... ]
>
>> @@ -2075,6 +2084,9 @@ static int parse_metadata(struct qcow2 *qcow2, struct qio **qio,
>> if (!qio_may_modify_image(qcow2, *qio))
>> return 0;
>>
>> + if (avoid_metadata_changes)
>> + return QIO_NEEDS_MAIN_KWORK;
>> +
>> /* Now refcounters table/block */
>> ret = qcow2_handle_r1r2_maps(qcow2, pos, qio, &map->r1,
>> &map->r2, map->compressed);
>
> [Severity: Medium]
> In this patch alone the flag is only ever passed as false, so this path is
> not reachable yet. It becomes live with the later patches that add per-qio
> fast processing and per-cpu fast workers. Those call process_one_qio(...,
> true) from outside the main kwork. Is the L2 mapping stable between
> parse_metadata() and the data submit in that case?
>
> parse_metadata() drops md_pages_lock at its unlock label before this point.
> A qio that continues on the fast path then carries map->data_clu_pos and
> map->l2.md with no lock, pin or wpc_readers reference:
>
> process_one_qio()
> handle_metadata()
> parse_metadata() /* md_pages_lock dropped at unlock */
> process_read_qio() or perform_rw_mapped()
> map_and_submit_rw(qcow2, map->data_clu_pos, qio)
>
> Meanwhile the main kwork can run the discard path:
>
> prepare_cluster_discard()
> prepare_l_entry_replace()
> issue_discard()
> qcow2_punch_hole(qcow2->file, pos, qio->bi_iter.bi_size)
>
> issue_discard() punches the hole immediately. It does not wait for fast
> path qios that already parsed the same L2 entry.
>
> Can a fast-path read or write then go to a cluster that a concurrent
> discard has already unmapped and freed, and which may have been reallocated
> to another guest offset? That would return stale or foreign data on a read,
> or write into a reused cluster.
>
> As far as I can see, wpc_readers is taken only for compressed and shared
> clusters in submit_read_whole_cow_clu() and submit_read_compressed(), and
> this patch excludes compressed clusters from the fast path. The other
> mechanisms only affect qios that parse later: delay_if_wpc_readers_locked()
> and md_index_set_locked() do not cover a qio that has already parsed.
> perform_rw_mapped() only revalidates the DIRTY and WRITEBACK state.
>
> Fully allocated, non-COW, non-compressed clusters were already untracked
> before this patch, so the new exposure is the added parse-to-submit window
> in the per-cpu worker. That window can stretch if the worker is preempted.
>
> The commit message says "if the request does not require metadata changes
> we should be able to immediately dispatch it." What keeps the L2 entry and
> data cluster stable for such a qio until its data I/O is submitted?
>
> [ ... ]
>
>> -static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
>> +static int process_one_qio(struct qcow2 *qcow2, struct qio *qio, bool avoid_metadata_changes)
>> {
>> struct qcow2_map map = { .qcow2 = qcow2, };
>> bool write;
>> + int ret;
>>
>> - if (!handle_metadata(qcow2, &qio, &map))
>> - return;
>> + ret = handle_metadata(qcow2, &qio, &map, avoid_metadata_changes);
>> + if (!ret)
>> + return 0;
>> + else if (ret == QIO_NEEDS_MAIN_KWORK)
>> + return 1;
>
> [Severity: Medium]
> This is the other side of the same concern. When handle_metadata() returns 1
> with avoid_metadata_changes set, the code below goes on to
> process_read_qio() or perform_rw_mapped() without any protection against a
> concurrent discard on the main kwork, as described above. Is there something
> that closes this window?
>
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
2026-11-10 11:59 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 14:36 ` Alexander Ivanov
2 siblings, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:25 +0300, Andrey Zhadchenko wrote:
> + if (avoid_metadata_changes)
> + return QIO_NEEDS_MAIN_KWORK;
No code changes in this patch for v3. On your answer to Sashiko about
the discard race, handled by item 3 of the cover letter reply:
- "The same scenario is possible for base dm-qcow2" holds only on
FUSE. On ext4 and xfs main kwork parses and submits a write in one
call, so its direct I/O is in flight before the discard is even
parsed, and the punch hole waits for it in inode_dio_wait().
fuse_file_fallocate() does not wait for direct I/O.
- "It should outsleep two inode-wide exclusive locks" does not cover a
fast worker that has parsed but not yet entered the filesystem: the
inode locks do not see it at all.
The window is new and needs a guest discarding and accessing the same
cluster at once, so a follow-up series is fine with me, as agreed. Say
it in the patches (item 3), and keep the switch of item 2 for the time
until it is fixed.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
2026-11-10 11:59 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
@ 2026-11-11 14:36 ` Alexander Ivanov
2 siblings, 0 replies; 26+ messages in thread
From: Alexander Ivanov @ 2026-11-11 14:36 UTC (permalink / raw)
To: Andrey Zhadchenko, khorenko; +Cc: devel
> -static void process_one_qio(struct qcow2 *qcow2, struct qio *qio)
> +static int process_one_qio(struct qcow2 *qcow2, struct qio *qio, bool avoid_metadata_changes)
> {
> struct qcow2_map map = { .qcow2 = qcow2, };
> bool write;
> + int ret;
>
> - if (!handle_metadata(qcow2, &qio, &map))
> - return;
> + ret = handle_metadata(qcow2, &qio, &map, avoid_metadata_changes);
> + if (!ret)
> + return 0;
> + else if (ret == QIO_NEEDS_MAIN_KWORK)
"else" here is redundant.
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
` (2 preceding siblings ...)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 3/7] dm-qcow2: prepare process_one_qio() for fastpath Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 12:02 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
` (3 subsequent siblings)
7 siblings, 2 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
dm-qcow2 was created to run non-concurrently, so qio never had
per work struct field. Let's add one and use it to queue first
dm-qcow2 pipeline step: qio preparation. It requires no metadata,
so everything is simple here.
We would later move more pipeline steps into qio.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: updated QLIST_EMBEDDED comment
drivers/md/dm-qcow2-map.c | 60 ++++++++++++++++++++++++++++-----------
drivers/md/dm-qcow2.h | 12 ++++++--
2 files changed, 54 insertions(+), 18 deletions(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index c495834a4d29f..7d220dd718021 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -362,6 +362,8 @@ struct qio *qcow2_alloc_qio(mempool_t *pool, bool zero)
}
ALLOW_ERROR_INJECTION(qcow2_alloc_qio, NULL);
+static void do_qio_work(struct work_struct *ws);
+
void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
{
qio->qcow2 = qcow2;
@@ -372,7 +374,9 @@ void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
qio->bi_io_vec = NULL;
qio->flags = 0;
qio->ref_index = REF_INDEX_INVALID;
+ qio->state = QIO_STATE_START;
atomic_set(&qio->remaining, 1);
+ INIT_WORK(&qio->worker, do_qio_work);
/*
* Initially set into BLK_STS_OK, while aio complete,
@@ -3798,10 +3802,10 @@ static void qio_init_iter_from_rq(struct qio *qio, struct request *rq)
qio->bi_iter.bi_bvec_done = 0;
}
-static void prepare_one_embedded_qio(struct qcow2 *qcow2, struct qio *qio,
- struct list_head *deferred_qios)
+static void prepare_one_embedded_qio(struct qio *qio, struct list_head *deferred_qios)
{
struct qcow2_rq *qrq = qio->endio_cb_data;
+ struct qcow2 *qcow2 = qio->qcow2;
struct request *rq = qrq->rq;
struct bio_vec *bvec = NULL;
LIST_HEAD(list);
@@ -3846,15 +3850,6 @@ static void prepare_one_embedded_qio(struct qcow2 *qcow2, struct qio *qio,
qio_endio(qio);
}
-static void process_embedded_qios(struct qcow2 *qcow2, struct list_head *qios,
- struct list_head *deferred_qios)
-{
- struct qio *qio;
-
- while ((qio = qio_list_pop(qios)))
- prepare_one_embedded_qio(qcow2, qio, deferred_qios);
-}
-
static void process_deferred_qios(struct qcow2 *qcow2, struct list_head *qios)
{
struct qio *qio;
@@ -4371,7 +4366,6 @@ static void process_seek_qios(struct qcow2 *qcow, struct list_head *qios)
void do_qcow2_work(struct work_struct *ws)
{
struct qcow2 *qcow2 = container_of(ws, struct qcow2, worker);
- LIST_HEAD(embedded_qios);
LIST_HEAD(deferred_qios);
LIST_HEAD(zread_qios);
LIST_HEAD(bwrite_qios);
@@ -4384,7 +4378,6 @@ void do_qcow2_work(struct work_struct *ws)
current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
spin_lock_irq(&qcow2->deferred_lock);
- list_splice_init(&qcow2->qios[QLIST_EMBEDDED], &embedded_qios);
list_splice_init(&qcow2->qios[QLIST_DEFERRED], &deferred_qios);
list_splice_init(&qcow2->qios[QLIST_ZREAD], &zread_qios);
list_splice_init(&qcow2->qios[QLIST_BMERGE_WRITE], &bwrite_qios);
@@ -4395,7 +4388,6 @@ void do_qcow2_work(struct work_struct *ws)
list_splice_init(&qcow2->qios[QLIST_SEEK], &seek_qios);
spin_unlock_irq(&qcow2->deferred_lock);
- process_embedded_qios(qcow2, &embedded_qios, &deferred_qios);
process_deferred_qios(qcow2, &deferred_qios);
process_compressed_read(&zread_qios, &cow_data_qios);
process_backward_merge_write(qcow2, &bwrite_qios);
@@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
if (blk_rq_bytes(rq)) {
queue_list_id = QLIST_EMBEDDED;
- worker = &qcow2->worker;
+ worker = &qio->worker;
} else {
WARN_ON_ONCE(qio->bi_op != REQ_OP_FLUSH);
queue_list_id = QLIST_FLUSH;
@@ -4535,7 +4527,10 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
qio->qcow2 = qcow2;
qio->queue_list_id = queue_list_id;
qio->ref_index = ref_index;
- list_add_tail(&qio->link, &qcow2->qios[qio->queue_list_id]);
+ if (queue_list_id == QLIST_EMBEDDED)
+ qio->state = QIO_STATE_START;
+ else
+ list_add_tail(&qio->link, &qcow2->qios[qio->queue_list_id]);
}
spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
@@ -4631,6 +4626,39 @@ static void handle_cleanup_mask(struct qio *qio)
}
}
+static void do_qio_work(struct work_struct *ws)
+{
+ struct qio *qio = container_of(ws, struct qio, worker);
+ struct qcow2 *qcow2 = qio->qcow2;
+ struct qcow2_target *tgt = qcow2->tgt;
+ unsigned int pflags = current->flags;
+
+ current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
+
+ switch (qio->state) {
+ case QIO_STATE_START: {
+ LIST_HEAD(deferred_qios);
+
+ prepare_one_embedded_qio(qio, &deferred_qios);
+ if (list_empty(&deferred_qios))
+ break;
+
+ spin_lock_irq(&qcow2->deferred_lock);
+ list_splice_tail(&deferred_qios, &qcow2->qios[QLIST_DEFERRED]);
+ spin_unlock_irq(&qcow2->deferred_lock);
+ queue_work(tgt->wq, &qcow2->worker);
+ break;
+ }
+
+ default:
+ qio->bi_status = BLK_STS_IOERR;
+ qio_endio(qio);
+ break;
+ }
+
+ current_restore_flags(pflags, PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO);
+}
+
struct qio_data_llseek_hole {
struct completion compl;
struct qio *higher;
diff --git a/drivers/md/dm-qcow2.h b/drivers/md/dm-qcow2.h
index ad5286b16883e..c044c630ca238 100644
--- a/drivers/md/dm-qcow2.h
+++ b/drivers/md/dm-qcow2.h
@@ -220,8 +220,10 @@ struct qcow2_target {
enum {
QLIST_EMBEDDED = 0, /*
- * List for initial setup embedded qios
- * related to prq (in process context).
+ * Only a queue_list_id, qios are never
+ * linked to this list: it marks embedded
+ * qios related to prq, which are set up
+ * outside main kwork.
* This is used only for top qcow2 image.
*/
QLIST_DEFERRED,
@@ -238,6 +240,10 @@ enum {
QLIST_INVALID = QLIST_COUNT,
};
+enum {
+ QIO_STATE_START = 0,
+};
+
struct qcow2 {
struct qcow2_target *tgt;
struct file *file;
@@ -371,6 +377,8 @@ struct qio {
struct qcow2 *qcow2;
qcow2_endio_t endio_cb;
void *endio_cb_data;
+ struct work_struct worker;
+ int state;
};
#define CLU_OFF(qcow2, pos) (pos & (qcow2->clu_size - 1))
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
@ 2026-11-10 12:02 ` no-reply
2026-11-11 15:34 ` Andrey Zhadchenko
2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 1 reply; 26+ messages in thread
From: no-reply @ 2026-11-10 12:02 UTC (permalink / raw)
To: khorenko, alexander.ivanov, Andrey Zhadchenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] dm-qcow2: concurrent qio preparers can deadlock on qio_pool under memory pressure
--
commit 1b2b6ed6f3d2b43e71cc7424d0502aa9998c0d28
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
dm-qcow2: add per-qio work infrastructure
This commit adds a work_struct and a state field to struct qio. It uses
them to queue the first pipeline step, qio preparation, as one work item
per data request on tgt->wq. Previously this step ran serialised inside the
single qcow2->worker.
Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
> --- a/drivers/md/dm-qcow2-map.c
> +++ b/drivers/md/dm-qcow2-map.c
[ ... ]
> @@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>
> if (blk_rq_bytes(rq)) {
> queue_list_id = QLIST_EMBEDDED;
> - worker = &qcow2->worker;
> + worker = &qio->worker;
> } else {
[ ... ]
> spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>
> if (queue)
> queue_work(tgt->wq, worker);
[Severity: Medium]
Can concurrent preparers deadlock on qio_pool under memory pressure?
Before this patch, preparation ran serialised inside qcow2->worker, so
only one preparer held a partial split list at a time. Now one work item
is queued per data request on the unbound tgt->wq, which has the default
max_active limit of 256.
Each work item runs:
do_qio_work()
prepare_one_embedded_qio()
qcow2_split_qio_to_list()
split_and_chain_qio()
qcow2_alloc_qio(qcow2->tgt->qio_pool, true)
mempool_alloc(GFP_NOIO)
The split qios stay on a local list and are only handed to QLIST_DEFERRED
in do_qio_work() after the whole request has been split:
> + prepare_one_embedded_qio(qio, &deferred_qios);
> + if (list_empty(&deferred_qios))
> + break;
The pool is created in alloc_qcow2_target() as:
tgt->qio_pool = mempool_create_kmalloc_pool(MIN_QIOS, sizeof(struct qio));
with 512 elements, shared by all preparers and the rest of the pipeline.
Up to 512 requests may be in flight (qrq_pool), and each request needs up
to request_size/clu_size splits (about 20 for a 64K cluster and a 1280K
request, and more for smaller clusters). Total demand can therefore exceed
512.
If kmalloc keeps failing under memory pressure, can several preparers each
hold part of the reserve while all of them sleep in mempool_alloc() waiting
for more? The only wake-up source is a free of a qio from a completed
request, and none of the blocked preparers' own splits can complete until
splitting finishes.
The later patches in the series do not appear to remove this. The final
state (d9084b95) drops the per-qio work_struct and runs do_qio_work() from
per-CPU qcow2_fast_work items on the same tgt->wq. That lowers the number
of preparers from up to 256 to one per CPU, but each preparer still builds
its full split list with blocking mempool_alloc() before releasing any of
it, so the hold-and-wait remains possible, mainly on many-core machines or
with small clusters.
Would it make sense to throttle or reserve pool space for preparers?
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=4
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
2026-11-10 12:02 ` no-reply
@ 2026-11-11 15:34 ` Andrey Zhadchenko
0 siblings, 0 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-11 15:34 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
On 11/10/26 13:02, no-reply@sashiko.virtuozzo.com wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] dm-qcow2: concurrent qio preparers can deadlock on qio_pool under memory pressure
> --
>
> commit 1b2b6ed6f3d2b43e71cc7424d0502aa9998c0d28
> Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
>
> dm-qcow2: add per-qio work infrastructure
>
> This commit adds a work_struct and a state field to struct qio. It uses
> them to queue the first pipeline step, qio preparation, as one work item
> per data request on tgt->wq. Previously this step ran serialised inside the
> single qcow2->worker.
>
> Link: https://virtuozzo.atlassian.net/browse/VSTOR-95494
>
>> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
>> --- a/drivers/md/dm-qcow2-map.c
>> +++ b/drivers/md/dm-qcow2-map.c
>
> [ ... ]
>
>> @@ -4519,7 +4511,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>>
>> if (blk_rq_bytes(rq)) {
>> queue_list_id = QLIST_EMBEDDED;
>> - worker = &qcow2->worker;
>> + worker = &qio->worker;
>> } else {
>
> [ ... ]
>
>> spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>>
>> if (queue)
>> queue_work(tgt->wq, worker);
>
> [Severity: Medium]
> Can concurrent preparers deadlock on qio_pool under memory pressure?
>
> Before this patch, preparation ran serialised inside qcow2->worker, so
> only one preparer held a partial split list at a time. Now one work item
> is queued per data request on the unbound tgt->wq, which has the default
> max_active limit of 256.
>
> Each work item runs:
>
> do_qio_work()
> prepare_one_embedded_qio()
> qcow2_split_qio_to_list()
> split_and_chain_qio()
> qcow2_alloc_qio(qcow2->tgt->qio_pool, true)
> mempool_alloc(GFP_NOIO)
>
> The split qios stay on a local list and are only handed to QLIST_DEFERRED
> in do_qio_work() after the whole request has been split:
>
>> + prepare_one_embedded_qio(qio, &deferred_qios);
>> + if (list_empty(&deferred_qios))
>> + break;
>
> The pool is created in alloc_qcow2_target() as:
>
> tgt->qio_pool = mempool_create_kmalloc_pool(MIN_QIOS, sizeof(struct qio));
>
> with 512 elements, shared by all preparers and the rest of the pipeline.
> Up to 512 requests may be in flight (qrq_pool), and each request needs up
> to request_size/clu_size splits (about 20 for a 64K cluster and a 1280K
> request, and more for smaller clusters). Total demand can therefore exceed
> 512.
>
> If kmalloc keeps failing under memory pressure, can several preparers each
> hold part of the reserve while all of them sleep in mempool_alloc() waiting
> for more? The only wake-up source is a free of a qio from a completed
> request, and none of the blocked preparers' own splits can complete until
> splitting finishes.
>
> The later patches in the series do not appear to remove this. The final
> state (d9084b95) drops the per-qio work_struct and runs do_qio_work() from
> per-CPU qcow2_fast_work items on the same tgt->wq. That lowers the number
> of preparers from up to 256 to one per CPU, but each preparer still builds
> its full split list with blocking mempool_alloc() before releasing any of
> it, so the hold-and-wait remains possible, mainly on many-core machines or
> with small clusters.
>
> Would it make sense to throttle or reserve pool space for preparers?
>
Even before the series amount of splits could exceed 512 and
theoretically exhaust this pool and sleep. Anyway mempool_alloc also has
kmalloc retry logic and other mitigation mechanisms. Not only this issue
is pre-existing but qio's are also quite small and if the node is out
of 0-order allocations we are likely doomed anyway.
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
2026-11-10 12:02 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:26 +0300, Andrey Zhadchenko wrote:
> + prepare_one_embedded_qio(qio, &deferred_qios);
Item 4 of the cover letter reply: answer Sashiko's qio_pool remark.
Why: preparation used to run in main kwork only, one request at a
time. Now several preparers each allocate all splits of their request
from tgt->qio_pool before handing any of them on. Under memory
pressure they can share out the reserve and all sleep in
mempool_alloc() waiting for each other. Say why this cannot stall, or
allocate the splits without sleeping first and serialize the sleeping
fallback, as dm-crypt does.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
` (3 preceding siblings ...)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 4/7] dm-qcow2: add per-qio work infrastructure Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 12:08 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
` (2 subsequent siblings)
7 siblings, 2 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
Each qio is handled by its own work and process_one_qio() can now
work with qios not requiring metadata. Try to process qios right
after splitting. Insert all leftovers into respective lists for
do_qcow2_work(), keeping the order of the splits.
From here on process_one_qio() runs in parallel with do_qcow2_work(),
so a write may reach perform_rw_mapped() after main kwork switched
its L2 page from DIRTY to WRITEBACK. perform_rw_mapped() parks such
a write until the writeback ends.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: list_add_tail() keeps the order of deferred splits; commit message
mentions the race with md writeback; fixed "it's" typo
drivers/md/dm-qcow2-map.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index 7d220dd718021..7a8deba4b0a83 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -4626,9 +4626,18 @@ static void handle_cleanup_mask(struct qio *qio)
}
}
+static void qio_try_fast(struct qio *qio, struct list_head *deferred_qios)
+{
+ list_del(&qio->link);
+
+ if (process_one_qio(qio->qcow2, qio, true))
+ list_add_tail(&qio->link, deferred_qios);
+}
+
static void do_qio_work(struct work_struct *ws)
{
struct qio *qio = container_of(ws, struct qio, worker);
+ struct qio *i, *next;
struct qcow2 *qcow2 = qio->qcow2;
struct qcow2_target *tgt = qcow2->tgt;
unsigned int pflags = current->flags;
@@ -4638,8 +4647,12 @@ static void do_qio_work(struct work_struct *ws)
switch (qio->state) {
case QIO_STATE_START: {
LIST_HEAD(deferred_qios);
+ LIST_HEAD(split_qios);
+
+ prepare_one_embedded_qio(qio, &split_qios);
+ list_for_each_entry_safe(i, next, &split_qios, link)
+ qio_try_fast(i, &deferred_qios);
- prepare_one_embedded_qio(qio, &deferred_qios);
if (list_empty(&deferred_qios))
break;
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
@ 2026-11-10 12:08 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: no-reply @ 2026-11-10 12:08 UTC (permalink / raw)
To: Andrey Zhadchenko, alexander.ivanov, khorenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] dm-qcow2: stale data_clu_pos in fast path can write to a freed or reused cluster
--
Commit 34396f8c5df05ca113c51e62c4510b07f74a9fdc
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
dm-qcow2: implement per-qio fast processing
This patch makes do_qio_work() try to process each split qio immediately
with process_one_qio(..., true). Qios that need metadata changes go onto
the deferred list for do_qcow2_work(). As a result, parse and submit now
run in parallel with the main kwork.
> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
> --- a/drivers/md/dm-qcow2-map.c
> +++ b/drivers/md/dm-qcow2-map.c
[ ... ]
> +static void qio_try_fast(struct qio *qio, struct list_head *deferred_qios)
> +{
> + list_del(&qio->link);
> +
> + if (process_one_qio(qio->qcow2, qio, true))
> + list_add_tail(&qio->link, deferred_qios);
> +}
[Severity: High]
Can a write submitted from here go to a cluster that has been freed or
reused?
process_one_qio() runs handle_metadata() -> parse_metadata(), which
drops md_pages_lock and caches the result:
unlock:
spin_unlock_irq(&qcow2->md_pages_lock);
if (ret <= 0) /* Err, delayed, DATA is not allocated, or zero read */
return ret;
map->data_clu_pos = pos;
The L2 entry is not re-validated after this point. Before this patch,
parse and submit ran sequentially in main kwork, so main kwork could not
change the index between the two steps.
The recheck added for the race with writeback in perform_rw_mapped()
looks only at MD_WRITEBACK, and at MD_DIRTY with the changed bit set:
spin_lock_irqsave(&qcow2->md_pages_lock, flags);
if (__delay_if_writeback(qcow2, md, index_in_page, &qio, false)) {
...
}
if ((md->status & MD_DIRTY) &&
test_bit(index_in_page, md->wbd->changed_indexes)) {
md->wbd->nr_submitted++;
qio->data = md->wbd;
}
spin_unlock_irqrestore(&qcow2->md_pages_lock, flags);
...
map_and_submit_rw(qcow2, map->data_clu_pos, qio);
It does not call delay_if_locked() and does not re-read the L2 entry. If
a discard's L2 update is DIRTY, the write is only attached to wbd and
still goes to the old data_clu_pos. If the discard's writeback has
already completed, the write goes to the stale position with no check.
On the main kwork side, the discard sequence is
prepare_cluster_discard() -> process_indexes_write() ->
process_indexes_end(). The last step waits only for wpc readers and then
calls dec_cluster_usage(), which frees the cluster for reuse. It does not
wait for fast-path writes that parsed the old mapping before the index
was locked.
A possible sequence:
fast worker: parse_metadata() caches data_clu_pos
(worker is delayed, for example by preemption)
main kwork: discard locks the index, rewrites the L2 entry, completes
writeback, and process_indexes_end() frees the cluster
fast worker: perform_rw_mapped() -> map_and_submit_rw() writes to the
old data_clu_pos, which may now belong to other data
Is there an inflight ref, an index-lock recheck, or a wait for in-flight
writes that closes this window? I could not find one. It also looks like
the later patches in the series do not close it: one only batches the
do_qio_work() calls per CPU, and the other only sets DM_TARGET_NOWAIT.
The commit message says "a write may reach perform_rw_mapped() after
main kwork switched its L2 page from DIRTY to WRITEBACK. perform_rw_mapped()
parks such a write until the writeback ends." Is this the only race
introduced by running process_one_qio() in parallel with
do_qcow2_work()? The discard and cluster reuse race described above is
not mentioned.
> +
> static void do_qio_work(struct work_struct *ws)
> {
[ ... ]
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
2026-11-10 12:08 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
1 sibling, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:27 +0300, Andrey Zhadchenko wrote:
> >From here on process_one_qio() runs in parallel with do_qcow2_work(),
> so a write may reach perform_rw_mapped() after main kwork switched
> its L2 page from DIRTY to WRITEBACK. perform_rw_mapped() parks such
> a write until the writeback ends.
Item 3 of the cover letter reply:
- add a paragraph on the second window: between the parse and the data
submit main kwork can discard, free and reuse the cluster, so a
stale write, read or partial punch hits the new owner; say it is a
known limitation, fixed by a follow-up series, with its Jira issue;
- reword the first line so it does not start with "From": the mail
got it escaped to ">From", and that went into the commit.
Why: this is the patch where process_one_qio() starts running in
parallel with main kwork, and right now the message reads as if the
writeback race were the only one.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
` (4 preceding siblings ...)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 5/7] dm-qcow2: implement per-qio fast processing Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
` (2 more replies)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT Andrey Zhadchenko
2026-11-10 17:56 ` [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Konstantin Khorenko
7 siblings, 3 replies; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
queue_work() is pretty costly under high load. It takes pool->lock;
unbound pools do not track running workers (WORKER_UNBOUND is part
of WORKER_NOT_RUNNING), so an idle kworker is woken (idle exit + IPI +
context switch) even if a running worker would pick the work up right
away. Every qio has its own work, so this is paid for every I/O. With
several submitters this ends up in several context switches per I/O
and in contention on pool->lock.
Batch requests with percpu llists. Additionally track if the worker
is busy and queue_work() only if it is not. The worker processes one
batch per run and requeues itself if more qios came meanwhile, so
flush_workqueue(), e.g. on backward merge completion with I/O still
running, does not wait for a worker which is refilled all the time.
The fast path is now serial per submitting CPU: qios queued from one
CPU are processed one by one by its work, while qios from different
CPUs are still processed in parallel. This is fine, since a fast path
qio is only mapped and submitted as direct AIO: O_DIRECT is enforced
for every delta. The exception is a partial discard, which punches
the hole synchronously in the worker, so the qios queued after it on
that CPU wait. Most of them would wait anyway: on ext4 and xfs punch
hole takes the inode lock exclusively, and direct I/O submission to
the same image sleeps on it.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: moved before "declare DM_TARGET_NOWAIT"; process one batch per run
and requeue, so flush_workqueue() does not wait for a work which is
refilled all the time; commit message: fixed the claim about
re-queued qios, describes that the fast path is serial per
submitting CPU and that partial discards punch the hole in the
worker, measurements moved to the thread
drivers/md/dm-qcow2-map.c | 100 +++++++++++++++++++++++++++++++----
drivers/md/dm-qcow2-target.c | 10 +++-
drivers/md/dm-qcow2.h | 6 ++-
3 files changed, 103 insertions(+), 13 deletions(-)
diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
index 7a8deba4b0a83..a87a396af56d0 100644
--- a/drivers/md/dm-qcow2-map.c
+++ b/drivers/md/dm-qcow2-map.c
@@ -87,6 +87,7 @@ static void process_read_qio(struct qcow2 *qcow2, struct qio *qio,
struct qcow2_map *map);
static void init_qrq_and_embedded_qio(struct qcow2_target *tgt, struct request *rq,
struct qcow2_rq *qrq, struct qio *qio);
+static void qcow2_fast_submit(struct qcow2_target *tgt, struct qio *qio);
static loff_t bytes_off_in_cluster(struct qcow2 *qcow2, struct qio *qio)
{
@@ -362,8 +363,6 @@ struct qio *qcow2_alloc_qio(mempool_t *pool, bool zero)
}
ALLOW_ERROR_INJECTION(qcow2_alloc_qio, NULL);
-static void do_qio_work(struct work_struct *ws);
-
void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
{
qio->qcow2 = qcow2;
@@ -376,7 +375,6 @@ void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
qio->ref_index = REF_INDEX_INVALID;
qio->state = QIO_STATE_START;
atomic_set(&qio->remaining, 1);
- INIT_WORK(&qio->worker, do_qio_work);
/*
* Initially set into BLK_STS_OK, while aio complete,
@@ -4511,7 +4509,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
if (blk_rq_bytes(rq)) {
queue_list_id = QLIST_EMBEDDED;
- worker = &qio->worker;
+ worker = NULL;
} else {
WARN_ON_ONCE(qio->bi_op != REQ_OP_FLUSH);
queue_list_id = QLIST_FLUSH;
@@ -4534,8 +4532,13 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
}
spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
- if (queue)
+ if (!queue)
+ return;
+
+ if (worker)
queue_work(tgt->wq, worker);
+ else
+ qcow2_fast_submit(tgt, qio);
}
void qcow2_submit_embedded_qios(struct qcow2_target *tgt, struct list_head *list)
@@ -4634,15 +4637,11 @@ static void qio_try_fast(struct qio *qio, struct list_head *deferred_qios)
list_add_tail(&qio->link, deferred_qios);
}
-static void do_qio_work(struct work_struct *ws)
+static void do_qio_work(struct qio *qio)
{
- struct qio *qio = container_of(ws, struct qio, worker);
- struct qio *i, *next;
struct qcow2 *qcow2 = qio->qcow2;
struct qcow2_target *tgt = qcow2->tgt;
- unsigned int pflags = current->flags;
-
- current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
+ struct qio *i, *next;
switch (qio->state) {
case QIO_STATE_START: {
@@ -4668,8 +4667,87 @@ static void do_qio_work(struct work_struct *ws)
qio_endio(qio);
break;
}
+}
+
+struct qcow2_fast_cpu {
+ struct llist_head qios;
+ struct work_struct work;
+ struct qcow2_target *tgt;
+ atomic_t busy;
+};
+
+static void qcow2_fast_submit(struct qcow2_target *tgt, struct qio *qio)
+{
+ struct qcow2_fast_cpu *fc = get_cpu_ptr(tgt->fast);
+
+ llist_add(&qio->fast_link, &fc->qios);
+ /* Pairs with smp_mb() in qcow2_fast_work() */
+ smp_mb();
+ if (!atomic_read(&fc->busy) && atomic_cmpxchg(&fc->busy, 0, 1) == 0)
+ queue_work(tgt->wq, &fc->work);
+ put_cpu_ptr(tgt->fast);
+}
+
+static void qcow2_fast_work(struct work_struct *work)
+{
+ struct qcow2_fast_cpu *fc = container_of(work, struct qcow2_fast_cpu, work);
+ unsigned int pflags = current->flags;
+ struct llist_node *node;
+ struct qio *qio, *next;
+
+ current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
+
+ node = llist_del_all(&fc->qios);
+ node = llist_reverse_order(node);
+ llist_for_each_entry_safe(qio, next, node, fast_link)
+ do_qio_work(qio);
current_restore_flags(pflags, PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO);
+
+ if (llist_empty(&fc->qios)) {
+ atomic_set(&fc->busy, 0);
+ /* just to be sure we look at the list after busy is posted */
+ smp_mb();
+ if (llist_empty(&fc->qios))
+ return;
+ if (atomic_cmpxchg(&fc->busy, 0, 1) != 0)
+ return;
+ }
+ queue_work(fc->tgt->wq, &fc->work);
+}
+
+void qcow2_fast_exit(struct qcow2_target *tgt)
+{
+ struct qcow2_fast_cpu *fc;
+ int cpu;
+
+ if (!tgt->fast)
+ return;
+ for_each_possible_cpu(cpu) {
+ fc = per_cpu_ptr(tgt->fast, cpu);
+ WARN_ON_ONCE(!llist_empty(&fc->qios));
+ }
+ free_percpu(tgt->fast);
+ tgt->fast = NULL;
+}
+
+int qcow2_fast_init(struct qcow2_target *tgt)
+{
+ struct qcow2_fast_cpu *fc;
+ int cpu;
+
+ tgt->fast = alloc_percpu(struct qcow2_fast_cpu);
+ if (!tgt->fast)
+ return -ENOMEM;
+ for_each_possible_cpu(cpu) {
+ fc = per_cpu_ptr(tgt->fast, cpu);
+ init_llist_head(&fc->qios);
+ INIT_WORK(&fc->work, qcow2_fast_work);
+ fc->tgt = tgt;
+ atomic_set(&fc->busy, 0);
+ }
+
+ return 0;
}
struct qio_data_llseek_hole {
diff --git a/drivers/md/dm-qcow2-target.c b/drivers/md/dm-qcow2-target.c
index 8337efc5c6238..3e85de5c8b050 100644
--- a/drivers/md/dm-qcow2-target.c
+++ b/drivers/md/dm-qcow2-target.c
@@ -253,6 +253,7 @@ static void qcow2_tgt_destroy(struct qcow2_target *tgt)
/* Now kill the queue */
destroy_workqueue(tgt->wq);
}
+ qcow2_fast_exit(tgt);
qcow2_merge_backward_cancel(tgt);
mempool_destroy(tgt->qio_pool);
@@ -477,6 +478,11 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
goto out_pool;
}
+ if (qcow2_fast_init(tgt)) {
+ ti->error = "Can't alloc fast path slots";
+ goto out_wq;
+ }
+
for (i = 0; i < 2; i++) {
release = i ? inflight_ref_exit1 : inflight_ref_exit0;
if (percpu_ref_init(&tgt->inflight_ref[i], release,
@@ -484,7 +490,7 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
if (i)
percpu_ref_exit(&tgt->inflight_ref[0]);
ti->error = "could not alloc percpu_ref";
- goto out_wq;
+ goto out_fast;
}
}
@@ -504,6 +510,8 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
qcow2_set_service_operations(ti, false);
return tgt;
+out_fast:
+ qcow2_fast_exit(tgt);
out_wq:
destroy_workqueue(tgt->wq);
out_pool:
diff --git a/drivers/md/dm-qcow2.h b/drivers/md/dm-qcow2.h
index c044c630ca238..d6884eba2f9de 100644
--- a/drivers/md/dm-qcow2.h
+++ b/drivers/md/dm-qcow2.h
@@ -4,6 +4,7 @@
#include <linux/percpu-refcount.h>
#include <linux/device-mapper.h>
+#include <linux/llist.h>
#include <linux/fs.h>
#include <linux/eventfd.h>
@@ -192,6 +193,7 @@ struct qcow2_target {
*/
struct qcow2 *top;
struct workqueue_struct *wq;
+ struct qcow2_fast_cpu __percpu *fast;
struct completion inflight_ref_comp;
struct percpu_ref inflight_ref[2];
@@ -377,7 +379,7 @@ struct qio {
struct qcow2 *qcow2;
qcow2_endio_t endio_cb;
void *endio_cb_data;
- struct work_struct worker;
+ struct llist_node fast_link;
int state;
};
@@ -394,6 +396,8 @@ loff_t qcow2_llseek_hole(struct dm_target *ti, loff_t offset, int whence);
void do_qcow2_work(struct work_struct *ws);
void do_qcow2_fsync_work(struct work_struct *ws);
+int qcow2_fast_init(struct qcow2_target *tgt);
+void qcow2_fast_exit(struct qcow2_target *tgt);
int qcow2_alloc_and_insert_md_page(struct qcow2 *qcow2, u64 index,
struct md_page **md);
struct md_page *qcow2_md_page_find_or_postpone(struct qcow2 *qcow2,
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
@ 2026-11-10 12:06 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 15:00 ` Alexander Ivanov
2 siblings, 0 replies; 26+ messages in thread
From: no-reply @ 2026-11-10 12:06 UTC (permalink / raw)
To: alexander.ivanov, khorenko, Andrey Zhadchenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] dm-qcow2: commit message omits sleeping steps run serially in fast-path worker
--
Commit 6cf4c03ea626fb7fe01c889ccd2361694ab98cf0
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
dm-qcow2: process fast path qios in per-cpu batches on a workqueue
This replaces the per-qio work item on the fast path with a per-CPU llist
and a single per-CPU work item. The worker drains one batch per run and
requeues itself if more qios arrived meanwhile.
> The fast path is now serial per submitting CPU: qios queued from one
> CPU are processed one by one by its work, while qios from different
> CPUs are still processed in parallel. This is fine, since a fast path
> qio is only mapped and submitted as direct AIO: O_DIRECT is enforced
> for every delta. The exception is a partial discard, which punches
> the hole synchronously in the worker, so the qios queued after it on
> that CPU wait.
[Severity: Low]
This isn't a bug in the code, but the commit message looks incomplete.
It says a fast path qio is "only mapped and submitted as direct AIO" and
names only a partial discard as an exception. Is that accurate?
qcow2_fast_work() runs the whole batch serially:
> + node = llist_del_all(&fc->qios);
> + node = llist_reverse_order(node);
> + llist_for_each_entry_safe(qio, next, node, fast_link)
> + do_qio_work(qio);
For QIO_STATE_START, do_qio_work() calls prepare_one_embedded_qio(). For
multi-bio requests this reaches create_bvec_from_rq(), which does:
bvec = kvmalloc_array(nr_bvec, sizeof(struct bio_vec), GFP_NOIO);
The qio split path also allocates through qcow2_alloc_qio(), which calls
mempool_alloc(GFP_NOIO). Both can sleep under memory pressure, and while
they do, every qio queued behind on that CPU is delayed.
The direct AIO submission via qcow2_call_rw_iter() can also block on the
inode lock. The partial discard case reaches issue_discard(), which calls
qcow2_punch_hole() synchronously, and the commit message already mentions
this.
Could the commit message list these other sleeping steps, or state that
only the discard case is the notable one and explain why the others are
acceptable?
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110114629.416695-1-andrey.zhadchenko@virtuozzo.com?part=6
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
@ 2026-11-10 17:56 ` Konstantin Khorenko
2026-11-11 15:00 ` Alexander Ivanov
2 siblings, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:28 +0300, Andrey Zhadchenko wrote:
> CPUs are still processed in parallel. This is fine, since a fast path
> qio is only mapped and submitted as direct AIO: O_DIRECT is enforced
> for every delta. The exception is a partial discard, which punches
Item 4 of the cover letter reply: answer Sashiko's remark that
preparation sleeps too, or add it here.
Why: before the qio is mapped, prepare_one_embedded_qio() allocates
the bvec array with kvmalloc_array(GFP_NOIO) and the splits with
mempool_alloc(), and both sleep under memory pressure. Everything
queued after it on that CPU waits. The batching code itself is
correct.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
2026-11-10 17:56 ` Konstantin Khorenko
@ 2026-11-11 15:00 ` Alexander Ivanov
2 siblings, 0 replies; 26+ messages in thread
From: Alexander Ivanov @ 2026-11-11 15:00 UTC (permalink / raw)
To: Andrey Zhadchenko, khorenko; +Cc: devel
On 11/10/26 12:46, Andrey Zhadchenko wrote:
> queue_work() is pretty costly under high load. It takes pool->lock;
> unbound pools do not track running workers (WORKER_UNBOUND is part
> of WORKER_NOT_RUNNING), so an idle kworker is woken (idle exit + IPI +
> context switch) even if a running worker would pick the work up right
> away. Every qio has its own work, so this is paid for every I/O. With
> several submitters this ends up in several context switches per I/O
> and in contention on pool->lock.
>
> Batch requests with percpu llists. Additionally track if the worker
> is busy and queue_work() only if it is not. The worker processes one
> batch per run and requeues itself if more qios came meanwhile, so
> flush_workqueue(), e.g. on backward merge completion with I/O still
> running, does not wait for a worker which is refilled all the time.
>
> The fast path is now serial per submitting CPU: qios queued from one
> CPU are processed one by one by its work, while qios from different
> CPUs are still processed in parallel. This is fine, since a fast path
> qio is only mapped and submitted as direct AIO: O_DIRECT is enforced
> for every delta. The exception is a partial discard, which punches
> the hole synchronously in the worker, so the qios queued after it on
> that CPU wait. Most of them would wait anyway: on ext4 and xfs punch
> hole takes the inode lock exclusively, and direct I/O submission to
> the same image sleeps on it.
>
> https://virtuozzo.atlassian.net/browse/VSTOR-95494
> Feature: dm-qcow2: block device over QCOW2 files driver
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
> ---
> v2: moved before "declare DM_TARGET_NOWAIT"; process one batch per run
> and requeue, so flush_workqueue() does not wait for a work which is
> refilled all the time; commit message: fixed the claim about
> re-queued qios, describes that the fast path is serial per
> submitting CPU and that partial discards punch the hole in the
> worker, measurements moved to the thread
>
> drivers/md/dm-qcow2-map.c | 100 +++++++++++++++++++++++++++++++----
> drivers/md/dm-qcow2-target.c | 10 +++-
> drivers/md/dm-qcow2.h | 6 ++-
> 3 files changed, 103 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/md/dm-qcow2-map.c b/drivers/md/dm-qcow2-map.c
> index 7a8deba4b0a83..a87a396af56d0 100644
> --- a/drivers/md/dm-qcow2-map.c
> +++ b/drivers/md/dm-qcow2-map.c
> @@ -87,6 +87,7 @@ static void process_read_qio(struct qcow2 *qcow2, struct qio *qio,
> struct qcow2_map *map);
> static void init_qrq_and_embedded_qio(struct qcow2_target *tgt, struct request *rq,
> struct qcow2_rq *qrq, struct qio *qio);
> +static void qcow2_fast_submit(struct qcow2_target *tgt, struct qio *qio);
>
> static loff_t bytes_off_in_cluster(struct qcow2 *qcow2, struct qio *qio)
> {
> @@ -362,8 +363,6 @@ struct qio *qcow2_alloc_qio(mempool_t *pool, bool zero)
> }
> ALLOW_ERROR_INJECTION(qcow2_alloc_qio, NULL);
>
> -static void do_qio_work(struct work_struct *ws);
> -
> void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
> {
> qio->qcow2 = qcow2;
> @@ -376,7 +375,6 @@ void qcow2_init_qio(struct qio *qio, unsigned int bi_op, struct qcow2 *qcow2)
> qio->ref_index = REF_INDEX_INVALID;
> qio->state = QIO_STATE_START;
> atomic_set(&qio->remaining, 1);
> - INIT_WORK(&qio->worker, do_qio_work);
>
> /*
> * Initially set into BLK_STS_OK, while aio complete,
> @@ -4511,7 +4509,7 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
>
> if (blk_rq_bytes(rq)) {
> queue_list_id = QLIST_EMBEDDED;
> - worker = &qio->worker;
> + worker = NULL;
> } else {
> WARN_ON_ONCE(qio->bi_op != REQ_OP_FLUSH);
> queue_list_id = QLIST_FLUSH;
> @@ -4534,8 +4532,13 @@ static void qcow2_submit_embedded_qio(struct qcow2_target *tgt, struct qio *qio)
> }
> spin_unlock_irqrestore(&qcow2->deferred_lock, flags);
>
> - if (queue)
> + if (!queue)
> + return;
> +
> + if (worker)
> queue_work(tgt->wq, worker);
> + else
> + qcow2_fast_submit(tgt, qio);
> }
>
> void qcow2_submit_embedded_qios(struct qcow2_target *tgt, struct list_head *list)
> @@ -4634,15 +4637,11 @@ static void qio_try_fast(struct qio *qio, struct list_head *deferred_qios)
> list_add_tail(&qio->link, deferred_qios);
> }
>
> -static void do_qio_work(struct work_struct *ws)
> +static void do_qio_work(struct qio *qio)
> {
> - struct qio *qio = container_of(ws, struct qio, worker);
> - struct qio *i, *next;
> struct qcow2 *qcow2 = qio->qcow2;
> struct qcow2_target *tgt = qcow2->tgt;
> - unsigned int pflags = current->flags;
> -
> - current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
> + struct qio *i, *next;
>
> switch (qio->state) {
> case QIO_STATE_START: {
> @@ -4668,8 +4667,87 @@ static void do_qio_work(struct work_struct *ws)
> qio_endio(qio);
> break;
> }
> +}
> +
> +struct qcow2_fast_cpu {
> + struct llist_head qios;
> + struct work_struct work;
> + struct qcow2_target *tgt;
> + atomic_t busy;
> +};
> +
> +static void qcow2_fast_submit(struct qcow2_target *tgt, struct qio *qio)
> +{
> + struct qcow2_fast_cpu *fc = get_cpu_ptr(tgt->fast);
> +
> + llist_add(&qio->fast_link, &fc->qios);
> + /* Pairs with smp_mb() in qcow2_fast_work() */
> + smp_mb();
If I’m not mistaken, the try_cmpxchg() in llist_add() is already fully
ordered, so the smp_mb() isn’t needed.
> + if (!atomic_read(&fc->busy) && atomic_cmpxchg(&fc->busy, 0, 1) == 0)
> + queue_work(tgt->wq, &fc->work);
> + put_cpu_ptr(tgt->fast);
> +}
> +
> +static void qcow2_fast_work(struct work_struct *work)
> +{
> + struct qcow2_fast_cpu *fc = container_of(work, struct qcow2_fast_cpu, work);
> + unsigned int pflags = current->flags;
> + struct llist_node *node;
> + struct qio *qio, *next;
> +
> + current->flags |= PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO;
> +
> + node = llist_del_all(&fc->qios);
> + node = llist_reverse_order(node);
> + llist_for_each_entry_safe(qio, next, node, fast_link)
> + do_qio_work(qio);
>
> current_restore_flags(pflags, PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO);
> +
> + if (llist_empty(&fc->qios)) {
> + atomic_set(&fc->busy, 0);
> + /* just to be sure we look at the list after busy is posted */
> + smp_mb();
> + if (llist_empty(&fc->qios))
> + return;
> + if (atomic_cmpxchg(&fc->busy, 0, 1) != 0)
> + return;
> + }
> + queue_work(fc->tgt->wq, &fc->work);
> +}
> +
> +void qcow2_fast_exit(struct qcow2_target *tgt)
> +{
> + struct qcow2_fast_cpu *fc;
> + int cpu;
> +
> + if (!tgt->fast)
> + return;
> + for_each_possible_cpu(cpu) {
> + fc = per_cpu_ptr(tgt->fast, cpu);
> + WARN_ON_ONCE(!llist_empty(&fc->qios));
> + }
> + free_percpu(tgt->fast);
> + tgt->fast = NULL;
> +}
> +
> +int qcow2_fast_init(struct qcow2_target *tgt)
> +{
> + struct qcow2_fast_cpu *fc;
> + int cpu;
> +
> + tgt->fast = alloc_percpu(struct qcow2_fast_cpu);
> + if (!tgt->fast)
> + return -ENOMEM;
> + for_each_possible_cpu(cpu) {
> + fc = per_cpu_ptr(tgt->fast, cpu);
> + init_llist_head(&fc->qios);
> + INIT_WORK(&fc->work, qcow2_fast_work);
> + fc->tgt = tgt;
> + atomic_set(&fc->busy, 0);
> + }
> +
> + return 0;
> }
>
> struct qio_data_llseek_hole {
> diff --git a/drivers/md/dm-qcow2-target.c b/drivers/md/dm-qcow2-target.c
> index 8337efc5c6238..3e85de5c8b050 100644
> --- a/drivers/md/dm-qcow2-target.c
> +++ b/drivers/md/dm-qcow2-target.c
> @@ -253,6 +253,7 @@ static void qcow2_tgt_destroy(struct qcow2_target *tgt)
> /* Now kill the queue */
> destroy_workqueue(tgt->wq);
> }
> + qcow2_fast_exit(tgt);
> qcow2_merge_backward_cancel(tgt);
>
> mempool_destroy(tgt->qio_pool);
> @@ -477,6 +478,11 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
> goto out_pool;
> }
>
> + if (qcow2_fast_init(tgt)) {
> + ti->error = "Can't alloc fast path slots";
> + goto out_wq;
> + }
> +
> for (i = 0; i < 2; i++) {
> release = i ? inflight_ref_exit1 : inflight_ref_exit0;
> if (percpu_ref_init(&tgt->inflight_ref[i], release,
> @@ -484,7 +490,7 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
> if (i)
> percpu_ref_exit(&tgt->inflight_ref[0]);
> ti->error = "could not alloc percpu_ref";
> - goto out_wq;
> + goto out_fast;
> }
> }
>
> @@ -504,6 +510,8 @@ static struct qcow2_target *alloc_qcow2_target(struct dm_target *ti)
> qcow2_set_service_operations(ti, false);
>
> return tgt;
> +out_fast:
> + qcow2_fast_exit(tgt);
> out_wq:
> destroy_workqueue(tgt->wq);
> out_pool:
> diff --git a/drivers/md/dm-qcow2.h b/drivers/md/dm-qcow2.h
> index c044c630ca238..d6884eba2f9de 100644
> --- a/drivers/md/dm-qcow2.h
> +++ b/drivers/md/dm-qcow2.h
> @@ -4,6 +4,7 @@
>
> #include <linux/percpu-refcount.h>
> #include <linux/device-mapper.h>
> +#include <linux/llist.h>
> #include <linux/fs.h>
> #include <linux/eventfd.h>
>
> @@ -192,6 +193,7 @@ struct qcow2_target {
> */
> struct qcow2 *top;
> struct workqueue_struct *wq;
> + struct qcow2_fast_cpu __percpu *fast;
>
> struct completion inflight_ref_comp;
> struct percpu_ref inflight_ref[2];
> @@ -377,7 +379,7 @@ struct qio {
> struct qcow2 *qcow2;
> qcow2_endio_t endio_cb;
> void *endio_cb_data;
> - struct work_struct worker;
> + struct llist_node fast_link;
> int state;
> };
>
> @@ -394,6 +396,8 @@ loff_t qcow2_llseek_hole(struct dm_target *ti, loff_t offset, int whence);
>
> void do_qcow2_work(struct work_struct *ws);
> void do_qcow2_fsync_work(struct work_struct *ws);
> +int qcow2_fast_init(struct qcow2_target *tgt);
> +void qcow2_fast_exit(struct qcow2_target *tgt);
> int qcow2_alloc_and_insert_md_page(struct qcow2 *qcow2, u64 index,
> struct md_page **md);
> struct md_page *qcow2_md_page_find_or_postpone(struct qcow2 *qcow2,
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
` (5 preceding siblings ...)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 6/7] dm-qcow2: process fast path qios in per-cpu batches on a workqueue Andrey Zhadchenko
@ 2026-11-10 11:46 ` Andrey Zhadchenko
2026-11-10 12:06 ` no-reply
2026-11-10 17:56 ` [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Konstantin Khorenko
7 siblings, 1 reply; 26+ messages in thread
From: Andrey Zhadchenko @ 2026-11-10 11:46 UTC (permalink / raw)
To: khorenko, alexander.ivanov; +Cc: devel
dm-qcow2 is a request-based target: REQ_NOWAIT handling is done by blk-mq
(tag allocation) before a request reaches us, and clone_and_map_rq()
never blocks. Still, the target does not declare DM_TARGET_NOWAIT, so
dm_table_set_restrictions() clears BLK_FEAT_NOWAIT on the mapped device
and the block device file gets no FMODE_NOWAIT.
As a result io_uring cannot issue I/O in nowait mode and punts every
request to an io-wq worker thread. Declare DM_TARGET_NOWAIT, so io_uring
submits inline and saves a thread handoff per I/O.
https://virtuozzo.atlassian.net/browse/VSTOR-95494
Feature: dm-qcow2: block device over QCOW2 files driver
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: moved after "process fast path qios in per-cpu batches on a
workqueue"; measurements moved to the thread
drivers/md/dm-qcow2-target.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/md/dm-qcow2-target.c b/drivers/md/dm-qcow2-target.c
index 3e85de5c8b050..79a667d755f29 100644
--- a/drivers/md/dm-qcow2-target.c
+++ b/drivers/md/dm-qcow2-target.c
@@ -1171,7 +1171,7 @@ static void qcow2_resume(struct dm_target *ti)
static struct target_type qcow2_target = {
.name = "qcow2",
.version = {1, 0, 0},
- .features = DM_TARGET_SINGLETON,
+ .features = DM_TARGET_SINGLETON | DM_TARGET_NOWAIT,
.module = THIS_MODULE,
.ctr = qcow2_ctr,
.dtr = qcow2_dtr,
--
2.43.5
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path
2026-11-10 11:46 [Devel] [PATCH hci-8.1 v2 0/7] dm-qcow2: per-qio fast path Andrey Zhadchenko
` (6 preceding siblings ...)
2026-11-10 11:46 ` [Devel] [PATCH hci-8.1 v2 7/7] dm-qcow2: declare DM_TARGET_NOWAIT Andrey Zhadchenko
@ 2026-11-10 17:56 ` Konstantin Khorenko
7 siblings, 0 replies; 26+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:56 UTC (permalink / raw)
To: Andrey Zhadchenko; +Cc: devel
I won't apply this version, please send v3 with the changes below.
On Tue, 10 Nov 2026 14:46:22 +0300, Andrey Zhadchenko wrote:
> Andrey Zhadchenko (7):
> dm-qcow2: repeat md page lookup if it was inserted in parallel
> dm-qcow2: wait for L2 writeback in perform_rw_mapped()
> dm-qcow2: prepare process_one_qio() for fastpath
> dm-qcow2: add per-qio work infrastructure
> dm-qcow2: implement per-qio fast processing
> dm-qcow2: process fast path qios in per-cpu batches on a workqueue
> dm-qcow2: declare DM_TARGET_NOWAIT
All v1 items are done, and the DIRTY -> WRITEBACK race is closed.
What to change for v3:
1. Patch 2, code and commit message: in perform_rw_mapped(), under the
lock it already takes, check that the L2 entry (and the ext_l2
word) is still what parse_l2() saw and that the index is not
locked; if not, hand the qio back to main kwork. In the message,
"X freed" becomes "X leaked", and md->wbd is dropped not only by
main kwork.
2. New patch: a writable module parameter (0644, on by default) that
turns the fast path off. Check it with READ_ONCE() in qio_try_fast()
and send every split to the deferred list when it is off.
3. Patch 5 and the cover letter: describe the window between the fast
path parse and the data submit, where main kwork can discard, free
and reuse the cluster, as a known limitation fixed by a follow-up
series, and name its Jira issue. In patch 5, reword the line that
starts with "From here on": it got mbox-escaped to ">From".
4. Answer Sashiko's open remarks: patch 1 (unbounded retry, the third
time now; a bound of one retry is enough), patch 4 (qio_pool
hold-and-wait) and patch 6 (preparation sleeps too).
Separate patches, not needed for v3:
1. Writes and reads racing with a discard of the same cluster:
per-md write counters or discard locking, as you planned.
Details are in the replies to the patches.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 26+ messages in thread