All Virtuozzo development lists (kernel + QEMU)
 help / color / mirror / Atom feed
* [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error
@ 2026-11-17 13:24 Alexey Kuznetsov
  2026-11-17 13:25 ` [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext Alexey Kuznetsov
  2026-11-17 14:00 ` [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error Konstantin Khorenko
  0 siblings, 2 replies; 4+ messages in thread
From: Alexey Kuznetsov @ 2026-11-17 13:24 UTC (permalink / raw)
  To: devel, khorenko, kui.liu, azaitsev

swapext ioctl (like mainstream move_extent) has to have
unorthodox behaviour on error. It can be partial like write()
and then partial success must be returned to calling user.
Unlike write() ioctl really returns an error, but also
ioctl request struct is modified with partial work results.

It was forgotten, but did not affect existing user space
csd, because it aborted on any error of this kind.
But now we work on better handling of errors, both
recoverable and not-recoverable and found this missing element.

Affects: #VSTOR-148028

Signed-off-by: Alexey Kuznetsov <kuznet@virtuozzo.com>
---
 fs/ext4/move_extent.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c
index f08604816892..0e8d1366da7a 100644
--- a/fs/ext4/move_extent.c
+++ b/fs/ext4/move_extent.c
@@ -714,7 +714,7 @@ swap_one_extent(struct file *o_filp, struct inode *donor_inode,
 	struct inode *orig_inode = file_inode(o_filp);
 	handle_t *handle;
 	int jblocks, retries = 0;
-	int replaced_count = 0;
+	int rc = 0;
 	struct super_block *sb = orig_inode->i_sb;
 
 	/*
@@ -731,13 +731,13 @@ swap_one_extent(struct file *o_filp, struct inode *donor_inode,
 	handle = ext4_journal_start(orig_inode, EXT4_HT_MOVE_EXTENTS, jblocks);
 	if (IS_ERR(handle)) {
 		*err = PTR_ERR(handle);
-		return 0;
+		return rc;
 	}
 
 	ext4_double_down_write_data_sem(orig_inode, donor_inode);
-	replaced_count = ext4_swap_extents_2(handle, orig_inode, donor_inode,
-					   orig_blk, donor_blk, len,
-					   insn, err, se);
+	rc += ext4_swap_extents_2(handle, orig_inode, donor_inode,
+				  orig_blk + rc, donor_blk + rc, len - rc,
+				  insn, err, se);
 	ext4_double_up_write_data_sem(orig_inode, donor_inode);
 
 	se->s_dirty += handle->h_requested_credits - handle->h_total_credits;
@@ -752,7 +752,7 @@ swap_one_extent(struct file *o_filp, struct inode *donor_inode,
 	if (*err == -EBUSY && retries++ < 4 && EXT4_SB(sb)->s_journal &&
 	    jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal))
 		goto again;
-	return replaced_count;
+	return rc;
 }
 
 static int
@@ -762,7 +762,7 @@ expose_one_extent(struct inode *donor_inode,
 {
 	handle_t *handle;
 	int jblocks, retries = 0;
-	int replaced_count = 0;
+	int rc = 0;
 	struct super_block *sb = donor_inode->i_sb;
 
 again:
@@ -771,13 +771,13 @@ expose_one_extent(struct inode *donor_inode,
 	handle = ext4_journal_start(donor_inode, EXT4_HT_MOVE_EXTENTS, jblocks);
 	if (IS_ERR(handle)) {
 		*err = PTR_ERR(handle);
-		return 0;
+		return rc;
 	}
 
 	down_write(&EXT4_I(donor_inode)->i_data_sem);
-	replaced_count = ext4_expose_extents_2(handle, donor_inode,
-					       donor_blk, len,
-					       insn, err, se);
+	rc += ext4_expose_extents_2(handle, donor_inode,
+				    donor_blk + rc, len - rc,
+				    insn, err, se);
 	up_write(&EXT4_I(donor_inode)->i_data_sem);
 
 	se->s_dirty += handle->h_requested_credits - handle->h_total_credits;
@@ -791,7 +791,7 @@ expose_one_extent(struct inode *donor_inode,
 	if (*err == -EBUSY && retries++ < 4 && EXT4_SB(sb)->s_journal &&
 	    jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal))
 		goto again;
-	return replaced_count;
+	return rc;
 }
 
 static unsigned int count_credits(struct inode *inode, ext4_lblk_t start, ext4_lblk_t len,
@@ -974,12 +974,12 @@ ext4_ioc_swap_extents(struct file *o_filp, struct file *d_filp, __u64 orig_blk,
 			ret = 0;
 			rc = cur_len;
 		}
+		se->moved_len += rc;
 		if (ret < 0)
 			break;
 		o_start += rc;
 		d_start += rc;
 	}
-	se->moved_len = d_start - donor_blk;
 
 out:
 	if (se->moved_len) {
-- 
2.47.1

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext
@ 2026-11-17 13:25 ` Alexey Kuznetsov
  2026-11-17 14:00   ` Konstantin Khorenko
  0 siblings, 1 reply; 4+ messages in thread
From: Alexey Kuznetsov @ 2026-11-17 13:25 UTC (permalink / raw)
  To: devel, khorenko, kui.liu, azaitsev

ext4 (like any other journaling fs I guess) has a problem
with estimating journal reservations required for particular
fs operation. swapext is pretty complicated operation, its
exact volume depends on set of extents currently present
in the inodes. Even though existing estimate works, there
is no warranty under some circumstances calculated credits
are not enough to complete the transaction.
This patch suggests a safe way to adjust credits in case
initial estimate was wrong without fatal impact on journal:
it checks credits before each pass, extends the handle if needed,
and otherwise restarts the transaction and continues
from the last completed pass.

The patch is suggested by Claude. It looks reasonable,
the technique is borrowed from ext4_block_write_begin().

Affects: #VSTOR-148028

Signed-off-by: Alexey Kuznetsov <kuznet@virtuozzo.com>
---
 fs/ext4/extents.c     | 33 +++++++++++++++++++++++++++++++++
 fs/ext4/move_extent.c |  6 ++++++
 2 files changed, 39 insertions(+)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index eca3f0312ac5..6ffee24cf386 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5821,6 +5821,7 @@ ext4_swap_extents_2(handle_t *handle, struct inode *inode1,
 	struct ext4_ext_path *path1 = NULL;
 	struct ext4_ext_path *path2 = NULL;
 	int replaced_count = 0;
+	int passes = 0;
 
 	BUG_ON(!rwsem_is_locked(&EXT4_I(inode1)->i_data_sem));
 	BUG_ON(!rwsem_is_locked(&EXT4_I(inode2)->i_data_sem));
@@ -5834,6 +5835,26 @@ ext4_swap_extents_2(handle_t *handle, struct inode *inode1,
 		int e1_len, e2_len, len;
 		int split = 0;
 
+		/*
+		 * A pass either splits at most one extent in each inode or
+		 * swaps one pair of extents. Running out of credits in the
+		 * middle of a pass aborts the journal, so before each pass
+		 * (but the first one, which the caller has reserved for) make
+		 * sure the handle can afford the worst case. If it cannot,
+		 * return to the caller with -EAGAIN, it will restart the
+		 * transaction and continue from lblk1 + replaced_count.
+		 */
+		if (passes++) {
+			int credits = ext4_chunk_trans_blocks(inode1, 1) +
+				      ext4_chunk_trans_blocks(inode2, 1);
+			int ret = __ext4_journal_ensure_credits(handle, credits,
+								credits, 0);
+			if (ret) {
+				*erp = ret < 0 ? ret : -EAGAIN;
+				break;
+			}
+		}
+
 		path1 = ext4_find_extent(inode1, lblk1, NULL, EXT4_EX_NOCACHE);
 		if (IS_ERR(path1)) {
 			*erp = PTR_ERR(path1);
@@ -5993,6 +6014,7 @@ ext4_expose_extents_2(handle_t *handle, struct inode *inode,
 {
 	struct ext4_ext_path *path = NULL;
 	int replaced_count = 0;
+	int passes = 0;
 
 	ext4_es_remove_extent(inode, lblk, count);
 
@@ -6002,6 +6024,17 @@ ext4_expose_extents_2(handle_t *handle, struct inode *inode,
 		int e_len, len;
 		int split = 0;
 
+		/* See comment in ext4_swap_extents_2() */
+		if (passes++) {
+			int credits = ext4_chunk_trans_blocks(inode, 1);
+			int ret = __ext4_journal_ensure_credits(handle, credits,
+								credits, 0);
+			if (ret) {
+				*erp = ret < 0 ? ret : -EAGAIN;
+				break;
+			}
+		}
+
 		path = ext4_find_extent(inode, lblk, NULL, EXT4_EX_NOCACHE);
 		if (IS_ERR(path)) {
 			*erp = PTR_ERR(path);
diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c
index 0e8d1366da7a..dfbdb571105f 100644
--- a/fs/ext4/move_extent.c
+++ b/fs/ext4/move_extent.c
@@ -743,6 +743,9 @@ swap_one_extent(struct file *o_filp, struct inode *donor_inode,
 	se->s_dirty += handle->h_requested_credits - handle->h_total_credits;
 
 	ext4_journal_stop(handle);
+	/* Handle is out of credits, continue in a new transaction */
+	if (*err == -EAGAIN)
+		goto again;
 	if (*err == -ENOSPC &&
 	    ext4_should_retry_alloc(sb, &retries))
 		goto again;
@@ -782,6 +785,9 @@ expose_one_extent(struct inode *donor_inode,
 
 	se->s_dirty += handle->h_requested_credits - handle->h_total_credits;
 	ext4_journal_stop(handle);
+	/* Handle is out of credits, continue in a new transaction */
+	if (*err == -EAGAIN)
+		goto again;
 	if (*err == -ENOSPC &&
 	    ext4_should_retry_alloc(sb, &retries))
 		goto again;
-- 
2.47.1

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error
  2026-11-17 13:24 [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error Alexey Kuznetsov
  2026-11-17 13:25 ` [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext Alexey Kuznetsov
@ 2026-11-17 14:00 ` Konstantin Khorenko
  1 sibling, 0 replies; 4+ messages in thread
From: Konstantin Khorenko @ 2026-11-17 14:00 UTC (permalink / raw)
  To: Alexey Kuznetsov; +Cc: azaitsev, devel, kui.liu

I won't apply this version, please send v2 with the changes below.

This series has no cover letter, so the whole list is here; my reply
to 2/2 points to it.

1. Patch 1 message: say what the patch fixes in the kernel, not only
   the moved_len user space reads back:
   - a retry after -ENOSPC or -EBUSY restarted from orig_blk and
     swapped the done part back;
   - a short or zero moved_len on error left the page cache of the
     swapped blocks in place.
   Drop the "It was forgotten ... found this missing element"
   paragraph.
2. Patch 1: add Fixes: 1a3d440e7fbc ("fs/ext4: swapext ioctl").
3. Both patches: the tracker URL
   https://virtuozzo.atlassian.net/browse/VSTOR-148028 and
   "Feature: ext4: swapext ioctl: swap range of blocks between two
   files", as the earlier swapext commits have.
4. Patch 2 message: name the case where the estimate falls short and
   what the user sees then (the handle aborts, an ext4 error, the fs
   goes read-only), and say that the restart relies on patch 1. Drop
   "The patch is suggested by Claude. It looks reasonable," and
   "I guess"; "warranty" -> "guarantee".
5. Patch 2: one donor extent can now be swapped in two transactions.
   Please confirm that csd recovers from a crash between them, and
   say it in the message.
6. Patch 2 code comment: "continue from lblk1 + replaced_count" is not
   where the caller resumes: lblk1 has already been advanced by
   replaced_count at that point.

Separate patches, not needed for this series. I created a task for
each:

1. On ext4 without a journal swap_one_extent() and expose_one_extent()
   dereference the fake handle (handle->h_requested_credits) and
   oops.
   https://virtuozzo.atlassian.net/browse/VSTOR-155599
2. jblocks grows with the number of orig extents; on a fragmented
   range it goes above the jbd2 per-handle limit, which WARNs and
   fails the ioctl with -ENOSPC. With patch 2 it can be capped.
   https://virtuozzo.atlassian.net/browse/VSTOR-155600
3. ext4_ioc_swap_extents() does not call inode_dio_wait() on the two
   inodes, unlike ext4_move_extents(), and does not mark the swap
   fast-commit ineligible.
   https://virtuozzo.atlassian.net/browse/VSTOR-155601

On Tue, 17 Nov 2026 21:24:01 +0800, Alexey Kuznetsov wrote:
> It was forgotten, but did not affect existing user space
> csd, because it aborted on any error of this kind.
> But now we work on better handling of errors, both
> recoverable and not-recoverable and found this missing element.

1. The code does more than this says. Before the patch a retry in
swap_one_extent() started again at orig_blk, so the extents already
swapped were swapped back: without UNWRITE_DONOR the ioctl reported
the full moved_len while that part held the old data, with it the
retry failed with -ENODATA. And truncate_pagecache_range() is sized
by moved_len, so a short one left stale cached pages over swapped
blocks. Mainline fixed the latter for move_extent in 55583e899a53
("ext4: fix double-free of blocks due to wrong extents moved_len").

Thanks.

-- 
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext
  2026-11-17 13:25 ` [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext Alexey Kuznetsov
@ 2026-11-17 14:00   ` Konstantin Khorenko
  0 siblings, 0 replies; 4+ messages in thread
From: Konstantin Khorenko @ 2026-11-17 14:00 UTC (permalink / raw)
  To: Alexey Kuznetsov; +Cc: azaitsev, devel, kui.liu

I won't apply this version, please send v2 with the changes below.

On Tue, 17 Nov 2026 21:25:08 +0800, Alexey Kuznetsov wrote:
> The patch is suggested by Claude. It looks reasonable,
> the technique is borrowed from ext4_block_write_begin().

Items 3-6 of my reply to 1/2.

5. Before this patch one call of swap_one_extent() swapped a donor
extent in one transaction. When jbd2_journal_extend() refuses, the
rest now goes into the next transaction, so a crash between the two
commits leaves that extent half swapped.

> +		 * sure the handle can afford the worst case. If it cannot,
> +		 * return to the caller with -EAGAIN, it will restart the
> +		 * transaction and continue from lblk1 + replaced_count.

6. lblk1 is advanced together with replaced_count in the loop, so the
caller resumes at lblk1 itself, not at lblk1 + replaced_count.

-- 
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-17 14:01 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 13:24 [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error Alexey Kuznetsov
2026-11-17 13:25 ` [Devel] [PATCH hci-8.1 2/2] fs/ext4: ensure ext4 transaction credits while swapext Alexey Kuznetsov
2026-11-17 14:00   ` Konstantin Khorenko
2026-11-17 14:00 ` [Devel] [PATCH hci-8.1 1/2] fs/ext4: rectify behaviour of swapext ioctl on error Konstantin Khorenko

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.