[Devel] [PATCH hci-8.1 v2 0/6] dm/push_backup: event fd interface and fixes

[Devel] [PATCH hci-8.1 v2 0/6] dm/push_backup: event fd interface and fixes

[Andrey Zhadchenko]

The push_backup target currently makes the backup tool wait for the next
cluster range inside the "push_backup_read" target message. dm runs
target_message() under dm_get_live_table(), i.e. with md->io_barrier SRCU
read-locked, so while the tool sleeps there anything that needs
synchronize_srcu(&md->io_barrier) on that device is stuck: suspend, table
reload/remove and dm_internal_suspend_fast() from "@stats_create". An
unrelated dm-stats user hangs, and everything that later queues behind
md->suspend_lock hangs with it.

Patch 6 adds a file descriptor based interface instead. The tool gets an
anon inode fd via the "push_backup_get_fd" message and then uses plain
read()/write() of struct push_backup_range to fetch the next range and to
acknowledge it, without going through dm messages on the data path. The
fd pins struct push_backup with a kref so it survives target removal and
simply returns -ENODEV afterwards. The legacy messages are kept for
compatibility.

Patches 1-5 are fixes found while working on this:

  1. init the lock, waitqueue and timer before anything that can fail so
     pb_destroy() is safe from any error path;
  2. clear pb->alive under pb->lock like the other users do;
  3. drop the redundant mod_timer() in push_backup_write(). The timer
     re-arms itself from pb->deadline_jiffies, so updating the deadline
     under the lock is enough. This was also the only place that armed
     the timer outside the window push_backup_stop() waits for, which
     matters once writes may arrive via the fd without pb->ctl_rwsem;
  4. pass absolute expiry to mod_timer(). Both remaining callers passed
     a relative value, so after ~5 min of uptime the timer fired and
     re-armed on every tick while a write was delayed;
  5. arm the timer under pb->lock. pb_timer_func() and
     postpone_if_required_for_backup() both decided under the lock and
     armed after dropping it, so a stale S64_MAX could overwrite a fresh
     deadline and park the timer ~12 days out with delayed writes still
     pending. With arming done under the lock the RCU dance between
     postpone and stop is no longer needed and is removed.

v2:
  - 2/6: reworded the commit message, dropped the KCSAN claim.
  - 5/6: new patch: arm the deadline timer under pb->lock and drop the
    rcu_read_lock()/synchronize_rcu() between
    postpone_if_required_for_backup() and push_backup_stop().
  - 6/6: resubmit_pbios() now frees the pbio before
    dm_requeue_original_rq(). The requeue drops the md reference held by
    the in-flight request, after which __dm_destroy() may run .dtr and
    destroy pbio_pool under the fd writer's feet.

https://virtuozzo.atlassian.net/browse/VSTOR-152912

Andrey Zhadchenko (6):
  drivers/md/push_backup: correct initialization
  drivers/md/push_backup: clear pb->alive under pb->lock
  drivers/md/push_backup: do not re-arm deadline timer in
    push_backup_write()
  drivers/md/push_backup: fix mod_timer() usage
  drivers/md/push_backup: arm deadline timer under pb->lock
  dm/push_backup: add event file descriptor interface

 drivers/md/dm-push-backup.c         | 252 +++++++++++++++++++++-------
 include/uapi/linux/dm-push-backup.h |  36 ++++
 2 files changed, 229 insertions(+), 59 deletions(-)
 create mode 100644 include/uapi/linux/dm-push-backup.h

-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 1/6] drivers/md/push_backup: correct initialization

[Andrey Zhadchenko]

Allocate pool and htable after lock, waitqueue and deadline
timer init so we are sure that pb_destroy() can be called
from anywhere. Otherwise we risk timer_delete_sync_try on
zeroed timer struct (currently luckily fine).

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: no changes

 drivers/md/dm-push-backup.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index 885ea7b31cf67..9a6252f2b21e2 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -658,24 +658,22 @@ static int pb_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 	pb = kzalloc(sizeof(*pb), GFP_KERNEL);
 	if (!pb)
 		goto err;
-
-	pb->pbio_pool = mempool_create_slab_pool(PUSH_BACKUP_POOL_SIZE,
-						 pbio_cache);
-        pb->pending_htable = kcalloc(PB_HASH_TABLE_SIZE,
-				     sizeof(struct hlist_head),
-				     GFP_KERNEL);
-	if (!pb->pbio_pool || !pb->pending_htable)
-		goto err;
-
 	spin_lock_init(&pb->lock);
 	init_rwsem(&pb->ctl_rwsem);
 	bio_list_init(&pb->deferred_bios);
 	INIT_LIST_HEAD(&pb->pending);
 	timer_setup(&pb->deadline_timer, pb_timer_func, 0);
-
 	init_waitqueue_head(&pb->waitq);
 	INIT_WORK(&pb->worker, do_pb_work);
 
+	pb->pbio_pool = mempool_create_slab_pool(PUSH_BACKUP_POOL_SIZE,
+						 pbio_cache);
+	pb->pending_htable = kcalloc(PB_HASH_TABLE_SIZE,
+				     sizeof(struct hlist_head),
+				     GFP_KERNEL);
+	if (!pb->pbio_pool || !pb->pending_htable)
+		goto err;
+
 	ti->private = pb;
 	pb->ti = ti;
 
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 1/6] drivers/md/push_backup: correct initialization

[no-reply]

> Allocate pool and htable after lock, waitqueue and deadline
> timer init so we are sure that pb_destroy() can be called
> from anywhere. Otherwise we risk timer_delete_sync_try on
> zeroed timer struct (currently luckily fine).
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-152912
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=1


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 2/6] drivers/md/push_backup: clear pb->alive under pb->lock

[Andrey Zhadchenko]

Most other users handle this under the pb->lock so let's
make it a bit more consistent.

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: reworded the commit message, dropped the KCSAN claim

 drivers/md/dm-push-backup.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index 9a6252f2b21e2..1ae4ccc208ffa 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -581,7 +581,9 @@ static int pb_message(struct dm_target *ti, unsigned int argc, char **argv,
 		if (!pb->map)
 			return -EBADF;
 
+		spin_lock_irq(&pb->lock);
 		pb->alive = false;
+		spin_unlock_irq(&pb->lock);
 		wake_up_interruptible(&pb->waitq); /* pb->alive = false */
 	}
 
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 2/6] drivers/md/push_backup: clear pb->alive under pb->lock

[no-reply]

> Most other users handle this under the pb->lock so let's
> make it a bit more consistent.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-152912
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=2


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 3/6] drivers/md/push_backup: do not re-arm deadline timer in push_backup_write()

[Andrey Zhadchenko]

pb_timer_func() checks pb->deadline_jiffies every time it fires,
so we actually don't need to re-arm the timer after setting the
filed.

This is also the only place that arms the timer outside the window
push_backup_stop() waits for with synchronize_rcu(). Today it is
harmless because write and stop are serialized by pb->ctl_rwsem,
but it would re-arm the timer behind timer_delete_sync_try() once
the write path is reachable without the rwsem (next patches).

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: no changes

 drivers/md/dm-push-backup.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index 1ae4ccc208ffa..8ce84a1a696e5 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -467,8 +467,8 @@ static int push_backup_write(struct push_backup *pb,
 			     unsigned int clu, unsigned int nr)
 {
 	u64 i, key, nr_clus = pb->nr_clus;
-	bool finished, has_more = false;
 	LIST_HEAD(pbio_list);
+	bool finished;
 	struct pb_bio *pbio;
 
 	if (!pb)
@@ -519,8 +519,7 @@ static int push_backup_write(struct push_backup *pb,
 		}
 	}
 
-	has_more = (pb->nr_delayed != 0);
-	if (has_more)
+	if (pb->nr_delayed)
 		pb->deadline_jiffies = get_jiffies_64() + pb->timeout_in_jiffies;
 	else
 		pb->deadline_jiffies = S64_MAX;
@@ -529,11 +528,8 @@ static int push_backup_write(struct push_backup *pb,
 	if (finished)
 		wake_up_interruptible(&pb->waitq);
 
-	if (!list_empty(&pbio_list)) {
+	if (!list_empty(&pbio_list))
 		resubmit_pbios(pb, &pbio_list);
-		if (has_more)
-			mod_timer(&pb->deadline_timer, pb->timeout_in_jiffies + 1);
-	}
 
 	return 0;
 }
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 3/6] drivers/md/push_backup: do not re-arm deadline timer in push_backup_write()

[no-reply]

> pb_timer_func() checks pb->deadline_jiffies every time it fires,
> so we actually don't need to re-arm the timer after setting the
> filed.
> 
> This is also the only place that arms the timer outside the window
> push_backup_stop() waits for with synchronize_rcu(). Today it is
> harmless because write and stop are serialized by pb->ctl_rwsem,
> but it would re-arm the timer behind timer_delete_sync_try() once
> the write path is reachable without the rwsem (next patches).
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-152912
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=3


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 4/6] drivers/md/push_backup: fix mod_timer() usage

[Andrey Zhadchenko]

mod_timer() takes an absolute jiffies value, but both callers pass a
relative one. After about 5 min uptime this causes the timer to fire
and re-arm on every tick. This has not exploded because we still have
checked pb->deadline_jiffies each time.

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: no changes

 drivers/md/dm-push-backup.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index 8ce84a1a696e5..b1861204f6e3e 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -244,7 +244,7 @@ static void pb_timer_func(struct timer_list *timer)
 	spin_unlock_irqrestore(&pb->lock, flags);
 
 	if (unlikely(time_before64(now, deadline)))
-		mod_timer(timer, deadline - now + 1);
+		mod_timer(timer, jiffies + (deadline - now) + 1);
 	else
 		queue_work(pb->wq, &pb->worker);
 }
@@ -278,7 +278,7 @@ static int postpone_if_required_for_backup(struct push_backup *pb,
 	spin_unlock_irqrestore(&pb->lock, flags);
 
 	if (queue_timer)
-		mod_timer(&pb->deadline_timer, pb->timeout_in_jiffies + 1);
+		mod_timer(&pb->deadline_timer, jiffies + pb->timeout_in_jiffies + 1);
 	rcu_read_unlock();
 
 	if (queue_timer)
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 4/6] drivers/md/push_backup: fix mod_timer() usage

[no-reply]

> mod_timer() takes an absolute jiffies value, but both callers pass a
> relative one. After about 5 min uptime this causes the timer to fire
> and re-arm on every tick. This has not exploded because we still have
> checked pb->deadline_jiffies each time.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-152912
> Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=4


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 5/6] drivers/md/push_backup: arm deadline timer under pb->lock

[Andrey Zhadchenko]

When push_backup_write acks last delayed request it sets
deadline_jiffies = S64_MAX and the next fired timer uses this
for mod_timer(): expiry is not set to some very distant future.

Previously a later push_backup_write() would accidentally
repair the expiry; since it no longer touches the timer, nothing
does until stop.

Make the decision and the arming atomic: call mod_timer() under
pb->lock at both sites, and do not re-arm when nothing is delayed.
mod_timer() and queue_work() are fine under a spinlock, and the timer
base lock is dropped while the callback runs, so there is no lock
inversion.

As a bonus the rcu_read_lock()/synchronize_rcu() dance between
postpone_if_required_for_backup() and push_backup_stop() is no longer
needed: cleanup_backup() clears pb->alive under pb->lock, the timer is
only armed under pb->lock while pb->alive is set, and the callback
re-arming itself is handled by timer_delete_sync_try(). Drop it.

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: new patch

 drivers/md/dm-push-backup.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index b1861204f6e3e..8f2015e8e29a4 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -241,23 +241,24 @@ static void pb_timer_func(struct timer_list *timer)
 
 	spin_lock_irqsave(&pb->lock, flags);
 	deadline = pb->deadline_jiffies;
-	spin_unlock_irqrestore(&pb->lock, flags);
-
+	if (deadline == S64_MAX)
+		goto unlock; /* Nothing is delayed, let the timer idle */
 	if (unlikely(time_before64(now, deadline)))
 		mod_timer(timer, jiffies + (deadline - now) + 1);
 	else
 		queue_work(pb->wq, &pb->worker);
+unlock:
+	spin_unlock_irqrestore(&pb->lock, flags);
 }
 
 static int postpone_if_required_for_backup(struct push_backup *pb,
 					   struct request *rq)
 {
-	bool queue_timer = false;
 	struct pb_bio *pbio;
 	unsigned long flags;
+	bool wake = false;
 	int ret = 0;
 
-	rcu_read_lock(); /* See push_backup_stop() */
 	spin_lock_irqsave(&pb->lock, flags);
 	if (!pb->alive)
 		goto unlock;
@@ -272,16 +273,13 @@ static int postpone_if_required_for_backup(struct push_backup *pb,
 	pb->nr_delayed += 1;
 	if (pb->nr_delayed == 1) {
 		pb->deadline_jiffies = get_jiffies_64() + pb->timeout_in_jiffies;
-		queue_timer = true;
+		mod_timer(&pb->deadline_timer, jiffies + pb->timeout_in_jiffies + 1);
+		wake = true;
 	}
 unlock:
 	spin_unlock_irqrestore(&pb->lock, flags);
 
-	if (queue_timer)
-		mod_timer(&pb->deadline_timer, jiffies + pb->timeout_in_jiffies + 1);
-	rcu_read_unlock();
-
-	if (queue_timer)
+	if (wake)
 		wake_up_interruptible(&pb->waitq);
 
 	return ret;
@@ -400,8 +398,6 @@ static int push_backup_stop(struct push_backup *pb,
 		return -EBADF;
 	cleanup_backup(pb);
 
-	/* Wait postpone_if_required_for_backup() starts timer */
-	synchronize_rcu();
 	WARN_ON_ONCE(timer_delete_sync_try(&pb->deadline_timer) < 0);
 	flush_workqueue(pb->wq);
 
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 5/6] drivers/md/push_backup: arm deadline timer under pb->lock

[no-reply]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] dm-push-backup: pb_timer_func() can re-arm timer or queue work after stop/destroy
- [Medium] dm-push-backup: pb_destroy() uses non-waiting timer_delete_sync_try() instead of timer_shutdown_sync()
- [Medium] dm-push-backup: cleanup_backup() leaves deadline_jiffies stale, so a racing timer callback can re-arm or queue_work()
--

commit f322079c9ebbcb0d0c2a4b8257ff8a52b4e79a15
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

drivers/md/push_backup: arm deadline timer under pb->lock

This patch moves the mod_timer() calls in pb_timer_func() and
postpone_if_required_for_backup() under pb->lock. It also makes the timer
callback idle when deadline_jiffies is S64_MAX. It drops the
rcu_read_lock()/synchronize_rcu() pairing between
postpone_if_required_for_backup() and push_backup_stop().

Link: https://virtuozzo.atlassian.net/browse/VSTOR-152912

> diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
> --- a/drivers/md/dm-push-backup.c
> +++ b/drivers/md/dm-push-backup.c

[ ... ]

> @@ -241,23 +241,24 @@ static void pb_timer_func(struct timer_list *timer)
>  
>  	spin_lock_irqsave(&pb->lock, flags);
>  	deadline = pb->deadline_jiffies;
> -	spin_unlock_irqrestore(&pb->lock, flags);
> -
> +	if (deadline == S64_MAX)
> +		goto unlock; /* Nothing is delayed, let the timer idle */
>  	if (unlikely(time_before64(now, deadline)))
>  		mod_timer(timer, jiffies + (deadline - now) + 1);
>  	else
>  		queue_work(pb->wq, &pb->worker);
> +unlock:
> +	spin_unlock_irqrestore(&pb->lock, flags);
>  }

[Severity: Medium]
Can pb_timer_func() re-arm the timer or queue work after push_backup_stop()
has returned?

The callback does not check pb->alive. It only tests deadline_jiffies
against S64_MAX. cleanup_backup() sets pb->alive = false and drains
nr_delayed to 0, but it leaves deadline_jiffies at its old finite value.
Only setup_pb() and push_backup_write() write S64_MAX.

Consider this sequence:

CPU1
pb_timer_func()
    blocks on spin_lock_irqsave(&pb->lock)

CPU0
push_backup_stop()
    cleanup_backup()
        pb->alive = false;
        (deadline_jiffies still finite)
    timer_delete_sync_try()   <- returns -1, callback is running

CPU1 then takes pb->lock, sees a finite deadline, and calls mod_timer()
or queue_work(pb->wq, &pb->worker). If push_backup_start() has already run
for a new session, the stale do_pb_work() could call cleanup_backup() and
expire that session early.

Should cleanup_backup() set pb->deadline_jiffies = S64_MAX under pb->lock,
or should pb_timer_func() check pb->alive?

> @@ -272,16 +273,13 @@ static int postpone_if_required_for_backup(struct push_backup *pb,

[ ... ]

> @@ -400,8 +398,6 @@ static int push_backup_stop(struct push_backup *pb,
>  		return -EBADF;
>  	cleanup_backup(pb);
>  
> -	/* Wait postpone_if_required_for_backup() starts timer */
> -	synchronize_rcu();
>  	WARN_ON_ONCE(timer_delete_sync_try(&pb->deadline_timer) < 0);
>  	flush_workqueue(pb->wq);

[Severity: Medium]
The commit message says "the callback re-arming itself is handled by
timer_delete_sync_try()". Is that accurate?

timer_delete_sync_try() calls __try_to_del_timer_sync(timer, false). That
returns -1 without waiting when base->running_timer == timer, and it does
not prevent re-arming. Here the -1 result only triggers WARN_ON_ONCE().

The removed synchronize_rcu() only covered
postpone_if_required_for_backup() arming the timer. It never covered the
callback re-arming itself.

So after this change, can the callback still call mod_timer() or
queue_work() after flush_workqueue() has returned? Would
timer_shutdown_sync(), or a loop that waits for the running callback,
be more appropriate here?

The same WARN_ON_ONCE(timer_delete_sync_try(&pb->deadline_timer) < 0)
pattern in pb_destroy() is not touched by this patch. It runs before
destroy_workqueue(pb->wq) and pb_put(pb).

If pb_timer_func() is running on another CPU and blocked on pb->lock, can
it call queue_work() on a destroyed workqueue? Can it also touch pb after
it has been freed? When pb_destroy() runs without a prior push_backup_stop(),
pb->alive is still true and a finite deadline can still exist. Would
timer_shutdown_sync() be the right call in pb_destroy()?

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH hci-8.1 v2 6/6] dm/push_backup: add event file descriptor interface

[Andrey Zhadchenko]

The backup tool currently waits for the next cluster range to back up
by sleeping inside the "push_backup_read" target message. target_message()
calls into the target under dm_get_live_table(), i.e. with md->io_barrier
SRCU read-locked, so for the whole time the tool waits for a write to be
intercepted anything doing synchronize_srcu(&md->io_barrier) on this device
is stuck: dm suspend, table reload/remove and in particular
dm_internal_suspend_fast() called by "@stats_create". An unrelated dm-stats
consumer therefore hangs together with everything that later queues behind
md->suspend_lock.

Add a file descriptor based interface that does not involve dm messages
on the data path:

  push_backup_get_fd        - returns an anon inode fd in the message
                              result
  read(fd, &range, 16)      - next struct push_backup_range to back up;
                              0 when the backup is finished,
                              -ESTALE when not running,
                              -ENODEV after the target is destroyed
  write(fd, &range, 16)     - acknowledge the range, same as
                              push_backup_write

This way we will avoid aforementioned problems.

We can't access/uninstall fd across all processes that have it, so add kref
on the push_backup struct and tie it to number of fds. Free only when
everything is gone.

The legacy messages keep working unchanged for compatibility but we will
clean it up later.

Also destroy pbio object first before requing in resubmit_pbios():
in-flight requests hold the target which we may have already stopped,
so dm_requeue_original may trigger .dtr and invalidate mempool.

https://virtuozzo.atlassian.net/browse/VSTOR-152912
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2: free the pbio before dm_requeue_original_rq() in resubmit_pbios(),
    the requeue may drop the last md reference and let .dtr run

 drivers/md/dm-push-backup.c         | 200 ++++++++++++++++++++++++----
 include/uapi/linux/dm-push-backup.h |  36 +++++
 2 files changed, 207 insertions(+), 29 deletions(-)
 create mode 100644 include/uapi/linux/dm-push-backup.h

diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
index 8f2015e8e29a4..f45ac679c5ec8 100644
--- a/drivers/md/dm-push-backup.c
+++ b/drivers/md/dm-push-backup.c
@@ -15,6 +15,10 @@
 #include <linux/ctype.h>
 #include <linux/dm-io.h>
 #include <linux/blk-mq.h>
+#include <linux/anon_inodes.h>
+#include <linux/kref.h>
+#include <linux/uaccess.h>
+#include <linux/dm-push-backup.h>
 
 
 #define DM_MSG_PREFIX "push-backup"
@@ -46,6 +50,9 @@ struct push_backup {
 	u64 clu_size;
 	u64 nr_clus;
 
+	struct kref ref;
+	bool dead;
+
 	bool alive;
 	void *map;
 	u64 map_bits;
@@ -69,6 +76,16 @@ struct push_backup {
 	struct rw_semaphore ctl_rwsem;
 };
 
+static void pb_release(struct kref *ref)
+{
+	kfree(container_of(ref, struct push_backup, ref));
+}
+
+static void pb_put(struct push_backup *pb)
+{
+	kref_put(&pb->ref, pb_release);
+}
+
 static u64 pbio_first_required_for_backup_clu(struct push_backup *pb, struct pb_bio *pbio)
 {
 	u64 clu;
@@ -196,9 +213,10 @@ static void resubmit_pbios(struct push_backup *pb, struct list_head *list)
 	struct pb_bio *pbio;
 
 	while ((pbio = list_first_entry_or_null(list, struct pb_bio, list)) != NULL) {
+		struct request *rq = pbio->rq;
 		list_del_init(&pbio->list);
-		dm_requeue_original_rq(pbio->rq);
 		mempool_free(pbio, pb->pbio_pool);
+		dm_requeue_original_rq(rq);
 	}
 }
 
@@ -411,35 +429,41 @@ static int push_backup_stop(struct push_backup *pb,
 	return 0;
 }
 
-static int push_backup_read(struct push_backup *pb,
-			    char *result, unsigned int maxlen)
+static bool pb_has_event(struct push_backup *pb)
+{
+	return READ_ONCE(pb->dead) || !READ_ONCE(pb->map) ||
+	       !READ_ONCE(pb->alive) || !READ_ONCE(pb->map_bits) ||
+	       !list_empty_careful(&pb->pending);
+}
+
+/*
+ * Returns:
+ *   1       - next range to back up is stored in @clu/@nr
+ *   0       - backup is finished (nothing left to back up)
+ *   -EAGAIN - nothing pending at the moment
+ *   -ESTALE - backup is not running (not started, stopped or expired)
+ *   -ENODEV - target was destroyed
+ */
+static int pb_next_range(struct push_backup *pb, u64 *clu, u64 *nr)
 {
-	unsigned int left, right, sz = 0;
 	struct pb_bio *pbio;
+	u64 left, right;
 	int ret;
 
-	if (!pb)
-		return -EBADF;
-	if (!pb->map)
-		return -ESTALE;
-again:
-	if (wait_event_interruptible(pb->waitq,
-				     !list_empty_careful(&pb->pending) ||
-				     !pb->alive || !pb->map_bits))
-		return -EINTR;
-
 	spin_lock_irq(&pb->lock);
+	ret = -ENODEV;
+	if (pb->dead)
+		goto unlock;
 	ret = -ESTALE;
-	if (!pb->alive)
+	if (!pb->map || !pb->alive)
 		goto unlock;
 	ret = 0;
 	if (!pb->map_bits)
 		goto unlock;
+	ret = -EAGAIN;
 	pbio = list_first_entry_or_null(&pb->pending, typeof(*pbio), list);
-	if (unlikely(!pbio)) {
-		spin_unlock_irq(&pb->lock);
-		goto again;
-	}
+	if (!pbio)
+		goto unlock;
 
 	ret = -EBADMSG;
 	left = pbio_first_required_for_backup_clu(pb, pbio);
@@ -452,15 +476,47 @@ static int push_backup_read(struct push_backup *pb,
 	else
 		right = pb->nr_clus - 1;
 
-	DMEMIT("%u:%u", left, right - left + 1);
+	*clu = left;
+	*nr = right - left + 1;
 	ret = 1;
 unlock:
 	spin_unlock_irq(&pb->lock);
 	return ret;
 }
 
-static int push_backup_write(struct push_backup *pb,
-			     unsigned int clu, unsigned int nr)
+static int pb_wait_range(struct push_backup *pb, bool nonblock,
+			 u64 *clu, u64 *nr)
+{
+	int ret;
+
+	while (1) {
+		ret = pb_next_range(pb, clu, nr);
+		if (ret != -EAGAIN || nonblock)
+			return ret;
+		if (wait_event_interruptible(pb->waitq, pb_has_event(pb)))
+			return -ERESTARTSYS;
+	}
+}
+
+static int push_backup_read(struct push_backup *pb,
+			    char *result, unsigned int maxlen)
+{
+	unsigned int sz = 0;
+	u64 clu, nr;
+	int ret;
+
+	if (!pb)
+		return -EBADF;
+
+	ret = pb_wait_range(pb, false, &clu, &nr);
+	if (ret == -ERESTARTSYS)
+		ret = -EINTR;
+	if (ret == 1)
+		DMEMIT("%llu:%llu", clu, nr);
+	return ret;
+}
+
+static int push_backup_write(struct push_backup *pb, u64 clu, u64 nr)
 {
 	u64 i, key, nr_clus = pb->nr_clus;
 	LIST_HEAD(pbio_list);
@@ -471,11 +527,13 @@ static int push_backup_write(struct push_backup *pb,
 		return -EBADF;
 	if (clu >= nr_clus || nr > nr_clus - clu)
 		return -E2BIG;
-	if (!pb->map)
-		return -ESTALE;
 
 	spin_lock_irq(&pb->lock);
-	if (!pb->alive) {
+	if (pb->dead) {
+		spin_unlock_irq(&pb->lock);
+		return -ENODEV;
+	}
+	if (!pb->map || !pb->alive) {
 		spin_unlock_irq(&pb->lock);
 		return -ESTALE;
 	}
@@ -530,6 +588,73 @@ static int push_backup_write(struct push_backup *pb,
 	return 0;
 }
 
+static ssize_t pb_fd_read(struct file *file, char __user *buf,
+			  size_t count, loff_t *ppos)
+{
+	struct push_backup *pb = file->private_data;
+	struct push_backup_range range;
+	int ret;
+
+	if (count < sizeof(range))
+		return -EINVAL;
+
+	ret = pb_wait_range(pb, file->f_flags & O_NONBLOCK,
+			    &range.clu, &range.nr);
+	if (ret <= 0)
+		return ret;
+
+	if (copy_to_user(buf, &range, sizeof(range)))
+		return -EFAULT;
+	return sizeof(range);
+}
+
+static ssize_t pb_fd_write(struct file *file, const char __user *buf,
+			   size_t count, loff_t *ppos)
+{
+	struct push_backup *pb = file->private_data;
+	struct push_backup_range range;
+	int ret;
+
+	if (count != sizeof(range))
+		return -EINVAL;
+	if (copy_from_user(&range, buf, sizeof(range)))
+		return -EFAULT;
+
+	ret = push_backup_write(pb, range.clu, range.nr);
+	return ret ? : sizeof(range);
+}
+
+static int pb_fd_release(struct inode *inode, struct file *file)
+{
+	pb_put(file->private_data);
+	return 0;
+}
+
+static const struct file_operations pb_fops = {
+	.owner		= THIS_MODULE,
+	.read		= pb_fd_read,
+	.write		= pb_fd_write,
+	.release	= pb_fd_release,
+};
+
+static int push_backup_get_fd(struct push_backup *pb,
+			      char *result, unsigned int maxlen)
+{
+	unsigned int sz = 0;
+	int fd;
+
+	kref_get(&pb->ref);
+	fd = anon_inode_getfd("[push_backup]", &pb_fops, pb,
+			      O_RDWR | O_CLOEXEC);
+	if (fd < 0) {
+		pb_put(pb);
+		return fd;
+	}
+
+	DMEMIT("%d", fd);
+	return 1;
+}
+
 static int push_backup_statistics(struct push_backup *pb, char *result,
 				  unsigned int maxlen)
 {
@@ -608,6 +733,10 @@ static int pb_message(struct dm_target *ti, unsigned int argc, char **argv,
 		ret = push_backup_write(pb, val, val2);
 	} else if (!strcmp(argv[0], "push_backup_statistics")) {
 		ret = push_backup_statistics(pb, result, maxlen);
+	} else if (!strcmp(argv[0], "push_backup_get_fd")) {
+		if (argc != 1)
+			goto unlock;
+		ret = push_backup_get_fd(pb, result, maxlen);
 	} else {
 		ret = -ENOTSUPP;
 	}
@@ -622,18 +751,30 @@ static int pb_message(struct dm_target *ti, unsigned int argc, char **argv,
 }
 static void pb_destroy(struct push_backup *pb)
 {
+	void *map, *pending_map;
+
 	WARN_ON_ONCE(pb->nr_delayed);
 
+	spin_lock_irq(&pb->lock);
+	pb->dead = true;
+	map = pb->map; /* Is's not zero if stop was not called */
+	pb->map = NULL;
+	pending_map = pb->pending_map;
+	pb->pending_map = NULL;
+	spin_unlock_irq(&pb->lock);
+
+	wake_up_interruptible(&pb->waitq); /* pb->dead = true */
+
 	WARN_ON_ONCE(timer_delete_sync_try(&pb->deadline_timer) < 0);
 	if (pb->wq)
 		destroy_workqueue(pb->wq);
-	kvfree(pb->map); /* Is's not zero if stop was not called */
-	kvfree(pb->pending_map);
+	kvfree(map);
+	kvfree(pending_map);
 	kvfree(pb->pending_htable);
 	mempool_destroy(pb->pbio_pool);
 	if (pb->origin_dev)
 		dm_put_device(pb->ti, pb->origin_dev);
-	kfree(pb);
+	pb_put(pb);
 }
 
 /*
@@ -652,6 +793,7 @@ static int pb_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 	pb = kzalloc(sizeof(*pb), GFP_KERNEL);
 	if (!pb)
 		goto err;
+	kref_init(&pb->ref);
 	spin_lock_init(&pb->lock);
 	init_rwsem(&pb->ctl_rwsem);
 	bio_list_init(&pb->deferred_bios);
@@ -760,7 +902,7 @@ static loff_t pb_llseek_hole(struct dm_target *ti, loff_t offset,
 
 static struct target_type pb_target = {
 	.name = "push_backup",
-	.version = {1, 0, 0},
+	.version = {1, 1, 0},
 	.features = DM_TARGET_SINGLETON,
 	.module = THIS_MODULE,
 	.ctr = pb_ctr,
diff --git a/include/uapi/linux/dm-push-backup.h b/include/uapi/linux/dm-push-backup.h
new file mode 100644
index 0000000000000..3a56f851a2d56
--- /dev/null
+++ b/include/uapi/linux/dm-push-backup.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ *  Copyright (c) 2026 Virtuozzo International GmbH. All rights reserved.
+ */
+
+/*
+ * Userspace interface of the dm-push-backup target's event file descriptor.
+ *
+ * The fd is obtained with the "push_backup_get_fd" target message; the fd
+ * number is returned in the message result buffer.
+ *
+ * read(fd, &range, sizeof(range)):
+ *   sizeof(range) - @range holds the next cluster range that must be backed
+ *                   up before the writes postponed on it may proceed.
+ *                   Reading does not consume anything: the same (or a wider)
+ *                   range is returned until it is acknowledged via write().
+ *   0             - backup has finished, nothing is left to back up.
+ *   -EAGAIN       - O_NONBLOCK is set and no writes are pending.
+ *   -ESTALE       - backup is not running (not started, stopped or expired).
+ *   -ENODEV       - the target has been destroyed.
+ *
+ * write(fd, &range, sizeof(range)):
+ *   acknowledge that @range has been backed up; equivalent to the
+ *   "push_backup_write clu:nr" message.
+ */
+#ifndef _UAPI_LINUX_DM_PUSH_BACKUP_H
+#define _UAPI_LINUX_DM_PUSH_BACKUP_H
+
+#include <linux/types.h>
+
+struct push_backup_range {
+	__u64 clu;	/* first cluster */
+	__u64 nr;	/* number of clusters */
+};
+
+#endif /* _UAPI_LINUX_DM_PUSH_BACKUP_H */
-- 
2.43.5

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1 v2 6/6] dm/push_backup: add event file descriptor interface

[no-reply]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] dm-push-backup: push_backup_get_fd() installs fd before result is delivered
- [Low] dm-push-backup: event fd has no .poll method, so it cannot be multiplexed
--

commit 586e161f2dd08e2697bfd3552aef44d435e3d52e
Author: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>

dm/push_backup: add event file descriptor interface

This adds a "push_backup_get_fd" target message that returns an anon inode
fd. The backup tool can read() the next range to back up and write() to
acknowledge it, without sleeping inside a dm message. A kref on struct
push_backup is tied to the number of open fds.

Link: https://virtuozzo.atlassian.net/browse/VSTOR-152912

> diff --git a/drivers/md/dm-push-backup.c b/drivers/md/dm-push-backup.c
> --- a/drivers/md/dm-push-backup.c
> +++ b/drivers/md/dm-push-backup.c

[ ... ]

> +static const struct file_operations pb_fops = {
> +	.owner		= THIS_MODULE,
> +	.read		= pb_fd_read,
> +	.write		= pb_fd_write,
> +	.release	= pb_fd_release,
> +};

[Severity: Low]
This isn't a bug, but pb_fops provides no .poll method, although
pb->waitq and pb_has_event() already exist and could back one.

Without .poll, poll() and select() report the fd as always ready, and
epoll_ctl() rejects it.

A consumer that wants to multiplex the fd must dedicate a thread to a
blocking read(), or retry read() on -EAGAIN with O_NONBLOCK. The UAPI
header documents the O_NONBLOCK -EAGAIN mode, but without .poll it is hard
to use.

Would it make sense to add .poll before the UAPI is finalized?

There is also no .llseek or stream_open(), so the fd is seekable even though
f_pos is meaningless. Should this be a stream-like file?

> +static int push_backup_get_fd(struct push_backup *pb,
> +			      char *result, unsigned int maxlen)
> +{
> +	unsigned int sz = 0;
> +	int fd;
> +
> +	kref_get(&pb->ref);
> +	fd = anon_inode_getfd("[push_backup]", &pb_fops, pb,
> +			      O_RDWR | O_CLOEXEC);
> +	if (fd < 0) {
> +		pb_put(pb);
> +		return fd;
> +	}
> +
> +	DMEMIT("%d", fd);
> +	return 1;
> +}

[Severity: Medium]
Can this leak a file descriptor in the calling process?

anon_inode_getfd() calls fd_install() before the result is known to be
deliverable. After that, DMEMIT("%d", fd) only formats into the kernel
result buffer. It silently truncates if maxlen is too small, and
push_backup_get_fd() still returns 1.

If the DMEMIT output is truncated, or the dm ioctl core later fails to copy
the result out to userspace, the caller never learns the fd number:

pb_message()
  push_backup_get_fd()
    kref_get(&pb->ref);
    anon_inode_getfd()      <- fd_install() already done
    DMEMIT("%d", fd);       <- may truncate
    return 1;

The caller cannot close that fd. The fd keeps its kref on struct push_backup
and a module reference, so the target memory and the module stay pinned until
the process exits.

Would it be better to reserve the fd with get_unused_fd_flags(), check that
the result fits, and call fd_install() last?

[ ... ]

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261103003246.712239-1-andrey.zhadchenko@virtuozzo.com?part=6

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel