[Devel] [PATCH hci-8.1 v2 0/4] dm-ploop: fix the pio submission stop around cmds

["Denis V. Lunev" <den@openvz.org>]

A cmd stops pio submission and waits for the inflight pios before it
rewrites the BAT levels or deltas[], and expects nothing to touch them
until it resumes. Konstantin found three ways this does not hold while
reviewing the VSTOR-151979 series (VSTOR-152985, VSTOR-152984,
VSTOR-152981):

- a submitter racing the stop either runs without the cmd waiting for
  it, or is parked after resume and never completes (patch 1);

- the ploop_allocator() thread is not stopped at all. It truncates the
  top delta while merge, notify_merged or flip moves it in deltas[],
  cutting the live image (patch 3). The thread can also miss a wakeup,
  which stalls the device by itself and would hang the new wait, so
  patch 2 fixes that first;

- a grow failing to copy a cluster resumes submission with that
  cluster's BAT page still busy, and writes to it hang (patch 4).

Patches 1 and 3 both lose data: a pio slipping past the stop can read
and write through a delta that merge has already released, and the
allocator cuts the live image. Patch 3 is the easiest to hit, a
snapshot delete under write load is enough.

Tested on dev-cl04, stock 6.12.0-211.52.1.22.11.vz10, dm-ploop built
out of tree:
- each bug reproduced on the stock module, the races with a test-only
  kprobe module widening the window: a pio running past the suspend of
  a cmd; a pio never completing after resume; the top image truncated
  from 512M to 128M by a merge and to 256M by a flip; after a grow
  failing with ENOSPC on a full host fs, a writer stuck in D state, an
  unsuspendable device and a later cmd failing with ETIMEDOUT. None of
  them reproduce with the series;
- set_noresume, suspend, a grow to 2T, which relocates 8 clusters and
  restarts the allocator, and flip_upper_deltas with the allocator held:
  v1 cut the old top image from 640M to 256M, v2 waits for the allocator
  and the image keeps its data. On a stock kernel the flip oopses
  first: the check in ploop_check_delta_before_flip() walks past the md
  pages of the lower delta once the BAT has grown. The test ran with a
  local fix for that, which will be sent separately;
- fio 4k random reads and writes, O_DIRECT, 1 to 48 jobs, on a fully
  allocated image over brd: IOPS and system CPU stay within the
  run-to-run spread of the stock module. This setup tops out at about
  420K IOPS elsewhere in dm-ploop and would not show a lock taken on
  submission either, so patch 1 keeps that path lockless;
- the ploop pytest suite, ploop and qcow2 formats, passes. A test for
  patch 4 goes to the ploop tests separately.

Changes in v2, after Konstantin's review:
- 1/4: notify_merged named among the cmds hit by the race;
- 2/4: the stall does not end by itself, and the reorder fixes a lost
  kthread_stop() wakeup as well;
- 3/4: also wait for the allocator in ploop_flip_upper_deltas(), as a
  resize of a dm suspended device restarts it after postsuspend; the
  message names notify_merged, which moves the top delta in both
  directions;
- 4/4: unchanged;
- rebased on rh10-6.12.0-211.52.1.22.17.vz10, Reviewed-by tags added
  to 1/4, 2/4 and 4/4.

v1: http://lore.virtuozzo.com/kernel/20261103083642.130885-1-den@openvz.org/

Denis V. Lunev (4):
  dm-ploop: fix pio submission racing with the suspend and resume of a
    cmd
  dm-ploop: fix a lost wakeup of the ploop_allocator() thread
  dm-ploop: wait for the ploop_allocator() thread before changing
    deltas[]
  dm-ploop: fix a failed cluster copy on grow leaving the BAT page busy

 drivers/md/dm-ploop-cmd.c    | 24 ++++++++++++++++++++++--
 drivers/md/dm-ploop-map.c    | 22 +++++++++++++++-------
 drivers/md/dm-ploop-target.c |  4 ++++
 drivers/md/dm-ploop.h        |  1 +
 4 files changed, 42 insertions(+), 9 deletions(-)


base-commit: 95ab11e2c57e8468412ca586d4be8cd75b6bfbf0
-- 
2.53.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel