* [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
@ 2026-11-10 16:45 Denis V. Lunev
2026-11-10 16:53 ` no-reply
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Denis V. Lunev @ 2026-11-10 16:45 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Denis V. Lunev <den@openvz.org>
increase_address_space() publishes the new root and mode and only then
points the DTEs at the new root. alloc_pte() reads pgtable->mode
without the domain lock, so a mapper on another CPU that sees the new
mode installs its PTEs under the new root and hands the IOVA to the
device while the IOMMU still walks the old table:
nvme 0000:63:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT
domain=0x0004 address=0xfffffffffff40000 flags=0x0030]
The v1 table starts with 3 levels and grows to 6 on the first IOVA
dma-iommu takes from the top of the 64-bit space, once the 32-bit space
runs out. That happens under heavy DMA load, so many CPUs map into the
window at once. A faulting device read of memory fails the command with
Data Transfer Error. A faulting device write to memory is posted and
gets dropped silently: an NVMe READ completes successfully and the
buffer keeps whatever it held before.
Update the DTEs inside the seqcount write section. alloc_pte() and
fetch_pte() spin in read_seqcount_begin() until it ends, so nothing is
mapped under the new root before the devices use it.
Mainline is not affected since commit 789a5913b29c ("iommu/amd: Use
the generic iommu page table"): increase_top() calls change_top()
before it publishes the new top.
https://virtuozzo.atlassian.net/browse/VSTOR-154276
Fixes: 6eedb59c18a3 ("iommu/amd: Remove amd_iommu_domain_get_pgtable")
Feature: fix ms/iommu
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
Reproduced on epyc06 (Supermicro H12DSU-iN, EPYC 7352, four Optane in
one IOMMU group): an out-of-tree module eats the group's 32-bit IOVA
space through a root port while fio reads the drives, and a kprobe
delays amd_iommu_update_and_flush_device_table() to hold the window
open. Stock 6.12.0-211.52.1.22.9.vz10 faults on every try, and fio
verify sees reads return stale buffers with no I/O error. With this
patch: no faults, no verify mismatches, and the first 64-bit IOVA is
mapped only after the last DTE flush.
drivers/iommu/amd/io_pgtable.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c
index 70c2f5b1631b..46dfed9bd7e2 100644
--- a/drivers/iommu/amd/io_pgtable.c
+++ b/drivers/iommu/amd/io_pgtable.c
@@ -131,12 +131,15 @@ static bool increase_address_space(struct amd_io_pgtable *pgtable,
*pte = PM_LEVEL_PDE(pgtable->mode, iommu_virt_to_phys(pgtable->root));
+ /*
+ * Keep the seqcount odd until the DTEs point at the new root, or a
+ * concurrent alloc_pte() maps an IOVA the device cannot translate yet.
+ */
write_seqcount_begin(&pgtable->seqcount);
pgtable->root = pte;
pgtable->mode += 1;
- write_seqcount_end(&pgtable->seqcount);
-
amd_iommu_update_and_flush_device_table(domain);
+ write_seqcount_end(&pgtable->seqcount);
pte = NULL;
ret = true;
--
2.53.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
2026-11-10 16:45 [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows Denis V. Lunev
@ 2026-11-10 16:53 ` no-reply
2026-11-10 16:56 ` Denis V. Lunev
2026-11-10 17:53 ` Konstantin Khorenko
2026-11-10 17:53 ` Konstantin Khorenko
2 siblings, 1 reply; 5+ messages in thread
From: no-reply @ 2026-11-10 16:53 UTC (permalink / raw)
To: Denis V. Lunev, khorenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] iommu/amd: seqcount write section spans sleepable DTE flush, so readers can livelock on PREEMPT_RT
--
commit 7634aee1eec1a71c7f83c233557bd0d61b816f56
Author: Denis V. Lunev <den@openvz.org>
iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
This commit moves amd_iommu_update_and_flush_device_table() inside the
seqcount write section in increase_address_space(). Mappers that call
alloc_pte() or fetch_pte() on another CPU then spin until the DTEs point
at the new root. This is meant to stop IOVAs from being mapped under the
new root before the device uses it.
> diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c
> --- a/drivers/iommu/amd/io_pgtable.c
> +++ b/drivers/iommu/amd/io_pgtable.c
[ ... ]
> @@ -131,12 +131,15 @@ static bool increase_address_space(struct amd_io_pgtable *pgtable,
>
> *pte = PM_LEVEL_PDE(pgtable->mode, iommu_virt_to_phys(pgtable->root));
>
> + /*
> + * Keep the seqcount odd until the DTEs point at the new root, or a
> + * concurrent alloc_pte() maps an IOVA the device cannot translate yet.
> + */
> write_seqcount_begin(&pgtable->seqcount);
> pgtable->root = pte;
> pgtable->mode += 1;
> - write_seqcount_end(&pgtable->seqcount);
> -
> amd_iommu_update_and_flush_device_table(domain);
> + write_seqcount_end(&pgtable->seqcount);
[Severity: Medium]
Can readers livelock on a PREEMPT_RT build here?
pgtable->seqcount is a plain seqcount_t, so write_seqcount_begin() does
not disable preemption itself. On non-RT builds, the enclosing
spin_lock_irqsave(&domain->lock) disables preemption and IRQs. The odd
window therefore runs atomically, and readers on other CPUs spin only for
the bounded flush time.
On PREEMPT_RT, spinlock_t is a sleepable lock and spin_lock_irqsave()
does not disable preemption or IRQs. The writer stays preemptible while
the seqcount is odd.
Before this patch the odd window covered two stores. Now it also covers
amd_iommu_update_and_flush_device_table():
amd_iommu_update_and_flush_device_table()
set_dte_entry() /* takes spinlock_t dev_data->dte_lock */
clone_aliases()
device_flush_dte() /* sends commands under raw iommu->lock */
domain_flush_complete() /* polls for completion */
The readers cannot sleep:
alloc_pte()
read_seqcount_begin(&pgtable->seqcount)
fetch_pte()
read_seqcount_begin(&pgtable->seqcount)
If the writer is preempted or blocked on dte_lock, a higher-priority task
on the same CPU that maps or unmaps through the same domain spins in
read_seqcount_begin() with no priority inheritance. fetch_pte() is also
reached from the unmap, iova_to_phys and dirty-read paths.
The existing write_seqcount_begin() under spin_lock_irqsave() already
conflicts with the seqcount preemption requirement on RT. This patch
widens that window from two stores to include blocking operations.
Is there a way to publish the new root and mode only after the DTEs are
updated, or to otherwise avoid holding the seqcount odd across the DTE
flush?
>
> pte = NULL;
> ret = true;
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261110164529.2794752-1-den@openvz.org?part=1
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
2026-11-10 16:53 ` no-reply
@ 2026-11-10 16:56 ` Denis V. Lunev
0 siblings, 0 replies; 5+ messages in thread
From: Denis V. Lunev @ 2026-11-10 16:56 UTC (permalink / raw)
To: devel
> [Severity: Medium]
> Can readers livelock on a PREEMPT_RT build here?
The vz10 kernels are not PREEMPT_RT: both x86_64 flavours build with
PREEMPT_VOLUNTARY and PREEMPT_RT unset. There spin_lock_irqsave() keeps
the write section atomic, readers on other CPUs spin for the DTE flush
only, and nothing in amd_iommu_update_and_flush_device_table() reads the
seqcount, so the writer cannot wait on itself. Tested with the section
held open for 5 ms at each of the three level increases: no lockups.
> Is there a way to publish the new root and mode only after the DTEs are
> updated, or to otherwise avoid holding the seqcount odd across the DTE
> flush?
On PREEMPT_RT a plain seqcount_t writer under a sleeping spinlock is
already broken before this patch, as the review notes. Fixing that means
publishing root and mode only after set_dte_entry() can take them
explicitly, as mainline does since 789a5913b29c. Not worth it for a
configuration we do not build.
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
2026-11-10 16:45 [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows Denis V. Lunev
2026-11-10 16:53 ` no-reply
@ 2026-11-10 17:53 ` Konstantin Khorenko
2026-11-10 17:53 ` Konstantin Khorenko
2 siblings, 0 replies; 5+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:53 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
Applying with these fixes on my side:
- "grows to 6" -> "grows up to 6"
- the second paragraph split in two: when the table grows, and what a
faulting read and write do, as a "- " list
On Tue, 10 Nov 2026 17:45:29 +0100, Denis V. Lunev wrote:
> The v1 table starts with 3 levels and grows to 6 on the first IOVA
> dma-iommu takes from the top of the 64-bit space, once the 32-bit space
> runs out.
The table stops at amd_iommu_hpt_level. On parts whose EFR[HATS] allows
only 4 or 5 levels it never reaches 6, and the race is the same there.
> - write_seqcount_end(&pgtable->seqcount);
> -
> amd_iommu_update_and_flush_device_table(domain);
> + write_seqcount_end(&pgtable->seqcount);
- Please send a separate backport of upstream 1e75a8255f11c ("iommu/amd:
Wait for completion instead of returning early in
iommu_completion_wait()"). I created a task for that:
https://virtuozzo.atlassian.net/browse/VSTOR-154424
This fix relies on domain_flush_complete() having waited for the DTE
invalidation before write_seqcount_end(). In vz10
iommu_completion_wait() still tests need_sync without iommu->lock.
If another CPU queues a completion wait on the same IOMMU right after
our invalidation, it clears need_sync. Our wait then returns before the
IOMMU has run the invalidation, and a mapper can hand out a 64-bit IOVA
while the old DTE is still cached.
The window is much smaller than the one this patch closes, but it opens
under the same heavy DMA load. The stable trees have the fix, vz10 does
not yet. It needs a small adaptation, cmd_sem_val is still an atomic64_t
here.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
2026-11-10 16:45 [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows Denis V. Lunev
2026-11-10 16:53 ` no-reply
2026-11-10 17:53 ` Konstantin Khorenko
@ 2026-11-10 17:53 ` Konstantin Khorenko
2 siblings, 0 replies; 5+ messages in thread
From: Konstantin Khorenko @ 2026-11-10 17:53 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
On Tue, 10 Nov 2026 17:45:29 +0100, Denis V. Lunev wrote:
> iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
Applied to branch-rh10-6.12.0-211.52.1.22.x.vz10-ovz, will be released in rh10-6.12.0-211.52.1.22.20.vz10, thanks!
[1/1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows
https://bitbucket.org/virtuozzocore/vzkernel.vzs/commits/1ed4816af9e9
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-10 17:54 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-10 16:45 [Devel] [PATCH hci-8.1] iommu/amd/pgtbl: fix DMA faults on IOVAs mapped while the table grows Denis V. Lunev
2026-11-10 16:53 ` no-reply
2026-11-10 16:56 ` Denis V. Lunev
2026-11-10 17:53 ` Konstantin Khorenko
2026-11-10 17:53 ` Konstantin Khorenko
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.