[Devel] [PATCH VZ10] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available

[Devel] [PATCH VZ10] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available

[Pavel Tikhomirov]

From: Sean Christopherson <seanjc@google.com>

Check for a "stale" page fault, i.e. for an invalid and/or obsolete root,
after making MMU pages available for the shadow MMU.  If reclaiming shadow
pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to
map memory into an invalid root.  On its own, populating an invalid root is
"fine", but because child shadow pages inherit their parent's role, any
children created during the map/fetch will be created as invalid pages,
thus violating KVM's invariant that invalid pages are never on the list of
active MMU pages.

Note, the underlying flaw has existed since KVM first started tracking
invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root
pagetables"), but the true badness only came along in 2020 (Linux 5.9)
with the invariant that invalid shadow pages can't be on the list of
active pages.

Note #2, inheriting role.invalid when creating child shadow pages is also
far from ideal; that flaw will be addressed separately.

Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

CVE-2026-64561 [Zapscape]
https://virtuozzo.atlassian.net/browse/VSTOR-140678
(cherry picked from commit 2abd5287f08319fa35764566b15c6e22cb1068db)
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
---
 arch/x86/kvm/mmu/mmu.c         |  9 +++++----
 arch/x86/kvm/mmu/paging_tmpl.h | 10 ++++++----
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 6472b0e1b45db..04591cfe4f425 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4788,16 +4788,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 	if (r != RET_PF_CONTINUE)
 		return r;
 
-	r = RET_PF_RETRY;
 	write_lock(&vcpu->kvm->mmu_lock);
 
-	if (is_page_fault_stale(vcpu, fault))
-		goto out_unlock;
-
 	r = make_mmu_pages_available(vcpu);
 	if (r)
 		goto out_unlock;
 
+	if (is_page_fault_stale(vcpu, fault)) {
+		r = RET_PF_RETRY;
+		goto out_unlock;
+	}
+
 	r = direct_map(vcpu, fault);
 
 out_unlock:
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index ed762bb4b007b..af220c9c8ab8d 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -827,15 +827,17 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 	}
 #endif
 
-	r = RET_PF_RETRY;
 	write_lock(&vcpu->kvm->mmu_lock);
 
-	if (is_page_fault_stale(vcpu, fault))
-		goto out_unlock;
-
 	r = make_mmu_pages_available(vcpu);
 	if (r)
 		goto out_unlock;
+
+	if (is_page_fault_stale(vcpu, fault)) {
+		r = RET_PF_RETRY;
+		goto out_unlock;
+	}
+
 	r = FNAME(fetch)(vcpu, fault, &walker);
 
 out_unlock:
-- 
2.55.0

Re: [Devel] [PATCH VZ10] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available

[Vasileios Almpanis]

Reviewed-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>

On 8/7/26 3:47 PM, Pavel Tikhomirov wrote:
> From: Sean Christopherson <seanjc@google.com>
>
> Check for a "stale" page fault, i.e. for an invalid and/or obsolete root,
> after making MMU pages available for the shadow MMU.  If reclaiming shadow
> pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to
> map memory into an invalid root.  On its own, populating an invalid root is
> "fine", but because child shadow pages inherit their parent's role, any
> children created during the map/fetch will be created as invalid pages,
> thus violating KVM's invariant that invalid pages are never on the list of
> active MMU pages.
>
> Note, the underlying flaw has existed since KVM first started tracking
> invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root
> pagetables"), but the true badness only came along in 2020 (Linux 5.9)
> with the invariant that invalid shadow pages can't be on the list of
> active pages.
>
> Note #2, inheriting role.invalid when creating child shadow pages is also
> far from ideal; that flaw will be addressed separately.
>
> Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
> Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages")
> Cc: stable at vger.kernel.org
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>
> CVE-2026-64561 [Zapscape]
> https://virtuozzo.atlassian.net/browse/VSTOR-140678
> (cherry picked from commit 2abd5287f08319fa35764566b15c6e22cb1068db)
> Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
> ---
>   arch/x86/kvm/mmu/mmu.c         |  9 +++++----
>   arch/x86/kvm/mmu/paging_tmpl.h | 10 ++++++----
>   2 files changed, 11 insertions(+), 8 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index 6472b0e1b45db..04591cfe4f425 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -4788,16 +4788,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
>   	if (r != RET_PF_CONTINUE)
>   		return r;
>   
> -	r = RET_PF_RETRY;
>   	write_lock(&vcpu->kvm->mmu_lock);
>   
> -	if (is_page_fault_stale(vcpu, fault))
> -		goto out_unlock;
> -
>   	r = make_mmu_pages_available(vcpu);
>   	if (r)
>   		goto out_unlock;
>   
> +	if (is_page_fault_stale(vcpu, fault)) {
> +		r = RET_PF_RETRY;
> +		goto out_unlock;
> +	}
> +
>   	r = direct_map(vcpu, fault);
>   
>   out_unlock:
> diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
> index ed762bb4b007b..af220c9c8ab8d 100644
> --- a/arch/x86/kvm/mmu/paging_tmpl.h
> +++ b/arch/x86/kvm/mmu/paging_tmpl.h
> @@ -827,15 +827,17 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
>   	}
>   #endif
>   
> -	r = RET_PF_RETRY;
>   	write_lock(&vcpu->kvm->mmu_lock);
>   
> -	if (is_page_fault_stale(vcpu, fault))
> -		goto out_unlock;
> -
>   	r = make_mmu_pages_available(vcpu);
>   	if (r)
>   		goto out_unlock;
> +
> +	if (is_page_fault_stale(vcpu, fault)) {
> +		r = RET_PF_RETRY;
> +		goto out_unlock;
> +	}
> +
>   	r = FNAME(fetch)(vcpu, fault, &walker);
>   
>   out_unlock:

-- 
Best regards, Vasileios Almpanis
Software Developer, Virtuozzo.

[Devel] [PATCH RHEL10 COMMIT] ms/KVM: x86: Check for invalid/obsolete root *after* making MMU pages available

[Konstantin Khorenko]

The commit is pushed to "branch-rh10-6.12.0-211.39.1.16.x.vz10-ovz" and will appear at git at bitbucket.org:openvz/vzkernel.git
after rh10-6.12.0-211.39.1.16.3.vz10
------>
commit 67f9b973e36d988ee141f0c7881255f77d430a6c
Author: Sean Christopherson <seanjc@google.com>
Date:   Fri Aug 7 15:47:27 2026 +0200

    ms/KVM: x86: Check for invalid/obsolete root *after* making MMU pages available
    
    Check for a "stale" page fault, i.e. for an invalid and/or obsolete root,
    after making MMU pages available for the shadow MMU.  If reclaiming shadow
    pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to
    map memory into an invalid root.  On its own, populating an invalid root is
    "fine", but because child shadow pages inherit their parent's role, any
    children created during the map/fetch will be created as invalid pages,
    thus violating KVM's invariant that invalid pages are never on the list of
    active MMU pages.
    
    Note, the underlying flaw has existed since KVM first started tracking
    invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root
    pagetables"), but the true badness only came along in 2020 (Linux 5.9)
    with the invariant that invalid shadow pages can't be on the list of
    active pages.
    
    Note #2, inheriting role.invalid when creating child shadow pages is also
    far from ideal; that flaw will be addressed separately.
    
    Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
    Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages")
    Cc: stable at vger.kernel.org
    Signed-off-by: Sean Christopherson <seanjc@google.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
    
    CVE-2026-64561 [Zapscape]
    Feature: fix ms/kvm
    https://virtuozzo.atlassian.net/browse/VSTOR-140678
    (cherry picked from commit 2abd5287f08319fa35764566b15c6e22cb1068db)
    Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
    Reviewed-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
---
 arch/x86/kvm/mmu/mmu.c         |  9 +++++----
 arch/x86/kvm/mmu/paging_tmpl.h | 10 ++++++----
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 6472b0e1b45db..04591cfe4f425 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -4788,16 +4788,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 	if (r != RET_PF_CONTINUE)
 		return r;
 
-	r = RET_PF_RETRY;
 	write_lock(&vcpu->kvm->mmu_lock);
 
-	if (is_page_fault_stale(vcpu, fault))
-		goto out_unlock;
-
 	r = make_mmu_pages_available(vcpu);
 	if (r)
 		goto out_unlock;
 
+	if (is_page_fault_stale(vcpu, fault)) {
+		r = RET_PF_RETRY;
+		goto out_unlock;
+	}
+
 	r = direct_map(vcpu, fault);
 
 out_unlock:
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index ed762bb4b007b..af220c9c8ab8d 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -827,15 +827,17 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 	}
 #endif
 
-	r = RET_PF_RETRY;
 	write_lock(&vcpu->kvm->mmu_lock);
 
-	if (is_page_fault_stale(vcpu, fault))
-		goto out_unlock;
-
 	r = make_mmu_pages_available(vcpu);
 	if (r)
 		goto out_unlock;
+
+	if (is_page_fault_stale(vcpu, fault)) {
+		r = RET_PF_RETRY;
+		goto out_unlock;
+	}
+
 	r = FNAME(fetch)(vcpu, fault, &walker);
 
 out_unlock: