[PATCH hci-8.0/hci-8.1] i386: hide la57 at fresh start on vz machine types up to vz10.0.0 #VSTOR-152405

[PATCH hci-8.0/hci-8.1] i386: hide la57 at fresh start on vz machine types up to vz10.0.0 #VSTOR-152405

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

The vz10 OVMF enables 5-level paging whenever the CPU has la57, so on
AMD EPYC 9004/9005 hosts with "-cpu host" Windows boots its LA57
kernel. That kernel bugchecks with UNSUPPORTED_PROCESSOR on the Intel
only ARCH_CAPABILITIES MSR, which pc-q35-vz10.0.0 and older still show
to AMD guests (x-arch-cap-always-on). There is no IDT yet, so the guest
triple-faults and reboots in a loop (ASUP-3871). The vz9 OVMF never
used 5-level paging, so these guests did not see this before.

Add x-no-la57-on-start and turn it on in pc_vz_10_0_compat, which
covers vz10.0.0 and every older vz machine type. On a fresh start a
"-cpu host" guest loses la57 unless la57=on is given explicitly. The
bit is reported in filtered-features, so libvirt writes la57=off into
the live definition and migration, save and snapshots carry it.

An incoming QEMU keeps la57: a guest from an older node may already run
with CR4.LA57 set. A hotplugged vCPU follows the boot vCPUs. Named CPU
models are not touched, la57 is part of their definition.

Signed-off-by: Denis V. Lunev <den@openvz.org>
---
The patch applies to hci-8.0 without changes; the base given below is
hci-dev, on top of "i386: keep CPU features of guests from older nodes
on vz machine types".

Customer case ASUP-3871, bug VSTOR-152405. New VMs get vz10.0.0 only
because nova pins it instead of using the q35 alias; that is
VSTOR-152413. pc-q35-vz10.1.0 already hides ARCH_CAPABILITIES on AMD and
is not touched here.

Testing, hci-8.0 build, bg-sdc2-e19-07 (EPYC 9355, VHI 8.0.0 packages):
- -cpu host, fresh start: la57 hidden and listed in
  unavailable-features on vz10.0.0 and vz9.2.0; kept with la57=on,
  on vz10.1.0 and with EPYC-Genoa.
- live migration from stock qemu-kvm 10.1.0-20.vz10.16 to the patched
  binary: completes, la57 kept on the destination.
- vCPU hotplug follows the boot vCPU, after a fresh start and after an
  incoming migration.
- Windows Server 2025 GA ISO (26100.1742) with the nova command line of
  the customer VM, pc-q35-vz10.0.0, -cpu host and the Hyper-V
  enlightenments: OVMF stays in 4-level paging and Setup starts. Stock
  qemu-kvm triple-faults there with bugcheck 0x5D (MSR 0x10a).
- Not tested: libvirt writing la57=off into the live XML and carrying
  it to the destination. That follows from virCPUx86UpdateLive for
  host-passthrough with check='none' and does not depend on the libvirt
  relaxations dropped in libvirt.ovz PR 392.

 hw/i386/pc.c         |  5 +++++
 hw/i386/pc_piix.c    |  2 +-
 hw/i386/pc_q35.c     |  2 +-
 include/hw/i386/pc.h |  3 +++
 target/i386/cpu.c    | 27 +++++++++++++++++++++++++++
 target/i386/cpu.h    |  6 ++++++
 6 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 9b6b51bd5f7..2834982dab4 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -548,6 +548,11 @@ GlobalProperty pc_vz_compat[] = {
 };
 const size_t pc_vz_compat_len = G_N_ELEMENTS(pc_vz_compat);
 
+GlobalProperty pc_vz_10_0_compat[] = {
+    { TYPE_X86_CPU, "x-no-la57-on-start", "on" },
+};
+const size_t pc_vz_10_0_compat_len = G_N_ELEMENTS(pc_vz_10_0_compat);
+
 GlobalProperty pc_vz_9_2_compat[] = {
     { TYPE_X86_CPU, "x-tsx-compat", "on" },
     { TYPE_X86_CPU, "x-spec-ctrl-compat", "on" },
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index b1b595078eb..ab29ea0cd12 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -1057,7 +1057,7 @@ DEFINE_I440FX_RHEL_MACHINE(7, 2, 0);
 /*  maj min mic         basemt      isdefault                              */
 #define VZ_PC_MT(X) \
     X(10, 1, 0, MT_VER_SYM(10, 0, 0), true)                                 \
-    X(10, 0, 0, MT_VER_SYM(10, 0, 0), false)                                \
+    X(10, 0, 0, MT_VER_SYM(10, 0, 0), false, pc_vz_10_0_compat)             \
     X(7, 16, 0, MT_VER_SYM(7, 6, 0),  false, pc_vz_9_2_compat,              \
                                              hw_compat_vz_9_1,              \
                                              hw_compat_vz_7_16,             \
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 019239bec6a..51539f22399 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -973,7 +973,7 @@ DEFINE_Q35_RHEL_MACHINE(7, 3, 0);
 /*  maj min mic         basemt      isdefault                              */
 #define VZ_Q35_MT(X)                                                        \
     X(10, 1, 0, MT_VER_SYM(10, 2, 0), true)                                 \
-    X(10, 0, 0, MT_VER_SYM(10, 0, 0), false)                                \
+    X(10, 0, 0, MT_VER_SYM(10, 0, 0), false, pc_vz_10_0_compat)             \
     X(9, 2,  0, MT_VER_SYM(9, 6, 0),  false, pc_vz_9_2_compat)              \
     X(9, 1,  0, MT_VER_SYM(9, 4, 0),  false, hw_compat_vz_9_1)              \
     X(9, 0,  0, MT_VER_SYM(9, 2, 0),  false)                                \
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index bd73426644e..021b139b872 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -364,6 +364,9 @@ extern const size_t pc_rhel_7_2_compat_len;
 extern GlobalProperty pc_vz_compat[];
 extern const size_t pc_vz_compat_len;
 
+extern GlobalProperty pc_vz_10_0_compat[];
+extern const size_t pc_vz_10_0_compat_len;
+
 extern GlobalProperty pc_vz_9_2_compat[];
 extern const size_t pc_vz_9_2_compat_len;
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 29e7b4b7e15..8d2ce279a72 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -40,6 +40,7 @@
 #ifndef CONFIG_USER_ONLY
 #include "confidential-guest.h"
 #include "system/reset.h"
+#include "system/runstate.h"
 #include "qapi/qapi-commands-machine.h"
 #include "system/address-spaces.h"
 #include "hw/boards.h"
@@ -9107,6 +9108,27 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
     }
 }
 
+/*
+ * Reported as filtered, so that libvirt records la57=off in the live
+ * definition and passes it on to migration, save and snapshots.
+ */
+static bool x86_cpu_hide_la57(X86CPU *cpu)
+{
+    CPUX86State *env = &cpu->env;
+
+    if (!cpu->no_la57_on_start || !X86_CPU_GET_CLASS(cpu)->max_features ||
+        !kvm_enabled() ||
+        !(env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_LA57) ||
+        (env->user_features[FEAT_7_0_ECX] & CPUID_7_0_ECX_LA57)) {
+        return false;
+    }
+    if (DEVICE(cpu)->hotplugged) {
+        return first_cpu && !(X86_CPU(first_cpu)->env.features[FEAT_7_0_ECX] &
+                              CPUID_7_0_ECX_LA57);
+    }
+    return !runstate_check(RUN_STATE_INMIGRATE);
+}
+
 /* Requested features that a vz compat property keeps despite the host */
 static uint64_t x86_cpu_compat_kept_features(X86CPU *cpu, FeatureWord w)
 {
@@ -9404,6 +9426,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         }
     }
 
+    if (x86_cpu_hide_la57(cpu)) {
+        mark_unavailable_features(cpu, FEAT_7_0_ECX, CPUID_7_0_ECX_LA57, NULL);
+    }
+
     /* On AMD CPUs, some CPUID[8000_0001].EDX bits must match the bits on
      * CPUID[1].EDX.
      */
@@ -10120,6 +10146,7 @@ static const Property x86_cpu_properties[] = {
                      bus_lock_detect_compat, false),
     DEFINE_PROP_BOOL("x-v-vmsave-vmload-compat", X86CPU,
                      v_vmsave_vmload_compat, false),
+    DEFINE_PROP_BOOL("x-no-la57-on-start", X86CPU, no_la57_on_start, false),
     DEFINE_PROP_BOOL("x-svm-features-compat", X86CPU,
                      svm_features_compat, false),
 };
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 97e38013e46..5c1a692aca3 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2342,6 +2342,12 @@ struct ArchCPU {
     bool v_vmsave_vmload_compat;
     bool svm_features_compat;
 
+    /*
+     * Virtuozzo machine types up to vz10.0.0: hide la57 from "-cpu host"
+     * when the VM starts fresh, keep it on incoming migration.
+     */
+    bool no_la57_on_start;
+
     /* Number of physical address bits supported */
     uint32_t phys_bits;
 

base-commit: 02f55118bcb0533ce4c6544458c30803d96bea4a
-- 
2.53.0

Re: [PATCH hci-8.0/hci-8.1] i386: hide la57 at fresh start on vz machine types up to vz10.0.0 #VSTOR-152405

[no-reply]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] i386: x86_cpu_hide_la57() hides la57 again on -loadvm restore
- [Low] i386: x86_cpu_hide_la57() uses runstate_check() outside !CONFIG_USER_ONLY
- [Low] i386: x86_cpu_hide_la57() also hides la57 for -cpu max, not only -cpu host
--

commit 0fe19500ac631308cc76d8b2992da28d4b1f5fb1
Author: Denis V. Lunev <den@openvz.org>

i386: hide la57 at fresh start on vz machine types up to vz10.0.0 #VSTOR-152405

This adds the x-no-la57-on-start CPU property and enables it in
pc_vz_10_0_compat for vz10.0.0 and older machine types. On a fresh
start under KVM, a "-cpu host" guest loses la57 unless la57=on is given.
An incoming migration keeps la57, and a hotplugged vCPU follows the boot
vCPUs.

> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -40,6 +40,7 @@
>  #ifndef CONFIG_USER_ONLY
>  #include "confidential-guest.h"
>  #include "system/reset.h"
> +#include "system/runstate.h"
>  #include "qapi/qapi-commands-machine.h"
>  #include "system/address-spaces.h"
>  #include "hw/boards.h"

[ ... ]

> +/*
> + * Reported as filtered, so that libvirt records la57=off in the live
> + * definition and passes it on to migration, save and snapshots.
> + */
> +static bool x86_cpu_hide_la57(X86CPU *cpu)
> +{
> +    CPUX86State *env = &cpu->env;
> +
> +    if (!cpu->no_la57_on_start || !X86_CPU_GET_CLASS(cpu)->max_features ||

[Severity: Low]
The commit message and the cpu.h comment describe this as applying to
"-cpu host". Is that the intended scope?

X86_CPU_GET_CLASS(cpu)->max_features is also set for the "max" CPU
class, not only the host class. x86_cpu_expand_features() uses the same
predicate to auto-enable host-supported features for both.

On vz10.0.0 and older machine types under KVM, "-cpu max" guests would
therefore also lose la57. Should the commit message and the comment
mention this, or should the check be narrowed to the host model?

> +        !kvm_enabled() ||
> +        !(env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_LA57) ||
> +        (env->user_features[FEAT_7_0_ECX] & CPUID_7_0_ECX_LA57)) {
> +        return false;
> +    }
> +    if (DEVICE(cpu)->hotplugged) {
> +        return first_cpu && !(X86_CPU(first_cpu)->env.features[FEAT_7_0_ECX] &
> +                              CPUID_7_0_ECX_LA57);
> +    }
> +    return !runstate_check(RUN_STATE_INMIGRATE);

[Severity: Low]
Will this build for linux-user and bsd-user i386 and x86_64 targets?

The only declaration of runstate_check() is in system/runstate.h, and
this patch adds that include inside the #ifndef CONFIG_USER_ONLY block
near the top of target/i386/cpu.c:

#ifndef CONFIG_USER_ONLY
#include "confidential-guest.h"
#include "system/reset.h"
#include "system/runstate.h"

But x86_cpu_hide_la57() is defined outside any CONFIG_USER_ONLY guard,
and x86_cpu_realizefn() calls it unconditionally. kvm_enabled() is
(0) in user-only builds, but that does not remove the runstate_check()
call from compilation, so it would be an implicit declaration there.

Could the helper and its call site be wrapped in #ifndef
CONFIG_USER_ONLY, or could a user-only stub be provided?

[Severity: Medium]
Is the la57 bit hidden again when a guest is restored with -loadvm?

Here only RUN_STATE_INMIGRATE keeps la57. With -loadvm, the vCPUs are
realized during machine init and load_snapshot() runs afterwards from
qemu_init():

    if (loadvm) {
        ...
        load_snapshot(loadvm, NULL, false, NULL, &error_fatal);

-incoming and -loadvm are mutually exclusive, so the runstate during
realize is PRELAUNCH and x86_cpu_hide_la57() returns true when
x-no-la57-on-start is set and la57=on was not given.

If the snapshot was taken from a guest started by an older QEMU, or
without la57=off, the guest may already run with CR4.LA57 set. After
-loadvm the vCPU CPUID lacks la57, so I would expect CR4 validation in
KVM_SET_SREGS to fail or the guest to misbehave. I did not confirm the
KVM behaviour in this tree.

Is there any check that detects an la57 mismatch between the saved
state and the restored CPU? As far as I can tell, nothing in QEMU
enforces consistency, and migration depends on the management layer
passing la57=off to the destination. The commit message notes that this
libvirt step was not tested.

> +}
> +
>  /* Requested features that a vz compat property keeps despite the host */
>  static uint64_t x86_cpu_compat_kept_features(X86CPU *cpu, FeatureWord w)
>  {

[ ... ]

> @@ -9404,6 +9426,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>      }
>  
> +    if (x86_cpu_hide_la57(cpu)) {
> +        mark_unavailable_features(cpu, FEAT_7_0_ECX, CPUID_7_0_ECX_LA57, NULL);
> +    }
> +

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261029174221.1068858-1-den@openvz.org?part=1