Re: [PATCH v3 hci-8.0/hci-8.1 0/6] i386: keep CPU features of guests from older nodes on vz machine types #VSTOR-152264

[Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>]

On 10/29/26 3:30 PM, Andrey Drobyshev wrote:
>> A guest started on an older node can have a CPU feature that the
>> destination host no longer offers: TSX disabled by microcode, a newer
>> kernel that stops offering spec-ctrl or bus-lock-detect on AMD, or SVM
>> leaf features that QEMU 2.12 on Vz7 kept with svm off. QEMU filters the
>> feature on the destination, and libvirt refuses the live migration,
>> restore or snapshot revert:
>>
>>   guest CPU doesn't match specification: missing features: hle,rtm
>>
>> The guest read its CPUID at boot, so the bit has to stay until the guest
>> restarts.
>>
>> Until now libvirt relaxed the check, in three places that behave
>> differently. One of them crashes libvirtd on the destination of any
>> migration of a host-passthrough VM with a vz7 machine type
>> (VSTOR-152247). This series keeps the bits in QEMU instead: one
>> x-...-compat X86CPU property per feature, all turned on by a new
>> pc_vz_9_2_compat on vz9.2.0 and every older vz machine type. Release
>> 7.x ships up to vz9.1.0, and vz9.2.0 is the last type based on a RHEL 9
>> machine type. vz10.x types and -cpu host expansion stay strict. A
>> property keeps a bit only when the CPU model or the command line asks
>> for it and the condition in the commit message holds; KVM ignores
>> CPUID bits it does not support.
>>
>> Patch 1 lets a row of the vz machine type tables name a fourth compat
>> array, which the i440fx vz7.16.0 row needs. Patches 2 to 6 add one
>> property each:
>>
>>   2  hle, rtm                      PSBM-149402 (field: VSTOR-75689,
>>                                    VSTOR-105458)
>>   3  spec-ctrl, AMD                VSTOR-151770
>>   4  bus-lock-detect, AMD          VSTOR-152291
>>   5  v-vmsave-vmload, AMD          code review, nested guests on Zen 4
>>                                    client parts, no field report
>>   6  SVM leaf features, svm off    PSBM-144630
>>
>> The same patches apply to hci-8.0 and hci-8.1 without changes; the
>> base given below is hci-dev. On 8.0 they also fix the migration on
>> their own: hci-8.0 libvirt keeps its relaxations, and they no longer
>> see a filtered bit. That covers bus-lock-detect (VSTOR-152291), which
>> hci-8.0 libvirt does not relax. The vz7 CPU fixup there still runs on
>> migration (with the NULL check from VSTOR-152247) and forces npt and
>> nrip-save off, which the guest does not see, because leaf 0x8000000a
>> is hidden without svm. Only the destination needs the new qemu-kvm:
>> the compat acts when the incoming QEMU process starts.
>>
>> On hci-dev the libvirt side drops its relaxations and the qemu.d hooks
>> in libvirt.ovz pr/den-VSTOR-152247-drop-vz-cpu-relaxations. It needs
>> this series in qemu-kvm first.
>>
>> Testing:
>> - dev-cl04, EPYC 7443P: a saved guest whose migration cookie CPU was
>>   rewritten to the shape of a Vz7 host-model guest
>>   (svm=off,npt=on,nrip-save=on, plus spec-ctrl) fails to restore
>>   without the series with "missing features: npt,nrip-save" and
>>   "missing features: spec-ctrl". With it the guest restores, the
>>   features stay on and filtered-features is empty. Stock 8.1 libvirt,
>>   which carries the same relaxations as hci-8.0, restores it with no
>>   warning as well.
>> - sqvm2-1015, Broadwell without TSX: the VSTOR-75689 guest CPU fails
>>   with "missing features: hle,rtm" without the series and keeps hle
>>   and rtm with it.
>> - sqvm2-1013 <-> sqvm2-1015, with the libvirt PR: live migration in
>>   both directions of a pc-i440fx-vz7.12.0 host-passthrough guest and
>>   of the VSTOR-75689 Broadwell-IBRS guest. No crash, no warning,
>>   hle/rtm kept, the CPU stays match='exact'. Stock libvirt crashes on
>>   the first of them.
>> - hci-8.0 and hci-8.1 builds: -cpu EPYC-Milan,svm=off,npt=on,
>>   nrip-save=on,enforce and -cpu EPYC-Milan,spec-ctrl=on,enforce start
>>   on vz7.12.0, vz9.0.0 and vz9.2.0 and are refused on vz10.x. A fresh
>>   Broadwell-IBRS start with check='full' is refused on vz10.1.0.
>> - both builds, TCG: -cpu EPYC-Milan,svm=off warns about the same
>>   features on vz9.2.0 as on vz10.0.0, none from leaf 0x8000000a.
>> - both builds, qtest-x86_64: ivshmem-test, test-hmp and qom-test fail
>>   as on the base (qemu_ram_free assertion, isapc aborting at start).
>> - both builds, spec-ctrl on vz9.2.0 and vz7.16.0: EPYC-Milan with
>>   spec-ctrl=on,enforce starts, also with only one of ibrs, amd-stibp
>>   and amd-ssbd left; EPYC-IBPB, or EPYC-Milan without ibpb or without
>>   all three, is refused. Everything is refused on vz10.1.0.
>> - bus-lock-detect and v-vmsave-vmload are build-tested only: no Zen 5
>>   or Zen 4 client host was available.
>>
>> Changes v2 -> v3:
>> - 3/6: keep spec-ctrl only when the guest also has the amd-* bits that
>>   give it the MSRs, not only the host (Andrey)
>> - 2/6: pc_vz_9_2_compat first in the vz7.16.0 row (Andrey); TSX_CTRL
>>   makes XBEGIN abort, not #UD, and libvirt check='partial' still
>>   refuses a fresh start
>> - 5/6, 6/6: commit messages clarified
>> - R-b on 1/6, A-b on 2/6, 4/6, 5/6 and 6/6
>>
>> Changes v1 -> v2:
>> - 6/6: keep SVM leaf features past the svm dependency under KVM only,
>>   TCG warned about them
>> - 2/6, 6/6, cover: commit messages clarified after the Sashiko review
>>
>> v2: http://lore.virtuozzo.com/qemu/20261029113436.93521-1-den@openvz.org/
>> v1: http://lore.virtuozzo.com/kernel/20261029095645.4173762-1-den@openvz.org/
>>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>>
>> Denis V. Lunev (6):
>>   i386/pc: raise the limit of compat arrays per Virtuozzo machine type
>>     to four #VSTOR-152264
>>   i386: keep hle and rtm on Virtuozzo machine types up to vz9.2.0
>>     #PSBM-149402
>>   i386: keep spec-ctrl for AMD guests on vz machine types up to vz9.2.0
>>     #VSTOR-151770
>>   i386: keep bus-lock-detect for AMD guests on vz machine types up to
>>     vz9.2.0 #VSTOR-152291
>>   i386: keep v-vmsave-vmload for nested AMD guests on vz types up to
>>     vz9.2.0 #VSTOR-152264
>>   i386: keep SVM leaf features without svm on vz types up to vz9.2.0
>>     #PSBM-144630
>>
>>  hw/i386/pc.c         |  9 ++++++
>>  hw/i386/pc_piix.c    |  5 ++--
>>  hw/i386/pc_q35.c     |  4 +--
>>  include/hw/i386/pc.h | 10 +++++--
>>  target/i386/cpu.c    | 68 +++++++++++++++++++++++++++++++++++++++++++-
>>  target/i386/cpu.h    | 11 +++++++
>>  6 files changed, 100 insertions(+), 7 deletions(-)
> 
> Major: vz10.x AMD guests from 55.x-kernel nodes are left out (patch 3 / cover letter)
> - The SPEC_CTRL gate change (AMD_IBPB_RET required) entered vzkernel with the RHEL 10.1 import, commit 9f055df113432, first tag rh10-6.12.0-211.16.1..vz10.
>   All earlier vz10 kernels (rh10-6.12.0-55..vz10) still advertise spec-ctrl on every AMD host with IBPB+IBRS, i.e. on Zen 1-3.
> - Release-kernel tags rk-6.12.0-55.52.1.5.28.vz10-v10.2 and rk-6.12.0-55.52.1.5.22.vz10-v10.3 show shipped 8.0 nodes ran 55.x. Host-model guests started
>   there carry vz10.0.0/vz10.1.0 and spec-ctrl as require.
> - After such a node moves to a 211.x kernel those guests fail migration, restore and revert exactly as in VSTOR-151770, but pc_vz_9_2_compat does not apply
>   to vz10.x. The libvirt kludge being reverted in PR 392 (d7e2f96f877f) covered every machine type, so the pair of series regresses this population relative to the kludge.
> - Fix: enable x-spec-ctrl-compat for vz10.0.0 and vz10.1.0 as well, with only the next new type strict, or keep the libvirt spec-ctrl kludge until then. The other four properties are fine at vz9.2.0: BLD and VMSAVE/VMLOAD changes are in every vz10 kernel, TSX is host-config, not kernel-version.
> 
> Medium: tsx_compat has no host guard (patch 2)
> - kept = HLE|RTM whenever the vendor is Intel. The message says it "also covers Haswell parts with the TSX erratum" and that XBEGIN aborts. On
>   HSW136-microcode parts and on silicon without TSX, XBEGIN raises #UD, so a guest that selected RTM paths at startup gets SIGILL.
> - Not a regression against the libvirt skip, which had the same hole, but the message overclaims. Guard suggestion: keep only when the host still has TSX
>   silicon, i.e. ARCH_CAPABILITIES.TSX_CTRL, CPUID.7.EDX rtm-always-abort or tsx-force-abort, and never when host_tsx_broken().
> 
> Low: Hygon hosts excluded (patches 3, 4)
> - IS_AMD_CPU matches AuthenticAMD only. Dhyana (Zen 1) hosts lose spec-ctrl on 211.x the same way and get nothing from the series. Add the HygonGenuine
>   vendor check if Hygon is in the fleet.
> 
> Low: bus-lock-detect kept bit on 211.x (patch 4)
>   - svm_set_msr rejects any DEBUGCTL bit but LBR (DEBUGCTL_RESERVED_BITS = ~DEBUGCTLMSR_LBR), so a rebooted guest enabling BLD gets #GP and one "unchecked MSR access error" splat. A migrated guest loses BLD silently since QEMU does not migrate DEBUGCTL. Cosmetic, worth a sentence in the message.
> 

Disregard this. .211 is the first VZ10 kernel in our production.

Ack for the series
Applied onto hci-dev