[Devel] [PATCH hci-8.1] KVM: nVMX: Keep the emulated VMX-preemption timer on L1's TSC

[Devel] [PATCH hci-8.1] KVM: nVMX: Keep the emulated VMX-preemption timer on L1's TSC

[Eva Kurchatova]

KVM emulates the VMX-preemption timer of L2 with an hrtimer, and turns
the timer value into nanoseconds with the TSC frequency of the vCPU.
The hrtimer runs on CLOCK_MONOTONIC, which NTP slews against the TSC,
so the VM-exit comes late or early in L1's TSC while the host clock is
slewed. vmx_preemption_timer_test then fails with "L2 PT expiry TSC >
L2 TSC deadline" when the clock is slowed down, and with "L1 PT expiry
TSC < L1 TSC deadline" when it is sped up.

Arm the hrtimer an eighth early, and re-arm it from its callback until
L1's TSC reaches the deadline that KVM keeps since
commit 850448f35aaf ("KVM: nVMX: Fix VMX preemption timer migration").
Count the timer value that is left from that deadline when starting
the timer and when saving it to vmcs12, as the hrtimer no longer tracks
it.

Fixes: f4124500c2c1 ("KVM: nVMX: Fully emulate preemption timer")
https://virtuozzo.atlassian.net/browse/VSTOR-152991
Feature: fix ms/kvm

Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
 arch/x86/kvm/vmx/nested.c | 67 +++++++++++++++++++++++++--------------
 1 file changed, 43 insertions(+), 24 deletions(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 8c771d8c35e3..3f6a606e97a0 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -2206,15 +2206,51 @@ void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu)
 	vmx->nested.need_vmcs12_to_shadow_sync = false;
 }
 
+/* Ticks left until L1's TSC reaches the deadline, at most the 32-bit value */
+static u64 vmx_preemption_timer_left(struct kvm_vcpu *vcpu)
+{
+	u64 deadline = to_vmx(vcpu)->nested.preemption_timer_deadline;
+	u64 now = kvm_read_l1_tsc(vcpu, rdtsc()) >>
+		  VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE;
+
+	return deadline > now ? min(deadline - now, (u64)U32_MAX) : 0;
+}
+
+/*
+ * The hrtimer runs on CLOCK_MONOTONIC, which NTP may slew against the TSC by
+ * up to 10%: fire an eighth early while more than 100us are left, and let the
+ * callback re-arm until the deadline is reached.
+ */
+static ktime_t vmx_preemption_timer_expiry(struct kvm_vcpu *vcpu, u64 ticks)
+{
+	u64 ns = (ticks << VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE) * 1000000;
+
+	do_div(ns, vcpu->arch.virtual_tsc_khz);
+	if (ns > 100 * NSEC_PER_USEC)
+		ns -= ns >> 3;
+	return ktime_add_ns(ktime_get(), ns);
+}
+
+static void vmx_preemption_timer_expire(struct kvm_vcpu *vcpu)
+{
+	to_vmx(vcpu)->nested.preemption_timer_expired = true;
+	kvm_make_request(KVM_REQ_EVENT, vcpu);
+	kvm_vcpu_kick(vcpu);
+}
+
 static enum hrtimer_restart vmx_preemption_timer_fn(struct hrtimer *timer)
 {
 	struct vcpu_vmx *vmx =
 		container_of(timer, struct vcpu_vmx, nested.preemption_timer);
+	u64 left = vmx_preemption_timer_left(&vmx->vcpu);
 
-	vmx->nested.preemption_timer_expired = true;
-	kvm_make_request(KVM_REQ_EVENT, &vmx->vcpu);
-	kvm_vcpu_kick(&vmx->vcpu);
+	if (left) {
+		hrtimer_set_expires(timer,
+				    vmx_preemption_timer_expiry(&vmx->vcpu, left));
+		return HRTIMER_RESTART;
+	}
 
+	vmx_preemption_timer_expire(&vmx->vcpu);
 	return HRTIMER_NORESTART;
 }
 
@@ -2231,7 +2267,7 @@ static u64 vmx_calc_preemption_timer_value(struct kvm_vcpu *vcpu)
 			vmcs12->vmx_preemption_timer_value + l1_scaled_tsc;
 		vmx->nested.has_preemption_timer_deadline = true;
 	}
-	return vmx->nested.preemption_timer_deadline - l1_scaled_tsc;
+	return vmx_preemption_timer_left(vcpu);
 }
 
 static void vmx_start_preemption_timer(struct kvm_vcpu *vcpu,
@@ -2244,18 +2280,15 @@ static void vmx_start_preemption_timer(struct kvm_vcpu *vcpu,
 	 * a VMExit prior to executing any instructions in the guest.
 	 */
 	if (preemption_timeout == 0) {
-		vmx_preemption_timer_fn(&vmx->nested.preemption_timer);
+		vmx_preemption_timer_expire(vcpu);
 		return;
 	}
 
 	if (vcpu->arch.virtual_tsc_khz == 0)
 		return;
 
-	preemption_timeout <<= VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE;
-	preemption_timeout *= 1000000;
-	do_div(preemption_timeout, vcpu->arch.virtual_tsc_khz);
 	hrtimer_start(&vmx->nested.preemption_timer,
-		      ktime_add_ns(ktime_get(), preemption_timeout),
+		      vmx_preemption_timer_expiry(vcpu, preemption_timeout),
 		      HRTIMER_MODE_ABS_PINNED);
 }
 
@@ -4428,20 +4461,6 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu)
 	return vmx_complete_nested_posted_interrupt(vcpu);
 }
 
-static u32 vmx_get_preemption_timer_value(struct kvm_vcpu *vcpu)
-{
-	ktime_t remaining =
-		hrtimer_get_remaining(&to_vmx(vcpu)->nested.preemption_timer);
-	u64 value;
-
-	if (ktime_to_ns(remaining) <= 0)
-		return 0;
-
-	value = ktime_to_ns(remaining) * vcpu->arch.virtual_tsc_khz;
-	do_div(value, 1000000);
-	return value >> VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE;
-}
-
 static bool is_vmcs12_ext_field(unsigned long field)
 {
 	switch (field) {
@@ -4597,7 +4616,7 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
 	    vmcs12->vm_exit_controls & VM_EXIT_SAVE_VMX_PREEMPTION_TIMER &&
 	    !vmx->nested.nested_run_pending)
 		vmcs12->vmx_preemption_timer_value =
-			vmx_get_preemption_timer_value(vcpu);
+			vmx_preemption_timer_left(vcpu);
 
 	/*
 	 * In some cases (usually, nested EPT), L2 is allowed to change its
-- 
2.56.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH hci-8.1] KVM: nVMX: Keep the emulated VMX-preemption timer on L1's TSC

[no-reply]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] KVM: nVMX: no slew margin in the last 100us of the emulated preemption timer
- [Low] KVM: nVMX: commit message carries downstream-only Jira URL and Feature tag
--

commit 88673c58251bfdcecfe5cfb0887dacefd547c04a
Author: Eva Kurchatova <eva.kurchatova@virtuozzo.com>

KVM: nVMX: Keep the emulated VMX-preemption timer on L1's TSC

This patch arms the hrtimer that emulates the L2 VMX-preemption timer an
eighth early. The timer callback re-arms it until L1's TSC reaches the saved
deadline. The remaining timer value is now computed from that deadline, both
when starting the timer and when saving it to vmcs12.

> KVM: nVMX: Keep the emulated VMX-preemption timer on L1's TSC

[ ... ]

> Fixes: f4124500c2c1 ("KVM: nVMX: Fully emulate preemption timer")
> https://virtuozzo.atlassian.net/browse/VSTOR-152991
> Feature: fix ms/kvm

[Severity: Low]
This isn't a bug, but the commit message carries a private Jira URL
(virtuozzo.atlassian.net/browse/VSTOR-152991) and a "Feature: fix ms/kvm"
tag. Neither is meaningful to upstream reviewers. Could they be dropped
before this is posted upstream?

> diff --git a/arch/x86/kernel/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c

[ ... ]

> +static ktime_t vmx_preemption_timer_expiry(struct kvm_vcpu *vcpu, u64 ticks)
> +{
> +	u64 ns = (ticks << VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE) * 1000000;
> +
> +	do_div(ns, vcpu->arch.virtual_tsc_khz);
> +	if (ns > 100 * NSEC_PER_USEC)
> +		ns -= ns >> 3;
> +	return ktime_add_ns(ktime_get(), ns);
> +}

[Severity: Low]
Can the emulated timer still fire late in L1's TSC when CLOCK_MONOTONIC is
slewed slower than the TSC?

The early margin is only applied when more than 100us remain:

    if (ns > 100 * NSEC_PER_USEC)
        ns -= ns >> 3;

When 100us or less remain, either at the initial hrtimer_start() or on the
last re-arm hop, the hrtimer is armed for the unadjusted conversion.

vmx_preemption_timer_fn() only re-arms when the timer fires early. A late
fire is not corrected. The lateness is bounded by roughly slew times 100us,
which is about 10us at the 10% slew the comment assumes.

The commit message says the "L2 PT expiry TSC > L2 TSC deadline" failure is
fixed for a slowed clock. Could a strict test such as
vmx_preemption_timer_test still fail intermittently for short timeouts?
Should the commit message mention this limitation?

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261106134316.84189-1-eva.kurchatova@virtuozzo.com?part=1

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel