[PATCH hci-8.0 00/12] system: add a watchdog for the Big QEMU Lock

[PATCH hci-8.0 00/12] system: add a watchdog for the Big QEMU Lock

[Denis V. Lunev]

A vCPU thread reacquires the BQL on every exit that needs userspace. A
holder that keeps it for tens of seconds stops every vCPU, the guest
reports a soft lockup or panics, and qemu itself records nothing. The
guest log is the only evidence, and it cannot say whether the hypervisor
held the lock or the guest wedged on its own.

This series adds a BQL watchdog. A sampler thread watches a sequence
counter that bql_update_status() advances on every acquire and release.
One odd value across the whole window means one owner held the lock for
the whole window. On expiry the watchdog names the holder and prints the
state of every other thread:

  qemu-system-x86_64: 2026-09-16T13:21:04Z BQL held for more than 100 ms by thread 927240
    927240  qemu-system-x86  S
        pc 0x7f35a4871e06 ppoll+0xa6
        [<0>] poll_schedule_timeout.constprop.0+0x3a/0x80
        [<0>] do_sys_poll+0x1eb/0x290
        [<0>] __x64_sys_ppoll+0xe0/0x160
    927242  qemu-system-x86  S
        pc 0x7f35a487c57b syscall+0x1b
        [<0>] futex_do_wait+0x38/0x70
        [<0>] __futex_wait+0x92/0x100

Without CAP_SYS_ADMIN the kernel stack collapses to a single wchan line.
The pc line comes from signalling each thread and reading the program
counter out of its signal frame, which is what tells a thread spinning
in userspace from one asleep in a syscall.

The watchdog is off by default and stays quiet under a debugger. Two
machine properties drive it: bql-watchdog-ms sets the deadline, on the
command line or over qom-set at any time, and bql-watchdog-abort turns
the report into a core dump for hosts where the answer is worth more
than the guest.

Patches 3 to 8 are the pieces: a fix for bql_locked across the timed
condvar wait, a debugger check, the per-thread state reader and its
kernel stack and program counter extensions, and the signal reserved for
the latter. Patches 9 to 11 are the watchdog itself. Patch 12 is a
functional test that holds the BQL through an HMP qemu-io write to a
null-co node with a latency longer than the deadline.

This is the hci-8.0 port of the series posted upstream on 2026-09-15:
https://lore.kernel.org/qemu-devel/20260915203455.580999-1-den@openvz.org/

Patches 1 and 2 are the two upstream commits hci-8.0 does not carry and
the rest needs. Patch 1 gives bql_update_status(), which is where the
sequence counter hooks in, so patch 9 does not build without it; patch 2
makes real_time_iso8601() public, because the watchdog stamps its own
report rather than going through error_report(). Both are unmodified
cherry-picks.

Three things differ from the upstream posting, all because 10.1 lays the
tree out differently: the riscv signal-pc.h header goes in
host/include/riscv rather than host/include/riscv64, since meson maps
both riscv32 and riscv64 to host_arch 'riscv'; the functional test sits
in tests/functional rather than tests/functional/generic, and registers
in tests_generic_system there; and hw/core/machine.c has no
x-change-vmfd-on-reset property to anchor against. No code changed.

Built and tested on dev-cl04 with --enable-debug: func-quick passes,
including the two bql_watchdog subtests, and the report above is what
this build prints under -machine none,bql-watchdog-ms=100.

The program counter lives at a host specific offset in the signal frame,
so patch 8 reads it through a host/include header. The Linux headers for
aarch64, ppc64, riscv, s390x and loongarch64 repeat what linux-user's
host-signal.h already knows about those frames, but only the x86_64 one
has been compiled, here as upstream. The Windows half of patch 3 has not
been compiled either.

Signed-off-by: Denis V. Lunev <den@openvz.org>

Denis V. Lunev (10):
  util/qemu-thread: track bql_locked across the timed condvar wait
    #VSTOR-144507
  oslib: add qemu_debugger_attached() #VSTOR-144507
  oslib: add qemu_thread_states() #VSTOR-144507
  oslib: report kernel stacks in qemu_thread_states() #VSTOR-144507
  util/qemu-thread: reserve a signal for thread introspection
    #VSTOR-144507
  oslib: report each thread's program counter #VSTOR-144507
  system/cpus: report a BQL held past a deadline #VSTOR-144507
  system/cpus: hold off the BQL watchdog under a debugger #VSTOR-144507
  system/cpus: add bql-watchdog-abort #VSTOR-144507
  tests/functional: cover the BQL watchdog #VSTOR-144507

Peter Xu (1):
  bql: Fix bql_locked status with condvar APIs #VSTOR-144507

Vladimir Sementsov-Ogievskiy (1):
  error-report: make real_time_iso8601() public #VSTOR-144507

 host/include/aarch64/host/signal-pc.h     |  20 ++
 host/include/generic/host/signal-pc.h     |  11 +
 host/include/loongarch64/host/signal-pc.h |  20 ++
 host/include/ppc64/host/signal-pc.h       |  22 ++
 host/include/riscv/host/signal-pc.h       |  20 ++
 host/include/s390x/host/signal-pc.h       |  20 ++
 host/include/x86_64/host/signal-pc.h      |  20 ++
 hw/core/machine.c                         |  44 +++
 include/qemu/error-report.h               |   6 +
 include/qemu/main-loop.h                  |  18 ++
 include/qemu/osdep.h                      |  29 ++
 include/system/cpus.h                     |   8 +
 meson.build                               |   2 +
 qemu-options.hx                           |  11 +
 stubs/iothread-lock.c                     |   9 +
 system/cpus.c                             | 157 ++++++++++-
 tests/functional/meson.build              |   1 +
 tests/functional/test_bql_watchdog.py     |  60 ++++
 util/error-report.c                       |   3 +-
 util/oslib-posix.c                        | 316 ++++++++++++++++++++++
 util/oslib-win32.c                        |  10 +
 util/qemu-thread-common.h                 |   7 +
 util/qemu-thread-posix.c                  |   8 +-
 util/qemu-thread-win32.c                  |   4 +-
 24 files changed, 818 insertions(+), 8 deletions(-)
 create mode 100644 host/include/aarch64/host/signal-pc.h
 create mode 100644 host/include/generic/host/signal-pc.h
 create mode 100644 host/include/loongarch64/host/signal-pc.h
 create mode 100644 host/include/ppc64/host/signal-pc.h
 create mode 100644 host/include/riscv/host/signal-pc.h
 create mode 100644 host/include/s390x/host/signal-pc.h
 create mode 100644 host/include/x86_64/host/signal-pc.h
 create mode 100755 tests/functional/test_bql_watchdog.py

-- 
2.53.0

[PATCH hci-8.0 01/12] bql: Fix bql_locked status with condvar APIs #VSTOR-144507

[Denis V. Lunev]

From: Peter Xu <peterx@redhat.com>

QEMU has a per-thread "bql_locked" variable stored in TLS section, showing
whether the current thread is holding the BQL lock.

It's a pretty handy variable.  Function-wise, QEMU have codes trying to
conditionally take bql, relying on the var reflecting the locking status
(e.g. BQL_LOCK_GUARD), or in a GDB debugging session, we could also look at
the variable (in reality, co_tls_bql_locked), to see which thread is
currently holding the bql.

When using that as a debugging facility, sometimes we can observe multiple
threads holding bql at the same time. It's because QEMU's condvar APIs
bypassed the bql_*() API, hence they do not update bql_locked even if they
have released the mutex while waiting.

It can cause confusion if one does "thread apply all p co_tls_bql_locked"
and see multiple threads reporting true.

Fix this by moving the bql status updates into the mutex debug hooks.  Now
the variable should always reflect the reality.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250904223158.1276992-1-peterx@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
(cherry picked from commit c89d1c879ab0281d481e460644eb5d56ace1a038)

Backported because the BQL watchdog reads the lock status from these
hooks: it is the only place that sees every handover, including the
ones the condvar wrappers make.

Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/main-loop.h  | 18 ++++++++++++++++++
 stubs/iothread-lock.c     |  9 +++++++++
 system/cpus.c             | 14 ++++++++++++--
 util/qemu-thread-common.h |  7 +++++++
 4 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 4e2436b1968..0d55c636b21 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -270,6 +270,24 @@ void rust_bql_mock_lock(void);
  */
 bool bql_locked(void);
 
+/**
+ * mutex_is_bql:
+ *
+ * @mutex: the mutex pointer
+ *
+ * Returns whether the mutex is the BQL.
+ */
+bool mutex_is_bql(QemuMutex *mutex);
+
+/**
+ * bql_update_status:
+ *
+ * @locked: update status on whether the BQL is locked
+ *
+ * NOTE: this should normally only be invoked when the status changed.
+ */
+void bql_update_status(bool locked);
+
 /**
  * bql_block: Allow/deny releasing the BQL
  *
diff --git a/stubs/iothread-lock.c b/stubs/iothread-lock.c
index 6050c081f53..c89c9c7228f 100644
--- a/stubs/iothread-lock.c
+++ b/stubs/iothread-lock.c
@@ -34,3 +34,12 @@ void bql_block_unlock(bool increase)
     assert((new_value > bql_unlock_blocked) == increase);
     bql_unlock_blocked = new_value;
 }
+
+bool mutex_is_bql(QemuMutex *mutex)
+{
+    return false;
+}
+
+void bql_update_status(bool locked)
+{
+}
diff --git a/system/cpus.c b/system/cpus.c
index cad32a25342..5c2bdc3f3d0 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -518,6 +518,18 @@ bool qemu_in_vcpu_thread(void)
 
 QEMU_DEFINE_STATIC_CO_TLS(bool, bql_locked)
 
+bool mutex_is_bql(QemuMutex *mutex)
+{
+    return mutex == &bql;
+}
+
+void bql_update_status(bool locked)
+{
+    /* This function should only be used when an update happened.. */
+    assert(bql_locked() != locked);
+    set_bql_locked(locked);
+}
+
 static uint32_t bql_unlock_blocked;
 
 void bql_block_unlock(bool increase)
@@ -558,14 +570,12 @@ void bql_lock_impl(const char *file, int line)
 
     g_assert(!bql_locked());
     bql_lock_fn(&bql, file, line);
-    set_bql_locked(true);
 }
 
 void bql_unlock(void)
 {
     g_assert(bql_locked());
     g_assert(!bql_unlock_blocked);
-    set_bql_locked(false);
     qemu_mutex_unlock(&bql);
 }
 
diff --git a/util/qemu-thread-common.h b/util/qemu-thread-common.h
index 2af6b120853..09331843ba9 100644
--- a/util/qemu-thread-common.h
+++ b/util/qemu-thread-common.h
@@ -14,6 +14,7 @@
 #define QEMU_THREAD_COMMON_H
 
 #include "qemu/thread.h"
+#include "qemu/main-loop.h"
 #include "trace.h"
 
 static inline void qemu_mutex_post_init(QemuMutex *mutex)
@@ -39,6 +40,9 @@ static inline void qemu_mutex_post_lock(QemuMutex *mutex,
     mutex->line = line;
 #endif
     trace_qemu_mutex_locked(mutex, file, line);
+    if (mutex_is_bql(mutex)) {
+        bql_update_status(true);
+    }
 }
 
 static inline void qemu_mutex_pre_unlock(QemuMutex *mutex,
@@ -49,6 +53,9 @@ static inline void qemu_mutex_pre_unlock(QemuMutex *mutex,
     mutex->line = 0;
 #endif
     trace_qemu_mutex_unlock(mutex, file, line);
+    if (mutex_is_bql(mutex)) {
+        bql_update_status(false);
+    }
 }
 
 #endif
-- 
2.53.0

[PATCH hci-8.0 02/12] error-report: make real_time_iso8601() public #VSTOR-144507

[Denis V. Lunev]

From: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

To be reused in the following commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-ID: <20260201173633.413934-3-vsementsov@yandex-team.ru>
(cherry picked from commit 2eed5472ec424d217786af97e6212e68e49c5d0b)

Backported because the BQL watchdog stamps its report itself: the
report goes to stderr from a thread, not through error_report()'s
timestamp.

Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/error-report.h | 6 ++++++
 util/error-report.c         | 3 +--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/include/qemu/error-report.h b/include/qemu/error-report.h
index 3ae2357fda5..dc423d36071 100644
--- a/include/qemu/error-report.h
+++ b/include/qemu/error-report.h
@@ -74,4 +74,10 @@ extern bool message_with_timestamp;
 extern bool error_with_guestname;
 extern const char *error_guest_name;
 
+/*
+ * Return current datetime in ISO 8601 format.
+ * Caller is responsible to g_free() the returned string.
+ */
+char *real_time_iso8601(void);
+
 #endif
diff --git a/util/error-report.c b/util/error-report.c
index 1b17c11de19..d6b7448183a 100644
--- a/util/error-report.c
+++ b/util/error-report.c
@@ -169,8 +169,7 @@ static void print_loc(void)
     }
 }
 
-static char *
-real_time_iso8601(void)
+char *real_time_iso8601(void)
 {
     g_autoptr(GDateTime) dt = g_date_time_new_now_utc();
     return g_date_time_format_iso8601(dt);
-- 
2.53.0

[PATCH hci-8.0 03/12] util/qemu-thread: track bql_locked across the timed condvar wait #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

qemu_cond_timedwait_ts() releases and retakes the mutex around
pthread_cond_timedwait(), but calls the two trace points by hand
instead of going through the mutex hooks, so it never reaches
bql_update_status(). A thread waiting there still reports bql_locked
true while the mutex is released, which is the confusion the hooks were
given that job to end. migration/cpu-throttle.c reaches this through
qemu_cond_timedwait_bql(). The Windows qemu_cond_timedwait_impl() has
the same two trace calls, next to a qemu_cond_wait_impl() that already
uses the hooks.

Call the hooks, which emit the same two trace points.

Fixes: c89d1c879a ("bql: Fix bql_locked status with condvar APIs")
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Cc: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 util/qemu-thread-posix.c | 4 ++--
 util/qemu-thread-win32.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index ba725444ba6..013436a51e2 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -235,9 +235,9 @@ qemu_cond_timedwait_ts(QemuCond *cond, QemuMutex *mutex, struct timespec *ts,
     int err;
 
     assert(cond->initialized);
-    trace_qemu_mutex_unlock(mutex, file, line);
+    qemu_mutex_pre_unlock(mutex, file, line);
     err = pthread_cond_timedwait(&cond->cond, &mutex->lock, ts);
-    trace_qemu_mutex_locked(mutex, file, line);
+    qemu_mutex_post_lock(mutex, file, line);
     if (err && err != ETIMEDOUT) {
         error_exit(err, __func__);
     }
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index ca2e0b512e2..a1dfb28926b 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -177,11 +177,11 @@ bool qemu_cond_timedwait_impl(QemuCond *cond, QemuMutex *mutex, int ms,
     int rc = 0;
 
     assert(cond->initialized);
-    trace_qemu_mutex_unlock(mutex, file, line);
+    qemu_mutex_pre_unlock(mutex, file, line);
     if (!SleepConditionVariableSRW(&cond->var, &mutex->lock, ms, 0)) {
         rc = GetLastError();
     }
-    trace_qemu_mutex_locked(mutex, file, line);
+    qemu_mutex_post_lock(mutex, file, line);
     if (rc && rc != ERROR_TIMEOUT) {
         error_exit(rc, __func__);
     }
-- 
2.53.0

[PATCH hci-8.0 04/12] oslib: add qemu_debugger_attached() #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

Code that watches for a hang has to tell a hang apart from a debugger
stop, because under a debugger every thread stops and whatever liveness
signal the watcher samples stops changing with them.

Linux reports the tracer in /proc/self/status and Windows has
IsDebuggerPresent(). Answer false elsewhere, which leaves the caller
behaving as it did before the check.

Match the Linux field at the start of a line. The Name: line holds
fifteen characters of comm, enough to spell the field name and appear
ahead of the real one.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Cc: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/osdep.h |  9 +++++++++
 util/oslib-posix.c   | 22 ++++++++++++++++++++++
 util/oslib-win32.c   |  5 +++++
 3 files changed, 36 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index f5fd5f4c4d0..801dec40467 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -635,6 +635,15 @@ bool qemu_write_pidfile(const char *pidfile, Error **errp);
 
 int qemu_get_thread_id(void);
 
+/**
+ * qemu_debugger_attached:
+ *
+ * Returns true if a debugger is currently tracing this process. Meant for
+ * code that would otherwise mistake a debugger stop for a hang, and false
+ * on hosts where this cannot be determined.
+ */
+bool qemu_debugger_attached(void);
+
 /**
  * qemu_kill_thread:
  * @tid: thread id.
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 3c14b726659..e3593f57eee 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -112,6 +112,28 @@ int qemu_get_thread_id(void)
 #endif
 }
 
+bool qemu_debugger_attached(void)
+{
+#ifdef CONFIG_LINUX
+    static const char key[] = "TracerPid:";
+    g_autofree char *status = NULL;
+    const char *p = NULL;
+
+    if (!g_file_get_contents("/proc/self/status", &status, NULL, NULL)) {
+        return false;
+    }
+    while ((p = strstr(p ? p : status, key))) {
+        if (p == status || p[-1] == '\n') {
+            return atoi(p + sizeof(key) - 1) > 0;
+        }
+        p += sizeof(key) - 1;
+    }
+    return false;
+#else
+    return false;
+#endif
+}
+
 int qemu_kill_thread(int tid, int sig)
 {
 #if defined(__linux__)
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index b3cc5725e55..4a6e66d021f 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -228,6 +228,11 @@ int qemu_get_thread_id(void)
     return GetCurrentThreadId();
 }
 
+bool qemu_debugger_attached(void)
+{
+    return IsDebuggerPresent();
+}
+
 char *
 qemu_get_local_state_dir(void)
 {
-- 
2.53.0

[PATCH hci-8.0 05/12] oslib: add qemu_thread_states() #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

A watchdog that reports a hang wants to say what every thread was doing,
and that answer is host specific. Put it beside the other process
introspection helpers rather than in the code that happens to need it
first.

Describe each thread by id, name, scheduler state and the wait channel
it is parked on, all of which Linux exposes in /proc/self/task without
privilege. That is already enough to separate a thread blocked on a
userspace lock, which waits in a futex, from one blocked in the kernel,
which names the primitive it sits on. Skip the caller, which is
reporting rather than hanging.

Hosts with no way to answer return NULL, leaving the caller to print
its own message and nothing more.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Cc: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/osdep.h | 10 +++++++
 util/oslib-posix.c   | 71 ++++++++++++++++++++++++++++++++++++++++++++
 util/oslib-win32.c   |  5 ++++
 3 files changed, 86 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 801dec40467..d3e85c606f0 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -644,6 +644,16 @@ int qemu_get_thread_id(void);
  */
 bool qemu_debugger_attached(void);
 
+/**
+ * qemu_thread_states:
+ *
+ * Returns a newly allocated, newline terminated description of every thread
+ * in this process apart from the caller: its id, name, scheduler state and
+ * the wait channel it is parked on, or NULL on a host that cannot report any
+ * of that.
+ */
+char *qemu_thread_states(void);
+
 /**
  * qemu_kill_thread:
  * @tid: thread id.
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index e3593f57eee..c4856f8cba5 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -134,6 +134,77 @@ bool qemu_debugger_attached(void)
 #endif
 }
 
+#ifdef CONFIG_LINUX
+
+static char *read_task_file(const char *tid, const char *name)
+{
+    g_autofree char *path = g_strdup_printf("/proc/self/task/%s/%s", tid, name);
+    char *buf = NULL;
+
+    g_file_get_contents(path, &buf, NULL, NULL);
+    return buf;
+}
+
+static void append_thread_state(GString *out, const char *tid)
+{
+    g_autofree char *comm = read_task_file(tid, "comm");
+    g_autofree char *stat = NULL;
+    g_autofree char *wchan = NULL;
+    char state = '?';
+    char *rparen;
+
+    if (!comm) {
+        return;
+    }
+    g_strchomp(comm);
+
+    stat = read_task_file(tid, "stat");
+    if (stat) {
+        /* comm can hold anything, ')' included, so scan from the right */
+        rparen = strrchr(stat, ')');
+        if (rparen && rparen[1] == ' ') {
+            state = rparen[2];
+        }
+    }
+    g_string_append_printf(out, "  %-7s %-16s %c\n", tid, comm, state);
+
+    wchan = read_task_file(tid, "wchan");
+    if (wchan && wchan[0]) {
+        g_string_append_printf(out, "      wchan %s\n", wchan);
+    }
+}
+
+char *qemu_thread_states(void)
+{
+    g_autoptr(GDir) dir = g_dir_open("/proc/self/task", 0, NULL);
+    const char *tid;
+    GString *out;
+
+    if (!dir) {
+        return NULL;
+    }
+    out = g_string_new(NULL);
+    while ((tid = g_dir_read_name(dir))) {
+        if (atoi(tid) != qemu_get_thread_id()) {
+            append_thread_state(out, tid);
+        }
+    }
+    if (!out->len) {
+        g_string_free(out, TRUE);
+        return NULL;
+    }
+    return g_string_free(out, FALSE);
+}
+
+#else
+
+char *qemu_thread_states(void)
+{
+    return NULL;
+}
+
+#endif /* CONFIG_LINUX */
+
 int qemu_kill_thread(int tid, int sig)
 {
 #if defined(__linux__)
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 4a6e66d021f..af39bab5620 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -233,6 +233,11 @@ bool qemu_debugger_attached(void)
     return IsDebuggerPresent();
 }
 
+char *qemu_thread_states(void)
+{
+    return NULL;
+}
+
 char *
 qemu_get_local_state_dir(void)
 {
-- 
2.53.0

[PATCH hci-8.0 06/12] oslib: report kernel stacks in qemu_thread_states() #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

A wait channel names the primitive a thread is parked on but not the
path that got it there, and for a thread wedged in a syscall that path
is the answer.

Read /proc/tid/stack when the report is built, and keep the wait
channel for when it cannot be. The file is restricted to CAP_SYS_ADMIN
in the init namespace, so ask once rather than failing per thread, and
say why the report came back thinner. Skip a running thread, which has
no settled stack to unwind, and cap the frames, because the deep end of
a stack is rarely the part that explains the hang.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/osdep.h |  4 ++--
 util/oslib-posix.c   | 52 +++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index d3e85c606f0..e118f12693b 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -649,8 +649,8 @@ bool qemu_debugger_attached(void);
  *
  * Returns a newly allocated, newline terminated description of every thread
  * in this process apart from the caller: its id, name, scheduler state and
- * the wait channel it is parked on, or NULL on a host that cannot report any
- * of that.
+ * kernel stack, or NULL on a host that cannot report any of that. Where the
+ * kernel stack is privileged the description falls back to a wait channel.
  */
 char *qemu_thread_states(void);
 
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index c4856f8cba5..d5aa7d398f1 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -145,13 +145,39 @@ static char *read_task_file(const char *tid, const char *name)
     return buf;
 }
 
+#define THREAD_STACK_FRAMES 12
+
+/*
+ * /proc/tid/stack is restricted to CAP_SYS_ADMIN in the init namespace, so
+ * ask once instead of failing per thread, and say why the report is thinner.
+ */
+static bool thread_stacks_readable(void)
+{
+    static int readable = -1;
+    g_autofree char *path = NULL;
+    g_autofree char *buf = NULL;
+
+    if (readable >= 0) {
+        return readable;
+    }
+    path = g_strdup_printf("/proc/self/task/%d/stack", qemu_get_thread_id());
+    readable = g_file_get_contents(path, &buf, NULL, NULL);
+    if (!readable) {
+        warn_report("kernel stacks need CAP_SYS_ADMIN, falling back to wchan");
+    }
+    return readable;
+}
+
 static void append_thread_state(GString *out, const char *tid)
 {
     g_autofree char *comm = read_task_file(tid, "comm");
     g_autofree char *stat = NULL;
+    g_autofree char *stack = NULL;
     g_autofree char *wchan = NULL;
+    g_auto(GStrv) lines = NULL;
     char state = '?';
     char *rparen;
+    int i;
 
     if (!comm) {
         return;
@@ -168,9 +194,26 @@ static void append_thread_state(GString *out, const char *tid)
     }
     g_string_append_printf(out, "  %-7s %-16s %c\n", tid, comm, state);
 
-    wchan = read_task_file(tid, "wchan");
-    if (wchan && wchan[0]) {
-        g_string_append_printf(out, "      wchan %s\n", wchan);
+    /* A running thread has no settled kernel stack to unwind */
+    if (state != 'R' && thread_stacks_readable()) {
+        stack = read_task_file(tid, "stack");
+    }
+    if (!stack) {
+        wchan = read_task_file(tid, "wchan");
+        if (wchan && wchan[0]) {
+            g_string_append_printf(out, "      wchan %s\n", wchan);
+        }
+        return;
+    }
+
+    lines = g_strsplit(stack, "\n", 0);
+    for (i = 0; i < THREAD_STACK_FRAMES && lines[i]; i++) {
+        if (lines[i][0]) {
+            g_string_append_printf(out, "      %s\n", lines[i]);
+        }
+    }
+    if (lines[i]) {
+        g_string_append(out, "      ...\n");
     }
 }
 
@@ -183,6 +226,9 @@ char *qemu_thread_states(void)
     if (!dir) {
         return NULL;
     }
+    /* Emit the permission notice ahead of the listing, not inside it */
+    thread_stacks_readable();
+
     out = g_string_new(NULL);
     while ((tid = g_dir_read_name(dir))) {
         if (atoi(tid) != qemu_get_thread_id()) {
-- 
2.53.0

[PATCH hci-8.0 07/12] util/qemu-thread: reserve a signal for thread introspection #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

qemu_thread_create() blocks every signal in the threads it starts so
that process directed signals reach the main loop and nowhere else.
That reasoning covers delivery the kernel is free to route: kill(2)
picks any thread not blocking the signal, so leaving SIGTERM or SIGCHLD
unblocked in a worker would let signalfd miss them.

A thread directed signal is not routed. tgkill(2) delivers to the named
thread whatever every other mask says, so a signal only ever sent that
way was never the main loop's to receive, and unblocking it takes
nothing from it.

Reserve SIGRTMIN for asking a thread what it is doing, and leave that
one unblocked. Nothing sends it yet, and until something installs a
handler the change is invisible: a real time signal terminates by
default, which is already what a stray one does through the main
thread, where it was never blocked either.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 include/qemu/osdep.h     | 9 +++++++++
 util/qemu-thread-posix.c | 4 ++++
 2 files changed, 13 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index e118f12693b..31a13926013 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -644,6 +644,15 @@ int qemu_get_thread_id(void);
  */
 bool qemu_debugger_attached(void);
 
+/*
+ * Signal reserved for asking a thread what it is doing. It is only ever
+ * sent thread directed, so it does not compete for the process directed
+ * signals that qemu routes to the main loop.
+ */
+#if defined(SIGRTMIN) && !defined(_WIN32)
+#define QEMU_SIG_INTROSPECT SIGRTMIN
+#endif
+
 /**
  * qemu_thread_states:
  *
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 013436a51e2..3515d47d788 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -423,6 +423,10 @@ void qemu_thread_create(QemuThread *thread, const char *name,
     sigdelset(&set, SIGFPE);
     sigdelset(&set, SIGILL);
     /* TODO avoid SIGBUS loss on macOS */
+#ifdef QEMU_SIG_INTROSPECT
+    /* Thread directed, so it was never the iothread's to receive */
+    sigdelset(&set, QEMU_SIG_INTROSPECT);
+#endif
     pthread_sigmask(SIG_SETMASK, &set, &oldset);
 
     qemu_thread_args = g_new0(QemuThreadArgs, 1);
-- 
2.53.0

[PATCH hci-8.0 08/12] oslib: report each thread's program counter #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

A kernel stack describes a thread asleep in a syscall and says nothing
about one spinning in userspace, which is the other way a thread stops
making progress.

Ask each thread where it is by signalling it and reading the program
counter out of the signal frame. That is a register read and a store,
which a handler may do, unlike unwinding: backtrace() is not async
signal safe and takes the loader lock the first time it runs.

Ask all of them at once, once every passive read is taken. The signal
interrupts whatever a thread was blocked in, so probing earlier would
make the kernel stack and the wait channel describe the interruption
instead of the hang, and probing one at a time would spend the timeout
again for every thread that never answers. Insert the answers from the
back, so the offsets recorded during the reads still point where they
did.

A thread that never answers within the deadline can still run the
handler later, once whatever held it lets go, and by then the slots it
would write to are gone. Count handlers between entry and exit,
withdraw the slots when the deadline expires and wait for the last
handler out before returning to the caller that frees them. A handler
that arrives after the withdrawal finds nothing to write to.

Where the counter sits in the signal frame is a host property, so it is
answered from host/include like the other per-host questions, with a
generic fallback that reports no counter and leaves the host otherwise
untouched. The Linux hosts covered are the ones whose signal frames
linux-user already describes in host-signal.h.

Name the address through dladdr() where there is a symbol for it. There
often is not, because dladdr() sees exported symbols only and a thread
parked inside a library sits in its internals, so fall back to the
object and the offset within it. An offset resolves against the library
afterwards; the raw address does not survive the process.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 host/include/aarch64/host/signal-pc.h     |  20 +++
 host/include/generic/host/signal-pc.h     |  11 ++
 host/include/loongarch64/host/signal-pc.h |  20 +++
 host/include/ppc64/host/signal-pc.h       |  22 +++
 host/include/riscv/host/signal-pc.h       |  20 +++
 host/include/s390x/host/signal-pc.h       |  20 +++
 host/include/x86_64/host/signal-pc.h      |  20 +++
 include/qemu/osdep.h                      |   7 +-
 meson.build                               |   2 +
 util/oslib-posix.c                        | 181 +++++++++++++++++++++-
 10 files changed, 318 insertions(+), 5 deletions(-)
 create mode 100644 host/include/aarch64/host/signal-pc.h
 create mode 100644 host/include/generic/host/signal-pc.h
 create mode 100644 host/include/loongarch64/host/signal-pc.h
 create mode 100644 host/include/ppc64/host/signal-pc.h
 create mode 100644 host/include/riscv/host/signal-pc.h
 create mode 100644 host/include/s390x/host/signal-pc.h
 create mode 100644 host/include/x86_64/host/signal-pc.h

diff --git a/host/include/aarch64/host/signal-pc.h b/host/include/aarch64/host/signal-pc.h
new file mode 100644
index 00000000000..fe60ed1c96c
--- /dev/null
+++ b/host/include/aarch64/host/signal-pc.h
@@ -0,0 +1,20 @@
+/*
+ * Program counter of an interrupted aarch64 thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef AARCH64_HOST_SIGNAL_PC_H
+#define AARCH64_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.pc;
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/host/include/generic/host/signal-pc.h b/host/include/generic/host/signal-pc.h
new file mode 100644
index 00000000000..8ce3822069a
--- /dev/null
+++ b/host/include/generic/host/signal-pc.h
@@ -0,0 +1,11 @@
+/*
+ * No host specific way to read the program counter from a signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HOST_SIGNAL_PC_H
+#define HOST_SIGNAL_PC_H
+
+#define HAVE_HOST_SIGNAL_PC 0
+
+#endif
diff --git a/host/include/loongarch64/host/signal-pc.h b/host/include/loongarch64/host/signal-pc.h
new file mode 100644
index 00000000000..ccb2ac1c56f
--- /dev/null
+++ b/host/include/loongarch64/host/signal-pc.h
@@ -0,0 +1,20 @@
+/*
+ * Program counter of an interrupted loongarch64 thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef LOONGARCH64_HOST_SIGNAL_PC_H
+#define LOONGARCH64_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.__pc;
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/host/include/ppc64/host/signal-pc.h b/host/include/ppc64/host/signal-pc.h
new file mode 100644
index 00000000000..1473570a39a
--- /dev/null
+++ b/host/include/ppc64/host/signal-pc.h
@@ -0,0 +1,22 @@
+/*
+ * Program counter of an interrupted ppc64 thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef PPC64_HOST_SIGNAL_PC_H
+#define PPC64_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#include <asm/ptrace.h>
+
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.gp_regs[PT_NIP];
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/host/include/riscv/host/signal-pc.h b/host/include/riscv/host/signal-pc.h
new file mode 100644
index 00000000000..8db92513465
--- /dev/null
+++ b/host/include/riscv/host/signal-pc.h
@@ -0,0 +1,20 @@
+/*
+ * Program counter of an interrupted riscv thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef RISCV_HOST_SIGNAL_PC_H
+#define RISCV_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.__gregs[REG_PC];
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/host/include/s390x/host/signal-pc.h b/host/include/s390x/host/signal-pc.h
new file mode 100644
index 00000000000..6ad809c9116
--- /dev/null
+++ b/host/include/s390x/host/signal-pc.h
@@ -0,0 +1,20 @@
+/*
+ * Program counter of an interrupted s390x thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef S390X_HOST_SIGNAL_PC_H
+#define S390X_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.psw.addr;
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/host/include/x86_64/host/signal-pc.h b/host/include/x86_64/host/signal-pc.h
new file mode 100644
index 00000000000..c94ad2e976f
--- /dev/null
+++ b/host/include/x86_64/host/signal-pc.h
@@ -0,0 +1,20 @@
+/*
+ * Program counter of an interrupted x86_64 thread, from its signal frame.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef X86_64_HOST_SIGNAL_PC_H
+#define X86_64_HOST_SIGNAL_PC_H
+
+#ifdef CONFIG_LINUX
+#define HAVE_HOST_SIGNAL_PC 1
+
+static inline uintptr_t host_signal_pc(const ucontext_t *uc)
+{
+    return uc->uc_mcontext.gregs[REG_RIP];
+}
+#else
+#define HAVE_HOST_SIGNAL_PC 0
+#endif
+
+#endif
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 31a13926013..8aebb639e4c 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -657,9 +657,10 @@ bool qemu_debugger_attached(void);
  * qemu_thread_states:
  *
  * Returns a newly allocated, newline terminated description of every thread
- * in this process apart from the caller: its id, name, scheduler state and
- * kernel stack, or NULL on a host that cannot report any of that. Where the
- * kernel stack is privileged the description falls back to a wait channel.
+ * in this process apart from the caller: its id, name, scheduler state, the
+ * program counter it stopped at and its kernel stack, or NULL on a host that
+ * cannot report any of that. A privileged kernel stack falls back to a wait
+ * channel, and a host with no way to read a program counter omits it.
  */
 char *qemu_thread_states(void);
 
diff --git a/meson.build b/meson.build
index 21ca6d0e2a9..1e799f7346a 100644
--- a/meson.build
+++ b/meson.build
@@ -2749,6 +2749,8 @@ config_host_data.set('HAVE_GLIB_WITH_SLICE_ALLOCATOR', glib_has_gslice)
 config_host_data.set('HAVE_GLIB_WITH_ALIGNED_ALLOC', glib_has_aligned_alloc)
 config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependencies: util))
 config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul', prefix: osdep_prefix))
+config_host_data.set('CONFIG_DLADDR', cc.has_function('dladdr',
+                     prefix: osdep_prefix + '#include <dlfcn.h>'))
 config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', prefix: '#include <stdlib.h>'))
 if rbd.found()
   config_host_data.set('HAVE_RBD_NAMESPACE_EXISTS',
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index d5aa7d398f1..c960df6a313 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -58,8 +58,13 @@
 #include <lwp.h>
 #endif
 
+#ifdef CONFIG_DLADDR
+#include <dlfcn.h>
+#endif
+
 #include "qemu/memalign.h"
 #include "qemu/mmap-alloc.h"
+#include "host/signal-pc.h"
 
 #define MAX_MEM_PREALLOC_THREAD_COUNT 16
 
@@ -168,8 +173,164 @@ static bool thread_stacks_readable(void)
     return readable;
 }
 
-static void append_thread_state(GString *out, const char *tid)
+/* Where a thread was, and where its line belongs in the report */
+typedef struct {
+    int tid;
+    size_t at;
+    void *pc;
+    int done;
+} ThreadPc;
+
+#if HAVE_HOST_SIGNAL_PC
+
+/* How long to wait for the answers, however many threads were asked */
+#define THREAD_PC_WAIT_MS 100
+
+/*
+ * Published to the handler, which runs in whichever thread was interrupted
+ * and so may only touch this with atomics: every qemu synchronisation
+ * primitive is built on a mutex, and none of them may be taken here.
+ * pc_handlers counts handlers between entry and exit, so the slots can be
+ * withdrawn and the last handler waited out before the caller frees them.
+ */
+static ThreadPc *pc_slots;
+static int pc_nslots;
+static int pc_handlers;
+
+static void thread_pc_handler(int sig, siginfo_t *si, void *ctx)
+{
+    ThreadPc *slots;
+    int me, i, n;
+
+    qatomic_inc(&pc_handlers);
+    smp_mb__after_rmw();
+    slots = qatomic_load_acquire(&pc_slots);
+    if (slots) {
+        n = qatomic_read(&pc_nslots);
+        me = qemu_get_thread_id();
+        for (i = 0; i < n; i++) {
+            if (qatomic_read(&slots[i].tid) == me) {
+                slots[i].pc = (void *)host_signal_pc(ctx);
+                qatomic_store_release(&slots[i].done, 1);
+                break;
+            }
+        }
+    }
+    qatomic_dec(&pc_handlers);
+}
+
+static bool thread_pc_armed(void)
+{
+    static int armed = -1;
+    struct sigaction sa;
+
+    if (armed >= 0) {
+        return armed;
+    }
+    memset(&sa, 0, sizeof(sa));
+    sa.sa_sigaction = thread_pc_handler;
+    sa.sa_flags = SA_SIGINFO | SA_RESTART;
+    sigemptyset(&sa.sa_mask);
+    armed = sigaction(QEMU_SIG_INTROSPECT, &sa, NULL) == 0;
+    return armed;
+}
+
+/*
+ * Ask every thread at once. Asking one at a time would spend the timeout
+ * again for each thread that never answers, and the threads that never
+ * answer are the ones a hung qemu has most of.
+ */
+static void thread_pc_collect(GArray *slots)
 {
+    int64_t deadline;
+    guint i;
+    int left;
+
+    if (!slots->len || !thread_pc_armed()) {
+        return;
+    }
+    qatomic_set(&pc_nslots, slots->len);
+    qatomic_store_release(&pc_slots, &g_array_index(slots, ThreadPc, 0));
+
+    for (i = 0; i < slots->len; i++) {
+        ThreadPc *slot = &g_array_index(slots, ThreadPc, i);
+
+        if (qemu_kill_thread(slot->tid, QEMU_SIG_INTROSPECT) < 0) {
+            qatomic_store_release(&slot->done, -1);
+        }
+    }
+
+    deadline = g_get_monotonic_time() + THREAD_PC_WAIT_MS * 1000;
+    do {
+        left = 0;
+        for (i = 0; i < slots->len; i++) {
+            if (!qatomic_load_acquire(&g_array_index(slots, ThreadPc,
+                                                     i).done)) {
+                left++;
+            }
+        }
+        if (!left) {
+            break;
+        }
+        g_usleep(1000);
+    } while (g_get_monotonic_time() < deadline);
+
+    qatomic_set(&pc_slots, NULL);
+    /* Pairs with the barrier in thread_pc_handler() */
+    smp_mb();
+    while (qatomic_read(&pc_handlers)) {
+        g_usleep(1000);
+    }
+}
+
+static void insert_thread_pc(GString *out, const ThreadPc *slot)
+{
+    g_autofree char *line = NULL;
+    void *pc = slot->pc;
+#ifdef CONFIG_DLADDR
+    Dl_info info = { };
+#endif
+
+    if (qatomic_read(&slot->done) != 1 || !pc) {
+        return;
+    }
+#ifdef CONFIG_DLADDR
+    if (dladdr(pc, &info) && info.dli_sname) {
+        line = g_strdup_printf("      pc %p %s+0x%tx\n", pc, info.dli_sname,
+                               (char *)pc - (char *)info.dli_saddr);
+    } else if (info.dli_fname && info.dli_fbase) {
+        /*
+         * dladdr() only knows exported symbols, so an address inside a
+         * library's own internals has none. The object and the offset
+         * within it still resolve later, the raw address does not.
+         */
+        g_autofree char *obj = g_path_get_basename(info.dli_fname);
+
+        line = g_strdup_printf("      pc %p %s+0x%tx\n", pc, obj,
+                               (char *)pc - (char *)info.dli_fbase);
+    }
+#endif
+    if (!line) {
+        line = g_strdup_printf("      pc %p\n", pc);
+    }
+    g_string_insert(out, slot->at, line);
+}
+
+#else
+
+static void thread_pc_collect(GArray *slots)
+{
+}
+
+static void insert_thread_pc(GString *out, const ThreadPc *slot)
+{
+}
+
+#endif /* HAVE_HOST_SIGNAL_PC */
+
+static void append_thread_state(GString *out, GArray *slots, const char *tid)
+{
+    ThreadPc slot = { };
     g_autofree char *comm = read_task_file(tid, "comm");
     g_autofree char *stat = NULL;
     g_autofree char *stack = NULL;
@@ -194,6 +355,10 @@ static void append_thread_state(GString *out, const char *tid)
     }
     g_string_append_printf(out, "  %-7s %-16s %c\n", tid, comm, state);
 
+    slot.tid = atoi(tid);
+    slot.at = out->len;
+    g_array_append_val(slots, slot);
+
     /* A running thread has no settled kernel stack to unwind */
     if (state != 'R' && thread_stacks_readable()) {
         stack = read_task_file(tid, "stack");
@@ -220,8 +385,10 @@ static void append_thread_state(GString *out, const char *tid)
 char *qemu_thread_states(void)
 {
     g_autoptr(GDir) dir = g_dir_open("/proc/self/task", 0, NULL);
+    g_autoptr(GArray) slots = g_array_new(FALSE, TRUE, sizeof(ThreadPc));
     const char *tid;
     GString *out;
+    guint i;
 
     if (!dir) {
         return NULL;
@@ -232,13 +399,23 @@ char *qemu_thread_states(void)
     out = g_string_new(NULL);
     while ((tid = g_dir_read_name(dir))) {
         if (atoi(tid) != qemu_get_thread_id()) {
-            append_thread_state(out, tid);
+            append_thread_state(out, slots, tid);
         }
     }
     if (!out->len) {
         g_string_free(out, TRUE);
         return NULL;
     }
+
+    /*
+     * Only now, with every passive read taken, disturb the threads. Insert
+     * from the back so the offsets recorded above stay where they were.
+     */
+    thread_pc_collect(slots);
+    for (i = slots->len; i > 0; i--) {
+        insert_thread_pc(out, &g_array_index(slots, ThreadPc, i - 1));
+    }
+
     return g_string_free(out, FALSE);
 }
 
-- 
2.53.0

[PATCH hci-8.0 09/12] system/cpus: report a BQL held past a deadline #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

A vCPU thread reacquires the BQL on every exit that needs userspace, so
a lock held for tens of seconds stops those vCPUs dead and the guest
reports its own soft lockup. Nothing in qemu records that this
happened, which leaves a guest panic as the only evidence and no way to
tell a stuck holder from a guest that wedged on its own.

Add a BQL watchdog. A thread of its own samples a counter advanced on
every acquire and every release, so it is odd while the lock is held.
One odd value seen throughout the window means a single owner held the
lock for the whole window, because any handover would have advanced the
counter twice.

Advance it in bql_update_status(), which the mutex hooks already call
for exactly those two transitions, so a cond-wait that drops and
retakes the lock is covered without the wrappers knowing, and so is any
caller that reaches the BQL through the plain mutex API. Record the
holder's tid beside it, under the same lock, because the thread listing
alone cannot tell a thread parked on its own condvar from one blocked
on the BQL. Cache the tid per thread, since gettid is a syscall and
this is the lock path.

On expiry, name the holder and list every thread's state. The listing
separates a thread blocked on the BQL, which waits in a futex, from one
idle inside KVM, which waits in kvm_vcpu_block, and names the syscall a
wedged holder is stuck in. Report once per episode, or a lock held for
a minute buries the log in copies of one answer. Put the time on the
line unless -msg timestamp=on already does: a hold is only worth
reporting if it can be lined up against what the guest saw, and that
option has to be asked for in advance.

The deadline has to clear the longest legitimate hold, and migration
setup can hold the BQL for a while, so the watchdog stays off unless
asked for. The bql-watchdog-ms machine property is the whole interface:
it sets the deadline, on the command line or over qom-set at any time,
and 0 turns the watchdog off. Reject a deadline above a minute: a hold
that long has already shown up as a soft lockup inside the guest.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 hw/core/machine.c     |  27 ++++++++++
 include/system/cpus.h |   4 ++
 qemu-options.hx       |   6 +++
 system/cpus.c         | 120 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 157 insertions(+)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index bd61aa69ccc..d5bb047744d 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -923,6 +923,27 @@ static void machine_set_dump_guest_core(Object *obj, bool value, Error **errp)
     ms->dump_guest_core = value;
 }
 
+static void machine_get_bql_watchdog_ms(Object *obj, Visitor *v,
+                                        const char *name, void *opaque,
+                                        Error **errp)
+{
+    uint64_t value = bql_watchdog_get_timeout_ms();
+
+    visit_type_uint64(v, name, &value, errp);
+}
+
+static void machine_set_bql_watchdog_ms(Object *obj, Visitor *v,
+                                        const char *name, void *opaque,
+                                        Error **errp)
+{
+    uint64_t value;
+
+    if (!visit_type_uint64(v, name, &value, errp)) {
+        return;
+    }
+    bql_watchdog_set_timeout_ms(value, errp);
+}
+
 static bool machine_get_mem_merge(Object *obj, Error **errp)
 {
     MachineState *ms = MACHINE(obj);
@@ -1671,6 +1692,12 @@ static void machine_class_init(ObjectClass *oc, const void *data)
     object_class_property_set_description(oc, "dump-guest-core",
         "Include guest memory in a core dump");
 
+    object_class_property_add(oc, "bql-watchdog-ms", "uint64",
+        machine_get_bql_watchdog_ms, machine_set_bql_watchdog_ms, NULL, NULL);
+    object_class_property_set_description(oc, "bql-watchdog-ms",
+        "Watchdog deadline for a Big QEMU Lock hold, in ms. "
+        "Default: 0 (disabled)");
+
     object_class_property_add_bool(oc, "mem-merge",
         machine_get_mem_merge, machine_set_mem_merge);
     object_class_property_set_description(oc, "mem-merge",
diff --git a/include/system/cpus.h b/include/system/cpus.h
index 4aebec48705..45adcacdc38 100644
--- a/include/system/cpus.h
+++ b/include/system/cpus.h
@@ -41,4 +41,8 @@ void cpu_synchronize_all_post_reset(void);
 void cpu_synchronize_all_post_init(void);
 void cpu_synchronize_all_pre_loadvm(void);
 
+/* Watchdog deadline for a BQL hold. 0 disables the watchdog. */
+bool bql_watchdog_set_timeout_ms(uint64_t ms, Error **errp);
+uint64_t bql_watchdog_get_timeout_ms(void);
+
 #endif
diff --git a/qemu-options.hx b/qemu-options.hx
index aae51823d70..2809fe42fb9 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -32,6 +32,7 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
     "                vmport=on|off|auto controls emulation of vmport (default: auto)\n"
     "                dump-guest-core=on|off include guest memory in a core dump (default=on)\n"
     "                mem-merge=on|off controls memory merge support (default: on)\n"
+    "                bql-watchdog-ms=ms report a Big QEMU Lock held longer than ms (default: 0, off)\n"
     "                aes-key-wrap=on|off controls support for AES key wrapping (default=on)\n"
     "                dea-key-wrap=on|off controls support for DEA key wrapping (default=on)\n"
     "                suppress-vmdesc=on|off disables self-describing migration (default=off)\n"
@@ -84,6 +85,11 @@ SRST
         supported by the host, de-duplicates identical memory pages
         among VMs instances (enabled by default).
 
+    ``bql-watchdog-ms=ms``
+        Report when the Big QEMU Lock has been held for more than ``ms``
+        milliseconds, up to a minute. The default is 0, which disables the
+        watchdog.
+
     ``aes-key-wrap=on|off``
         Enables or disables AES key wrapping support on s390-ccw hosts.
         This feature controls whether AES wrapping keys will be created
diff --git a/system/cpus.c b/system/cpus.c
index 5c2bdc3f3d0..fe7f47ab1c3 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -36,6 +36,7 @@
 #include "exec/cpu-common.h"
 #include "qemu/thread.h"
 #include "qemu/main-loop.h"
+#include "qemu/error-report.h"
 #include "qemu/plugin.h"
 #include "system/cpus.h"
 #include "qemu/guest-random.h"
@@ -419,12 +420,129 @@ static QemuCond qemu_cpu_cond;
 /* system init */
 static QemuCond qemu_pause_cond;
 
+/*
+ * Advanced by bql_update_status() below, so odd while the BQL is held and
+ * even while it is free. A sampler that sees one odd value throughout its
+ * window knows a single owner held the lock for the whole window: any
+ * handover would have advanced it twice.
+ */
+static unsigned int bql_seq;
+
+/* Who advanced bql_seq to its current odd value, 0 while the BQL is free */
+static int bql_owner_tid;
+
+static int bql_this_tid(void)
+{
+    static __thread int tid;
+
+    if (tid) {
+        return tid;
+    }
+    tid = qemu_get_thread_id();
+    return tid;
+}
+
+/* Poll this many times per timeout, to bound how late the watchdog reacts */
+#define BQL_WATCHDOG_POLLS 4
+
+#define BQL_WATCHDOG_MAX_MS (60 * 1000)
+
+static QemuThread bql_watchdog_thread;
+static QemuSemaphore bql_watchdog_sem;
+static uint64_t bql_watchdog_timeout_ms;
+
+static void bql_report_thread_states(void)
+{
+    g_autofree char *states = qemu_thread_states();
+
+    if (!states) {
+        return;
+    }
+    qemu_flockfile(stderr);
+    fputs(states, stderr);
+    qemu_funlockfile(stderr);
+}
+
+static void *bql_watchdog_fn(void *opaque)
+{
+    unsigned int last = qatomic_read(&bql_seq);
+    unsigned int reported = 0;
+    int stuck_polls = 0;
+
+    for (;;) {
+        uint64_t timeout_ms = qatomic_read(&bql_watchdog_timeout_ms);
+        g_autofree char *when = NULL;
+        unsigned int seq;
+
+        if (!timeout_ms) {
+            /* Park rather than exit, so re-arming needs no new thread */
+            qemu_sem_wait(&bql_watchdog_sem);
+            last = qatomic_read(&bql_seq);
+            stuck_polls = 0;
+            continue;
+        }
+        qemu_sem_timedwait(&bql_watchdog_sem,
+                           MAX(timeout_ms / BQL_WATCHDOG_POLLS, 1));
+
+        seq = qatomic_read(&bql_seq);
+        if (seq != last || !(seq & 1)) {
+            last = seq;
+            stuck_polls = 0;
+            continue;
+        }
+        if (++stuck_polls < BQL_WATCHDOG_POLLS) {
+            continue;
+        }
+        if (seq == reported) {
+            continue;
+        }
+        reported = seq;
+        if (!message_with_timestamp) {
+            when = real_time_iso8601();
+        }
+        error_report("%s%sBQL held for more than %" PRIu64 " ms by thread %d",
+                     when ?: "", when ? " " : "", timeout_ms,
+                     qatomic_read(&bql_owner_tid));
+        bql_report_thread_states();
+    }
+}
+
+uint64_t bql_watchdog_get_timeout_ms(void)
+{
+    return qatomic_read(&bql_watchdog_timeout_ms);
+}
+
+bool bql_watchdog_set_timeout_ms(uint64_t ms, Error **errp)
+{
+    static bool started;
+
+    assert(bql_locked());
+    if (ms > BQL_WATCHDOG_MAX_MS) {
+        error_setg(errp, "BQL watchdog deadline must not exceed %d ms",
+                   BQL_WATCHDOG_MAX_MS);
+        return false;
+    }
+    qatomic_set(&bql_watchdog_timeout_ms, ms);
+    if (!ms) {
+        return true;
+    }
+    if (!started) {
+        started = true;
+        qemu_thread_create(&bql_watchdog_thread, "bql-watchdog",
+                           bql_watchdog_fn, NULL, QEMU_THREAD_DETACHED);
+    } else {
+        qemu_sem_post(&bql_watchdog_sem);
+    }
+    return true;
+}
+
 void qemu_init_cpu_loop(void)
 {
     qemu_init_sigbus();
     qemu_cond_init(&qemu_cpu_cond);
     qemu_cond_init(&qemu_pause_cond);
     qemu_mutex_init(&bql);
+    qemu_sem_init(&bql_watchdog_sem, 0);
 
     qemu_thread_get_self(&io_thread);
 }
@@ -528,6 +646,8 @@ void bql_update_status(bool locked)
     /* This function should only be used when an update happened.. */
     assert(bql_locked() != locked);
     set_bql_locked(locked);
+    qatomic_set(&bql_owner_tid, locked ? bql_this_tid() : 0);
+    qatomic_set(&bql_seq, qatomic_read(&bql_seq) + 1);
 }
 
 static uint32_t bql_unlock_blocked;
-- 
2.53.0

[PATCH hci-8.0 10/12] system/cpus: hold off the BQL watchdog under a debugger #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

A debugger stops every thread, so the sequence counter freezes at an
odd value and the watchdog cannot tell that apart from an owner that
never lets go. Attaching gdb to a running guest would report a hang
that is not one.

Skip the report while a tracer is attached, and restart the count so
that a detach gets a full deadline before the watchdog can fire. Any
ptracer sets TracerPid, so strace holds the watchdog off as well.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 system/cpus.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/system/cpus.c b/system/cpus.c
index fe7f47ab1c3..f838cb6af7c 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -493,6 +493,10 @@ static void *bql_watchdog_fn(void *opaque)
         if (++stuck_polls < BQL_WATCHDOG_POLLS) {
             continue;
         }
+        if (qemu_debugger_attached()) {
+            stuck_polls = 0;
+            continue;
+        }
         if (seq == reported) {
             continue;
         }
-- 
2.53.0

[PATCH hci-8.0 11/12] system/cpus: add bql-watchdog-abort #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

The report says where each thread was when the watchdog looked, not
how it got there. A core carries every thread's whole stack, at the
price of the guest. A qemu whose BQL has been held this long is not
running its vCPUs anyway, so on a host where the answer is worth more
than the guest, abort after the report.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 hw/core/machine.c     | 17 +++++++++++++++++
 include/system/cpus.h |  4 ++++
 qemu-options.hx       |  5 +++++
 system/cpus.c         | 19 +++++++++++++++++++
 4 files changed, 45 insertions(+)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index d5bb047744d..be98c0f9ca9 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -944,6 +944,17 @@ static void machine_set_bql_watchdog_ms(Object *obj, Visitor *v,
     bql_watchdog_set_timeout_ms(value, errp);
 }
 
+static bool machine_get_bql_watchdog_abort(Object *obj, Error **errp)
+{
+    return bql_watchdog_get_abort();
+}
+
+static void machine_set_bql_watchdog_abort(Object *obj, bool value,
+                                           Error **errp)
+{
+    bql_watchdog_set_abort(value);
+}
+
 static bool machine_get_mem_merge(Object *obj, Error **errp)
 {
     MachineState *ms = MACHINE(obj);
@@ -1698,6 +1709,12 @@ static void machine_class_init(ObjectClass *oc, const void *data)
         "Watchdog deadline for a Big QEMU Lock hold, in ms. "
         "Default: 0 (disabled)");
 
+    object_class_property_add_bool(oc, "bql-watchdog-abort",
+        machine_get_bql_watchdog_abort, machine_set_bql_watchdog_abort);
+    object_class_property_set_description(oc, "bql-watchdog-abort",
+        "Abort after the bql-watchdog-ms report, to leave a core dump. "
+        "Default: off");
+
     object_class_property_add_bool(oc, "mem-merge",
         machine_get_mem_merge, machine_set_mem_merge);
     object_class_property_set_description(oc, "mem-merge",
diff --git a/include/system/cpus.h b/include/system/cpus.h
index 45adcacdc38..6dbb89f3de9 100644
--- a/include/system/cpus.h
+++ b/include/system/cpus.h
@@ -45,4 +45,8 @@ void cpu_synchronize_all_pre_loadvm(void);
 bool bql_watchdog_set_timeout_ms(uint64_t ms, Error **errp);
 uint64_t bql_watchdog_get_timeout_ms(void);
 
+/* Abort after reporting, to leave a core behind. */
+void bql_watchdog_set_abort(bool enable);
+bool bql_watchdog_get_abort(void);
+
 #endif
diff --git a/qemu-options.hx b/qemu-options.hx
index 2809fe42fb9..f3711b92707 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -33,6 +33,7 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
     "                dump-guest-core=on|off include guest memory in a core dump (default=on)\n"
     "                mem-merge=on|off controls memory merge support (default: on)\n"
     "                bql-watchdog-ms=ms report a Big QEMU Lock held longer than ms (default: 0, off)\n"
+    "                bql-watchdog-abort=on|off abort after the bql-watchdog-ms report (default: off)\n"
     "                aes-key-wrap=on|off controls support for AES key wrapping (default=on)\n"
     "                dea-key-wrap=on|off controls support for DEA key wrapping (default=on)\n"
     "                suppress-vmdesc=on|off disables self-describing migration (default=off)\n"
@@ -90,6 +91,10 @@ SRST
         milliseconds, up to a minute. The default is 0, which disables the
         watchdog.
 
+    ``bql-watchdog-abort=on|off``
+        Abort after the ``bql-watchdog-ms`` report, to leave a core dump.
+        The default is off.
+
     ``aes-key-wrap=on|off``
         Enables or disables AES key wrapping support on s390-ccw hosts.
         This feature controls whether AES wrapping keys will be created
diff --git a/system/cpus.c b/system/cpus.c
index f838cb6af7c..1cf393879fc 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -450,6 +450,7 @@ static int bql_this_tid(void)
 static QemuThread bql_watchdog_thread;
 static QemuSemaphore bql_watchdog_sem;
 static uint64_t bql_watchdog_timeout_ms;
+static bool bql_watchdog_abort;
 
 static void bql_report_thread_states(void)
 {
@@ -463,6 +464,16 @@ static void bql_report_thread_states(void)
     qemu_funlockfile(stderr);
 }
 
+bool bql_watchdog_get_abort(void)
+{
+    return qatomic_read(&bql_watchdog_abort);
+}
+
+void bql_watchdog_set_abort(bool enable)
+{
+    qatomic_set(&bql_watchdog_abort, enable);
+}
+
 static void *bql_watchdog_fn(void *opaque)
 {
     unsigned int last = qatomic_read(&bql_seq);
@@ -508,6 +519,14 @@ static void *bql_watchdog_fn(void *opaque)
                      when ?: "", when ? " " : "", timeout_ms,
                      qatomic_read(&bql_owner_tid));
         bql_report_thread_states();
+        if (!qatomic_read(&bql_watchdog_abort)) {
+            continue;
+        }
+        /*
+         * A core carries the whole of every stack, and a qemu whose BQL has
+         * been held this long has stopped running its vCPUs anyway.
+         */
+        abort();
     }
 }
 
-- 
2.53.0

[PATCH hci-8.0 12/12] tests/functional: cover the BQL watchdog #VSTOR-144507

[Denis V. Lunev]

From: Denis V. Lunev <den@openvz.org>

Hold the BQL from the main thread and check that the watchdog says so.

human-monitor-command is not a coroutine command, so a write issued
through it runs to completion under AIO_WAIT_WHILE() in the main
thread, which keeps the BQL for as long as the backend takes to answer.
Point that at a null-co node with a latency ten times the deadline and
the hold is both long enough to report and short enough to end on its
own.

Arm the watchdog after launch rather than on the command line, because
machine creation takes the BQL in bursts of its own and a deadline this
short is meant for the hold, not for startup.

Check the quiet case too. A counter that stops distinguishing a held
lock from a free one would report a hang on a healthy qemu, and that
failure is invisible to a test that only looks for the report.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Cc: Zhao Liu <zhao1.liu@intel.com>
Cc: Thomas Huth <th.huth+qemu@posteo.eu>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/functional/meson.build          |  1 +
 tests/functional/test_bql_watchdog.py | 60 +++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100755 tests/functional/test_bql_watchdog.py

diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 305b25703d9..698a045ed84 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -68,6 +68,7 @@ test_timeouts = {
 }
 
 tests_generic_system = [
+  'bql_watchdog',
   'empty_cpu_model',
   'info_usernet',
   'version',
diff --git a/tests/functional/test_bql_watchdog.py b/tests/functional/test_bql_watchdog.py
new file mode 100755
index 00000000000..193dad15e37
--- /dev/null
+++ b/tests/functional/test_bql_watchdog.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Test the BQL watchdog against a main thread blocked on IO.
+#
+# Copyright (C) 2026 Virtuozzo International GmbH
+
+from qemu_test import QemuSystemTest
+
+deadline_ms = 100
+hold_ns = 1000 * 1000 * 1000
+
+
+class BqlWatchdog(QemuSystemTest):
+
+    def setUp(self):
+        super().setUp()
+        self.set_machine('none')
+        self.vm.add_args('-nodefaults')
+        self.vm.add_args('-blockdev',
+                         f'driver=null-co,node-name=null0,size=1048576,'
+                         f'latency-ns={hold_ns}')
+        self.vm.launch()
+
+    def arm(self, ms):
+        self.vm.cmd('qom-set', path='/machine',
+                    property='bql-watchdog-ms', value=ms)
+
+    def hold_the_bql(self):
+        """
+        human-monitor-command is not a coroutine command, so the write runs
+        to completion under AIO_WAIT_WHILE() in the main thread, which keeps
+        the BQL for as long as null-co takes to answer.
+        """
+        self.vm.cmd('human-monitor-command',
+                    command_line='qemu-io null0 "write 0 4k"')
+
+    def log_after_shutdown(self):
+        """The log only reaches the harness once the VM is reaped"""
+        self.vm.shutdown()
+        return self.vm.get_log() or ''
+
+    def test_reports_a_hold(self):
+        self.arm(deadline_ms)
+        self.hold_the_bql()
+        self.arm(0)
+
+        log = self.log_after_shutdown()
+        self.assertIn(f'BQL held for more than {deadline_ms} ms', log)
+
+    def test_silent_while_disarmed(self):
+        self.hold_the_bql()
+
+        log = self.log_after_shutdown()
+        self.assertNotIn('BQL held for more than', log)
+
+
+if __name__ == '__main__':
+    QemuSystemTest.main()
-- 
2.53.0