Re: [Devel] [PATCH hci-8.1 0/4] configs: a User Mode Linux kernel for the KUnit tests

[Konstantin Khorenko <khorenko@virtuozzo.com>]

Eva Kurchatova wrote:
> The config is faithful. I built kernel-um-rhel.config, and compared it
> with what kunit-uml-config.sh makes from the shipped x86_64 config: the
> same 77 KUnit tests built in, the same vz options, the same sanity
> checks. The only differences are the debug info (4/4) and the KASAN
> self-test (2/4, 3/4). Built with ARCH=um and booted with
> kasan_multi_shot: 182 pass, 0 fail, no kernel report.

Thank you for checking it and for running it.

> 1. Please drop 2/4 and keep the KASAN self-test. hci-kunit does not fail
> on its reports: since "hci-kunit: leave the reports of a suite that
> provokes them out of the check" (PR #9241) reports inside the kasan
> suite are expected, and the run is 183 pass, 0 fail with it.

There is no such commit in PR #9241. The branch still ends with
"hci-kunit: run the UML kernel for KUnit tests" of October 5, and
REPORT_MARKERS has no exception for any suite. Probably the filter has
not been pushed yet.

> The test is also the only proof that KASAN catches anything in this
> kernel. Without it, a KASAN that silently stopped working would look
> like a clean run. And 2/4 would be a vz-only change to
> lib/Kconfig.kasan that mainline does not need.

Even with the filter in place, I would rather not run the KASAN
self-test in the UML kernel:

- The filter is blind for as long as the kasan suite runs. A real
  report printed in that window, by deferred work of an earlier suite in
  a workqueue, an RCU callback or a timer, is dropped together with the
  expected ones.
- It depends on the KTAP framing and on the suite name, so the runner
  has to follow kasan_test from now on. A change there turns the run
  red rather than green, but it is still upkeep for no new coverage.
- The way KASAN can really disappear here is the config. kernel.spec
  runs olddefconfig with the real compiler, and a toolchain without
  KASAN support drops CONFIG_KASAN silently; process_configs.sh does not
  see it, as it runs with dummy-tools. The self-test does not catch that
  either: KASAN_KUNIT_TEST depends on KASAN, so the kasan suite just
  vanishes from a green run. A check of the final .config does catch
  it, see 2 below.
- At run time, the KASAN code is the upstream one. Neither mm/kasan nor
  the UML KASAN support carry vz changes, and upstream runs this very
  self-test on every release.

2/4 is a vz-only change indeed, a one-line one. I prefer it to a blind
spot in the runner.

> 2. The static list of builtin tests will drift. 69 of the override files
> only turn a generic =m test into =y. A KUnit test that a RHEL import
> adds lands in common/generic as =m and stays a module here, which the
> UML kernel never loads, so it silently does not run. process_configs.sh
> does not catch it, =m is a valid value. The script built every KUnit
> test in by itself.
>
> A check in kernel.spec after olddefconfig would cover it: fail when a
> symbol defaulting to KUNIT_ALL_TESTS is =m, except the tests whose
> subsystem is a module (cfg80211, mac80211, iwlwifi, xe, vkms, hid,
> rpcsec_gss_krb5, prp).

Most of them do not drift. Nearly every KUnit test hides its prompt
behind "if !KUNIT_ALL_TESTS", so with KUNIT_ALL_TESTS=y Kconfig ignores
the =m of the generic layer and builds the test in by itself. Their =y
files are there only to keep process_configs.sh -c quiet, and a new test
of that kind is built in without anyone touching the um layer.

A test whose prompt stays visible does stay =m, as you say:
REGMAP_KUNIT, USERCOPY_KUNIT_TEST and the SND_*_TEST ones today. So yes,
let's have the check. The plan:

- A script next to the um layer in redhat/configs, which kernel.spec
  calls on the final .config, after olddefconfig and before the build.
- It fails the build when a symbol defaulting to KUNIT_ALL_TESTS is =m.
- It also fails when an option the variant exists for did not survive
  olddefconfig: KUNIT, KASAN, UBSAN and the other checks. That is the
  config check from 1.

No exception list. The tests you name cannot be turned off under
KUNIT_ALL_TESTS either, most of them hide their prompt the same way.
What keeps them =m is a subsystem we ship as a module only, so the um
layer turns that one off instead:

- CFG80211, mac80211 and iwlwifi go with it
- DRM_XE
- DRM_VKMS
- HID_UCLOGIC
- RPCSEC_GSS_KRB5
- HSR

A module the UML kernel never loads is of no use there. Their tests stay
with the modules mode of hci-kunit on the real kernel, except the xe one,
which our kernel does not build at all.

A new test of that kind then fails the build, and whoever fixes it
decides whether its subsystem goes off in UML or gets built in.

> Optional: CONFIG_SLUB_RCU_DEBUG=y. It is a KASAN feature that catches
> use after free of RCU-freed slab objects, off here because the generic
> layer sets it off for kernels without KASAN. My config lacked it too,
> though.

Agreed, cheap and useful. I'll add it to 3/4.

--
Best regards,

Konstantin Khorenko,
Virtuozzo Linux Kernel Team

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel