[Devel] [PATCH vz10 1/3] selftests: drv-net: let NetDrvEnv take nsim_test

[Devel] [PATCH vz10 1/3] selftests: drv-net: let NetDrvEnv take nsim_test

[Eva Kurchatova]

rss_api.py opens its environment with

  with NetDrvEnv(__file__, nsim_test=False) as cfg:

NetDrvEnv hands every keyword it does not name to NetdevSimDev(), which
takes port_count, queue_count and ns only, so the test ends in

  TypeError: NetdevSimDev.__init__() got an unexpected
  keyword argument 'nsim_test'

The test came in with the RHEL10 import, commit 9f055df11343 ("rh10:
import RHEL10 kernel-6.12.0-211.16.1.el10"); the parameter it asks for
did not. Take the env.py hunk of commit 185646a8a0a8 ("selftests:
drv-net: add tests for napi IRQ affinity notifiers"), which added that
parameter upstream. The test that commit adds is left out, it needs a
real device.

https://virtuozzo.atlassian.net/browse/VSTOR-139651
Feature: fix vz selftests
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
 tools/testing/selftests/drivers/net/lib/py/env.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/drivers/net/lib/py/env.py b/tools/testing/selftests/drivers/net/lib/py/env.py
index 987e452d3a45..4677c3df4ec6 100644
--- a/tools/testing/selftests/drivers/net/lib/py/env.py
+++ b/tools/testing/selftests/drivers/net/lib/py/env.py
@@ -38,14 +38,20 @@ class NetDrvEnv:
     """
     Class for a single NIC / host env, with no remote end
     """
-    def __init__(self, src_path, **kwargs):
+    def __init__(self, src_path, nsim_test=None, **kwargs):
         self._ns = None
 
         self.env = _load_env_file(src_path)
 
         if 'NETIF' in self.env:
+            if nsim_test is True:
+                raise KsftXfailEx("Test only works on netdevsim")
+
             self.dev = ip("link show dev " + self.env['NETIF'], json=True)[0]
         else:
+            if nsim_test is False:
+                raise KsftXfailEx("Test does not work on netdevsim")
+
             self._ns = NetdevSimDev(**kwargs)
             self.dev = self._ns.nsims[0].dev
         self.ifname = self.dev['ifname']
-- 
2.55.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH vz10 2/3] selftests: drv-net: run ping in its own netns

[Eva Kurchatova]

In netdevsim mode NetDrvEpEnv gives the remote end a namespace of its
own and leaves the local end in the one the test was started in, so
the filtering of the host applies to the traffic the test sends to
itself. A host that drops it fails the test:

  # nft add rule inet t input ip saddr 192.0.2.0/24 tcp flags syn drop
  # ./ping.py
  ok 1 ping.test_v4
  ok 2 ping.test_v6
  not ok 3 ping.test_tcp
  # Stopping tests due to KsftTerminate.

ICMP is let through where TCP is not, which is why the first two pass
and the test then sits there until it is killed.

Re-execute the test in a namespace of its own, the way nft_audit.sh and
nft_concat_range.sh already do, and mount sysfs again there so that the
netdevsim devices the test creates are the ones it sees.

https://virtuozzo.atlassian.net/browse/VSTOR-139651
Feature: fix selftests
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
 .../selftests/drivers/net/lib/py/__init__.py  |  2 +-
 tools/testing/selftests/drivers/net/ping.py   |  5 ++++
 tools/testing/selftests/net/lib/py/utils.py   | 24 +++++++++++++++++++
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/drivers/net/lib/py/__init__.py b/tools/testing/selftests/drivers/net/lib/py/__init__.py
index fce5d9218f1d..2d3fdaa81550 100644
--- a/tools/testing/selftests/drivers/net/lib/py/__init__.py
+++ b/tools/testing/selftests/drivers/net/lib/py/__init__.py
@@ -15,7 +15,7 @@ try:
         NlError, RtnlFamily, DevlinkFamily
     from net.lib.py import CmdExitFailure
     from net.lib.py import bkg, cmd, defer, ethtool, fd_read_timeout, ip, \
-        rand_port, tool, wait_port_listen
+        rand_port, rexec_in_netns, tool, wait_port_listen
     from net.lib.py import fd_read_timeout
     from net.lib.py import KsftSkipEx, KsftFailEx, KsftXfailEx
     from net.lib.py import ksft_disruptive, ksft_exit, ksft_pr, ksft_run, \
diff --git a/tools/testing/selftests/drivers/net/ping.py b/tools/testing/selftests/drivers/net/ping.py
index dfcf6ddd4c01..4799c94369d4 100755
--- a/tools/testing/selftests/drivers/net/ping.py
+++ b/tools/testing/selftests/drivers/net/ping.py
@@ -5,6 +5,7 @@ from lib.py import ksft_run, ksft_exit
 from lib.py import ksft_eq
 from lib.py import NetDrvEpEnv
 from lib.py import bkg, cmd, wait_port_listen, rand_port
+from lib.py import rexec_in_netns
 
 
 def test_v4(cfg) -> None:
@@ -42,6 +43,10 @@ def test_tcp(cfg) -> None:
 
 
 def main() -> None:
+    # In netdevsim mode the local end of the test stays in the namespace
+    # the test was started in, so the rules of the host apply to it.
+    rexec_in_netns(__file__)
+
     with NetDrvEpEnv(__file__) as cfg:
         ksft_run(globs=globals(), case_pfx={"test_"}, args=(cfg, ))
     ksft_exit()
diff --git a/tools/testing/selftests/net/lib/py/utils.py b/tools/testing/selftests/net/lib/py/utils.py
index afce547bcab0..13d7a7e6562e 100644
--- a/tools/testing/selftests/net/lib/py/utils.py
+++ b/tools/testing/selftests/net/lib/py/utils.py
@@ -8,6 +8,7 @@ import re
 import select
 import socket
 import subprocess
+import sys
 import time
 
 
@@ -131,6 +132,29 @@ def ethtool(args, json=None, ns=None, host=None):
     return tool('ethtool', args, json=json, ns=ns, host=host)
 
 
+def rexec_in_netns(path):
+    """
+    Re-exec the test in a network namespace of its own.
+
+    A test that puts one end of its traffic in the namespace it was
+    started in otherwise depends on what the host filters.  sysfs is
+    mounted again in the new namespace, without it the netdevsim
+    devices the test creates are not the ones it sees.  The test is
+    left where it is if unshare is not there to move it.
+    """
+    if os.environ.get("KSFT_IN_NETNS"):
+        return
+
+    os.environ["KSFT_IN_NETNS"] = "1"
+    try:
+        os.execvp("unshare",
+                  ["unshare", "-n", "-m",
+                   "sh", "-c", 'mount -t sysfs sysfs /sys && exec "$@"',
+                   "sh", sys.executable, os.path.abspath(path)] + sys.argv[1:])
+    except OSError:
+        del os.environ["KSFT_IN_NETNS"]
+
+
 def rand_port(type=socket.SOCK_STREAM):
     """
     Get a random unprivileged port.
-- 
2.55.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

[Devel] [PATCH vz10 3/3] selftests: drv-net: read the channel count over netlink

[Eva Kurchatova]

napi_threaded needs the number of combined channels of the device and
asks ethtool to print it as JSON. Not every ethtool prints that one:
ethtool 6.11 does it for -k, -a, -c, -g and -x, but not for -l, and
the test ends in the first case:

  CmdExitFailure: Command failed: ethtool --json -l eth4
  STDERR: b'ethtool: bad command line argument(s)
  JSON output not available for this subcommand

  not ok 1 napi_threaded.napi_init

Ask the kernel for the count over netlink, which is where ethtool reads
it too, so the test no longer depends on the ethtool version installed.
Changing the count stays with ethtool -L, which needs no JSON. No other
test in the group asks ethtool for something it cannot print, so this is
the only one that needs it.

https://virtuozzo.atlassian.net/browse/VSTOR-139651
Feature: fix selftests
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
 tools/testing/selftests/drivers/net/napi_threaded.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/drivers/net/napi_threaded.py b/tools/testing/selftests/drivers/net/napi_threaded.py
index 20f1a12552da..6662aee3442b 100755
--- a/tools/testing/selftests/drivers/net/napi_threaded.py
+++ b/tools/testing/selftests/drivers/net/napi_threaded.py
@@ -7,7 +7,7 @@ Test napi threaded states.
 
 from lib.py import ksft_run, ksft_exit
 from lib.py import ksft_eq, ksft_ne, ksft_ge
-from lib.py import NetDrvEnv, NetdevFamily
+from lib.py import EthtoolFamily, NetDrvEnv, NetdevFamily
 from lib.py import cmd, defer, ethtool
 
 
@@ -28,7 +28,13 @@ def _set_threaded_state(cfg, threaded) -> None:
 
 
 def _setup_deferred_cleanup(cfg) -> None:
-    combined = ethtool(f"-l {cfg.ifname}", json=True)[0].get("combined", 0)
+    # Not every ethtool prints the channel counts as JSON, the one on
+    # the test machine here does not:
+    #   ethtool: bad command line argument(s)
+    #   JSON output not available for this subcommand
+    # Ask the kernel for them instead, it is the same number.
+    chan = EthtoolFamily().channels_get({'header': {'dev-index': cfg.ifindex}})
+    combined = chan.get('combined-count', 0)
     ksft_ge(combined, 2)
     defer(ethtool, f"-L {cfg.ifname} combined {combined}")
 
-- 
2.55.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH vz10 1/3] selftests: drv-net: let NetDrvEnv take nsim_test

[Konstantin Khorenko]

> rss_api.py opens its environment with
> 
>   with NetDrvEnv(__file__, nsim_test=False) as cfg:
> 
> NetDrvEnv hands every keyword it does not name to NetdevSimDev(), which
> takes port_count, queue_count and ns only, so the test ends in
> 
>   TypeError: NetdevSimDev.__init__() got an unexpected
>   keyword argument 'nsim_test'
> 
> The test came in with the RHEL10 import, commit 9f055df11343 ("rh10:
> import RHEL10 kernel-6.12.0-211.16.1.el10"); the parameter it asks for
> did not. Take the env.py hunk of commit 185646a8a0a8 ("selftests:
> drv-net: add tests for napi IRQ affinity notifiers"), which added that
> parameter upstream. The test that commit adds is left out, it needs a
> real device.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-139651
> Feature: fix vz selftests

Feature: fix selftests

-- 
Konstantin Khorenko <khorenko@virtuozzo.com>
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH vz10 2/3] selftests: drv-net: run ping in its own netns

[Konstantin Khorenko]

> In netdevsim mode NetDrvEpEnv gives the remote end a namespace of its
> own and leaves the local end in the one the test was started in, so
> the filtering of the host applies to the traffic the test sends to
> itself. A host that drops it fails the test:
> 
>   # nft add rule inet t input ip saddr 192.0.2.0/24 tcp flags syn drop
>   # ./ping.py
>   ok 1 ping.test_v4
>   ok 2 ping.test_v6
>   not ok 3 ping.test_tcp
>   # Stopping tests due to KsftTerminate.
> 
> ICMP is let through where TCP is not, which is why the first two pass
> and the test then sits there until it is killed.
> 
> Re-execute the test in a namespace of its own, the way nft_audit.sh and
> nft_concat_range.sh already do, and mount sysfs again there so that the
> netdevsim devices the test creates are the ones it sees.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-139651
> Feature: fix selftests
> Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
>
> diff --git a/tools/testing/selftests/drivers/net/lib/py/__init__.py b/tools/testing/selftests/drivers/net/lib/py/__init__.py
> index fce5d9218f1d6..2d3fdaa815504 100644
> --- a/tools/testing/selftests/drivers/net/lib/py/__init__.py
> +++ b/tools/testing/selftests/drivers/net/lib/py/__init__.py
> @@ -15,7 +15,7 @@ KSFT_DIR = (Path(__file__).parent / "../../../..").resolve()
>          NlError, RtnlFamily, DevlinkFamily
>      from net.lib.py import CmdExitFailure
>      from net.lib.py import bkg, cmd, defer, ethtool, fd_read_timeout, ip, \
> -        rand_port, tool, wait_port_listen
> +        rand_port, rexec_in_netns, tool, wait_port_listen
>      from net.lib.py import fd_read_timeout
>      from net.lib.py import KsftSkipEx, KsftFailEx, KsftXfailEx
>      from net.lib.py import ksft_disruptive, ksft_exit, ksft_pr, ksft_run, \
> diff --git a/tools/testing/selftests/drivers/net/ping.py b/tools/testing/selftests/drivers/net/ping.py
> index eb83e7b487978..e1cc74084a244 100755
> --- a/tools/testing/selftests/drivers/net/ping.py
> +++ b/tools/testing/selftests/drivers/net/ping.py
> @@ -5,6 +5,7 @@ from lib.py import ksft_run, ksft_exit
>  from lib.py import ksft_eq
>  from lib.py import NetDrvEpEnv
>  from lib.py import bkg, cmd, wait_port_listen, rand_port
> +from lib.py import rexec_in_netns
>  
>  
>  def test_v4(cfg) -> None:
> @@ -42,6 +43,10 @@ from lib.py import bkg, cmd, wait_port_listen, rand_port
>  
>  
>  def main() -> None:
> +    # In netdevsim mode the local end of the test stays in the namespace
> +    # the test was started in, so the rules of the host apply to it.
> +    rexec_in_netns(__file__)
> +

This re-execs unconditionally, but ping.py lives in drivers/net/, and
per the README the tests there run against a real driver as well, with
NETIF set in the environment or in net.config. _load_env_file() starts
from os.environ, so NETIF survives the exec, but the device itself
stays in init_net, and "ip link show dev $NETIF" fails in the new
namespace. The commit message says "in netdevsim mode"; the code does
not check for it.

Please re-exec only when NETIF is not set. Either in main(), or better
inside rexec_in_netns() itself, since the helper went into the shared
library and every caller will need the same check.

>      with NetDrvEpEnv(__file__) as cfg:
>          ksft_run(globs=globals(), case_pfx={"test_"}, args=(cfg, ))
>      ksft_exit()
> diff --git a/tools/testing/selftests/net/lib/py/utils.py b/tools/testing/selftests/net/lib/py/utils.py
> index afce547bcab09..13d7a7e6562e6 100644
> --- a/tools/testing/selftests/net/lib/py/utils.py
> +++ b/tools/testing/selftests/net/lib/py/utils.py
> @@ -8,6 +8,7 @@ import re
>  import select
>  import socket
>  import subprocess
> +import sys
>  import time
>  
>  
> @@ -131,6 +132,29 @@ global_defer_queue = []
>      return tool('ethtool', args, json=json, ns=ns, host=host)
>  
>  
> +def rexec_in_netns(path):
> +    """
> +    Re-exec the test in a network namespace of its own.
> +
> +    A test that puts one end of its traffic in the namespace it was
> +    started in otherwise depends on what the host filters.  sysfs is

drop double spaces

> +    mounted again in the new namespace, without it the netdevsim
> +    devices the test creates are not the ones it sees.  The test is

drop double spaces

> +    left where it is if unshare is not there to move it.
> +    """
> +    if os.environ.get("KSFT_IN_NETNS"):
> +        return
> +
> +    os.environ["KSFT_IN_NETNS"] = "1"
> +    try:
> +        os.execvp("unshare",
> +                  ["unshare", "-n", "-m",
> +                   "sh", "-c", 'mount -t sysfs sysfs /sys && exec "$@"',

lo is down in the new namespace. ping.py does not depend on loopback
and rand_port() binds to "", so nothing breaks today, but the helper is
shared, and the next test that talks to 127.0.0.1 will trip over it. An
"ip link set lo up" in the wrapper closes that for one line.

-- 
Konstantin Khorenko <khorenko@virtuozzo.com>
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

Re: [Devel] [PATCH vz10 3/3] selftests: drv-net: read the channel count over netlink

[Konstantin Khorenko]

> napi_threaded needs the number of combined channels of the device and
> asks ethtool to print it as JSON. Not every ethtool prints that one:
> ethtool 6.11 does it for -k, -a, -c, -g and -x, but not for -l, and
> the test ends in the first case:
> 
>   CmdExitFailure: Command failed: ethtool --json -l eth4
>   STDERR: b'ethtool: bad command line argument(s)
>   JSON output not available for this subcommand
> 
>   not ok 1 napi_threaded.napi_init
> 
> Ask the kernel for the count over netlink, which is where ethtool reads
> it too, so the test no longer depends on the ethtool version installed.
> Changing the count stays with ethtool -L, which needs no JSON. No other
> test in the group asks ethtool for something it cannot print, so this is
> the only one that needs it.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-139651
> Feature: fix selftests
> Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
>
> diff --git a/tools/testing/selftests/drivers/net/napi_threaded.py b/tools/testing/selftests/drivers/net/napi_threaded.py
> index ed66efa481b0f..73c81a319b49f 100755
> --- a/tools/testing/selftests/drivers/net/napi_threaded.py
> +++ b/tools/testing/selftests/drivers/net/napi_threaded.py
> @@ -7,7 +7,7 @@ Test napi threaded states.
>  
>  from lib.py import ksft_run, ksft_exit
>  from lib.py import ksft_eq, ksft_ne, ksft_ge
> -from lib.py import NetDrvEnv, NetdevFamily
> +from lib.py import EthtoolFamily, NetDrvEnv, NetdevFamily
>  from lib.py import cmd, defer, ethtool
>  
>  
> @@ -28,7 +28,13 @@ from lib.py import cmd, defer, ethtool
>  
>  
>  def _setup_deferred_cleanup(cfg) -> None:
> -    combined = ethtool(f"-l {cfg.ifname}", json=True)[0].get("combined", 0)
> +    # Not every ethtool prints the channel counts as JSON, the one on
> +    # the test machine here does not:
> +    #   ethtool: bad command line argument(s)
> +    #   JSON output not available for this subcommand
> +    # Ask the kernel for them instead, it is the same number.

The long description with error messages examples should be in the commit
message, not in the code comment. A short comment like
"not every ethtool prints -l as JSON, ask the kernel"
should be enough.

-- 
Konstantin Khorenko <khorenko@virtuozzo.com>
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel