net: pin system percpu page_pools to the corresponding NUMA nodes

System page_pools are percpu and one instance can be used only on
one CPU.
%NUMA_NO_NODE is fine for allocating pages, as the PP core always
allocates local pages in this case. But for the struct &page_pool
itself, this node ID means they are allocated on the boot CPU,
which may belong to a different node than the target CPU.
Pin system page_pools to the corresponding nodes when creating,
so that all the allocated data will always be local. Use
cpu_to_mem() to account memless nodes.
Nodes != 0 win some Kpps when testing with xdp-trafficgen.

Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Link: https://lore.kernel.org/r/20240325160635.3215855-1-aleksander.lobakin@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

diff --git a/net/core/dev.c b/net/core/dev.c
index 9ce34164bcb1cd26ae6e49e402f87f6f97067a7c..5d36a634f468ffdeaca598c3dd033fe06d240bd0 100644 (file)
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11762,7 +11762,7 @@ static int net_page_pool_create(int cpuid)
        struct page_pool_params page_pool_params = {
                .pool_size = SYSTEM_PERCPU_PAGE_POOL_SIZE,
                .flags = PP_FLAG_SYSTEM_POOL,
-               .nid = NUMA_NO_NODE,
+               .nid = cpu_to_mem(cpuid),
        };
        struct page_pool *pp_ptr;