page_pool: try direct bulk recycling
authorAlexander Lobakin <aleksander.lobakin@intel.com>
Fri, 29 Mar 2024 16:55:07 +0000 (17:55 +0100)
committerJakub Kicinski <kuba@kernel.org>
Wed, 3 Apr 2024 01:13:49 +0000 (18:13 -0700)
commit39806b96c89ae5d52092c8f86393ecbfaae26697
tree78701c158045e79d05961247fd068dad9fe12d3c
parent4a96a4e807c390a9d91b450ebe04eeb2e0ecc076
page_pool: try direct bulk recycling

Now that the checks for direct recycling possibility live inside the
Page Pool core, reuse them when performing bulk recycling.
page_pool_put_page_bulk() can be called from process context as well,
page_pool_napi_local() takes care of this at the very beginning.
Under high .ndo_xdp_xmit() traffic load, the win is 2-3% Pps assuming
the sending driver uses xdp_return_frame_bulk() on Tx completion.

Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Link: https://lore.kernel.org/r/20240329165507.3240110-3-aleksander.lobakin@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
net/core/page_pool.c