mm/zswap: invalidate zswap entry when swap entry free
authorChengming Zhou <zhouchengming@bytedance.com>
Sun, 4 Feb 2024 03:06:00 +0000 (03:06 +0000)
committerAndrew Morton <akpm@linux-foundation.org>
Thu, 22 Feb 2024 18:24:54 +0000 (10:24 -0800)
commit0827a1fb143fae588cb6f5b9a97c405d6c2ddec9
tree76de3ff51d3ad34fbc26d8743134ce1b383d4fd9
parentf9c0f1c32cb568e16ef0676d8e7827a3ad443742
mm/zswap: invalidate zswap entry when swap entry free

During testing I found there are some times the zswap_writeback_entry()
return -ENOMEM, which is not we expected:

bpftrace -e 'kr:zswap_writeback_entry {@[(int32)retval]=count()}'
@[-12]: 1563
@[0]: 277221

The reason is that __read_swap_cache_async() return NULL because
swapcache_prepare() failed.  The reason is that we won't invalidate zswap
entry when swap entry freed to the per-cpu pool, these zswap entries are
still on the zswap tree and lru list.

This patch moves the invalidation ahead to when swap entry freed to the
per-cpu pool, since there is no any benefit to leave trashy zswap entry on
the tree and lru list.

With this patch:
bpftrace -e 'kr:zswap_writeback_entry {@[(int32)retval]=count()}'
@[0]: 259744

Note: large folio can't have zswap entry for now, so don't bother
to add zswap entry invalidation in the large folio swap free path.

Link: https://lkml.kernel.org/r/20240201-b4-zswap-invalidate-entry-v2-2-99d4084260a0@bytedance.com
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Yosry Ahmed <yosryahmed@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
include/linux/zswap.h
mm/swap_slots.c
mm/swapfile.c
mm/zswap.c