riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU
authorJisheng Zhang <jszhang@kernel.org>
Tue, 19 Dec 2023 17:50:45 +0000 (01:50 +0800)
committerPalmer Dabbelt <palmer@rivosinc.com>
Wed, 24 Jan 2024 23:55:55 +0000 (15:55 -0800)
commit69be3fb111e73bd025ce6d2322371da5aa497c70
treeb46277ed0bfe4a2e1e1aa83bd0a56cce12398624
parent40d1bb92a49313b3e0dc5513fdd2578362c40312
riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU

In order to implement fast gup we need to ensure that the page
table walker is protected from page table pages being freed from
under it.

riscv situation is more complicated than other architectures: some
riscv platforms may use IPI to perform TLB shootdown, for example,
those platforms which support AIA, usually the riscv_ipi_for_rfence is
true on these platforms; some riscv platforms may rely on the SBI to
perform TLB shootdown, usually the riscv_ipi_for_rfence is false on
these platforms. To keep software pagetable walkers safe in this case
we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the
comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in
include/asm-generic/tlb.h for more details.

This patch enables MMU_GATHER_RCU_TABLE_FREE, then use

*tlb_remove_page_ptdesc() for those platforms which use IPI to perform
TLB shootdown;

*tlb_remove_ptdesc() for those platforms which use SBI to perform TLB
shootdown;

Both case mean that disabling interrupts will block the free and
protect the fast gup page walker.

Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231219175046.2496-4-jszhang@kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
arch/riscv/Kconfig
arch/riscv/include/asm/pgalloc.h
arch/riscv/include/asm/tlb.h