arm64/mm: remove now-superfluous ISBs from TTBR writes
authorJamie Iles <quic_jiles@quicinc.com>
Tue, 13 Jun 2023 14:19:59 +0000 (15:19 +0100)
committerCatalin Marinas <catalin.marinas@arm.com>
Thu, 15 Jun 2023 16:47:54 +0000 (17:47 +0100)
commitb9293d457ff3de415fa07d7e35978bceb29c3827
treea7cf67a4110f65dfe879c01118bc93bd40f82a80
parent601eaec513cc814c3dc6ed504c7c51ce1b80d0ea
arm64/mm: remove now-superfluous ISBs from TTBR writes

At the time of authoring 7655abb95386 ("arm64: mm: Move ASID from TTBR0
to TTBR1"), the Arm ARM did not specify any ordering guarantees for
direct writes to TTBR0_ELx and TTBR1_ELx and so an ISB was required
after each write to ensure TLBs would only be populated from the
expected (or reserved tables).

In a recent update to the Arm ARM, the requirements have been relaxed to
reflect the implementation of current CPUs and required implementation
of future CPUs to read (RDYDPX in D8.2.3 Translation table base address
register):

  Direct writes to TTBR0_ELx and TTBR1_ELx occur in program order
  relative to one another, without the need for explicit
  synchronization. For any one translation, all indirect reads of
  TTBR0_ELx and TTBR1_ELx that are made as part of the translation
  observe only one point in that order of direct writes.

Remove the superfluous ISBs to optimize uaccess helpers and context
switch.

Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/r/20230613141959.92697-1-quic_jiles@quicinc.com
[catalin.marinas@arm.com: rename __cpu_set_reserved_ttbr0 to ..._nosync]
[catalin.marinas@arm.com: move the cpu_set_reserved_ttbr0_nosync() call to cpu_do_switch_mm()]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
arch/arm64/include/asm/asm-uaccess.h
arch/arm64/include/asm/mmu_context.h
arch/arm64/include/asm/uaccess.h
arch/arm64/mm/context.c