arm64: atomic: always inline the assembly
authorMark Rutland <mark.rutland@arm.com>
Wed, 17 Aug 2022 15:59:14 +0000 (16:59 +0100)
committerCatalin Marinas <catalin.marinas@arm.com>
Fri, 9 Sep 2022 12:58:33 +0000 (13:58 +0100)
commit78f6f5c994ed22a35ce1cd3ec9aeda8e2fa328e6
tree7a26f895fa5f3083c72d11b853b07ee9afd0a00c
parentb2c3ccbd0011bb3b51d0fec24cb3a5812b1ec8ea
arm64: atomic: always inline the assembly

The __lse_*() and __ll_sc_*() atomic implementations are marked as
inline rather than __always_inline, permitting a compiler to generate
out-of-line versions, which may be instrumented.

We marked the atomic wrappers as __always_inline in commit:

  c35a824c31834d94 ("arm64: make atomic helpers __always_inline")

... but did not think to do the same for the underlying implementations.

If the compiler were to out-of-line an LSE or LL/SC atomic, this could
break noinstr code. Ensure this doesn't happen by marking the underlying
implementations as __always_inline.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20220817155914.3975112-3-mark.rutland@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
arch/arm64/include/asm/atomic_ll_sc.h
arch/arm64/include/asm/atomic_lse.h