static_call: Properly initialise DEFINE_STATIC_CALL_RET0()
authorChristophe Leroy <christophe.leroy@csgroup.eu>
Mon, 14 Mar 2022 10:27:35 +0000 (11:27 +0100)
committerPeter Zijlstra <peterz@infradead.org>
Tue, 5 Apr 2022 07:59:38 +0000 (09:59 +0200)
commit5517d500829c683a358a8de04ecb2e28af629ae5
treeda7f5fefbdd46e59e7a62647e0b10277e0245b3e
parent8fd4ddda2f49a66bf5dd3d0c01966c4b1971308b
static_call: Properly initialise DEFINE_STATIC_CALL_RET0()

When a static call is updated with __static_call_return0() as target,
arch_static_call_transform() set it to use an optimised set of
instructions which are meant to lay in the same cacheline.

But when initialising a static call with DEFINE_STATIC_CALL_RET0(),
we get a branch to the real __static_call_return0() function instead
of getting the optimised setup:

c00d8120 <__SCT__perf_snapshot_branch_stack>:
c00d8120: 4b ff ff f4  b       c00d8114 <__static_call_return0>
c00d8124: 3d 80 c0 0e  lis     r12,-16370
c00d8128: 81 8c 81 3c  lwz     r12,-32452(r12)
c00d812c: 7d 89 03 a6  mtctr   r12
c00d8130: 4e 80 04 20  bctr
c00d8134: 38 60 00 00  li      r3,0
c00d8138: 4e 80 00 20  blr
c00d813c: 00 00 00 00  .long 0x0

Add ARCH_DEFINE_STATIC_CALL_RET0_TRAMP() defined by each architecture
to setup the optimised configuration, and rework
DEFINE_STATIC_CALL_RET0() to call it:

c00d8120 <__SCT__perf_snapshot_branch_stack>:
c00d8120: 48 00 00 14  b       c00d8134 <__SCT__perf_snapshot_branch_stack+0x14>
c00d8124: 3d 80 c0 0e  lis     r12,-16370
c00d8128: 81 8c 81 3c  lwz     r12,-32452(r12)
c00d812c: 7d 89 03 a6  mtctr   r12
c00d8130: 4e 80 04 20  bctr
c00d8134: 38 60 00 00  li      r3,0
c00d8138: 4e 80 00 20  blr
c00d813c: 00 00 00 00  .long 0x0

Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/1e0a61a88f52a460f62a58ffc2a5f847d1f7d9d8.1647253456.git.christophe.leroy@csgroup.eu
arch/powerpc/include/asm/static_call.h
arch/x86/include/asm/static_call.h
include/linux/static_call.h