locking/atomic/x86: Introduce arch_try_cmpxchg64_local()
authorUros Bizjak <ubizjak@gmail.com>
Sun, 14 Apr 2024 16:12:43 +0000 (18:12 +0200)
committerIngo Molnar <mingo@kernel.org>
Sun, 14 Apr 2024 20:40:54 +0000 (22:40 +0200)
commitd26e46f6bf329cfcc469878709baa41d3bfc7cc3
treef76ffaccd1fc92741244b9deeebca6756b0aca16
parent91095666125a666c8f20c2323b742c53165c0325
locking/atomic/x86: Introduce arch_try_cmpxchg64_local()

Introduce arch_try_cmpxchg64_local() for 64-bit and 32-bit targets
to improve code using cmpxchg64_local().  On 64-bit targets, the
generated assembly improves from:

    3e28: 31 c0                 xor    %eax,%eax
    3e2a: 4d 0f b1 7d 00        cmpxchg %r15,0x0(%r13)
    3e2f: 48 85 c0              test   %rax,%rax
    3e32: 0f 85 9f 00 00 00     jne    3ed7 <...>

to:

    3e28: 31 c0                 xor    %eax,%eax
    3e2a: 4d 0f b1 7d 00        cmpxchg %r15,0x0(%r13)
    3e2f: 0f 85 9f 00 00 00     jne    3ed4 <...>

where a TEST instruction after CMPXCHG is saved.  The improvements
for 32-bit targets are even more noticeable, because double-word
compare after CMPXCHG8B gets eliminated.

Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Waiman Long <longman@redhat.com>
Link: https://lore.kernel.org/r/20240414161257.49145-1-ubizjak@gmail.com
arch/x86/include/asm/cmpxchg_32.h
arch/x86/include/asm/cmpxchg_64.h