x86/entry: Optimize common_interrupt_return()
authorPeter Zijlstra <peterz@infradead.org>
Mon, 20 Nov 2023 14:33:45 +0000 (15:33 +0100)
committerIngo Molnar <mingo@kernel.org>
Tue, 21 Nov 2023 12:57:30 +0000 (13:57 +0100)
commitc516213726fb572700cce4a5909aa8d82b77192a
tree58e59e4af6716d5276dd928e75e4a0d3fbd1a048
parent98b1cc82c4affc16f5598d4fa14b1858671b2263
x86/entry: Optimize common_interrupt_return()

The code in common_interrupt_return() does a bunch of unconditional
work that is really only needed on PTI kernels. Specifically it
unconditionally copies the IRET frame back onto the entry stack,
swizzles onto the entry stack and does IRET from there.

However, without PTI we can simply IRET from whatever stack we're on.

  ivb-ep, mitigations=off, gettid-1m:

  PRE:
       140,118,538      cycles:k                                                      ( +-  0.01% )
       236,692,878      instructions:k            #    1.69  insn per cycle           ( +-  0.00% )

  POST:
       140,026,608      cycles:k                                                      ( +-  0.01% )
       236,696,176      instructions:k            #    1.69  insn per cycle           ( +-  0.00% )

(this is with --repeat 100 and the run-to-run variance is bigger than
the difference shown)

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20231120143626.638107480@infradead.org
arch/x86/entry/calling.h
arch/x86/entry/entry_64.S