From: Alexei Starovoitov <ast@kernel.org>
Date: Wed, 17 Apr 2024 21:44:06 +0000 (-0700)
Subject: bpf: Fix JIT of is_mov_percpu_addr instruction.
X-Git-Url: http://git.maquefel.me/?a=commitdiff_plain;h=462e5e2a5938d0241ad146d21dd0da1be8e7eaf0;p=linux.git

bpf: Fix JIT of is_mov_percpu_addr instruction.

The codegen for is_mov_percpu_addr instruction works for rax/r8 registers
only. Fix it to generate proper x86 byte code for other registers.

Fixes: 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20240417214406.15788-1-alexei.starovoitov@gmail.com
---

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 2b5a475c4dd0d..673fdbd765d78 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1439,7 +1439,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 #ifdef CONFIG_SMP
 				/* add <dst>, gs:[<off>] */
 				EMIT2(0x65, add_1mod(0x48, dst_reg));
-				EMIT3(0x03, add_1reg(0x04, dst_reg), 0x25);
+				EMIT3(0x03, add_2reg(0x04, 0, dst_reg), 0x25);
 				EMIT((u32)(unsigned long)&this_cpu_off, 4);
 #endif
 				break;