KVM: arm64: Load the stage-2 MMU context in kvm_vcpu_load_vhe()

To date the VHE code has aggressively reloaded the stage-2 MMU context
on every guest entry, despite the fact that this isn't necessary. This
was probably done for consistency with the nVHE code, which needs to
switch in/out the stage-2 MMU context as both the host and guest run at
EL1.

Hoist __load_stage2() into kvm_vcpu_load_vhe(), thus avoiding a reload
on every guest entry/exit. This is likely to be beneficial to systems
with one of the speculative AT errata, as there is now one fewer context
synchronization event on the guest entry path. Additionally, it is
possible that implementations have hitched correctness mitigations on
writes to VTTBR_EL2, which are now elided on guest re-entry.

Note that __tlb_switch_to_guest() is deliberately left untouched as it
can be called outside the context of a running vCPU.

Link: https://lore.kernel.org/r/20231018233212.2888027-6-oliver.upton@linux.dev
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index d05b6a08dcde0db51b43167cbb69ddcb1892f4f9..b0cafd7c5f8fdf90386578e02975acba4f649ea1 100644 (file)
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -120,6 +120,7 @@ void kvm_vcpu_load_vhe(struct kvm_vcpu *vcpu)
 {
        __vcpu_load_switch_sysregs(vcpu);
        __vcpu_load_activate_traps(vcpu);
+       __load_stage2(vcpu->arch.hw_mmu, vcpu->arch.hw_mmu->arch);
 }
 
 void kvm_vcpu_put_vhe(struct kvm_vcpu *vcpu)
@@ -182,17 +183,11 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)
        sysreg_save_host_state_vhe(host_ctxt);
 
        /*
-        * ARM erratum 1165522 requires us to configure both stage 1 and
-        * stage 2 translation for the guest context before we clear
-        * HCR_EL2.TGE.
-        *
-        * We have already configured the guest's stage 1 translation in
-        * kvm_vcpu_load_sysregs_vhe above.  We must now call
-        * __load_stage2 before __activate_traps, because
-        * __load_stage2 configures stage 2 translation, and
-        * __activate_traps clear HCR_EL2.TGE (among other things).
+        * Note that ARM erratum 1165522 requires us to configure both stage 1
+        * and stage 2 translation for the guest context before we clear
+        * HCR_EL2.TGE. The stage 1 and stage 2 guest context has already been
+        * loaded on the CPU in kvm_vcpu_load_vhe().
         */
-       __load_stage2(vcpu->arch.hw_mmu, vcpu->arch.hw_mmu->arch);
        __activate_traps(vcpu);
 
        __kvm_adjust_pc(vcpu);