efi/unaccepted: Avoid load_unaligned_zeropad() stepping into unaccepted memory

load_unaligned_zeropad() can lead to unwanted loads across page boundaries.
The unwanted loads are typically harmless. But, they might be made to
totally unrelated or even unmapped memory. load_unaligned_zeropad()
relies on exception fixup (#PF, #GP and now #VE) to recover from these
unwanted loads.

But, this approach does not work for unaccepted memory. For TDX, a load
from unaccepted memory will not lead to a recoverable exception within
the guest. The guest will exit to the VMM where the only recourse is to
terminate the guest.

There are two parts to fix this issue and comprehensively avoid access
to unaccepted memory. Together these ensure that an extra "guard" page
is accepted in addition to the memory that needs to be used.

1. Implicitly extend the range_contains_unaccepted_memory(start, end)
   checks up to end+unit_size if 'end' is aligned on a unit_size
   boundary.
2. Implicitly extend accept_memory(start, end) to end+unit_size if 'end'
   is aligned on a unit_size boundary.

Side note: This leads to something strange. Pages which were accepted
	   at boot, marked by the firmware as accepted and will never
	   _need_ to be accepted might be on unaccepted_pages list
	   This is a cue to ensure that the next page is accepted
	   before 'page' can be used.

This is an actual, real-world problem which was discovered during TDX
testing.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/r/20230606142637.5171-7-kirill.shutemov@linux.intel.com

diff --git a/drivers/firmware/efi/unaccepted_memory.c b/drivers/firmware/efi/unaccepted_memory.c
index 08a9a843550a5934cebf76a6db018faee13d4468..853f7dc3c21d8f0dafed9d43f57ebdf13fcf911a 100644 (file)
--- a/drivers/firmware/efi/unaccepted_memory.c
+++ b/drivers/firmware/efi/unaccepted_memory.c
@@ -46,6 +46,34 @@ void accept_memory(phys_addr_t start, phys_addr_t end)
        start -= unaccepted->phys_base;
        end -= unaccepted->phys_base;
 
+       /*
+        * load_unaligned_zeropad() can lead to unwanted loads across page
+        * boundaries. The unwanted loads are typically harmless. But, they
+        * might be made to totally unrelated or even unmapped memory.
+        * load_unaligned_zeropad() relies on exception fixup (#PF, #GP and now
+        * #VE) to recover from these unwanted loads.
+        *
+        * But, this approach does not work for unaccepted memory. For TDX, a
+        * load from unaccepted memory will not lead to a recoverable exception
+        * within the guest. The guest will exit to the VMM where the only
+        * recourse is to terminate the guest.
+        *
+        * There are two parts to fix this issue and comprehensively avoid
+        * access to unaccepted memory. Together these ensure that an extra
+        * "guard" page is accepted in addition to the memory that needs to be
+        * used:
+        *
+        * 1. Implicitly extend the range_contains_unaccepted_memory(start, end)
+        *    checks up to end+unit_size if 'end' is aligned on a unit_size
+        *    boundary.
+        *
+        * 2. Implicitly extend accept_memory(start, end) to end+unit_size if
+        *    'end' is aligned on a unit_size boundary. (immediately following
+        *    this comment)
+        */
+       if (!(end % unit_size))
+               end += unit_size;
+
        /* Make sure not to overrun the bitmap */
        if (end > unaccepted->size * unit_size * BITS_PER_BYTE)
                end = unaccepted->size * unit_size * BITS_PER_BYTE;
@@ -93,6 +121,13 @@ bool range_contains_unaccepted_memory(phys_addr_t start, phys_addr_t end)
        start -= unaccepted->phys_base;
        end -= unaccepted->phys_base;
 
+       /*
+        * Also consider the unaccepted state of the *next* page. See fix #1 in
+        * the comment on load_unaligned_zeropad() in accept_memory().
+        */
+       if (!(end % unit_size))
+               end += unit_size;
+
        /* Make sure not to overrun the bitmap */
        if (end > unaccepted->size * unit_size * BITS_PER_BYTE)
                end = unaccepted->size * unit_size * BITS_PER_BYTE;