linux-user: detect overflow of MAP_FIXED mmap

Relaxing the restrictions on 64 bit guests leads to the user being
able to attempt to map right at the edge of addressable memory. This
in turn lead to address overflow tripping the assert in page_set_flags
when the end address wrapped around.

Detect the wrap earlier and correctly -ENOMEM the guest (in the
reported case LTP mmap15).

Fixes: 7d8cbbabcb
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20200605154929.26910-15-alex.bennee@linaro.org>

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index caab62909eb1c518030e331ebe8e9acdb3d6e34e..0019447892e0fd51e0259d766fbee1a385b53b0e 100644 (file)
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -467,7 +467,7 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
          * It can fail only on 64-bit host with 32-bit target.
          * On any other target/host host mmap() handles this error correctly.
          */
-        if (!guest_range_valid(start, len)) {
+        if (end < start || !guest_range_valid(start, len)) {
             errno = ENOMEM;
             goto fail;
         }