io_uring/rsrc: check for nonconsecutive pages

Pages that are from the same folio do not necessarily need to be
consecutive. In that case, we cannot consolidate them into a single bvec
entry. Before applying the huge page optimization from commit 57bebf807e2a
("io_uring/rsrc: optimise registered huge pages"), check that the memory
is actually consecutive.

Cc: stable@vger.kernel.org
Fixes: 57bebf807e2a ("io_uring/rsrc: optimise registered huge pages")
Signed-off-by: Tobias Holl <tobias@tholl.xyz>
[axboe: formatting]
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index ddee7adb400604865f8ea56d1c1da237dca456a9..00affcf811ad98b71fbcafcfbcb77caf85ba6791 100644 (file)
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -1117,7 +1117,12 @@ static int io_sqe_buffer_register(struct io_ring_ctx *ctx, struct iovec *iov,
        if (nr_pages > 1) {
                folio = page_folio(pages[0]);
                for (i = 1; i < nr_pages; i++) {
-                       if (page_folio(pages[i]) != folio) {
+                       /*
+                        * Pages must be consecutive and on the same folio for
+                        * this to work
+                        */
+                       if (page_folio(pages[i]) != folio ||
+                           pages[i] != pages[i - 1] + 1) {
                                folio = NULL;
                                break;
                        }