io_uring: maintain CQE order of a failed link

Arguably we want CQEs of linked requests be in a strict order of
submission as it always was. Now if init of a request fails its CQE may
be posted before all prior linked requests including the head of the
link. Fix it by failing it last.

Fixes: de59bc104c24f ("io_uring: fail links more in io_submit_sqe()")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/b7a96b05832e7ab23ad55f84092a2548c4a888b0.1616699075.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 8c5789b96dbb6e6615554467c1479a143e2e5891..54ea561db4a556b76463474637da3b692c8169c7 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -6480,8 +6480,6 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct io_kiocb *req,
        ret = io_init_req(ctx, req, sqe);
        if (unlikely(ret)) {
 fail_req:
-               io_put_req(req);
-               io_req_complete(req, ret);
                if (link->head) {
                        /* fail even hard links since we don't submit */
                        link->head->flags |= REQ_F_FAIL_LINK;
@@ -6489,6 +6487,8 @@ fail_req:
                        io_req_complete(link->head, -ECANCELED);
                        link->head = NULL;
                }
+               io_put_req(req);
+               io_req_complete(req, ret);
                return ret;
        }
        ret = io_req_prep(req, sqe);