From: Pavel Begunkov <asml.silence@gmail.com>
Date: Thu, 21 Nov 2019 18:24:56 +0000 (+0300)
Subject: io_uring: add likely/unlikely in io_get_sqring()
X-Git-Url: http://git.maquefel.me/?a=commitdiff_plain;h=9835d6fafba58e6d9386a6d5af800789bdb52e5b;p=linux.git

io_uring: add likely/unlikely in io_get_sqring()

The number of SQEs to submit is specified by a user, so io_get_sqring()
in most of the cases succeeds. Hint compilers about that.

Checking ASM genereted by gcc 9.2.0 for x64, there is one branch
misprediction.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 956a61cb0b132..63e0448f3f8d9 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -3129,11 +3129,11 @@ static bool io_get_sqring(struct io_ring_ctx *ctx, struct sqe_submit *s)
 	 */
 	head = ctx->cached_sq_head;
 	/* make sure SQ entry isn't read before tail */
-	if (head == smp_load_acquire(&rings->sq.tail))
+	if (unlikely(head == smp_load_acquire(&rings->sq.tail)))
 		return false;
 
 	head = READ_ONCE(sq_array[head & ctx->sq_mask]);
-	if (head < ctx->sq_entries) {
+	if (likely(head < ctx->sq_entries)) {
 		s->ring_file = NULL;
 		s->sqe = &ctx->sq_sqes[head];
 		s->sequence = ctx->cached_sq_head;