From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mon, 1 Nov 2021 16:41:33 +0000 (-0700)
Subject: Merge tag 'for-5.16/io_uring-2021-10-29' of git://git.kernel.dk/linux-block
X-Git-Url: http://git.maquefel.me/?a=commitdiff_plain;h=8d1f01775f8ead7ee313403158be95bffdbb3638;p=linux.git

Merge tag 'for-5.16/io_uring-2021-10-29' of git://git.kernel.dk/linux-block

Pull io_uring updates from Jens Axboe:
 "Light on new features - basically just the hybrid mode support.

  Outside of that it's just fixes, cleanups, and performance
  improvements.

  In detail:

   - Add ring related information to the fdinfo output (Hao)

   - Hybrid async mode (Hao)

   - Support for batched issue on block (me)

   - sqe error trace improvement (me)

   - IOPOLL efficiency improvements (Pavel)

   - submit state cleanups and improvements (Pavel)

   - Completion side improvements (Pavel)

   - Drain improvements (Pavel)

   - Buffer selection cleanups (Pavel)

   - Fixed file node improvements (Pavel)

   - io-wq setup cancelation fix (Pavel)

   - Various other performance improvements and cleanups (Pavel)

   - Misc fixes (Arnd, Bixuan, Changcheng, Hao, me, Noah)"

* tag 'for-5.16/io_uring-2021-10-29' of git://git.kernel.dk/linux-block: (97 commits)
  io-wq: remove worker to owner tw dependency
  io_uring: harder fdinfo sq/cq ring iterating
  io_uring: don't assign write hint in the read path
  io_uring: clusterise ki_flags access in rw_prep
  io_uring: kill unused param from io_file_supports_nowait
  io_uring: clean up timeout async_data allocation
  io_uring: don't try io-wq polling if not supported
  io_uring: check if opcode needs poll first on arming
  io_uring: clean iowq submit work cancellation
  io_uring: clean io_wq_submit_work()'s main loop
  io-wq: use helper for worker refcounting
  io_uring: implement async hybrid mode for pollable requests
  io_uring: Use ERR_CAST() instead of ERR_PTR(PTR_ERR())
  io_uring: split logic of force_nonblock
  io_uring: warning about unused-but-set parameter
  io_uring: inform block layer of how many requests we are submitting
  io_uring: simplify io_file_supports_nowait()
  io_uring: combine REQ_F_NOWAIT_{READ,WRITE} flags
  io_uring: arm poll for non-nowait files
  fs/io_uring: Prioritise checking faster conditions first in io_write
  ...
---

8d1f01775f8ead7ee313403158be95bffdbb3638
diff --cc fs/io_uring.c
index 057d07cee9f8e,c887e4e19e9e4..ca10dbb012010
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@@ -6945,7 -6906,33 +6903,33 @@@ static void io_queue_linked_timeout(str
  	io_put_req(req);
  }
  
- static void __io_queue_sqe(struct io_kiocb *req)
+ static void io_queue_sqe_arm_apoll(struct io_kiocb *req)
+ 	__must_hold(&req->ctx->uring_lock)
+ {
+ 	struct io_kiocb *linked_timeout = io_prep_linked_timeout(req);
+ 
+ 	switch (io_arm_poll_handler(req)) {
+ 	case IO_APOLL_READY:
+ 		if (linked_timeout) {
 -			io_unprep_linked_timeout(req);
++			io_queue_linked_timeout(linked_timeout);
+ 			linked_timeout = NULL;
+ 		}
+ 		io_req_task_queue(req);
+ 		break;
+ 	case IO_APOLL_ABORTED:
+ 		/*
+ 		 * Queued up for async execution, worker will release
+ 		 * submit reference when the iocb is actually submitted.
+ 		 */
+ 		io_queue_async_work(req, NULL);
+ 		break;
+ 	}
+ 
+ 	if (linked_timeout)
+ 		io_queue_linked_timeout(linked_timeout);
+ }
+ 
+ static inline void __io_queue_sqe(struct io_kiocb *req)
  	__must_hold(&req->ctx->uring_lock)
  {
  	struct io_kiocb *linked_timeout;
@@@ -10645,11 -10697,9 +10703,11 @@@ static __cold int io_unregister_iowq_af
  	return io_wq_cpu_affinity(tctx->io_wq, NULL);
  }
  
- static int io_register_iowq_max_workers(struct io_ring_ctx *ctx,
- 					void __user *arg)
+ static __cold int io_register_iowq_max_workers(struct io_ring_ctx *ctx,
+ 					       void __user *arg)
 +	__must_hold(&ctx->uring_lock)
  {
 +	struct io_tctx_node *node;
  	struct io_uring_task *tctx = NULL;
  	struct io_sq_data *sqd = NULL;
  	__u32 new_count[2];