io-wq: write next_work before dropping acct_lock
authorGabriel Krisman Bertazi <krisman@suse.de>
Tue, 16 Apr 2024 02:10:53 +0000 (22:10 -0400)
committerJens Axboe <axboe@kernel.dk>
Wed, 17 Apr 2024 14:20:32 +0000 (08:20 -0600)
commit068c27e32e51e94e4a9eb30ae85f4097a3602980
tree2a780cd67afbf9a64419aff1dbce510685622348
parentc4ce0ab27646f4206a9eb502d6fe45cb080e1cae
io-wq: write next_work before dropping acct_lock

Commit 361aee450c6e ("io-wq: add intermediate work step between pending
list and active work") closed a race between a cancellation and the work
being removed from the wq for execution.  To ensure the request is
always reachable by the cancellation, we need to move it within the wq
lock, which also synchronizes the cancellation.  But commit
42abc95f05bf ("io-wq: decouple work_list protection from the big
wqe->lock") replaced the wq lock here and accidentally reintroduced the
race by releasing the acct_lock too early.

In other words:

        worker                |     cancellation
work = io_get_next_work()     |
raw_spin_unlock(&acct->lock); |
      |
                              | io_acct_cancel_pending_work
                              | io_wq_worker_cancel()
worker->next_work = work

Using acct_lock is still enough since we synchronize on it on
io_acct_cancel_pending_work.

Fixes: 42abc95f05bf ("io-wq: decouple work_list protection from the big wqe->lock")
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Link: https://lore.kernel.org/r/20240416021054.3940-2-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/io-wq.c