On 9/29/19 3:47 AM, Jackie Liu wrote: > > >> 在 2019年9月29日，07:23，Jens Axboe <[email protected]> 写道： >> >> Currently any dependent link is executed from a new workqueue context, >> which means that we'll be doing a context switch per link in the chain. >> If we are running the completion of the current request from our async >> workqueue and find that the next request is a link, then run it directly >> from the workqueue context instead of forcing another switch. >> >> This improves the performance of linked SQEs, and reduces the CPU >> overhead. >> >> Signed-off-by: Jens Axboe <[email protected]> >> >> --- >> >> 2-3x speedup doing read-write links, where the read often ends up >> blocking. Tested with examples/link-cp.c >> >> diff --git a/fs/io_uring.c b/fs/io_uring.c >> index aa8ac557493c..742d95563a54 100644 >> --- a/fs/io_uring.c >> +++ b/fs/io_uring.c >> @@ -667,7 +667,7 @@ static void __io_free_req(struct io_kiocb *req) >> kmem_cache_free(req_cachep, req); >> } >> >> -static void io_req_link_next(struct io_kiocb *req) >> +struct io_kiocb *io_req_link_next(struct io_kiocb *req) >> { >> struct io_kiocb *nxt; >> >> @@ -686,9 +686,19 @@ static void io_req_link_next(struct io_kiocb *req) >> } >> >> nxt->flags |= REQ_F_LINK_DONE; >> + /* >> + * If we're in async work, we can continue processing this, >> + * we can continue processing the chain in this context instead >> + * of having to queue up new async work. >> + */ >> + if (current_work()) >> + return nxt; >> INIT_WORK(&nxt->work, io_sq_wq_submit_work); >> io_queue_async_work(req->ctx, nxt); >> + nxt = NULL; >> } >> + >> + return nxt; >> } >> >> /* >> @@ -707,8 +717,10 @@ static void io_fail_links(struct io_kiocb *req) >> } >> } >> >> -static void io_free_req(struct io_kiocb *req) >> +static struct io_kiocb *io_free_req(struct io_kiocb *req) >> { >> + struct io_kiocb *nxt = NULL; >> + >> /* >> * If LINK is set, we have dependent requests in this chain. If we >> * didn't fail this request, queue the first one up, moving any other >> @@ -719,16 +731,30 @@ static void io_free_req(struct io_kiocb *req) >> if (req->flags & REQ_F_FAIL_LINK) >> io_fail_links(req); >> else >> - io_req_link_next(req); >> + nxt = io_req_link_next(req); >> } >> >> __io_free_req(req); >> + return nxt; >> } >> > > LGTM, Reviewed-by: Jackie Liu <[email protected]> > > The function io_free_req has been used not only for free req, but also for > the task > of finding the next link entry. I think it is possible to change a name to > avoid > confusion, of course, only personal opinion.
That's a good point. I also changed how we handle the return of that, so there's no confusion as to a caller getting a nxt request returned and not handling it. See here: http://git.kernel.dk/cgit/linux-block/commit/?h=for-5.5/io_uring&id=778fd7a24868b329ff8da2784fd8ced5e35af78c I'll send out a v2 that's the above, and your naming suggestion. -- Jens Axboe
