On Sat, Dec 21, 2019 at 02:07:19AM +0000, Wangyong wrote: > > From: Stefan Hajnoczi [mailto:stefa...@gmail.com] > > Sent: Friday, December 20, 2019 5:53 PM > > To: wangyong (Cloud) <wang.yo...@h3c.com> > > Cc: Stefan Hajnoczi <stefa...@redhat.com>; pbonz...@redhat.com; > > mark.ka...@oracle.com; h...@lst.de; qemu-devel@nongnu.org > > Subject: Re: issue about virtio-blk queue size > > > > On Thu, Dec 05, 2019 at 01:30:09AM +0000, Wangyong wrote: > > > > > > > > On Thu, Nov 28, 2019 at 08:44:43AM +0000, Wangyong wrote: > > > > > Hi all, > > > > > > > > This looks interesting, please continue this discussion on the QEMU > > > > mailing list <qemu-devel@nongnu.org> so that others can participate. > > > > > > > > > > > > > > This patch makes virtio_blk queue size configurable > > > > > > > > > > commit 6040aedddb5f474a9c2304b6a432a652d82b3d3c > > > > > Author: Mark Kanda <mark.ka...@oracle.com> > > > > > Date: Mon Dec 11 09:16:24 2017 -0600 > > > > > > > > > > virtio-blk: make queue size configurable > > > > > > > > > > But when we set the queue size to more than 128, it will not take > > > > > effect. > > > > > > > > > > That's because linux aio's maximum outstanding requests at a time > > > > > is always less than or equal to 128 > > > > > > > > > > The following code limits the outstanding requests at a time: > > > > > > > > > > #define MAX_EVENTS 128 > > > > > > > > > > laio_do_submit() > > > > > { > > > > > > > > > > if (!s->io_q.blocked && > > > > > (!s->io_q.plugged || > > > > > s->io_q.in_flight + s->io_q.in_queue >= MAX_EVENTS)) { > > > > > ioq_submit(s); > > > > > } > > > > > } > > > > > > > > > > Should we make the value of MAX_EVENTS configurable ? > > > > > > > > Increasing MAX_EVENTS to a larger hardcoded value seems reasonable > > > > as a shortterm fix. Please first check how /proc/sys/fs/aio-max-nr > > > > and > > > > io_setup(2) handle this resource limit. The patch must not break > > > > existing systems where 128 works today. > > > [root@node2 ~]# cat /etc/centos-release CentOS Linux release 7.5.1804 > > > (Core) > > > > > > [root@node2 ~]# cat /proc/sys/fs/aio-max-nr > > > 4294967296 > > > > > > > > MAX_EVENTS should have the same value as queue size ? > > > > > > > > Multiple virtio-blk devices can share a single AioContext, > > > Is multiple virtio-blk configured with one IOThread? > > > Multiple virtio-blk performance will be worse. > > > > Yes. By default IOThreads are not used and all virtio-blk devices share the > > main loop's AioContext. > > > > When IOThreads are configured it's up to the user how to assign devices to > > IOThreads. Assigning multiple devices to one IOThread is realistic because > > it's common to create only num_vcpus IOThreads. > > > > A good starting point would be a patch that raises the limit to a higher > > hardcoded number. Then you can investigate how to size the AioContext > > appropriately (maybe dynamically?) for a full fix. > > > Okay. Raising MAX_EVENT to 1024 may be more appropriate? If possible I can > try to make a patch.
Sounds good. Stefan
signature.asc
Description: PGP signature
