On Fri, Aug 07, 2020 at 11:35:08AM +0800, Jason Wang wrote: > > On 2020/8/6 下午8:37, Stefan Hajnoczi wrote: > > On Wed, Aug 05, 2020 at 08:13:29AM -0400, Michael S. Tsirkin wrote: > > > On Wed, Aug 05, 2020 at 01:11:07PM +0100, Stefan Hajnoczi wrote: > > > > On Thu, Jul 30, 2020 at 07:46:09AM +0000, Yajun Wu wrote: > > > > > I'm doing iperf test on VIRTIO net through vhost-user(HW VDPA). > > > > > Find maximal acceptable tx_queue_size/rx_queue_size is 1024. > > > > > Basically increase queue size can get better RX rate for my case. > > > > > > > > > > Can we increase the limit(VIRTQUEUE_MAX_SIZE) to 8192 to possibly > > > > > gain better performance? > > > > Hi, > > > > The VIRTIO 1.1 specification says the maximum number of descriptors is > > > > 32768 for both split and packed virtqueues. > > > > > > > > The vhost kernel code seems to support 32768. > > > > > > > > The 1024 limit is an implementation limit in QEMU. Increasing it would > > > > require QEMU code changes. For example, VIRTQUEUE_MAX_SIZE is used as > > > > the size of arrays. > > > > > > > > I can't think of a fundamental reason why QEMU needs to limit itself to > > > > 1024 descriptors. Raising the limit would require fixing up the code and > > > > ensuring that live migration remains compatible with older versions of > > > > QEMU. > > > > > > > > Stefan > > > There's actually a reason for a limit: in theory the vq size > > > also sets a limit on the number of scatter/gather entries. > > > both QEMU and vhost can't handle a packet split over > 1k chunks. > > > > > > We could add an extra limit for s/g size like block and scsi do, > > > this will need spec, guest and host side work. > > Interesting, thanks for explaining! This could be made explicit by > > changing the QEMU code to: > > > > include/hw/virtio/virtio.h:#define VIRTQUEUE_MAX_SIZE IOV_MAX > > > > Looking more closely at the vhost kernel code I see that UIO_MAXIOV is > > used in some places but not in vhost_vring_set_num() (ioctl > > VHOST_SET_VRING_NUM). Is there a reason why UIO_MAXIOV isn't enforced > > when the application sets the queue size? > > > Actually three things: > > 1) queue size > 2) #descriptors in a list > 3) IOV size > > Spec limit the 2) to 1) but 2) may not equal to 3). > > So enforcing UIO_MAXIOV can not solve the problem completely. > > For vhost-net, it depends on socket to build skb which requires an iov array > to work. We need remove this limitation by: > > - build skb by vhost-net itself > - do piecewise copying > > Then we're not limited with #iov and more and support up to what spec > supports.
If I understand correctly, you are saying vhost_net.ko does not support more than UIO_MAXIOV descriptors today but it could be fixed as you described. Thanks! Stefan
signature.asc
Description: PGP signature
