On Thu, May 5, 2011 at 3:50 PM, Paolo Bonzini <pbonz...@redhat.com> wrote: > On 05/05/2011 04:29 PM, Hannes Reinecke wrote: >>> >>> I chose 1 requestq per target so that, with MSI-X support, each >>> target can be associated to one MSI-X vector. >>> >>> If you want a large number of units, you can subdivide targets into >>> logical units, or use multiple adapters if you prefer. We can have >>> 20-odd SCSI adapters, each with 65534 targets. I think we're way >>> beyond the practical limits even before LUN support is added to QEMU. >> >> But this will make queue full tracking harder. >> If we have one queue per LUN the SCSI stack is able to track QUEUE FULL >> states and will adjust the queue depth accordingly. >> When we have only one queue per target we cannot track QUEUE FULL >> anymore and have to rely on the static per-host 'can_queue' setting. >> Which doesn't work as well, especially in a virtualized environment >> where the queue full conditions might change at any time. > > So you want one virtqueue per LUN? I had it in the first version, but then > you had to associate a (target, 8-byte LUN) pair to each virtqueue manually. > That was very hairy, so I changed it to one target per queue. > >> But read on: >> >>> For comparison, Windows supports up to 1024 targets per adapter >>> (split across 8 channels); IBM vSCSI provides up to 128; VMware >>> supports a maximum of 15 SCSI targets per adapter and 4 adapters per >>> VM. >>> >> We don't have to impose any hard limits here. The virtio scsi transport >> would need to be able to detect the targets, and we would be using >> whatever targets have been found. > > Yes, that's what I wrote above. Right now "detect the targets" means "send > INQUIRY for LUN0 and/or REPORT LUNS to each virtqueue", thanks to the 1:1 > relationship. In my first version it would mean: > > - associate each target's LUN0 to a virtqueue > - if needed, send INQUIRY for LUN0 and/or REPORT LUNS > - if needed, deassociate the LUN0 and the virtqueue > > Really, it was ugly. It also brings a lot more the question, such as what > to do if a virtqueue has pending requests at deassociation time. > >>> Yes, just add the first LUN to it (it will be LUN0 which must be >>> there anyway). The target's existence will be reported on the >>> control receiveq. >>> >> ?? How is this supposed to work? >> How can I detect the existence of a virtqueue ? > > Config space tells you how many virtqueue exist. That gives how many > targets you can address at most. If some of them are empty at the beginning > of the guest's life, their LUN0 will fail to answer INQUIRY and REPORT LUNS. > > (It is the same for vmw_pvscsi by the way, except simpler: the maximum # of > targets is not configurable, and there is just one queue + one interrupt).
Okay, this explains how you plan to handle targets appearing - you want to set a maximum number of targets. I was wondering how we would add virtqueues dynamically (and why the control vqs are placed last at n,n+1 instead of 0,1). Like Hannes said, why introduce a limit here if we don't have to? I'm really not sure I understand the win of creating lots of virtqueues. I just want a pipe out onto the SCSI bus so I can talk to all devices in the SCSI domain. Creating separate virtqueues increases complexity in the driver and emulation IMO. What is the MSI-X win you mentioned? I guess if an application on vcpu0 is accessing target0 a lot then interrupt handling can be handled on vcpu0 while other vcpus handle interrupts for other SCSI targets? I remember VMware pv scsi has a trick here, each request can contain the vcpu number which influences interrupt routing somehow. Stefan
