On 11/30/2017 10:50 AM, Cornelia Huck wrote: > On Wed, 29 Nov 2017 19:51:23 +0100 > Christian Borntraeger <borntrae...@de.ibm.com> wrote: > >> On 11/28/2017 03:45 PM, Cornelia Huck wrote: >>> On Tue, 28 Nov 2017 15:17:49 +0100 >>> Christian Borntraeger <borntrae...@de.ibm.com> wrote: >>> >>>> On 11/28/2017 03:01 PM, Cornelia Huck wrote: >>>>> On Tue, 28 Nov 2017 14:25:08 +0100 >>>>> Christian Borntraeger <borntrae...@de.ibm.com> wrote: >>> >>>>>> What I want now is to enable vfio-ccw for libvirt and Linux guests and >>>>>> for that >>>>>> we need to lift the restriction of having vfio not in FE. >>>>> >>>>> This, however, worries me. I don't really consider vfio-ccw ready for >>>>> prime time yet. We still need to figure out path handling at the very >>>>> least. And I'm still not sure that our non-handling of halt/clear won't >>>>> cause major issues with e.g. a storage server error recovery. >>>>> >>>>> Can we flag vfio-ccw as experimental or such in libvirt? >>>> >>>> We should have flagged vfio-ccw experimental in QEMU then. >>>> e.g. by using x-vfio-ccw or whatever.I dont think we can do that >>>> after the facts. >>> >>> Yes, we should have done that... can't fix that now, unfortunately. >>> >>>> >>>> I am not that deep into vfio-ccw, but I was under the impression that >>>> the missing features should not affect the vfio interface that is >>>> created by libvirt. After all this should just be a "-device >>>> vfio-ccw,....." >>>> statement and some setup steps beforehand (unbind + setup of vfio-ccw) >>> >>> The halt/clear stuff is highly unlikely to influence the configuration >>> interface. I'm still not too clear about path handling, though. Will we >>> need different modes (hypervisor managed vs. full passthrough? probably >>> only passthrough)? What about pgid handling - do we need some kind of >>> pgid manager? (Keep in mind that you get to set the pgid once. This has >>> implications for e.g. reserve/release.) >>> >>> Also, what about devices other than ECKD DASD? Has there been any >>> testing? Tapes should be interesting; the other interesting ccw devices >>> are QDIO-based and I'm not sure we want to spend anything on supporting >>> those. >> >> DASD is probably the most important thing today, QDIO might never happen >> (or very late). > > One thing I'd find interesting about tapes is very long running channel > programs (like rewind) and how they interact with halt/clear. But yes, > I would think that DASD is the most important one in practice. > >> See my proposal below. >> >>> >>> The interface is probably fine, but I'd like to get an idea about the >>> path stuff (is the attachment spec that contains the pgid stuff >>> publicly available, btw.?) >>> >>>> >>>> If your concern is the serviceability I think it would be valid for a RHEL >>>> KVM to disable vfio-ccw in the kernel. Maybe we could even provide a config >>>> for QEMU? >>> >>> It's fine just to turn off vfio-ccw in the kernel. >>> >>>> PS: I learned from the PCI and CCW experience that I do not want >>>> to upstream kernel/qemu code unless I have a working libvirt code that >>>> shows that the thing will work. >>> >>> Yes, I understand the wish to verify the interface, and I think it's a >>> good idea. What I'm worried about is that this might be the precursor >>> to a push to support it, and I don't think vfio-ccw is ready for that >>> yet. >>> >> FWIW, libvirt should not care if a device works in all cases or not because >> libvirt versions should work with all kind of QEMU/kernel combinations. >> Fencing in libvirt that the kernel part is not up to the task is making >> me feel the same way as you when you see the css-unrestricted property >> at a device ;-) > > I'm more worried about the political angle than the technical. If we > don't get pressure to support this too soon, I don't care that much > about experimental or not. > >> Having said that, I think that having vfio-ccw support has a value (and it >> actually works for an unnamed test infrastructure). In addition to that I >> am much more likely to actually test this continuously if I have libvirt >> support. > > Good to hear that it works for a non-Linux guest. Any plans to test > something like storage server failover? > > [I'd really love to do something about the path handling stuff, but the > combination of scant documentation and scantier time works against me.] > >> >> >> So what about the following: >> >> 1. We will implement the libvirt support with either a or b: >> a: if we find a solution to our "where to put the property" dispute use that >> to decide >> if we can add vfio-ccw >> b if not: just provide a patch that lifts the restriction without any >> property. >> in libvirt blindy assume that free assignment will work. old QEMUs will >> complain at >> startup and libvirt will print the QEMU error. This is similar to other >> situations >> where libvirt cannot fully bprobe if the QEMU will work or not. (not having >> vfio-ccw >> support in the kernel will certainly allow libvirt to reject this upfront) > > I hope we end up with a solution that works for everyone... > >> >> 2. at the same time we mark vfio-ccw experimental in the kernel to make clear >> that this is still work in progress > > I'm not sure we can do that after the fact, but let's do it if we can.
I think we can change that. Its just a Kconfig dependency. >> >> 3. (optional) we could even whitelist devices that work in a reasonable >> fashion (e.g. >> do not whitelist qdio but whitelist DASD) > > I think it's not quite that easy with vfio-ccw working at the > subchannel level. >
