Wei and others, It has been a long, long road to getting a rumprun-based stubdom implemented. The earliest discussion of pursuing this approach appears to be a post by IanJ on August 21, 2013, with the first serious indication of development work again posted by IanJ, on April 8, 2014 - over 19 months ago:
http://lists.xenproject.org/archives/html/xen-devel/2014-04/msg01066.html >From what is described below, it looks like there remains a long way to go, as what appears to be running is only printing out the output from "qemu --help" (pretty much what was described at XPDS15 more than three months ago). Once upon a time, one of the release managers was talking about making QEMU upstream stubdom a blocker for 4.5: http://lists.xen.org/archives/html/xen-devel/2014-04/msg01065.html Obviously that did not happen. Time and again, QEMU upstream stubdom has gotten shelved in favor of other priorities, which testifies as to where it falls on the Xen project's list of priorities. I believe it is time to reassess the current development plan for QEMU upstream stubdom. From a project management perspective, the current plan relies far too heavily on a very small handful of people that are simply too busy with other things to bring it to a working, let alone releasable, state. Plus, the delay in realizing this has now reached a point where it imposes frequent, tangible, and possibly significant drains on development resources in order to cope with shortcomings of the current situation (QEMU upstream in dom0, and separate qemu-traditional codebase for stubdom). Plus, having QEMU upstream run in dom0 is likely Xen's single longest running identified security issue - and certainly the longest-lived one with an identified and actionable solution. Consider how many of the QEMU-related XSAs reported something along the lines of "systems using qemu-dm stubdomain device models are NOT vulnerable." Recently, following several XSAs demonstrating an interest in finding (and likely exploiting) QEMU vulnerabilities, the Xen team has devoted resources to running QEMU as non-root (now on patchset v10) - apparently trying to replicate the bandaid used by KVM. This effort would not have been necessary if QEMU upstream stubdom had been implemented. Additionally, the Xen team bears the burdens of maintaining the qemu-traditional tree, backporting fixes originating from various places into qemu-traditional, and having libxl cope with the differences between QEMU upstream and traditional and associated development headaches. These represent underappreciated and unnecessary drain on the Xen project's limited resources imposed by not getting QEMU upstream stubdom implemented in a timely manner. Also, it looks like QEMU upstream would be the single most complex piece of software to be implemented under rumprun, which puts the current plan development way out on the bleeding edge. It seems more like an experiment with lots of unknowns than a development project with a defined and predictable plan that simply needs man hours thrown at it. Although rumprun has technical "sex appeal," and it taps into the community's recent interest in unikernels, rumprun still seems to be very immature at this time. On top of that, getting this project implemented - even at a basic level of functionality - appears to require VERY specialized expertise held by VERY few people. Is there really anyone other than Wei, IanJ, and Antti that has the skills needed to realize the current plan? Further tightening the resource crunch, those three are in very high demand for other projects: Wei had (and has?) release management duties, IanJ is involved in pretty much everything Xen, and Antti has broader pursuits to worry about. The fact that no one on xen-devel has commented on Wei's October update in over a month indicates that he is essentially alone in his efforts. When does anyone _realistically_ see a rumprun-based QEMU reaching something approaching just a basic level of functionality as a stubdomain - let's say being able to run Windows 7 with a working display and networking? Unless the answer is along the lines of hitting at least tech preview status in Xen 4.7, I suggest that rumprun is not the right approach at this time. I am fairly certain - based on my direct efforts in helping to move it along - that a Linux-based stubdom is something that CAN be realized in the 4.7 release timeframe. Almost all of the problems you are trying to solve at this time with rumprun are already resolved today under Linux. Today, on a Linux-based QEMU upstream stubdom, you can SSH into a headless Linux distribution with fully working networking - this is light years beyond where rumprun is or will be any time soon. Unsurprisingly, QEMU runs just fine in a Linux environment. The only thing that remains to reach the "basic level of functionality" mentioned above is getting the display and keyboard/mouse input working, which I suspect will not be that big of an effort for someone familiar with QEMU upstream internals (and would have to be done anyway for rumprun). I also believe that what has been described as the main problem for Linux-based stubdom - building a Linux-based image - is not as big as it has been made out to be: raisin may now reduce some of the previously perceived difficulties, and there is so much Linux expertise in the Xen community that I find it incredible that it is too hard to sort out how to build Linux on a Linux system that is already capable of building Xen and QEMU upstream. I am not suggesting that rumprun should be abandoned - just that in terms of actually getting QEMU upstream stubdom released and finally kicking qemu-traditional and other development resource drains to the curb, a Linux-based stubdom represents a practical and effective solution that can be used until rumprun is more mature and predictable. Almost all of the work (with the exception of building the Linux-based image) is the same work that will have to be done to get the rumprun implementation working: getting the display working, and getting QEMU upstream to provide its current range of features from within a stubdom (as much of what is in there today depends on running in dom0). So, when the day arrives that a rumprun-based stubdom is ready, little of what needs to be done for a Linux-based stubdom will end up being wasted effort. Plus, there might just be some benefits from having QEMU running in a Linux PV domain (firewalling provided in the stubdom and easier debug, as possible examples) that could justify keeping it around even after rumprun is practical. Thank you for taking the time to read this. I have been following, and participating a bit in, this issue for more than 2.5 years, and I haven't observed any progress in that time - if anything, things are beginning to make backwards now that QEMU has become a target. I believe it is time for the Xen project to acknowledge that a rumprun-based stubdom is beyond the level of resources and expertise that the project is actually willing or able to commit to it (as Xen has - quite rationally - set it aside time and again to pursue other matters). In the face of finite and limited development resources, it is sensible to set aside a "perfect" solution aside, and instead pursue a path that has a genuine prospect of getting released sometime soon. All the best, Eric > Hi all > > I've been working on and off on upstream QEMU stubdom. Unfortunately in > the last few months I didn't make much progress as I had expected. Some > folks expressed interests during Xen developer summit on this project > and I think it would be better I post what I have as soon as possible. > > In short, it is now possible to compile and run QEMU as a PV guest, > but I haven't finished the plumbing of toolstack code, hence I don't > know whether it functions as a device model. > > Note that there are quite a bit of hacks inside those scripts to work > around problems in build systems of different piece of software. > > There is a small program to configure network topology inside the > guest. It creates a bridge, then adds a bunch of tap interfaces and > existing Xen vif interface(s) to the bridge. > > What are missing: > > 1. Inter-domain communication mechanism: this can be subsidised by > network interface for now. > 2. Plumbing through libxl. It will be using rumprun to launch QEMU > stubdom. > 3. Integration with Raisin build system. This is the ulterior goal to > get rid of all customised build scripts. > > I haven't checked everything, but some of the rump kernel interfaces > are not yet declared stable. Relevant discussions should mainly take > place on rump kernel mailing list and CC xen-devel when necessary. > > Here are the step-by-step guide for building a QEMU unikernel. > > Note that all of my scripts have hardcoded paths (/local/scratch/...), > so anyone who is interested in using them would need to replace the > hardcoded paths with something sensible. > > $XEN refers to Xen source code. I will always refer to upstream > repositories and only provide git branches unless there are local > changes needed. > > Rumprun is a fast moving target so I will also state the working > version of things. > > # Build rumprun toolchain (based on a5f58548) > > $ git clone https://github.com/rumpkernel/rumprun.git rumprun.git > $ cd rumprun.git > $ export XEN_HEADERS=$XEN/xen/include/public > $ ./build-rr xen -- -j8 > > If nothing goes wrong, the toolchain will be available under app-tools > directory. > > # Build QEMU dependency > > Currently only a minimum set of libraries that QEMU depends on are > built. They are zlib, libffi, glib, libpixman, Xen libxenctrl and > libxenstore. More libraries might be needed as we move forward, but > overall the build process should be the same. > > I have a git repository for build scripts. > > $ git clone https://xenbits.xen.org/people/liuw/rump-build-scrpt-junk.git rump-build-script-junk > $ cd rump-build-script-junk > $ ./script 2>&1 | tee buildlog > > The script will download tarballs from various websites and expects > Xen source tree resides at a fix path (see script). > > If everything goes well, those libraries will be inside > /local/scratch/Rump-kernels/prefix. If something goes wrong, dig into > buildlog and figure out why. :-) > > # Build QEMU > > I have prepared a branch for my WIP patches for QEMU. > > $ cd rump-build-junk-script > $ git clone https://xenbits.xen.org/people/liuw/qemu.git -b wip.rump qemu > $ ./script-qemu 2>&1 | tee buildlog > > If everything goes well, qemu will be inside > /local/scratch/Rump-kernels/prefix-qemu/bin. > > But that's not yet a working domain image. > > # "Baking" QEMU into a unikernel > > $ git clone https://xenbits.xen.org/people/liuw/qemu-rump-config.git qemu- rump-config > $ cd rump-build-junk-script > $ ./script-bake > > Then the final binary will be in > /local/scratch/Rump-kernels/prefix-qemu/bin/qemu-system-i386-rump. > > # Test the unikernel > > Note that I omit path prefixes in following commands. > > $ scp qemu-system-i386-rump testbox:~ > $ scp app-tools/rumprun testbox:~ > > Then on testbox. > > $ ./rumprun -S xen -i qemu-system-i386-rump --help > > This rune should start the guest, attach to its console and show you > the help information from QEMU. > > To add network interface, see ./rumprun -h. > > > Wei.
_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
