Chen Yufei wrote: > On 2010-7-22, at 上午1:04, Stefan Weil wrote: > >> Am 21.07.2010 09:03, schrieb Chen Yufei: >>> On 2010-7-21, at 上午5:43, Blue Swirl wrote: >>> >>> >>>> On Sat, Jul 17, 2010 at 10:27 AM, Chen Yufei<cyfde...@gmail.com> wrote: >>>> >>>>> We are pleased to announce COREMU, which is a "multicore-on-multicore" >>>>> full-system emulator built on Qemu. (Simply speaking, we made Qemu >>>>> parallel.) >>>>> >>>>> The project web page is located at: >>>>> http://ppi.fudan.edu.cn/coremu >>>>> >>>>> You can also download the source code, images for playing on sourceforge >>>>> http://sf.net/p/coremu >>>>> >>>>> COREMU is composed of >>>>> 1. a parallel emulation library >>>>> 2. a set of patches to qemu >>>>> (We worked on the master branch, commit >>>>> 54d7cf136f040713095cbc064f62d753bff6f9d2) >>>>> >>>>> It currently supports full-system emulation of x64 and ARM MPcore >>>>> platforms. >>>>> >>>>> By leveraging the underlying multicore resources, it can emulate up to >>>>> 255 cores running commodity operating systems (even on a 4-core machine). >>>>> >>>>> Enjoy, >>>>> >>>> Nice work. Do you plan to submit the improvements back to upstream QEMU? >>>> >>> It would be great if we can submit our code to QEMU, but we do not know the >>> process. >>> Would you please give us some instructions? >>> >>> -- >>> Best regards, >>> Chen Yufei >>> >> Some hints can be found here: >> http://wiki.qemu.org/Contribute/StartHere >> >> Kind regards, >> Stefan Weil > > The patch is in the attachment, produced with command > git diff 54d7cf136f040713095cbc064f62d753bff6f9d2 > > In order to separate what need to be done to make QEMU parallel, we created a > separate library, and the patched QEMU need to be compiled and linked with > that library. To submit our enhancement to QEMU, maybe we need to incorporate > this library into QEMU. I don't know what would be the best solution.
For upstream QEMU, the goal should be to integrate your modifications and enhancements into the existing architecture in a mostly seamless way. The library approach may help maintaining your changes out of tree, but it likely cannot contribute any benefit to an in-tree extension of QEMU for parallel TCG VCPUs. > > Our approach to make QEMU parallel can be found at > http://ppi.fudan.edu.cn/coremu > > I will give a short summary here: > > 1. Each emulated core thread runs a separate binary translator engine and has > private code cache. We marked some variables in TCG as thread local. We also > modified the TB invalidation mechanism. > > 2. Each core has a queue holding pending interrupts. The COREMU library > provides this queue, and interrupt notification is done by sending realtime > signals to the emulated core thread. > > 3. Atomic instruction emulation has to be modified for parallel emulation. We > use lightweight memory transaction which requires only compare-and-swap > instruction to emulate atomic instruction. > > 4. Some code in the original QEMU may cause data race bug after we make it > parallel. We fixed these problems. > Upstream integration requires such iterative steps as well - in form of ideally small, focused patches that finally convert QEMU into a parallel emulator. Also note that upstream already supports threaded VCPUs - in KVM mode. You obviously have resolved the major blocking points to apply this on TCG mode as well. But I don't see yet why we may need a new VCPU threading infrastructure for this. Rather only small tuning of what KVM already uses should suffice - if that's required at all. To give it a start, you could identify some more trivial changes in your patches, split them out and rebase them over latest qemu.git, then post them as a patch series for inclusion (see the mailing list for various examples). Make sure to describe the reason for your changes as clear as possible, specifically if they are not (yet) obvious in the absence of COREMU features in upstream QEMU. Be prepared that merging your code can be a lengthy process with quite a few discussions about why and how things are done, likely also with requests to change your current solution in some aspects. However, the result should be an optimal solution for the overall goal, parallel VCPU emulation - and no longer any need to maintain your private set of patches against quickly evolving QEMU. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
