Markus, Thank you for your comments! Based on your questions and suggestions of writing a more complete explanation in my commits, I decided to start to describe our whole work on the wiki: https://wiki.qemu.org/Internships/ProjectIdeas/TCGCodeQuality I will update and expand it weekly, so I can link and cite it in future patches to give a better whole vision for anyone interested.
Btw, thank you for your QMP tips, I will discuss with Alex how to address it in our approach. []'s Vanderson M. Rosario On Sat, Jul 6, 2019 at 3:09 AM Markus Armbruster <arm...@redhat.com> wrote: > Cc: Marc-André, who has patches that might be useful here. > > Alex Bennée <alex.ben...@linaro.org> writes: > > > Markus Armbruster <arm...@redhat.com> writes: > > > >> vandersonmr <vanderson...@gmail.com> writes: > >> > > <snip> > > > > I'll leave Vanderson to address your other comments. > > > >> > >> Debugging commands are kind of borderline. Debugging is commonly a > >> human activity, where HMP is just fine. However, humans create tools to > >> assist with their activities, and then QMP is useful. While I wouldn't > >> encourage HMP-only for the debugging use case, I wouldn't veto it. > >> > >> Your (overly terse!) commit message and help texts make me guess the > >> commands are for gathering statistics. Statistics can have debugging > >> uses. But they often have non-debugging uses as well. What use cases > >> can you imagine for these commands? > > > > So this is all really aimed at making TCG go faster - but before we can > > make it go faster we need better tools for seeing where the time is > > being spent and examining the code that we generate. So I expect the > > main users of this functionality will be QEMU developers. > > > > That said I can see a good rationale for supporting QMP because it is > > more amenable to automation. However this is early days so I would > > caution about exposing this stuff too early least we bake in a woolly > > API. > > Development tools should exempt themselves from QMP's interface > stability promise: prefix the command names with 'x-'. > > > The other wrinkle is we do have to take control of the emulator to > > safely calculate some of the numbers we output. This essentially means > > the HMP commands are asynchronous - we kick of safe work which waits > > until all vCPU threads are stopped before we go through the records and > > add up numbers. This is fine for the HMP because we just output to the > > monitor FD when we are ready. I assume for QMP commands there is more > > housekeeping to do? Can QMP commands wait for a response to be > > calculated by another thread? Are there any existing commands that have > > to support this sort of pattern? > > Let me clarify "synchronous" to avoid confusion. > > Both QMP and HMP commands are synchronous protocols in the sense that > commands are executed one after the other, without overlap. When a > client sends multiple commands, it can assume that each one starts only > after the previous one completed. > > Both HMP and QMP commands execute synchronously in the sense that the > command runs to completion without ever yielding the thread. Any > blocking operations put the thread to sleep (but see below). > > HMP runs in the main thread. Putting the main thread to sleep is > generally undesirable. > > QMP used to run in the main thread, too. Nowadays, the QMP core runs in > an I/O thread shared by all monitors, and dispatches commands to the > main thread. Moving command execution out of the main thread as well > requires careful review of the command's code for hidden assumptions. > Major project. > > Fine print: OOB commands are a special case, but I doubt you want to > know more. > > Fine print: certain character devices can't support use of an I/O > thread; QMP runs in the main thread then. The ones you want to use with > QMP all support I/O threads. > > You wrote "we kick of safe work which waits until all vCPU threads are > stopped before we go through the records and add up numbers [...] we > just output to the monitor FD". Does this mean the HMP command kicks > off the work, terminates, and some time later something else prints > results to the monitor? How much later? > > If "later" is actually "soon", for a suitable value of "soon", > Marc-André's work on "asynchronous" QMP might be pertinent. I put > "asynchronous" in scare quotes, because of the confusion it has caused. > My current understanding (Marc-André, please correct me if wrong): it > lets QMP commands to block without putting their thread to sleep. It > does not make QMP an asynchronous protocol. > > If "later" need not be "soon", read on. > > In QMP, there are two established ways to do potentially long-running > work. Both ways use a command that kicks off the work, then terminates > without waiting for it to complete. > > The first way is traditional: pair the kick off command with a query > command and optionally an event. > > When the work completes, it fires off the event. The event is broadcast > to all QMP monitors (we could implement unicast if we have a compelling > use case). > > The query command reports whether the work has completed, and if yes, > the work's results, if any. > > You need the event if you want to avoid polling. > > Even with an event, you still need a query command. If your management > application loses its QMP connection temporarily, you can miss the > event. You want to poll on reconnect, with the query command. > > If more than one instance of the work can be pending at any one time, > event and query need to identify the instance somehow. This is > completely ad hoc. > > The second way is a full-blown "job". This provides more control: you > can cancel, pause, resume, ... It also provides a job ID. More > featureful and more structured. > > Jobs have grown out of block jobs. I'd love to see some uses outside > the block subsystem. > > Hope this helps! >
