On 1/11/21 6:47 AM, Stefan Hajnoczi wrote:
Dear QEMU, KVM, and rust-vmm community,
QEMU will apply for Google Summer of Code
(https://summerofcode.withgoogle.com/) again this year. This internship
program offers paid, 10-week, remote work internships for
contributing to open source. QEMU can act as an umbrella organization
for KVM kernel and rust-vmm projects too.
Please post project ideas on the QEMU wiki before February 14th:
https://wiki.qemu.org/Google_Summer_of_Code_2021
What's new this year:
* The number of internship hours has been halved to 175 hours over
10 weeks. Project ideas must be smaller to fit and students will have
more flexibility with their working hours.
* Eligibility has been expanded to include "licensed coding school or
similar type of program".
Good project ideas are suitable for 175 hours (10 weeks half-day) work by a
competent programmer who is not yet familiar with the codebase. In
addition, they are:
* Well-defined - the scope is clear
* Self-contained - there are few dependencies
* Uncontroversial - they are acceptable to the community
* Incremental - they produce deliverables along the way
Feel free to post ideas even if you are unable to mentor the project.
It doesn't hurt to share the idea!
I have one that is probably way too ambitious, but requires a particular
skillset that might be of good interest to a student that has some
experience in the area already.
The idea is for a TUI qmp-shell (maybe using urwid?) to create an
irssi-like REPL interface for QMP. The idea would be to mimic the
mitmproxy TUI interface (Check it out if you haven't!)
All the ideas below are extremely tentative to give a basic gist of what
I mean; exact layouts/hotkeys/etc are for the sake of explanation only.
Essentially, I want an interface like this:
-----------------------------------------------------------
| QMP Mode |
|=========================================================|
| |
| Welcome to the Qemu Machine Protocol shell. |
| Please type /help or Ctrl+H to see available commands. |
| |
| |
| |
|---------------------------------------------------------|
| > |
-----------------------------------------------------------
commands are entered in the bottom and appear in a log window above,
appearing most-recent last, like irssi works.
As an example, let's say we issue block-dirty-bitmap-add:
--------------------------------------------------------
| > block-dirty-bitmap-add node=ide0hd0 name=mybitmap0 |
--------------------------------------------------------
(...syntax up for debate...! We want something easy to parse, but easy
to type. For structured data, that's Hard.)
we press [Enter] to submit the command, and the history now shows:
-------------------------------------------------
| QMP Mode |
|===============================================|
| |
| |
| |
| [11:28] block-dirty-bitmap-add [Pending ...] |
|-----------------------------------------------|
| > |
-------------------------------------------------
After a few moments, the command resolves:
----------------------------------------
| QMP Mode |
|======================================|
| |
| |
| |
| [11:28] block-dirty-bitmap-add [OK] |
----------------------------------------
| > |
----------------------------------------
The basic unit of history here is either an RPC call/response pair or an
asynchronous QMP event.
Clicking on the history pane or pressing Alt+↑ or Alt+↓ (Mimics irssi
keys for switching buffers, let's say it's user-configurable) to change
focus to the history pane:
------------------------------------------------------------
| Command History Mode |
|==========================================================|
| |
| |
| |
| [11:28] block-dirty-bitmap-add [OK] |
-----------------------------------------------------------|
| Use ↑ or ↓ to select a command, press [Enter] to inspect |
------------------------------------------------------------
Using the arrow keys, you can highlight the history item and then press
<Enter> to change the log pane to a details pane for that item, showing:
------------------------------------------------------
| Details: block-dirty-bitmap-add [X] |
|====================================================|
| |
| [11:28:01] -> { |
| "execute": "block-dirty-bitmap-add", |
| "arguments": { |
| "node": "ide0hd0", |
| "name": "mybitmap0" |
| } |
| } |
| |
| [11:28:02] <- { |
| "return": {} |
| } |
| |
|----------------------------------------------------|
| Press 'Q' or Backspace to close details pane |
------------------------------------------------------
From here, maybe you could do a few things, like press a button to view
the raw QMP I/O for the purposes of copy-pasting elsewhere, or just
press Q/H/Backspace to go back to the history view.
Interesting abilities this style of shell can provide:
(1) The ability to render asynchronous events from the server in the log
window without overwriting the user's command buffer. (In contrast to
qmp-shell today, which cannot show async events until the user presses
enter again to prompt a check for them.)
(2) The ability to asynchronously reconnect to servers on connection
failures.
(3) The ability to collapse command:response pairs into one summarized
line in the log, which can be expanded for more information later with a
keypress.
(4) (As a future add-on, beyond current scope) The ability to add a
status pane widget that might show current running jobs, the current
machine execution status, etc.
As options, users should be able to:
1. Choose the history display mode live at runtime:
a) Summarized command history (as shown above)
b) Summarized JSON history
(one item per message, no grouping, purely linear,
shows a one-line summary of the message.)
c) Full JSON history
(one item per message, JSON messages are rendered in full.)
d) Pretty JSON history
(one item per message, JSON messages are pretty-printed.)
e.g. Ctrl+V could change the "verbosity" of the messages, and the
history log will be re-rendered to reflect the new setting.
2. Customize hotkeys for switching between command mode and history
mode; quitting the details view, etc.
3. toggle on/off automatic command execution logging; or manually
(Ctrl+S?) save history to file. The format might need some light markup
to indicate directionality, timestamps, and so on.
Other things that might be of interest, but are beyond the scope of a
GSoC/Outreachy internship:
1. readline-esque hotkey functionality for the command input bar,
including command history, autocomplete for command names, etc. If we
aren't using readline itself anymore, these things will have to be
implemented somehow. There are libraries that support this:
https://github.com/rr-/urwid_readline
2. Recording execution macros; subsets of commands you wish to replay
easily.
3. Raw input mode: allow the user to paste a raw JSON command directly
into the command bar.
4. Configuring our GTK interface to open up qmp-shell inside one of the
terminal panes. (Kevin Wolf suggested this or something like it once,
and I rather like the idea.)
5. "Shell Builtins" -- analogous to the same in bash -- additional
commands and routines that help perform certain actions that might
require complex, multi-part steps.
6. Augmented commands -- commands that don't require multi-part phases,
but ones in which the shell might offer enhanced readouts for. An
example would be integrating Vladimir's block graph rendering script to
automatically fire off when block-query is issued. (If the user has a
graphical session running, the libraries are installed, etc.)
See alsos; prior art, &c:
- irssi as an example of continuous, asynchronous history:
https://irssi.org/
- mitmproxy, which heavily inspired this idea with its use of urwid to
show a history of "packets" which can be interrogated later from within
the interface
https://mitmproxy.org/
- aioconsole -- an async python REPL loop for interactively writing
async code in python. It might have good ideas to steal!
https://pypi.org/project/aioconsole/
- urwid: a TUI framework for Python; it is used by mitmproxy to
implement its interface.
http://urwid.org/
- urwid readline library: implements readline-esque hotkeys for urwid,
but (maybe) doesn't respect readline config, if any. It is a
reimplementation.
https://github.com/rr-/urwid_readline
Target skillset for students:
- Intermediate Python
- Some exposure to async programming, coroutines, etc.
- Some experience with GUI programming concepts, in any toolkit or language
- A willingness to use gradually typed python O:-)
I don't expect this project to involve much, if any, C; but familiarity
with the linux shell ecosystem will be an asset when choosing default
behaviors, mimicking other tools, etc.
I have already written a fully asyncio QMP library; I would expect this
new library to be used. It should be fully operational, but might
require API changes when we begin to use it for a serious endeavor.
I will review project ideas and keep you up-to-date on QEMU's
acceptance into GSoC.
For more background on QEMU internships, check out this video:
https://www.youtube.com/watch?v=xNVCX7YMUL8
Stefan
