ср, 17 нояб. 2021 г. в 09:10, Jason Wang <jasow...@redhat.com>:
> On Tue, Nov 16, 2021 at 11:30 PM Vladislav Yaroshchuk > <yaroshchuk2...@gmail.com> wrote: > > > > > > > > вт, 16 нояб. 2021 г. в 10:23, Jason Wang <jasow...@redhat.com>: > >> > >> On Mon, Nov 15, 2021 at 6:45 PM Vladislav Yaroshchuk > >> <yaroshchuk2...@gmail.com> wrote: > >> > > >> > > >> > > >> > пн, 15 нояб. 2021 г. в 07:53, Jason Wang <jasow...@redhat.com>: > >> >> > >> >> On Fri, Nov 12, 2021 at 5:14 PM Vladislav Yaroshchuk > >> >> <yaroshchuk2...@gmail.com> wrote: > >> >> > > >> >> > macOS provides networking API for VMs called 'vmnet.framework': > >> >> > https://developer.apple.com/documentation/vmnet > >> >> > > >> >> > We can provide its support as the new QEMU network backends which > >> >> > represent three different vmnet.framework interface usage modes: > >> >> > > >> >> > * `vmnet-shared`: > >> >> > allows the guest to communicate with other guests in shared > mode and > >> >> > also with external network (Internet) via NAT. Has > (macOS-provided) > >> >> > DHCP server; subnet mask and IP range can be configured; > >> >> > > >> >> > * `vmnet-host`: > >> >> > allows the guest to communicate with other guests in host mode. > >> >> > By default has enabled DHCP as `vmnet-shared`, but providing > >> >> > network unique id (uuid) can make `vmnet-host` interfaces > isolated > >> >> > from each other and also disables DHCP. > >> >> > > >> >> > * `vmnet-bridged`: > >> >> > bridges the guest with a physical network interface. > >> >> > > >> >> > This backends cannot work on macOS Catalina 10.15 cause we use > >> >> > vmnet.framework API provided only with macOS 11 and newer. Seems > >> >> > that it is not a problem, because QEMU guarantees to work on two > most > >> >> > recent versions of macOS which now are Big Sur (11) and Monterey > (12). > >> >> > > >> >> > Also, we have one inconvenient restriction: vmnet.framework > interfaces > >> >> > can create only privileged user: > >> >> > `$ sudo qemu-system-x86_64 -nic vmnet-shared` > >> >> > > >> >> > Attempt of `vmnet-*` netdev creation being unprivileged user fails > with > >> >> > vmnet's 'general failure'. > >> >> > > >> >> > This happens because vmnet.framework requires > `com.apple.vm.networking` > >> >> > entitlement which is: "restricted to developers of virtualization > software. > >> >> > To request this entitlement, contact your Apple representative." > as Apple > >> >> > documentation says: > >> >> > > https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking > >> >> > >> >> Do you know how multipass work? Looks like it uses vmnet without > privileges. > >> >> > >> > > >> > I've just checked this, and they still need root privileges. They > have a > >> > `multipassd` daemon which is launched as root by launchd by default. > >> > > >> > ``` > >> > bash-5.1$ ps axo ruser,ruid,comm | grep multipass > >> > root 0 /Library/Application > Support/com.canonical.multipass/bin/multipassd > >> > root 0 /Library/Application > Support/com.canonical.multipass/bin/hyperkit > >> > ``` > >> > > >> > That's the reason why it's required to 'enter a password' while > multipass installation: > >> > it creates launchd plist (kinda launch rule) and places it to > /Library/LaunchDaemons/, > >> > which can be done only with root privileges. > >> > > >> > ``` > >> > bash-5.1$ ll /Library/LaunchDaemons | grep multipass > >> > -rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47 > com.canonical.multipassd.plist > >> > ``` > >> > > >> > And after this launchd launches this multipassd daemon as root every > boot. > >> > > >> > So an unprivileged user can launch a multipass VM instance, but > actually the > >> > `hyperkit` process which interacts with vmnet is gonna be launched by > multipassd > >> > running as root. > >> > >> I wonder how it passes the vmnet object to qemu? Nothing obvious from > >> the qemu command line that multipass launched: > >> > >> -nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4 > >> > >> (But I haven't had time to check their qemu codes). > >> > > > > It seems they just use QEMU with patch by Phillip Tennen: > > https://patchew.org/QEMU/20210218134947.1860-1-phillip.en...@gmail.com/ > > > > In that patch he does quite the same as we in this series, the > > difference remains in foreground: he creates one new 'vmnet-macos' > > netdev, and uses 'mode=shared' property to choose vmnet > > operating mode. I decided to create three different netdevs instead > > (vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some > > features related to isolation and ipv6. > > Ok. > > > > > > I wonder how it passes the vmnet object to qemu > > I hope I clearly described this. > > A silly question, how did the 'hyperkit' process pass the vmnet object to > qemu? > > I think I didn't describe it very well in the previous mail. The 'hyperkit' and QEMU are two multipass 'drivers'. hyperkit is an independent hypervisor based on xhyve (bhyve): https://github.com/moby/hyperkit So there are many ways to launch multipass VMs: * VM <-> hyperkit <-> host * VM <-> QEMU <-> host * VM <-> virtualbox <-> host * etc. https://discourse.ubuntu.com/t/announcing-the-first-release-candidate-for-apple-m1-support/24445 So hyperkit doesn't pass anything to QEMU, it's a separate hypervisor and multipass driver. But it uses vmnet backend too, the same way as we do: https://github.com/moby/hyperkit/blob/2f061e447e1435cdf1b9eda364cea6414f2c606b/src/lib/pci_virtio_net_vmnet.c#L250 > > >> > > >> > tl;dr sadly, we can't interact with vmnet.framework without having > our binary correctly > >> > signed and being an unprivileged user. Root privileges or special > signature with > >> > entitlement is required. > >> > >> This is something similar to what happens in other OS. E.g for the tap > >> backend, it can't be created without privileges. So qemu allows to: > >> > >> 1) the TAP to be created by privileged program like libvirt, and its > >> fd could be passed to qemu via SCM_RIGHTS > >> 2) run a set-uid helper to create and config TAP > >> > >> This is something we need to consider now or in the future probably. > >> > > > > Seems we can do nothing with this if we have qemu-bundled & > > direct vmnet.framework interaction, it always requires privileges > > or entitlement. > > The workaround can be moving vmnet-related things to > > another helper binary running with privileges, and usage of > > this helper somewhere between qemu and vmnet. > > Yes, that's my point. > > > > > I think for now it's applicable to leave it as is, having qemu > > that requires privileges for vmnet.framework usage. > > Right, but we need to consider it for the future. > > Btw, if you wish, you can list yourself as the maintainer for this backend. > > Ok, thank you! Let me do this in the next patch version, after we finish our discussion and I fix all the issues you pointed in your review. > Thanks > > > > >> > >> Thanks > >> > >> > > >> > > >> >> Thanks > >> >> > >> > > >> > Thank you for your review, I will check it this week and reply as > soon as possible. > >> > > >> >> > >> >> > > >> >> > One more note: we still have quite useful but not supported > >> >> > 'vmnet.framework' features as creating port forwarding rules, IPv6 > >> >> > NAT prefix specifying and so on. > >> >> > > >> >> > Nevertheless, new backends work fine and tested within > `qemu-system-x86-64` > >> >> > on macOS Bir Sur 11.5.2 host with such nic models: > >> >> > * e1000-82545em > >> >> > * virtio-net-pci > >> >> > * vmxnet3 > >> >> > > >> >> > The guests were: > >> >> > * macOS 10.15.7 > >> >> > * Ubuntu Bionic (server cloudimg) > >> >> > > >> >> > > >> >> > This series partially reuses patches by Phillip Tennen: > >> >> > > https://patchew.org/QEMU/20210218134947.1860-1-phillip.en...@gmail.com/ > >> >> > So I included his signed-off line into one of the commit messages > and > >> >> > also here. > >> >> > > >> >> > v1 -> v2: > >> >> > Since v1 minor typos were fixed, patches rebased onto latest > master, > >> >> > redundant changes removed (small commits squashed) > >> >> > > >> >> > v2 -> v3: > >> >> > - QAPI style fixes > >> >> > - Typos fixes in comments > >> >> > - `#include`'s updated to be in sync with recent master > >> >> > v3 -> v4: > >> >> > - Support vmnet interfaces isolation feature > >> >> > - Support vmnet-host network uuid setting feature > >> >> > - Refactored sources a bit > >> >> > v4 -> v5: > >> >> > - Missed 6.2 boat, now 7.0 candidate > >> >> > - Fix qapi netdev descriptions and styles > >> >> > (@subnetmask -> @subnet-mask) > >> >> > - Support vmnet-shared IPv6 prefix setting feature > >> >> > > >> >> > Signed-off-by: Phillip Tennen <phil...@axleos.com> > >> >> > Signed-off-by: Vladislav Yaroshchuk <yaroshchuk2...@gmail.com> > >> >> > > >> >> > Vladislav Yaroshchuk (6): > >> >> > net/vmnet: add vmnet dependency and customizable option > >> >> > net/vmnet: add vmnet backends to qapi/net > >> >> > net/vmnet: implement shared mode (vmnet-shared) > >> >> > net/vmnet: implement host mode (vmnet-host) > >> >> > net/vmnet: implement bridged mode (vmnet-bridged) > >> >> > net/vmnet: update qemu-options.hx > >> >> > > >> >> > meson.build | 4 + > >> >> > meson_options.txt | 2 + > >> >> > net/clients.h | 11 ++ > >> >> > net/meson.build | 7 + > >> >> > net/net.c | 10 ++ > >> >> > net/vmnet-bridged.m | 111 ++++++++++++ > >> >> > net/vmnet-common.m | 325 > ++++++++++++++++++++++++++++++++++ > >> >> > net/vmnet-host.c | 111 ++++++++++++ > >> >> > net/vmnet-shared.c | 92 ++++++++++ > >> >> > net/vmnet_int.h | 48 +++++ > >> >> > qapi/net.json | 127 ++++++++++++- > >> >> > qemu-options.hx | 25 +++ > >> >> > scripts/meson-buildoptions.sh | 3 + > >> >> > 13 files changed, 874 insertions(+), 2 deletions(-) > >> >> > create mode 100644 net/vmnet-bridged.m > >> >> > create mode 100644 net/vmnet-common.m > >> >> > create mode 100644 net/vmnet-host.c > >> >> > create mode 100644 net/vmnet-shared.c > >> >> > create mode 100644 net/vmnet_int.h > >> >> > > >> >> > -- > >> >> > 2.23.0 > >> >> > > >> >> > >> > > >> > > >> > -- > >> > Best Regards, > >> > > >> > Vladislav Yaroshchuk > >> > > > > > > -- > > Best Regards, > > > > Vladislav Yaroshchuk > >
