I am working on an OpenBSD-based QubesOS TemplateVM, and have run into a few problems.
In QubesOS, all volumes of a TemplateVM are persistent. AppVMs based on a TemplateVM use a (copy of) the TemplateVM’s root partition, but have their own private partition, which is set to zero when the VM first boots up. Finally, DispVMs have no persistent storage at all. This leads to a few difficulties. First, I found that I need to mount various directories from the persistent volume (which I mounted at /mnt/rw) over the root volumes (mounted at / and /usr in my case). Linux AppVMs use bind mounts for that purpose, but OpenBSD doesn’t have bind mounts, so I use NFS over loopback. Since that is slow, I symlinked /home to /mnt/rw/home. Furthermore, when an AppVM first starts up, /mnt/rw will not be mountable, as the underlying storage will consist entirely of zeroes. I wrote a C program that uses disklabel(5) ioctls to handle this. However, this breaks sysupgrade(8), since /home is symlinked to /mnt/rw/home, which doesn’t exist until /etc/rc.securelevel runs. I fixed this by monkeypatching sysupgrade(8) to download into /var/_sysupgrade instead, but that is an ugly hack. The other problem is that I do not know of a reliable way to make the boot process fail if my /etc/rc.securelevel fails. This is security-critical, as it does tasks like deleting SSH keys from /etc/ssh so that they will be regenerated after /mnt/rw/export/etc/ssh has been mounted over /etc/ssh. If this fails, there is a chance that two AppVMs could share the same SSH host keys, which would be bad. Furthermore, if mounting /mnt/rw fails, I would much prefer to drop into a single-user shell than to try to continue. This is a trivial patch to /etc/rc, but patching /etc/rc is strongly discouraged in afterboot(8). Are the solutions I came up with the best possible, or are there better ones that I missed? Any help will be greatly appreciated. Sincerely, Demi
signature.asc
Description: OpenPGP digital signature
