Re: "relaxed" `-hda file.qcow2` equivalent ?
Le ven. 15 nov. 2024 à 15:21, Frantisek Rysanek a écrit : > > And, I have an idea: rather than refer to driver=qcow2 and > > file.filename, how about referring to the loopback device (NBD) that > > you already have, courtesy of qemu-nbd ? Would that perhaps circumvent > > the file lock? ;-) > > > > -blockdev node-name=xy,driver=raw,file.driver=host_device,\ > > file.filename=/dev/loop0,file.locking=off > > > > -device virtio-scsi-pci -device scsi-hd,drive=xy > > > > I mean: the QEMU device emulation would not run on top of the QCOW > file directly (and the underlying filesystem, and its locking > feature), but would instead share a block device with your host-side > mount. Thus, it would also plug directly into any block-level > buffering going on, on the host side. > > On the guest, I'm wondering if you should mount the partition with > -o direct. This should prevent any write-back buffering in the guest, > which however you will not be doing, as you say. > On the other hand, if you make changes to the FS on the host side, > while the QEMU guest instance is already running, the guest probably > will not get to know about any changes, probably unless you umount > and remount, that with "-o direct" (to avoid local read caching in > the guest). > > Even if this crazy stuff works in the end, I'm wondering if it's all > worth the implied pitfalls :-) > Apparently you still need to keep stuff in sync in some way... > > Frank > > after reading page 17 @ https://vmsplice.net/~stefan/qemu-block-layer-features-and-concepts.pdf, I'm almost there with : qemu -snapshot \ -blockdev driver=file,node-name=file-driver,filename=file.qcow2,locking=off \ -blockdev driver=qcow2,node-name=qcow-driver,file=file-driver \ -device ide-hd,drive=qcow-driver \ -hdb file2.qcow2 the difference lies in the fact that it's not `hda` but `hdc` : on the guest side, the disk appears second after the one passed by `hdb`
Re: "relaxed" `-hda file.qcow2` equivalent ?
Le ven. 15 nov. 2024 à 16:11, lacsaP Patatetom a écrit : > Le ven. 15 nov. 2024 à 15:21, Frantisek Rysanek > a écrit : > >> > And, I have an idea: rather than refer to driver=qcow2 and >> > file.filename, how about referring to the loopback device (NBD) that >> > you already have, courtesy of qemu-nbd ? Would that perhaps circumvent >> > the file lock? ;-) >> > >> > -blockdev node-name=xy,driver=raw,file.driver=host_device,\ >> > file.filename=/dev/loop0,file.locking=off >> > >> > -device virtio-scsi-pci -device scsi-hd,drive=xy >> > >> >> I mean: the QEMU device emulation would not run on top of the QCOW >> file directly (and the underlying filesystem, and its locking >> feature), but would instead share a block device with your host-side >> mount. Thus, it would also plug directly into any block-level >> buffering going on, on the host side. >> >> On the guest, I'm wondering if you should mount the partition with >> -o direct. This should prevent any write-back buffering in the guest, >> which however you will not be doing, as you say. >> On the other hand, if you make changes to the FS on the host side, >> while the QEMU guest instance is already running, the guest probably >> will not get to know about any changes, probably unless you umount >> and remount, that with "-o direct" (to avoid local read caching in >> the guest). >> >> Even if this crazy stuff works in the end, I'm wondering if it's all >> worth the implied pitfalls :-) >> Apparently you still need to keep stuff in sync in some way... >> >> Frank >> >> > after reading page 17 @ > https://vmsplice.net/~stefan/qemu-block-layer-features-and-concepts.pdf, > I'm almost there with : > > qemu -snapshot \ > -blockdev > driver=file,node-name=file-driver,filename=file.qcow2,locking=off \ > -blockdev driver=qcow2,node-name=qcow-driver,file=file-driver \ > -device ide-hd,drive=qcow-driver \ > -hdb file2.qcow2 > > the difference lies in the fact that it's not `hda` but `hdc` : on the > guest side, the disk appears second after the one passed by `hdb` > adding `,bus=ide.0,unit=0` to `-device ide-hd,drive=qcow-driver` do the job :-) "relaxed" `-hda file.qcow2` is (seems to be) `-blockdev driver=file,node-name=file-driver,filename=file.qcow2,locking=off -blockdev driver=qcow2,node-name=qcow-driver,file=file-driver -device ide-hd,drive=qcow-driver,bus=ide.0,unit=0` ;-)
Re: "relaxed" `-hda file.qcow2` equivalent ?
> adding `,bus=ide.0,unit=0` to `-device ide-hd,drive=qcow-driver` do > the job :-) > > "relaxed" `-hda file.qcow2` is (seems to be) `-blockdev > driver=file,node-name=file-driver,filename=file.qcow2,locking=off > -blockdev driver=qcow2,node-name=qcow-driver,file=file-driver -device > ide-hd,drive=qcow-driver,bus=ide.0,unit=0` ;-) :-) thanks for sharing that detailed recipe. I actually find the second row a little convoluted, but I guess that cross-eyed expression on my face will fade away later tonight. You have hereby enriched the official documentation, and the world at large. Even if your antic eventually gets proven fraught by some unforseen FS consistency issues, you have shown us the way. Thanks for educating me :-) Frank
Re: PASST networking with onboard NIC
On 14/11/2024 20.09, Lytle, Ben wrote: >> I have a question about the proper way to use passt networking with a machine's onboard >> NIC (ie. without specifying a |-device|). >> Is there a way to connect |-netdev stream,id=net0,server=off,addr.type=unix,addr.path=/ >> tmp/passt_1.socket| to an onboard nic using |-nic| ? > > No, I don't think the -nic option is able to parse the stream backend parameters (like > "addr.type", "addr.path", ...). I figured that was the case, as I've tried many combinations for those options. I appreciate the confirmation! I think you might be able to work-around the problem by using the legacy "-net nic" option: -netdev stream,id=net0,server=off,addr.type=unix,addr.path=/tmp/passt_1.socket -net nic,netdev=net0 HTH, Thomas
Re: "relaxed" `-hda file.qcow2` equivalent ?
Le ven. 15 nov. 2024 à 13:35, Frantisek Rysanek a écrit : > > # qemu-nbd -c /dev/nbd0 file.qcow2 > > # mount /dev/nbd0p1 /mnt -o uid=me > > $ # do some changes in /mnt/... > > > Are you sure the changes have made it to the underlying file? > If you do an umount here, a sync() is guaranteed. > If you do not umount, at this point you may have some dirty > writeback. > So that the following QEMU instance may find the filesystem in a > "factually out of date" state. I dare not say "inconsistent state", > because this should theoretically be taken care of... maybe... > > I manually flush the system caches to make sure my `file.qcow2` file is up to date . > $ qemu-system-x86_64 -snapshot -hda file.qcow2 # with locking=off > > Read-only access is one thing. Maybe the QEMU instance will cope with > that. > > Do you want the QEMU instance to *write* to the filesystem, by any > chance? > no, I don't need it. `-snapshot` is there to prevent corruption of the assembly left in place. > If you manage to force-mount the filesystem for writing, and you > write some changes to it, how does your "outer host-side instance of > the mounted FS" get to know? > This is a recipe for filesystem breakage :-) > > You know, these are exactly the sort of problems, that get avoided by > the locking :-) > > They also get avoided by "shared access filesystems" such as GFS2 or > OCFS2, or network filesystems such as NFS or CIFS=SMBFS. Maybe CEPH > is also remotely in this vein, although this is more of a distributed > clustered FS, and an overkill for your scenario. > > Frank > > in any case, to avoid this kind of problem and make sure I have the right data, I manually clear the system caches with `sudo bash -c “sync && sysctl -q vm.drop_caches=3”` before calling qemu. `-snapshot` is there to prevent corruption of the assembly left in place. regards, lacsaP.
Re: "relaxed" `-hda file.qcow2` equivalent ?
Le jeu. 14 nov. 2024 à 17:00, Frantisek Rysanek a écrit : > > my image `fie.qcow2` is mounted elsewhere (nbd) and running > > `qemu-system-x86_64 -snapshot -hda file.qcow2` fails with the error > > "qemu-system-x86_64: -hda usb1.disk: Failed to get shared “write” > > lock / Is another process using the image [usb1.disk] ?". > > Oh... > > So you want concurrent access to the block device. > 1) have it mounted on the host=HV using qemu-nbd > 2) while at the same time, you want to give it to a guest VM? > > Unless you use a filesystem such as GFS2 or OCFS2 inside the > partitions that exist inside that QCOW2 black box, or mount it > read-only at both ends, you are asking for trouble :-) > That is, if QEMU permits unlocked access by a VM, parallel to the > qemu-nbd loop-mount. > > Note that this is just a tangential pointer on my part. Just a start > of a further journey. Further reading and study. > > If you really want concurrent access to a directory tree, consider > using something like Samba or NFS as the simpler options. > > Then again, concurrent access is definitely fun to play with and grow > :-) > > Frank > > my problem is exactly this, the link between `device`, `drive` and `blockdev` : while `device` and `drive` link quite easily, I can't link `blockdev` to the other two... setting aside the fact that the image is used elsewhere, my aim is to mimic `-hda file.qcow2` but without taking the lock into account (`locking=off`). I currently do this: ``` # qemu-nbd -c /dev/nbd0 file.qcow2 # mount /dev/nbd0p1 /mnt -o uid=me $ # do some changes in /mnt/... # umount /mnt # qemu-nbd -d /dev/nbd0 $ qemu-system-x86_64 -snapshot -hda file.qcow2 # qemu-nbd -c /dev/nbd0 file.qcow2 # mount /dev/nbd0p1 /mnt -o uid=me $ # do some changes in /mnt/... # umount /mnt # qemu-nbd -d /dev/nbd0 $ qemu-system-x86_64 -snapshot -hda file.qcow2 # qemu-nbd -c /dev/nbd0 file.qcow2 # mount /dev/nbd0p1 /mnt -o uid=me $ # do some changes in /mnt/... etc... ``` and I'd like to be able to do this : ``` # qemu-nbd -c /dev/nbd0 file.qcow2 # mount /dev/nbd0p1 /mnt -o uid=me $ # do some changes in /mnt/... $ qemu-system-x86_64 -snapshot -hda file.qcow2 # with locking=off $ # do some changes in /mnt/... $ qemu-system-x86_64 -snapshot -hda file.qcow2 # with locking=off $ # do some changes in /mnt/... etc... ```
Re: "relaxed" `-hda file.qcow2` equivalent ?
Le ven. 15 nov. 2024 à 14:00, lacsaP Patatetom a écrit : > Le ven. 15 nov. 2024 à 13:35, Frantisek Rysanek > a écrit : > >> > # qemu-nbd -c /dev/nbd0 file.qcow2 >> > # mount /dev/nbd0p1 /mnt -o uid=me >> > $ # do some changes in /mnt/... >> > >> Are you sure the changes have made it to the underlying file? >> If you do an umount here, a sync() is guaranteed. >> If you do not umount, at this point you may have some dirty >> writeback. >> So that the following QEMU instance may find the filesystem in a >> "factually out of date" state. I dare not say "inconsistent state", >> because this should theoretically be taken care of... maybe... >> >> > I manually flush the system caches to make sure my `file.qcow2` file is up > to date . > > > $ qemu-system-x86_64 -snapshot -hda file.qcow2 # with locking=off >> >> Read-only access is one thing. Maybe the QEMU instance will cope with >> that. >> >> Do you want the QEMU instance to *write* to the filesystem, by any >> chance? >> > > no, I don't need it. > `-snapshot` is there to prevent corruption of the assembly left in place. > > >> If you manage to force-mount the filesystem for writing, and you >> write some changes to it, how does your "outer host-side instance of >> the mounted FS" get to know? >> This is a recipe for filesystem breakage :-) >> >> You know, these are exactly the sort of problems, that get avoided by >> the locking :-) >> >> They also get avoided by "shared access filesystems" such as GFS2 or >> OCFS2, or network filesystems such as NFS or CIFS=SMBFS. Maybe CEPH >> is also remotely in this vein, although this is more of a distributed >> clustered FS, and an overkill for your scenario. >> >> Frank >> >> in any case, to avoid this kind of problem and make sure I have the right > data, I manually clear the system caches with `sudo bash -c “sync && sysctl > -q vm.drop_caches=3”` before calling qemu. > `-snapshot` is there to prevent corruption of the assembly left in place. > > regards, lacsaP. > after rereading the documentation (introduction @ https://qemu-project.gitlab.io/qemu/system/qemu-block-drivers.html#disk-image-file-locking), I've just had a look at “Linux OFD” to try to remove the lock on my `file.qcow2` file, but I haven't found anything there : if someone has a clue, I'm interested...
Re: "relaxed" `-hda file.qcow2` equivalent ?
Hmm. By now I'm thinking aloud. According to the manpage (see below), "-blockdev" is the most "down to earth", true and modern way to specify the emulated disk - possibly needs to be combined with a "-device" to specify the guest-side emulated interface? Curiously, I cannot find a complete example... The manpage says, that: -hda (etc) is the oldest form, nowadays a macro for -drive + -device (controller). -drive is a shorthand for "-blockdev + -device" So... where I'm using -drive, I might as well be using -blockdev . Plus something more? No clue... :-) Or perhaps use "-blockdev node-name=.." instead of "-drive id=.." ? The qemu docs chapter on "QEMU block drivers" gives this example: -blockdev driver=qcow2,file.filename=/path/to/image,file.locking=off,file.driver =file I.e., you should put your locking=off option into a -blockdev definition as you say. And, I have an idea: rather than refer to driver=qcow2 and file.filename, how about referring to the loopback device (NBD) that you already have, courtesy of qemu-nbd ? Would that perhaps circumvent the file lock? ;-) -blockdev node-name=xy,driver=raw,file.driver=host_device,\ file.filename=/dev/loop0,file.locking=off -device virtio-scsi-pci -device scsi-hd,drive=xy Further reading: https://manpages.debian.org/testing/qemu-system-x86/qemu-system-x86_64 .1.en.html#Block_device_options https://unix.stackexchange.com/questions/753092/what-is-the-difference -between-these-two-blockdev-declarations https://www.qemu.org/docs/master/system/qemu-block-drivers.html#disk-i mage-file-locking Frank
Re: "relaxed" `-hda file.qcow2` equivalent ?
> And, I have an idea: rather than refer to driver=qcow2 and > file.filename, how about referring to the loopback device (NBD) that > you already have, courtesy of qemu-nbd ? Would that perhaps circumvent > the file lock? ;-) > > -blockdev node-name=xy,driver=raw,file.driver=host_device,\ > file.filename=/dev/loop0,file.locking=off > > -device virtio-scsi-pci -device scsi-hd,drive=xy > I mean: the QEMU device emulation would not run on top of the QCOW file directly (and the underlying filesystem, and its locking feature), but would instead share a block device with your host-side mount. Thus, it would also plug directly into any block-level buffering going on, on the host side. On the guest, I'm wondering if you should mount the partition with -o direct. This should prevent any write-back buffering in the guest, which however you will not be doing, as you say. On the other hand, if you make changes to the FS on the host side, while the QEMU guest instance is already running, the guest probably will not get to know about any changes, probably unless you umount and remount, that with "-o direct" (to avoid local read caching in the guest). Even if this crazy stuff works in the end, I'm wondering if it's all worth the implied pitfalls :-) Apparently you still need to keep stuff in sync in some way... Frank
Re: "relaxed" `-hda file.qcow2` equivalent ?
> # qemu-nbd -c /dev/nbd0 file.qcow2 > # mount /dev/nbd0p1 /mnt -o uid=me > $ # do some changes in /mnt/... > Are you sure the changes have made it to the underlying file? If you do an umount here, a sync() is guaranteed. If you do not umount, at this point you may have some dirty writeback. So that the following QEMU instance may find the filesystem in a "factually out of date" state. I dare not say "inconsistent state", because this should theoretically be taken care of... maybe... > $ qemu-system-x86_64 -snapshot -hda file.qcow2 # with locking=off Read-only access is one thing. Maybe the QEMU instance will cope with that. Do you want the QEMU instance to *write* to the filesystem, by any chance? If you manage to force-mount the filesystem for writing, and you write some changes to it, how does your "outer host-side instance of the mounted FS" get to know? This is a recipe for filesystem breakage :-) You know, these are exactly the sort of problems, that get avoided by the locking :-) They also get avoided by "shared access filesystems" such as GFS2 or OCFS2, or network filesystems such as NFS or CIFS=SMBFS. Maybe CEPH is also remotely in this vein, although this is more of a distributed clustered FS, and an overkill for your scenario. Frank
