On Thu, Nov 16, 2023 at 06:48:09PM +0100, Alberto Garcia wrote: > Hi, > > I haven't written here in a while :) but I have something small that I > would like to discuss. > > Using qemu-img to convert an image and writing the result directly to > stdout is a question that has already been raised in the past (see > [1] for an example) and it's clear that it's generally not possible > because the images need to be seekable. > > While I think that there's almost certainly no generic way to do > something like that for every combination of input and output formats, > I do think that it should be relatively easy to produce a qcow2 file > directly to stdout as long as the input file is on disk.
Indeed, it does seem like this should be easy enough to cobble together. > > I'm interested in this use case, and I think that the method would be > as simple as this: > > 1. Decide a cluster size for the output qcow2 file. > 2. Read the input file once to determine which clusters need to be > allocated in the output file and which ones don't. > 3. That infomation is enough to determine the number and contents of > the refcount table, refcount blocks, and L1/L2 tables. > 4. Write the qcow2 header + metadata + allocated data to stdout. It may also be possible to write a qcow2 file that uses the external data bit, so that you are only writing the qcow2 header + metadata, and reusing the existing input file as the external data. > > Since this would be qcow2-specific I would probably implement this as > a separate tool instead of adding it directly to qemu-img. This could > go to the scripts/ directory because I can imagine that such a tool > could be useful for other people. Also sounds reasonable. > > Because this would be an external tool it would only support a qcow2 > file with the default options. Other features like compression would > be out of scope. Why is compression not viable? Are you worried that the qcow2 metadata (such as refcounts) becomes too complex? I've also wondered how easy or hard it would be to write a tool that can take an existing qcow2 file and defragment and/or compress it in-place (rather than having to copy it to a second qcow2 file). > > For the same reason the input would be a raw file for simplicity, > other input files could be presented in raw format using > qemu-storage-daemon. Yes, getting seekable input from any other format is easy enough; it's the streaming output that is the trickier task. > > And that's the general idea, comments and questions are welcome. > > Thanks! > > Berto > > [1] https://lists.nongnu.org/archive/html/qemu-discuss/2020-01/msg00014.html > -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org
