Am 09.02.2026 um 09:38 hat Peter Krempa geschrieben: > Adding qemu-block list. I'll summarize the question for qemu folks at > the end. > > On Fri, Dec 12, 2025 at 10:00:17 -0300, João Jandre wrote: > > Hi, all > > > > I've been reading the QEMU documentation, the Libvirt documentation, as well > > as doing some testing and code investigation regarding disk cache modes. > > > > In the Libvirt documentation, it is stated that the disk driver cache modes > > accepted are "default", "none", "writethrough", "writeback", "directsync" > > and "unsafe". Analyzing the code, we can see that this is somewhat reflected > > on the cache modes that QEMU documents on the -drive option > > (https://qemu-project.gitlab.io/qemu/system/invocation.html): > > https://github.com/libvirt/libvirt/blob/04c1f458313e9001f5a804a898408e1f4985262b/src/qemu/qemu_domain.c#L10020-L10028 > > The intention behind that function was to preserve the behaviour from > the time when libvirt was using -drive when I was converting libvirt to > use -blockdev instead where the cache mode is configured by the 3 bool > options. > > > However, the default option (when we do not define the "cache" option) is > > different from other options and does not seem to map to any option on the > > While I no longer remeber the specifics (it's beel already almost 8 years > since I've wrote the code) the code was designed to avoid setting > any of the cache options in qemu (thus honour the 'hypervisor > default'-type behaviour). > > Specifically the 'write-cache' option for the disk frontend device is > defined as an "OnOffAuto" option and defaults to 'auto', which means > we do set it to "default", whatever that means. > > Now the comment in the aforementioned function which I've copied from > the qemu documentation doesn't have the line: > > "The default mode is cache=writeback." > > I unfortunately don't remember why thoug. It was present at the time I > was copying it from there based on the commit history. > > > Qemu side. I see that, from testing and analyzing the code, if the default > > (not defining the "cache" options) is used, cache.direct and cache.no-flush > > are not informed; also, the write-cache option is not informed. > > > > Just for comparison, here is a disk defined with cache=writeback: > > > > XML: > > > > |<disk type='file' device='disk'> <driver name='qemu' type='qcow2' > > cache='writeback'/> <source > > file='/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/c0e925a9-6697-4ee9-8abf-d36c7a95cea4' > > index='2'/> <backingStore type='file' index='3'> <format type='qcow2'/> > > <source > > file='/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/0db52809-9f33-43ac-8151-5452e9dee330'/> > > <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> > > <serial>c0e925a966974ee98abf</serial> <alias name='virtio-disk0'/> <address > > type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> | > > > > Generated QEMU command: > > > > |-blockdev > > {"driver":"file","filename":"/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/c0e925a9-6697-4ee9-8abf-d36c7a95cea4","node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":"libvirt-3-format"} > > -device > > virtio-blk-pci,bus=pci.0,addr=0x5,drive=libvirt-2-format,id=virtio-disk0,bootindex=2,write-cache=on,serial=c0e925a966974ee98abf > > | > > > > And here is a disk that does not have the cache option: > > > > XML: > > > > |<disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> > > <source > > file='/mnt/b59eff0f-5b97-37ed-a513-9e1983d1d19b/b430296b-0924-412e-a7b1-ddc3d4c90f83' > > index='2'/> <backingStore/> <target dev='vdc' bus='virtio'/> > > <serial>b430296b0924412ea7b1</serial> <alias name='virtio-disk2'/> <address > > type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </disk> | > > > > Generated QEMU command: > > > > |blockdev > > {"driver":"file","filename":"/mnt/b59eff0f-5b97-37ed-a513-9e1983d1d19b/b430296b-0924-412e-a7b1-ddc3d4c90f83","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null} > > -device > > virtio-blk-pci,bus=pci.0,addr=0x8,drive=libvirt-2-format,id=virtio-disk2,serial=b430296b0924412ea7b1 > > | > > Indeed the differences in the commandlines what you see here are > "itended" (in terms that I've likely wanted qemu to use the 'auto' mode > of 'write-cache' whatever it is.). > > > The QEMU documentation states in the -drive option that the default cache > > mode is |Writeback| (see > > https://qemu-project.gitlab.io/qemu/system/invocation.html), so I expected > > that setting |Writeback| or |Default| as the cache mode on Libvirt would > > also be the same. I ran some performance tests using both options to try and > > verify if they were the same, but they are clearly different. For each of > > the configuration profiles below, I created a VM and ran some fio tests: > > > > Combination Disk cache mode Disk Controller IO > > thread IO policy > > hds0i default virtio-scsi 0 iouring > > hds0t default virtio-scsi 0 threads > > hds1i default virtio-scsi 1 iouring > > hds1t default virtio-scsi 1 threads > > hdv0i default virtio 0 iouring > > hdv0t default virtio 0 threads > > hdv1i default virtio 1 iouring > > hdv1t default virtio 1 threads > > wbs0i writeback virtio-scsi 0 iouring > > wbs0t writeback virtio-scsi 0 threads > > wbs1i writeback virtio-scsi 1 iouring > > wbs1t writeback virtio-scsi 1 threads > > wbv0i writeback virtio 0 iouring > > wbv0t writeback virtio 0 threads > > wbv1i writeback virtio 1 iouring > > wbv1t writeback virtio 1 threads > > > > The fio test results are in a html attached to the email. > > > > I would like to understand what the practical difference is between not > > informing the cache and any other option. I can see that the disk is defined > > in different way, but just by analyzing the code I was not able to > > understand what that leads to; also, looking at the performance tests, we > > clearly see a difference in the results. Would anyone be able to explain > > what is the expected behavior when the disk is defined in such a way? > > > > By the way, I'm using Libvirt 8.0.0 and Qemu 6.2.0. > > Libvirt uses the '-blockdev' mode and thus the above logic to pick the > cache mode starting with qemu-4.2. In fact the support for '-drive' mode > is no longer present in current upstream as we no-longer support > anything older than qemu-6.2 thus always use the -blockdev mode.a > > While the _DEFAULT mode surely gives us the possibility to pick another > combination of parameters, qemu-4.2 and thus our switch to -blockdev > was 6 years ago. This gives a very long time when this combination of > parameters was used and so I'm wary about changing the mode, unless > it's bad for some reason. > > I'll thus CC the qemu folks with this message. > > TL;DR for qemu-block: > > When libvirt switched to '-blockdev' mode I've added a function which > maps the cache modes used with '-drive' to the 3 options with blockdev. > > I for some reason chose that the "VIR_DOMAIN_DISK_CACHE_DEFAULT" mode > is that *all* of the cache mode flags are just omitted relying on the > default in qemu rather than going with the documented 'writeback' mode > setting as default. > > Is the above combination problematic in any way? Should libvirt change > the "default"?
Using QEMU's defaults as the libvirt default makes sense to me. It's currently the same as "writeback" in QEMU anyway, so as far as the cache mode is concerned, both command lines quoted above should be identical in behaviour today. As for the fio benchmarks, testing cache.direct=off modes is notoriously hard to do in a meaningful way because you get a mix of requests that hit the host page cache and are effectively just a memcpy() and other requests that actually hit the disk. Obviously, performance will be very different for the two cases and their distribution is hard to control. I don't know if there is any reason to believe that in the tested scenario, disk 0 should have different performance from disk 1, but either way in some tests disk 0 is faster by a factor and in other tests disk 1 is. So it looks to me like you just have wild variation and you can't really read anything from the fio results. Kevin
