Thank you, Kashyap! For a general audience, let me annotate this just a little bit. I'll try to avoid being too partial in this email, and will follow up with some opinions in a separate email, if any.
If I have misrepresented the spirit of a concern or objection, please do correct my annotations. On 02/23/2016 09:25 AM, Kashyap Chamarthy wrote: > On Tue, Feb 16, 2016 at 02:02:58PM +0000, Stefan Hajnoczi wrote: >> Hi, >> There are several ongoing efforts to implement incremental >> backup-related features. >> >> Let's have a voice/video conference to get everyone on the same page, >> avoid duplicated work, and get patches merged faster. >> >> Agenda: >> >> * External incremental backup API. Summarize requirements common to >> third-party backup appliances and agree on suitable API direction. >> >> * Persistent dirty bitmaps. Agree on how qcow2 and/or qbm will hold >> bitmaps in various use cases. >> >> * Anything else? > > I was in a listen-only mode on the call, here's some quick minutes > (read: haphazard scribbling) so please excuse grammar > thinkos/typos/missed points. Feel free to correct/adjust. > > ----------------------------------------------------------------------- > - [denis-lunev] Migration support for dirty bitmaps? > - [stefanha] One discussion that might need to reset things is Fam's > QEMU QBM (Qemu Bit Map) QBM is a proposed image format that is essentially an on-disk wrapper around existing formats. It's a plaintext JSON file that describes a disk image resource and an existing format (e.g. raw, qcow2, etc) and a number of raw bitmap data files. It is essentially a "filter" format, but one that exists on-disk. > - [stefanha] Incremental backups _should_ be possible with RAW > - [jsnow] Apart from just from having incremental backups for RAW > devices, any reason why qcow2 _cannot_ be the bitmap provider? One of the existing arguments against augmenting non-native image formats (e.g. adding features to RAW) is complexity of maintenance and wanting to bolster the strength of the qcow2 format. The refrain is "If qcow2 can't do it, we should fix qcow2." However, I wonder if there are any cases we wish to support incremental backup for that do not or cannot use .qcow2 as the root format for the drive -- principally SAN devices and other networked block devices where qcow2 is not the primary handle to the resource tree. > - [kwolf] What are the actual requirements? > - [stefanha] Vladimir's series that modified qcow2 - the idea was that > you don't _have_ to use qcow2 format, but you could use it as a bitmap > container for other images. The original proposal for bitmap persistence involved using .qcow2 as a container format to store bitmaps for *any* image, not just ones related to this particular qcow2. The one that was merged recently pares down this idea into storing only bitmaps related to that qcow2. > - [kwolf] Something about VM state internal snapshots: where the VM > state is saved into qcow2 file: > - Don't store bitmaps in a qcow2 file Kevin feels that storing VM snapshots inside of .qcow2 files was a mistake, because it contains data not related necessarily to the image stored inside of the qcow2. His thought was that storing any bitmaps inside of a qcow2 is similar to the mistake that was made with VM snapshots. There is no major opposition to storing bitmaps that describe a qcow2 inside of that qcow2. > - Stack the qcow2 on the specific image that the bitmap is for Instead of using QBM as our filter format, perhaps qcow2 can be extended to fulfill the same role, which avoids us cooking up an entirely new primary/first-party image format that we'd have to support. > - [stefanha] What happens when there's guest I/O (?) qcow2 does not currently support a mode where the top-layer is a filter. writes currently always land at the top layer, which would be undesirable for a "bitmap filter." > - [kwolf] The question is: if it wouldn't be simple to extend qcow2 - to > forward all writes to the backing file > - If we _don't_ want to use qcow2 to store this: > - [kwolf] Concern is consistency: doing one thing for qcow2; and other > thing for other formats? We are concerned over the redundancy of having both per-image-format bitmap saving routines AND having a global bitmap saving mechanic. To date, we have approved a qcow2 extension for bitmap saving, and Denis Lunev at al either already have or are prepared to write a similar extension for the parallels image format. Is it too messy or complicated to support qcow2 native saving, parallels saving, AND an "any-format" saving mechanism? Will this lead to too much code duplication and an undue maintenance burden caused by a lack of planning? > - [eblake] This is not the first time we're adding something on top of > raw images > - [eblake] bitmap on top of RAW - this concept has been floating on > the list for several years > - https://lists.nongnu.org/archive/html/qemu-devel/2013-08/msg03682.html > - [kwolf] What is the use case? > - Backing files with raw images > - If you have a raw image file, you can use backing files with > it; if at some point in time Desires to augment RAW in QEMU have existed for a long time, but qcow2's performance has continued to improve, lessening some of the technical merit to past justifications. Are there any hard technical reasons why RAW *must* still be augmented? We don't have a smoking gun. > - [eblake] Mentioned LUKS encryption of danpb (in comparision with QBM > work?) > - A seperate driver (general purpose) 'luks' format driver which can > be layered over any other existing block driver. > - Embedded LUKS inside qcow2 > - it's encrypted as _part_ of the qcow2 file > - A general purpose 'luks' format driver which can be layered over > any other existing block driver. > - To differentiate intentionally encrypted as part of qcow2 vs. the > guest data > - [kwolf] One important difference: Dan didn't introduce a new file > _format_ for qcow2. > - [eblake] In a sense, LUKS _is_ a new file format - but indeed it's > interoperating with existing format > > - [stefanha] Is the least controversial part: getting the Qcow2 support > in that Vladimir is working on? > - [kwolf] Concern: is the relationship with QBM -- I don't want to end > up with duplicated things at the end > - Once we accept this API, we can't change it Can we progress on the qcow2-native bitmap persistence feature for now and worry about QBM later? There are concerns that by solidifying the qcow2 method now, we may forever prohibit ourselves from a clean, non-redundant universal persistence solution. > - [denis-lunev] A lot of resources/time has been invested in this. > Specification about bitmaps. Version-17 for this spec is too much. > - [kwolf] If you go forward with qcow2 path, then, we're not going to do > the QBM approach. > - [fam] Qcow2 extension has made its progress > - We should go ahead and make the extension in qcow2 and see how it > goes > - People will start complaining about missing features in raw (?) > - Concern: in certain protocols like Ceph, people would tend to use > Raw images, without any tools on top > - [stefanha] If we do what Kevin told: qcow2 will have a node - where > the writes will be forwarded to backing file, what's holding up there > (?) > - You have a qcow2 file that doesn't have much data, except metadata > and including bitmap > - [jsnow] We should move forward with qcow2 persistence approach. > - We can re-discuss the merits of duplication again Consensus appears to be to move forward with per-format bitmap saving for now (starting with qcow2, parallels perhaps to follow.) We can re-investigate the QBM/qcow2 as a filter-format argument later after more client use cases come to light. We can debate about the merits/demerits of code duplication between a per-format persistence and a global persistence when the need arises. The qcow2 format is an obvious use case that is obviously desired and one that we have the most justification for, so it is probably unreasonable to delay work pending future debates. > - [stefanha] Question for Eric: In terms of committing an API, client > applications, external plugins for backup/ related libvirt work? > - [eblake] From libvirt's perspective, the biggest thing we need for > 2.6 is having QMP 'blockdev-add' for everything: > - We want to get Ceph, NBD and Gluster all of those under QAPI, to > be able to programatically address them from libvirt > - We currently sometimes fallback to HMP > - [kwolf] I don't think it's possible for 2.6 yet. So far, we > have NBD support on the list > - [eblake] Going to look at 'blockdev-add' to see what's missing > and what needs to be added > - [eblake] Ability to track persistent node names in libvirt XML > - We're at the point now that every node creation can have a name. > - libvirt does have some work to do take advantage of node names, > remembering what names it has assigned > ----------------------------------------------------------------------- > > --js
