On Wed, Sep 8, 2010 at 3:37 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Sep 7, 2010 at 8:25 PM, Blue Swirl <blauwir...@gmail.com> wrote:
>> On Mon, Sep 6, 2010 at 10:04 AM, Stefan Hajnoczi
>> <stefa...@linux.vnet.ibm.com> wrote:
>>> QEMU Enhanced Disk format is a disk image format that forgoes features
>>> found in qcow2 in favor of better levels of performance and data
>>> integrity. Due to its simpler on-disk layout, it is possible to safely
>>> perform metadata updates more efficiently.
>>>
>>> Installations, suspend-to-disk, and other allocation-heavy I/O workloads
>>> will see increased performance due to fewer I/Os and syncs. Workloads
>>> that do not cause new clusters to be allocated will perform similar to
>>> raw images due to in-memory metadata caching.
>>>
>>> The format supports sparse disk images. It does not rely on the host
>>> filesystem holes feature, making it a good choice for sparse disk images
>>> that need to be transferred over channels where holes are not supported.
>>>
>>> Backing files are supported so only deltas against a base image can be
>>> stored.
>>>
>>> The file format is extensible so that additional features can be added
>>> later with graceful compatibility handling.
>>>
>>> Internal snapshots are not supported. This eliminates the need for
>>> additional metadata to track copy-on-write clusters.
>>
>> It would be nice to support external snapshots, so another file
>> besides the disk images can store the snapshots. Then snapshotting
>> would be available even with raw or QED disk images. This is of course
>> not QED specific.
>>
>>> + *
>>> + * +--------+----------+----------+----------+-----+
>>> + * | header | L1 table | cluster0 | cluster1 | ... |
>>> + * +--------+----------+----------+----------+-----+
>>> + *
>>> + * There is a 2-level pagetable for cluster allocation:
>>> + *
>>> + * +----------+
>>> + * | L1 table |
>>> + * +----------+
>>> + * ,------' | '------.
>>> + * +----------+ | +----------+
>>> + * | L2 table | ... | L2 table |
>>> + * +----------+ +----------+
>>> + * ,------' | '------.
>>> + * +----------+ | +----------+
>>> + * | Data | ... | Data |
>>> + * +----------+ +----------+
>>> + *
>>> + * The L1 table is fixed size and always present. L2 tables are allocated
>>> on
>>> + * demand. The L1 table size determines the maximum possible image size;
>>> it
>>> + * can be influenced using the cluster_size and table_size values.
>>
>> The formula for calculating the maximum size would be nice. Is the
>> image_size the limit? How many clusters can there be? What happens if
>> the image_size is not equal to multiple of cluster size? Wouldn't
>> image_size be redundant if cluster_size and table_size determine the
>> image size?
>
> image_size is the logical image size, whereas TABLE_NELEMS *
> TABLE_NELEMS * cluster_size is the maximum logical image size
> (TABLE_NELEMS depends on table_size and cluster_size). I have updated
> the wiki page with the constraint.
Based on these:
#define TABLE_NOFFSETS (table_size * cluster_size / sizeof(uint64_t))
header.image_size <= TABLE_NOFFSETS * TABLE_NOFFSETS * header.cluster_size,
the maximum image size equals to table_size^2 * cluster_size^3 /
sizeof(uint64_t)^2. Is the squaring and cubing of the terms
beneficial? I mean, the size scales up fast to unusable numbers,
whereas with a more linear equation (for example, allow different L1
and L2 sizes), more values could be actually usable. Again, I'm not
sure if this matters at all.
I think the minimum size should be table_size = 1, cluster_size = 4
bytes, 1^2 * 4^3 / 8^2 = 2 bytes, or is the minimum bigger? What's
the minimum for cluster_size?
>
> I don't think the specification needs to mention error behavior, that
> would depend on the implementation. But the specification needs to
> mention alignment constraints so I have added them.
>
>>
>>> + *
>>> + * All fields are little-endian on disk.
>>> + */
>>> +
>>> +typedef struct {
>>> + uint32_t magic; /* QED */
>>> +
>>> + uint32_t cluster_size; /* in bytes */
>>
>> Doesn't cluster_size need to be a power of two?
>>
>>> + uint32_t table_size; /* table size, in clusters */
>>> + uint32_t first_cluster; /* first usable cluster */
>>
>> This introduces some limits to the location of first cluster, with 4k
>> clusters it must reside within the first 16TB. I guess it doesn't
>> matter.
>
> It shouldn't matter since any header that is >=16 TB means something
> mutated, escaped the lab, and is terrorizing the world as a qed
> monster image.
In the Wiki version this has changed to header_size in clusters. With
2GB clusters, there will be some wasted bits.
By the way, perhaps cluster_size of 0 should mean 4GB? Or maybe all
sizes should be expressed as an exponent to 2, then 16 bits would
allow cluster sizes up to 2^64?
>>
>>> +
>>> + uint64_t features; /* format feature bits */
>>> + uint64_t compat_features; /* compatible feature bits */
>>> + uint64_t l1_table_offset; /* L1 table offset, in bytes */
>>> + uint64_t image_size; /* total image size, in bytes */
>>> +
>>> + uint32_t backing_file_offset; /* in bytes from start of header */
>>> + uint32_t backing_file_size; /* in bytes */
>>> + uint32_t backing_fmt_offset; /* in bytes from start of header */
>>> + uint32_t backing_fmt_size; /* in bytes */
>>> +} QEDHeader;
>>> +
>>> +typedef struct {
>>> + uint64_t offsets[0]; /* in bytes */
>>> +} QEDTable;
>>
>> Is this for both L1 and L2 tables?
>
> Yes, they both have the same size.
>
> Stefan
>
