Yes, preallocation=metadata creates a large file full of holes. and maybe I need to modify storage consumption measuring method of CloudStack to care this kind sparse file.
To satisfy my needs, just turning on preallocation=metadata by default or making it configurable from agent.properties is enough, but I'm still thinking about making it configurable from disk offerings. Thanks, 2014-01-31 Marcus <shadow...@gmail.com>: > Ok, so not technically sparse in the sense that you have a large file > full of holes, but it's still not allocating the entire disk upfront. > You get the same result of disk savings either way, existing > cloudstack installs with qcow2 are still saving space in the same > manner. > > On Fri, Jan 31, 2014 at 11:43 PM, Marcus <shadow...@gmail.com> wrote: >> Here are my tests, using stock centos 6.5 qemu-img: >> >> [root@server ~]# qemu-img create -f qcow2 image.qcow2 10G >> Formatting 'image.qcow2', fmt=qcow2 size=10737418240 encryption=off >> cluster_size=65536 >> [root@server ~]# qemu-img info image.qcow2 >> image: image.qcow2 >> file format: qcow2 >> virtual size: 10G (10737418240 bytes) >> disk size: 136K >> cluster_size: 65536 >> >> No options leaves the disk as 136K, it's sparse. >> >> [root@server ~]# qemu-img create -f qcow2 -o preallocation=metadata >> imagepre.qcow2 10G >> Formatting 'imagepre.qcow2', fmt=qcow2 size=10737418240 encryption=off >> cluster_size=65536 preallocation='metadata' >> [root@server ~]# qemu info imagepre.qcow2 >> -bash: qemu: command not found >> [root@server ~]# qemu-img info imagepre.qcow2 >> image: imagepre.qcow2 >> file format: qcow2 >> virtual size: 10G (10737418240 bytes) >> disk size: 1.7M >> cluster_size: 65536 >> >> preallocation=metadata leaves disk as 1.7M, it's also sparse, but a bit >> bigger. >> >> On Fri, Jan 31, 2014 at 11:41 PM, Marcus <shadow...@gmail.com> wrote: >>> Yes, no_option.img is only 256k in size. it's sparse. >>> >>> On Fri, Jan 31, 2014 at 11:38 PM, Yoshikazu Nojima <m...@ynojima.net> wrote: >>>>>I don't think preallocation=metadata is required for sparse qcow2. >>>>>Just if you want it to NOT be sparse for metadata. >>>> >>>> I think it is required. Please see the output below. Only >>>> "metadata.img" is a sparse file. >>>> >>>> ============== >>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>> preallocation=off off.img 3G >>>> Formatting 'off.img', fmt=qcow2 size=3221225472 encryption=off >>>> cluster_size=65536 preallocation='off' >>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>> preallocation=metadata metadata.img 3G >>>> Formatting 'metadata.img', fmt=qcow2 size=3221225472 encryption=off >>>> cluster_size=65536 preallocation='metadata' >>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 -o >>>> preallocation=full full.img 3G >>>> Formatting 'full.img', fmt=qcow2 size=3221225472 encryption=off >>>> cluster_size=65536 preallocation='full' >>>> [root@ut1-qmb-rd5c test]# qemu-img create -f qcow2 no_option.img 3G >>>> Formatting 'no_option.img', fmt=qcow2 size=3221225472 encryption=off >>>> cluster_size=65536 >>>> [root@ut1-qmb-rd5c test]# ls -alhs >>>> total 3.1G >>>> 4.0K drwxr-xr-x 2 root root 4.0K Feb 1 06:29 . >>>> 4.0K dr-xr-x---. 5 root root 4.0K Feb 1 06:19 .. >>>> 3.1G -rw-r--r-- 1 root root 3.1G Feb 1 06:23 full.img >>>> 592K -rw-r--r-- 1 root root 3.1G Feb 1 06:21 metadata.img >>>> 136K -rw-r--r-- 1 root root 256K Feb 1 06:29 no_option.img >>>> 136K -rw-r--r-- 1 root root 256K Feb 1 06:21 off.img >>>> ============== >>>> >>>> >>>>>I also remember hearing that either recent qemu-img has a decent >>>>>preallocation by default now, or that performance has improved such >>>>>that it doesn't matter. Will need to do some reading to remember, but >>>>>I think I remember hearing it's not a big deal nowadays. >>>> >>>> Ummm. I tested with "qemu-img version 0.12.1," and I confirmed the >>>> performance of "preallocation=off" is worse than >>>> "preallocation=metadata", >>>> but it may be not true with latest qemu. I will test it. >>>> >>>> Thank you for your comment! >>>> >>>> Regards, >>>> >>>> >>>> 2014-01-31 Marcus <shadow...@gmail.com>: >>>>> I don't think preallocation=metadata is required for sparse qcow2. >>>>> Just if you want it to NOT be sparse for metadata. >>>>> >>>>> I also remember hearing that either recent qemu-img has a decent >>>>> preallocation by default now, or that performance has improved such >>>>> that it doesn't matter. Will need to do some reading to remember, but >>>>> I think I remember hearing it's not a big deal nowadays. >>>>> >>>>> On Fri, Jan 31, 2014 at 10:04 PM, Yoshikazu Nojima <m...@ynojima.net> >>>>> wrote: >>>>>> Hello Nux, >>>>>> >>>>>> Thank you for your comment. I will prepare feature specification. >>>>>> >>>>>> Regards, >>>>>> >>>>>> 2014-01-31 Nux! <n...@li.nux.ro>: >>>>>>> On 31.01.2014 20:24, Yoshikazu Nojima wrote: >>>>>>>> >>>>>>>> Afternoon All, >>>>>>>> >>>>>>>> Is there anyone working on adding volume provisioning method option? >>>>>>>> As you know, thin provisioning of a volume save consumption of a >>>>>>>> storage, and fat provisioning improves IOPS performance. >>>>>>>> Especially, Qcow2 can save storage consumption and achive relatively >>>>>>>> better performance than default by provisioning a volume with an >>>>>>>> option "preallocation=metadata", which makes an image file a sparse >>>>>>>> file. >>>>>>>> >>>>>>>> http://itscblog.tamu.edu/improve-disk-io-performance-in-kvm/#.232---preallocation >>>>>>>> >>>>>>>> Any thoughts about this? >>>>>>>> If it is ok, I will write a feature specification on confluence and >>>>>>>> start implementation. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Noji >>>>>>>> >>>>>>>> >>>>>>>> Yoshikazu Nojima <m...@ynojima.net> >>>>>>> >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I thought preallocation=metadata is common practice since years. Now I >>>>>>> find >>>>>>> out ACS doesn't actually use it? >>>>>>> If so, this is really _bad_ and needs to be fixed ASAP... >>>>>>> >>>>>>> Thanks Noji >>>>>>> >>>>>>> -- >>>>>>> Sent from the Delta quadrant using Borg technology! >>>>>>> >>>>>>> Nux! >>>>>>> www.nux.ro
