thanks for opening this thread mike, since i only use nfs as my secondary storage provider, i didn't see this issue till date.
is this issue occurring even using a S3 secondary storage with staging nfs store ? if so like edison pointed we need to fetch the virtual size from the nfs store instead of S3 in the deploymentmanager. thanks On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski < mike.tutkow...@solidfire.com> wrote: > Hey Edison, > > Do you know how difficult/easy of a fix this is, who might be available to > put this fix in, and for what release (hopefully 4.4.1) this fix could find > its way in? > > Thanks! > Mike > > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault < > fgaudrea...@cloudops.com> wrote: > > > Min, > > > > Ok, but this is not the behavior I see. Even without requesting a VM > > create, the template is pushed to the staging NFS at least once. Is it > > downloaded there or pushed after download, that I am not sure. I was > > assuming the swift upload bash script was executed after the template is > on > > the staging. > > > > Anyway... the focus is on the virt size, and you all know the code better > > than I do :) > > > > FG > > On Aug 22, 2014 5:28 PM, "Min Chen" <min.c...@citrix.com> wrote: > > > >> No. For S3/Swift, register template will directly upload templates to S3 > >> without going through staging NFS. It will only be copied to staging NFS > >> when we first use that template to provision a VM. > >> > >> Thanks > >> -min > >> > >> On 8/22/14 2:25 PM, "Francois Gaudreault" <fgaudrea...@cloudops.com> > >> wrote: > >> > >> >Edison, > >> > > >> >Isnt the templates downloaded to the Staging NFS first? > >> > > >> >FG > >> >On Aug 22, 2014 5:20 PM, "Edison Su" <edison...@citrix.com> wrote: > >> > > >> >> I know the reason why the size of template doesn¹t have correct > virtual > >> >> size if it¹s registered in S3/Swift: > >> >> In case of s3/swift, the template is directly stored into s3/swift > >> >>through > >> >> swift/s3 api, there is no place for cloudstack to look into template, > >> to > >> >> find out the virtual size during template registration. > >> >> While, if secondary storage is NFS, the template is first stored on > >> >> NFS(which has file system), cloudstack can unzip the template(if > it¹s a > >> >> zipped file), and read virtual size from the file, then report back > to > >> >>mgt > >> >> server. > >> >> In order to fix it, we can add some code as: all the templates > >> >>registered > >> >> on Swift/S3, need to be downloaded to a NFS intermediate storage > before > >> >>it > >> >> can be consumed by primary storage. After the download finished, then > >> we > >> >> check virtual size of template, and report back to mgt server/update > DB > >> >>etc. > >> >> > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] > >> >> Sent: Friday, August 22, 2014 1:38 PM > >> >> To: dev@cloudstack.apache.org > >> >> Cc: Edison Su > >> >> Subject: S3/Swift Problem around Virtual Size > >> >> > >> >> Hi, > >> >> > >> >> This was brought up in a different e-mail thread, but I wanted to > make > >> >>it > >> >> more clear that it's related to CloudStack's download code around > >> >>S3/Swift, > >> >> so I'm opening up a new thread. > >> >> > >> >> Francois (from CloudOps) noticed today that when he downloaded a > >> >>template > >> >> (VHD format) to Swift (but it looks like the same applies for S3) > that > >> >>the > >> >> physical and virtual sizes are set to be the same. > >> >> > >> >> This appears to have the following consequence: > >> >> > >> >> You can download a template with a physical size of, say, 3 GB and a > >> >>root > >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual size > >> >>showing > >> >> as 20 GB, it shows as 3 GB. > >> >> > >> >> This is not an issue with NFS. In that situation, the two sizes are > >> >> correctly accounted for. > >> >> > >> >> What later can happen is the template is downloaded from Swift and > >> takes > >> >> up an unexpected amount of space on the XenServer storage repository > >> >>(SR). > >> >> > >> >> If there is enough space on the SR, this isn't too big of a deal. > >> >>However, > >> >> for so-called managed storage plug-ins (examples are SolidFire and > >> >> CloudByte), this will lead to them dynamically creating a SAN volume > of > >> >>the > >> >> wrong size. > >> >> > >> >> Francois opened up the following ticket: > >> >> > >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406 > >> >> > >> >> Thanks! > >> >> > >> >> -- > >> >> Mike Tutkowski > >> >> Senior CloudStack Developer, SolidFire Inc. > >> >> e: mike.tutkow...@solidfire.com<mailto:mike.tutkow...@solidfire.com> > >> >> o: 303.746.7302 > >> >> Advancing the way the world uses the cloud< > >> >> http://solidfire.com/solution/overview/?video=play> > >> >> > >> > >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the cloud > <http://solidfire.com/solution/overview/?video=play>*™* > -- regards, punith s cloudbyte.com
