Punith, I highly anticipate the same issue with S3... it shares a lot of code with swift.
My focus would be swift, but we should fix for both :) FG On Aug 25, 2014 6:33 AM, "Punith S" <punit...@cloudbyte.com> wrote: > thanks for opening this thread mike, > > since i only use nfs as my secondary storage provider, i didn't see this > issue till date. > > is this issue occurring even using a S3 secondary storage with staging nfs > store ? > > if so like edison pointed we need to fetch the virtual size from the nfs > store instead of S3 in the deploymentmanager. > > thanks > > > On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > > > Hey Edison, > > > > Do you know how difficult/easy of a fix this is, who might be available > to > > put this fix in, and for what release (hopefully 4.4.1) this fix could > find > > its way in? > > > > Thanks! > > Mike > > > > > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault < > > fgaudrea...@cloudops.com> wrote: > > > > > Min, > > > > > > Ok, but this is not the behavior I see. Even without requesting a VM > > > create, the template is pushed to the staging NFS at least once. Is it > > > downloaded there or pushed after download, that I am not sure. I was > > > assuming the swift upload bash script was executed after the template > is > > on > > > the staging. > > > > > > Anyway... the focus is on the virt size, and you all know the code > better > > > than I do :) > > > > > > FG > > > On Aug 22, 2014 5:28 PM, "Min Chen" <min.c...@citrix.com> wrote: > > > > > >> No. For S3/Swift, register template will directly upload templates to > S3 > > >> without going through staging NFS. It will only be copied to staging > NFS > > >> when we first use that template to provision a VM. > > >> > > >> Thanks > > >> -min > > >> > > >> On 8/22/14 2:25 PM, "Francois Gaudreault" <fgaudrea...@cloudops.com> > > >> wrote: > > >> > > >> >Edison, > > >> > > > >> >Isnt the templates downloaded to the Staging NFS first? > > >> > > > >> >FG > > >> >On Aug 22, 2014 5:20 PM, "Edison Su" <edison...@citrix.com> wrote: > > >> > > > >> >> I know the reason why the size of template doesn¹t have correct > > virtual > > >> >> size if it¹s registered in S3/Swift: > > >> >> In case of s3/swift, the template is directly stored into s3/swift > > >> >>through > > >> >> swift/s3 api, there is no place for cloudstack to look into > template, > > >> to > > >> >> find out the virtual size during template registration. > > >> >> While, if secondary storage is NFS, the template is first stored on > > >> >> NFS(which has file system), cloudstack can unzip the template(if > > it¹s a > > >> >> zipped file), and read virtual size from the file, then report back > > to > > >> >>mgt > > >> >> server. > > >> >> In order to fix it, we can add some code as: all the templates > > >> >>registered > > >> >> on Swift/S3, need to be downloaded to a NFS intermediate storage > > before > > >> >>it > > >> >> can be consumed by primary storage. After the download finished, > then > > >> we > > >> >> check virtual size of template, and report back to mgt > server/update > > DB > > >> >>etc. > > >> >> > > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] > > >> >> Sent: Friday, August 22, 2014 1:38 PM > > >> >> To: dev@cloudstack.apache.org > > >> >> Cc: Edison Su > > >> >> Subject: S3/Swift Problem around Virtual Size > > >> >> > > >> >> Hi, > > >> >> > > >> >> This was brought up in a different e-mail thread, but I wanted to > > make > > >> >>it > > >> >> more clear that it's related to CloudStack's download code around > > >> >>S3/Swift, > > >> >> so I'm opening up a new thread. > > >> >> > > >> >> Francois (from CloudOps) noticed today that when he downloaded a > > >> >>template > > >> >> (VHD format) to Swift (but it looks like the same applies for S3) > > that > > >> >>the > > >> >> physical and virtual sizes are set to be the same. > > >> >> > > >> >> This appears to have the following consequence: > > >> >> > > >> >> You can download a template with a physical size of, say, 3 GB and > a > > >> >>root > > >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual size > > >> >>showing > > >> >> as 20 GB, it shows as 3 GB. > > >> >> > > >> >> This is not an issue with NFS. In that situation, the two sizes are > > >> >> correctly accounted for. > > >> >> > > >> >> What later can happen is the template is downloaded from Swift and > > >> takes > > >> >> up an unexpected amount of space on the XenServer storage > repository > > >> >>(SR). > > >> >> > > >> >> If there is enough space on the SR, this isn't too big of a deal. > > >> >>However, > > >> >> for so-called managed storage plug-ins (examples are SolidFire and > > >> >> CloudByte), this will lead to them dynamically creating a SAN > volume > > of > > >> >>the > > >> >> wrong size. > > >> >> > > >> >> Francois opened up the following ticket: > > >> >> > > >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406 > > >> >> > > >> >> Thanks! > > >> >> > > >> >> -- > > >> >> Mike Tutkowski > > >> >> Senior CloudStack Developer, SolidFire Inc. > > >> >> e: mike.tutkow...@solidfire.com<mailto: > mike.tutkow...@solidfire.com> > > >> >> o: 303.746.7302 > > >> >> Advancing the way the world uses the cloud< > > >> >> http://solidfire.com/solution/overview/?video=play> > > >> >> > > >> > > >> > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the cloud > > <http://solidfire.com/solution/overview/?video=play>*™* > > > > > > -- > regards, > > punith s > cloudbyte.com >
