ot;Is it too late in the game to
>>>>>>>>> interrogate the template to discover its virtual size if we're just
>>>>>>>>> about
>>>>>>>>> to copy the template to primary storage?"
>>>>>>>
t;>>>>>> about
>>>>>>>> to copy the template to primary storage?"
>>>>>>>>
>>>>>>>> If it's not, this might be the place to run the logic to figure out
>>>>>&
>>>
>>>>>>> Really, there are three big possibilities:
>>>>>>>
>>>>>>> 1) Just ask the end user to provide the virtual size (not commenting
>>>>>>> here on what happens for already-uploaded templates)
>>>
gt;>>> here on what happens for already-uploaded templates)
>>>>>>
>>>>>> or
>>>>>>
>>>>>> 2) Figure out the virtual size when the template is copied from
>>>>>> object storage to secondary storage an
e the DB with this info (not sure
>>>>> what happens if the template has already been copied to
>>>>> (secondary-storage)
>>>>> NFS because it was used before)
>>>>>
>>>>> or
>>>>>
>>>>> 3)
>>>>
>>>> or
>>>>
>>>> 3) Figure out the virtual size when the template is about to be copied
>>>> from secondary storage to primary storage
>>>>
>>>> On Mon, Sep 8, 2014 at 11:35 PM, Sanjeev Neelarapu &l
t;>> On Mon, Sep 8, 2014 at 11:35 PM, Sanjeev Neelarapu <
>>> sanjeev.neelar...@citrix.com> wrote:
>>>
>>>> Mike,
>>>>
>>>> You are right. Template gets copied to (secondary-storage) NFS before
>>>> being copied to primar
elar...@citrix.com> wrote:
>>
>>> Mike,
>>>
>>> You are right. Template gets copied to (secondary-storage) NFS before
>>> being copied to primary storage
>>>
>>> -Sanjeev
>>>
>>> -Original Message-
>>> From: Mike Tu
sage-
>> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>> Sent: Tuesday, September 09, 2014 10:55 AM
>> To: dev@cloudstack.apache.org
>> Cc: Punith S; Francois Gaudreault
>> Subject: Re: S3/Swift Problem around Virtual Size
>>
>> Hi W
gt; Sent: Tuesday, September 09, 2014 10:55 AM
> To: dev@cloudstack.apache.org
> Cc: Punith S; Francois Gaudreault
> Subject: Re: S3/Swift Problem around Virtual Size
>
> Hi Will,
>
> Thanks for the input!
>
> I like the idea of storing the virtual size as metadata in S3 or Swift
> although
; Francois Gaudreault
Subject: Re: S3/Swift Problem around Virtual Size
Hi Will,
Thanks for the input!
I like the idea of storing the virtual size as metadata in S3 or Swift although
this could require that the end user provide this value when uploading the
template.
However, if we have the ability
Hi Will,
Thanks for the input!
I like the idea of storing the virtual size as metadata in S3 or Swift
although this could require that the end user provide this value when
uploading the template.
However, if we have the ability to determine the virtual size of the
template after it gets download
My two cents on the topic.
Ideally we would save the size in the object store metadata and retrieve it
from the metadata if it is set. If it is not set in the object store
metadata, then when it is fetched, we have to put it on NFS and determine
the size (then ideally save the metadata back to th
Hi Punith,
Thanks for putting in time on this!
So, option 1 occurs after we download the template to S3 or Swift. We then
have to copy it to NFS so that we can determine the virtual size?
Relatively slow, but it would work.
If we went this route, would we then delete that template from the NFS
s
hi mike,
i have figured out the issue, in NFS secondary storage the virtual size is
been calculated by the VHD Processor by accessing the vhd template.
but in case of S3, cloudstack is not able to access the template but it
only gets to know the physical size of the template!
in order to solve t
Hi Punith,
Have you been able to make any progress with regards to this Swift/S3 issue?
Thanks!
Mike
On Wed, Aug 27, 2014 at 7:43 AM, Punith S wrote:
> hi
>
> think i had a timeout problem!
> on the second try the template has been downloaded to the S3 bucket and
> the management server shows
That's a good question, Marcus.
I've CCed Edison to bring this to his attention.
Edison - What do you think?
Thanks
On Tue, Aug 26, 2014 at 9:48 PM, Marcus wrote:
> Per Edisons comments about not knowing the image size, can't we just set
> some headers and store metadata with the template in
hi
think i had a timeout problem!
on the second try the template has been downloaded to the S3 bucket and the
management server shows the status as download complete with template size
as 1.6G instead of its virtual size 20G.
and i see the template's status as Download Complete but it seems it is
Per Edisons comments about not knowing the image size, can't we just set
some headers and store metadata with the template in S3 to save the virtual
size when the template is registered? I'm assuming here that the SSVM does
the work of pulling the template in and uploading to S3. Or it could be
sto
Looks like your SSVM cannot reach Internet properly?
FG
On 2014-08-26, 11:14 AM, Punith S wrote:
hi francois,
since i'm not having a swift setup, i'm using the s3 bucket.
and as you recommended i got the SSVM up with seeded nfs storage,
post that i removed the nfs secondary storage and added
hi francois,
since i'm not having a swift setup, i'm using the s3 bucket.
and as you recommended i got the SSVM up with seeded nfs storage,
post that i removed the nfs secondary storage and added the S3 with staging
nfs store as the new sec storage, since you cannot have any nfs secondary
storag
unzip the template(if
> > > it¹s a
> > > >> >> zipped file), and read virtual size from
the file, then report
> back
> > > to
> > > >> &g
n a VM.
>>>> > > > >>
>>>> > > > >> Thanks
>>>> > > > >> -min
>>>> > > > >>
>>>> > > > >> On 8/22/14 2:25 PM, "Francois Gaudreault" <
>>>>
lt;
>>> fgaudrea...@cloudops.com
>>> > >
>>> > > > >> wrote:
>>> > > > >>
>>> > > > >> >Edison,
>>> > > > >> >
>>> > > > >> >Isnt the templates down
;> >FG
>> > > > >> >On Aug 22, 2014 5:20 PM, "Edison Su"
>> wrote:
>> > > > >> >
>> > > > >> >> I know the reason why the size of template doesn¹t have
>> correct
>> > >
gt; > >> >> size if it¹s registered in S3/Swift:
> > > > >> >> In case of s3/swift, the template is directly stored into
> > s3/swift
> > > > >> >>through
> > > > >> >> swift/s3 api, there is no place
hrough
> > > >> >> swift/s3 api, there is no place for cloudstack to look into
> > template,
> > > >> to
> > > >> >> find out the virtual size during template registration.
> > > >> >> While, if secondary storage is N
>> to
> > >> >> find out the virtual size during template registration.
> > >> >> While, if secondary storage is NFS, the template is first stored on
> > >> >> NFS(which has file system), cloudstack can unzip the template(if
> > it
if
> it¹s a
> >> >> zipped file), and read virtual size from the file, then report back
> to
> >> >>mgt
> >> >> server.
> >> >> In order to fix it, we can add some code as: all the templates
> >> >>registered
> >> >&
>> >> server.
>> >> In order to fix it, we can add some code as: all the templates
>> >>registered
>> >> on Swift/S3, need to be downloaded to a NFS intermediate storage before
>> >>it
>> >> can be consumed by primary storage. Afte
to a NFS intermediate storage before
> >>it
> >> can be consumed by primary storage. After the download finished, then we
> >> check virtual size of template, and report back to mgt server/update DB
> >>etc.
> >>
> >> From: Mike Tutkowski [mail
o a NFS intermediate storage before
>>it
>> can be consumed by primary storage. After the download finished, then we
>> check virtual size of template, and report back to mgt server/update DB
>>etc.
>>
>> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>
storage. After the download finished, then we
> check virtual size of template, and report back to mgt server/update DB etc.
>
> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> Sent: Friday, August 22, 2014 1:38 PM
> To: dev@cloudstack.apache.org
> Cc: Edison Su
>
...@solidfire.com]
Sent: Friday, August 22, 2014 1:38 PM
To: dev@cloudstack.apache.org
Cc: Edison Su
Subject: S3/Swift Problem around Virtual Size
Hi,
This was brought up in a different e-mail thread, but I wanted to make it more
clear that it's related to CloudStack's download code around S3/Swi
I forgot to say that this was noticed with 4.4 (but is probably an earlier
issue).
On Fri, Aug 22, 2014 at 2:38 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:
> Hi,
>
> This was brought up in a different e-mail thread, but I wanted to make it
> more clear that it's related to CloudSta
Hi,
This was brought up in a different e-mail thread, but I wanted to make it
more clear that it's related to CloudStack's download code around S3/Swift,
so I'm opening up a new thread.
Francois (from CloudOps) noticed today that when he downloaded a template
(VHD format) to Swift (but it looks l
36 matches
Mail list logo
