Hey Sanjeev, I've got system VMs downloading from S3 to the NFS cache so hopefully things will come alive soon.
Can you explain (or point me to the documentation) what gets cached and how long it's kept for etc. Also there was a bug fix "Cannot Deploy VM when using S3 object store without NFS Cache" https://issues.apache.org/jira/browse/CLOUDSTACK-3215 which suggests to me that the cache should be required, could someone comment/clarify the situation (thanks everyone for their help) Regards Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus paul.an...@shapeblue.com -----Original Message----- From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] Sent: 11 February 2014 11:32 To: dev@cloudstack.apache.org Subject: RE: S3 based secondary Storage Yes , staging storage can be added after adding S3 storage. For adding staging storage we have createSecondaryStagingStore API. Please see the api doc http://cloudstack.apache.org/docs/api/apidocs-4.2/root_admin/createSecondaryStagingStore.html For adding S3 we use addImageStore API with provider as S3. -Sanjeev -----Original Message----- From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Tuesday, February 11, 2014 4:21 PM To: dev@cloudstack.apache.org Subject: RE: S3 based secondary Storage Thanks Sanjeev for the explanation, 1. can the staging area be added after the S3 storage is added or does it have to be done at the same time (I can't see a 'staging' api command) 2. the adds3 api documentation doesn't include anything about staging areas - what are the arguments? (I'll file a bug to get them added to the API documentation too) Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> -----Original Message----- From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] Sent: 11 February 2014 10:37 To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> Subject: RE: S3 based secondary Storage Hi, At least one NFS Staging storage is needed in each zone to copy storage artifacts from S3 to primary and vice-versa. While adding S3 storage NFS staging storage is optional because we can add more than one S3 storage in a region. So for every S3 storage addition we don't need to add NFS staging storage. In this case we can uncheck staging storage part. -Sanjeev -----Original Message----- From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Tuesday, February 11, 2014 2:51 PM To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> Subject: RE: S3 based secondary Storage Success! (some) I changed the endpoint to s3.amazonaws.com last night and it failed, but actually because download.cloud.com wasn't responding, I tried again today and it uploaded - Cool. BUT now, CloudStack is complaining that I don't have a cache/staging storage... I thought that the staging area was to migrate from NFS to s3 (why is it optional if you have to have one?) Sorry, still getting my head around this... 2014-02-11 09:08:24,396 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (secstorage-1:ctx-c5e3fb33) needCacheStorage true, dest at null dest role Primarytemplate/tmpl/1/1/routing-1/systemvm64template-2014-01-14-master-xen.vhd.bz2 src role Image 2014-02-11 09:08:24,398 DEBUG [o.a.c.s.c.a.StorageCacheRandomAllocator] (secstorage-1:ctx-c5e3fb33) Can't find staging storage in zone: 1 2014-02-11 09:08:24,398 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (secstorage-1:ctx-c5e3fb33) copy object failed: com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope id 1 type ZONE 2014-02-11 09:08:24,406 DEBUG [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:ctx-c5e3fb33) Unable to create Vol[11|vm=11|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope id 1 type ZONE 2014-02-11 09:08:24,406 INFO [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-c5e3fb33) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[11|vm=11|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope id 1 type ZONE Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> -----Original Message----- From: Min Chen [mailto:min.c...@citrix.com] Sent: 10 February 2014 22:57 To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> Subject: Re: S3 based secondary Storage It seems to me that your endpoint may not be right, see this link: http://docs.aws.amazon.com/general/latest/gr/rande.html, where it lists all valid endpoint for Amazon S3, your endpoint is not the one for Ireland. Thanks -min On 2/10/14 2:13 PM, "Paul Angus" <paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com>> wrote: >I removed it from the paste for security - it was there:) > >Regards > >Paul Angus >Cloud Architect >S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus >paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> > >-----Original Message----- >From: Min Chen [mailto:min.c...@citrix.com] >Sent: 10 February 2014 22:07 >To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> >Subject: Re: S3 based secondary Storage > >Look further into your API invoked to add S3 image store: > >command=addImageStore&response=json&sessionkey=xqZoRDVpFezJkBlS3vqGZQzU >ytY >=&name=SecStor-S3&provider=S3&details[0].key=accesskey&details[0].value >=<k >ey>&details[1].key=secretkey&details[1].value=<key>&details[2].key=buck >ey>et& >details[2].value=cloudstack-sec-stor&details[3].key=usehttps&details[3] >.va >lue=false&details[4].key=endpoint&details[4].value=s3-website-eu-west-1 >.am >azonaws.com > >See highlighted part, you didn't provide access key and secret key there. > >Thanks >-min > >On 2/10/14 1:36 PM, "Paul Angus" ><paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com%3cmailto:paul.an...@shapeblue.com>>> > wrote: > >Hey Min, > >S3cmd works ok too, but what's odd is that it says that the bucket is >in the US, but the AWS S3 control panel says it's in Ireland. I'm using >s3-website-eu-west-1.amazonaws.com as my endpoint in CloudStack > >Here's my s3cfg file > >(I'll change the access keys soon :)) > > >[default] >access_key = AKIAIF25KWKAPF4E74KQ >bucket_location = US >cloudfront_host = cloudfront.amazonaws.com cloudfront_resource = >/2010-07-15/distribution default_mime_type = binary/octet-stream >delete_removed = False dry_run = False encoding = UTF-8 encrypt = False >follow_symlinks = False force = False get_continue = False gpg_command >= /usr/bin/gpg gpg_decrypt = %(gpg_command)s -d --verbose >--no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o >%(output_file)s %(input_file)s gpg_encrypt = %(gpg_command)s -c >--verbose --no-use-agent --batch --yes --passphrase-fd >%(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_passphrase = >mcTavish99 guess_mime_type = True host_base = s3.amazonaws.com >host_bucket = %(bucket)s.s3.amazonaws.com human_readable_sizes = False >list_md5 = False >log_target_prefix = >preserve_attrs = True >progress_meter = True >proxy_host = >proxy_port = 0 >recursive = False >recv_chunk = 4096 >reduced_redundancy = False >secret_key = 8/C8W1VcauYm0BLQqAUwpUbjDFzkOqD/lUN+8hta >send_chunk = 4096 >simpledb_host = sdb.amazonaws.com >skip_existing = False >socket_timeout = 300 >urlencoding_mode = normal >use_https = False >verbosity = WARNING > > > >Regards, > >Paul Angus >Cloud Architect >S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus >paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com%3cmailto:paul.an...@shapeblue.com>> > >-----Original Message----- >From: Min Chen [mailto:min.c...@citrix.com] >Sent: 10 February 2014 17:16 >To: >dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org%3cmailto:dev@cloudstack.apache.org>> >Subject: Re: S3 based secondary Storage > >Can you try with s3cmd? If still not working, please post your .s3cfg >file. > >Thanks >-min > >On 2/10/14 12:41 AM, "Paul Angus" ><paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com%3cmailto:paul.an...@shapeblue.com>>> > wrote: > >Yes, I'm using S3Browser, no problem uploading/downloading files > >Regards, > >Paul Angus >Cloud Architect >S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus >paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com%3cmailto:paul.an...@shapeblue.com>> > >-----Original Message----- >From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] >Sent: 10 February 2014 08:38 >To: >dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org%3cmailto:dev@cloudstack.apache.org>> >Subject: RE: S3 based secondary Storage > >Hi, > >To make sure everything is working fine with S3 , try to upload a file >to >s3 storage using any s3client and use the same credentials. > >-Sanjeev > >From: Paul Angus [mailto:paul.an...@shapeblue.com] >Sent: Sunday, February 09, 2014 1:50 AM >To: >dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org%3cmailto:dev@cloudstack.apache.org>> >Subject: S3 based secondary Storage > >Hi Guys, > >Any idea what's happening here? (see last two lines of log). > >I haven't seeded sec storage, because it's going to be S3 based. > >Unless you have to create NFS first and then migrate to it? But then >the option to 'not create nfs staging area' doesn't make sense... > >2014-02-08 20:11:16,770 DEBUG [c.c.a.ApiServlet] >(catalina-exec-11:ctx-13ef9d35 ctx-c6f8549a) ===END=== 10.0.0.17 -- >GET >command=addImageStore&response=json&sessionkey=xqZoRDVpFezJkBlS3vqGZQzU >ytY >%3D&name=SecStor-S3&provider=S3&details%5B0%5D.key=accesskey&details%5B >0%5 >D.value=<key>&details%5B1%5D.key=secretkey&details%5B1%5D.value=<key>&d >eta >ils%5B2%5D.key=bucket&details%5B2%5D.value=cloudstack-sec-stor&details% >5B3 >%5D.key=usehttps&details%5B3%5D.value=false&details%5B4%5D.key=endpoint >&de >tails%5B4%5D.value=s3-website-eu-west-1.amazonaws.com&_=1391890278321 > >2014-02-08 20:11:26,925 INFO [c.c.s.t.S3TemplateDownloader] >(pool-10-thread-1:ctx-045258b1) No credentials configured for >host=download.cloud.com:80 >2014-02-08 20:11:27,579 INFO [c.c.s.t.S3TemplateDownloader] >(pool-9-thread-1:ctx-efcb9d5b) Starting download from >http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-m >ast >er-xen.vhd.bz2 to s3 bucket cloudstack-sec-stor remoteSize=236599855 , >max size=53687091200 > >2014-02-08 20:11:27,580 DEBUG [c.c.u.S3Utils] >(pool-9-thread-1:ctx-efcb9d5b) Sending stream as S3 object using >PutObjectRequest >2014-02-08 20:11:27,580 DEBUG [c.c.u.S3Utils] >(pool-9-thread-1:ctx-efcb9d5b) Creating S3 client with configuration: >[protocol: http, connectionTimeOut: 50000, maxErrorRetry: 3, >socketTimeout: 50000] >2014-02-08 20:11:27,588 DEBUG [c.c.u.S3Utils] >(pool-9-thread-1:ctx-efcb9d5b) Setting the end point for S3 client >com.amazonaws.services.s3.AmazonS3Client@756ff826<mailto:com.amazonaws<mailto:com.amazonaws.services.s3.AmazonS3Client@756ff826%3cmailto:com.amazonaws>. >ser vices.s3.AmazonS3Client@756ff826<mailto:vices.s3.AmazonS3Client@756ff826>> >to >s3-website-eu-west-1.amazonaws.com. > >2014-02-08 20:13:18,734 INFO [o.a.c.s.t.DownloadManagerImpl] >(pool-9-thread-1:ctx-efcb9d5b) Download Completion for jobId: >30aad9b0-a968-4896-bf2c-1d5e31973934, status=UNRECOVERABLE_ERROR >2014-02-08 20:13:18,735 INFO [o.a.c.s.t.DownloadManagerImpl] >(pool-9-thread-1:ctx-efcb9d5b) local: >template/tmpl/1/1/routing-1/systemvm64template-2014-01-14-master-xen.vh >d.b >z2, bytes=0, error=null, pct=0 > > >Regards, > >Paul Angus >Senior Consultant / Cloud Architect >[cid:image003.png@01CEF0F0.9C9104D0<mailto:image003.png@01CEF0F0.9C9104 >D0> >] > >S: +44 20 3603 0540<tel:+442036030540> | M: >+4<tel:+447968161581>47711418784 | T: @CloudyAngus >paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com><mailto:paul.a<mailto:paul.an...@shapeblue.com%3cmailto:paul.an...@shapeblue.com%3e%3cmailto:paul.a> >ngu s...@shapeblue.com<mailto:s...@shapeblue.com>> | >www.shapeblue.com<htp://www.shapeblue.com/<http://www.shapeblue.com%3chtp:/www.shapeblue.com/>> > | >Twitter:@shapeblue<https://twitter.com/> >ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS > >Need Enterprise Grade Support for Apache CloudStack? >Our CloudStack Infrastructure >Support<http://shapeblue.com/cloudstack-infrastructure-support/> offers >the best 24/7 SLA for CloudStack Environments. > >Apache CloudStack Bootcamp training courses > >**NEW!** CloudStack 4.2.1 >training<http://shapeblue.com/cloudstack-training/> >18th-19th February 2014, Brazil. >Classroom<http://shapeblue.com/cloudstack-training/> >17th-23rd March 2014, Region A. Instructor led, >On-line<http://shapeblue.com/cloudstack-training/> >24th-28th March 2014, Region B. Instructor led, >On-line<http://shapeblue.com/cloudstack-training/> >16th-20th June 2014, Region A. Instructor led, >On-line<http://shapeblue.com/cloudstack-training/> >23rd-27th June 2014, Region B. Instructor led, >On-line<http://shapeblue.com/cloudstack-training/> > >This email and any attachments to it may be confidential and are >intended solely for the use of the individual to whom it is addressed. >Any views or opinions expressed are solely those of the author and do >not necessarily represent those of Shape Blue Ltd or related companies. >If you are not the intended recipient of this email, you must neither >take any action based upon its contents, nor copy or show it to anyone. >Please contact the sender if you believe you have received this email >in error. >Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >Services India LLP is a company incorporated in India and is operated >under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda >is a company incorporated in Brasil and is operated under license from >Shape Blue Ltd. ShapeBlue is a registered trademark. >This email and any attachments to it may be confidential and are >intended solely for the use of the individual to whom it is addressed. >Any views or opinions expressed are solely those of the author and do >not necessarily represent those of Shape Blue Ltd or related companies. >If you are not the intended recipient of this email, you must neither >take any action based upon its contents, nor copy or show it to anyone. >Please contact the sender if you believe you have received this email >in error. >Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >Services India LLP is a company incorporated in India and is operated >under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda >is a company incorporated in Brasil and is operated under license from >Shape Blue Ltd. ShapeBlue is a registered trademark. > >This email and any attachments to it may be confidential and are >intended solely for the use of the individual to whom it is addressed. >Any views or opinions expressed are solely those of the author and do >not necessarily represent those of Shape Blue Ltd or related companies. >If you are not the intended recipient of this email, you must neither >take any action based upon its contents, nor copy or show it to anyone. >Please contact the sender if you believe you have received this email in error. >Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >Services India LLP is a company incorporated in India and is operated >under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda >is a company incorporated in Brasil and is operated under license from >Shape Blue Ltd. ShapeBlue is a registered trademark. > >This email and any attachments to it may be confidential and are >intended solely for the use of the individual to whom it is addressed. >Any views or opinions expressed are solely those of the author and do >not necessarily represent those of Shape Blue Ltd or related companies. >If you are not the intended recipient of this email, you must neither >take any action based upon its contents, nor copy or show it to anyone. >Please contact the sender if you believe you have received this email in error. >Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >Services India LLP is a company incorporated in India and is operated >under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda >is a company incorporated in Brasil and is operated under license from >Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
