Hi Min/Jessica,
I attached an image to that issue to show what what my browser is
showing.
https://issues.apache.org/jira/browse/CLOUDSTACK-3220
Tom.
On Fri, 2013-06-28 at 09:45 +0900, Thomas O'Dowd wrote:
> Hi Min,
>
> Yes. I'll try it again today to check again but when I added Amazon S3
> as the S3 secondary storage and uploaded a template, I was not shown the
> "download template" link. However - for Cloudian S3, I am shown it so
> I'm wondering why.
>
> Tom.
>
> On Fri, 2013-06-28 at 00:26 +0000, Min Chen wrote:
> > Hi Tom,
> >
> > Are you saying that you cannot see a Download Template button from UI
> > when Amazon S3 is added as secondary storage? I only tested with RiakCS
> > and Cloudian, so didn't see this issue. But I am CC Jessica her to confirm
> > what special handling is done in UI to enable/disable a button from UI.
> >
> > Thanks
> > -min
> >
> > On 6/27/13 5:23 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote:
> >
> > >Hi Min,
> > >
> > >Can you check this bug? I'm trying to test this feature for Amazon but
> > >having no luck getting the Download template link/button to appear.
> > >
> > >https://issues.apache.org/jira/browse/CLOUDSTACK-3220
> > >
> > >Thanks,
> > >
> > >Tom.
> > >
> > >On Fri, 2013-06-21 at 17:21 +0000, Min Chen wrote:
> > >> John,
> > >>
> > >> For S3, the api call createEntityExtractUrl is done on management
> > >>server
> > >> side; while for NFS secondary storage, if the implementation of
> > >> createEntityExtractUrl will involve some code be executed in ssvm to
> > >>copy
> > >> template from the install location to a public accessible web server
> > >> location.
> > >> I don't quite understand some of your comments below. This API is not
> > >> used to write any information to S3 bucket/directory. This is used for
> > >> object already existed on S3, and we just provide a URL for user to
> > >> download a template from S3, just like how Amazon provided user a way to
> > >> user to extract a S3 object through generatePresignedUrl. We can discuss
> > >> more on this on collaboration conference.
> > >>
> > >> Thanks
> > >> -min
> > >>
> > >>
> > >>
> > >> On 6/21/13 7:25 AM, "John Burwell" <jburw...@basho.com> wrote:
> > >>
> > >> >Min,
> > >> >
> > >> >(I apologize for my belated reply -- I lost track of this draft in the
> > >> >chaos of the last couple of days.)
> > >> >
> > >> >Upon further review, I think I feel into the confusion between
> > >>management
> > >> >server and ssvm. This code is executing on the management server side,
> > >> >correct? Based on my "corrected" understanding is correct, I would
> > >>like
> > >> >to amend my thoughts. Namely, I would like to see the driver
> > >>operations
> > >> >pushed out to the SSVM where we can use the stream. As I think about
> > >>it,
> > >> >the management server should not need to interact with the driver.
> > >> >Simply yard up the DataStore attributes + details map and other extract
> > >> >parameters, and send them to the SSVM. Using this information, the S3
> > >> >driver could open a stream to write the template out to the
> > >> >bucket/directory. I recognize it changes the protocol between the
> > >> >management server and SSVM, but it simply both sides of the operation
> > >>by
> > >> >allowing the DataStore information to be treated opaquely until it is
> > >> >consumed by the driver to execute the write operation. I also
> > >>recognize
> > >> >that we may a little late in the cycle to address it for 4.2, and it
> > >>may
> > >> >need to be part of the 4.3 enhancements.
> > >> >
> > >> >Thanks,
> > >> >-John
> > >> >
> > >> >On Jun 18, 2013, at 3:55 PM, Min Chen <min.c...@citrix.com> wrote:
> > >> >
> > >> >> John,
> > >> >> In that case, how do we keep backward compatibility of
> > >>extractTemplate
> > >> >> api, which requires a URL in the response?
> > >> >>
> > >> >> Thanks
> > >> >> -min
> > >> >>
> > >> >> On 6/18/13 11:53 AM, "John Burwell" <jburw...@basho.com> wrote:
> > >> >>
> > >> >>> Min,
> > >> >>>
> > >> >>> Looking through the code, I think we can simplify driver operation
> > >>and
> > >> >>> increase robustness by changing
> > >> >>>ImageStoreDriver#createEntityExtractUrl()
> > >> >>> : String to ImageStoreDriver#readEntity(Š) : InputStream. My first
> > >> >>> concern with the current implementation is that it circumvents any
> > >> >>> connection pooling/resource management underlying client libraries
> > >> >>> provide. I/O streams provide a higher-level abstraction that allows
> > >> >>> drivers to provide the orchestration components with actual
> > >>resources
> > >> >>> rather String references. Second, the current interface seems to
> > >> >>>appears
> > >> >>> to assume that an http/https URL will be returned. With I/O
> > >>streams,
> > >> >>>we
> > >> >>> can support any client library capable of using the standard I/O
> > >> >>> framework -- enabling us to support other protocols for downloading
> > >> >>> templates in the future (e.g. RBD, local filesystem, NBD, etc).
> > >> >>>
> > >> >>> Thanks,
> > >> >>> -John
> > >> >>>
> > >> >>> On Jun 18, 2013, at 1:11 PM, Min Chen <min.c...@citrix.com> wrote:
> > >> >>>
> > >> >>>> A new version of using generatePresignedUrl in
> > >>S3ImageStoreDriverImpl
> > >> >>>>is
> > >> >>>> checked into object_store.
> > >> >>>>
> > >> >>>> THanks
> > >> >>>> -min
> > >> >>>>
> > >> >>>> On 6/18/13 8:29 AM, "Min Chen" <min.c...@citrix.com> wrote:
> > >> >>>>
> > >> >>>>> Yes, current code is in
> > >> >>>>>S3ImageStoreDriverImpl.createEntityExtractUrl,
> > >> >>>>> which has a security issue mentioned in CLOUDSTACK-3030. I am
> > >>going
> > >> >>>>>to
> > >> >>>>> change it to use generatePresignedUrl api from AWS S3 api.
> > >> >>>>>
> > >> >>>>> Thanks
> > >> >>>>> -min
> > >> >>>>>
> > >> >>>>> From: John Burwell <jburw...@basho.com<mailto:jburw...@basho.com>>
> > >> >>>>> Date: Tuesday, June 18, 2013 8:07 AM
> > >> >>>>> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>>
> > >> >>>>> Cc: Thomas O'Dowd
> > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
> > >> >>>>> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
> > >> >>>>> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
> > >> >>>>> Subject: Re: Query String Request Authentication(QSRA) support by
> > >>S3
> > >> >>>>> providers
> > >> >>>>>
> > >> >>>>> Min,
> > >> >>>>>
> > >> >>>>> Is the code checked into the object_store branch? If so, which
> > >>lines
> > >> >>>>> in
> > >> >>>>> S3TemplateDownloader?
> > >> >>>>>
> > >> >>>>> Thanks,
> > >> >>>>> -John
> > >> >>>>>
> > >> >>>>> On Jun 18, 2013, at 12:39 AM, Min Chen
> > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote:
> > >> >>>>>
> > >> >>>>> Hi John,
> > >> >>>>>
> > >> >>>>> This is regarding extractTemplate api, where for extractable
> > >> >>>>>template,
> > >> >>>>> users can click "Download Template" button from UI to get a http
> > >>url
> > >> >>>>>to
> > >> >>>>> download the template already stored at S3 without providing S3
> > >> >>>>> credentials. In 4.1, we don't have this issue, since the URL
> > >>returned
> > >> >>>>> is
> > >> >>>>> the public web server location hosted in ssvm, and in 4.2, we are
> > >> >>>>> returning URL pointing to s3 object. Without setting ACL to the S3
> > >> >>>>> object, user cannot directly click the URL returned from
> > >> >>>>> extractTemplate
> > >> >>>>> api to download the template without providing credentials. By
> > >> >>>>>reading
> > >> >>>>> the AWS SDK doc today, I ran across the following API that I may
> > >>be
> > >> >>>>> able
> > >> >>>>> to use for this purpose:
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html?is-ext
> > >>>>>>>er
> > >> >>>>>na
> > >> >>>>> l=
> > >> >>>>> true>
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSDK/latest/ja
> > >>>>>>>va
> > >> >>>>>do
> > >> >>>>> c/
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>com/amazonaws/services/s3/AmazonS3Client.html#generatePresignedUrl%2
> > >>>>>>>8j
> > >> >>>>>av
> > >> >>>>> a.
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>lang.String,%20java.lang.String,%20java.util.Date,%20com.amazonaws.H
> > >>>>>>>tt
> > >> >>>>>pM
> > >> >>>>> et
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Str
> > >>>>>>>in
> > >> >>>>>g.
> > >> >>>>> ht
> > >> >>>>> ml?is-external=true> bucketName,
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html
> > >>>>>>>?i
> > >> >>>>>s-
> > >> >>>>> ex
> > >> >>>>> ternal=true> key,
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date.html?is-
> > >>>>>>>ex
> > >> >>>>>te
> > >> >>>>> rn
> > >> >>>>> al=true> expiration,
> > >> >>>>>
> > >> >>>>>
> > >>
> > >>>>>>>HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/
> > >>>>>>>am
> > >> >>>>>az
> > >> >>>>> on
> > >> >>>>> aws/HttpMethod.html> method)
> > >> >>>>> Returns a pre-signed URL for accessing an Amazon S3
> > >>resource.
> > >> >>>>>
> > >> >>>>> This is along the same line as QSRA mentioned by Tom, by wrapped
> > >>in
> > >> >>>>> AmazonS3Client for easy consumption. By using this method, I think
> > >> >>>>> that I
> > >> >>>>> don't need to change ACL of S3 object to open a security hole.
> > >> >>>>>
> > >> >>>>> Thanks
> > >> >>>>> -min
> > >> >>>>>
> > >> >>>>> From: John Burwell <jburw...@basho.com<mailto:jburw...@basho.com>>
> > >> >>>>> Date: Monday, June 17, 2013 7:38 PM
> > >> >>>>> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>>
> > >> >>>>> Cc: Thomas O'Dowd
> > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
> > >> >>>>> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
> > >> >>>>> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
> > >> >>>>> Subject: Re: Query String Request Authentication(QSRA) support by
> > >>S3
> > >> >>>>> providers
> > >> >>>>>
> > >> >>>>> Min,
> > >> >>>>>
> > >> >>>>> Why are we mucking with ACLs at all? The best security practice
> > >> >>>>>would
> > >> >>>>> be
> > >> >>>>> to create a bucket for CloudStack's use and assign it a dedicated
> > >> >>>>> access
> > >> >>>>> key and secret key pair with read/write access only to that
> > >>bucket.
> > >> >>>>> Requiring an administrative account to an object store opens an
> > >> >>>>> unnecessarily large attack surface. Therefore, as implemented in
> > >> >>>>>4.1,
> > >> >>>>> we
> > >> >>>>> should defer bucket creation, ACL assignment, and credential
> > >>creation
> > >> >>>>> to
> > >> >>>>> the administrator/operator.
> > >> >>>>>
> > >> >>>>> Thanks,
> > >> >>>>> -John
> > >> >>>>>
> > >> >>>>> On Jun 17, 2013, at 1:15 PM, Min Chen
> > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote:
> > >> >>>>>
> > >> >>>>> Tom filed a very good bug for ACL setting change on S3 object when
> > >> >>>>> users
> > >> >>>>> issue extractTemplate API
> > >> >>>>> (https://issues.apache.org/jira/browse/CLOUDSTACK-3030), and his
> > >> >>>>> recommendation of using Query String Request Authentication (QSRA)
> > >> >>>>> alternative sounds like a right approach to fix this bug. Before
> > >> >>>>> implementing it, I would like to confirm if QSRA should be
> > >>supported
> > >> >>>>>by
> > >> >>>>> all S3 providers if they claim that they are AWS s3 compatible. If
> > >> >>>>>so,
> > >> >>>>> we
> > >> >>>>> will make this assumption in our code. Based on Tom, Cloudian is
> > >> >>>>> supporting it. How about RiakCS, John?
> > >> >>>>>
> > >> >>>>> Thanks
> > >> >>>>> -min
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >> >
> > >>
> > >
> > >--
> > >Cloudian KK - http://www.cloudian.com/get-started.html
> > >Fancy 100TB of full featured S3 Storage?
> > >Checkout the Cloudian(R) Community Edition!
> > >
> >
>
--
Cloudian KK - http://www.cloudian.com/get-started.html
Fancy 100TB of full featured S3 Storage?
Checkout the Cloudian® Community Edition!
