Sanjeev, Thanks for the confirmation.
Jessica -----Original Message----- From: Sanjeev Neelarapu Sent: Thursday, July 18, 2013 11:05 PM To: dev@cloudstack.apache.org; Jessica Wang Cc: Edison Su; Thomas O'Dowd Subject: RE: Query String Request Authentication(QSRA) support by S3 providers Hi, I have not seen the issue in extract template/iso after the changes. Works fine. Thanks, Sanjeev -----Original Message----- From: Min Chen [mailto:min.c...@citrix.com] Sent: Friday, July 19, 2013 5:08 AM To: Jessica Wang Cc: dev@cloudstack.apache.org; Edison Su; Thomas O'Dowd Subject: Re: Query String Request Authentication(QSRA) support by S3 providers Thanks Jessica. Tom, did you still see the issue? -min On 7/8/13 1:20 PM, "Jessica Wang" <jessica.w...@citrix.com> wrote: >Min, > >> would you please take a look at this to see if UI can disable >>decoding in displaying this download template url returned from API? > >I just changed UI to not decode the URL returned in extractTemplate, >extractIso API. > >Jessica > > >-----Original Message----- >From: Min Chen >Sent: Wednesday, July 03, 2013 5:53 PM >To: dev@cloudstack.apache.org; Thomas O'Dowd >Cc: Jessica Wang >Subject: Re: Query String Request Authentication(QSRA) support by S3 >providers > >Jessica, would you please take a look at this to see if UI can disable >decoding in displaying this download template url returned from API? > >Thanks >-min > >On 7/3/13 5:38 PM, "Min Chen" <min.c...@citrix.com> wrote: > >>By examining further what returned from extractTemplateCmd api, I >>realized that the URL returned from API is different from what is >>displayed from pop-up dialog from UI. Directly using the link returned >>from API (with / encoded as %2F can successfully download the >>template. So the issue may not be that bad to upgrade Amazon SDK, but >>a simple UI issue. That is, is it possible for UI not to decode >>URLEncoded string in this case? >> >>Thanks >>-min >> >>On 7/3/13 5:22 PM, "Min Chen" <min.c...@citrix.com> wrote: >> >>>Hi Tom, >>> >>> I can reproduce this issue using Cloudian, after investigation, I >>>realized that this is a bug in Amazon SDK we have used, based on this >>>thread: >>>http://stackoverflow.com/questions/15473582/amazon-s3-presigned-urls- >>>esc >>>a >>>p >>>e >>>-the-slashes-in-the-key. When generatePresignedUrl is called it takes >>>the entire key and escapes it, and then creates a signature using the >>>escaped key. You cannot use the signature from the escaped key and >>>combine it with the unescaped key in the URL. See the bug code here: >>> >>> String resourcePath = "/" + >>> ((bucketName != null) ? bucketName + "/" : "") + >>> ((key != null) ? ServiceUtils.urlEncode(key) : "") + >>> ((subResource != null) ? "?" + subResource : ""); >>> >>>We have two options to fix this: >>> 1. Either upgrade Amazon SDK to use 1.4.3 version, someone in that >>>thread claimed that it is fixed in that version, but I haven't >>>checked that. >>>Currently CloudStack is using 1.3.21. Not sure if this will break >>>CloudStack cloud_bridge. >>> 2. Workaround by creating customized AmazonS3Client to change the >>>internal implementation on this. >>> >>> Thanks >>> -min >>> >>> >>> >>>On 7/2/13 11:31 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote: >>> >>>>Excellent. The link is there now. Thank you Min. I verified that bug >>>>and closed it. >>>> >>>>However - now that I can finally click the download link... I ran >>>>into the issue that the link doesn't work on AWS or Cloudian. Please >>>>see this bug for details (latest 4.2 updates included in my test). >>>> >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3341 >>>> >>>>Tom. >>>> >>>>On Tue, 2013-07-02 at 22:54 +0000, Min Chen wrote: >>>>> Hi Tom, >>>>> I investigated this issue through the db dump you provided in the >>>>>bug, this is an issue with our db view template_view creation >>>>>script, and it has been fixed in resolving another bug >>>>>(https://issues.apache.org/jira/browse/CLOUDSTACK-3314). >>>>>I >>>>> have verified the fix using your db dump on my local setup. Please >>>>>check out latest 4.2 or master code to try again. >>>>> >>>>> Thanks >>>>> -min >>>>> >>>>> On 7/2/13 2:18 PM, "Min Chen" <min.c...@citrix.com> wrote: >>>>> >>>>> >Tom, this seems like an issue with entry stored in our DB. I will >>>>>take >>>>>a >>>>> >look at this bug and update you. Just to clarify, this symptom >>>>> >only happens when you register these templates to Amazon S3, not >>>>> >for >>>>>Cloudian >>>>> >or RiakCS S3, right? >>>>> > >>>>> >Thanks >>>>> >-min >>>>> > >>>>> >On 7/1/13 7:27 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote: >>>>> > >>>>> >>Yes thanks Jessica. I re-opened the bug again. I know its not a >>>>> >>gui problem per-say in that the template is not ready to show >>>>> >>the >>>>>download >>>>> >>link. However, it never becomes ready is the actual problem. >>>>> >>What >>>>>sets >>>>> >>the "isready" property to true? As far as I can see, the objects >>>>> >>in >>>>>the >>>>> >>S3 stores (AWS or Cloudian) are complete and from my perspective >>>>>"ready" >>>>> >>to download/use. It sounds like a bug when registering the >>>>>template. >>>>> >> >>>>> >>Tom. >>>>> >> >>>>> >>On Mon, 2013-07-01 at 18:54 +0000, Jessica Wang wrote: >>>>> >>> Thomas, >>>>> >>> >>>>> >>> I checked the data you provided. >>>>> >>> >>>>> >>> The reason that the 2 templates("MyTiny", "AnotherTiny") have >>>>> >>>no download button is because they are not ready (i.e. their >>>>> >>>"isready" property is false). >>>>> >>> >>>>> >>> Download button is only available when "isready" property is >>>>>true. >>>>> >>> >>>>> >>> Jessica >>>>> >>> >>>>> >>> -----Original Message----- >>>>> >>> From: Thomas O'Dowd [mailto:tpod...@cloudian.com] >>>>> >>> Sent: Thursday, June 27, 2013 8:04 PM >>>>> >>> To: Min Chen >>>>> >>> Cc: dev@cloudstack.apache.org; Jessica Wang >>>>> >>> Subject: Re: Query String Request Authentication(QSRA) support >>>>> >>> by >>>>>S3 >>>>> >>>providers >>>>> >>> >>>>> >>> Hi Min/Jessica, >>>>> >>> >>>>> >>> I attached an image to that issue to show what what my browser >>>>> >>> is showing. >>>>> >>> >>>>> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-3220 >>>>> >>> >>>>> >>> Tom. >>>>> >>> >>>>> >>> On Fri, 2013-06-28 at 09:45 +0900, Thomas O'Dowd wrote: >>>>> >>> > Hi Min, >>>>> >>> > >>>>> >>> > Yes. I'll try it again today to check again but when I added >>>>>Amazon >>>>> >>>S3 >>>>> >>> > as the S3 secondary storage and uploaded a template, I was >>>>> >>> > not >>>>>shown >>>>> >>>the >>>>> >>> > "download template" link. However - for Cloudian S3, I am >>>>> >>> > shown >>>>>it so >>>>> >>> > I'm wondering why. >>>>> >>> > >>>>> >>> > Tom. >>>>> >>> > >>>>> >>> > On Fri, 2013-06-28 at 00:26 +0000, Min Chen wrote: >>>>> >>> > > Hi Tom, >>>>> >>> > > >>>>> >>> > > Are you saying that you cannot see a Download Template >>>>>button >>>>>from >>>>> >>>UI >>>>> >>> > > when Amazon S3 is added as secondary storage? I only >>>>> >>> > > tested >>>>>with >>>>> >>>RiakCS >>>>> >>> > > and Cloudian, so didn't see this issue. But I am CC >>>>> >>> > > Jessica >>>>>her >>>>>to >>>>> >>>confirm >>>>> >>> > > what special handling is done in UI to enable/disable a >>>>>button >>>>>from >>>>> >>>UI. >>>>> >>> > > >>>>> >>> > > Thanks >>>>> >>> > > -min >>>>> >>> > > >>>>> >>> > > On 6/27/13 5:23 PM, "Thomas O'Dowd" <tpod...@cloudian.com> >>>>>wrote: >>>>> >>> > > >>>>> >>> > > >Hi Min, >>>>> >>> > > > >>>>> >>> > > >Can you check this bug? I'm trying to test this feature >>>>> >>> > > >for >>>>>Amazon >>>>> >>>but >>>>> >>> > > >having no luck getting the Download template link/button >>>>> >>> > > >to >>>>> >>>appear. >>>>> >>> > > > >>>>> >>> > > >https://issues.apache.org/jira/browse/CLOUDSTACK-3220 >>>>> >>> > > > >>>>> >>> > > >Thanks, >>>>> >>> > > > >>>>> >>> > > >Tom. >>>>> >>> > > > >>>>> >>> > > >On Fri, 2013-06-21 at 17:21 +0000, Min Chen wrote: >>>>> >>> > > >> John, >>>>> >>> > > >> >>>>> >>> > > >> For S3, the api call createEntityExtractUrl is done on >>>>> >>>management >>>>> >>> > > >>server >>>>> >>> > > >> side; while for NFS secondary storage, if the >>>>>implementation >>>>>of >>>>> >>> > > >> createEntityExtractUrl will involve some code be >>>>> >>> > > >> executed >>>>>in >>>>> >>>ssvm to >>>>> >>> > > >>copy >>>>> >>> > > >> template from the install location to a public >>>>> >>> > > >>accessible >>>>>web >>>>> >>>server >>>>> >>> > > >> location. >>>>> >>> > > >> I don't quite understand some of your comments below. >>>>>This >>>>>API >>>>> >>>is not >>>>> >>> > > >> used to write any information to S3 bucket/directory. >>>>> >>> > > >> This >>>>>is >>>>> >>>used for >>>>> >>> > > >> object already existed on S3, and we just provide a URL >>>>>for >>>>>user >>>>> >>>to >>>>> >>> > > >> download a template from S3, just like how Amazon >>>>> >>> > > >> provided >>>>>user >>>>> >>>a way to >>>>> >>> > > >> user to extract a S3 object through generatePresignedUrl. >>>>>We >>>>>can >>>>> >>>discuss >>>>> >>> > > >> more on this on collaboration conference. >>>>> >>> > > >> >>>>> >>> > > >> Thanks >>>>> >>> > > >> -min >>>>> >>> > > >> >>>>> >>> > > >> >>>>> >>> > > >> >>>>> >>> > > >> On 6/21/13 7:25 AM, "John Burwell" <jburw...@basho.com> >>>>>wrote: >>>>> >>> > > >> >>>>> >>> > > >> >Min, >>>>> >>> > > >> > >>>>> >>> > > >> >(I apologize for my belated reply -- I lost track of >>>>> >>> > > >> >this >>>>>draft >>>>> >>>in the >>>>> >>> > > >> >chaos of the last couple of days.) >>>>> >>> > > >> > >>>>> >>> > > >> >Upon further review, I think I feel into the confusion >>>>>between >>>>> >>> > > >>management >>>>> >>> > > >> >server and ssvm. This code is executing on the >>>>>management >>>>> >>>server side, >>>>> >>> > > >> >correct? Based on my "corrected" understanding is >>>>>correct, >>>>>I >>>>> >>>would >>>>> >>> > > >>like >>>>> >>> > > >> >to amend my thoughts. Namely, I would like to see the >>>>>driver >>>>> >>> > > >>operations >>>>> >>> > > >> >pushed out to the SSVM where we can use the stream. >>>>> >>> > > >> >As I >>>>>think >>>>> >>>about >>>>> >>> > > >>it, >>>>> >>> > > >> >the management server should not need to interact with >>>>>the >>>>> >>>driver. >>>>> >>> > > >> >Simply yard up the DataStore attributes + details map >>>>> >>> > > >> >and >>>>>other >>>>> >>>extract >>>>> >>> > > >> >parameters, and send them to the SSVM. Using this >>>>>information, >>>>> >>>the S3 >>>>> >>> > > >> >driver could open a stream to write the template out >>>>> >>> > > >> >to >>>>>the >>>>> >>> > > >> >bucket/directory. I recognize it changes the protocol >>>>>between >>>>> >>>the >>>>> >>> > > >> >management server and SSVM, but it simply both sides >>>>> >>> > > >> >of >>>>>the >>>>> >>>operation >>>>> >>> > > >>by >>>>> >>> > > >> >allowing the DataStore information to be treated >>>>> >>> > > >> >opaquely >>>>>until >>>>> >>>it is >>>>> >>> > > >> >consumed by the driver to execute the write operation. >>>>> >>> > > >> >I >>>>>also >>>>> >>> > > >>recognize >>>>> >>> > > >> >that we may a little late in the cycle to address it >>>>> >>> > > >> >for >>>>>4.2, >>>>> >>>and it >>>>> >>> > > >>may >>>>> >>> > > >> >need to be part of the 4.3 enhancements. >>>>> >>> > > >> > >>>>> >>> > > >> >Thanks, >>>>> >>> > > >> >-John >>>>> >>> > > >> > >>>>> >>> > > >> >On Jun 18, 2013, at 3:55 PM, Min Chen >>>>><min.c...@citrix.com> >>>>> >>>wrote: >>>>> >>> > > >> > >>>>> >>> > > >> >> John, >>>>> >>> > > >> >> In that case, how do we keep backward compatibility >>>>> >>> > > >> >> of >>>>> >>> > > >>extractTemplate >>>>> >>> > > >> >> api, which requires a URL in the response? >>>>> >>> > > >> >> >>>>> >>> > > >> >> Thanks >>>>> >>> > > >> >> -min >>>>> >>> > > >> >> >>>>> >>> > > >> >> On 6/18/13 11:53 AM, "John Burwell" >>>>><jburw...@basho.com> >>>>> >>>wrote: >>>>> >>> > > >> >> >>>>> >>> > > >> >>> Min, >>>>> >>> > > >> >>> >>>>> >>> > > >> >>> Looking through the code, I think we can simplify >>>>>driver >>>>> >>>operation >>>>> >>> > > >>and >>>>> >>> > > >> >>> increase robustness by changing >>>>> >>> > > >> >>>ImageStoreDriver#createEntityExtractUrl() >>>>> >>> > > >> >>> : String to ImageStoreDriver#readEntity(Š) : >>>>>InputStream. >>>>> >>>My first >>>>> >>> > > >> >>> concern with the current implementation is that it >>>>> >>>circumvents any >>>>> >>> > > >> >>> connection pooling/resource management underlying >>>>>client >>>>> >>>libraries >>>>> >>> > > >> >>> provide. I/O streams provide a higher-level >>>>>abstraction >>>>> >>>that allows >>>>> >>> > > >> >>> drivers to provide the orchestration components >>>>> >>> > > >> >>> with >>>>>actual >>>>> >>> > > >>resources >>>>> >>> > > >> >>> rather String references. Second, the current >>>>>interface >>>>> >>>seems to >>>>> >>> > > >> >>>appears >>>>> >>> > > >> >>> to assume that an http/https URL will be returned. >>>>>With >>>>>I/O >>>>> >>> > > >>streams, >>>>> >>> > > >> >>>we >>>>> >>> > > >> >>> can support any client library capable of using the >>>>>standard >>>>> >>>I/O >>>>> >>> > > >> >>> framework -- enabling us to support other protocols >>>>>for >>>>> >>>downloading >>>>> >>> > > >> >>> templates in the future (e.g. RBD, local >>>>> >>> > > >> >>> filesystem, >>>>>NBD, >>>>> >>>etc). >>>>> >>> > > >> >>> >>>>> >>> > > >> >>> Thanks, >>>>> >>> > > >> >>> -John >>>>> >>> > > >> >>> >>>>> >>> > > >> >>> On Jun 18, 2013, at 1:11 PM, Min Chen >>>>><min.c...@citrix.com> >>>>> >>>wrote: >>>>> >>> > > >> >>> >>>>> >>> > > >> >>>> A new version of using generatePresignedUrl in >>>>> >>> > > >>S3ImageStoreDriverImpl >>>>> >>> > > >> >>>>is >>>>> >>> > > >> >>>> checked into object_store. >>>>> >>> > > >> >>>> >>>>> >>> > > >> >>>> THanks >>>>> >>> > > >> >>>> -min >>>>> >>> > > >> >>>> >>>>> >>> > > >> >>>> On 6/18/13 8:29 AM, "Min Chen" >>>>> >>> > > >> >>>> <min.c...@citrix.com> >>>>>wrote: >>>>> >>> > > >> >>>> >>>>> >>> > > >> >>>>> Yes, current code is in >>>>> >>> > > >> >>>>>S3ImageStoreDriverImpl.createEntityExtractUrl, >>>>> >>> > > >> >>>>> which has a security issue mentioned in >>>>>CLOUDSTACK-3030. I >>>>> >>>am >>>>> >>> > > >>going >>>>> >>> > > >> >>>>>to >>>>> >>> > > >> >>>>> change it to use generatePresignedUrl api from >>>>> >>> > > >> >>>>>AWS >>>>>S3 >>>>>api. >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Thanks >>>>> >>> > > >> >>>>> -min >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> From: John Burwell >>>>> >>><jburw...@basho.com<mailto:jburw...@basho.com>> >>>>> >>> > > >> >>>>> Date: Tuesday, June 18, 2013 8:07 AM >>>>> >>> > > >> >>>>> To: Min Chen >>>>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>> >>>>> >>> > > >> >>>>> Cc: Thomas O'Dowd >>>>> >>> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com> >>>>> >>> > > >> >>>>>>, >>>>> >>> > > >> >>>>> >>>>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >>>>> >>> > > >> >>>>> >>>>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >>>>> >>> > > >> >>>>> Subject: Re: Query String Request >>>>>Authentication(QSRA) >>>>> >>>support by >>>>> >>> > > >>S3 >>>>> >>> > > >> >>>>> providers >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Min, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Is the code checked into the object_store branch? >>>>>If >>>>>so, >>>>> >>>which >>>>> >>> > > >>lines >>>>> >>> > > >> >>>>> in >>>>> >>> > > >> >>>>> S3TemplateDownloader? >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Thanks, >>>>> >>> > > >> >>>>> -John >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> On Jun 18, 2013, at 12:39 AM, Min Chen >>>>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> >>>>>wrote: >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Hi John, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> This is regarding extractTemplate api, where for >>>>> >>>extractable >>>>> >>> > > >> >>>>>template, >>>>> >>> > > >> >>>>> users can click "Download Template" button from >>>>> >>> > > >> >>>>>UI >>>>>to >>>>>get >>>>> >>>a http >>>>> >>> > > >>url >>>>> >>> > > >> >>>>>to >>>>> >>> > > >> >>>>> download the template already stored at S3 >>>>> >>> > > >> >>>>>without >>>>> >>>providing S3 >>>>> >>> > > >> >>>>> credentials. In 4.1, we don't have this issue, >>>>> >>> > > >> >>>>> since >>>>>the >>>>> >>>URL >>>>> >>> > > >>returned >>>>> >>> > > >> >>>>> is >>>>> >>> > > >> >>>>> the public web server location hosted in ssvm, >>>>> >>> > > >> >>>>> and >>>>>in >>>>>4.2, >>>>> >>>we are >>>>> >>> > > >> >>>>> returning URL pointing to s3 object. Without >>>>> >>> > > >> >>>>> setting >>>>>ACL >>>>> >>>to the S3 >>>>> >>> > > >> >>>>> object, user cannot directly click the URL >>>>> >>> > > >> >>>>> returned >>>>>from >>>>> >>> > > >> >>>>> extractTemplate >>>>> >>> > > >> >>>>> api to download the template without providing >>>>> >>>credentials. By >>>>> >>> > > >> >>>>>reading >>>>> >>> > > >> >>>>> the AWS SDK doc today, I ran across the following >>>>>API >>>>>that >>>>> >>>I may >>>>> >>> > > >>be >>>>> >>> > > >> >>>>> able >>>>> >>> > > >> >>>>> to use for this purpose: >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL. >>>>>>>>>>>>>>>htm >>>>>>>>>>>>>>>l >>>>>>>>>>>>>>>? >>>>>>>>>>>>>>>i >>>>>>>>>>>>>>>s- >>>>> >>>>>>>>>>e >>>>> >>>>>>>>>>xt >>>>> >>> > > >>>>>>>er >>>>> >>> > > >> >>>>>na >>>>> >>> > > >> >>>>> l= >>>>> >>> > > >> >>>>> true> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSD >>>>>>>>>>>>>>>K/l >>>>>>>>>>>>>>>a >>>>>>>>>>>>>>>t >>>>>>>>>>>>>>>e >>>>>>>>>>>>>>>st >>>>> >>>>>>>>>>/ >>>>> >>>>>>>>>>ja >>>>> >>> > > >>>>>>>va >>>>> >>> > > >> >>>>>do >>>>> >>> > > >> >>>>> c/ >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>com/amazonaws/services/s3/AmazonS3Client.html#generatePre >>>>>>>>>>>>>>>sig >>>>>>>>>>>>>>>n >>>>>>>>>>>>>>>e >>>>>>>>>>>>>>>d >>>>>>>>>>>>>>>Ur >>>>> >>>>>>>>>>l >>>>> >>>>>>>>>>%2 >>>>> >>> > > >>>>>>>8j >>>>> >>> > > >> >>>>>av >>>>> >>> > > >> >>>>> a. >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>lang.String,%20java.lang.String,%20java.util.Date,%20com. >>>>>>>>>>>>>>>ama >>>>>>>>>>>>>>>z >>>>>>>>>>>>>>>o >>>>>>>>>>>>>>>n >>>>>>>>>>>>>>>aw >>>>> >>>>>>>>>>s >>>>> >>>>>>>>>>.H >>>>> >>> > > >>>>>>>tt >>>>> >>> > > >> >>>>>pM >>>>> >>> > > >> >>>>> et >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/ja >>>>>>>>>>>>>>>va/ >>>>>>>>>>>>>>>l >>>>>>>>>>>>>>>a >>>>>>>>>>>>>>>n >>>>>>>>>>>>>>>g/ >>>>> >>>>>>>>>>S >>>>> >>>>>>>>>>tr >>>>> >>> > > >>>>>>>in >>>>> >>> > > >> >>>>>g. >>>>> >>> > > >> >>>>> ht >>>>> >>> > > >> >>>>> ml?is-external=true> bucketName, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ >>>>>>>>>>>>>>>Str >>>>>>>>>>>>>>>i >>>>>>>>>>>>>>>n >>>>>>>>>>>>>>>g >>>>>>>>>>>>>>>.h >>>>> >>>>>>>>>>t >>>>> >>>>>>>>>>ml >>>>> >>> > > >>>>>>>?i >>>>> >>> > > >> >>>>>s- >>>>> >>> > > >> >>>>> ex >>>>> >>> > > >> >>>>> ternal=true> key, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date. >>>>>>>>>>>>>>>h >>>>>>>>>>>>>>>t >>>>>>>>>>>>>>>m >>>>>>>>>>>>>>>l? >>>>> >>>>>>>>>>i >>>>> >>>>>>>>>>s- >>>>> >>> > > >>>>>>>ex >>>>> >>> > > >> >>>>>te >>>>> >>> > > >> >>>>> rn >>>>> >>> > > >> >>>>> al=true> expiration, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>> > > >>>>> >>>>>>>>>>>>>>>HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/j >>>>>>>>>>>>>>>ava >>>>>>>>>>>>>>>d >>>>>>>>>>>>>>>o >>>>>>>>>>>>>>>c >>>>>>>>>>>>>>>/c >>>>> >>>>>>>>>>o >>>>> >>>>>>>>>>m/ >>>>> >>> > > >>>>>>>am >>>>> >>> > > >> >>>>>az >>>>> >>> > > >> >>>>> on >>>>> >>> > > >> >>>>> aws/HttpMethod.html> method) >>>>> >>> > > >> >>>>> Returns a pre-signed URL for accessing an >>>>>Amazon >>>>> >>>S3 >>>>> >>> > > >>resource. >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> This is along the same line as QSRA mentioned by >>>>>Tom, >>>>>by >>>>> >>>wrapped >>>>> >>> > > >>in >>>>> >>> > > >> >>>>> AmazonS3Client for easy consumption. By using >>>>> >>> > > >> >>>>> this >>>>>method, >>>>> >>>I think >>>>> >>> > > >> >>>>> that I >>>>> >>> > > >> >>>>> don't need to change ACL of S3 object to open a >>>>>security >>>>> >>>hole. >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Thanks >>>>> >>> > > >> >>>>> -min >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> From: John Burwell >>>>> >>><jburw...@basho.com<mailto:jburw...@basho.com>> >>>>> >>> > > >> >>>>> Date: Monday, June 17, 2013 7:38 PM >>>>> >>> > > >> >>>>> To: Min Chen >>>>> >>><min.c...@citrix.com<mailto:min.c...@citrix.com>> >>>>> >>> > > >> >>>>> Cc: Thomas O'Dowd >>>>> >>> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com> >>>>> >>> > > >> >>>>>>, >>>>> >>> > > >> >>>>> >>>>> >>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >>>>> >>> > > >> >>>>> >>>>> >>><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >>>>> >>> > > >> >>>>> Subject: Re: Query String Request >>>>>Authentication(QSRA) >>>>> >>>support by >>>>> >>> > > >>S3 >>>>> >>> > > >> >>>>> providers >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Min, >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Why are we mucking with ACLs at all? The best >>>>>security >>>>> >>>practice >>>>> >>> > > >> >>>>>would >>>>> >>> > > >> >>>>> be >>>>> >>> > > >> >>>>> to create a bucket for CloudStack's use and >>>>> >>> > > >> >>>>>assign >>>>>it >>>>>a >>>>> >>>dedicated >>>>> >>> > > >> >>>>> access >>>>> >>> > > >> >>>>> key and secret key pair with read/write access >>>>> >>> > > >> >>>>> only >>>>>to >>>>> >>>that >>>>> >>> > > >>bucket. >>>>> >>> > > >> >>>>> Requiring an administrative account to an object >>>>>store >>>>> >>>opens an >>>>> >>> > > >> >>>>> unnecessarily large attack surface. Therefore, >>>>> >>> > > >> >>>>> as >>>>> >>>implemented in >>>>> >>> > > >> >>>>>4.1, >>>>> >>> > > >> >>>>> we >>>>> >>> > > >> >>>>> should defer bucket creation, ACL assignment, and >>>>> >>>credential >>>>> >>> > > >>creation >>>>> >>> > > >> >>>>> to >>>>> >>> > > >> >>>>> the administrator/operator. >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Thanks, >>>>> >>> > > >> >>>>> -John >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> On Jun 17, 2013, at 1:15 PM, Min Chen >>>>> >>> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> >>>>>wrote: >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Tom filed a very good bug for ACL setting change >>>>> >>> > > >> >>>>> on >>>>>S3 >>>>> >>>object when >>>>> >>> > > >> >>>>> users >>>>> >>> > > >> >>>>> issue extractTemplate API >>>>> >>> > > >> >>>>> >>>>>(https://issues.apache.org/jira/browse/CLOUDSTACK-3030), >>>>> >>>and his >>>>> >>> > > >> >>>>> recommendation of using Query String Request >>>>> >>>Authentication (QSRA) >>>>> >>> > > >> >>>>> alternative sounds like a right approach to fix >>>>> >>> > > >> >>>>> this >>>>>bug. >>>>> >>>Before >>>>> >>> > > >> >>>>> implementing it, I would like to confirm if QSRA >>>>>should be >>>>> >>> > > >>supported >>>>> >>> > > >> >>>>>by >>>>> >>> > > >> >>>>> all S3 providers if they claim that they are AWS >>>>> >>> > > >> >>>>>s3 >>>>> >>>compatible. If >>>>> >>> > > >> >>>>>so, >>>>> >>> > > >> >>>>> we >>>>> >>> > > >> >>>>> will make this assumption in our code. Based on >>>>> >>> > > >> >>>>>Tom, >>>>> >>>Cloudian is >>>>> >>> > > >> >>>>> supporting it. How about RiakCS, John? >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> Thanks >>>>> >>> > > >> >>>>> -min >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>>> >>>>> >>> > > >> >>>> >>>>> >>> > > >> >>> >>>>> >>> > > >> >> >>>>> >>> > > >> > >>>>> >>> > > >> >>>>> >>> > > > >>>>> >>> > > >-- >>>>> >>> > > >Cloudian KK - http://www.cloudian.com/get-started.html >>>>> >>> > > >Fancy 100TB of full featured S3 Storage? >>>>> >>> > > >Checkout the Cloudian(R) Community Edition! >>>>> >>> > > > >>>>> >>> > > >>>>> >>> > >>>>> >>> >>>>> >> >>>>> >>-- >>>>> >>Cloudian KK - http://www.cloudian.com/get-started.html >>>>> >>Fancy 100TB of full featured S3 Storage? >>>>> >>Checkout the Cloudian(R) Community Edition! >>>>> >> >>>>> > >>>>> >>>> >>>>-- >>>>Cloudian KK - http://www.cloudian.com/get-started.html >>>>Fancy 100TB of full featured S3 Storage? >>>>Checkout the Cloudian(R) Community Edition! >>>> >>> >> >
