Min, Looking through the code, I think we can simplify driver operation and increase robustness by changing ImageStoreDriver#createEntityExtractUrl() : String to ImageStoreDriver#readEntity(…) : InputStream. My first concern with the current implementation is that it circumvents any connection pooling/resource management underlying client libraries provide. I/O streams provide a higher-level abstraction that allows drivers to provide the orchestration components with actual resources rather String references. Second, the current interface seems to appears to assume that an http/https URL will be returned. With I/O streams, we can support any client library capable of using the standard I/O framework -- enabling us to support other protocols for downloading templates in the future (e.g. RBD, local filesystem, NBD, etc).
Thanks, -John On Jun 18, 2013, at 1:11 PM, Min Chen <min.c...@citrix.com> wrote: > A new version of using generatePresignedUrl in S3ImageStoreDriverImpl is > checked into object_store. > > THanks > -min > > On 6/18/13 8:29 AM, "Min Chen" <min.c...@citrix.com> wrote: > >> Yes, current code is in S3ImageStoreDriverImpl.createEntityExtractUrl, >> which has a security issue mentioned in CLOUDSTACK-3030. I am going to >> change it to use generatePresignedUrl api from AWS S3 api. >> >> Thanks >> -min >> >> From: John Burwell <jburw...@basho.com<mailto:jburw...@basho.com>> >> Date: Tuesday, June 18, 2013 8:07 AM >> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>> >> Cc: Thomas O'Dowd <tpod...@cloudian.com<mailto:tpod...@cloudian.com>>, >> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> Subject: Re: Query String Request Authentication(QSRA) support by S3 >> providers >> >> Min, >> >> Is the code checked into the object_store branch? If so, which lines in >> S3TemplateDownloader? >> >> Thanks, >> -John >> >> On Jun 18, 2013, at 12:39 AM, Min Chen >> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote: >> >> Hi John, >> >> This is regarding extractTemplate api, where for extractable template, >> users can click "Download Template" button from UI to get a http url to >> download the template already stored at S3 without providing S3 >> credentials. In 4.1, we don't have this issue, since the URL returned is >> the public web server location hosted in ssvm, and in 4.2, we are >> returning URL pointing to s3 object. Without setting ACL to the S3 >> object, user cannot directly click the URL returned from extractTemplate >> api to download the template without providing credentials. By reading >> the AWS SDK doc today, I ran across the following API that I may be able >> to use for this purpose: >> >> >> URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html?is-external= >> true> >> generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/ >> com/amazonaws/services/s3/AmazonS3Client.html#generatePresignedUrl%28java. >> lang.String,%20java.lang.String,%20java.util.Date,%20com.amazonaws.HttpMet >> hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.ht >> ml?is-external=true> bucketName, >> String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html?is-ex >> ternal=true> key, >> Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date.html?is-extern >> al=true> expiration, >> HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazon >> aws/HttpMethod.html> method) >> Returns a pre-signed URL for accessing an Amazon S3 resource. >> >> This is along the same line as QSRA mentioned by Tom, by wrapped in >> AmazonS3Client for easy consumption. By using this method, I think that I >> don't need to change ACL of S3 object to open a security hole. >> >> Thanks >> -min >> >> From: John Burwell <jburw...@basho.com<mailto:jburw...@basho.com>> >> Date: Monday, June 17, 2013 7:38 PM >> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>> >> Cc: Thomas O'Dowd <tpod...@cloudian.com<mailto:tpod...@cloudian.com>>, >> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" >> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> >> Subject: Re: Query String Request Authentication(QSRA) support by S3 >> providers >> >> Min, >> >> Why are we mucking with ACLs at all? The best security practice would be >> to create a bucket for CloudStack's use and assign it a dedicated access >> key and secret key pair with read/write access only to that bucket. >> Requiring an administrative account to an object store opens an >> unnecessarily large attack surface. Therefore, as implemented in 4.1, we >> should defer bucket creation, ACL assignment, and credential creation to >> the administrator/operator. >> >> Thanks, >> -John >> >> On Jun 17, 2013, at 1:15 PM, Min Chen >> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote: >> >> Tom filed a very good bug for ACL setting change on S3 object when users >> issue extractTemplate API >> (https://issues.apache.org/jira/browse/CLOUDSTACK-3030), and his >> recommendation of using Query String Request Authentication (QSRA) >> alternative sounds like a right approach to fix this bug. Before >> implementing it, I would like to confirm if QSRA should be supported by >> all S3 providers if they claim that they are AWS s3 compatible. If so, we >> will make this assumption in our code. Based on Tom, Cloudian is >> supporting it. How about RiakCS, John? >> >> Thanks >> -min >> >> >
