The Etag created by both RIAK CS and Amazon S3 seems a little bit different, in case of multi part upload.
Here is the result I tested on both RIAK CS and Amazon S3, with s3cmd. Test environment: S3cmd: version: version 1.5.0-alpha1 Riak cs: Name : riak Arch : x86_64 Version : 1.3.1 Release : 1.el6 Size : 40 M Repo : installed >From repo : basho-products The command I used to put: s3cmd put some-file s3://some-path --multipart-chunk-size-mb=100 -v -d The etag created for the file, when using Riak CS is WxEUkiQzTWm_2C8A92fLQg== EBUG: Sending request method_string='POST', uri='http://imagestore.s3.amazonaws.com/tmpl/1/1/routing-1/test?uploadId=kfDkh7Q_QCWN7r0ZTqNq4Q==', headers={'content-length': '309', 'Authorization': 'AWS OYAZXCAFUC1DAFOXNJWI:xlkHI9tUfUV/N+Ekqpi7Jz/pbOI=', 'x-amz-date': 'Thu, 06 Jun 2013 22:54:28 +0000'}, body=(309 bytes) DEBUG: Response: {'status': 200, 'headers': {'date': 'Thu, 06 Jun 2013 22:40:09 GMT', 'content-length': '326', 'content-type': 'application/xml', 'server': 'Riak CS'}, 'reason': 'OK', 'data': '<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";><Location>http://imagestore.s3.amazonaws.com/tmpl/1/1/routing-1/test</Location><Bucket>imagestore</Bucket><Key>tmpl/1/1/routing-1/test</Key><ETag>kfDkh7Q_QCWN7r0ZTqNq4Q==</ETag></CompleteMultipartUploadResult>'} While the etag created by Amazon S3 is: "70e1860be687d43c039873adef4280f2-3" DEBUG: Sending request method_string='POST', uri='/fixes/icecake/systdfdfdfemvm.iso1?uploadId=vdkPSAtaA7g.fdfdfdfdf..iaKRNW_8QGz.bXdfdfdfdfdfkFXwUwLzRcG5obVvJFDvnhYUFdT6fYr1rig--', DEBUG: Response: {'status': 200, 'headers': {, 'server': 'AmazonS3', 'transfer-encoding': 'chunked', 'connection': 'Keep-Alive', 'x-amz-request-id': '8DFF5D8025E58E99', 'cache-control': 'proxy-revalidate', 'date': 'Thu, 06 Jun 2013 22:39:47 GMT', 'content-type': 'application/xml'}, 'reason': 'OK', 'data': '<?xml version="1.0" encoding="UTF-8"?>\n\n<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";><Location>http://fdfdfdfdfdfdf</Location>Key>fixes/icecake/systemvm.iso1</Key><ETag>"70e1860be687d43c039873adef4280f2-3"</ETag></CompleteMultipartUploadResult>'} So the etag created on Amazon S3 has "-"(dash) in it, but there is only "_" (underscore) on Riak cs. Do you know the reason? What should we need to do to make it compatible with Amazon S3 SDK? > -----Original Message----- > From: John Burwell [mailto:jburw...@basho.com] > Sent: Thursday, June 06, 2013 2:03 PM > To: dev@cloudstack.apache.org > Subject: Re: Object based Secondary storage. > > Min, > > Are you calculating the MD5 or letting the Amazon client do it? > > Thanks, > -John > > On Jun 6, 2013, at 4:54 PM, Min Chen <min.c...@citrix.com> wrote: > > > Thanks Tom. Indeed I have a S3 question that need some advise from > > some S3 experts. To support upload object > 5G, I have used > > TransferManager.upload to upload object to S3, upload went fine and > > object are successfully put to S3. However, later on when I am using > > "s3cmd get <object key>" to retrieve this object, I always got this > > exception: > > > > "MD5 signatures do not match: computed=Y, received="X" > > > > It seems that Amazon S3 kept a different Md5 sum for the multi-part > > uploaded object. We have been using Riak CS for our S3 testing. If I > > changed to not using multi-part upload and directly invoking S3 > > putObject, I will not run into this issue. Do you have such experience > before? > > > > -min > > > > On 6/6/13 1:56 AM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote: > > > >> Thanks Min. I've printed out the material and am reading new threads. > >> Can't comment much yet until I understand things a bit more. > >> > >> Meanwhile, feel free to hit me up with any S3 questions you have. I'm > >> looking forward to playing with the object_store branch and testing > >> it out. > >> > >> Tom. > >> > >> On Wed, 2013-06-05 at 16:14 +0000, Min Chen wrote: > >>> Welcome Tom. You can check out this FS > >>> > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu > >>> p+Obj > >>> ec > >>> t+Store+Plugin+Framework for secondary storage architectural work > >>> t+Store+Plugin+done > >>> in > >>> object_store branch.You may also check out the following recent > >>> threads regarding 3 major technical questions raised by community as > >>> well as our answers and clarification. > >>> > >>> http://mail-archives.apache.org/mod_mbox/cloudstack- > dev/201306.mbox/ > >>> %3C77 > >>> B3 > >>> > 37AF224FD84CBF8401947098DD87036A76%40SJCPEX01CL01.citrite.net%3E > >>> > >>> http://mail-archives.apache.org/mod_mbox/cloudstack- > dev/201306.mbox/ > >>> %3CCD > >>> D2 > >>> 2955.3DDDC%25min.chen%40citrix.com%3E > >>> > >>> http://mail-archives.apache.org/mod_mbox/cloudstack- > dev/201306.mbox/ > >>> %3CCD > >>> D2 > >>> 300D.3DE0C%25min.chen%40citrix.com%3E > >>> > >>> > >>> That branch is mainly worked on by Edison and me, and we are at PST > >>> timezone. > >>> > >>> Thanks > >>> -min > >> -- > >> Cloudian KK - http://www.cloudian.com/get-started.html > >> Fancy 100TB of full featured S3 Storage? > >> Checkout the Cloudian(r) Community Edition! > >> > >
