I managed to find out from SoftLayer that they're running 1.7.x
On Thu, Mar 13, 2014 at 11:39 AM, Ben McCann <b...@benmccann.com> wrote: > The segments do appear to be there. > > I don't see any X-Object-Manifest header though or anything like that > > URL: > https://dal05.objectstorage.softlayer.net/v1/AUTH_c4fd4142-xxxx-xxxx-xxxx-xxxxxxxxxxxx/mongodb-backups/mongodb_backup_2014.03.09.09.00.03.tar.gz > Auth Token: AUTH_tkced5xxxxxxxxxxxxxxxxxxxxxxxxxxxx > Account: AUTH_c4fd4142-xxxx-xxxx-xxxx-xxxxxxxxxxxx > Container: mongodb-backups > Object: mongodb_backup_2014.03.09.09.00.03.tar.gz > Content Type: application/x-tar > Content Length: 3895 > Last Modified: Sun, 09 Mar 2014 09:58:09 GMT > ETag: 0a8ec37fb235ac99e16a726e1b17375b > Meta Mtime: 1394358254.226578 > Accept-Ranges: bytes > X-Timestamp: 1394359089.40223 > X-Trans-Id: tx912c6e12d63a409aa5362b6075c5f930 > > Is this object likely to be stored wrong? Or maybe some misconfiguration > on their end like a load banalancer stripping out headers or something? > > > On Thu, Mar 13, 2014 at 11:30 AM, Clay Gerrard <clay.gerr...@gmail.com>wrote: > >> No, you can't really turn off DLO and it's been around for awhile (let's >> assume for now they support it). >> >> Can you verify the segments are showing up in the "container_segments" >> container? Can you verify the "X-Object-Manifest" header on the >> "container/file.tar.gz" object? >> >> "swift stat -v container file.tar.gz" should show it as just "Manifest", >> something like "container >> _segments/file.tar.gz/1394610073.993163/104857600/1000000/" >> >> Good Luck, >> >> -Clay >> >> >> On Thu, Mar 13, 2014 at 11:00 AM, Ben McCann <b...@benmccann.com> wrote: >> >>> Oh geeze, sorry, looking at those docs, it seems maybe I was mistaken >>> and am using Dynamic Large Objects because I uploaded the file using swift >>> -S. Is DLO also optional? Was it added at a certain version? >>> >>> Thanks so much for the help! >>> -Ben >>> >>> >>> On Thu, Mar 13, 2014 at 10:50 AM, Clay Gerrard >>> <clay.gerr...@gmail.com>wrote: >>> >>>> You should check with your deployer to see what version of Swift they >>>> are running. >>>> >>>> the /info (capabilities) feature was added in 1.11 and I think Havana >>>> shipped with 1.10 >>>> >>>> But I think SLO support has been around since 1.5 which should be in >>>> Havana, maybe even Grizzly. [1] >>>> >>>> Either way SLO support has always been optional (and off by default) - >>>> so your deployer may only support DLO. [2] >>>> >>>> 1. >>>> https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#OpenStack_Object_Storage_.28Swift.29 >>>> 2. >>>> http://docs.openstack.org/developer/swift/overview_large_objects.html >>>> >>>> >>>> On Thu, Mar 13, 2014 at 10:31 AM, Ben McCann <b...@benmccann.com> wrote: >>>> >>>>> Oh, you're right, I did miss the -v option. Thanks! >>>>> >>>>> The curl command isn't working for me, however. >>>>> >>>>> StorageURL: >>>>> https://dal05.objectstorage.softlayer.net/v1/AUTH_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx >>>>> >>>>> $ curl https://dal05.objectstorage.softlayer.net/info >>>>> Bad URL >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Mar 13, 2014 at 2:59 AM, Kuo Hugo <tonyt...@gmail.com> wrote: >>>>> >>>>>> Hi Ben, >>>>>> >>>>>> 1. What's the Swift CLI version are you using for now ? >>>>>> 2. The $>swift stat -v suppose to show you the storage_url. -v means >>>>>> verbose. >>>>>> 3. There's another choice via curl >>>>>> >>>>>> ~IRON_MAN$>curl -i https://$FQDN_of_AUTH_SERVER/auth/v1.0 -H >>>>>> "X-auth-user:johndoe" -H "x-auth-key:password_of_john" >>>>>> HTTP/1.1 200 OK >>>>>> X-Storage-Url: https://swift.example.com/v1/AUTH_johndoe >>>>>> X-Auth-Token: AUTH_tk0f4473fa1234567890 >>>>>> Content-Type: text/html; charset=UTF-8 >>>>>> Set-Cookie: X-Auth-Token=AUTH_tk0f4473fa1234567890; Path=/ >>>>>> X-Storage-Token: AUTH_tk0f4473fa1234567890 >>>>>> Content-Length: 0 >>>>>> X-Trans-Id: txcfefd5396a0f4f4694707-0053217efd >>>>>> >>>>>> >>>>>> And then you can send a GET to https://swift.example.com/info >>>>>> >>>>>> $> curl https://cloud.swiftstack.com/info >>>>>> >>>>>> >>>>>> One more information here, the /info was available since Swift 1.11.0 >>>>>> https://github.com/openstack/swift/blob/master/CHANGELOG#L120-L125 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 2014-03-13 14:41 GMT+08:00 Ben McCann <b...@benmccann.com>: >>>>>> >>>>>> Thanks Samuel. That's super helpful. I didn't realize that SLO could >>>>>>> be turned on/off on the server. I thought it was merely a client-side >>>>>>> convention. >>>>>>> >>>>>>> swift stat doesn't return a StorageUrl for me. It just returns: >>>>>>> Account: >>>>>>> Containers: >>>>>>> Objects: >>>>>>> Bytes: >>>>>>> Meta Cdn-Id: >>>>>>> Meta Nas-Id: >>>>>>> X-Timestamp: >>>>>>> X-Trans-Id: >>>>>>> Accept-Ranges: >>>>>>> >>>>>>> I'll see if I can find out from someone at SoftLayer whether they >>>>>>> have an old version or have large objects turned off unless you can >>>>>>> think >>>>>>> of any other ways I might be able to find out by querying the server. >>>>>>> >>>>>>> Thanks, >>>>>>> Ben >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Mar 12, 2014 at 11:09 PM, Samuel Merritt <s...@swiftstack.com >>>>>>> > wrote: >>>>>>> >>>>>>>> On 3/12/14 9:14 PM, Ben McCann wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm attempting to download a large file which I have uploaded to >>>>>>>>> SoftLayer Object Storage using the swift tool. It's downloading >>>>>>>>> only the >>>>>>>>> manifest and is not downloading the segments. Any tips on figuring >>>>>>>>> out >>>>>>>>> what is going wrong? >>>>>>>>> >>>>>>>> >>>>>>>> It's entirely possible that the cluster in question lacks static >>>>>>>> large object support. Maybe they're running an old version of Swift >>>>>>>> (pre-1.8.0/Grizzly), or maybe they haven't enabled the static large >>>>>>>> object >>>>>>>> feature. >>>>>>>> >>>>>>>> What you'll need to do is probe the cluster to see if static large >>>>>>>> objects are enabled. >>>>>>>> >>>>>>>> The easy way is to politely ask the cluster with a GET request to >>>>>>>> /info. Run `swift stat -v`, then take the "StorageURL" field and >>>>>>>> change the >>>>>>>> path to be just "/info". Strip off any path that's already there. >>>>>>>> >>>>>>>> For example, if the output looks like >>>>>>>> >>>>>>>> StorageURL: http://192.168.22.2:8080/v1/AUTH_test >>>>>>>> Auth Token: AUTH_tk89454677b67348e09eefc1d6fe4dfa68 >>>>>>>> Account: AUTH_test >>>>>>>> Containers: 1 >>>>>>>> [...] >>>>>>>> >>>>>>>> then you'd simply run >>>>>>>> >>>>>>>> curl http://192.168.22.2:8080/info >>>>>>>> >>>>>>>> The response will be a JSON hash, so maybe run it through a JSON >>>>>>>> pretty-printer, e.g. >>>>>>>> >>>>>>>> curl http://192.168.22.2:8080/info | python -mjson.tool >>>>>>>> >>>>>>>> On my development machine, the response looks something like this >>>>>>>> (some items removed for brevity): >>>>>>>> >>>>>>>> { >>>>>>>> "container_quotas": {}, >>>>>>>> "formpost": {}, >>>>>>>> "slo": { <---- LOOK FOR THIS >>>>>>>> "max_manifest_segments": 1000, >>>>>>>> "max_manifest_size": 2097152, >>>>>>>> "min_segment_size": 71 >>>>>>>> }, >>>>>>>> "swift": { >>>>>>>> "account_listing_limit": 10000, >>>>>>>> "container_listing_limit": 10000, >>>>>>>> "max_account_name_length": 256, >>>>>>>> "max_container_name_length": 256, >>>>>>>> "max_file_size": 5368709122, >>>>>>>> "max_meta_count": 90, >>>>>>>> "max_meta_name_length": 128, >>>>>>>> "max_meta_value_length": 256, >>>>>>>> "max_object_name_length": 1024, >>>>>>>> "version": "1.12.0.106.ga08e813" >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> If you don't see SLO in there, then you're out of luck. Bug >>>>>>>> Softlayer and see if they'll turn it on. >>>>>>>> >>>>>>>> Now, it's also possible that you'll get a 412 response with the >>>>>>>> body "Bad URL" when you run the curl command above. That will tell you >>>>>>>> that >>>>>>>> the cluster is running 1.11.0 or earlier, but it won't tell you >>>>>>>> anything >>>>>>>> about SLO support. In that case, try uploading a normal file (cat >>>>>>>> picture >>>>>>>> or something) with the header "X-Static-Large-Object: yes". If that >>>>>>>> succeeds, then SLO is disabled; if SLO is enabled, then the PUT will >>>>>>>> fail. >>>>>>>> >>>>>>>> A more verbose description is available here: >>>>>>>> https://ask.openstack.org/en/question/7005/static-large-objects/ >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/ >>>>>>>> openstack >>>>>>>> Post to : openstack@lists.openstack.org >>>>>>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/ >>>>>>>> openstack >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> about.me/benmccann >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Mailing list: >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>>> Post to : openstack@lists.openstack.org >>>>>>> Unsubscribe : >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>> Post to : openstack@lists.openstack.org >>>>>> Unsubscribe : >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> about.me/benmccann >>>>> >>>>> _______________________________________________ >>>>> Mailing list: >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>> Post to : openstack@lists.openstack.org >>>>> Unsubscribe : >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>> >>>>> >>>> >>> >>> >>> -- >>> about.me/benmccann >>> >> >> > > > -- > about.me/benmccann > -- about.me/benmccann
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
