Re: [ceph-users] rgw query bucket usage quickly

2016-07-29 Thread Dan van der Ster
Thanks for the tip.. It works now! ./s3curl.pl --id -- 'http:///admin/bucket?bucket=&stats' | jq .usage { "rgw.main": { "size_kb": 14754527122, "size_kb_actual": 14787626688, "num_objects": 16556819 } } -- Dan On Fri, Jul 29, 2016 at 6:05 PM, Brian Felton wrote: > If s3curl h

Re: [ceph-users] rgw query bucket usage quickly

2016-07-29 Thread Brian Felton
If s3curl has a major limitation, it's that you have to edit it to get signatures to match. Specifically, the @endpoints list on or around line 31 needs to be edited to include the endpoint you're calling to hit RGW (e.g. 192.168.100.100, my.ceph.cluster, etc.). Once you add that, you should stop

Re: [ceph-users] rgw query bucket usage quickly

2016-07-29 Thread Dan van der Ster
On Fri, Jul 29, 2016 at 12:06 PM, Wido den Hollander wrote: > >> Op 29 juli 2016 om 11:59 schreef Dan van der Ster : >> >> >> Oh yes, that should help. BTW, which client are people using for the >> Admin Ops API? Is there something better than s3curl.pl ... >> > > I wrote my own client a while ago

Re: [ceph-users] rgw query bucket usage quickly

2016-07-29 Thread Wido den Hollander
> Op 29 juli 2016 om 11:59 schreef Dan van der Ster : > > > Oh yes, that should help. BTW, which client are people using for the > Admin Ops API? Is there something better than s3curl.pl ... > I wrote my own client a while ago, but that's kind of buggy :) You might want to take a look at: htt

Re: [ceph-users] rgw query bucket usage quickly

2016-07-29 Thread Dan van der Ster
Oh yes, that should help. BTW, which client are people using for the Admin Ops API? Is there something better than s3curl.pl ... -- Dan On Thu, Jul 28, 2016 at 6:37 PM, Brian Andrus wrote: > I'm not sure what mechanism is used, but perhaps the Admin Ops API could > provide what you're looking f

Re: [ceph-users] rgw query bucket usage quickly

2016-07-28 Thread Brian Andrus
I'm not sure what mechanism is used, but perhaps the Admin Ops API could provide what you're looking for. http://docs.ceph.com/docs/master/radosgw/adminops/#get-usage I believe also that the usage log should be enabled for the gateway. On Thu, Jul 28, 2016 at 12:19 PM, Sean Redmond wrote: > Hi

Re: [ceph-users] rgw query bucket usage quickly

2016-07-28 Thread Sean Redmond
Hi, This seems pretty quick here on a jewel cluster here, But I guess the key questions is how large is large? Is it perhaps a large number of smaller files that is slowing this down? Is the bucket index shared / on SSD? [root@korn ~]# time s3cmd du s3://seanbackup 1656225129419 29 objects

Re: [ceph-users] rgw query bucket usage quickly

2016-07-28 Thread Dan van der Ster
On Thu, Jul 28, 2016 at 5:33 PM, Abhishek Lekshmanan wrote: > > Dan van der Ster writes: > >> Hi, >> >> Does anyone know a fast way for S3 users to query their total bucket >> usage? 's3cmd du' takes a long time on large buckets (is it iterating >> over all the objects?). 'radosgw-admin bucket sta

Re: [ceph-users] rgw query bucket usage quickly

2016-07-28 Thread Abhishek Lekshmanan
Dan van der Ster writes: > Hi, > > Does anyone know a fast way for S3 users to query their total bucket > usage? 's3cmd du' takes a long time on large buckets (is it iterating > over all the objects?). 'radosgw-admin bucket stats' seems to know the > bucket usage immediately, but I didn't find a

[ceph-users] rgw query bucket usage quickly

2016-07-28 Thread Dan van der Ster
Hi, Does anyone know a fast way for S3 users to query their total bucket usage? 's3cmd du' takes a long time on large buckets (is it iterating over all the objects?). 'radosgw-admin bucket stats' seems to know the bucket usage immediately, but I didn't find a way to expose that to end users. Hopi