By default, radosgw only returns the first 1000 objects. Looks like radosgw-admin has the same limit.
Looking at the man page, I don't see any way to page through the list. I must be missing something. The S3 API does have the ability to page through the list. I use the command line tool s3cmd to get the full bucket list. It does require user credentials though, so that might be a pain if you have many users. On Wed, Mar 11, 2015 at 6:47 PM, Sean Sullivan <seapasu...@uchicago.edu> wrote: > I have a single radosgw user with 2 s3 keys and 1 swift key. I have > created a few buckets and I can list all of the contents of bucket A and C > but not B with either S3 (boto) or python-swiftclient. I am able to list > the first 1000 entries using radosgw-admin 'bucket list --bucket=bucketB' > without any issues but this doesn't really help. > > The odd thing is I can still upload and download objects in the bucket. I > just can't list them. I tried setting the bucket canned_acl to private and > public but I still can't list the objects inside. > > I'm using ceph .87 (Giant) Here is some info about the cluster:: > http://pastebin.com/LvQYnXem -- ceph.conf > http://pastebin.com/efBBPCwa -- ceph -s > http://pastebin.com/tF62WMU9 -- radosgw-admin bucket list > http://pastebin.com/CZ8TkyNG -- python list bucket objects script > http://pastebin.com/TUCyxhMD -- radosgw-admin bucket stats --bucketB > http://pastebin.com/uHbEtGHs -- rados -p .rgw.buckets ls | grep > default.20283.2 (bucketB marker) > http://pastebin.com/WYwfQndV -- Python Error when trying to list BucketB > via boto > > I have no idea why this could be happening outside of the acl. Has anyone > seen this before? Any idea on how I can get access to this bucket again via > s3/swift? Also is there a way to list the full list of a bucket via > radosgw-admin and not the first 9000 lines / 1000 entries, or a way to page > through them? > > EDIT:: I just fixed it (I hope) but the fix doesn't make any sense: > > radosgw-admin bucket unlink --uid=user --bucket=bucketB > radosgw-admin bucket link --uid=user --bucket=bucketB > --bucket-id=default.20283.2 > > Now with swift or s3 (boto) I am able to list the bucket contents without > issue ^_^ > > Can someone elaborate on why this works and how it broken in the first > place when ceph was health_ok the entire time? With 3 replicas how did this > happen? Could this be a bug? sorry for the rambling. I am confused and > tired ;p > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
