Thanks for all the responses, much appreciated.
Upping the chunk size fixes my problem in the short term but I upgrade to
17.2.6 :)
Kind regards,
Tom
On Tue, 3 Oct 2023 at 15:28, Matt Benjamin wrote:
> Hi Thomas,
>
> If I'm not mistaken, the RGW will paginate ListBuckets essentially like
> Lis
Hi Thomas,
If I'm not mistaken, the RGW will paginate ListBuckets essentially like
ListObjectsv1 if the S3 client provides the appropriate "marker" parameter
values. COS does this too, I noticed. I'm not sure which S3 clients can
be relied on to do this, though.
Matt
On Tue, Oct 3, 2023 at 9:0
On Tue, Oct 3, 2023 at 9:06 AM Thomas Bennett wrote:
>
> Hi Jonas,
>
> Thanks :) that solved my issue.
>
> It would seem to me that this is heading towards something that the clients
> s3 should paginate, but I couldn't find any documentation on how to
> paginate bucket listings.
the s3 ListBucke
Den tis 3 okt. 2023 kl 11:59 skrev Thomas Bennett :
> Hi,
>
> I'm running a Ceph 17.2.5 Rados Gateway and I have a user with more than
> 1000 buckets.
>
> When the client tries to list all their buckets using s3cmd, rclone and
> python boto3, they all three only ever return the first 1000 bucket n
Hi Jonas,
Thanks :) that solved my issue.
It would seem to me that this is heading towards something that the clients
s3 should paginate, but I couldn't find any documentation on how to
paginate bucket listings. All the information points to paginating object
listing - which makes sense.
Just fo
Hi,
You should increase these default settings:
rgw_list_buckets_max_chunk // for buckets
rgw_max_listing_results // for objects
On Tue, Oct 3, 2023 at 12:59 PM Thomas Bennett wrote:
> Hi,
>
> I'm running a Ceph 17.2.5 Rados Gateway and I have a user with more than
> 1000 buckets.
>
> When the
