Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Василий Ангапов
But is there any way to recreate bucket index for existing bucket? Is it possible to change bucket's index pool to some new pool in its metadata and them tell RadosGW to rebuild (--check --fix) index? Sounds really crazy but will it work? Will the new index become sharded? 2016-06-14 13:18 GMT+03:

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Ansgar Jazdzewski
Hi, your cluster will be in warning state if you disable scrubbing, and you relay need it in case of some data loss cheers, Ansgar 2016-06-14 11:05 GMT+02:00 Wido den Hollander : > >> Op 14 juni 2016 om 11:00 schreef Василий Ангапов : >> >> >> Is it a good idea to disable scrub and deep-scrub fo

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Ansgar Jazdzewski
Hi, yes we have index sharding enabled, we have only tow big buckets at the moment with 15Mil objects each and some smaller ones cheers, Ansgar 2016-06-14 10:51 GMT+02:00 Wido den Hollander : > >> Op 14 juni 2016 om 10:10 schreef Ansgar Jazdzewski >> : >> >> >> Hi, >> >> we are using ceph and r

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Christian Balzer
Hello, On Tue, 14 Jun 2016 12:20:44 +0300 Nmz wrote: > > > > - Original Message - > From: Wido den Hollander > To: Василий Ангапов > Date: Tuesday, June 14, 2016, 12:05:51 PM > Subject: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs > &g

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Nmz
- Original Message - From: Wido den Hollander To: Василий Ангапов Date: Tuesday, June 14, 2016, 12:05:51 PM Subject: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs >> Op 14 juni 2016 om 11:00 schreef Василий Ангапов : >> >> >> Is it a go

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Василий Ангапов
Wido, can you please give more details about that? What sort of corruption may occur? What scrubbing actually does especially for bucket index pool? 2016-06-14 12:05 GMT+03:00 Wido den Hollander : > >> Op 14 juni 2016 om 11:00 schreef Василий Ангапов : >> >> >> Is it a good idea to disable scrub a

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Wido den Hollander
> Op 14 juni 2016 om 11:00 schreef Василий Ангапов : > > > Is it a good idea to disable scrub and deep-scrub for bucket.index > pool? What negative consequences it may cause? > No, I would not do that. Scrubbing is essential to detect (silent) data corruption. You should really scrub all you

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Василий Ангапов
Is it a good idea to disable scrub and deep-scrub for bucket.index pool? What negative consequences it may cause? 2016-06-14 11:51 GMT+03:00 Wido den Hollander : > >> Op 14 juni 2016 om 10:10 schreef Ansgar Jazdzewski >> : >> >> >> Hi, >> >> we are using ceph and radosGW to store images (~300kb e

Re: [ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Wido den Hollander
> Op 14 juni 2016 om 10:10 schreef Ansgar Jazdzewski > : > > > Hi, > > we are using ceph and radosGW to store images (~300kb each) in S3, > when in comes to deep-scrubbing we facing task timeouts (> 30s ...) > > my questions is: > > in case of that amount of objects/files is it better to cal

[ceph-users] 40Mil objects in S3 rados pool / how calculate PGs

2016-06-14 Thread Ansgar Jazdzewski
Hi, we are using ceph and radosGW to store images (~300kb each) in S3, when in comes to deep-scrubbing we facing task timeouts (> 30s ...) my questions is: in case of that amount of objects/files is it better to calculate the PGs on a object-bases instant of the volume size? and how it should be