Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-06-02 Thread Kenneth Waegeman
- Message from Guang Yang - Date: Fri, 30 May 2014 08:56:37 +0800 From: Guang Yang Subject: Re: [ceph-users] Expanding pg's of an erasure coded pool To: Gregory Farnum Cc: Kenneth Waegeman , ceph-users On May 28, 2014, at 5:31 AM, Gregory Farnum wrote:

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-29 Thread Guang Yang
On May 28, 2014, at 5:31 AM, Gregory Farnum wrote: > On Sun, May 25, 2014 at 6:24 PM, Guang Yang wrote: >> On May 21, 2014, at 1:33 AM, Gregory Farnum wrote: >> >>> This failure means the messenger subsystem is trying to create a >>> thread and is getting an error code back — probably due to a

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-27 Thread Gregory Farnum
On Sun, May 25, 2014 at 6:24 PM, Guang Yang wrote: > On May 21, 2014, at 1:33 AM, Gregory Farnum wrote: > >> This failure means the messenger subsystem is trying to create a >> thread and is getting an error code back — probably due to a process >> or system thread limit that you can turn up with

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-25 Thread Guang Yang
On May 21, 2014, at 1:33 AM, Gregory Farnum wrote: > This failure means the messenger subsystem is trying to create a > thread and is getting an error code back — probably due to a process > or system thread limit that you can turn up with ulimit. > > This is happening because a replicated PG pr

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-22 Thread Henrik Korkuc
On 2014.05.22 19:55, Gregory Farnum wrote: > On Thu, May 22, 2014 at 4:09 AM, Kenneth Waegeman > wrote: >> - Message from Gregory Farnum - >>Date: Wed, 21 May 2014 15:46:17 -0700 >> >>From: Gregory Farnum >> Subject: Re: [ceph-users] Exp

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-22 Thread Gregory Farnum
On Thu, May 22, 2014 at 4:09 AM, Kenneth Waegeman wrote: > > - Message from Gregory Farnum - >Date: Wed, 21 May 2014 15:46:17 -0700 > >From: Gregory Farnum > Subject: Re: [ceph-users] Expanding pg's of an erasure coded pool > To: Kenneth Waege

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-22 Thread Kenneth Waegeman
- Message from Gregory Farnum - Date: Wed, 21 May 2014 15:46:17 -0700 From: Gregory Farnum Subject: Re: [ceph-users] Expanding pg's of an erasure coded pool To: Kenneth Waegeman Cc: ceph-users On Wed, May 21, 2014 at 3:52 AM, Kenneth Waegeman wrote: Than

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-21 Thread Gregory Farnum
On Wed, May 21, 2014 at 3:52 AM, Kenneth Waegeman wrote: > Thanks! I increased the max processes parameter for all daemons quite a lot > (until ulimit -u 3802720) > > These are the limits for the daemons now.. > [root@ ~]# cat /proc/17006/limits > Limit Soft Limit Har

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-21 Thread Kenneth Waegeman
20 May 2014 10:33:30 -0700 From: Gregory Farnum Subject: Re: [ceph-users] Expanding pg's of an erasure coded pool To: Kenneth Waegeman Cc: ceph-users This failure means the messenger subsystem is trying to create a thread and is getting an error code back — probably due to a

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-20 Thread Gregory Farnum
This failure means the messenger subsystem is trying to create a thread and is getting an error code back — probably due to a process or system thread limit that you can turn up with ulimit. This is happening because a replicated PG primary needs a connection to only its replicas (generally 1 or 2

[ceph-users] Expanding pg's of an erasure coded pool

2014-05-20 Thread Kenneth Waegeman
Hi, On a setup of 400 OSDs (20 nodes, with 20 OSDs per node), I first tried to create a erasure coded pool with 4096 pgs, but this crashed the cluster. I then started with 1024 pgs, expanding to 2048 (pg_num and pgp_num), when I then try to expand to 4096 (not even quite enough) the cluster