Re: [ceph-users] Giant upgrade - stability issues

2014-11-28 Thread Andrei Mikhailovsky
ay, 27 November, 2014 1:05:30 PM > Subject: Re: [ceph-users] Giant upgrade - stability issues > Sam, > I've done more network testing, this time over 2 days and I believe I > have enough evidence to conclude that the osd disconnects are not > caused by the network. I have ran about 14

Re: [ceph-users] Giant upgrade - stability issues

2014-11-27 Thread Andrei Mikhailovsky
ly, which I did not. Not sure what to think now. Andrei - Original Message - > From: "Andrei Mikhailovsky" > To: sj...@redhat.com > Cc: ceph-users@lists.ceph.com > Sent: Thursday, 20 November, 2014 4:50:21 PM > Subject: Re: [ceph-users] Giant upgrade - sta

Re: [ceph-users] Giant upgrade - stability issues

2014-11-20 Thread Andrei Mikhailovsky
Thanks, I will try that. Andrei - Original Message - From: "Samuel Just" To: "Andrei Mikhailovsky" Cc: ceph-users@lists.ceph.com Sent: Thursday, 20 November, 2014 4:26:00 PM Subject: Re: [ceph-users] Giant upgrade - stability issues You can try to capture lo

Re: [ceph-users] Giant upgrade - stability issues

2014-11-20 Thread Samuel Just
; ________ > From: "Samuel Just" > To: "Andrei Mikhailovsky" > Cc: ceph-users@lists.ceph.com > Sent: Wednesday, 19 November, 2014 9:45:40 PM > > Subject: Re: [ceph-users] Giant upgrade - stability issues > > Well, the heartbeats are

Re: [ceph-users] Giant upgrade - stability issues

2014-11-20 Thread Andrei Mikhailovsky
Mikhailovsky" Cc: ceph-users@lists.ceph.com Sent: Wednesday, 19 November, 2014 9:45:40 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Well, the heartbeats are failing due to networking errors preventing the heartbeats from arriving. That is causing osds to go down, and that is ca

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Andrei Mikhailovsky
of logs. If we >> are >> talking about debug_ms 10 setting, I've got about 7gb worth of logs >> generated every half an hour or so. Not really sure what to do with that >> much data. Anything more constructive? >> >> Thanks >> >>

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Samuel Just
of logs. If we >> are >> talking about debug_ms 10 setting, I've got about 7gb worth of logs >> generated every half an hour or so. Not really sure what to do with that >> much data. Anything more constructive? >> >> Thanks >>

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Andrei Mikhailovsky
"Andrei Mikhailovsky" > Cc: ceph-users@lists.ceph.com > Sent: Tuesday, 18 November, 2014 8:53:47 PM > > Subject: Re: [ceph-users] Giant upgrade - stability issues > > pastebin or something, probably. > -Sam > > On Tue, Nov 18, 2

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Samuel Just
lovsky > wrote: >> Sam, the logs are rather large in size. Where should I post it to? >> >> Thanks >> ____ >> From: "Samuel Just" >> To: "Andrei Mikhailovsky" >> Cc: ceph-users@lists.ceph.com >&g

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
ginal Message - From: "Samuel Just" To: "Andrei Mikhailovsky" Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 8:53:47 PM Subject: Re: [ceph-users] Giant upgrade - stability issues pastebin or something, probably. -Sam On Tue, Nov 18, 2014 at 12:34 PM, An

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Samuel Just
i Mikhailovsky" > Cc: ceph-users@lists.ceph.com > Sent: Tuesday, 18 November, 2014 7:54:56 PM > Subject: Re: [ceph-users] Giant upgrade - stability issues > > > Ok, why is ceph marking osds down? Post your ceph.log from one of the > problematic periods. > -Sam > > On Tue, Nov

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
Sam, the logs are rather large in size. Where should I post it to? Thanks - Original Message - From: "Samuel Just" To: "Andrei Mikhailovsky" Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 7:54:56 PM Subject: Re: [ceph-users] Giant upgrade - s

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Samuel Just
Ok, why is ceph marking osds down? Post your ceph.log from one of the problematic periods. -Sam On Tue, Nov 18, 2014 at 1:35 AM, Andrei Mikhailovsky wrote: > Hello cephers, > > I need your help and suggestion on what is going on with my cluster. A few > weeks ago i've upgraded from Firefly to Gi

[ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
Hello cephers, I need your help and suggestion on what is going on with my cluster. A few weeks ago i've upgraded from Firefly to Giant. I've previously written about having issues with Giant where in two weeks period the cluster's IO froze three times after ceph down-ed two osds. I have in to