[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-05 Thread 西宮牧人
Hi, everyone. The problem was solved. A PG's epoch of active osds was dfferent from one of acting osds. When I removed head and TEMP directory of the PG from active osds, blocked request vanished. Thanks to your kindness. 2020年2月6日(木) 4:19 : > What do you guys Think about > > Ceph osd Set noo

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-05 Thread ceph
What do you guys Think about Ceph osd Set noout/Down And See if the osd will become healthy? Another idea which is in my mind is to remove the sayed osd from the Cluster... As long as the other osds on the same node dont have an issue i guess the disk has a Problem... Just my 2 Cents - Mehmet

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-03 Thread Janne Johansson
Den mån 3 feb. 2020 kl 08:25 skrev Wido den Hollander : > > The crash happens, when the osd wants to read from pipe when processing > > heartbeat. To me it sounds like a networking issue. > > It could also be that this OSD is so busy internally with other stuff > that it doesn't respond to heartbe

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-02 Thread Wido den Hollander
On 2/3/20 8:39 AM, wes park wrote: > How to know a OSD is super busy? Thanks. Check if it's using 100% CPU for example. And check the disk util with iostat. Wido > > Wido den Hollander mailto:w...@42on.com>> > > > > On 2/2/20 5:20 PM, Andreas John wrote: > > Hello, > > > >

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-02 Thread wes park
How to know a OSD is super busy? Thanks. Wido den Hollander > > > On 2/2/20 5:20 PM, Andreas John wrote: > > Hello, > > > > what you see is an stracktrace, so the OSD is hitting an unexpected > > state (Otherwise there would be an error handler). > > > > The crash happens, when the osd wants to

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-02 Thread Wido den Hollander
On 2/2/20 5:20 PM, Andreas John wrote: > Hello, > > what you see is an stracktrace, so the OSD is hitting an unexpected > state (Otherwise there would be an error handler). > > The crash happens, when the osd wants to read from pipe when processing > heartbeat. To me it sounds like a networking

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-02 Thread Andreas John
Hello, what you see is an stracktrace, so the OSD is hitting an unexpected state (Otherwise there would be an error handler). The crash happens, when the osd wants to read from pipe when processing heartbeat. To me it sounds like a networking issue. I see the other OSD an that host are healthy,

[ceph-users] Re: osd is immidietly down and uses CPU full.

2020-02-01 Thread Makito Nishimiya
Hi. This is the cluster informastion. -- /var/log/ceph/ceph.osd.1.log --- 2020-02-01 03:47:20.635504 7f86f4e40700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f86fe35e700' had timed out after 15 2020-02-01 03:47:20.635521 7f86f4f41700 1 heartbeat_m