Somnath, You are correct, there are dmesg errors about the drive. How can I replace the drive? Can I copy all of the readable contents from this drive to a new one? Because I have the following output from “ceph health detail”
HEALTH_WARN 43 pgs stale; 43 pgs stuck stale pg 7.5b7 is stuck stale for 5954121.993990, current state stale+active+clean, last acting [1] pg 7.42a is stuck stale for 5954121.993885, current state stale+active+clean, last acting [1] pg 7.669 is stuck stale for 5954121.994072, current state stale+active+clean, last acting [1] pg 7.121 is stuck stale for 5954121.993586, current state stale+active+clean, last acting [1] pg 7.4ec is stuck stale for 5954121.993956, current state stale+active+clean, last acting [1] pg 7.1e4 is stuck stale for 5954121.993670, current state stale+active+clean, last acting [1] pg 7.41f is stuck stale for 5954121.993901, current state stale+active+clean, last acting [1] pg 7.59f is stuck stale for 5954121.994024, current state stale+active+clean, last acting [1] pg 7.39 is stuck stale for 5954121.993490, current state stale+active+clean, last acting [1] pg 7.584 is stuck stale for 5954121.994026, current state stale+active+clean, last acting [1] pg 7.fd is stuck stale for 5954121.993600, current state stale+active+clean, last acting [1] pg 7.6fd is stuck stale for 5954121.994158, current state stale+active+clean, last acting [1] pg 7.4b5 is stuck stale for 5954121.993975, current state stale+active+clean, last acting [1] pg 7.328 is stuck stale for 5954121.993840, current state stale+active+clean, last acting [1] pg 7.4a9 is stuck stale for 5954121.993981, current state stale+active+clean, last acting [1] pg 7.569 is stuck stale for 5954121.994046, current state stale+active+clean, last acting [1] pg 7.629 is stuck stale for 5954121.994119, current state stale+active+clean, last acting [1] pg 7.623 is stuck stale for 5954121.994118, current state stale+active+clean, last acting [1] pg 7.6dd is stuck stale for 5954121.994179, current state stale+active+clean, last acting [1] pg 7.3d5 is stuck stale for 5954121.993935, current state stale+active+clean, last acting [1] pg 7.54b is stuck stale for 5954121.994058, current state stale+active+clean, last acting [1] pg 7.3cf is stuck stale for 5954121.993938, current state stale+active+clean, last acting [1] pg 7.c4 is stuck stale for 5954121.993633, current state stale+active+clean, last acting [1] pg 7.178 is stuck stale for 5954121.993719, current state stale+active+clean, last acting [1] pg 7.3b8 is stuck stale for 5954121.993946, current state stale+active+clean, last acting [1] pg 7.b1 is stuck stale for 5954121.993635, current state stale+active+clean, last acting [1] pg 7.5fb is stuck stale for 5954121.994146, current state stale+active+clean, last acting [1] pg 7.236 is stuck stale for 5954121.993801, current state stale+active+clean, last acting [1] pg 7.2f5 is stuck stale for 5954121.993881, current state stale+active+clean, last acting [1] pg 7.ac is stuck stale for 5954121.993643, current state stale+active+clean, last acting [1] pg 7.16d is stuck stale for 5954121.993738, current state stale+active+clean, last acting [1] pg 7.6b7 is stuck stale for 5954121.994223, current state stale+active+clean, last acting [1] pg 7.5ea is stuck stale for 5954121.994166, current state stale+active+clean, last acting [1] pg 7.a3 is stuck stale for 5954121.993654, current state stale+active+clean, last acting [1] pg 7.52d is stuck stale for 5954121.994110, current state stale+active+clean, last acting [1] pg 7.2d8 is stuck stale for 5954121.993904, current state stale+active+clean, last acting [1] pg 7.2db is stuck stale for 5954121.993903, current state stale+active+clean, last acting [1] pg 7.5d9 is stuck stale for 5954121.994181, current state stale+active+clean, last acting [1] pg 7.395 is stuck stale for 5954121.993989, current state stale+active+clean, last acting [1] pg 7.38e is stuck stale for 5954121.993988, current state stale+active+clean, last acting [1] pg 7.13a is stuck stale for 5954121.993766, current state stale+active+clean, last acting [1] pg 7.683 is stuck stale for 5954121.994255, current state stale+active+clean, last acting [1] pg 7.439 is stuck stale for 5954121.994079, current state stale+active+clean, last acting [1] It’s osd id=1 that’s problematic, but I should have a replica of the data somewhere else? Thanks! ~Noah > On Mar 22, 2015, at 2:04 PM, Somnath Roy <somnath....@sandisk.com> wrote: > > Are you seeing any error related to the disk (where OSD is mounted) in dmesg ? > Could be a leveldb corruption or ceph bug. > Now, unfortunately not enough log in that portion of the code base to reveal > exactly why we are not getting infoos object from leveldb :-( > > Thanks & Regards > Somnath > > -----Original Message----- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Noah > Mehl > Sent: Sunday, March 22, 2015 10:11 AM > To: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Can't Start OSD > > In production for over a year, and no upgrades. > > Thanks! > > ~Noah > >> On Mar 22, 2015, at 1:01 PM, Somnath Roy <somnath....@sandisk.com> wrote: >> >> Noah, >> Is this fresh installation or after upgrade ? >> >> It seems related to omap (leveldb) stuff. >> >> Thanks & Regards >> Somnath >> -----Original Message----- >> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of >> Noah Mehl >> Sent: Sunday, March 22, 2015 9:34 AM >> To: ceph-users@lists.ceph.com >> Subject: [ceph-users] Can't Start OSD >> >> I have an OSD that’s failing to start. I can’t make heads or tails of the >> error (pasted below). >> >> Thanks! >> >> ~Noah >> >> 2015-03-22 16:32:39.265116 7f4da7fa0780 0 ceph version 0.67.4 >> (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 13483 >> 2015-03-22 16:32:39.269499 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) mount detected xfs >> 2015-03-22 16:32:39.269509 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) disabling 'filestore replica fadvise' >> due to known issues with fadvise(DONTNEED) on xfs >> 2015-03-22 16:32:39.450031 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is supported and >> appears to work >> 2015-03-22 16:32:39.450069 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is disabled via >> 'filestore fiemap' config option >> 2015-03-22 16:32:39.450743 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount did NOT detect btrfs >> 2015-03-22 16:32:39.499753 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount syncfs(2) syscall fully supported >> (by glibc and kernel) >> 2015-03-22 16:32:39.500078 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount found snaps <> >> 2015-03-22 16:32:40.765736 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount: enabling WRITEAHEAD journal mode: >> btrfs not detected >> 2015-03-22 16:32:40.777156 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 2551: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> 2015-03-22 16:32:40.777278 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 2551: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> 2015-03-22 16:32:40.778223 7f4da7fa0780 1 journal close >> /var/lib/ceph/osd/ceph-1/journal >> 2015-03-22 16:32:41.066655 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) mount detected xfs >> 2015-03-22 16:32:41.150578 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is supported and >> appears to work >> 2015-03-22 16:32:41.150624 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is disabled via >> 'filestore fiemap' config option >> 2015-03-22 16:32:41.151359 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount did NOT detect btrfs >> 2015-03-22 16:32:41.225302 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount syncfs(2) syscall fully supported >> (by glibc and kernel) >> 2015-03-22 16:32:41.225498 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount found snaps <> >> 2015-03-22 16:32:42.375558 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount: enabling WRITEAHEAD journal mode: >> btrfs not detected >> 2015-03-22 16:32:42.382958 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 1429: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> 2015-03-22 16:32:42.383187 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 1481: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> 2015-03-22 16:32:43.076434 7f4da7fa0780 -1 osd/PG.cc: In function 'static >> epoch_t PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::bufferlist*)' thread 7f4da7fa0780 time 2015-03-22 16:32:43.075101 >> osd/PG.cc: 2270: FAILED assert(values.size() == 1) >> >> ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> 1: (PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::buffer::list*)+0x4d7) [0x70ebf7] >> 2: (OSD::load_pgs()+0x14ce) [0x694efe] >> 3: (OSD::init()+0x11be) [0x69cffe] >> 4: (main()+0x1d09) [0x5c3509] >> 5: (__libc_start_main()+0xed) [0x7f4da5bde76d] >> 6: /usr/bin/ceph-osd() [0x5c6e1d] >> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to >> interpret this. >> >> --- begin dump of recent events --- >> -75> 2015-03-22 16:32:39.259280 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command perfcounters_dump hook 0x1ae4010 >> -74> 2015-03-22 16:32:39.259373 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command 1 hook 0x1ae4010 >> -73> 2015-03-22 16:32:39.259393 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command perf dump hook 0x1ae4010 >> -72> 2015-03-22 16:32:39.259429 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command perfcounters_schema hook 0x1ae4010 >> -71> 2015-03-22 16:32:39.259445 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command 2 hook 0x1ae4010 >> -70> 2015-03-22 16:32:39.259453 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command perf schema hook 0x1ae4010 >> -69> 2015-03-22 16:32:39.259467 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command config show hook 0x1ae4010 >> -68> 2015-03-22 16:32:39.259481 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command config set hook 0x1ae4010 >> -67> 2015-03-22 16:32:39.259495 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command config get hook 0x1ae4010 >> -66> 2015-03-22 16:32:39.259505 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command log flush hook 0x1ae4010 >> -65> 2015-03-22 16:32:39.259519 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command log dump hook 0x1ae4010 >> -64> 2015-03-22 16:32:39.259536 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command log reopen hook 0x1ae4010 >> -63> 2015-03-22 16:32:39.265116 7f4da7fa0780 0 ceph version 0.67.4 >> (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 13483 >> -62> 2015-03-22 16:32:39.266443 7f4da7fa0780 1 -- 192.168.41.42:0/0 >> learned my addr 192.168.41.42:0/0 >> -61> 2015-03-22 16:32:39.266462 7f4da7fa0780 1 accepter.accepter.bind >> my_inst.addr is 192.168.41.42:6803/13483 need_addr=0 >> -60> 2015-03-22 16:32:39.266500 7f4da7fa0780 1 -- 192.168.42.42:0/0 >> learned my addr 192.168.42.42:0/0 >> -59> 2015-03-22 16:32:39.266537 7f4da7fa0780 1 accepter.accepter.bind >> my_inst.addr is 192.168.42.42:6802/13483 need_addr=0 >> -58> 2015-03-22 16:32:39.266551 7f4da7fa0780 1 -- 192.168.42.42:0/0 >> learned my addr 192.168.42.42:0/0 >> -57> 2015-03-22 16:32:39.266560 7f4da7fa0780 1 accepter.accepter.bind >> my_inst.addr is 192.168.42.42:6803/13483 need_addr=0 >> -56> 2015-03-22 16:32:39.266580 7f4da7fa0780 1 -- 192.168.41.42:0/0 >> learned my addr 192.168.41.42:0/0 >> -55> 2015-03-22 16:32:39.266602 7f4da7fa0780 1 accepter.accepter.bind >> my_inst.addr is 192.168.41.42:6808/13483 need_addr=0 >> -54> 2015-03-22 16:32:39.269108 7f4da7fa0780 5 asok(0x1aec1c0) init >> /var/run/ceph/ceph-osd.1.asok >> -53> 2015-03-22 16:32:39.269138 7f4da7fa0780 5 asok(0x1aec1c0) >> bind_and_listen /var/run/ceph/ceph-osd.1.asok >> -52> 2015-03-22 16:32:39.269185 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command 0 hook 0x1ae30b0 >> -51> 2015-03-22 16:32:39.269203 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command version hook 0x1ae30b0 >> -50> 2015-03-22 16:32:39.269206 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command git_version hook 0x1ae30b0 >> -49> 2015-03-22 16:32:39.269210 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command help hook 0x1ae40d0 >> -48> 2015-03-22 16:32:39.269231 7f4da7fa0780 5 asok(0x1aec1c0) >> register_command get_command_descriptions hook 0x1ae40c0 >> -47> 2015-03-22 16:32:39.269273 7f4da3c28700 5 asok(0x1aec1c0) entry start >> -46> 2015-03-22 16:32:39.269499 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) mount detected xfs >> -45> 2015-03-22 16:32:39.269509 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) disabling 'filestore replica fadvise' >> due to known issues with fadvise(DONTNEED) on xfs >> -44> 2015-03-22 16:32:39.450031 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is supported and >> appears to work >> -43> 2015-03-22 16:32:39.450069 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is disabled via >> 'filestore fiemap' config option >> -42> 2015-03-22 16:32:39.450743 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount did NOT detect btrfs >> -41> 2015-03-22 16:32:39.499753 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount syncfs(2) syscall fully supported >> (by glibc and kernel) >> -40> 2015-03-22 16:32:39.500078 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount found snaps <> >> -39> 2015-03-22 16:32:40.765736 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount: enabling WRITEAHEAD journal mode: >> btrfs not detected >> -38> 2015-03-22 16:32:40.777088 7f4da7fa0780 2 journal open >> /var/lib/ceph/osd/ceph-1/journal fsid e2ad61ec-c581-4159-8671-bab77c7d4e97 >> fs_op_seq 81852815 >> -37> 2015-03-22 16:32:40.777156 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 2551: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> -36> 2015-03-22 16:32:40.777242 7f4da7fa0780 2 journal No further valid >> entries found, journal is most likely valid >> -35> 2015-03-22 16:32:40.777252 7f4da7fa0780 2 journal No further valid >> entries found, journal is most likely valid >> -34> 2015-03-22 16:32:40.777255 7f4da7fa0780 3 journal journal_replay: end >> of journal, done. >> -33> 2015-03-22 16:32:40.777278 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 2551: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> -32> 2015-03-22 16:32:40.777874 7f4d9fc20700 1 FileStore::op_tp worker >> finish >> -31> 2015-03-22 16:32:40.777930 7f4da0421700 1 FileStore::op_tp worker >> finish >> -30> 2015-03-22 16:32:40.778223 7f4da7fa0780 1 journal close >> /var/lib/ceph/osd/ceph-1/journal >> -29> 2015-03-22 16:32:41.066043 7f4da7fa0780 10 monclient(hunting): >> build_initial_monmap >> -28> 2015-03-22 16:32:41.066137 7f4da7fa0780 5 adding auth protocol: cephx >> -27> 2015-03-22 16:32:41.066147 7f4da7fa0780 5 adding auth protocol: cephx >> -26> 2015-03-22 16:32:41.066384 7f4da7fa0780 1 -- 192.168.41.42:6803/13483 >> messenger.start >> -25> 2015-03-22 16:32:41.066418 7f4da7fa0780 1 -- :/0 messenger.start >> -24> 2015-03-22 16:32:41.066444 7f4da7fa0780 1 -- 192.168.41.42:6808/13483 >> messenger.start >> -23> 2015-03-22 16:32:41.066469 7f4da7fa0780 1 -- 192.168.42.42:6803/13483 >> messenger.start >> -22> 2015-03-22 16:32:41.066512 7f4da7fa0780 1 -- 192.168.42.42:6802/13483 >> messenger.start >> -21> 2015-03-22 16:32:41.066610 7f4da7fa0780 2 osd.1 0 mounting >> /var/lib/ceph/osd/ceph-1 /var/lib/ceph/osd/ceph-1/journal >> -20> 2015-03-22 16:32:41.066655 7f4da7fa0780 1 >> filestore(/var/lib/ceph/osd/ceph-1) mount detected xfs >> -19> 2015-03-22 16:32:41.150578 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is supported and >> appears to work >> -18> 2015-03-22 16:32:41.150624 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount FIEMAP ioctl is disabled via >> 'filestore fiemap' config option >> -17> 2015-03-22 16:32:41.151359 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount did NOT detect btrfs >> -16> 2015-03-22 16:32:41.225302 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount syncfs(2) syscall fully supported >> (by glibc and kernel) >> -15> 2015-03-22 16:32:41.225498 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount found snaps <> >> -14> 2015-03-22 16:32:42.375558 7f4da7fa0780 0 >> filestore(/var/lib/ceph/osd/ceph-1) mount: enabling WRITEAHEAD journal mode: >> btrfs not detected >> -13> 2015-03-22 16:32:42.382825 7f4da7fa0780 2 journal open >> /var/lib/ceph/osd/ceph-1/journal fsid e2ad61ec-c581-4159-8671-bab77c7d4e97 >> fs_op_seq 81852815 >> -12> 2015-03-22 16:32:42.382958 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 1429: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> -11> 2015-03-22 16:32:42.383091 7f4da7fa0780 2 journal No further valid >> entries found, journal is most likely valid >> -10> 2015-03-22 16:32:42.383108 7f4da7fa0780 2 journal No further valid >> entries found, journal is most likely valid >> -9> 2015-03-22 16:32:42.383111 7f4da7fa0780 3 journal journal_replay: end >> of journal, done. >> -8> 2015-03-22 16:32:42.383187 7f4da7fa0780 1 journal _open >> /var/lib/ceph/osd/ceph-1/journal fd 1481: 5368709120 bytes, block size 4096 >> bytes, directio = 1, aio = 1 >> -7> 2015-03-22 16:32:42.383761 7f4da7fa0780 2 osd.1 0 boot >> -6> 2015-03-22 16:32:42.388322 7f4da7fa0780 1 <cls> >> cls/rgw/cls_rgw.cc:1596: Loaded rgw class! >> -5> 2015-03-22 16:32:42.389272 7f4da7fa0780 1 <cls> >> cls/log/cls_log.cc:313: Loaded log class! >> -4> 2015-03-22 16:32:42.392742 7f4da7fa0780 1 <cls> >> cls/refcount/cls_refcount.cc:231: Loaded refcount class! >> -3> 2015-03-22 16:32:42.393520 7f4da7fa0780 1 <cls> >> cls/statelog/cls_statelog.cc:306: Loaded log class! >> -2> 2015-03-22 16:32:42.394181 7f4da7fa0780 1 <cls> >> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! >> -1> 2015-03-22 16:32:42.394476 7f4da7fa0780 1 <cls> >> cls/version/cls_version.cc:227: Loaded version class! >> 0> 2015-03-22 16:32:43.076434 7f4da7fa0780 -1 osd/PG.cc: In function >> 'static epoch_t PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::bufferlist*)' thread 7f4da7fa0780 time 2015-03-22 16:32:43.075101 >> osd/PG.cc: 2270: FAILED assert(values.size() == 1) >> >> ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> 1: (PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::buffer::list*)+0x4d7) [0x70ebf7] >> 2: (OSD::load_pgs()+0x14ce) [0x694efe] >> 3: (OSD::init()+0x11be) [0x69cffe] >> 4: (main()+0x1d09) [0x5c3509] >> 5: (__libc_start_main()+0xed) [0x7f4da5bde76d] >> 6: /usr/bin/ceph-osd() [0x5c6e1d] >> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to >> interpret this. >> >> --- logging levels --- >> 0/ 5 none >> 0/ 1 lockdep >> 0/ 1 context >> 1/ 1 crush >> 1/ 5 mds >> 1/ 5 mds_balancer >> 1/ 5 mds_locker >> 1/ 5 mds_log >> 1/ 5 mds_log_expire >> 1/ 5 mds_migrator >> 0/ 1 buffer >> 0/ 1 timer >> 0/ 1 filer >> 0/ 1 striper >> 0/ 1 objecter >> 0/ 5 rados >> 0/ 5 rbd >> 0/ 5 journaler >> 0/ 5 objectcacher >> 0/ 5 client >> 0/ 5 osd >> 0/ 5 optracker >> 0/ 5 objclass >> 1/ 3 filestore >> 1/ 3 journal >> 0/ 5 ms >> 1/ 5 mon >> 0/10 monc >> 1/ 5 paxos >> 0/ 5 tp >> 1/ 5 auth >> 1/ 5 crypto >> 1/ 1 finisher >> 1/ 5 heartbeatmap >> 1/ 5 perfcounter >> 1/ 5 rgw >> 1/ 5 hadoop >> 1/ 5 javaclient >> 1/ 5 asok >> 1/ 1 throttle >> -2/-2 (syslog threshold) >> -1/-1 (stderr threshold) >> max_recent 10000 >> max_new 1000 >> log_file /var/log/ceph/ceph-osd.1.log >> --- end dump of recent events --- >> 2015-03-22 16:32:43.079587 7f4da7fa0780 -1 *** Caught signal (Aborted) ** >> in thread 7f4da7fa0780 >> >> ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> 1: /usr/bin/ceph-osd() [0x8001ea] >> 2: (()+0xfcb0) [0x7f4da743acb0] >> 3: (gsignal()+0x35) [0x7f4da5bf3425] >> 4: (abort()+0x17b) [0x7f4da5bf6b8b] >> 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f4da654569d] >> 6: (()+0xb5846) [0x7f4da6543846] >> 7: (()+0xb5873) [0x7f4da6543873] >> 8: (()+0xb596e) [0x7f4da654396e] >> 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char >> const*)+0x1df) [0x8c5e7f] >> 10: (PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::buffer::list*)+0x4d7) [0x70ebf7] >> 11: (OSD::load_pgs()+0x14ce) [0x694efe] >> 12: (OSD::init()+0x11be) [0x69cffe] >> 13: (main()+0x1d09) [0x5c3509] >> 14: (__libc_start_main()+0xed) [0x7f4da5bde76d] >> 15: /usr/bin/ceph-osd() [0x5c6e1d] >> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to >> interpret this. >> >> --- begin dump of recent events --- >> 0> 2015-03-22 16:32:43.079587 7f4da7fa0780 -1 *** Caught signal (Aborted) >> ** in thread 7f4da7fa0780 >> >> ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> 1: /usr/bin/ceph-osd() [0x8001ea] >> 2: (()+0xfcb0) [0x7f4da743acb0] >> 3: (gsignal()+0x35) [0x7f4da5bf3425] >> 4: (abort()+0x17b) [0x7f4da5bf6b8b] >> 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f4da654569d] >> 6: (()+0xb5846) [0x7f4da6543846] >> 7: (()+0xb5873) [0x7f4da6543873] >> 8: (()+0xb596e) [0x7f4da654396e] >> 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char >> const*)+0x1df) [0x8c5e7f] >> 10: (PG::peek_map_epoch(ObjectStore*, coll_t, hobject_t&, >> ceph::buffer::list*)+0x4d7) [0x70ebf7] >> 11: (OSD::load_pgs()+0x14ce) [0x694efe] >> 12: (OSD::init()+0x11be) [0x69cffe] >> 13: (main()+0x1d09) [0x5c3509] >> 14: (__libc_start_main()+0xed) [0x7f4da5bde76d] >> 15: /usr/bin/ceph-osd() [0x5c6e1d] >> NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to >> interpret this. >> >> --- logging levels --- >> 0/ 5 none >> 0/ 1 lockdep >> 0/ 1 context >> 1/ 1 crush >> 1/ 5 mds >> 1/ 5 mds_balancer >> 1/ 5 mds_locker >> 1/ 5 mds_log >> 1/ 5 mds_log_expire >> 1/ 5 mds_migrator >> 0/ 1 buffer >> 0/ 1 timer >> 0/ 1 filer >> 0/ 1 striper >> 0/ 1 objecter >> 0/ 5 rados >> 0/ 5 rbd >> 0/ 5 journaler >> 0/ 5 objectcacher >> 0/ 5 client >> 0/ 5 osd >> 0/ 5 optracker >> 0/ 5 objclass >> 1/ 3 filestore >> 1/ 3 journal >> 0/ 5 ms >> 1/ 5 mon >> 0/10 monc >> 1/ 5 paxos >> 0/ 5 tp >> 1/ 5 auth >> 1/ 5 crypto >> 1/ 1 finisher >> 1/ 5 heartbeatmap >> 1/ 5 perfcounter >> 1/ 5 rgw >> 1/ 5 hadoop >> 1/ 5 javaclient >> 1/ 5 asok >> 1/ 1 throttle >> -2/-2 (syslog threshold) >> -1/-1 (stderr threshold) >> max_recent 10000 >> max_new 1000 >> log_file /var/log/ceph/ceph-osd.1.log >> --- end dump of recent events --- >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> ________________________________ >> >> PLEASE NOTE: The information contained in this electronic mail message is >> intended only for the use of the designated recipient(s) named above. If the >> reader of this message is not the intended recipient, you are hereby >> notified that you have received this message in error and that any review, >> dissemination, distribution, or copying of this message is strictly >> prohibited. If you have received this communication in error, please notify >> the sender by telephone or e-mail (as shown above) immediately and destroy >> any and all copies of this message in your possession (whether hard copies >> or electronically stored copies). >> > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
