Re: [ceph-users] LevelDB corruption

2017-08-01 Thread Mazzystr
Sorry to take so long in replying I ended up evacuating data and rebuilding using Luminous with BlueStore OSDs. I need my usual drive/host failure testing before going live. Of course other things are burning right now and have my attention. Hopefully I can finish that work in the next few

Re: [ceph-users] LevelDB corruption

2017-06-28 Thread Mazzystr
I should be able to try that tomorrow. I'll report back in afterward On Wed, Jun 28, 2017 at 10:09 PM, Brad Hubbard wrote: > On Thu, Jun 29, 2017 at 11:58 AM, Mazzystr wrote: > > just one MON > > Try just replacing that MON then? > > > > > On Wed, Jun 28, 2017 at 8:05 PM, Brad Hubbard > wrote

Re: [ceph-users] LevelDB corruption

2017-06-28 Thread Brad Hubbard
On Thu, Jun 29, 2017 at 11:58 AM, Mazzystr wrote: > just one MON Try just replacing that MON then? > > On Wed, Jun 28, 2017 at 8:05 PM, Brad Hubbard wrote: >> >> On Wed, Jun 28, 2017 at 10:18 PM, Mazzystr wrote: >> > The corruption is back in mons logs... >> > >> > 2017-06-28 08:16:53.078495 7

Re: [ceph-users] LevelDB corruption

2017-06-28 Thread Mazzystr
just one MON On Wed, Jun 28, 2017 at 8:05 PM, Brad Hubbard wrote: > On Wed, Jun 28, 2017 at 10:18 PM, Mazzystr wrote: > > The corruption is back in mons logs... > > > > 2017-06-28 08:16:53.078495 7f1a0b9da700 1 leveldb: Compaction error: > > Corruption: bad entry in block > > 2017-06-28 08:16:

Re: [ceph-users] LevelDB corruption

2017-06-28 Thread Brad Hubbard
On Wed, Jun 28, 2017 at 10:18 PM, Mazzystr wrote: > The corruption is back in mons logs... > > 2017-06-28 08:16:53.078495 7f1a0b9da700 1 leveldb: Compaction error: > Corruption: bad entry in block > 2017-06-28 08:16:53.078499 7f1a0b9da700 1 leveldb: Waiting after background > compaction error: C

Re: [ceph-users] LevelDB corruption

2017-06-28 Thread Mazzystr
The corruption is back in mons logs... 2017-06-28 08:16:53.078495 7f1a0b9da700 1 leveldb: Compaction error: Corruption: bad entry in block 2017-06-28 08:16:53.078499 7f1a0b9da700 1 leveldb: Waiting after background compaction error: Corruption: bad entry in block On Tue, Jun 27, 2017 at 10:42

Re: [ceph-users] LevelDB corruption

2017-06-27 Thread Mazzystr
22:16 ccallegar: good grief...talk about a handful of sand in your eye! I've been chasing down a "leveldb: Compaction error: Corruption: bad entry in block " in mons logs... 22:17 ccallegar: I ran a python leveldb.repair() and restarted osd's and mons and my cluster crashed and burned 22:18 ccalleg