Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

-- Od: Matthew Von-Maszewski Datum: 21. 11. 2012 Předmět: Re: Riak performance problems when LevelDB database grows beyond 16GB map reduce is currently outside my skill set. I have forwarded the question to others on the team. I have also asked the team if they can give me the key specifications

Re: Riak performance problems when LevelDB database grows beyond 16GB

rithm. Matthew On Nov 21, 2012, at 10:52 AM, wrote: > -- Původní zpráva -- >> Od: Matthew Von-Maszewski >> Datum: 22. 10. 2012 >> Předmět: Re: Riak performance problems when LevelDB database grows beyond >> 16GB >> Jan, >> >> ... >&g

Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

-- Původní zpráva -- > Od: Matthew Von-Maszewski > Datum: 22. 10. 2012 > Předmět: Re: Riak performance problems when LevelDB database grows beyond 16GB > Jan, > > ... > The next question from me is whether the drive / disk array problems are your > on

Re: Riak performance problems when LevelDB database grows beyond 16GB

> 1. Did you realize that the "log_jan.txt" file from #1 below documented a hard disk failure? I did not know about the corruption (I did not know that LevelDB logs are human readable application logs), thanks for telling me. I did look at syslog and I did not find any traces of a disk failure

Re: Riak performance problems when LevelDB database grows beyond 16GB

Jan, I apologize for the delayed response. 1. Did you realize that the "log_jan.txt" file from #1 below documented a hard disk failure? You mentioned a failed drive once. I am not sure if this is the same drive. 2. The "sse4_2" tells me that your Intel cpu supports hardware CRC32c calcul

Re: Riak performance problems when LevelDB database grows beyond 16GB

> d. you comment about using O_NOATIME is valid. The issue is that the flag > is relatively new. We are supporting some really old compilers and > linux/solaris versions. It is easier to ask everyone to work noatime at the > mount level than have conditional code for some and mount level tun

Re: Riak performance problems when LevelDB database grows beyond 16GB

lang:nodes() on other nodes). I would suspect a bug in LevelDB, but people > are using it in production, aren't they? > > I intend to retry the test without the software RAID. Any other hints? > > Best regards, Jan > > ------ Původní zpráva -- > Od: Evan Vi

Re: Riak performance problems when LevelDB database grows beyond 16GB

Greetings, I am currently responsible for tuning Google's leveldb implementation for Riak. I have read through most of the thread and have a couple of information requests. Then I will try to address various questions and comments from the thread. In general, you are filling leveldb faster t

Re: Riak performance problems when LevelDB database grows beyond 16GB

rds, Jan -- Původní zpráva -- Od: Evan Vigil-McClanahan Datum: 12. 10. 2012 Předmět: Re: Re: Riak performance problems when LevelDB database grows beyond 16GB Hi there, Jan, The lsof issue is that max_open_files is per backend, iirc, so if you're maxed out you'll see vnode count *

Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

vodní zpráva -- Od: Eli Janssen Datum: 16. 10. 2012 Předmět: Re: Riak performance problems when LevelDB database grows beyond 16GB Anything in the system logs or dmesg? With vm.swappiness set to the defaults, the oom-killer could be doing its job a bit too well. On Oct 15, 2012, at 12:

Re: Riak performance problems when LevelDB database grows beyond 16GB

a/#4Riak_2K_2.1RC2_3d_edited.jpg > ). All the nodes crashed silently, there is nothing interesting in Riak logs. > > Thanks, Jan > > -- Původní zpráva -- > Od: Evan Vigil-McClanahan > Datum: 12. 10. 2012 > Předmět: Re: Re: Riak performance problems when Le

Re: Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

://janevangelista.rajce.idnes.cz/nastenka/#4Riak_2K_2.1RC2_3d_edited.jpg ). All the nodes crashed silently, there is nothing interesting in Riak logs. Thanks, Jan -- Původní zpráva -- Od: Evan Vigil-McClanahan Datum: 12. 10. 2012 Předmět: Re: Re: Riak performance problems when LevelDB

Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

Hi there, Jan, The lsof issue is that max_open_files is per backend, iirc, so if you're maxed out you'll see vnode count * max_open_files. I think on the second try, you may have set the cache too high. I'd drop it back to 8 or 16 MB, and possibly up the open files a bit more, but you don't see

Re: Re: Riak performance problems when LevelDB database grows beyond 16GB

> Can you attach the eleveldb portion of your app.config file? > Configuration problems, especially max_open_files being too low, can > often cause issues like this. > > If it isn't sensitive, the whole app.config and vm.args files are also > often helpful. Hello Evan, thanks for responding. I o

Re: Riak performance problems when LevelDB database grows beyond 16GB

Can you attach the eleveldb portion of your app.config file? Configuration problems, especially max_open_files being too low, can often cause issues like this. If it isn't sensitive, the whole app.config and vm.args files are also often helpful. On Thu, Oct 11, 2012 at 9:12 AM, wrote: > Hello,