Re: Local state write throughput

2015-01-25 Thread Jay Kreps
To change the readahead amount iirc is something like blockdev --setra 16 /dev/sda where the number is the readahead buffer size in KB I think. -Jay On Sun, Jan 25, 2015 at 3:50 PM, Roger Hoover wrote: > I haven't had a chance to try it yet. Hopefully next week. I'll let you > know what I

Re: Local state write throughput

2015-01-25 Thread Roger Hoover
I haven't had a chance to try it yet. Hopefully next week. I'll let you know what I find. On Sun, Jan 25, 2015 at 2:40 PM, Chris Riccomini wrote: > Awesome, I'll have a look at this. @Roger, did setting this improve your > RocksDB throughput? > > On Sun, Jan 25, 2015 at 12:53 PM, Jay Kreps wr

Re: Check website

2015-01-25 Thread Chris Riccomini
Hey all, Thanks for checking! Milinda, I've opened: https://issues.apache.org/jira/browse/INFRA-9066 To fix the Github issue. Cheers, Chris On Sun, Jan 25, 2015 at 7:58 AM, Milinda Pathirage wrote: > Hi Chris, > > https://github.com/apache/samza still says 'mirrored from > git://git.apache.o

Re: Local state write throughput

2015-01-25 Thread Chris Riccomini
Awesome, I'll have a look at this. @Roger, did setting this improve your RocksDB throughput? On Sun, Jan 25, 2015 at 12:53 PM, Jay Kreps wrote: > I have seen a similar thing from the OS tunable readahead. I think Linux > defaults to reading a full 128K into pagecache with every read. This is > s

Re: Local state write throughput

2015-01-25 Thread Jay Kreps
I have seen a similar thing from the OS tunable readahead. I think Linux defaults to reading a full 128K into pagecache with every read. This is sensible for spinning disks where maybe blowing 500us may mean you get lucky and save a 10ms seek. But for SSDs, especially a key-value store doing purely

Re: Local state write throughput

2015-01-25 Thread Roger Hoover
FYI, for Linux with SSDs, changing the io scheduler to deadline or noop can make a 500x improvement. I haven't tried this myself. http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/hardware.html#_disks On Tue, Jan 20, 2015 at 9:28 AM, Chris Riccomini < criccom...@linkedin.com.inva

Re: Samza has graduated!

2015-01-25 Thread Zhao Weinan
Awesome! Thanks to your works! 2015-01-23 4:55 GMT+08:00 Chris Riccomini : > Hey all, > > The Apache board has unanimously passed the resolution to establish Apache > Samza as a top level project. We've graduated! > > Jakob has opened this umbrella ticket to handle the graduation work that > need

Re: Check website

2015-01-25 Thread Milinda Pathirage
Hi Chris, https://github.com/apache/samza still says 'mirrored from git://git.apache.org/incubator-samza.git'. Thanks Milinda On Sun, Jan 25, 2015 at 2:35 AM, Yan Fang wrote: > Yes, verified. It's working. Thanks! > > Cheers, > > Fang, Yan > yanfang...@gmail.com > +1 (206) 849-4108 > > On Sat,

Review Request 30255: SAMZA-475

2015-01-25 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30255/ --- Review request for samza. Bugs: SAMZA-475 https://issues.apache.org/jira/br