Re: key value store restore time

2016-02-19 Thread Tao Feng
Hi Leo, At linkedin when we switched to using RocksDB for Samza last year, we did some tests to see how well RocksDB performs. We used the rocksdb microbenchmark( https://github.com/facebook/rocksdb/blob/master/java/benchmark/src/main/java/org/rocksdb/benchmark/DbBenchmark.java) to conduct serval

Re: key value store restore time

2016-02-18 Thread Jagadish Venkatraman
Samza 0.10 introduces the feature of Yarn host affinity for this exact reason. For jobs that need to bootstrap lots of state, downtime during bootstrapping is not acceptable. In our production usecases, we've observed bootstrap times from 25 mins to about 30 seconds. Please refer https://samza.apa