Awesome. I knew there were basho optimizations. Just wondering if there was parity. Cool!
@siculars http://siculars.posthaven.com Sent from my iRotaryPhone > On Jan 27, 2014, at 17:53, Matthew Von-Maszewski <matth...@basho.com> wrote: > > > P.S. I failed to include the link to the wiki page that discusses each of > Basho's optimizations for the Riak environment: > > https://github.com/basho/leveldb/wiki > > >> On Jan 27, 2014, at 8:49 PM, Matthew Von-Maszewski <matth...@basho.com> >> wrote: >> >> Basho's leveldb requirements have lead to different optimizations. Facebook >> has a captive hardware environment and usage case that does not match ours. >> I am not saying their changes are better or worse, only different. >> >> Basho needs: >> >> - multiple databases running simultaneously: 6 to 64 >> - support for slow solitary SATA drives all the way to rocking' fast >> FusionIO PCI arrays >> - fast disaster recovery / repair >> - aggressive delete logic to help remove large chunks of deleted data now >> instead of next year >> - (and a few things still not unleashed in Riak 2.0) >> >> So our compaction strategy has come a long way: from Google's original >> single thread to a thread pool that can run simultaneous companions on the >> same database as well as other databases. We have a dedicated thread that >> constantly computes the speed of the local array and gradually manipulates >> each write request to prevent Google's designed stall … maximizing total >> throughput for the simultaneous databases while maintaining a constant user >> experience. And in between all that, we have retuned the skip list, the >> bloom filter, and integrated Intel's hardware CRC assembly code. >> >> That is what meets Riak's needs. >> >> Matthew >> >> >>> On Jan 27, 2014, at 6:05 PM, Alexander Sicular <sicul...@gmail.com> wrote: >>> >>> Hey Basho gang, >>> >>> Have you all taken any sort of look at FB's leveldb fork, rocksdb, >>> http://rocksdb.org/? Just watching Dhruba's video it seems like it may be a >>> useful upgrade, >>> https://code.facebook.com/videos/174548089418739/rocksdb-a-high-performance-embedded-key-value-store-for-flash-storage-data-scale/. >>> The main talking points seems to be that rocksdb upgrades leveldb such >>> that it is more efficient when storing data more than system ram, has more >>> efficient access patternsto take advantage of ssd's and has a better >>> compaction story. >>> >>> Care to share your thoughts? >>> >>> Thanks, >>> >>> -Alexander Sicular >>> >>> @siculars >>> >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
