Any volunteer to test EMDB as a new Riak backend: https://github.com/alepharchives/emdb
Regards, Zabrane On Sep 30, 2012, at 6:15 PM, Howard Chu wrote: > Kresten Krab Thorup wrote: >> Howard, >> >> I took a look at MDB, and it does look quite promising! >> >> Perhaps you can enlighten us a bit on these aspects of MDB: >> >> - How do you back up an MDB instance? Can you do that while it is actively > updating, or do you need to stop it? I'm asking because the operational ease > of the log-structured stores is one of the features that many Riak'ers like > quite a lot. > > Since this was originally developed for OpenLDAP slapd we currently use > slapcat to backup an MDB instance. That won't be useful in general though, so > we will be providing an mdb_dump/mdb_load utility shortly. There is no need > to stop the database while performing a backup, but... > MDB uses MVCC, so once a read transaction starts, it is guaranteed a > self-consistent view of the database until the transaction ends. > Keeping a read txn open for a very long time will prevent the dirty page > reclaimer from re-using the pages that are referenced by that read txn. As > such, any ongoing write activity will be forced to use new pages. The DB size > can grow very rapidly in these situations, until the read txn ends. > >> - What is the typical response-time distribution for inserts? I've tried >> to > work with BDB some time back, and one of the issues with that is that every > once in a while it slows down quite significantly as B-tree rebalancing makes > some requests unreasonably slow. > > MDB is also a B+tree; at a high level it will have similar characteristics to > BDB. It's always possible for an insert that results in a page split to have > a cascade effect, causing page splits all the way back up to the root of the > tree. But in general MDB is still more efficient than BDB so the actual > variance will be much smaller. > > Also note that if you're bulk loading records in sorted order, using the > MDB_APPEND option basically degenerates into sequential write operations - > when one page fills, instead of splitting it in half as usual, we just > allocate a new sibling page and continue filling it. Page "splits" of this > sort can still ripple upward, but they're so cheap as to be unmeasurable. > >> - Does an MDB instance exploit multiple cores? If so, what is the >> structure > of this usage? In Riak, we have the benefit that a typical Riak node runs > multiple independent databases (one for each data partition/vnode), and so at > least that can provide some concurrency to better leverage I/O and CPU > concurrency. > > Within an MDB environment, multiple readers can run concurrently with a > single writer. Readers are never blocked by anything. (Readers don't block > writers, but as mentioned above, can prevent old pages from being reused.) > >> Kresten >> >> >> On Sep 29, 2012, at 9:50 PM, Kresten Krab Thorup >> <k...@trifork.com<mailto:k...@trifork.com>> wrote: >> >> BETS May be a good template for MDB NIFs since the MDB API looks like bdb. >> It doesn't implement the Riak backend API but an ets subset. >> >> https://github.com/krestenkrab/bets >> >> Kresten >> Trifork >> >> On 28/09/2012, at 13.50, "Howard Chu" >> <h...@symas.com<mailto:h...@symas.com>> wrote: >> >> yaoxinming wrote: >> mdb looks very good ,you can use nif ,maybe you can look the eleveldb >> project,that's a good example. >> >> Thanks for the tip, I'll look into eleveldb. >> >> 2012/9/27 Howard Chu <h...@symas.com<mailto:h...@symas.com> >> <mailto:h...@symas.com>> >> >> Hi, I'm interested in developing a backend using OpenLDAP's memory-mapped >> database library (MDB). Unfortunately Erlang is not a language I've used >> before. Any tips on where to get started? >> >> You can find information on MDB here: http://highlandsun.com/hyc/__mdb/ >> <http://highlandsun.com/hyc/mdb/> >> >> Its read performance is several times faster than anything else out there, >> and scales perfectly linearly since readers require no locks. Its write >> performance is also quite fast... >> -- >> > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
