Re: Question about Bitcask

2010-11-28 Thread Dan Reverri
No, the portion of Bitcask written in C only manages the in memory keydir structure. Reading and writing of files, coordination of merges, etc. is all done in Erlang. Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Sun, Nov 28, 2010 at 3:20 AM, Kostya V

Re: Python Driver Write Times

2010-11-28 Thread Derek Sanderson
>From just the switch to the PBC Transport, I see this kind of increase: REST HTTP: MakeRiakPerson took 40.139 ms MakeRiakPerson took 40.472 ms MakeRiakPerson took 40.651 ms MakeRiakPerson took 51.630 ms MakeRiakPerson took 36.733 ms ProtoBuffers: MakeRiakPerson took 1.989 ms MakeRiakPerson took

Re: Python Driver Write Times

2010-11-28 Thread Derek Sanderson
About VM resources: I had suspected there would be a hit in this sense, but I wasn't aware of any actual numbers. Thanks for that About 3 writes / the need for 2 more nodes: I had no idea about that (x# of writes per object). Riak is very unfamiliar territory for me. I'll read the guide that has b

Re: Python Driver Write Times

2010-11-28 Thread Derek Sanderson
I'm using the defaults for the python library, so that would be the HTTP Rest interface. There is support for the PBC interface, which I'm looking into using now. I had suspected that since I wasn't really using Riak in such a way as to let it shine (ie, in a cluster of nodes), that might be part

Re: Python Driver Write Times

2010-11-28 Thread Daniel Lindsley
You're using the HTTP interface, which is slower than the Protocal Buffers interface. You should change your code from: riak_conn = RiakClient() bucket = riak_conn.bucket("peopledb") for i in range(1,100): try: new_obj = bucket.new("p" + str(i),MakePerso

Re: Python Driver Write Times

2010-11-28 Thread Alexander Sicular
Keep in mind that if you are using the standard riak install each of your writes are actually written three times to disk. The default configuration is an n_val of 3 for all buckets. For this reason I usually change the default config file or change the n_val of the specific bucket i'm using for

Re: Python Driver Write Times

2010-11-28 Thread Jeremiah Peschka
To add to this, you won't see write speeds as fast as people have reported in a variety of benchmarks because of I/O subsystem virtualization. You take a 10-15% performance hit with virtualized disk when using a pure hypervisor like VMWare ESX. Depending on your VM software, you could be taking a m

Re: Python Driver Write Times

2010-11-28 Thread Greg Steffensen
This is due to two factors: 1) Durability. MongoDB stores writes in RAM and flushes them to disk periodically (by default, every 60 seconds, according to this page: http://www.mongodb.org/display/DOCS/Durability+and+Repair). This means that its writes can seem very, very fast, but if the machine

Python Driver Write Times

2010-11-28 Thread Derek Sanderson
Hello, I've recently started to explore using Riak (v0.13.0-2) from Python (v2.6.5) as a datastore, and I've run into a performance issue that I'm unsure of the true origin of, and would like some input from users who have been working with Riak and its Python drivers. I have 2 tests set up, one

Question about Bitcask

2010-11-28 Thread Kostya V
Is there any way to use it without Erlang, just with C ? ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com