To make it simple. No more networking. Just one node (with n = 1) and local
tests.
The producing of data is a simple CSV file read (ruled out too cause this
is fast).
Switched to bitcask:
{storage_backend, riak_kv_bitcask_backend},
Removed the index from the RiakObject (so now we have simple k/v).
Created a new bucket (just to be "clean").
And rerun the test.
Now it's better:
~450 inserts / second
I still think it's slow.
Hardware is pretty good:
Intel® Core™ i7-920 Quadcore incl. Hyper-Threading Technology
RAM: 8GB DDR3
HDD: 2 x 750 GB SATA 2 (RAID1)
What insert rate should i expect on a normal Debian 6.0 64 bit installation
(no tweaks) ?
I can only compare it with other DBs i have tested on the same machine:
ex: mongodb, kyototycoon
On Sun, Nov 20, 2011 at 10:56 PM, Aphyr <ap...@aphyr.com> wrote:
> On 11/20/2011 12:14 PM, Catalin Constantin wrote:
>
>> I am 100% sure the transfer rate is 10MBytes / second. This is not the
>> problem.
>>
>
> In ten years of network administration I have never encountered an
> ethernet device with a wire rate of 10 MBps. I have, however, encountered
> frequent confusion over units. Perhaps you understand my suspicion here. :)
>
>
> IOWAIT is also pretty low. iostat shows: iowait 4.69%
>>
>
> If you read my email, I suggested that even values as low as 2.6% may
> suggest contention. It depends on your CPU arch and utilization. I would
> investigate your disks more closely. Are they spinning or solid-state?
> Median seek time? 95/99 seek times? Disk cache? Does the riak process spend
> disproportionate time in IO_WAIT relative to USER? Filesystem
> atime/relatime/noatime? FS block size properly aligned for your disk?
> Insufficient filesystem or leveldb cache? hdparm options? Is riak on an
> independent disk or competing with other processes, i.e. syslog, file
> servers, etc? Does strace show an unusual amount of time spent in certain
> system calls?
>
> It might just be leveldb, too. I only have experience with bitcask in
> production.
>
>
> I have retried the test with one node, a new bucket newly created where
>> i have set: n to 1.
>> bucket.set_n_val(1)
>>
>> Results are the same. Less than 300 inserts / second.
>>
>
> This is good; it rules out replication.
>
>
> Any idea why riak is so slow on inserting data ?
>>
>
> Disk, disk disk disk, disk disk? Disk!
>
> You could also look at the client. Are you writing to a local node or a
> remote one? Is your client's threading model getting in the way? Can you
> actually produce data fast enough to insert it? Is your client fighting for
> the same resources as the riak process? Presuming you've ruled these out,
> it's almost certainly network or disk.
>
> --Kyle
>
--
Catalin Constantin
Dazoot Software
http://www.dazoot.eu/
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
