I am working for client that needs to persist 100K-200K records per second
for later querying.  As a proof of concept, we are looking at several
options including nosql (Cassandra and MongoDB).

I have been running some tests on my laptop (MacBook Pro, 4GB RAM, 2.66 GHz,
Dual Core/4 logical cores) and have not been happy with the results.

The best I have been able to accomplish is 100K records in approximately 30
seconds.  Each record has 30 columns, mostly made up of integers.  I have
tried both the Hector and Pelops APIs, and have tried writing in batches
versus one at a time.  The times have not varied much.

I am using the out of the box configuration for Cassandra, and while I know
using 1 disk will have an impact on performance, I would expect to see
better write numbers than I am.

As a point of reference, the same test using MongoDB I was able to
accomplish 100K records in 3.5 seconds.

Any tips would be appreciated.

- Steve

Reply via email to