A few things:
1. can you provide the output from `riak-admin member-status`?
2. using five VMs per physical node likely means that you're bottleneck is
that host running the VMs not Riak
3. uses a shared disk via iSCSI for storage is most certainly also a
bottleneck
A benchmark that scales linear
While LevelDB does eventually free up space on disk by eliminating deleted
keys during SST compactions it doesn't do so "aggressively", as Matthew
indicated and hence our name for the fix. This is problem you've been
experiencing and we're working on it and hope to have it tested and
integrated in
another offering in
> the key/value storage space. Comparison of 20 hour runs, maybe. But those
> benchmarks need to validate known performance differences of Bitcask and
> Leveldb to establish credibility. 10 minutes runs have barely flushed
> memory caches.
>
> Matthew
>
&
Amir,
I'll add one more major consideration to Ryan's excellent list, check your
network for TCP Incast. Every cluster at reasonable scale will have to manage
this issue carefully, 20 nodes is more than enough to create this kind of
problem (I see it with as few as 9). Here's more information
Max,
This sounds a bit complex, what would need to happen if you didn't process an
event (or batch of events) in time? What about using time-based expiry for
your events which is supported by the Bitcask backend. You could use
Multi-backend to setup a bucket that expires in N seconds. When
A while ago on this list Nico Meyer did an amazingly detailed job of examining
the overhead required when storing data in Riak using Bitcask
(http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-May/004292.html).
That post inspired me to redo our wiki page, which I just published a
