---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop
On Thu, Sep 12, 2013 at 9:45 AM, Alex Rice <a...@mindlube.com> wrote:
> Hey all, thanks for the feedback, this is interesting!
>
> re: Those CRDT white papers look pretty complicated, I will definitely
> will leave that to you hardcore mathematicians and computer scientists
> :)
> re: Riak 2.0, I see there are git branches for 2.0 related stuff in
> Riak and in corrugatediron, I am guessing it's still too rough around
> the edges for a newbie like me to be messing around with?
>
I can't speak for Riak 2.0, but CorrugatedIron's support doesn't have any
tests supporting it... I hope to get on that next week to see where things
are. The only hold up is time on my end. I believe the riak branch should
be workable for development. There are features in CI that need clean up,
but I'll start pushing out RC builds through nuget for the adventurous.
> re: RabbitMQ I had heard of it but am hesitant to add another
> daemon/system dependency. But I am definitely considering your
> advice, Guido!
>
> Also for this particular project (a multiplayer game) I may be able to
> get away with Sets, don't actually really need proper Queues. Actually
> I'm not sure. Just thinking out loud here.
>
> For a Set implementation in Riak, I am thinking of modeling it after
> this blog post, I cannot find which at the moment, which described how
> to implement a lock-free Set in Redis. Redis is not distributed, but I
> think the data structure is still relevant:
>
> * To get around the eventual-consistency issue, a prerequisite would
> be to always use R=1 and W=all. This will optimize it for reads
> because there will be a process polling Riak to remove messages out of
> the Set.
> * In the Redis implementation, they used the atomic APPEND operation
> to append entries into the set, as a list but with notation to add or
> remove, prefixed with + or -. e.g.
> +itemA
> +itemB
> -itemA
> +itemC
> => yields { itemB, itemC }
> * Obviously Riak doesn't have atomic appends, however setting a bucket
> to allow_mult = true, and always Put-ing with null vector clock, then
> the Siblings feature should be equivalent. Instead of a value being
> appended to, the Siblings would be populated with +itemA, +itemB,
> -itemA, +itemC, etc.
>
>From an API perspective, you'll do something similar in Riak 2.0. The
CorrugatedIron operation would allow to atomically put individual items
into a set. I'm hoping to create change tracking data structures for use
with the get/put/delete functionality of Riak's server side CRDTs.
> * Any process reading the Set is also be responsible for collating and
> Put-ing the condensed set back, after it reaches some size limit or I
> think they called it 'fragmentation'. This condense and re-Put
> operation would be the tricky part, and would need to use vector clock
> and ensure there are 0 siblings when finished. But it should be
> possible? It seems like this is an uber-simplified form of a CRDT data
> structure?
>
If you're implementing the CRDT on the client side, you'll need to figure
out the garbage collection yourself. If you're able to use Riak's data
structures, then you can safely rely on someone else handling the problems
for you.
If it weren't for my day job, I'd have this CRDT support in CI by now ;)
>
> Thanks,
> Alex
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Sep 12, 2013 at 2:12 AM, Guido Medina <guido.med...@temetra.com>
> wrote:
> > Alex,
> >
> > RabbitMQ which is a good high performer, developed in Erlang and scales
> just
> > as Riak.
> >
> > The old saying, the right tool for the right job, I like how fast Riak is
> > fetching/storing key values on a distributed environment, I don't like
> Riak
> > for queues, is it because it wasn't designed for that? CRDT like CAS
> > operations on structures (to mention some for Java) like
> LinkedBlockingQueue
> > and ConcurrentLinkedQueue should be painful enough to develop in a CRDT
> > environment, cause now you would be talking of guarding states of few
> within
> > the same structure; say AtomicReference of several variables to warranty
> a
> > non-blocking CAS operation, in this case heap and tail of the queue (Not
> so
> > true for LinkedBlockingQueue)
> >
> > And performance of several CRDT for high performance queue I don't
> actually
> > think is going to be good.
> >
> > If I were to use Riak for my big data environment and would like to match
> > Riak on a distributed queueing system then I would use like I said,
> > RabbitMQ.
> >
> > Guido.
> >
> >
> > On 12/09/13 00:29, Alex Rice wrote:
> >>
> >> Hi I'm very new to Riak. The allow_mult = true / Siblings feature is
> >> very interesting. Could it be used to implement a high performance
> >> collection like a Queue or Set, in a lock free manner? The Riak docs
> >> make it sound like allow_mult is mainly for confict resolution and
> >> degenerate cases, rather than a feature which a data structure could
> >> be designed around. Does anyone have any links to share, or thoughts
> >> about this broader issue?
> >> Thanks much!
> >> Alex
> >>
> >> _______________________________________________
> >> riak-users mailing list
> >> riak-users@lists.basho.com
> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
> >
> > _______________________________________________
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> _______________________________________________
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
