Re: Absolute consistency

2012-01-11 Thread Les Mikesell
On Wed, Jan 11, 2012 at 9:14 PM, Joseph Blomstedt wrote: > > Some thoughts to ponder: > > 1. Do you allow multiple clients to write to Riak at the same time? > With concurrent writers, "atomic" can mean multiple things. Do you > want linearizability? Do you want one writer to fail? In our particu

Re: Absolute consistency

2012-01-11 Thread Joseph Blomstedt
entation, there are cases where this is not guaranteed and a request may be sent to a fallback node. This leads to the behavior discussed earlier. Of course, even if we had perfect PW/PR semantics, Riak still only gives you a limited form of "read your own writes" consistency. The labels

Re: Absolute consistency

2012-01-11 Thread Les Mikesell
On Wed, Jan 11, 2012 at 10:56 AM, Vishal Shah wrote: > To add to Ian's comment, for me personally, this specific characteristic is > in fact a very important distinguishing feature of Riak vs other scalable KV > systems. To me, this is what separates Yes, but it makes it unusable for anything tha

Re: Absolute consistency

2012-01-11 Thread Les Mikesell
any key. > This is how Riak makes its availability guarantees, as well as why "absolute > consistency" is difficult. But who makes the decision that a partition needs to migrate and where a key is at any time during that migration? That isn't independently decided by e

Re: Absolute consistency

2012-01-11 Thread Vishal Shah
ere will be N (3 by default) partitions responsible for > storing the associated value. In effect, there are N primaries for any key. > This is how Riak makes its availability guarantees, as well as why "absolute > consistency" is difficult. > > -- > Ian Plosker >

Re: Absolute consistency

2012-01-11 Thread Ian Plosker
why "absolute consistency" is difficult. -- Ian Plosker mailto:i...@basho.com)> Developer Advocate Basho Technologies, Inc. On Tuesday, January 10, 2012 at 10:09 PM, Les Mikesell wrote: > On Tue, Jan 10, 2012 at 8:52 PM, Justin Sheehy (mailto:jus...@basho.com)> wrote: >

Re: Absolute consistency

2012-01-10 Thread Les Mikesell
On Tue, Jan 10, 2012 at 8:52 PM, Justin Sheehy wrote: > > On Jan 10, 2012, at 9:42 PM, Les Mikesell wrote: > >> How do things like mongo and elasticsearch manage atomic operations >> while still being redundant? > > Most such systems use some variant of primary copy replication, also known as > m

Re: Absolute consistency

2012-01-10 Thread Justin Sheehy
On Jan 10, 2012, at 9:42 PM, Les Mikesell wrote: > How do things like mongo and elasticsearch manage atomic operations > while still being redundant? Most such systems use some variant of primary copy replication, also known as master/slave replication. That approach can provide consistency, b

Re: Absolute consistency

2012-01-10 Thread Les Mikesell
On Tue, Jan 10, 2012 at 4:51 PM, Jon Meredith wrote: > Hi list, > > After some discussion internally we've agreed that setting PR=R, PW=W=DW and > PR+PW > N is insufficient to guarantee reading your writes. How do things like mongo and elasticsearch manage atomic operations while still being redu

Re: Absolute consistency

2012-01-10 Thread Jon Meredith
Hi list, After some discussion internally we've agreed that setting PR=R, PW=W=DW and PR+PW > N is insufficient to guarantee reading your writes. In the case where PR=quorum, PW=quorum, say for N=3 that would mean PR=R=PW=W=DW=2 there is at least one case where you would not be *guaranteed* to rea

Re: Absolute consistency

2012-01-10 Thread Andrew Thompson
Thomas, I just replicated your setup (at least for the PR gets) and you can indeed violate PR/PW when you pause a node on a VM. The reason this happens is that riak's check for PR/PW simply looks at the ring's preflist for a partition and checks that the required number of partitions for that pref

Re: Absolute consistency

2012-01-06 Thread Thomas Bakketun
On 2012-01-05 22:52, Tim Robinson wrote: > On a side note, it looks like we've completely highjacked the > "Absolute consistency" question initially proposed. Yes, the answers so far doesn't explain the behaviour I have observed. If that particular key would happen t

Re: Absolute consistency

2012-01-05 Thread Tim Robinson
Got it. Thanks for the responses and the patience. Tim -Original Message- From: "Joseph Blomstedt" Sent: Thursday, January 5, 2012 2:14pm To: "Tim Robinson" Cc: "Aphyr" , riak-users@lists.basho.com Subject: Re: Absolute consistency Internet went down as

Re: Absolute consistency

2012-01-05 Thread Joseph Blomstedt
Internet went down as I was writing an email. Looks like everyone already did a great job answering the availability issues. Although, I might as well chime in as a Basho engineer. On a side note, it looks like we've completely highjacked the "Absolute consistency" question ini

Re: Absolute consistency

2012-01-05 Thread Aphyr
On 01/05/2012 12:53 PM, Tim Robinson wrote: So with the original thread where with N=3 on 3 nodes. The developer believed each node was getting a copy. When in fact 2 copies went to a single node. So yes, there's redundancy and the "shock" value can go away :) My apologies. That said, I have no

Re: Absolute consistency

2012-01-05 Thread Les Mikesell
On Thu, Jan 5, 2012 at 2:53 PM, Tim Robinson wrote: > So with the original thread where with N=3 on 3 nodes. The developer believed > each node was getting a copy. When in fact 2 copies went to a single node. So > yes, there's redundancy and the "shock" value can go away :) My apologies. > > Tha

Re: Absolute consistency

2012-01-05 Thread Jeremiah Peschka
y. > > > -Original Message- > From: "Aphyr" > Sent: Thursday, January 5, 2012 1:29pm > To: "Tim Robinson" > Cc: "Runar Jordahl" , riak-users@lists.basho.com > Subject: Re: Absolute consistency > > On 01/05/2012 12:12 PM, Tim Robinson wrote

Re: Absolute consistency

2012-01-05 Thread Tim Robinson
-- From: "Aphyr" Sent: Thursday, January 5, 2012 1:29pm To: "Tim Robinson" Cc: "Runar Jordahl" , riak-users@lists.basho.com Subject: Re: Absolute consistency On 01/05/2012 12:12 PM, Tim Robinson wrote: > Thank you for this info. I'm still somewhat confused. &g

Re: Absolute consistency

2012-01-05 Thread Aphyr
On 01/05/2012 12:12 PM, Tim Robinson wrote: Thank you for this info. I'm still somewhat confused. Why would anyone ever want 2 copies on one physical PC? Correct me if I am wrong, but part of the sales pitch for Riak is that the cost of hardware is lessened by distributing your data across a clu

Re: Absolute consistency

2012-01-05 Thread Tim Robinson
2012 1:01pm To: "Tim Robinson" Cc: "Runar Jordahl" , riak-users@lists.basho.com Subject: Re: Absolute consistency On 01/05/2012 11:44 AM, Tim Robinson wrote: > Ouch. > > I'm shocked that is not considered a major bug. At minimum that kind of stuff > should be fro

Re: Absolute consistency

2012-01-05 Thread Aphyr
On 01/05/2012 11:44 AM, Tim Robinson wrote: Ouch. I'm shocked that is not considered a major bug. At minimum that kind of stuff should be front and center in their wiki/docs. Here I am thinking n 2 on a 3 node cluster means I'm covered when in fact I am not. It's the whole reason I gave Riak

Re: Absolute consistency

2012-01-05 Thread Tim Robinson
m Robinson" Cc: "Runar Jordahl" , riak-users@lists.basho.com Subject: Re: Absolute consistency shocked? dude. chillax. shocked would be like finding out the pope's gots kids. or we didn't land on the moon. or did we? there is so much magic abstraction going on here and ju

Re: Absolute consistency

2012-01-05 Thread Alexander Sicular
n fact I am not. It's the whole reason I > gave Riak consideration. > > Tim > > -Original Message- > From: "Runar Jordahl" > Sent: Thursday, January 5, 2012 12:13pm > To: "Thomas Bakketun" > Cc: riak-users@lists.basho.com > Subject: Re:

Re: Absolute consistency

2012-01-05 Thread Alexander Sicular
Thursday, January 5, 2012 12:13pm > To: "Thomas Bakketun" > Cc: riak-users@lists.basho.com > Subject: Re: Absolute consistency > > As Alexander pointed out, N does not mean "a separate PC". There was a > thread about this earlier: > > http://lists.basho

Re: Absolute consistency

2012-01-05 Thread Tim Robinson
riginal Message- From: "Runar Jordahl" Sent: Thursday, January 5, 2012 12:13pm To: "Thomas Bakketun" Cc: riak-users@lists.basho.com Subject: Re: Absolute consistency As Alexander pointed out, N does not mean "a separate PC". There was a thread about this ea

Re: Absolute consistency

2012-01-05 Thread Runar Jordahl
As Alexander pointed out, N does not mean "a separate PC". There was a thread about this earlier: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-February/003316.html Kind regards Runar Jordahl blog.epigent.com ___ riak-users mailing l

Re: Absolute consistency

2012-01-05 Thread Alexander Sicular
it's > desirable to have absolute consistency. I though it would be possible to > implement this on top of Riak with the help of pr and wr parameters of > fetch and store, but when testing this I sometimes get worrying results. > > I have at test setup with Riak nodes in three v

Absolute consistency

2012-01-05 Thread Thomas Bakketun
Hello, Riak is based on eventual consistency concept, but sometimes it's desirable to have absolute consistency. I though it would be possible to implement this on top of Riak with the help of pr and wr parameters of fetch and store, but when testing this I sometimes get worrying results. I