Re: Read Before Writes on Distributed Counters

2013-10-17 Thread wjossey
And, just to close the loop, I went ahead and patched the Go library to support the above functionality. Thanks for the help everyone. -- View this message in context: http://riak-users.197444.n3.nabble.com/Read-Before-Writes-on-Distributed-Counters-tp4029492p4029513.html Sent from the Riak

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Russell Brown
On 17 Oct 2013, at 17:21, Jeremiah Peschka wrote: > When you 'update' a counter, you send in an increment operation. That's added > to an internal list in Riak. The operations are then zipped up to provide the > correct counter value on read. The worst that you'll do is add a large(ish) > num

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Russell Brown
Hi Daniil, On 17 Oct 2013, at 16:55, Daniil Churikov wrote: > Correct me if I wrong, but when you blindly do update without previous read, > you create a sibling, which should be resolved on read. In case if you make > a lot of increments for counter and rarely reads it will lead to siblings > e

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Jeremiah Peschka
That's why I linked to the video - it's 60 minutes of Cribbs™ brand pedantry. --- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Thu, Oct 17, 2013 at 10:45 AM, Sean Cribbs wrote: > Since Jeremiah loves it when I'm

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Sean Cribbs
Since Jeremiah loves it when I'm pedantic, it bears mentioning that the list of operations are rolled up immediately (not kept around), grouping by which partition took the increment. So if I increment by 2 and then by 50, and the increment goes to different replicas, my counter will look like [{a,

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Sean Cribbs
The reasons counters are interesting are: 1) You send an "increment" or "decrement" operation rather than the new value. 2) Any conflicts that were created by that operation get resolved automatically. So, no, sibling explosion will not occur. On Thu, Oct 17, 2013 at 3:55 PM, Daniil Churikov w

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Jeremiah Peschka
When you 'update' a counter, you send in an increment operation. That's added to an internal list in Riak. The operations are then zipped up to provide the correct counter value on read. The worst that you'll do is add a large(ish) number of values to the op list inside Riak. Siblings will be crea

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Daniil Churikov
Correct me if I wrong, but when you blindly do update without previous read, you create a sibling, which should be resolved on read. In case if you make a lot of increments for counter and rarely reads it will lead to siblings explosion. I am not familiar with new counters datatypes, so I am curio

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Russell Brown
I have some from a while back, if I can find my graphs I'll put them up somewhere. Cheers Russell On 17 Oct 2013, at 16:35, Weston Jossey wrote: > Great everyone, thank you. > > @Russell: I specifically work with either Go > (https://github.com/tpjg/goriakpbc) or Ruby (basho client). I

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Weston Jossey
Great everyone, thank you. @Russell: I specifically work with either Go ( https://github.com/tpjg/goriakpbc) or Ruby (basho client). I haven't tested the ruby client, but I'd assume it will perform the write without the read (based on my reading of the code). The Go library, on the other hand,

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Russell Brown
Hi Wes, The client application does not need to perform a read before a write, the riak server must read from disk before updating the counter. Or at least it must with our current implementation. What PRs did you have in mind? I'm curious. Oh, it looks like Sam beat me to it…to elaborate on h

Re: Read Before Writes on Distributed Counters

2013-10-17 Thread Sam Elliott
It is perfectly safe with Counters to "blindly" issue an update. Clients (for counters) should allow a way to blindly send updates. You should only be aware that your updates are *not* idempotent - if you retry an update to a counter, both updates could be preserved. Sam -- Sam Elliott Enginee