Re: Absolute consistency

2012-01-10 Thread Les Mikesell
On Tue, Jan 10, 2012 at 8:52 PM, Justin Sheehy wrote: > > On Jan 10, 2012, at 9:42 PM, Les Mikesell wrote: > >> How do things like mongo and elasticsearch manage atomic operations >> while still being redundant? > > Most such systems use some variant of primary copy replication, also known as > m

Re: Absolute consistency

2012-01-10 Thread Justin Sheehy
On Jan 10, 2012, at 9:42 PM, Les Mikesell wrote: > How do things like mongo and elasticsearch manage atomic operations > while still being redundant? Most such systems use some variant of primary copy replication, also known as master/slave replication. That approach can provide consistency, b

Re: Absolute consistency

2012-01-10 Thread Les Mikesell
On Tue, Jan 10, 2012 at 4:51 PM, Jon Meredith wrote: > Hi list, > > After some discussion internally we've agreed that setting PR=R, PW=W=DW and > PR+PW > N is insufficient to guarantee reading your writes. How do things like mongo and elasticsearch manage atomic operations while still being redu

Re: Absolute consistency

2012-01-10 Thread Jon Meredith
Hi list, After some discussion internally we've agreed that setting PR=R, PW=W=DW and PR+PW > N is insufficient to guarantee reading your writes. In the case where PR=quorum, PW=quorum, say for N=3 that would mean PR=R=PW=W=DW=2 there is at least one case where you would not be *guaranteed* to rea

Re: Newbie question 1/3

2012-01-10 Thread John DeTreville
That's good to know; thanks. I imagine I may have to vary my physical node count as time goes by, and I wondering how much planning ahead that might take. Going by your example, if my n_val is 4 and an object hashes to partition 6, then my object will be stored only on two physical nodes, right?

Re: Newbie question 1/3

2012-01-10 Thread Jeffrey Massung
On Jan 10, 2012, at 2:46 PM, John DeTreville wrote: > A related question. You say that if my n_val is 3, some data may reside only > on 2 physical nodes. Ignoring failures, might some of it of reside on just > one node? That's a function of ring size and number of nodes in your cluster. For

Re: Newbie question 1/3

2012-01-10 Thread John DeTreville
Excellent answer; thank you. I imagine the unavailability I see will depend strongly on the speed of read repairs. Since I have quite a lot of data, I imagine that they might be quite slow, but I probably can't say more than that without real measurements. A related question. You say that if my

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jason J. W. Williams
That's terrific. Thank you. -J On Tue, Jan 10, 2012 at 1:08 PM, David Smith wrote: > On Tue, Jan 10, 2012 at 12:58 PM, Jason J. W. Williams > wrote: > > How does Riak Control differ from the management tools included in the > > enterprise version? > > Riak Control is a redesign of those manage

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread David Smith
On Tue, Jan 10, 2012 at 12:58 PM, Jason J. W. Williams wrote: > How does Riak Control differ from the management tools included in the > enterprise version? Riak Control is a redesign of those management tools and a release of that functionality to Open Source. We want it to be drop-dead easy for

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jeremy Raymond
Ah, OK thanks. I misread in the Riak Control README of the availability of the self signed certs in Riak 1.0.2 and later as an indication Riak Control was available in 1.0.2. -- Jeremy On Tue, Jan 10, 2012 at 2:56 PM, Andrew Thompson wrote: > Riak control will be part of the 1.1 series, it is

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jeffrey Massung
Jeremy, When you upgrade, your /etc/app.config file isn't altered (we don't want to blow away your previous settings). With a fresh install, there will be a new section in app.config for riak_control, which you can also manually add. But, as Andrew already replied, riak_control will be "offici

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jason J. W. Williams
How does Riak Control differ from the management tools included in the enterprise version? -J On Tue, Jan 10, 2012 at 12:56 PM, Andrew Thompson wrote: > Riak control will be part of the 1.1 series, it is not present in 1.0.2 > or any of the 1.0 releases. > > Andrew > > __

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Andrew Thompson
Riak control will be part of the 1.1 series, it is not present in 1.0.2 or any of the 1.0 releases. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Rebalancing pain and 100 percentile stat

2012-01-10 Thread Ian Plosker
Elias, Which Riak release are you running? Riak 1.0.2? -- Ian Plosker Developer Advocate Basho Technologies On Tuesday, January 10, 2012 at 2:48 PM, Elias Levy wrote: > On Tue, Jan 10, 2012 at 11:27 AM, David Smith (mailto:diz...@basho.com)> wrote: > > > > Rebalancing can be quite expensi

Re: Rebalancing pain and 100 percentile stat

2012-01-10 Thread Elias Levy
On Tue, Jan 10, 2012 at 11:27 AM, David Smith wrote: > > Rebalancing can be quite expensive with the default claim algorithm > that ships in all releases to date. In the 1.0.3 release (dropping > this week, I hope), we will be switching to a new default claim > algorithm that will dramatically de

Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jeremy Raymond
Hi, I was following the instructions here ( https://github.com/basho/riak_control ) to enable Riak Control. After enabling https/ssl and changing the https port (to ) I get a 404 when navigating to https://host:/admin. Would Riak Control be installed when upgrading a cluster or only on fre

Re: Rebalancing pain and 100 percentile stat

2012-01-10 Thread David Smith
On Tue, Jan 10, 2012 at 11:17 AM, Elias Levy wrote: > Is this expected behavior?  Granted, these are EC2 boxes and Leveldb depends > heavily on disk, but I can't imagine folks using Riak on production his this > type of performance hit resulting from rebalancing. tl;dr - yes, adding nodes can cr

Re: Absolute consistency

2012-01-10 Thread Andrew Thompson
Thomas, I just replicated your setup (at least for the PR gets) and you can indeed violate PR/PW when you pause a node on a VM. The reason this happens is that riak's check for PR/PW simply looks at the ring's preflist for a partition and checks that the required number of partitions for that pref

Re: Rebalancing pain and 100 percentile stat

2012-01-10 Thread Joseph Blomstedt
I'll leave the stats questions for someone else, but I'll try to answer your rebalancing questions. First off, what ring size is your cluster? If it's larger than 256, you could be hitting a known gossip overload condition. Generally, anything under 1024 is fine, but for EC2, 256/512 may be the re

Re: Adding a new machine to a three node cluster cause partition handoff problems

2012-01-10 Thread Joseph Blomstedt
> There's a code snippet in riak 1.0.1 or 1.0.2 release notes which addresses > this. Sorry can't find it for you, network here is useless. :( The code snippet in the release notes deals with a different issue. The case where Riak believes all handoff has already occurred, but refuses to actually

Rebalancing pain and 100 percentile stat

2012-01-10 Thread Elias Levy
Good day, Yesterday we went through the exercise of doubling the size of our initial 3 node cluster. The rebalancing took four hours or so. Each node now has between 7 and 8 GB of data stored in the Leveldb backend with an n_val of 3. During rebalancing the cluster became nearly useless. The CP

Re: Adding a new machine to a three node cluster cause partition handoff problems

2012-01-10 Thread Aphyr
There's a code snippet in riak 1.0.1 or 1.0.2 release notes which addresses this. Sorry can't find it for you, network here is useless. :( Ivaylo Panitchkov wrote: > >Hello All, > >We have a cluster of three machines (Debian 6.0, 4GB RAM, >riak_1.0.2-1_amd64.deb, n_val: 3) that serves an appli

Change Riak temp folder location

2012-01-10 Thread Jeremy Raymond
Is there a way to change riak's temp folder location from /tmp/riak to something else? I didn't see an option for this in the app.config or elsewhere. -- Jeremy ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinf

Re: Adding a new machine to a three node cluster cause partition handoff problems

2012-01-10 Thread Joseph Blomstedt
Ivaylo, It appears that handoff is stuck. This should not happen, and I have filed a bug report to track this issue: https://issues.basho.com/show_bug.cgi?id=1317 I believe I know the cause, and believe it is fixed in master leading up to our next major release. I'll update the ticket with my fin

Re: Newbie question 2/3

2012-01-10 Thread John DeTreville
Let me elaborate a tiny bit more. Consider the write(2) syscall on Unix and likealooks. If it succeeds, it returns the number of bytes written. If it fails, it returns -1. One must sometimes learn the hard way that some bytes may have been written even in the case of failure, but that there is

Re: documents disappear from riak search index

2012-01-10 Thread Ryan Zezeski
Typically, something like this would be caused by replicas being lost as Search currently has no anti-entropy. I imagine you would have mentioned replica loss if that was the case, though. You've seen this problem at least twice, correct? Were there other occurrences? If so how many and about h

Re: documents disappear from riak search index

2012-01-10 Thread francisco treacy
2012/1/10 Ryan Zezeski : > When you say "doesn't show up on the search query results *anymore*" does > that imply that at one time they did? Absolutely. As a matter of fact that's what some of my users were asking: "why can't I see X anymore?". >  I'm trying to understand if the index > entries a

Re: Question regarding recent riak_core_vnode_master changes

2012-01-10 Thread Jeff Thompson
Hi Joseph, Thanks for getting back with me, and my apologies that it's taken me so long to respond. Thank you for the explanation! You also came up with the right diagnosis of my problem. I fixed it sometime just before Christmas and everything has been working fine with riak_core since. Once I g

Adding a new machine to a three node cluster cause partition handoff problems

2012-01-10 Thread Ivaylo Panitchkov
Hello All, We have a cluster of three machines (Debian 6.0, 4GB RAM, riak_1.0.2-1_amd64.deb, n_val: 3) that serves an application for a while. As we go to production soon added a fourth machine to the cluster (exactly the same as the first three) yesterday. The partition handoff began in the

Re: documents disappear from riak search index

2012-01-10 Thread Ryan Zezeski
Francisco, When you say "doesn't show up on the search query results *anymore*" does that imply that at one time they did? I'm trying to understand if the index entries appear to have been lost or if it was never successfully written in the first place. Any errors you see in the logs may be rele

Re: missing writes / inconsistent number of keys read after a bulk write

2012-01-10 Thread Ian Plosker
Karthik, I'm think you need to call execute on the StoreObject, like so: > > > riakClient = RiakFactory.pbcClient(); > > > myBucket = > > > riakClient.createBucket("myBucket").nVal(1).execute(); > > > for (int i = 1; i <= 100; ++i) { > > >

Re: Issue in make all

2012-01-10 Thread suresh chandran
Thanks all for the support. I ran the tarball instead of the source :) I shall try these if I need to run the source again. Thanks again Suresh Chandran From: Bip Thelin To: Sean Cribbs Cc: suresh chandran ; "riak-users@lists.basho.com" Sent: Monday, Janu

Re: documents disappear from riak search index

2012-01-10 Thread francisco treacy
Around 5% of the keys on that bucket are missing from the search results. Not reassuring. 2012/1/10 francisco treacy : > Hi all, > > I am running (still) 0.14.2 in production, and using Riak Search to > index certain buckets. Those buckets have the Riak Search pre-commit > hook enabled. > > Once i

documents disappear from riak search index

2012-01-10 Thread francisco treacy
Hi all, I am running (still) 0.14.2 in production, and using Riak Search to index certain buckets. Those buckets have the Riak Search pre-commit hook enabled. Once in a while I get complaints of missing documents from my clients, and it just happened. I checked and the document: - is correctly