Replication behavior

2010-05-13 Thread Jimmy Thrasher
Hi all, I just added another node to my cluster (moving from 1 to 2) and was surprised by some behavior. As I understand replication, with an n_val of 3 (default), all nodes will have all data until I add my 4th node. First, is this right? Second, I watched my disk usage on node 1 go from 1GB t

Riak Search release date

2010-05-13 Thread Senthilkumar Peelikkampatti
Hi, It has been while Basho announced about Riak Search (I guess it is almost an year back, if I remember correctly), do you have any time frame of release? Or will it be available only to the Enterprise DS customer? I am one of the persons looking for Riak Search after moving from Couchdb to

Re: Riak Search release date

2010-05-13 Thread Eric Gaumer
On Thu, May 13, 2010 at 11:02 AM, Senthilkumar Peelikkampatti < senthilkumar.peelikkampa...@gmail.com> wrote: > Hi, > It has been while Basho announced about Riak Search (I guess it is > almost an year back, if I remember correctly), do you have any time frame of > release? Or will it be ava

Re: Replication behavior

2010-05-13 Thread Justin Sheehy
Hi, Jimmy. With an n_val of 3, there will be 3 copies of each data item in the cluster even when there are less than 3 hosts. With 2 nodes in that situation, each node will have either 1 or 2 copies of each item. Does that help with your understanding? -Justin _

Re: Replication behavior

2010-05-13 Thread Jimmy Thrasher
Ah, yes. So with 1 node, I had a guaranteed 3 copies of the data, and when I added another node, some of those got reduced to 1 or 2 on that node. I understand now, thanks. Jimmy On Thu, May 13, 2010 at 11:34 AM, Justin Sheehy wrote: > Hi, Jimmy. > > With an n_val of 3, there will be 3 copies

Re: FastTrack Slowing Point

2010-05-13 Thread Ted Karmel
For future reference, the problem was the GNU Coreutils version. My version was relatively old (6.5) and did not have -n as an option. I updated GNU Coreutils to 8.5 via this source: http://ftp.gnu.org/gnu/coreutils/ I no longer get the invalid option error. But I now get the following error wh

Re: FastTrack Slowing Point

2010-05-13 Thread Dan Reverri
Running "make dev" first will create the "dev/riak" directory. The "make devrel" command will copy the generated "dev/riak" directory into three additional release directories "dev/dev1", "dev/dev2", "dev/dev3". The name, web_port, pb_port, and handoff_port settings are then updated in the three r

Riak Search release date

2010-05-13 Thread Dan Young
I'm keen on hearing more about this as well. I'm at the National Snow and Ice Data Center and we're looking at an Enterprise Database that will need to support terabytes (and getting close to a petabyte) of research data. Thinking of storing all this in a RDBMSwait...just shoot me! I'm trying

Re: Riak Search release date

2010-05-13 Thread Mark Phillips
Senthilkumar, Eric, others - Thanks for your detailed and thoughtful questions about Riak Search. We are well aware that many of you are building and prototyping applications on Riak and that you are expecting to use Riak Search heavily once it is available. This is great. MapReduce is a great fe

Re: install on freebsd and dragonflybsd

2010-05-13 Thread Vitaly Martynenko
Hi Ryan, I have the same issue under FreeBSD. And your Makefile.test gives this result: CURDIR = "" If change $(CURDIR) to $(.CURDIR) I get CURDIR = "/usr/home/vitalka" I had a little progress with building Riak, but stopped on building erlang_js, can't find a way to build it.

CAP controls

2010-05-13 Thread Jeremy Hinegardner
I am thinking about how to possibly replace an existing system that has heavy I/O load, low CPU usage, with riak. Its a file storage system, with smallish files, a few K normally, but billions of them. The architecture, I think, would be one riak node per disk on the hardware, and probably run ab

Re: CAP controls

2010-05-13 Thread Justin Sheehy
Hi, Jeremy. It sounds like an interesting project. At this time, there is no way to indicate in Riak that two nodes are actually on the same host (and therefore should not overlap in replica sets). It could certainly be done, but to do so today would require modification to the ring partition cl

Re: CAP controls

2010-05-13 Thread Jon Meredith
Hi Jeremy, If it were me I would start off with the simplest set up possible - one riak node per physical machine. It'll be easier to see what is going on and reason about performance if there are less moving parts. For the storage configuration - if you are using innostore then keep one for

Re: CAP controls

2010-05-13 Thread Jeremy Hinegardner
Thanks Justin, Good to know. I'll throw it out there as a feature request :-). Or at least something to think about. I believe it is HDFS that has something along these lines, where you can say which nodes are in which racks and which data centers. This is so it knows who its nearby neighbors

question about postcommit hooks and patch

2010-05-13 Thread Bruce Lowekamp
I've been playing with using postcommit hooks in some code. I couldn't find an example, so looking at the source, I think the right way to set one up is something like: PHook = {struct, [ {<<"mod">>, <>}, {<<"fun">>, <<"notify_change">>}]}, RiakClient:set_bucket(<>, [{postcommit, [PHook]}]), Is

Re: CAP controls

2010-05-13 Thread Jeremy Hinegardner
Thanks Jon, I was thinking in terms of what we have right now, which is many processes per physical machine, each one of them managing a dedicated disk to optimize I/O throughput. If I get a chance to do some prototyping on this, I'll try out the RAID0 option. Although I'm a bit suspect at the mo

Re: question about postcommit hooks and patch

2010-05-13 Thread Andy Gross
Hi Bruce, Thanks for the patch - it's definitely worthwhile and we'll likely commit it to tip soon. - Andy -- Andy Gross VP, Engineering Basho Technologies, Inc. On Thu, May 13, 2010 at 6:16 PM, Bruce Lowekamp wrote: > I've been playing with using postcommit hooks in some code. I > couldn't

returning multiple documents

2010-05-13 Thread Gareth Stokes
hey guys, given that i have an array of keys and want to return the related documents. what would be the best way to do this? so far i can think of two ways 1. open a socket and call RpbGetReq asynchronously for each key 2. run a map reduce through the rest interface and deserialise the results