Any performance comparison / best practice advice for choosing a riak backend ?

2010-08-28 Thread Neville Burnell
Hi, I'm new to riak, and have been busily reading though the wiki, watching the videos, and catching up on the mail list, so I will have lots of questions over the next few weeks - so sorry To begin, I'm curious about the characteristics of the seven backends for riak [1] 1. riak_kv_bitcask_

Re: Any performance comparison / best practice advice for choosing a riak backend ?

2010-08-29 Thread Neville Burnell
quot;; the latter uses a > global LRU timeout. If you want to use several of the backends in the same > cluster (for different buckets), use the "riak_kv_multi_backend" and > configure each backend separately. > > Sean Cribbs > Developer Advocate > Basho Technologies

Re: Practical limit on total size?

2010-09-04 Thread Neville Burnell
I guess the corollary to his question is: Are there any size/performance details of large riak db instances and the hardware they are running on ?? Thanks, Neville Burnell On 5 September 2010 14:16, Sean Cribbs wrote: > There are no architectural limits on the size of your entire data

Re: Riak Search

2010-09-19 Thread Neville Burnell
Hey Mark, Any chance of getting hold of a beta or some docs? Kind Regards Neville Waiting excitedly for Riak Search On 24 August 2010 05:51, Mark Phillips wrote: > Hey Nickolay, > > On Tue, Aug 17, 2010 at 10:46 AM, Nickolay Platonov > wrote: > > Hello, > > > > Any information when the Riak

Riak Search

2010-10-11 Thread Neville Burnell
Great work to release Riak 0.13 and Riak Search. I'm curious to understand how much of Riak Search is pure Erlang, and when does Riak Search use the JVM? Also are the Riak Search Indexes and config/schema files all stored in Bitcask, or does Riak Search use the local node file system for storage

Re: Riak Search

2010-10-11 Thread Neville Burnell
es, Inc. > d...@basho.com > > > > On Mon, Oct 11, 2010 at 3:40 PM, Neville Burnell < > neville.burn...@gmail.com> wrote: > >> Great work to release Riak 0.13 and Riak Search. >> >> I'm curious to understand how much of Riak Search is pure Erlang, a

Riak Search performance experiences ?

2010-10-13 Thread Neville Burnell
Hi, Congratulations on releasing Riak Search - effectively implementing Lucene and Solr in Erlang is a great effort! Since Riak Search has been in use for some time with beta testers, I'm wondering if Basho might share some performance insights, especially if its possible to compare customers who

Re: Riak Search performance experiences ?

2010-10-18 Thread Neville Burnell
> integration index into Riak KV data. The Solr interface is a nice bonus. It > helps users familiar with Solr get started with Riak Search more easily and > it gives users an additional way to access there data, so it is something we > will continue to enhance over future rel

Re: couchdb performace 10x: using NIF for file io

2010-10-24 Thread Neville Burnell
erlang groups link: http://groups.google.com/group/erlang-programming/browse_thread/thread/f667a9a87ada3e7d On 25 October 2010 08:05, NevB wrote: > Hi, > > I came across this thread in the Erlang group and thought the Riak > team might find it interesting. > > Please forgive me if its not relev

Re: RiakSearch Benchmark Test Invitation.

2010-10-28 Thread Neville Burnell
Put it on S3 On 28 October 2010 20:20, francisco treacy wrote: > Very good idea! > > 2010/10/28 Prometheus WillSurvive : > > Hi All, > > We have prepare wikipedia database output ready to submit RiakSearch. It > is > > XML and described format for solr submit. Each file has 20.000 Document > and

Re: RiakSearch Reached Its Limit and gave below Error ..

2010-10-31 Thread Neville Burnell
Have you increased your ulimit? On 31 October 2010 19:23, Prometheus WillSurvive < prometheus.willsurv...@gmail.com> wrote: > Hi, > > We started a batch index test (wikipedia) when we reached around 600K > docs system gave below error.. Any idea ? > > We can not index any more doc in this ind

Riak and Locks

2010-11-08 Thread Neville Burnell
Are there any plans for a Distributed Lock Service for Riak, to allow for apps that *need* locking for some KV ? A lease based DLM based on something like PaxosLease [1] would be a great service. Kind Regards Neville [1] http://scalien.com/pdf/PaxosLease.pdf

Re: Riak and Locks

2010-11-08 Thread Neville Burnell
10:35 PM, Neville Burnell > wrote: > > > Are there any plans for a Distributed Lock Service for Riak, to allow for > > apps that *need* locking for some KV ? > > It has been discussed and agreed that it would be interesting, but > there is nothing currently being deve

Riak Roadmap ?

2010-11-14 Thread Neville Burnell
Hi, Does riak have a public roadmap somewhere ? I've googled and seen mentions of a roadmap, but failed to get any further. Thanks, Nev ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.bash

Re: Riak Roadmap ?

2010-11-14 Thread Neville Burnell
> Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On Nov 14, 2010, at 8:15 PM, Neville Burnell wrote: > > > Hi, > > > > Does riak have a public roadmap somewhere ? I've googled and seen > mentions of a roa

Re: riak-search default_field

2010-11-17 Thread Neville Burnell
heh, in a previous job working with Solr, my team created the "any" field On 18 November 2010 08:49, Wilson MacGyver wrote: > thanks for the info. that means for multiple fields search > as default. We'd have to create some sort of combined field > > > On Wed, Nov 17, 2010 at 4:07 PM, Dan Reverr

Re: riak-search default_field

2010-11-17 Thread Neville Burnell
t; > On Wed, Nov 17, 2010 at 5:18 PM, Neville Burnell > wrote: > > heh, in a previous job working with Solr, my team created the "any" field > > > > On 18 November 2010 08:49, Wilson MacGyver wrote: > >> > >> thanks for the info. that means for

Re: Whole cluster times out if one node is gone

2010-11-23 Thread Neville Burnell
Just a thought ... have you verified your switch, cables, nics, etc On 24 November 2010 09:33, Jay Adkisson wrote: > (many profuse apologies to Dan - hit "reply" instead of "reply all") > > Alrighty, I've done a little more digging. When I throttle the writes > heavily (2/sec) and set R and W t

Riak and PGM

2010-12-04 Thread Neville Burnell
Does/can Riak use PGM [1] for replicating writes ? It seems PGM would be more efficient when the number of physical nodes starts to get high. Just curious! [1] http://en.wikipedia.org/wiki/Pragmatic_General_Multicast ___ riak-users mailing list riak-us

Re: Riak and PGM

2010-12-04 Thread Neville Burnell
ridge/>[2] https://github.com/rabbitmq/rmq-0mq <https://github.com/rabbitmq/rmq-0mq> On 5 December 2010 17:32, Neville Burnell wrote: > Does/can Riak use PGM [1] for replicating writes ? > > It seems PGM would be more efficient when the number of physical nodes > starts to get

Re: Riak and PGM

2010-12-05 Thread Neville Burnell
And 0MQ has Erlang bindings anyhow, so rabbitMQ now required! http://www.zeromq.org/bindings:erlang On 5 December 2010 18:04, Neville Burnell wrote: > A quick follow up - it seems RabbitMQ now supports 0MQ [1][2] which might > be a good fit with Riak > > From the blog post [1

Re: Riak and PGM

2010-12-05 Thread Neville Burnell
that should be "RabbitMQ *not* required" On 5 December 2010 19:27, Neville Burnell wrote: > And 0MQ has Erlang bindings anyhow, so rabbitMQ now required! > > http://www.zeromq.org/bindings:erlang > > > On 5 December 2010 18:04, Neville Burnell wrote: > >>

Re: Riak and PGM

2010-12-05 Thread Neville Burnell
ld be more than happy if > someone fixes these problems and builds something cool with it. > > I don't think there's really any communication in the Riak model that > would benefit from better multicast. Maybe some ring stuff, but not > the reads or writes. > > On Sun, D

Re: {error,<<"{precommit_fail,notfound}">>}

2010-12-15 Thread Neville Burnell
Cant you simply PUT all the objects again, ie, dont bother deleting? On 16 December 2010 12:49, Xiaopong Tran wrote: > Well, that's what I had been trying to do: delete all the objects > in the bucket and re-insert again. > > So now, I can't delete the objects, and there's no delete-bucket > ope

Re: Performance issues with small dataset

2011-01-12 Thread Neville Burnell
On 13 January 2011 12:33, Alexander Staubo wrote: > Here's something else that is weird. I repeated the steps above on a > new, empty bucket, again using just 1,000 items, but after loading 1.5 > million items into a separate, empty bucket. The numbers now are very > odd: > > * 4.5 seconds to list

Re: Riak Recap for Jan. 12 - 13.

2011-01-14 Thread Neville Burnell
Dumb Question: could Riak be changed to perform read repair before responding, to improve consistency of response? On 15 January 2011 11:12, Sean Cribbs wrote: > > > Crap, the second after I hit "send" the lightbulb goes on! Why is that? > > The quorum _was_ met (all vnodes just migrated to the

Re: Riak Recap for Jan. 12 - 13.

2011-01-14 Thread Neville Burnell
Or to include a "read repair in progress" header to indicate the request should be retried? On 15 January 2011 11:33, Neville Burnell wrote: > Dumb Question: could Riak be changed to perform read repair before > responding, to improve consistency of response? > > On 15 Ja

Fwd: Riak Recap for Jan. 12 - 13.

2011-01-14 Thread Neville Burnell
oops, forgot to include the list >> This sucks, so as an optimization you decided that if a majority of the cluster thinks it's X, well then it must be X! I'm not sure I explained that well, but I'm sure I understand it now :) All true, except when Riak knows X is out of date, and Y is the corre

Re: Getting all the Keys

2011-01-22 Thread Neville Burnell
>As of Riak 0.14 your m/r can filter on key name. I would highly recommend that your data architecture take this into account by using keys that have meaningful names. >>This will allow you to not scan every key in your cluster. Is this part true? I understood that key filtering just means yo