Hi list,
I'm wondering how riaksearch performance will degrade as I add documents.
For my purpose I limit rows at 1k and sorting is not necessary. I have a
single node cluster for development. I know I can increase performance if I
add nodes but I'd like to understand this before I do.
My docu
Sorry I feel compelled to chime in.
Maybe you could assess your physical node limits and start with a small
configuration, then increase it and increase it until you hit a limit.
Work small to large.
Once you find the pain point, lets us know what resource ran out.
You will learn a lot along t
Good points, Dave.
Also, it's worth mentioning that we've seen that many customers and open-source
users think they will need many more nodes than they actually do. Many are able
to start with 5 nodes and are happy for quite a while. The only way to tell
what you actually need is to start with
hi guys,
i'm try to design a log storage based on riak.
the requiment are:
1 the system needs to be able to store any number of these log event
2 the system needs to be able to apply a label to some of these event
3 the user can access the most recent log, and then can go back in time (cannot
deci
I actually did this exact thing in the MongoDB Log4J appender I wrote. I
haven't had time to port it Riak yet, but that's my plan when I get a few spare
minutes. I even wrote a dojo-based front-end for it that I'm beefing up to
include sorting entries, searching, etc... Kind of a graylog2 baby c
Hi Alessandro,
I think it is a god idea. however my question is how do you plan to sort the
data? or is it not important for you?
@jpartogi
On Apr 14, 2011 11:40 PM, "Alessandro Testa" wrote:
> hi guys,
> i'm try to design a log storage based on riak.
> the requiment are:
> 1 the system needs to
1. How much data do you anticipate?
2. Linking to the previous as you suggest really limits the log because it can
then only write one thing at a time, which would be a pity for a Riak :-) I'd
go for a model that does not imply such constraints.
I had some experience recently you can read about
Daniel the max_search_results only applies to searches done via the solr
interface. From
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-January/002974.html:
- System now aborts queries that would queue up too many documents in
a result set. This is controlled by a 'max_searc
Hello.
I've set up a three node Riak cluster on some decent hardware as
specified in the basic cluster setup guide.
I wanted to get a rough idea of performance before tuning the cluster to
my needs, so I've written a Perl script that inserts a sample set of the
sort of data I might be using Riak
Hello Brian,
I am one of the maintainers of Net::Riak, the next version will
support the ProtocolBuffers interface,the write performance of this is
naturally a lot faster.
Theres a trial release ready for download, I am planning to release
this to CPAN next week sometime.
https://github.com/down
I'd be happy to as we are thinking o using this combination where I work :-).
Sent from my iPhone
On Apr 13, 2011, at 7:26 PM, Andrew Thompson wrote:
> I actually have a couple pull requests open to add support for all the
> BSDs for riak. You're welcome to contribute to the effort.
>
> https:
On Wed, Apr 13, 2011 at 7:26 PM, Andrew Thompson wrote:
> I actually have a couple pull requests open to add support for all the
> BSDs for riak. You're welcome to contribute to the effort.
>
> https://github.com/basho/erlang_js/pull/11
>
> https://github.com/basho/skerl/pull/5
>
> Piotr actually
On Thu, Apr 14, 2011 at 09:34:26AM -0700, David Leimbach wrote:
> 1. GNUMakefile instead of Makefile
This would work if we added a regular Makefile that invoked GNU make
because GNU make looks for GNUMakefile first.
> 2. Allow overriding the configuration
Rebar doesn't support this, to my knowledge
On Thu, Apr 14, 2011 at 9:40 AM, Andrew Thompson wrote:
> On Thu, Apr 14, 2011 at 09:34:26AM -0700, David Leimbach wrote:
> > 1. GNUMakefile instead of Makefile
> This would work if we added a regular Makefile that invoked GNU make
> because GNU make looks for GNUMakefile first.
> > 2. Allow over
It works!
https://github.com/basho/erlang_js/commit/25d9edfbe1956ece87ed20454b1e2ed53ee3881d
This makes erlang_js run the right make flavor on both BSD and linux.
Thanks a lot for the suggestion, I think this neatly solves the problem.
Andrew
___
riak
To be clear, I'm only talking about the solr interface. I'm wondering if my
query time will remain fixed (since it's capped at rows=1000) as I add
several million docs to the index.
If I use my search as an input into Map/Reduce, won't my response time grow
with my index? My search query would qu
That's great! Does it still work on Linux too though? :-) I'm having one
of those interrupt heavy days, and all the context switching is making it
difficult to focus on this task :-)
On Thu, Apr 14, 2011 at 10:02 AM, Andrew Thompson wrote:
> It works!
>
>
> https://github.com/basho/erlang_js
We have a exact idea of the amount of data we'll be storing, and the kinds of
machines we'll be storing them on. The simple math of (total data we'll be
storing 6 months from now) / (total capacity of a single node) * (number of
duplicates of each datum we'd like to store for redundancy) gives u
On Thu, Apr 14, 2011 at 10:44:53AM -0700, David Leimbach wrote:
> That's great! Does it still work on Linux too though? :-) I'm having one
> of those interrupt heavy days, and all the context switching is making it
> difficult to focus on this task :-)
>
Yes it still works on linux (this was th
On Thu, Apr 14, 2011 at 10:57 AM, Andrew Thompson wrote:
> On Thu, Apr 14, 2011 at 10:44:53AM -0700, David Leimbach wrote:
> > That's great! Does it still work on Linux too though? :-) I'm having
> one
> > of those interrupt heavy days, and all the context switching is making it
> > difficult t
Hi Morten,
Thanks for sending the log files. I was able to figure out, at least
partially, what's going on here.
The "Failed to compact" message is a result of trying to index a token
that's greater than 32kb in size. (The index storage engine, called
merge_index, assumes tokens sizes smaller tha
Hi Greg,
I played with this a little last night and this morning and I can reproduce
the behavior you are seeing - my two nodes ate more than a combined 15gig of
memory with 16384 partitions and were promptly killed by the O/S.
I haven't had a chance to analyze yet, so this is pure speculation. I
thank you so much for the help.. i do really appreciate it.. now i could move
on to my testing...
--- On Wed, 4/13/11, Mathias Meyer wrote:
> From: Mathias Meyer
> Subject: Re: QUERY: riaksearch
> To: "khyqo"
> Cc: riak-users@lists.basho.com
> Date: Wednesday, April 13, 2011, 6:15 PM
> Hi,
>
good day everyone..
i encountered another problem.. i am confused on how to integrate riaksearch
and mapreduce.. according to
http://blog.basho.com/2010/07/27/webinar-recap---mapreduce-querying-in-riak/
(second question), a webinar would be released but i was not able to find one..
could anyo
Hi there,
this wiki page sums it up pretty nicely:
http://wiki.basho.com/Riak-Search---Querying.html#Querying-Integrated-with-Map-Reduce.
It's just a different kind of input to MapReduce.
In PHP, you could simply do:
$result = $client->search("bucket", "this:that")
->map("function (v) { return
Hi Daniel,
If you use search to provide (streaming) input to map/reduce then you can do
additional processing in the M/R phases to condition and limit your results.
For example you can do additional filtering in a map phase if necessary, as
well as perhaps extracting some subset of the data th
Hi all,
Given the recent discussion around David's riakpool project, I thought
I would share the connection pooling solution that I cooked up. It's
called pooler and you can find it here:
https://github.com/seth/pooler
It is not tightly coupled to Riak's protocol buffer client, but was
des
I've checked out the branch for the last riak release, made a few
modifications to the downloaded rebar.conf files for riak_kv, and one
adjustment to the erlang_js/ebin/.app file (forget whole path), and it looks
like I now have a tarball I can build on FreeBSD, and run some very basic
tests agains
Greg,
What is the size of the HW or VM you plan to deploy as 1000 nodes (memory
and disk space)?
I'm very interested in the trade-off between hardware and software...
Dave
On Thu, Apr 14, 2011 at 2:00 PM, Jon Meredith wrote:
> Hi Greg,
>
> I played with this a little last night and this mornin
On Thu, Apr 14, 2011 at 01:29:49PM -0700, David Leimbach wrote:
> I've checked out the branch for the last riak release, made a few
> modifications to the downloaded rebar.conf files for riak_kv, and one
> adjustment to the erlang_js/ebin/.app file (forget whole path), and it looks
> like I now hav
On Thu, Apr 14, 2011 at 2:01 PM, Andrew Thompson wrote:
> On Thu, Apr 14, 2011 at 01:29:49PM -0700, David Leimbach wrote:
> > I've checked out the branch for the last riak release, made a few
> > modifications to the downloaded rebar.conf files for riak_kv, and one
> > adjustment to the erlang_js
On Thu, Apr 14, 2011 at 02:09:35PM -0700, David Leimbach wrote:
> And now it builds.
Oh yeah, just switching to my branch. I thought you had to patch
something.
>
> To get all the bash scripts to run I hunted down all the instances of:
> #!/bin/bash
>
> and made them
>
> #!/usr/bin/env bash
>
32 matches
Mail list logo
