On Jun 13, 2011, at 9:40 PM, Antonio Rohman Fernandez wrote:
>
> Can a same job be performed by several servers distributed as a
> cluster? or only 1 Pipe server will do the job and Riak KV will be the
> distributed cluster?... the good thing of Hadoop is that you can have a
> Hadoop cluster and
> Riak Pipe on the other hand, could be used to build jobs like you would in
> Hadoop
Can a same job be performed by several servers distributed as a
cluster? or only 1 Pipe server will do the job and Riak KV will be the
distributed cluster?... the good thing of Hadoop is that you can have a
Hado
I think you misunderstand Riak Pipe; it's a distributed processing framework,
not a security layer.
Sean Cribbs
Developer Advocate
Basho Technologies, Inc.
http://basho.com/
On Jun 13, 2011, at 9:09 PM, Eric Fong wrote:
> Hi
>
> Can we use the Riak Pipe to handle all HTTP requests so that we
You can already use Riak KV as persistence for Hadoop (although it can be a bit
intensive on IO), I know of one production setup already doing this. Riak Pipe
on the other hand, could be used to build jobs like you would in Hadoop, with
greater independence and flexibility than Riak KV's standa
Can Riak Pipe be used with Hadoop? That would be wonderful!
Rohman
Sent from my iPad
On Jun 14, 2011, at 12:20 AM, Bryan Fink wrote:
> Hello again, Community.
>
> I'm excited to announce the opening of a new beta-status Basho project
> today: Riak Pipe.
>
> http://github.com/basho/riak_pipe
Hi
Can we use the Riak Pipe to handle all HTTP requests so that we can use that
as security/application layer and remove the need to put something in front
of Riak.
Eric
On Tue, Jun 14, 2011 at 12:20 AM, Bryan Fink wrote:
> Hello again, Community.
>
> I'm excited to announce the opening of a n
Hi Steve,
The reason those files are not being merged is because of the point Justin
made earlier. Expiry is not a condition that affects the merge check.
Stated earlier:
That is, bitcask wasn't originally designed around the expiry-centric
way of removing old data, and data that has simply expi
Q: It looks like I have files in my bitcask directories that are not being
actively used (I've restarted, and they seem to still be a pretty descent
size still, and the mtime is several days old):
root@ha2:/data/riaksearch/bitcask/1027618338748291114361965898003636498195577569280#
ls -la
total
My current app.config bitcask section is looking like this:
%% Bitcask Config
{bitcask, [
{data_root, "/var/lib/riaksearch/bitcask" },
{dead_bytes_merge_trigger, 10242880 },
{dead_bytes_threshold, 5242880 },
{max_file_size, 8000 },
I'm going to experiment with the bitcask max_file_size and reduce it to
80MB or so (my current files are 200+MB) so hopefully, this will force a
merge on the files earlier and will discover the expired records. I'll
let you know how it goes.
- Steve
--
Steve Webb - Senior System Administrato
Hi, Steve.
The key to your situation was in my earlier email:
One note that is relevant for your specific use: the expiry_secs
parameter will cause a given item to disappear from the client
API immediately after expiry, and to be cleaned if it is in a file
already being merged, bu
Ahh, that's HEX notation in erlang. Sorry for the stupid question.
- Steve
--
Steve Webb - Senior System Administrator for gnip.com
http://twitter.com/GnipWebb
On Mon, 13 Jun 2011, Steve Webb wrote:
Dan -
Q: What does the syntax: 16#8000 represent in the max_file_size
parameter? It's
that's a hex number or 2147483648 decimal
Chad DePue
inakanetworks.com - development consulting | skype cdepue | @chaddepue
+1 206.866.5707
On Mon, Jun 13, 2011 at 7:08 PM, Steve Webb wrote:
> Dan -
>
> Q: What does the syntax: 16#8000 represent in the max_file_size
> parameter? It's sup
Dan -
Q: What does the syntax: 16#8000 represent in the max_file_size
parameter? It's supposed to be 2GB, but I can't see where that means 2GB
anywhere.
Even if that meant 16 files of 80MB each, that only comes out to slightly
over 1GB.
- Steve
--
Steve Webb - Senior System Administr
If you still want to build from the source, use some git-fu:
git clone https://github.com/basho/riak.git
cd riak
#make sure you have all the latest branches and tags
git fetch --all
#list all tags
#git tag -l
#checkout the latest stable (riak-0.14.2, i believe)
git checkout riak-0.14.2
Now you've
Hi Steve,
The article points out that the active data file is not considered during
merge checks. Your 250-ish MB data file is the active file and not
considered during the merge check. The file will eventually role over to a
non-active file when it hits 2 GB in size. Once the file is not active i
What about Ubuntu? (assuming your usual Debian package works for Ubuntu)
On Mon, Jun 13, 2011 at 3:37 PM, Sean Cribbs wrote:
> Sorry, I've been corrected by our release manager. We have quit building
> and supporting 32-bit binaries for RHEL as of 0.14.x (0.14.0 was the last,
> and a manual bui
Dan -
I've got dead_bytes_threshold=5242880 (5M) and
dead_bytes_merge_trigger=10242880. My bitcask *.data files are 250-ish MB
in size:
root@ha2:/data/riaksearch/bitcask/1027618338748291114361965898003636498195577569280#
ls -lah
total 771M
drwxr-xr-x 2 riak riak 4.0K 2011-06-12 01:08 .
drw
Thomas,
This is a minor artifact of our build process that will be alleviated by the
merger (we have i386 packages of standard Riak for RHEL already).
Sean Cribbs
Developer Advocate
Basho Technologies, Inc.
http://basho.com/
On Jun 13, 2011, at 5:15 PM, Thomas Fee wrote:
> I noticed the riak-
I noticed the riak-search prebuilt binaries for RedHat are all 64bits. Is
this merely a reflection of memory needs or of something more fundamental?
Mainly for development and QA, it would be bad if the unified riak couldn't
be downloaded and installed for 32bits.
On Mon, Jun 13, 2011 at 3:09 PM,
Riak Users,
I'm happy to announce that Riak Search has now been integrated with Riak in
the master branch.
https://github.com/basho/riak/commit/1a4daa717ce9bb627e72fb7c32dda2ec84e02e92
To use Search you no longer checkout the riak_search repo. Instead, you
checkout riak, build a release as you
Hi,
we recently upgraded to riak-search 0.14.2 and it seems that the numFound
value returned from SOLR searches is no longer correct.
In one particular search, there are actually 22 results. If I set start = 30
and count = 10, I get 0 results as expected. However, no matter what I set
those to, n
On Mon, Jun 13, 2011 at 4:01 PM, Les Mikesell wrote:
> On 6/13/2011 2:40 PM, Bryan Fink wrote:
>>
>> Or if I could take a look at your mock system, maybe I could help with
>> some suggestions for moving in the direction I proposed? We are
>> generally trying to move toward more well-defined APIs
On 6/13/2011 2:40 PM, Bryan Fink wrote:
Or if I could take a look at your mock system, maybe I could help with
some suggestions for moving in the direction I proposed? We are
generally trying to move toward more well-defined APIs (such as
PBC/HTTP) and to discourage people from using distribute
Afternoon, Evening, Morning to All -
For today's Recap: slides, more slides, jobs, code, and more.
Also, I'll be in Portland, Oregon for about the next 24 hours. If
anyone is around and wants to tell me about the Riak cluster you've
just deployed, drinks/food are on me :) Get in touch.
Enjoy -
On Fri, Jun 10, 2011 at 11:18 AM, Kresten Krab Thorup wrote:
> We are using those APIs in our new stuff for riak sync/mobile. My problem
> with your change would be that we have a mock riak_client which is super
> convenient for unit testing, and it would be good to still be able to have
> that
Ok, I've changed my two VMs to each have:
3 CPUs, 1GB ram, 120GB disk
I'm ingesting the twitter spritzer stream (about 10-20 tweets per second,
approx 2k of data per tweet). One bucket is storing the non-indexed
tweets in full. Another bucket is storing the indexed tweet string, id,
date an
Hi Steve,
This Knowledge Base article may be related:
https://help.basho.com/entries/20141178-why-does-it-seem-that-bitcask-merging-is-only-triggered-when-a-riak-node-is-restarted
Thanks,
Dan
Daniel Reverri
Developer Advocate
Basho Technologies, Inc.
d...@basho.com
On Mon, Jun 13, 2011 at 10:2
Justin -
My current bitcask settings are:
%% Bitcask Config
{bitcask, [
{data_root, "/var/lib/riaksearch/bitcask" },
{dead_bytes_merge_trigger, 10242880 },
{dead_bytes_threshold, 5242880 },
{expiry_secs, 86400}
]},
My understandin
Hello again, Community.
I'm excited to announce the opening of a new beta-status Basho project
today: Riak Pipe.
http://github.com/basho/riak_pipe
Riak Pipe is a new way to distribute work around a Riak cluster.
The README explains much more than I can here, but essentially Riak
Pipe allows you
This really does sound like an environmental issue. As a workaround,
you could try running it w/ Rscript --vanilla from command line.
D.
On Thu, Jun 9, 2011 at 3:36 PM, Ken Perkins wrote:
> I've followed the directions verbatim for setting up basho bench. I
> successfully ran a test, but now I g
Nice! I wouldn't mind seeing a cleaned up version if you have a moment...
Thanks!
Mark
On Thu, Jun 9, 2011 at 5:15 PM, Sylvain Niles wrote:
> This calls for silly hats.
> It's a re-cap, heh.
>
>
> http://rookery9.aviary.com.s3.amazonaws.com/8461000/8461482_11a7_625x625.jpg
>
> If anyone likes s
I've been wanting to say this for weeks now...
I really like how communicative and responsive all of you at Basho are with us,
your users.
Thanks for that, as well as for riak and the riak ruby client, which makes
using riak so easy.
Sean and all, your same day response, including code change,
On Mon, Jun 13, 2011 at 9:36 PM, David Smith wrote:
> All,
>
> The next few months will see a number of major new features and
> changes dropping into the tip (aka HEAD) of the Riak codebase. The
> team at Basho is very excited to get these new pieces of code out
> there and we believe that this
All,
The next few months will see a number of major new features and
changes dropping into the tip (aka HEAD) of the Riak codebase. The
team at Basho is very excited to get these new pieces of code out
there and we believe that this upcoming release will make Riak even
more reliable and easier to
35 matches
Mail list logo
