Is Riak the right software for me?

2012-03-04 Thread Philip
I am looking for a software to host millions of files in the order of 2-20MB for web serving. A few (~5%) files are "hot" and accessed heavily but most of the files are cold. The system is going to work like a big cache and therefore I need to make sure that the cluster won't go out of space while

riak_core app using riak_kv

2012-03-04 Thread Adam Schepis
What is the best way for a riak_core app to use riak_kv for persistent storage? I thought that riak search did this but haven't found it in that code yet. Should I use the erlang interface via http or protobuf or is there an API or module I can use since my app is running as a member of the cluster

Re: Is Riak the right software for me?

2012-03-04 Thread Philip
It looks like these high wait times occur when the request hits a node which does not store the requested file. The node will download the whole file into memory and after that he will send the data to the client. How about directly pipe'ing the download to the output stream or (even better) redire

Re: Is Riak the right software for me?

2012-03-04 Thread Philip
A memory based caching layer will be added to the frontends so the hot files won't really be a problem. I just mentioned it because the memory layer might fail and result in increased use of the data source. However, I am still not sure if it is possible to use MapReduce to get a ordered list of t

Re: Is Riak the right software for me?

2012-03-04 Thread Shekhar Vemuri
Are you looking for something like this? https://help.basho.com/entries/358219-can-riak-be-used-to-build-a-secure-s3-alternative Ideally you would run a reverse proxy in front of riak. So riak would be your store and the reverse proxy would serve files directly to the client... Sent from my iP

Search quite slow

2012-03-04 Thread Bogunov
Was doing some benchmarks with search and while thinking how to implement something more-or-less relevant i found this: http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/. So I forked and added riak https://github.com/Techmind/opensearch (though b

Unicode String problem

2012-03-04 Thread Buri Arslon
Hi everybody! I can't put unicode string to riak. I was following the riak-erlang-client docs, and this doesn't work: Object = riakc_obj:new(<<"snippet">>, <<"odam">>, <<"Одамлардан тинглаб хикоя">>). ** exception error: bad argument I googled but couldn't find anything meaningful about this

Questions on configuring public and private ips for riak on ubuntu

2012-03-04 Thread Tim Robinson
Hello all, I have a few questions on networking configs for riak. I have both a public ip and a private ip for each riak node. I want Riak to communicate over the private ip addresses to take advantage of free bandwidth, but I would also like the option to interface with riak using the public

Re: Questions on configuring public and private ips for riak on ubuntu

2012-03-04 Thread Alexander Sicular
this is a "Very Bad" idea. do not expose your riak instance over a public ip address. riak has no internal security mechanism to keep people from doing very bad things to your data, configuration, etc. -Alexander Sicular @siculars On Mar 5, 2012, at 12:43 AM, Tim Robinson wrote: > Hello all,

Re: Questions on configuring public and private ips for riak on ubuntu

2012-03-04 Thread Tim Robinson
Right now I am just loading data for test purposes. It's nice to be able to do some benchmarks against the private network (which is @1Gbit/s)... while being able to poke a hole in the firewall when I want to do a test/demo. Tim -Original Message- From: "Alexander Sicular" Sent: Sunda

Re: Questions on configuring public and private ips for riak on ubuntu

2012-03-04 Thread Tim Robinson
Yeah, I read your blog post when it first came out. I liked it. I appreciate the warning, but practically speaking I'm really just not worried about it. It's a test environment on an external VPS that no one knows the info for. Demo to the company means show image/content-type load, JSON via bro

Re: Questions on configuring public and private ips for riak on ubuntu

2012-03-04 Thread Aphyr
ssh -NL 8098:localhost:8098 your.vps.com --Kyle On 03/04/2012 09:55 PM, Tim Robinson wrote: Yeah, I read your blog post when it first came out. I liked it. I appreciate the warning, but practically speaking I'm really just not worried about it. It's a test environment on an external VPS that

Re: licenses (was Re: riakkit, a python riak object mapper, has hit beta!(

2012-03-04 Thread Greg Stein
Hey Andrey, I've spent well over a decade dealing with licensing issues. One thing that I've learned is that licensing is a personal choice and decision, and it is nearly impossible to alter somebody's philosophy. I find people fall into the GPL camp ("free software"), or the Apache/BSD camp ("per