Luwak is definitely EOL'd and RiakCS is our large object store going forward. It is a far superior design compared to Luwak and handles very large file sizes.
On Thursday, June 26, 2014, Jason Campbell <xia...@xiaclo.net> wrote: > Riak CS is designed to be the same interface as Amazon S3. It can be run > anywhere you can run Riak. > > Think of it as a way to run your own S3. > > I think the hard limit on Riak objects is 50MB, but the recommended size > is less than 1MB per object. The main reason for that is latency. It's > faster to get 100x1MB chunks from the entire cluster than it is to get > 1x100MB chunk from a single server. It's also easier for disks to pull a > 1MB object without delaying any other requests than pulling a 100MB object, > which may end up queuing other requests made after. > > There used to be a project called luwak that was built into Riak and > handled chunking files like this, but it has been EOL'd. Perhaps someone > can shed some light on this. Is RiakCS the final solution for binary > storage? Or is there another way to store larger objects in Riak? > > RiakCS is wonderful, but it does seem overkill for some uses where you > don't need to expose an S3 interface, don't need the S3 permission > structure, and really just want file storage. > > ----- Original Message ----- > From: "Alex De la rosa" <alex.rosa....@gmail.com <javascript:;>> > To: "Hector Castro" <hec...@basho.com <javascript:;>> > Cc: "riak-users" <riak-users@lists.basho.com <javascript:;>> > Sent: Friday, 27 June, 2014 6:55:13 AM > Subject: Re: Riak for streaming video > > > > Hi Hector, > > > Thanks for your response :) highly appreciated. I have more questions > though. > > > Riak CS has to be implemented in Amazon S3? Or can I house it myself with > own managed servers? > > > What is the limit for an object in Standard Riak? Large objects should go > to Riak CS, but what would be a limit for the Standard Riak? I may want to > save pictures, for example, in binary data... not a RAW version, but like > processed photo uploads from users... I think Standard Riak can hold that > with no problem, but it would be good to know the size limit for a key's > content. > > > Thanks, > Alex > > > > On Thu, Jun 26, 2014 at 10:45 PM, Hector Castro < hec...@basho.com > <javascript:;> > wrote: > > > Hey Alex, > > My response are inline below. > > -- > Hector > > > > On Thu, Jun 26, 2014 at 4:26 PM, Alex De la rosa > < alex.rosa....@gmail.com <javascript:;> > wrote: > > Hi Hector, > > > > I see, I always thought that Riak CS was the same as Riak just that you > run > > it in Amazon S3... they are actually different... > > Correct, but it is important to note that Riak is still at the core of > Riak CS. > > > > So... if I have a social network and one of the features is that they can > > share video, I would use a normal Riak cluster for the webapp and data > and a > > Riak CS cluster for the video storage/streaming... am I right? > > That seems like a reasonable conclusion. > > Riak is fundamentally a distributed key/value store for low latency > access to smaller pieces of data. Riak CS is an S3/Swift > API-compatible object storage platform with a little higher request > latency, but also the added ability to house very large objects > (backups, raw images, video). > > > > By the way, couldn't this "range header" be implemented in standard Riak? > > might be a good thing to have so you don't need 2 clusters for this > matter. > > It could some day. Right now, separating the clusters is best because > tuning a single cluster for both use cases would be difficult. > > > > > Thanks, > > Alex > > > > > > On Thu, Jun 26, 2014 at 9:28 PM, Hector Castro < hec...@basho.com > <javascript:;> > wrote: > >> > >> Hi Alex, > >> > >> For this type of problem, you may want to look into Riak CS. [0] It is > >> an object storage platform built on top of Riak. > >> > >> When you GET an object from Riak CS, you can specify a Range header to > >> get the object by its byte offset. [1] > >> > >> -- > >> Hector > >> > >> [0] http://docs.basho.com/riakcs/latest/ > >> [1] > >> > http://docs.basho.com/riakcs/latest/references/apis/storage/s3/RiakCS-GET-Object/#Examples > >> > >> On Thu, Jun 26, 2014 at 3:22 PM, Alex De la rosa > >> < alex.rosa....@gmail.com <javascript:;> > wrote: > >> > Hi there, > >> > > >> > Imagine that I want to use Riak for a video service like Youtube and I > >> > save > >> > the video file in Riak... is there a way to stream the contents of 1 > >> > key? I > >> > know we can stream keys... but what about its content? > >> > > >> > It would be pretty troublesome to have to wait for the full data to be > >> > downloaded $bucket->get('myvideo') to be able to serve it... as they > can > >> > be > >> > pretty big files. > >> > > >> > Is there any recommendation to store/get big files like videos? or it > >> > would > >> > be better to use some other system than Riak for the job? > >> > > >> > Thanks, > >> > Alex > >> > > >> > _______________________________________________ > >> > riak-users mailing list > >> > riak-users@lists.basho.com <javascript:;> > >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >> > > > > > > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com <javascript:;> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com <javascript:;> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
