Hi, Putting varnish in front of Riak CS nodes seems a good idea, if the load concentrates on small number of videos whose total size fits in varnish's memory. Yet I'm not sure how varnish works in caching. Putting varnish between Riak CS and Riak is not a good idea because they communicates with protocol buffers, not HTTP.
On Tue, Aug 13, 2013 at 2:17 PM, Andre Lohmann <lohmann.an...@gmail.com> wrote: > Hi Kota, > > one idea that came up was to work with varnish. > > You are right, Video Filesize can range from mega to gigybytes. So it would > be usefull, if the cache is self organizing and deleting lesser used files, > when space is needed. > > Varnish is a nice solution for that, but it lacks one problem. The first > range request to a file, before it is fully loaded into the cache can result > in a big latency (there are some workarounds, but they are all not that > satisfying). > > So why not put varnish between RIAK CS and the corresponding RIAK node. If I > got it right, Riak doesn't allow Range requests, but Riak CS does. Also the > communication between them seems to be simple HTTP. Is this might an option, > or are there some things missing, that do not work? > > > > > 2013/8/13 Kota Uenishi <k...@basho.com> >> >> Currently Riak CS and Riak don't have caching system inside. Because >> in the context of Riak CS, file size might range so much, say, from >> kilobytes to terabytes and it's hard to build an efficient caching >> layer. Also, in the context of Riak, once you read a data from Riak, >> it is cached under the kernel disk cache within Riak node. >> I remember some people arguing on using Spark/Shark to cache data on >> top of Riak CS... or if you are distributing the video over the >> internet you might adopt some CDN to do much more efficient caching >> and distributing. >> >> Hope it helps, >> >> >> On Sun, Aug 11, 2013 at 4:50 PM, Andre Lohmann <lohmann.an...@gmail.com> >> wrote: >> > Hi there, >> > >> > say, I have a CS Cluster of 50 Servers filled with Videofiles of in >> > general >> > 1GB of size. >> > >> > I now hit server 1 for a file, then the pieces are spread over a subset >> > of >> > all 50 servers and get fetched now, to be delivered by server 1. >> > >> > When Hitting Server 1 again for the same file, will it be fetched again >> > piece by piece from the other servers, or is there some kind of caching >> > (prefered in memory) available, to deliver this file from cache the next >> > time? >> > >> > -- >> > Andre Lohmann >> > >> > Alte Gärtnerei 20a >> > 24232 Schönkirchen >> > >> > Tel.: +49 (0) 431 260 93 833 >> > mobil: +49 (0) 176 55 910 913 >> > mail: lohmann.an...@gmail.com >> > >> > LinkedIn: http://de.linkedin.com/pub/andre-lohmann/26/a60/a92 >> > Xing: http://www.xing.com/profile/Andre_Lohmann >> > >> > _______________________________________________ >> > riak-users mailing list >> > riak-users@lists.basho.com >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > >> >> >> >> -- >> Kota UENISHI / @kuenishi >> Basho Japan KK > > > > > -- > Andre Lohmann > > Alte Gärtnerei 20a > 24232 Schönkirchen > > Tel.: +49 (0) 431 260 93 833 > mobil: +49 (0) 176 55 910 913 > mail: lohmann.an...@gmail.com > > LinkedIn: http://de.linkedin.com/pub/andre-lohmann/26/a60/a92 > Xing: http://www.xing.com/profile/Andre_Lohmann -- Kota UENISHI / @kuenishi Basho Japan KK _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
