+1 to what Ed said. CFS is a good facilitator for running MR jobs on Cassandra to fill the HDFS requirement (you just want to run MR, but you don't want the whole Hadoop stack). The source data for your MR jobs should be in Cassandra KS/CFs.
On Mon, Nov 18, 2013 at 3:21 PM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > CFS was written so that Brisk (now defunct) did not need a separate hadoop > HDFS stack (NN + DataNodes) to do map reduce work. It is better served as > an alternative to HDFS not as a general purpose distributed file system. > > > On Mon, Nov 18, 2013 at 2:02 PM, Robert Coli <rc...@eventbrite.com> wrote: > >> On Sat, Nov 16, 2013 at 9:10 PM, Willie Slepecki <scpha...@gmail.com>wrote: >> >>> The last issue i'm dealing with before starting to write code is random >>> file storage. The application will have the ability to upload whatever, >>> images, pdf, etc, and i need to put them somewhere. (for the record, >>> Amazon S3 is not an option, long story) So i'm looking at a hugely >>> expensive raid array, or an insanely complex distributed file system. >>> >> >> >>> <cdfs> From what I can tell, it is a file system that does not like >>> small files ... [not a fs] I mean that in the sense, it wasn't >>> designed to map a drive to and drop files in with explorer ... Third, >>> it looks like it commercial, from that stax something company. ... Am >>> i wrong about any of this? >>> >> >> No. >> >> If you don't have the requirement of a POSIX filesystem with locking etc. >> (and if you do, you are probably Doing It Wrong..), you may want to use >> MogileFS. >> >> https://code.google.com/p/mogilefs/ >> >> Summary : >> >> - distributed file system designed to keep redundant copies of arbitrary >> sized files, which are uploaded and accessed via HTTP >> - uses MySQL as the meta-data store, so you keep it available in the same >> way you (probably already know how to) keep MySQL available >> - scales to more files than almost anyone has >> >> =Rob >> >> > -- Ben Coverston DataStax -- The Apache Cassandra Company
