I'm not sure HDFS has the right properties for a media-storage file system. We have, however, built a WebDAV server on top of Cassandra that avoids any pretension of being a general-purpose, POSIX-compliant file system. We mount it on our servers using davfs2, which is also nice for a few reasons:
* We can use standard HTTP load-balancing and dead host avoidance strategies with WebDAV. * Encrypting access and authenticating clients with PKI/HTTPS works seamlessly. * WebDAV + davfs2 is etag-header aware, allowing clients to efficiently validate cached items. * HTTP is browser and CDN/reverse proxy cache friendly for distributing content to people who don't need to mount the file system. * We could extend the server's support to allow connections from a broad variety of interactive desktop clients. On Wed, Jul 6, 2011 at 13:11, Joseph Stein <crypt...@gmail.com> wrote: > Hey folks, I am going to start prototyping our media tier using cassandra as > a file system (meaning upload video/audio/images to web server save in > cassandra and then streaming them out) > Has anyone done this before? > I was thinking brisk's CassandraFS might be a fantastic implementation for > this but then I feel that I need to run another/different Cassandra cluster > outside of what our ops folks do with Apache Cassandra 0.8.X > Am I best to just compress files uploaded to the web server and then start > chunking and saving chunks in rows and columns so the mem issue does not > smack me in the face? And use our existing cluster and build it out > accordingly? > I am sure our ops people would like the command line aspect of CassandraFS > but looking for something that makes the most sense all around. > It seems to me there is a REALLY great thing in CassandraFS and would love > to see it as part of 1.0 =8^) or at a minimum some streamlined > implementation to-do the same thing. > If comparing to HDFS that is part of Hadoop project even though Cloudera has > a distribution of Hadoop :) maybe that can work here too _fingers_crosed_ > (or mongodb->gridfs) > happy to help as I am moving down this road in general > Thanks! > > /* > Joe Stein > http://www.linkedin.com/in/charmalloc > Twitter: @allthingshadoop > */ > -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile]
