Just adding one more item to the discussion. I believe this was announced on the list some time ago. I haven't tried it out yet, just pointing it out since it's on the OP's topic:
http://pithos.io/ It's a Cassandra-backed object store using an S3 API. On Tue, Jan 19, 2016 at 2:07 PM, Richard L. Burton III <mrbur...@gmail.com> wrote: > I would ask why do this over say HDFS, S3, etc. seems like this problem > has been solved with other solutions that are specifically designed for > blob storage? > > On Tue, Jan 19, 2016 at 4:23 PM, <l...@airstreamcomm.net> wrote: > >> I recently started noodling with this concept and built a working blob >> storage service using node.js and C*. I setup a basic web server using the >> express web server where you could POST binary files to the server where >> they would get chunked and assigned to a user and bucket, in the spirit of >> S3. Then when you retrieved the file with a GET it would reassemble the >> chunks and push it out. What was really nice about node.js is I could >> retrieve all the chunks in parallel asynchronously. It was just for fun, >> but we have considered fleshing it out for use in our organization. Here >> is the schema I developed: >> >> CREATE TABLE object.users ( >> name text PRIMARY KEY, >> buckets set<text>, >> password text >> ); >> >> CREATE TABLE object.objects ( >> user text, >> bucket text, >> name text, >> chunks int, >> id uuid, >> size int, >> type text, >> PRIMARY KEY ((user, bucket), name) >> ); >> >> CREATE TABLE object.chunks ( >> file_id uuid, >> num int, >> data blob, >> PRIMARY KEY (file_id, num) >> ); >> >> For your purposes you could modify the objects table to keep a revision >> id or timeuuid for looking at previous versions. If you want some insight >> into how the code worked give me a shout. >> > > > > -- > -Richard L. Burton III > @rburton > -- Steve Robenalt Software Architect sroben...@highwire.org <bza...@highwire.org> (office/cell): 916-505-1785 HighWire Press, Inc. 425 Broadway St, Redwood City, CA 94063 www.highwire.org Technology for Scholarly Communication
