Re: Storing (python) objects

2011-09-23 Thread Edward Capriolo
On Fri, Sep 23, 2011 at 1:41 PM, Ian Danforth wrote: > Good feedback from all. Thanks! > > Ian > > On Fri, Sep 23, 2011 at 7:48 AM, Tristan Seligmann < > mithra...@mithrandi.net> wrote: > >> On Fri, Sep 23, 2011 at 1:09 AM, Alexis Lê-Quôc >> wrote: >> > For data accessed through a single path, I

Re: Storing (python) objects

2011-09-23 Thread Ian Danforth
Good feedback from all. Thanks! Ian On Fri, Sep 23, 2011 at 7:48 AM, Tristan Seligmann wrote: > On Fri, Sep 23, 2011 at 1:09 AM, Alexis Lê-Quôc wrote: > > For data accessed through a single path, I use the same trick: pickle, > bz2 > > and insert. > > Note that unpickling a pickle in Python inv

Re: Storing (python) objects

2011-09-23 Thread Tristan Seligmann
On Fri, Sep 23, 2011 at 1:09 AM, Alexis Lê-Quôc wrote: > For data accessed through a single path, I use the same trick: pickle, bz2 > and insert. Note that unpickling a pickle in Python involves a) arbitrary code execution, and b) relies on your code being the same (or close enough) to what it wa

Re: Storing (python) objects

2011-09-23 Thread Koert Kuipers
i would advise not to use a language specific storage format, you might regret it later on if you want to add an application to your system that is written in anything else than python. i mean python is great, but it is not necessary the right tool for every job look at thrift/protobuf/avro/bson/j

Re: Storing (python) objects

2011-09-23 Thread David Allsopp
We have done exactly as you describe (nested dicts etc) - works fine as long as you are happy to read the whole lump of data, i.e. don't need to read at a finer granularity. This approach can also save a lot of storage space as you don't have the overhead of many small columns. Some folks also wri

Re: Storing (python) objects

2011-09-22 Thread Alexis Lê-Quôc
On Thu, Sep 22, 2011 at 3:50 PM, Aaron Turner wrote: > On Thu, Sep 22, 2011 at 11:28 AM, Ian Danforth > wrote: > > All, > > I find myself considering storing serialized python dicts in Cassandra. > I'd > > like to store fairly complex, nested dicts, and it's just easier to do > this > > rather

Re: Storing (python) objects

2011-09-22 Thread Aaron Turner
On Thu, Sep 22, 2011 at 11:28 AM, Ian Danforth wrote: > All, >  I find myself considering storing serialized python dicts in Cassandra. I'd > like to store fairly complex, nested dicts, and it's just easier to do this > rather than work out a lot of super columns / columns etc. >  Do others find t

Storing (python) objects

2011-09-22 Thread Ian Danforth
All, I find myself considering storing serialized python dicts in Cassandra. I'd like to store fairly complex, nested dicts, and it's just easier to do this rather than work out a lot of super columns / columns etc. Do others find themselves storing serialized data structures in Cassandra or is