On Oct 9, 2015, at 1:06 PM, Clayton Daley wrote:
> I (OP) was looking into these libraries/options as part of a side project
> that's been delayed by real work. I'm also relegating Mongo to a caching
> layer after stumbling upon Sarah Mei's "Why You Should Never Use MongoDB" and
> realizing I was going to have the exact same problems
>
Thanks for the update!
FWIW, (sorry, non-twisted!) if you want to avoid mongodb:
1. We use redis for KV caching, and its really, really, really fast.
2. Riak can use LevelDB as a backing store -- leveldb supports snappy
compression on documents
3. If you need the document search, i really like Elastic
I don't really like/trust mongodb from past experience myself. I would
normally not use it for anything other than a read-through cache... except for
my current need. We have a web spider and need to archive the snapshots.
WireTiger, which was bought by mongodb and integrated into the 3.x branch, is
the only storage backend I've found that will do "bucket level compression". I
forget if it's applied on pages or blocks, but it will basically apply zlib or
snappy compression onto sets of documents. I looked for weeks on something
that could do that, it was the only option. We did a lot of storage tests on
our typical data, and bucketed compression saves way more space than document
compression or document compression using a seed. _______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
