> Chris (or anyone else), could you comment on ZODB's performance? I've > Googled around a bit and haven't been able to find anything concrete, so > I'm really curious to know how ZODB does with a few hundred thousand > objects. > Specifically, what level of complexity do your ZODB queries/searches have? > Any idea on how purely ad hoc searches perform? Obviously it will be > affected by the nature of the objects, but any insight into ZODB's > performance on large data sets would be helpful. What's the general ratio > of reads to writes in your application?
This is a somewhat weak point of zodb. Zodb simply lets you store arbitrary object graphs. There is no indices created to access these, and no query language either. You can of course create indices yourself - and store them as simply as all other objects. But you've got to hand-tailor these to the objects you use, and create your querying code yourself - no 4gl like sql available. Of course writing queries as simple predicates evaluated against your whole object graph is straightforward - but unoptimized. The retrieval of objects themselves is very fast - I didn't compare to a rdbms, but as there is no networking involved it should be faster. And of course no joins are needed. So in the end, if you have always the same kind of queries that you only parametrize and create appropriate indices and hand-written "execution plans" things are nice. But I want to stress another point that can cause trouble when using zodb and that I didn't mention in replies to Paul so far, as he explicitly didn't want to use an rdbms: For rdbms'ses, a well-defined textual representation of the entities stored in the db is available. So while you have to put some effort on creating on OR-mapping (if you want to deal with objects) that will most likely evolve over time, migrating the underlying data usually is pretty straightforward, and even toolsupport is available. Basically, you're only dealing with CSV-Data that can be easily manipulated and stored back. ZODB on the other side is way easier to code for - but the hard times begin if you have a rolled out application that has a bunch of objects inside zodb that have to be migrated to newer versions and possibly changed object graph layouts. This made me create elaborate yaml/xml serializations to allow for im- and exports and use with xslt and currently I'm investigating a switch to postgres. This point is important, and future developments of mine will take that into consideration more than they did so far. -- Regards, Diez B. Roggisch -- http://mail.python.org/mailman/listinfo/python-list
