Diez> Sion Arrowsmith wrote:
>> I've got an established client-server application here where there
>> is now a need to shovel huge amounts of data (structured as lists of
>> lists) between the two, and the performance bottleneck has become
>> the amount of time spent parsing XML ...
Diez> CORBA. Or TCP/IP directly. I know that isn't really the answer
Diez> you're looking for - but there is an overhead parsing XML, and
Diez> that can't go away.
Ah, but if both ends of the pipe happen to be running Python, you can
cheat. ;-) Run your list through marshal.dumps() before passing it to
xmlrpclib, then undo the marshalling on the other end.
>>> biglist = ["abc"] * 50
>>> biglist = [biglist] * 50
>>> import xmlrpclib
>>> import marshal
>>> x = xmlrpclib.dumps((biglist,))
>>> len(x)
92331
>>> x.count("<")
10310
>>> y = xmlrpclib.dumps((marshal.dumps(biglist),))
>>> len(y)
20324
>>> y.count("<")
8
If you can swing it, I'd be willing to bet you a beer your XML parsing time
will all but disappear.
If you're not using xmlrpclib with some sort of C helper (like sgmlop), try
that as well. Big difference parsing XML in C and Python.
Skip
--
http://mail.python.org/mailman/listinfo/python-list
