<[EMAIL PROTECTED]> wrote: > Diez> Sion Arrowsmith wrote: > >> I've got an established client-server application here where there > >> is now a need to shovel huge amounts of data (structured as lists of > >> lists) between the two, and the performance bottleneck has become > >> the amount of time spent parsing XML ... > Diez> CORBA. Or TCP/IP directly. I know that isn't really the answer > Diez> you're looking for - but there is an overhead parsing XML, and > Diez> that can't go away. >Ah, but if both ends of the pipe happen to be running Python, you can >cheat. ;-) Run your list through marshal.dumps() before passing it to >xmlrpclib, then undo the marshalling on the other end. > [ ... ] >If you can swing it, I'd be willing to bet you a beer your XML parsing time >will all but disappear.
I wouldn't take you up on that bet, because I'd already considered that as a likely solution, and while I was asking here another member of the team went away and implemented it for me. Except with cPickle (as suggested elsewhere in the thread) -- we've got DateTime objects in there, which marshal chokes on. Initial tests show you'd've comfortably won your beer: 16s XML parsing down to 0.5s parsing plus 1.5s unpickling. >If you're not using xmlrpclib with some sort of C helper (like sgmlop), try >that as well. Big difference parsing XML in C and Python. Diez edited out the bit where I said we were using sgmlop and worried about what it would be like if we weren't 8-) -- \S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/ ___ | "Frankly I have no feelings towards penguins one way or the other" \X/ | -- Arthur C. Clarke her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
-- http://mail.python.org/mailman/listinfo/python-list
