Re: Writing big XML files where beginning depends on end.

""" Then we need something that allows parts of the XML file to be written to file and purged from RAM to avoid the memory problem. Suggestions for solutions are appreciated. """ Multiple XML files is not an option, but what about general entities or XInclude? That way you don't need to change

Re: Writing big XML files where beginning depends on end.

Magnus Lycka wrote: > We're using DOM to create XML files that describes fairly > complex calculations. The XML is structured as a big tree, > where elements in the beginning have values that depend on > other values further down in the tree. Imagine something > like below, but much bigger and much

Re: Writing big XML files where beginning depends on end.

Gerard Flanagan wrote: > what about multiple xml files? We have deployed code that relies of the XML files looking the way they do, and we'd prefer not to change that. -- http://mail.python.org/mailman/listinfo/python-list

Re: Writing big XML files where beginning depends on end.

Magnus Lycka wrote: > We're using DOM to create XML files that describes fairly > complex calculations. The XML is structured as a big tree, > where elements in the beginning have values that depend on > other values further down in the tree. Imagine something > like below, but much bigger and muc

Re: Writing big XML files where beginning depends on end.

Magnus Lycka wrote: > This won't help if we have problems keeping the whole > structure / call graph in memory at one time. Ah, I had assumed that the main problem was just that the resulting DOM was too big. Still, I don't think there would be a problem if you just constructed a very small tree

Re: Writing big XML files where beginning depends on end.

Ben Sizer wrote: > output = [] > > def write_tree(node): > output.append("") > sum = 0 > for child in reversed(node.children): > if child.type = leaf: > output.extend(["", "%d" % node.value, ""]) > sum += node.value > else: > sum += w

Re: Writing big XML files where beginning depends on end.

Fredrik Lundh wrote: > what's the typical overall structure for this tree ? is it short and wide, > or tall and > narrow ? That varies quite a bit. It's basically a call graph for a functional language, where also aggregated results are stored in the XML tree. In other words, the shape depends o

Re: Writing big XML files where beginning depends on end.

output = [] def write_tree(node): output.append("") sum = 0 for child in reversed(node.children): if child.type = leaf: output.extend(["", "%d" % node.value, ""]) sum += node.value else: sum += write_tree(node) output.append("" %

Re: Writing big XML files where beginning depends on end.

Magnus Lycka wrote: > We're using DOM to create XML files that describes fairly > complex calculations. The XML is structured as a big tree, > where elements in the beginning have values that depend on > other values further down in the tree. Imagine something > like below, but much bigger and muc

Re: Writing big XML files where beginning depends on end.

Magnus Lycka wrote: > > >In some cases, building up a DOM tree in memory takes up >several GB of RAM, which is a real showstopper. The actual >file is maybe a magnitute smaller than the DOM tree. The >app is using libxml2. It's actually written in C++. Some >library that used much less memory over

Writing big XML files where beginning depends on end.

We're using DOM to create XML files that describes fairly complex calculations. The XML is structured as a big tree, where elements in the beginning have values that depend on other values further down in the tree. Imagine something like below, but much bigger and much more complex: