On Fri, 20 May 2016, LacaK wrote:
Btw is there any limit for number of memory block, which can be requested
?
As far as my XML file has milions of nodes, may be that for each node is
created instance of TDOMNode class (or TDOMElement I do not know), which
in turn can cause that there is large amount of relative small memory
blocks ...
There is no limit, other than the fact that every block has some overhead,
and hence you can allocate less memory in total if you allocate tons of
small blocks instead of a few large ones. It's just 16 or 32 bytes per
block though, so even with millions of blocks you won't lose more than a
few tens of megabytes.
I think that, I found root of my problem.
As far as I have million nodes under few nodes, there is maintained
TDOMNodeList, which stores nodes in TFPList
As new nodes are appended TFPList.Expand is called
And there is:
if FCapacity > 127 then Inc(IncSize, FCapacity shr 2);
So if I have in list 1 000 000 items, then at once list is expanded by 250
000, which causes in one step out of memory
So my question is: can I somehow control increment count ? I think that ATM
no.
So second question is can TFPList.Expand be modified, that for large
FCapacity will be used smaller increment ? :-)
This is of course possible, but I think that you would be served better by
either
- going to 64-bit. Today you have million nodes, tomorrow maybe 2, and then
it will be a problem just as well.
- Using SAX and not store the whole DOM in memory, and at the same time
use your own memory allocation routines for your structures.
Michael.
_______________________________________________
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
