[charlie strauss] > I want to clarify that, on my computer, the first instance of the gap occurs > way > before the memory if filled. (at about 20% of physical ram). Additionally the > process monitor shows no page faults.
Python has no idea of how much RAM you have, or even of how much RAM it's using. See the `gc` module docs, function set_threshold(), for a description of when Python decides to run a cyclic gc pass. That's all about the current excess (if any) of the number of container allocations over the number of container deallocations since the last time cyclic gc ran. > ... > (note that the array if filled as [1]*10, so there is actually only one > "integer", > but 10 array elements refering to it, per foo class.) Nevertheless cyclic gc has to examine all 10 array elements each time the list is scanned. > ... > For example when I write > > me.memory = [1]*nfoo > > perhaps internally, python is allocating an array of size foo and then > __copying__ it > into me.memory??? No. Assignments in Python never copy anything. > Since there is no reference to the intermediate it would then marked for > future > garbage collection. Nothing in Python is "marked for future garbage collection". From time to time /all/ container objects are scanned to determine whether any have become cyclic trash. This takes time proportional to the total number of containees across all container objects. ... > foo was an election ballot holding 10 outcomes, and bar was a set of 100 > ballots > from 100 voting machines, and the total array was holding the ballot sets > from a few > thousand voting machines. > > Almost any inventory program is likely to have such a simmilar set of nested > array, > so it hardly seems unusual. For memory-consumption reasons alone, a serious <wink> such program is likely to use array.array objects at leaf level instead. array.array objects hold raw bits, not Python objects. Since they don't hold Python objects, they can't possibly be part of a cycle, so cyclic gc doesn't need to scan array.array contents either. -- http://mail.python.org/mailman/listinfo/python-list
