Python Memory Manager
I've been looking at the Python source code recently, more specifically trying to figure out how it's garbage collector works. I've gathered that it uses refcounting as well as some cycle-detection algorithms, but I haven't been able to figure out some other things. Does Python actually have a single 'heap' where all the data is stored? Because PyObject_HEAD seemed to imply to me it was just a linked list of objects, although perhaps I didnt understand this correctly. Also, if it does, how does it deal with memory segmentation? This question bothers me because I've been trying to implement a moving garbage collector, and am not sure how to deal with updating all program pointers to objects on the heap, and thought perhaps an answer to this question would give me some ideas. (Also, if you know any resources for things like this, I'd be grateful for links/names) Thanks in advance, Pie Squared -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Memory Manager
On Feb 17, 1:57 pm, Paul Rubin <http://[EMAIL PROTECTED]> wrote: > Pie Squared <[EMAIL PROTECTED]> writes: > > Also, if it does, how does it deal with memory segmentation? This > > question bothers me because I've been trying to implement a moving > > garbage collector, and am not sure how to deal with updating all > > program pointers to objects on the heap, and thought perhaps an answer > > to this question would give me some ideas. > > As I understand it, Python primarily uses reference counting, with a > mark and sweep scheme for cycle breaking tacked on as an afterthought. > It doesn't move objects in memory during GC so you can get > fragmentation. It's probably not feasible to change this in CPython > without extensive rewriting of CPython and maybe a lot of external C > modules. > > > (Also, if you know any > > resources for things like this, I'd be grateful for links/names) > > If you mean about GC in general, the old book by Jones and Lins is > still standard, I think. Thanks for the quick reply! That answered my question, and I'll check out the book you're referring to - it's exactly what I need, I think. Again, thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Memory Manager
On Feb 17, 3:05 pm, [EMAIL PROTECTED] wrote: > I researched this for some Java I wrote. Try to avoid shuffling > physical memory - you'll write a lot less code and it will be faster, > too. > > Use an "allocated" list and an "available" list. Keep them in address > order. Inserting (moving list elements from insertion point to end) > and deleting (vice-versa) are near-zero cost operations on Intel > boxes. ( Two millis to move a million ints at 1GHz 5 years ago when I > wrotehttp://www.martinrinehart.com/articles/repz.html- probably half > that today.) It seems to me that another, perhaps better strategy, would be to allocate a large heap space, then store a pointer to the base of the heap, the current heap size, and the beginning of the free memory. When you need to 'allocate' more room, just return a pointer to some location in the heap and increment the start-of-free-memory pointer. That way, allocation really IS free, more or less. Wouldn't that be more efficient? Perhaps I'm missing something. As a side note, I'm new to Usenet, so I'm not exactly sure... are 'tangents' like this - since this IS a Python newsgroup, after all - okay? Anyway, thanks for the suggestion. -- http://mail.python.org/mailman/listinfo/python-list
