With 32 bits address space, you have exactly 4GB. Of course out of that 4GB, the OS reserves some of that space for itself. So I think that 2GB of available memory for a user process is good approximation. However, on top of that, you need to account for space overhead that Python and the OS heap management will generate. I have written an article discussing memory allocation overhead:
http://www3.sympatico.ca/olanglois/CFixAlloc.html It is for C++ but I believe that concepts still apply for Python users and could give the original poster a better understanding about the problem he is facing. I have kept thinking about the original problem and I now believe that the only solution if he wants to store 3.6GB of data in a Python script is to recompile Python in 64 bits. I do not know if this is something that someone has already done successfully... Olivier Langlois http://www.quazal.com > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:python- > [EMAIL PROTECTED] On Behalf Of Burton Samograd > Sent: April 11, 2006 2:10 PM > To: python-list@python.org > Subject: Re: Memory limit to dict? > > Peter Beattie <[EMAIL PROTECTED]> writes: > > <snip> > > I've no idea how far the Windows task manager's resource monitor can be > > trusted -- probably not as far as I could throw a heavy-set St Bernard > > --, but it seems to stop roughly when that monitor records a swap file > > size of 2.2 GB. > > Not being a windows expert at all, but I would assume with 32 bit > windows each process in the system can have an address space of ~2 > gigs. In linux the process address space is split in half, bottom 2 > gigs for OS mappings, top for the process, so it looks like you might > just be hitting the maximum allowed address space mapping. > > You should partition your data into hierarchial modules and let python > do the swapping for you...although you have 16 gigs (I have to put a > holy crap after that!) you will always run into process limits, at > least until true 64 bit os's are in vouge. > > -- > burton samograd kruhft .at. gmail > kruhft.blogspot.com www.myspace.com/kruhft metashell.blogspot.com > -- > http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
