I used the array module and loaded all the data into an array. Everything works fine now. On Aug 14, 2006, at 4:01 PM, John Machin wrote:
> Yi Xing wrote: >> Thanks! I just found that that I have no problem with >> x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work. >> > > range(1*2560*2560*30) is creating a list of 196M *unique* ints. > Assuming 32-bit ints and pointers: that's 4 bytes each for the value, 4 > for the type pointer, 4 for the refcount and 4 for the actual list > element (a pointer to the 12-byte object). so that's one chunk of > 4x196M = 786MB of contiguous list, plus 196M chunks each whatever size > gets allocated for a request of 12 bytes. Let's guess at 16. So the > total memory you need is 3920M. > > Now let's look at [[10.0]*2560*2560]*500. > Firstly that creates a tiny list [10.0]. then you create a list that > contains 2560*2560 = 6.5 M references to that *one* object containing > 10.0. That's 26MB. Then you make a list of 500 references to that big > list. This new list costs you 2000 bytes. Total required: about 26.2MB. > The minute you start having non-unique numbers instead of 10.0, this > all falls apart. > > In any case, your above comparison is nothing at all to do with the > solution that you need, which as already explained will involve > array.array or numpy. > > What you now need to do is answer the questions about your pagefile > etc. > > Cheers, > John > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
