Yi Xing wrote: > Hi, > > I need to read a large amount of data into a list. So I am trying to > see if I'll have any memory problem. When I do > x=range(2700*2700*3) I got the following message: > > Traceback (most recent call last): > File "<stdin>", line 1, in ? > MemoryError > > Any way to get around this problem? I have a machine of 4G memory. The > total number of data points (float) that I need to read is in the order > of 200-300 millions. >
2700*2700*3 is only 21M. Your computer shouldn't have raised a sweat, let alone MemoryError. Ten times that got me a MemoryError on a 1GB machine. A raw Python float takes up 8 bytes. On a 32-bit machine a float object will have another 8 bytes of (type, refcount). Instead of a list, you probably need to use an array.array (which works on homogenous contents, so it costs 8 bytes each float, not 16), or perhaps numeric/numpy/scipy/... HTH, John -- http://mail.python.org/mailman/listinfo/python-list
