Yi Xing wrote:
> I need to read a large amount of data into a list. So I am trying to
> see if I'll have any memory problem. When I do
> x=range(2700*2700*3) I got the following message:
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> MemoryError
> Any way to get around this problem? I have a machine of 4G memory. The
> total number of data points (float) that I need to read is in the order
> of 200-300 millions.
If you know that you need floats only, then you can use a typed array
(an array.array) instead of an untyped array (a Python list):
import array
a = array.array("f")
You can also try with a numerical library like scipy, it may support up
to 2 GB long arrays.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
