On Thu, 14 Jan 2010 02:03:52 +, Steven D'Aprano wrote:
> Again, the technique you are using does a pointless amount of extra
> work. The values in the xrange object are already floats, calling float
> on them just wastes time.
Er what?
Sorry, please ignore that. This is completely untrue --
On Wed, 13 Jan 2010 06:24:04 -0800, Allard Warrink wrote:
> Within a python script I'm using a couple of different lists containing
> a large number of floats (+8M). The execution of this script fails
> because of an memory error (insufficient memory). I thought this was
> strange because I delete
En Wed, 13 Jan 2010 11:24:04 -0300, Allard Warrink
escribió:
Within a python script I'm using a couple of different lists
containing a large number of floats (+8M). The execution of this
script fails because of an memory error (insufficient memory).
I thought this was strange because I delete
Allard Warrink, 13.01.2010 15:24:
so I did some investigation on the memory use of the script. I found
out that when i populated the lists with floats using a for ... in
range() loop a lot of overhead memory is used and that this memory is
not freed after populating the list and is also not freed
Allard Warrink, 13.01.2010 15:24:
I found out that when i populated the lists with floats using a for ... in
range() loop a lot of overhead memory is used
Note that range() returns a list in Python 2.x. For iteration, use
xrange(), or switch to Python 3 where range() returns an iterable.
Ste
On Jan 16, 5:24 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
> Hi!
>
> I made a string parser program, it has a main function and a working
> thread class. When it is running in 24h non-stop, the memory run out.
> I dont Know why. Do anybody know somekind of debugger that can i see
> what is
Hi,
thank you for your comments and your hints (I probably deserve some kind
of subtle irony). I found the problem:
I thought a numpy array A has shape (n,) but actually it had shape
(n,1). In the loop I sampled a value from that array:
v.append(A[i])
So what happened was that I got a view of
En Thu, 15 Nov 2007 10:10:06 -0300, Peter Otten <[EMAIL PROTECTED]>
escribió:
> Rolf Wester wrote:
>
>> Sorry, of course your are wright. I'm running Python2.5 on Linux, my
>> program imports numpy, matplotlib, sys and a python module of my own.
>> This module uses numpy and scipy.weave for imbe
Rolf Wester wrote:
> Diez B. Roggisch wrote:
>> Rolf Wester wrote:
>>> I have a strange (for me) memory problem. When running a loop in a
>>> Python program memory usage increases from about 4% up to 100%. I do a
>>> gc.collect() every loop cycle but this doesn't help. There are about
>>> 67000 o
Sorry, of course your are wright. I'm running Python2.5 on Linux, my
program imports numpy, matplotlib, sys and a python module of my own.
This module uses numpy and scipy.weave for imbedded C-code but no
extension modules. I though the code to be to large to show, I hoped you
could give me hint
Rolf Wester wrote:
> Hi,
>
> I have a strange (for me) memory problem. When running a loop in a
> Python program memory usage increases from about 4% up to 100%. I do a
> gc.collect() every loop cycle but this doesn't help. There are about
> 67000 objects that are tracked by the garbage collector
En Tue, 18 Sep 2007 12:24:46 -0300, Christoph Scheit
<[EMAIL PROTECTED]> escribi�:
>> >> > # add row i and increment number of rows
>> >> > self.rows.append(DBRow(self, self.nRows))
>> >> > self.nRows += 1
>>
>> This looks suspicious, and may indicate that your structure contains
>> cycles, a
Hi, Thank you all very much,
so I will consider using a database. Anyway I would like
how to detect cycles, if there are.
> >> > # add row i and increment number of rows
> >> > self.rows.append(DBRow(self, self.nRows))
> >> > self.nRows += 1
>
> This looks suspicious, and may indicate that you
Christoph Scheit a écrit :
> On Tuesday 18 September 2007 15:10, Marc 'BlackJack' Rintsch wrote:
>> On Tue, 18 Sep 2007 14:06:22 +0200, Christoph Scheit wrote:
>>> Then the data is added to a table, which I use for the actual
>>> Post-Processing. The table is actually a Class with several "Columns"
En Tue, 18 Sep 2007 10:58:42 -0300, Christoph Scheit
<[EMAIL PROTECTED]> escribi�:
> I have to deal with several millions of data, actually I'm trying an
> example
> with
> 360 grid points and 1 time steps, i.e. 3 600 000 entries (and each
> row
> consits of 4 int and one float)
> Of cou
On Tuesday 18 September 2007 15:10, Marc 'BlackJack' Rintsch wrote:
> On Tue, 18 Sep 2007 14:06:22 +0200, Christoph Scheit wrote:
> > Then the data is added to a table, which I use for the actual
> > Post-Processing. The table is actually a Class with several "Columns",
> > each column internally b
On Tue, 18 Sep 2007 14:06:22 +0200, Christoph Scheit wrote:
> Then the data is added to a table, which I use for the actual Post-Processing.
> The table is actually a Class with several "Columns", each column internally
> being represented by array.
Array or list?
> # create reader
> breader = B
Squzer Crawler wrote:
> On Jun 18, 11:06 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
>> On Jun 17, 8:51 pm, Squzer Crawler <[EMAIL PROTECTED]> wrote:
>>
>>> i am developing distributed environment in my college using Python. I
>>> am using therads in client for downloading wepages. Even thou
On Jun 18, 11:06 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
> On Jun 17, 8:51 pm, Squzer Crawler <[EMAIL PROTECTED]> wrote:
>
> > i am developing distributed environment in my college using Python. I
> > am using therads in client for downloading wepages. Even though i am
> > reusing the th
On Jun 17, 8:51 pm, Squzer Crawler <[EMAIL PROTECTED]> wrote:
> i am developing distributed environment in my college using Python. I
> am using therads in client for downloading wepages. Even though i am
> reusing the thread, memory usage get increased. I don know why.? I am
> using BerkelyDB for
I used the array module and loaded all the data into an array.
Everything works fine now.
On Aug 14, 2006, at 4:01 PM, John Machin wrote:
> Yi Xing wrote:
>> Thanks! I just found that that I have no problem with
>> x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work.
>>
>
> range(1
In <[EMAIL PROTECTED]>, Yi Xing wrote:
> Is there a way that I can define a two-dimensional array in
> array.array()? Thanks.
If you need more than one dimension you really should take a look at
`numarray` or `numpy`. What are you going to do with the data once it's
loaded into memory?
Ciao,
Martin v. Löwis wrote:
> John Machin wrote:
> > Incredible. That's only 34 MB. What is the size of your paging file?
> > What memory guzzlers were you running at the same time? What was the
> > Task Manager "Performance" pane showing while your test was running?
> > What version of Python?
>
> He
Yi Xing wrote:
> Thanks! I just found that that I have no problem with
> x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work.
>
range(1*2560*2560*30) is creating a list of 196M *unique* ints.
Assuming 32-bit ints and pointers: that's 4 bytes each for the value, 4
for the type pointe
Yi Xing wrote:
> Thanks! I just found that that I have no problem with
> x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work.
That's no surprise. In the first case, try
x[0][0] = 20.0
print x[1][0]
You have the very same (identical) list of 2560*2560 values in x
500 times.
To cre
John Machin wrote:
> Incredible. That's only 34 MB. What is the size of your paging file?
> What memory guzzlers were you running at the same time? What was the
> Task Manager "Performance" pane showing while your test was running?
> What version of Python?
He didn't say Windows (so far). AFAICT,
Yi Xing wrote:
> On a related question: how do I initialize a list or an array with a
> pre-specified number of elements, something like
> int p[100] in C? I can do append() for 100 times but this looks silly...
>
> Thanks.
>
> Yi Xing
In the case of an array, you may wish to consider the fromfil
Is there a way that I can define a two-dimensional array in
array.array()? Thanks.
On Aug 14, 2006, at 2:28 PM, John Machin wrote:
> Yi Xing wrote:
>> I tried the following code:
>>
> i=0
> n=2600*2600*30
> a=array.array("f")
> while (i<=n):
>> .. i=i+1
>> .. a.append(floa
Thanks! I just found that that I have no problem with
x=[[10.0]*2560*2560]*500, but x=range(1*2560*2560*30) doesn't work.
-Yi
On Aug 14, 2006, at 3:08 PM, Larry Bates wrote:
> Yi Xing wrote:
>> On a related question: how do I initialize a list or an array with a
>> pre-specified number of elemen
Yi Xing wrote:
> On a related question: how do I initialize a list or an array with a
> pre-specified number of elements, something like
> int p[100] in C? I can do append() for 100 times but this looks silly...
>
> Thanks.
>
> Yi Xing
>
Unlike other languages this is seldom done in Python. I t
Yi Xing wrote:
> I tried the following code:
>
> >>> i=0
> >>> n=2600*2600*30
> >>> a=array.array("f")
> >>> while (i<=n):
> .. i=i+1
> .. a.append(float(i))
Not a good idea. The array has to be resized, which may mean that a
realloc won't work because of fragmentation, you're out of luck
Yi Xing wrote:
> On a related question: how do I initialize a list or an array with a
> pre-specified number of elements, something like
> int p[100] in C? I can do append() for 100 times but this looks silly...
>
> Thanks.
>
> Yi Xing
Use [0]*100 for a list.
THN
--
http://mail.python.org/ma
Yi Xing wrote:
> On a related question: how do I initialize a list or an array with a
> pre-specified number of elements, something like
> int p[100] in C? I can do append() for 100 times but this looks silly...
>
> Thanks.
>
> Yi Xing
You seldom need to do that in python, but it's easy enough:
On a related question: how do I initialize a list or an array with a
pre-specified number of elements, something like
int p[100] in C? I can do append() for 100 times but this looks silly...
Thanks.
Yi Xing
--
http://mail.python.org/mailman/listinfo/python-list
Yi Xing wrote:
> Hi,
>
> I need to read a large amount of data into a list. So I am trying to see
> if I'll have any memory problem. When I do
> x=range(2700*2700*3) I got the following message:
>
> Traceback (most recent call last):
> File "", line 1, in ?
> MemoryError
>
> Any way to get a
> I need to read a large amount of data into a list. So I am trying to
> see if I'll have any memory problem. When I do
> x=range(2700*2700*3) I got the following message:
>
> Traceback (most recent call last):
> File "", line 1, in ?
> MemoryError
>
> Any way to get around this problem? I
[EMAIL PROTECTED] wrote:
> If you know that you need floats only, then you can use a typed array
> (an array.array) instead of an untyped array (a Python list):
>
> import array
> a = array.array("f")
>
Clarification: typecode 'f' stores a Python float (64-bits, equivalent
to a C double) as a 32
Yi Xing wrote:
> Hi,
>
> I need to read a large amount of data into a list. So I am trying to
> see if I'll have any memory problem. When I do
> x=range(2700*2700*3) I got the following message:
>
> Traceback (most recent call last):
> File "", line 1, in ?
> MemoryError
>
> Any way to get ar
Yi Xing wrote:
> I need to read a large amount of data into a list. So I am trying to
> see if I'll have any memory problem. When I do
> x=range(2700*2700*3) I got the following message:
> Traceback (most recent call last):
> File "", line 1, in ?
> MemoryError
> Any way to get around this pr
39 matches
Mail list logo
