Python 2.4.3 array memory leak
I am using the array module to instantiate many arrays in my
application. It seems like there is a memory leak in there somewhere.
Can anyone confim this and let me know what, if anything, I can do
about it? I am using Fedora Core 5 Linux:
import commands
import array
import itertools
import sys
from itertools import repeat
print '*** before ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
for i in range(100):
a = array.array('I', repeat(0, int(2E6)))
del a
print '*** after ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
Output:
*** before ***
MemFree: 1459772 kB
*** after ***
MemFree: 1457688 kB
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4.3 array memory leak
On May 7, 3:31 pm, "danmcle...@yahoo.com"
wrote:
> I am using the array module to instantiate many arrays in my
> application. It seems like there is a memory leak in there somewhere.
> Can anyone confim this and let me know what, if anything, I can do
> about it? I am using Fedora Core 5 Linux:
>
> import commands
> import array
> import itertools
> import sys
>
> from itertools import repeat
>
> print '*** before ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> for i in range(100):
> a = array.array('I', repeat(0, int(2E6)))
> del a
> print '*** after ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> Output:
> *** before ***
> MemFree: 1459772 kB
> *** after ***
> MemFree: 1457688 kB
I hate to reply to my own thread but I wanted to update it. I tried
the same code using ctypes arrays and see no memory leak:
import commands
import array
import itertools
import sys
import ctypes
from itertools import repeat
from ctypes import *
print '*** before ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
for i in range(100):
a = ARRAY(c_uint, int(2E6))
del a
print '*** after ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
*** before ***
MemFree: 1416364 kB
*** after ***
MemFree: 1416364 kB
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4.3 array memory leak
On May 7, 3:47 pm, "danmcle...@yahoo.com"
wrote:
> On May 7, 3:31 pm, "danmcle...@yahoo.com"
> wrote:
>
>
>
> > I am using the array module to instantiate many arrays in my
> > application. It seems like there is a memory leak in there somewhere.
> > Can anyone confim this and let me know what, if anything, I can do
> > about it? I am using Fedora Core 5 Linux:
>
> > import commands
> > import array
> > import itertools
> > import sys
>
> > from itertools import repeat
>
> > print '*** before ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> > for i in range(100):
> > a = array.array('I', repeat(0, int(2E6)))
> > del a
> > print '*** after ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> > Output:
> > *** before ***
> > MemFree: 1459772 kB
> > *** after ***
> > MemFree: 1457688 kB
>
> I hate to reply to my own thread but I wanted to update it. I tried
> the same code using ctypes arrays and see no memory leak:
>
> import commands
> import array
> import itertools
> import sys
> import ctypes
>
> from itertools import repeat
> from ctypes import *
>
> print '*** before ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> for i in range(100):
> a = ARRAY(c_uint, int(2E6))
> del a
> print '*** after ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> *** before ***
> MemFree: 1416364 kB
> *** after ***
> MemFree: 1416364 kB
The above code was not correct. I actually do see a memory leak when
using ctypes arrays also. What is going on?
import commands
import array
import itertools
import sys
import ctypes
from itertools import repeat
from ctypes import *
print '*** before ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
for i in range(100):
a = ARRAY(c_uint, int(2E6))()
del a
print '*** after ***'
print commands.getoutput('cat /proc/meminfo').split('\n')[1]
Output
*** before ***
MemFree: 1564096 kB
*** after ***
MemFree: 1556884 kB
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4.3 array memory leak
On May 7, 3:58 pm, "danmcle...@yahoo.com"
wrote:
> On May 7, 3:47 pm, "danmcle...@yahoo.com"
> wrote:
>
>
>
> > On May 7, 3:31 pm, "danmcle...@yahoo.com"
> > wrote:
>
> > > I am using the array module to instantiate many arrays in my
> > > application. It seems like there is a memory leak in there somewhere.
> > > Can anyone confim this and let me know what, if anything, I can do
> > > about it? I am using Fedora Core 5 Linux:
>
> > > import commands
> > > import array
> > > import itertools
> > > import sys
>
> > > from itertools import repeat
>
> > > print '*** before ***'
> > > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> > > for i in range(100):
> > > a = array.array('I', repeat(0, int(2E6)))
> > > del a
> > > print '*** after ***'
> > > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> > > Output:
> > > *** before ***
> > > MemFree: 1459772 kB
> > > *** after ***
> > > MemFree: 1457688 kB
>
> > I hate to reply to my own thread but I wanted to update it. I tried
> > the same code using ctypes arrays and see no memory leak:
>
> > import commands
> > import array
> > import itertools
> > import sys
> > import ctypes
>
> > from itertools import repeat
> > from ctypes import *
>
> > print '*** before ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> > for i in range(100):
> > a = ARRAY(c_uint, int(2E6))
> > del a
> > print '*** after ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> > *** before ***
> > MemFree: 1416364 kB
> > *** after ***
> > MemFree: 1416364 kB
>
> The above code was not correct. I actually do see a memory leak when
> using ctypes arrays also. What is going on?
>
> import commands
> import array
> import itertools
> import sys
> import ctypes
>
> from itertools import repeat
> from ctypes import *
>
> print '*** before ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> for i in range(100):
> a = ARRAY(c_uint, int(2E6))()
> del a
> print '*** after ***'
> print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> Output
> *** before ***
> MemFree: 1564096 kB
> *** after ***
> MemFree: 1556884 kB
Maybe the problem is with the way I was looking at free memory. I
changed my code to look at virtual memory allocated to the process in
question and I do not see any appreciable before/after difference.
import commands
import array
import itertools
import sys
import os
from itertools import repeat
deltas = []
print '*** before ***'
s = commands.getoutput('cat /proc/%d/status' % os.getpid()).split('\n')
[12]
print s
for i in range(100):
a = array.array('I', repeat(0, int(2E6)))
del a
print '*** after ***'
s = commands.getoutput('cat /proc/%d/status' % os.getpid()).split('\n')
[12]
print s
Output:
*** before ***
VmSize: 5952 kB
*** after ***
VmSize: 5956 kB
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.4.3 array memory leak
On May 7, 4:41 pm, Terry Reedy wrote:
> danmcle...@yahoo.com wrote:
> > I am using the array module to instantiate many arrays in my
> > application. It seems like there is a memory leak in there somewhere.
> > Can anyone confim this and let me know what, if anything, I can do
> > about it? I am using Fedora Core 5 Linux:
>
> > import commands
> > import array
> > import itertools
> > import sys
>
> > from itertools import repeat
>
> > print '*** before ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
> > for i in range(100):
> > a = array.array('I', repeat(0, int(2E6)))
> > del a
> > print '*** after ***'
> > print commands.getoutput('cat /proc/meminfo').split('\n')[1]
>
> > Output:
> > *** before ***
> > MemFree: 1459772 kB
> > *** after ***
> > MemFree: 1457688 kB
>
> What happens if you remove the loop? I would not be surprised if Python
> grabs the memory once, reuses it, and does not let go. That is not a
> leak. What happens if you put the after inside the loop? Does mem
> usage steadily increase, and continue if you increase range to 1000,
> 1? That would be a leak.
>
> If there actually is a problem, try a later version of Python.
I'm not convinced there is a problem anymore. In my latest code, I
record the virtual mem allocated to my specific process and I do not
see it increasing. I would think that if I was leaking memory, I
should see a steady increase in virtual memory consumed by my process,
which I do not.
thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to execute a script from another script and other script does not do busy wait.
On Jan 7, 9:18 am, Jorgen Grahn wrote: > On Thu, 2010-01-07, Rajat wrote: > > I want to run a python script( aka script2) from another python script > > (aka script1). While script1 executes script2 it waits for script2 to > > complete and in doing so it also does some other useful work.(does not > > do a busy wait). > > > My intention is to update a third party through script1 that script2 > > is going to take longer. > > I do not understand that sentence. > What are you trying to do, more exactly? The best solution can be > threads, os.popen, os.system or something different -- depending on > the details of what you want to do. > > > Please suggest how should I go about implementing it. > > > I'm currently executing it as: > > > import main from script2 > > ret_code = main() > > return ret_code > > > which surely is not going to achieve me what I intend. > > > Thanks, > > Rajat. > > /Jorgen > > -- > // Jorgen Grahn \X/ snipabacken.se> O o . I personally use subprocess. Once you launch via subprocess you can wait or not. p = subprocess.Popen(...) p.wait() #or not. See subprocess docs. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to run python without python
On Apr 1, 5:54 pm, Chris Rebert wrote: > On Thu, Apr 1, 2010 at 4:46 PM, Krister Svanlund > > wrote: > > On Fri, Apr 2, 2010 at 1:36 AM, Spencer wrote: > >> Is there a way to developing a script on linux and give it > >> to someone on microsoft, so that they could run it on microsoft > >> without installing python? > > > Short answer: No. > > Long answer: > No indeed. But if he were to have a Windows computer, he could > generate a standalone executable from a Python program using one of > the following tools: > py2exe:http://www.py2exe.org/ > PyInstaller:http://www.pyinstaller.org/ > > But one can't generate such a standalone executable for a different > operating system from that which one's computer runs. > > Cheers, > Chris > --http://blog.rebertia.com that's not entirely true. i just built a standalone exe for win 7 from my win xp machine. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to run python without python
On Apr 2, 11:09 am, "danmcle...@yahoo.com" wrote: > On Apr 1, 5:54 pm, Chris Rebert wrote: > > > > > On Thu, Apr 1, 2010 at 4:46 PM, Krister Svanlund > > > wrote: > > > On Fri, Apr 2, 2010 at 1:36 AM, Spencer wrote: > > >> Is there a way to developing a script on linux and give it > > >> to someone on microsoft, so that they could run it on microsoft > > >> without installing python? > > > > Short answer: No. > > > Long answer: > > No indeed. But if he were to have a Windows computer, he could > > generate a standalone executable from a Python program using one of > > the following tools: > > py2exe:http://www.py2exe.org/ > > PyInstaller:http://www.pyinstaller.org/ > > > But one can't generate such a standalone executable for a different > > operating system from that which one's computer runs. > > > Cheers, > > Chris > > --http://blog.rebertia.com > > that's not entirely true. i just built a standalone exe for win 7 from > my win xp machine. using py2exe. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to run python without python
On Apr 2, 11:23 am, Chris Rebert wrote: > On Fri, Apr 2, 2010 at 10:09 AM, danmcle...@yahoo.com > > wrote: > > On Apr 1, 5:54 pm, Chris Rebert wrote: > >> On Thu, Apr 1, 2010 at 4:46 PM, Krister Svanlund > >> wrote: > >> > On Fri, Apr 2, 2010 at 1:36 AM, Spencer > >> > wrote: > >> >> Is there a way to developing a script on linux and give it > >> >> to someone on microsoft, so that they could run it on microsoft > >> >> without installing python? > > >> one can't generate such a standalone executable for a different > >> operating system from that which one's computer runs. > > > that's not entirely true. i just built a standalone exe for win 7 from > > my win xp machine. > > s/operating system/platform > > Good luck getting PyInstaller to output for Windows when being run on > a *nix box. > > Cheers, > Chris > --http://blog.rebertia.com you think virtualbox could help? i wonder if one could run linux/ py2exe virtually on a win machine and get it working. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting Local MAC Address
On Apr 2, 2:14 pm, Booter wrote:
> Hello all,
>
> I am new to python ans was wondering if there was a way to get the mac
> address from the local NIC?
>
> Thanks for your help.
>
> Gerad
for windows parse p.stdout.read():
import subprocess
p = subprocess.Popen('ipconfig', shell = True, stdout =
subprocess.PIPE)
p.wait()
print p.stdout.read()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Getting Local MAC Address
On Apr 2, 2:52 pm, "danmcle...@yahoo.com"
wrote:
> On Apr 2, 2:14 pm, Booter wrote:
>
> > Hello all,
>
> > I am new to python ans was wondering if there was a way to get the mac
> > address from the local NIC?
>
> > Thanks for your help.
>
> > Gerad
>
> for windows parse p.stdout.read():
>
> import subprocess
>
> p = subprocess.Popen('ipconfig', shell = True, stdout =
> subprocess.PIPE)
>
> p.wait()
>
> print p.stdout.read()
sorry, posted too soon. looks like this is for ip address only.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Getting Local MAC Address
On Apr 2, 2:52 pm, "danmcle...@yahoo.com"
wrote:
> On Apr 2, 2:14 pm, Booter wrote:
>
> > Hello all,
>
> > I am new to python ans was wondering if there was a way to get the mac
> > address from the local NIC?
>
> > Thanks for your help.
>
> > Gerad
>
> for windows parse p.stdout.read():
>
> import subprocess
>
> p = subprocess.Popen('ipconfig', shell = True, stdout =
> subprocess.PIPE)
>
> p.wait()
>
> print p.stdout.read()
try this instead:
import subprocess
p = subprocess.Popen('ipconfig /all', shell = True, stdout =
subprocess.PIPE)
p.wait()
print p.stdout.read()
--
http://mail.python.org/mailman/listinfo/python-list
Re: What is the best way to handle a missing newline in the following case
> The problem is when I get to the last line. When the program sees '\n' > after the 9, everything works fine. However, when there isn't a '\n', > the program doesn't process the last line. > > What would be the best approach to handle the case of the possible > missing '\n' at the end of the file? use readines to read all lines into a list and then iterate thru the list: f = open(r'c:\test.txt', 'rb') print f.readlines() >> ['1\r\n', '3\r\n', '5\r\n', '7\r\n', '3\r\n', '9'] -- http://mail.python.org/mailman/listinfo/python-list
Re: A matter of queues, tasks and multiprocessing
If you are using Python 2.6 or greater, look into the multiprocessing module. It may contain 90% of what you need. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.tmpfile() vs. tempfile.TemporaryFile()
On Nov 11, 11:32 am, John Nagle wrote: > Is there any reason to prefer "tempfile.TemporaryFile()" > over "os.tmpfile()"? Both create a nameless temporary file > that will be deleted on close. > > John Nagle tempfile.TemporaryFile has more options, e.g. file prefix, suffix, dir location, mode, and buf size. if you don't care about any of that, then it probably makes no difference. -- http://mail.python.org/mailman/listinfo/python-list
