Hi, Gabriel,
> -----Original Message-----
> From: Gabriel Genellina [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, January 09, 2007 4:06 PM
> To: python-list@python.org
> Subject: RE: File Closing Problem in 2.3 and 2.4, Not in 2.5 (Final
> report)
>
> At Tuesday 9/1/2007 20:31, Carroll, Barry wrote:
>
> >I've spent about a day investigating our "too many open files" error. I
> >found the following:
> >
> > 1. Windows XP allows a Python 2.5 script to open 509 concurrent
> > files.
>
> And do you actually need so many open files simultaneously?
No. In fact, that was the original point of my post. We expected that the
maximum number of file objects (open and otherwise) would never be more than
five or six. We believed (apparently incorrectly) that destroying the table
parser object would destroy its associated file object and close the referenced
file. Here is the code that does this:
##########
def unloadtable(self, log=True):
"""Unload current table and pop prior processor from call stack.
"""
# SNIP
# force clean up for unloaded table
self.processor = None
gc.collect()
# pop prior processor from stack
self.processor = self.pstack.pop()
# SNIP
##########
The first assumption was correct: the file object is destroyed. The second
assumption is apparently incorrect: the file handle is not released when the
associated object is destroyed. These 'orphaned' file handles build up and
when the OS handle limit is reached, "too many open files" errors show up.
> Try to close them explicitly when you finish working on them - do
> not rely on GC for closing files. This has *always* been the
> recomended practice (except maybe, inside a short script that
> finishes quickly).
This is what we have done. The unloadtable function now looks like this:
##########
def unloadtable(self, log=True):
"""Unload current table and pop prior processor from call stack.
"""
# SNIP
# force clean up for unloaded table
self.processor.source.close()
self.processor = None
gc.collect()
# pop prior processor from stack
self.processor = self.pstack.pop()
# SNIP
##########
> On Python 2.5 you can use the new `with` statement.
>
Not only that, the file errors don't appear when we run our program/table
combination under 2.5. I can't explain the difference, unless the work done to
implement the 'with' statement has an impact outside the scope of that feature.
In any event, it does us no good. Other constraints on the test system
require us to stick with Python 2.3, at least for the present.
>
> --
> Gabriel Genellina
> Softlab SRL
>
> __________________________________________________
> Preguntá. Respondé. Descubrí.
> Todo lo que querías saber, y lo que ni imaginabas,
> está en Yahoo! Respuestas (Beta).
> ¡Probalo ya!
> http://www.yahoo.com.ar/respuestas
>
Thanks for your interest.
Barry
[EMAIL PROTECTED]
541-302-1107
________________________
We who cut mere stones must always be envisioning cathedrals.
-Quarry worker's creed
--
http://mail.python.org/mailman/listinfo/python-list
