New submission from philipspencer <[EMAIL PROTECTED]>:
Python's os.listdir function has the following code in
Modules/posixmodule.c:
errno = 0
...
for (;;) {
Py_BEGIN_ALLOW_THREADS
ep = readdir(dirp);
Py_END_ALLOW_THREADS
if (ep == NULL)
break;
...
a bunch of other stuff, including PyString_FromStringAndSize
which calls malloc
...
}
if (errno != 0 && d != NULL) {
The assumption is that errno will be nonzero only if readdir failed.
However, this is not the case. GLibc's malloc will, in some rare cases,
set errno even when it succeeds. So, during one pass through the loop
errno gets set to ENOMEM and then it is still set to ENOMEM when the
final readdir returns null at the end of the directory listing.
The fix is to move the line "errno = 0" from outside the loop to
right before the readdir inside the loop. That is the only way to
ensure that, when the loop is exited with readdir returning null,
the condition "errno != 0" is equivalent to "readdir actually failed."
The attached patch does this. Without this patch, we experience
frequent failures with the python tools "creatrepo" and "repomanage"
when run on very large directories (> 5000 packages) on Fedora 8
systems; see RedHat bugzilla entry #451494. With the patch, the commands
work as expected.
The patch is against 2.5.1 but from what I can see of the posixmodule.c
code from SVN it should apply cleanly there too.
----------
components: Extension Modules
messages: 68223
nosy: philipspencer
severity: normal
status: open
title: os.listdir randomly fails on occasions when it shouldn't
type: behavior
versions: Python 2.5
_______________________________________
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3115>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
