Bugs item #1425127, was opened at 2006-02-06 05:44
Message generated for change (Comment added) made by tim_one
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1425127&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: cheops (atila-cheops)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.remove OSError: [Errno 13] Permission denied
Initial Comment:
When running the following program I get frequent
errors like this one
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Python24\lib\threading.py", line 442, in
__bootstrap
self.run()
File "os.remove.py", line 25, in run
os.remove(filename)
OSError: [Errno 13] Permission denied:
'c:\\docume~1\\joag\\locals~1\\temp\\tmpx91tkx'
When leaving out the touch statement(line 24) in the
loop of the class, I do not get any errors.
This is on Windows XP SP2 with python-2.4.2 (you should
have an exe touch somewhere in you path for this to
work) Can somebody shed any light on this please?
Thanks in advance
Joram Agten
----------------------------------------------------------------------
>Comment By: Tim Peters (tim_one)
Date: 2006-02-06 23:07
Message:
Logged In: YES
user_id=31435
[/F]
> Except that he's hitting the file system quite heavily,
Except that _without_ the call to touch(), he's hitting it
even more heavily, creating and destroying little files just
as fast as the OS can do it in each of 10 threads -- but
there aren't any errors then.
> and asyncronously.
What's asynch here? The OP's touch() function waits for the
spawned process to terminate, and the test driver doesn't
try to delete the file until after that.
> My guess is that Windows simply gets behind
> (a quick filemon test indicates that this is indeed the
> case; just before a crash, I see the events
> CREATE/SUCCESS, QUERY/SUCCESS, QUERY/SUCCESS,
> WRITE/SUCCESS, and OPEN/SHARING VIOLATION for the
> failing file, with lots of requests for other files
> interleaved).
That's consistent with the symptom reported: an exception
raised upon trying to remove the file, but not during any
other file operation. Does it tell you more than _just_
that? It doesn't for me.
> Unless someone wants to fix Windows,
As above, because removing the call to the internal `touch`
function makes all problems go away it's not obvious that
this is a Windows problem.
> a simple workaround would be to retry the os.remove a
> few times before giving up (with a time.sleep(0.1) in
> between).
Because of the internal threading errors in subprocess.py
(see my first comment), the threads in the test program
still usually die, but with instances of list.remove(x)
ValueErrors internal to subprocess.py.
If I hack around that, then this change to the test
program's file-removal code appears adequate to eliminate
all errors on my box (which is a zippy 3.4 GHz):
try:
os.remove(filename)
except OSError:
time.sleep(0.1)
os.remove(filename)
It's possible that some virus-scanning or file-indexing
gimmick on my box is opening these little files for its own
purposes -- although, if so, I'm at a loss to account for
why a single "os.remove(filename)" never raises an exception
when the `touch()` call is commented out.
OTOH, with the `touch()` call intact, the time.sleep(0.1)
above is not adequate to prevent os.remove() errors if I
change the file-writing code to:
f.write("test" * 250000)
Even boosting the sleep() to 0.4 isn't enough then.
That does (mildly) suggest there's another process opening
the temp files, and doing something with them that takes
time proportional to the file size. However, the
os.remove() errors persist when I disable all such gimmicks
(that I know about ;-)) on my box.
It seems likely I'll never determine a cause for that. The
bad thread behavior in subprocess.py is independent, and
should be repaired regardless.
----------------------------------------------------------------------
Comment By: Fredrik Lundh (effbot)
Date: 2006-02-06 18:05
Message:
Logged In: YES
user_id=38376
> The problem is that there's no reason to believe anything
> he did here _does_ leave files open.
Except that he's hitting the file system quite heavily, and
asyncronously. My guess is that Windows simply gets behind
(a quick filemon test indicates that this is indeed the
case; just before a crash, I see the events CREATE/SUCCESS,
QUERY/SUCCESS, QUERY/SUCCESS, WRITE/SUCCESS, and
OPEN/SHARING VIOLATION for the failing file, with lots of
requests for other files interleaved).
Unless someone wants to fix Windows, a simple workaround
would be to retry the os.remove a few times before giving up
(with a time.sleep(0.1) in between).
----------------------------------------------------------------------
Comment By: cheops (atila-cheops)
Date: 2006-02-06 17:53
Message:
Logged In: YES
user_id=1276121
I did post on the python mailing list first, but got no
responce there, after further looking into it, I seriously
think there is at least one bug here.
here is the link to the post:
http://mail.python.org/pipermail/python-list/2006-
February/323650.html
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2006-02-06 17:19
Message:
Logged In: YES
user_id=31435
The problem is that there's no reason to believe anything he
did here _does_ leave files open. I can confirm the
"permission denied" symptom, and even if I arrange for the
call to "touch" to run a touch.bat that doesn't even look at
the filename passed to it (let alone open or modify the file).
I also see a large number of errors of this sort:
Exception in thread Thread-8:
Traceback (most recent call last):
File "C:\python24\lib\threading.py", line 442, in __bootstrap
self.run()
File "osremove.py", line 21, in run
touch(filename)
File "osremove.py", line 8, in touch
stdin=None, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
File "C:\python24\lib\subprocess.py", line 490, in __init__
_cleanup()
File "C:\python24\lib\subprocess.py", line 398, in _cleanup
inst.poll()
File "C:\python24\lib\subprocess.py", line 739, in poll
_active.remove(self)
ValueError: list.remove(x): x not in list
Those are clearly due to subprocess.py internals on Windows,
where the poll() and wait() methods and the module internal
_cleanup() function aren't called in mutually threadsafe
ways. _Those_ errors can be stopped by commenting out the
_cleanup() call at the start of Popen.__init__() (think
about what happens when multiple threads call _cleanup() at
overlapping times on Windows: all those threads can end up
trying to remove the same items from _active, but only one
thread per item can succeed).
The "permission denied" errors persist, though.
So there's at least one class of subprocess.py Windows bugs
here, and another class of Windows mysteries. I believe
subprocess.py is a red herring wrt the latter, though. For
example, I see much the same if I use os.system() to run
`touch` instead.
----------------------------------------------------------------------
Comment By: Fredrik Lundh (effbot)
Date: 2006-02-06 16:50
Message:
Logged In: YES
user_id=38376
If Python gives you a permission error, that's because
Windows cannot remove the file. Windows does, in general,
not allow you to remove files that are held open by some
process.
I suggest taking this issue to comp.lang.python. The bug
tracker is not the right place for code review and other
support issues.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1425127&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
