Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > You misunderstood. The 300 secs is not in waiting for unlink() to > return, it is in waiting for its effects to be seen by rmdir() (i.e. for > the entry to actually be cleared from the directory). unlink() is > returning very quickly. If the bgwriter

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> One relatively low-impact workaround would be to force a checkpoint >> (on Windows only) during DROP DATABASE, just before we actually fire >> the rmtree() operation. The bgwriter is already coded to close all its >> open files after

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Andrew Dunstan
Zeugswetter Andreas DAZ SD wrote: You misunderstood. The 300 secs is not in waiting for unlink() to return, it is in waiting for its effects to be seen by rmdir() (i.e. for because the bgwriter obviously keeps them open, no ? Yes. cheers andrew ---(end of broadca

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Before I tried anything like that I tried one more thing. I disabled the background writer and the problem stopped. So now we know the "culprit". Okay. So what that says is that win32_open's claim to allow unlinking an open file

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Before I tried anything like that I tried one more thing. I disabled the > background writer and the problem stopped. So now we know the "culprit". Okay. So what that says is that win32_open's claim to allow unlinking an open file is a lie; or at leas

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Zeugswetter Andreas DAZ SD
> >>> 300 secs (!) fs timeout is really broken. > >>> Looks more like a locking or network timeout issue. > >>> What error codes does unlink(3) return? > >> > > > >> success. > > > > > > Oops! 5min timeout for success is certainly problematic. > > > > > > You misunderstood. The 300 secs is not in

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Bruce Momjian
Andrew Dunstan wrote: > > > Reini Urban wrote: > > >>> > >>> 300 secs (!) fs timeout is really broken. > >>> Looks more like a locking or network timeout issue. > >>> What error codes does unlink(3) return? > >> > > > >> success. > > > > > > Oops! 5min timeout for success is certainly problemati

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Andrew Dunstan
Reini Urban wrote: 300 secs (!) fs timeout is really broken. Looks more like a locking or network timeout issue. What error codes does unlink(3) return? success. Oops! 5min timeout for success is certainly problematic. You misunderstood. The 300 secs is not in waiting for unlink() to return,

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Reini Urban
Andrew Dunstan schrieb: problem area found. see below. Reini Urban wrote: Andrew Dunstan schrieb: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've

Re: [HACKERS] rmtree() failure on Windows

2004-10-27 Thread Andrew Dunstan
problem area found. see below. Reini Urban wrote: Andrew Dunstan schrieb: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several time

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Reini Urban
Andrew Dunstan schrieb: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks like we are the victim of some cachin

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It look

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Here is some more info. Below is a trace from dropdb. There is a loop > around the rmdir() calls which I have set to time out at 600 seconds. > The call eventually succeeds after around 300 seconds (I've seen this > several times). It looks like we ar

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Andrew Dunstan
Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks like we are the victim of some caching - the directory stil

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: If I increase the sleep time before dropdb enormously (35 secs) the unlink errors seem to go away, and instead they become rmdir errors like the rest. Do you have anything equivalent to ps that you could use to check whether ther

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > If I increase the sleep time before dropdb enormously (35 secs) the > unlink errors seem to go away, and instead they become rmdir errors like > the rest. Do you have anything equivalent to ps that you could use to check whether there is still an old

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Another data point - when rmdir() fails it fails quickly, but when unlink (i.e. our pg_unlink()) fails it takes a very long time (minutes) to fail. And the file is actually not there. So it looks like we loop over and over and keep g

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Another data point - when rmdir() fails it fails quickly, but when > unlink (i.e. our pg_unlink()) fails it takes a very long time (minutes) > to fail. And the file is actually not there. So it looks like we loop > over and over and keep getting EACCE

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Reini Urban
Tom Lane schrieb: Andrew Dunstan <[EMAIL PROTECTED]> writes: Shown below is an extract from the traces of make installcheck in contrib. It is decorated with some extra traces I built into src/port/dirmod.c::rmtree(). It shows quite reproducible failure of rmtree(), mostly at the rmdir calls, but

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I kinda suspect that what you are looking at is a problem with the >> delayed-unlinking feature that we built to cope with Windows' inability >> to unlink open files, ie, it's being a little too slow to do the >> unlinks. Would you re

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Shown below is an extract from the traces of make installcheck in contrib. It is decorated with some extra traces I built into src/port/dirmod.c::rmtree(). It shows quite reproducible failure of rmtree(), mostly at the rmdir calls, b

Re: [HACKERS] rmtree() failure on Windows

2004-10-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Shown below is an extract from the traces of make installcheck in > contrib. It is decorated with some extra traces I built into > src/port/dirmod.c::rmtree(). It shows quite reproducible failure of > rmtree(), mostly at the rmdir calls, but even more

[HACKERS] rmtree() failure on Windows

2004-10-25 Thread Andrew Dunstan
Houston, we have a problem. Shown below is an extract from the traces of make installcheck in contrib. It is decorated with some extra traces I built into src/port/dirmod.c::rmtree(). It shows quite reproducible failure of rmtree(), mostly at the rmdir calls, but even more worryingly there are