Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue jun 09 17:10:10 -0400 2011: > Alvaro Herrera writes: > > Excerpts from Alvaro Herrera's message of jue jun 09 16:34:13 -0400 2011: > >> I have pushed it now. > > > ... and it caused a failure on the buildfarm, so I panicked and reverted > > it. I think the

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Alvaro Herrera's message of jue jun 09 16:34:13 -0400 2011: >> I have pushed it now. > ... and it caused a failure on the buildfarm, so I panicked and reverted > it. I think the patch below fixes it. Actually, I think what you want is what closeAllVfds doe

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of jue jun 09 16:34:13 -0400 2011: > I have pushed it now. ... and it caused a failure on the buildfarm, so I panicked and reverted it. I think the patch below fixes it. Let me know if you think I should push the whole thing again. *** a/src/backend/stora

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Tom Lane
Alvaro Herrera writes: > That was pretty weird. I had rm'd the build tree and rebuilt, failure > still there. I then did a git reset FETCH_HEAD, recompiled, and the > problem was gone. git reset to my revision, failed. Then git clean > -dfx, nuked the build tree, redid the whole thing from scr

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of jue jun 09 16:02:14 -0400 2011: > Excerpts from Tom Lane's message of jue jun 09 14:45:31 -0400 2011: > > My thought is that it needs some beta testing. Perhaps it'd be sane to > > push it into beta2 now, and then back-patch sometime after 9.1 final, > >

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue jun 09 14:45:31 -0400 2011: > Alvaro Herrera writes: > > Excerpts from Tom Lane's message of mié jun 08 14:28:02 -0400 2011: > >> Alvaro Herrera writes: > >>> This customer is running on 8.4 so I started from there; should I > >>> backpatch this to 8.2, or

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Robert Haas
On Thu, Jun 9, 2011 at 3:02 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jun 9, 2011 at 2:45 PM, Tom Lane wrote: >>> My thought is that it needs some beta testing.  Perhaps it'd be sane to >>> push it into beta2 now, and then back-patch sometime after 9.1 final, >>> if no problems pop up

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 9, 2011 at 2:45 PM, Tom Lane wrote: >> My thought is that it needs some beta testing.  Perhaps it'd be sane to >> push it into beta2 now, and then back-patch sometime after 9.1 final, >> if no problems pop up. > I think it'd be sensible to back-patch it. I'm no

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Robert Haas
On Thu, Jun 9, 2011 at 2:45 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Tom Lane's message of mié jun 08 14:28:02 -0400 2011: >>> Alvaro Herrera writes: This customer is running on 8.4 so I started from there; should I backpatch this to 8.2, or not at all? > >>> I'm n

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Tom Lane's message of mié jun 08 14:28:02 -0400 2011: >> Alvaro Herrera writes: >>> This customer is running on 8.4 so I started from there; should I >>> backpatch this to 8.2, or not at all? >> I'm not excited about back-patching it... > Bummer. Well, o

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-09 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié jun 08 14:28:02 -0400 2011: > Alvaro Herrera writes: > > Okay, here's a patch implementing this idea. It seems to work quite > > well, and it solves the problem in a limited testing scenario -- I > > haven't yet tested on the customer machines. > > This se

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-08 Thread Tom Lane
Alvaro Herrera writes: > Okay, here's a patch implementing this idea. It seems to work quite > well, and it solves the problem in a limited testing scenario -- I > haven't yet tested on the customer machines. This seems mostly sane, except I think you have not considered the issue of when to cle

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-08 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar jun 07 12:26:34 -0400 2011: > It's not *that* many levels: in fact, I think md.c is the only level > that would just have to pass it through without doing anything useful. > I think that working from there is a saner and more efficient approach > than what y

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-07 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Tom Lane's message of lun jun 06 12:49:46 -0400 2011: >> Hmm, there's already a mechanism for closing "temp" FDs at the end of a >> query ... maybe blind writes could use temp-like FDs? > I don't think it can be made to work exactly like that. If I understa

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jun 06 12:49:46 -0400 2011: > Robert Haas writes: > > Instead of closing them immediately, how about flagging the FD and > > closing all the flagged FDs at the end of each query, or something > > like that? > > Hmm, there's already a mechanism for closing

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jun 06 12:49:46 -0400 2011: > Robert Haas writes: > > On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane wrote: > >> On reflection I think this behavior is probably limited to the case > >> where we've done what we used to call a "blind write" of a block that > >> i

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane wrote: >> On reflection I think this behavior is probably limited to the case >> where we've done what we used to call a "blind write" of a block that >> is unrelated to our database or tables.  For normal SQL-driven accesses, >> the

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: >>> Yeah, I wasn't that thrilled with the suggestion either.  But we can't >>> just have backends constantly closing every open FD they hold, or >>> per

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: >> Yeah, I wasn't that thrilled with the suggestion either. But we can't >> just have backends constantly closing every open FD they hold, or >> performance will suffer. I don't see any very good place t

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Kevin Grittner
Alvaro Herrera wrote: > That doesn't solve the WAL problem Kevin found, of course ... I wouldn't worry about that too much -- the WAL issue is self-limiting and not likely to ever cause a failure. The biggest risk is that every now and then some new individual will notice it and waste a bit o

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: > Alvaro Herrera writes: > > Hmm interesting. I don't think the placement suggested by Tom would be > > useful, because the Zabbix backends are particularly busy all the time, > > so they wouldn't run ProcessCatchupEvent at all.

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Kevin Grittner's message of lun jun 06 11:58:51 -0400 2011: >> Alvaro Herrera wrote: >>> What we found out after more careful investigation is that the >>> file is kept open by a backend connected to a different database. >>> I have a suspicion that what ha

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of lun jun 06 11:58:51 -0400 2011: > Alvaro Herrera wrote: > > > What we found out after more careful investigation is that the > > file is kept open by a backend connected to a different database. > > I have a suspicion that what happened here is that thi

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Kevin Grittner
Alvaro Herrera wrote: > What we found out after more careful investigation is that the > file is kept open by a backend connected to a different database. > I have a suspicion that what happened here is that this backend > was forced to flush out a page from shared buffers to read some > other

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of sáb jun 04 12:49:05 -0400 2011: > Alvaro Herrera writes: > > What surprises me is that the open references remain after a database > > drop. Surely this means that no backends keep open file descriptors to > > any table in that database, because there are no co

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-04 Thread Tom Lane
Alvaro Herrera writes: > What surprises me is that the open references remain after a database > drop. Surely this means that no backends keep open file descriptors to > any table in that database, because there are no connections. bgwriter ... regards, tom lane -- Sen

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-03 Thread Alvaro Herrera
Excerpts from Alexander Shulgin's message of vie jun 03 17:45:28 -0400 2011: > There were about 450 such (or similar) files, all of them having /2613 in the > filename. Since 2613 is a regclass of pg_largeobject and we are indeed > working with quite a few large objects in that DB so this is wh

[HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-03 Thread Alexander Shulgin
Hello Hackers, There is some strange behavior we're experiencing with one of the customer's DBs (8.4) We've noticed that free disk space went down heavily on a system, and after a short analysis determined that the reason was that postmaster was holding lots of unlinked files open. A sample o