Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-03 Thread Kevin Grittner
"Kevin Grittner" wrote: > Any objections to my putting it on the TODO list? Hearing none, added. (Apologies for missing the box where I should have commented on the what the change did.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > Perhaps, if you didn't mind sometimes getting a wrong answer. Well, it would be a heuristic which would close the deleted file *almost* all the time. When it didn't, the next check would probably catch it. Assuming that you would never get an indication that it was differen

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> Determining whether it's still the current append target is not so >> cheap though; it would require examining shared-memory status >> which means taking a lock on that status (and it's a high-traffic >> lock already). > I haven't reviewed the inter

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > Determining whether it's still the current append target is not so > cheap though; it would require examining shared-memory status > which means taking a lock on that status (and it's a high-traffic > lock already). I haven't reviewed the internal locking techniques, so this

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Tom Lane
"Kevin Grittner" writes: > Is there a reasonably cheap way to check whether the backend has a > WAL file open and whether that one is the current append target? Detecting whether we have a WAL file open is trivial (just look at the static variable holding the file descriptor). Determining whethe

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > How old were the sessions you were looking at? Days to months old. > If we think this is worth doing something about > (I'm not convinced yet) Once one knows about the issue, it's only a minor annoyance, and that infrequently, so it's not worth anything that would take si

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> There has to be something causing those sessions to touch WAL, and >> the dirty-buffer scenario doesn't seem reliable enough. > This is seeming fairly likely to be the cause, though. It may be a > combination of the nightly VACUUM FREEZE ANALYZE w

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > There has to be something causing those sessions to touch WAL, and > the dirty-buffer scenario doesn't seem reliable enough. This is seeming fairly likely to be the cause, though. It may be a combination of the nightly VACUUM FREEZE ANALYZE we typically do on every database

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > It's not about the size of a temp table, because writes to the > temp table itself aren't WAL-logged. However, the system catalog > entries for a temp table *are* WAL-logged. Definitely not issuing any CREATE TEMP statements of any kind, unless the JDBC driver is doing that

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> You sure it's not creating any temp tables? You didn't mention >> revoking TEMP privilege. > They have not been revoked, but I am sure the software publisher > doesn't explicitly create any, and I'd be very surprised if the > monitoring software d

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > You sure it's not creating any temp tables? You didn't mention > revoking TEMP privilege. They have not been revoked, but I am sure the software publisher doesn't explicitly create any, and I'd be very surprised if the monitoring software did. The tables are small enough t

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> A backend would never open a WAL file unless it had to write a WAL >> record, so I'm having a hard time believing that these were >> totally read-only transactions. Can you give specifics? > You will note that the connections logged in as "viewer" (

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: >> It seemed strange that the only backends which were holding open >> deleted WAL files were ones where the connection was established >> with a login which has no write permissions. > > A backend would never open a WAL file unless it had to write a WAL > record, so I'm having a

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-25 Thread Tom Lane
"Kevin Grittner" writes: > I ran across the subject issue and spent some time puzzling over it. > ... > I'm not sure whether Tom's comment that "There is zero hope of > making that work." referred to the idea that we could close deleted > WAL files or to something else. Is a fix feasible? The re

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-25 Thread Itagaki Takahiro
"Kevin Grittner" wrote: > I guess it is a stretch to imagine that a database would have > enough read-only connections to exhaust resources by holding open > one deleted WAL file each; unless they have, say, 200 such > connections and they're cutting things so close that a wasted 3.2GB > of disk