Re: Reaping Temp tables to avoid XID wraparound

2019-03-08 Thread Michael Paquier
On Fri, Mar 08, 2019 at 11:14:46AM -0800, Magnus Hagander wrote: > On Mon, Feb 25, 2019 at 10:45 PM Michael Paquier > wrote: >> One problem that I can see with your patch is that you would set the >> XID once any temporary object created, including when objects other >> than tables are created in

Re: Reaping Temp tables to avoid XID wraparound

2019-03-08 Thread Magnus Hagander
On Mon, Feb 25, 2019 at 10:45 PM Michael Paquier wrote: > On Fri, Feb 22, 2019 at 04:01:02PM +0100, Magnus Hagander wrote: > > I did the "insert column in the middle of pg_stat_get_activity", I'm not > > sure that is right -- how do we treate that one? Do we just append at the > > end because peo

Re: Reaping Temp tables to avoid XID wraparound

2019-02-25 Thread Michael Paquier
On Fri, Feb 22, 2019 at 04:01:02PM +0100, Magnus Hagander wrote: > I did the "insert column in the middle of pg_stat_get_activity", I'm not > sure that is right -- how do we treate that one? Do we just append at the > end because people are expected to use the pg_stat_activity view? It's a > nontri

Re: Reaping Temp tables to avoid XID wraparound

2019-02-22 Thread Magnus Hagander
On Wed, Feb 20, 2019 at 3:41 AM Michael Paquier wrote: > On Tue, Feb 19, 2019 at 09:56:28AM +0100, Magnus Hagander wrote: > > 2. Or probably even better, just put it in PgBackendStatus? Overhead here > > is a lot cheaper than PGPROC. > > > > ISTM 2 is probably the most reasonable option here? > >

Re: Reaping Temp tables to avoid XID wraparound

2019-02-19 Thread Michael Paquier
On Tue, Feb 19, 2019 at 09:56:28AM +0100, Magnus Hagander wrote: > 2. Or probably even better, just put it in PgBackendStatus? Overhead here > is a lot cheaper than PGPROC. > > ISTM 2 is probably the most reasonable option here? Yes, I forgot this one. That would be more consistent, even if the

Re: Reaping Temp tables to avoid XID wraparound

2019-02-19 Thread Magnus Hagander
On Mon, Feb 18, 2019 at 2:31 AM Michael Paquier wrote: > On Sun, Feb 17, 2019 at 05:47:09PM +0100, Magnus Hagander wrote: > > We could I guess add a field specifically for temp_namespace_xid or such. > > The question is if it's worth the overhead to do that. > > That would mean an extra 4 bytes i

Re: Reaping Temp tables to avoid XID wraparound

2019-02-18 Thread Michael Paquier
On Tue, Feb 19, 2019 at 10:52:54AM +1100, James Sewell wrote: > I agree the use case is narrow - but it's also pretty critical. Yeah.. > I suppose an in-core way of disconnecting idle sessions after x time would > work too - but that seems like a sledgehammer approach. Such solutions at SQL leve

Re: Reaping Temp tables to avoid XID wraparound

2019-02-18 Thread James Sewell
On Mon, 18 Feb 2019 at 12:31, Michael Paquier wrote: > On Sun, Feb 17, 2019 at 05:47:09PM +0100, Magnus Hagander wrote: > > We could I guess add a field specifically for temp_namespace_xid or such. > > The question is if it's worth the overhead to do that. > > That would mean an extra 4 bytes in

Re: Reaping Temp tables to avoid XID wraparound

2019-02-17 Thread Michael Paquier
On Sun, Feb 17, 2019 at 05:47:09PM +0100, Magnus Hagander wrote: > We could I guess add a field specifically for temp_namespace_xid or such. > The question is if it's worth the overhead to do that. That would mean an extra 4 bytes in PGPROC, which is something we could live with, still the use-cas

Re: Reaping Temp tables to avoid XID wraparound

2019-02-17 Thread James Sewell
> Yeah, possibly. I think that it could be tricky though to get that at >> a global level in a cheap way. It makes also little sense to only >> show the temp namespace OID if that information is not enough. >> > > We could I guess add a field specifically for temp_namespace_xid or such. > The que

Re: Reaping Temp tables to avoid XID wraparound

2019-02-17 Thread Magnus Hagander
On Thu, Feb 14, 2019 at 1:43 AM Michael Paquier wrote: > On Wed, Feb 13, 2019 at 05:48:39PM +0100, Magnus Hagander wrote: > > On Wed, Feb 13, 2019 at 2:26 AM Michael Paquier > wrote: > >> The temporary namespace OID is added to PGPROC since v11, so it could > >> be easy enough to add a system fu

Re: Reaping Temp tables to avoid XID wraparound

2019-02-13 Thread Michael Paquier
On Wed, Feb 13, 2019 at 05:48:39PM +0100, Magnus Hagander wrote: > On Wed, Feb 13, 2019 at 2:26 AM Michael Paquier wrote: >> The temporary namespace OID is added to PGPROC since v11, so it could >> be easy enough to add a system function which maps a temp schema to a >> PID. Now, it could actuall

Re: Reaping Temp tables to avoid XID wraparound

2019-02-13 Thread James Sewell
It's easy to identify the temp tables which are causing the problem, yes. The issue here is just getting rid of them. In an ideal world I wouldn't actually have to care about the session and I could just drop the table (or vacuum the table?). Dropping the session was just the best way I could fin

Re: Reaping Temp tables to avoid XID wraparound

2019-02-13 Thread Magnus Hagander
On Wed, Feb 13, 2019 at 6:05 PM Tom Lane wrote: > Magnus Hagander writes: > > And while at it, what would in this particular case have been even more > > useful to the OP would be to actually identify that there is a temp table > > *and which xid it's blocking at*. For regular transactions we ca

Re: Reaping Temp tables to avoid XID wraparound

2019-02-13 Thread Tom Lane
Magnus Hagander writes: > And while at it, what would in this particular case have been even more > useful to the OP would be to actually identify that there is a temp table > *and which xid it's blocking at*. For regular transactions we can look at > backend_xid, but IIRC that doesn't work for te

Re: Reaping Temp tables to avoid XID wraparound

2019-02-13 Thread Magnus Hagander
On Wed, Feb 13, 2019 at 2:26 AM Michael Paquier wrote: > On Wed, Feb 13, 2019 at 12:38:51AM +, Andrew Gierth wrote: > > Doesn't work - that function's idea of "backend id" doesn't match the > > real one, since it's looking at a local copy of the stats from which > > unused slots have been rem

RE: Reaping Temp tables to avoid XID wraparound

2019-02-12 Thread Tsunakawa, Takayuki
From: Andrew Gierth [mailto:and...@tao11.riddles.org.uk] > Tsunakawa> SELECT pg_stat_get_backend_pid(backendid); > > Doesn't work - that function's idea of "backend id" doesn't match the > real one, since it's looking at a local copy of the stats from which > unused slots have been removed. Ouch

Re: Reaping Temp tables to avoid XID wraparound

2019-02-12 Thread Michael Paquier
On Wed, Feb 13, 2019 at 12:38:51AM +, Andrew Gierth wrote: > Doesn't work - that function's idea of "backend id" doesn't match the > real one, since it's looking at a local copy of the stats from which > unused slots have been removed. The temporary namespace OID is added to PGPROC since v11,

Re: Reaping Temp tables to avoid XID wraparound

2019-02-12 Thread Andrew Gierth
> "Tsunakawa" == Tsunakawa, Takayuki > writes: >> From the temp table namespace I can get the backend ID using a regex >> - but I have no idea how I can map that to a PID - any thoughts? Tsunakawa> SELECT pg_stat_get_backend_pid(backendid); Doesn't work - that function's idea of "ba

RE: Reaping Temp tables to avoid XID wraparound

2019-02-12 Thread Tsunakawa, Takayuki
From: James Sewell [mailto:james.sew...@jirotech.com] > From the temp table namespace I can get the backend ID using a regex - but > I have no idea how I can map that to a PID - any thoughts? > SELECT pg_stat_get_backend_pid(backendid); https://www.postgresql.org/docs/devel/monitoring-stats.html