Re: Using ProcSignal to get memory context stats from a running backend

Did this go anywhere? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Using ProcSignal to get memory context stats from a running backend

On 27.12.2017 10:44, Craig Ringer wrote: On 22 December 2017 at 23:19, Maksim Milyutin > wrote: Noticing the interest in the calling some routines on the remote backend through signals, in parallel thread[1] I have proposed the possibility to define user

Re: Using ProcSignal to get memory context stats from a running backend

On 22 December 2017 at 23:19, Maksim Milyutin wrote: > On 22.12.2017 16:56, Craig Ringer wrote: > > On 22 December 2017 at 20:50, Maksim Milyutin > wrote: > >> On 19.12.2017 16:54, Pavel Stehule wrote: >> >> sorry for small offtopic. Can be used this mechanism for log of executed >> plan or full

Re: Using ProcSignal to get memory context stats from a running backend

On 22.12.2017 16:56, Craig Ringer wrote: On 22 December 2017 at 20:50, Maksim Milyutin > wrote: On 19.12.2017 16:54, Pavel Stehule wrote: sorry for small offtopic. Can be used this mechanism for log of executed plan or full query? That's a really go

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-22 13:50 GMT+01:00 Maksim Milyutin : > On 19.12.2017 16:54, Pavel Stehule wrote: > > Hi > > 2017-12-19 14:44 GMT+01:00 Craig Ringer : > >> On 18 December 2017 at 10:05, Robert Haas wrote: >> >>> On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer >>> wrote: >>> > On 15 December 2017 at 09:24,

Re: Using ProcSignal to get memory context stats from a running backend

On 22 December 2017 at 20:50, Maksim Milyutin wrote: > On 19.12.2017 16:54, Pavel Stehule wrote: > > Hi > > 2017-12-19 14:44 GMT+01:00 Craig Ringer : > >> On 18 December 2017 at 10:05, Robert Haas wrote: >> >>> On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer >>> wrote: >>> > On 15 December 2017 a

Re: Using ProcSignal to get memory context stats from a running backend

On 19.12.2017 16:54, Pavel Stehule wrote: Hi 2017-12-19 14:44 GMT+01:00 Craig Ringer >: On 18 December 2017 at 10:05, Robert Haas mailto:robertmh...@gmail.com>> wrote: On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer mailto:cr...@2ndquadrant.com

Re: Using ProcSignal to get memory context stats from a running backend

On 22 December 2017 at 13:05, Craig Ringer wrote: > On 21 December 2017 at 15:55, Craig Ringer wrote: > >> On 21 December 2017 at 15:24, Andres Freund wrote: >> >>> Hi, >>> >>> On 2017-12-21 15:13:13 +0800, Craig Ringer wrote: >>> > There tons of callers to enlargeStringInfo, so a 'noerror' pa

Re: Using ProcSignal to get memory context stats from a running backend

On 21 December 2017 at 15:55, Craig Ringer wrote: > On 21 December 2017 at 15:24, Andres Freund wrote: > >> Hi, >> >> On 2017-12-21 15:13:13 +0800, Craig Ringer wrote: >> > There tons of callers to enlargeStringInfo, so a 'noerror' parameter >> would >> > be viable. >> >> Not sure what you mean

Re: Using ProcSignal to get memory context stats from a running backend

On 21 December 2017 at 15:24, Andres Freund wrote: > Hi, > > On 2017-12-21 15:13:13 +0800, Craig Ringer wrote: > > There tons of callers to enlargeStringInfo, so a 'noerror' parameter > would > > be viable. > > Not sure what you mean with that sentence? > Mangled in editing and sent prematurely

Re: Using ProcSignal to get memory context stats from a running backend

Hi, On 2017-12-21 15:13:13 +0800, Craig Ringer wrote: > There tons of callers to enlargeStringInfo, so a 'noerror' parameter would > be viable. Not sure what you mean with that sentence? > But I'm not convinced it's worth it personally. If we OOM in response to a > ProcSignal request for memor

Re: Using ProcSignal to get memory context stats from a running backend

On 21 December 2017 at 14:58, Andres Freund wrote: > Hi, > > On 2017-12-21 14:49:28 +0800, Craig Ringer wrote: > > +/* > > + * Accumulate writes into the buffer in diag_request_buf, > > + * for use with functions that expect a printf-like callback. > > + */ > > +static void > > +printwrapper_stri

Re: Using ProcSignal to get memory context stats from a running backend

Hi, On 2017-12-21 14:49:28 +0800, Craig Ringer wrote: > +/* > + * Accumulate writes into the buffer in diag_request_buf, > + * for use with functions that expect a printf-like callback. > + */ > +static void > +printwrapper_stringinfo(void *extra, const char * fmt, ...) > +{ > + StringInfo out

Re: Using ProcSignal to get memory context stats from a running backend

On 20 December 2017 at 08:46, Craig Ringer wrote: > On 20 December 2017 at 02:35, Andres Freund wrote: > > >> > Yeah. But please don't mess with MemoryContextStats per se --- >> > I dunno about you guys but "call MemoryContextStats(TopMemoryContext)" >> > is kinda wired into my gdb reflexes. I

Re: Using ProcSignal to get memory context stats from a running backend

On 20 December 2017 at 02:35, Andres Freund wrote: > > > > Yeah. But please don't mess with MemoryContextStats per se --- > > I dunno about you guys but "call MemoryContextStats(TopMemoryContext)" > > is kinda wired into my gdb reflexes. I think what'd make sense > > is a new function "MemoryC

Re: Using ProcSignal to get memory context stats from a running backend

Andres Freund writes: > On 2017-12-19 13:17:52 -0500, Tom Lane wrote: >> Yeah. But please don't mess with MemoryContextStats per se --- >> I dunno about you guys but "call MemoryContextStats(TopMemoryContext)" >> is kinda wired into my gdb reflexes. I think what'd make sense >> is a new function

Re: Using ProcSignal to get memory context stats from a running backend

On 2017-12-19 13:17:52 -0500, Tom Lane wrote: > Robert Haas writes: > > On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: > >> I didn't want to mess with the MemoryContextMethods and expose a > >> printf-wrapper style typedef in memnodes.h, so I went with a hook global. > > > That looks prett

Re: Using ProcSignal to get memory context stats from a running backend

Robert Haas writes: > On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: >> I didn't want to mess with the MemoryContextMethods and expose a >> printf-wrapper style typedef in memnodes.h, so I went with a hook global. > That looks pretty grotty to me. I think if you want to elog/ereport > thi

Re: Using ProcSignal to get memory context stats from a running backend

On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: > I didn't want to mess with the MemoryContextMethods and expose a > printf-wrapper style typedef in memnodes.h, so I went with a hook global. That looks pretty grotty to me. I think if you want to elog/ereport this, you need to pass another a

Re: Using ProcSignal to get memory context stats from a running backend

Hi 2017-12-19 14:44 GMT+01:00 Craig Ringer : > On 18 December 2017 at 10:05, Robert Haas wrote: > >> On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer >> wrote: >> > On 15 December 2017 at 09:24, Greg Stark wrote: >> >> Another simpler option would be to open up a new file in the log >> >> directo

Re: Using ProcSignal to get memory context stats from a running backend

On 18 December 2017 at 10:05, Robert Haas wrote: > On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer > wrote: > > On 15 December 2017 at 09:24, Greg Stark wrote: > >> Another simpler option would be to open up a new file in the log > >> directory > > > > ... if we have one. > > > > We might be logg

Re: Using ProcSignal to get memory context stats from a running backend

On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer wrote: > On 15 December 2017 at 09:24, Greg Stark wrote: >> Another simpler option would be to open up a new file in the log >> directory > > ... if we have one. > > We might be logging to syslog, or whatever else. > > I'd rather keep it simple(ish).

Re: Using ProcSignal to get memory context stats from a running backend

On 15 December 2017 at 09:24, Greg Stark wrote: > Another simpler option would be to open up a new file in the log > directory ... if we have one. We might be logging to syslog, or whatever else. I'd rather keep it simple(ish). -- Craig Ringer http://www.2ndQuadrant.com/

Re: Using ProcSignal to get memory context stats from a running backend

Another simpler option would be to open up a new file in the log directory something like "debug_dump..txt" and print whatever you want there. Then print out a reasonable size log entry saying "Debug dump output to file 'debug_dump..txt'". You could provide a function that reads such files out of t

Re: Using ProcSignal to get memory context stats from a running backend

On 13 December 2017 at 12:10, Michael Paquier wrote: > On Mon, Dec 11, 2017 at 10:07 PM, Craig Ringer > wrote: > > On 12 December 2017 at 12:43, Andres Freund wrote: > >> On 2017-12-12 11:57:41 +0800, Craig Ringer wrote: > >> But that'd have > >> the disadvanatage that it possibly would take a

Re: Using ProcSignal to get memory context stats from a running backend

On Mon, Dec 11, 2017 at 10:07 PM, Craig Ringer wrote: > On 12 December 2017 at 12:43, Andres Freund wrote: >> On 2017-12-12 11:57:41 +0800, Craig Ringer wrote: >> But that'd have >> the disadvanatage that it possibly would take a while till the >> MemoryContextStats() is executed. Not sure if tha

Re: Using ProcSignal to get memory context stats from a running backend

On 2017-12-12 14:16:59 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund wrote: > > I agree that it's more reliable - I hope there's no meaningful safety > > difference. I think you overestimate users a bit however - far from > > most of them are going to be able to extra

Re: Using ProcSignal to get memory context stats from a running backend

On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund wrote: > I agree that it's more reliable - I hope there's no meaningful safety > difference. I think you overestimate users a bit however - far from > most of them are going to be able to extract a very long log entry from > a busy log file. There's

Re: Using ProcSignal to get memory context stats from a running backend

On 2017-12-12 14:04:55 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund wrote: > > I do wonder if the right thing here wouldn't be to put the result into a > > dsm segment, and then return that to the UDF on the requesting > > side. Logging to the server log and then have

Re: Using ProcSignal to get memory context stats from a running backend

On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund wrote: > I do wonder if the right thing here wouldn't be to put the result into a > dsm segment, and then return that to the UDF on the requesting > side. Logging to the server log and then have the requestor dig that out > doesn't seem particularly u

Re: Using ProcSignal to get memory context stats from a running backend

On 2017-12-12 14:25:48 +0800, Craig Ringer wrote: > If we were willing to wrap variadic calls in a callback and rely on > vfprintf, we could: I don't think there's problems with relying on variadic calls, we do that in plenty places. > and let the ProcSignal based caller pass an elog wrapper ins

Re: Using ProcSignal to get memory context stats from a running backend

On 12 December 2017 at 14:06, Tom Lane wrote: > > Another question is whether printing to stderr, bypassing proper > > logging!, is good enough for something like this. > > Yeah, this is an issue. MemoryContextStats is designed to print > to stderr in the (possibly vain) hope that it will work

Re: Using ProcSignal to get memory context stats from a running backend

On 12 December 2017 at 12:43, Andres Freund wrote: > Hi, > > On 2017-12-12 11:57:41 +0800, Craig Ringer wrote: > > TL;DR: Lets add a ProcSignalReason that makes a backend > > call MemoryContextStats when it sees it and a C func that users can use > to > > set it on a proc. Sane? > > It's not unpr

Re: Using ProcSignal to get memory context stats from a running backend

Andres Freund writes: > On 2017-12-12 11:57:41 +0800, Craig Ringer wrote: >> TL;DR: Lets add a ProcSignalReason that makes a backend >> call MemoryContextStats when it sees it and a C func that users can use to >> set it on a proc. Sane? > It's not unproblematic. procsignal_sigusr1_handler() runs

Re: Using ProcSignal to get memory context stats from a running backend

On 12 December 2017 at 12:25, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > From: Craig Ringer [mailto:cr...@2ndquadrant.com] > > TL;DR: Lets add a ProcSignalReason that makes a backend call > > MemoryContextStats when it sees it and a C func that users can use to set > > it on a

Re: Using ProcSignal to get memory context stats from a running backend

Hi, On 2017-12-12 11:57:41 +0800, Craig Ringer wrote: > TL;DR: Lets add a ProcSignalReason that makes a backend > call MemoryContextStats when it sees it and a C func that users can use to > set it on a proc. Sane? It's not unproblematic. procsignal_sigusr1_handler() runs in a signal handler, so

RE: Using ProcSignal to get memory context stats from a running backend

From: Craig Ringer [mailto:cr...@2ndquadrant.com] > TL;DR: Lets add a ProcSignalReason that makes a backend call > MemoryContextStats when it sees it and a C func that users can use to set > it on a proc. Sane? > So how about borrowing a ProcSignalReason entry for "dump a memory context > summary