Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-13 Thread Heikki Linnakangas
On 08/08/18 19:19, Heikki Linnakangas wrote: So, with this commit, the various SIGQUIT quickdie handlers are in a better shape. The regular backend's quickdie() handler still calls ereport(), which is not safe, but it's a step in the right direction. And we haven't addressed the original complain

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-08 Thread Nico Williams
On Wed, Aug 08, 2018 at 11:47:34AM -0500, Nico Williams wrote: > Yes. Would that snprintf() and vsnprintf() were async-signal-safe -- > they can be, and some implementations probably are, but they aren't > required to be, then making ereport() safe would be easier. So, I took a look at glibc's im

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-08 Thread Nico Williams
On Wed, Aug 08, 2018 at 07:19:34PM +0300, Heikki Linnakangas wrote: > On 20/07/18 18:03, Andres Freund wrote: > >It's much less the exit() that's unsafe, than the callbacks themselves, > >right? Perhaps just restate that we wouldn't want to trigger atexit > >processing due to signal safety? > > W

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-08 Thread Heikki Linnakangas
On 20/07/18 18:03, Andres Freund wrote: On 2018-07-20 11:53:32 +0300, Heikki Linnakangas wrote: But I think we have consensus on replacing the exit(2) calls with _exit(2). If we do just that, it would be better than the status quo, even if it doesn't completely fix the problem. This would preven

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-23 Thread Robert Haas
On Fri, Jul 20, 2018 at 4:53 AM, Heikki Linnakangas wrote: > ISTM that no-one has any great ideas on what to do about the ereport() in > quickdie(). But I think we have consensus on replacing the exit(2) calls > with _exit(2). If we do just that, it would be better than the status quo, > even if i

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Ashwin Agrawal
On Fri, Jul 20, 2018 at 1:54 AM Heikki Linnakangas wrote: > On 19/07/18 23:13, Andres Freund wrote: > > On 2018-07-19 14:54:52 -0500, Nico Williams wrote: > >> On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: > >>> On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > Ugh.

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Andres Freund
On 2018-07-20 11:53:32 +0300, Heikki Linnakangas wrote: > ISTM that no-one has any great ideas on what to do about the ereport() in > quickdie(). I think we actually have some decent ideas how to make that less problematic in a number of cases. Avoid translation (thereby avoiding direct malloc())

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Heikki Linnakangas
On 19/07/18 23:13, Andres Freund wrote: On 2018-07-19 14:54:52 -0500, Nico Williams wrote: On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: Ugh. Yeah, in wal_quickdie, and other aux process *_quickdie() handlers, I agree we

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Heikki Linnakangas
On 19/07/18 23:04, Nico Williams wrote: On Thu, Jun 22, 2017 at 03:10:31PM -0400, Tom Lane wrote: Andres Freund writes: Or, probably more robust: Simply _exit(2) without further ado, and rely on postmaster to output an appropriate error message. Arguably it's not actually useful to see hundred

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:46:12PM -0700, Andres Freund wrote: > On 2018-07-19 15:42:46 -0500, Nico Williams wrote: > > Yes, but that's in libc. None of that is in the PG code itself. > > That's simply entirely completely wrong. PG has a good chunk of memory > management layers (facilitating memo

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
On 2018-07-19 15:44:23 -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 03:42:46PM -0500, Nico Williams wrote: > > On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > > > Uhm, this'd already require a fair bit of threadsafety. Like at least > > > all of the memory allocator / cont

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
On 2018-07-19 15:42:46 -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > > On 2018-07-19 15:27:06 -0500, Nico Williams wrote: > > > No, the other thread does NOT continue to do whatever -- it > > > blocks/sleeps forever waiting for the coming exit(3). >

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 03:42:46PM -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > > Uhm, this'd already require a fair bit of threadsafety. Like at least > > all of the memory allocator / context code. Nor is having threads > > around unproblematic f

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > On 2018-07-19 15:27:06 -0500, Nico Williams wrote: > > No, the other thread does NOT continue to do whatever -- it > > blocks/sleeps forever waiting for the coming exit(3). > > > > I.e., quickdie() would look like this: > > > >

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
Hi, On 2018-07-19 15:27:06 -0500, Nico Williams wrote: > No, the other thread does NOT continue to do whatever -- it > blocks/sleeps forever waiting for the coming exit(3). > > I.e., quickdie() would look like this: > > /* Marshall info in to an automatic */ > struct quickdie_inf

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:35:02PM -0700, Andres Freund wrote: > On 2018-07-19 15:17:26 -0500, Nico Williams wrote: > > You can create that thread with a really small stack given that its only > > purpose is to do this error reporting and exit. > > You still have a full kernel process backing it,

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
Hi, On 2018-07-19 16:16:31 -0400, Tom Lane wrote: > Nico Williams writes: > > I dunno if it is or isn't helpful. But I do know that this must be done > > in an async-signal-safe way. > > I haven't actually heard a convincing reason why that's true. As per > the previous discussion, if we happe

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
On 2018-07-19 15:17:26 -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 01:10:14PM -0700, Andres Freund wrote: > > On 2018-07-19 15:04:15 -0500, Nico Williams wrote: > > > Besides making ereport() async-signal-safe, which is tricky, you could > > > write(2) the arguments to a pipe that another

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 04:16:31PM -0400, Tom Lane wrote: > Nico Williams writes: > > I dunno if it is or isn't helpful. But I do know that this must be done > > in an async-signal-safe way. > > I haven't actually heard a convincing reason why that's true. As per It's undefined behavior. The

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:10:14PM -0700, Andres Freund wrote: > On 2018-07-19 15:04:15 -0500, Nico Williams wrote: > > Besides making ereport() async-signal-safe, which is tricky, you could > > write(2) the arguments to a pipe that another thread in the same process > > is reading from and which w

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Tom Lane
Nico Williams writes: > I dunno if it is or isn't helpful. But I do know that this must be done > in an async-signal-safe way. I haven't actually heard a convincing reason why that's true. As per the previous discussion, if we happen to service the SIGQUIT at an unfortunate moment, we might get

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
On 2018-07-19 14:54:52 -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: > > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > > > Ugh. Yeah, in wal_quickdie, and other aux process *_quickdie() handlers, I > > > agree we should just _exit(2). All w

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
On 2018-07-19 15:04:15 -0500, Nico Williams wrote: > Besides making ereport() async-signal-safe, which is tricky, you could > write(2) the arguments to a pipe that another thread in the same process > is reading from and which will then call ereport() and exit(3). This > would be less work if you'

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
Hi, On 2018-07-19 15:49:35 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > >> The regular backend's quickdie() function is more tricky. It should also > >> call _exit(2) rather than exit(2). But it also tries to ereport a WARNING, > >>

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 04:04:01PM -0400, Tom Lane wrote: > Nico Williams writes: > > What I'd do is have a volatile sig_atomic_t in_signal_handler_context > > variable to indicate that we're dying, and then when that is non-zero, > > ereport() and friends could use all-async-signal-safe codepaths

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jun 22, 2017 at 03:10:31PM -0400, Tom Lane wrote: > Andres Freund writes: > > Or, probably more robust: Simply _exit(2) without further ado, and rely > > on postmaster to output an appropriate error message. Arguably it's not > > actually useful to see hundreds of "WARNING: terminating con

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Tom Lane
Nico Williams writes: > What I'd do is have a volatile sig_atomic_t in_signal_handler_context > variable to indicate that we're dying, and then when that is non-zero, > ereport() and friends could use all-async-signal-safe codepaths. I eagerly await your patch with an async-safe implementation of

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 03:49:35PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > >> The regular backend's quickdie() function is more tricky. It should also > >> call _exit(2) rather than exit(2). But it also tries to ereport a WARNING

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > > Ugh. Yeah, in wal_quickdie, and other aux process *_quickdie() handlers, I > > agree we should just _exit(2). All we want to do is to exit the process > > immediately. > > I

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Tom Lane
Andres Freund writes: > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: >> The regular backend's quickdie() function is more tricky. It should also >> call _exit(2) rather than exit(2). But it also tries to ereport a WARNING, >> and that is quite useful. There's already an on_exit_reset i

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Andres Freund
Hi, On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > On 19/07/18 03:26, Asim R P wrote: > > On Thu, Jun 22, 2017 at 10:50 AM, Andres Freund wrote: > > > > > > Or, probably more robust: Simply _exit(2) without further ado, and rely > > > on postmaster to output an appropriate error messa

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 03:26, Asim R P wrote: On Thu, Jun 22, 2017 at 10:50 AM, Andres Freund wrote: Or, probably more robust: Simply _exit(2) without further ado, and rely on postmaster to output an appropriate error message. Arguably it's not actually useful to see hundreds of "WARNING: terminating con

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-18 Thread Asim R P
On Thu, Jun 22, 2017 at 10:50 AM, Andres Freund wrote: > > Or, probably more robust: Simply _exit(2) without further ado, and rely > on postmaster to output an appropriate error message. Arguably it's not > actually useful to see hundreds of "WARNING: terminating connection because of > crash of a