Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-11-14 Thread Andres Freund
On 2014-10-27 16:09:30 +0530, Abhijit Menon-Sen wrote: > At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > > > On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: > > > > > > 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to > > >earlier in StartupXLOG. > >

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-10-27 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: > > > > 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to > >earlier in StartupXLOG. > > > > 2. Inside that function, issue fsync()s for the main forks we

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
On 2014-09-26 02:34:06 +0530, Abhijit Menon-Sen wrote: > At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > Unless I miss something this isn't sufficient. We need to fsync the > > files in the data directory, not just the toplevel directory? > > No, of course you're right. So a separa

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > * Also recovery shouldn't be regarded successful if the reset fails - > * e.g. because of ENOSPC. OK. > * Doing this in a separate pass is advantageous for performance reasons > * because it allows the kernel to perform all the flu

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
Hi, On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: > Hi Andres, Robert. > > I've attached four patches here. Cool. Will review. > 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to >earlier in StartupXLOG. > > 2. Inside that function, issue fsync()s for the main

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-24 Thread Abhijit Menon-Sen
Hi Andres, Robert. I've attached four patches here. 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to earlier in StartupXLOG. 2. Inside that function, issue fsync()s for the main forks we create by copying the _init fork. 3. A small fixup to add a const to "typedef cha

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-22 Thread Robert Haas
On Thu, Sep 18, 2014 at 4:31 AM, Andres Freund wrote: > On 2014-09-12 14:44:48 -0400, Robert Haas wrote: >> On Fri, Sep 12, 2014 at 7:22 AM, Andres Freund >> wrote: >> > What I like even less is that pg_control is actually marked as >> > DB_SHUTDOWNED due to END_OF_RECOVERY. That's just plain wr

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-18 Thread Andres Freund
On 2014-09-12 14:44:48 -0400, Robert Haas wrote: > On Fri, Sep 12, 2014 at 7:22 AM, Andres Freund wrote: > > What I like even less is that pg_control is actually marked as > > DB_SHUTDOWNED due to END_OF_RECOVERY. That's just plain wrong. Obviously > > the database was *NOT* shutdown peacefully. I

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-12 Thread Robert Haas
On Fri, Sep 12, 2014 at 7:22 AM, Andres Freund wrote: > What I like even less is that pg_control is actually marked as > DB_SHUTDOWNED due to END_OF_RECOVERY. That's just plain wrong. Obviously > the database was *NOT* shutdown peacefully. I don't see active bugs due > it besides this, but I think

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-12 Thread Andres Freund
Hi, On 2014-09-12 13:22:46 +0200, Andres Freund wrote: > Because at least the former is something that obviously we can't (and > don't want) to change in the back branches I think the solution for this > particular problem is to simply move the ResetUnloggedRelations() call a > couple lines up to

[HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-12 Thread Andres Freund
Hi, Abhijit and I investigated a customer problem which has showed that crash recovery + unlogged relations don't always work well together: A condensed version of how crash recovery works is: StartupXLOG() { ... if (ControlFile->state != DB_SHUTDOWNED) InRecovery = true; if