Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Tom Lane
Peter Geoghegan writes: > On 9 May 2012 00:21, Peter Geoghegan wrote: >> Yes, there is some checking of flags before the potential ResetLatch() >> call, which may be acted on. The code there is almost identical to >> that of the extant bgwriter code. I was under the impression that this >> did no

Re: [HACKERS] synchronous_commit and remote_write

2012-05-08 Thread Bruce Momjian
On Tue, May 08, 2012 at 10:29:31PM -0400, Aidan Van Dyk wrote: > On Tue, May 8, 2012 at 10:09 PM, Bruce Momjian wrote: > > >> And then, I could envision (if it continues down this road): > >>   off > >>   local > >>   remote_accept > >>   remote_write > >>   remote_sync > >>   remote_apply (impli

Re: [HACKERS] synchronous_commit and remote_write

2012-05-08 Thread Aidan Van Dyk
On Tue, May 8, 2012 at 10:09 PM, Bruce Momjian wrote: >> And then, I could envision (if it continues down this road): >>   off >>   local >>   remote_accept >>   remote_write >>   remote_sync >>   remote_apply (implies visible to new connections on the standby) >> >> Not saying all off these are

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Tom Lane
On further reflection I've realized that there's a really unpleasant consequence of the walwriter change as-committed: it breaks the former guarantee that async commits would reach disk within at most 3 times the WalWriterDelay setting. They will still get written within at most 3 walwriter cycles

Re: [HACKERS] WIP Patch: Selective binary conversion of CSV file foreign tables

2012-05-08 Thread Etsuro Fujita
> -Original Message- > From: David Fetter [mailto:da...@fetter.org] > Sent: Wednesday, May 09, 2012 9:25 AM > To: Etsuro Fujita > Cc: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] WIP Patch: Selective binary conversion of CSV file > foreign tables > > On Tue, May 08, 2012 at 08:26:

Re: [HACKERS] synchronous_commit and remote_write

2012-05-08 Thread Bruce Momjian
On Tue, May 08, 2012 at 10:04:46PM -0400, Aidan Van Dyk wrote: > On Tue, May 8, 2012 at 9:13 PM, Bruce Momjian wrote: > > It seems pretty confusing that synchronous_commit = 'remote_write' means > > write confirmed to the remote socket, not write to the file system.  Is > > there no better term we

Re: [HACKERS] synchronous_commit and remote_write

2012-05-08 Thread Aidan Van Dyk
On Tue, May 8, 2012 at 9:13 PM, Bruce Momjian wrote: > It seems pretty confusing that synchronous_commit = 'remote_write' means > write confirmed to the remote socket, not write to the file system.  Is > there no better term we could some up with?  remote_pipe? > remote_transfer? remote_accept?

[HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-08 Thread Amit Kapila
While going through Readme in backend\storage\buffer, I found some point misleading. Normal Buffer Replacement Strategy -- -- .. .. Each buffer header contains a usage counter, which is incremented (up to a small limit value) whenever the buffer

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Tom Lane
I've applied the walwriter/checkpointer patch, with the mentioned re-simplification of the logic. While measuring that, I noticed that the stats collector was now the biggest repeated-wakeup culprit, so I took the time to latch-ify it as well. AFAICS we no longer have anything that wakes up often

[HACKERS] synchronous_commit and remote_write

2012-05-08 Thread Bruce Momjian
It seems pretty confusing that synchronous_commit = 'remote_write' means write confirmed to the remote socket, not write to the file system. Is there no better term we could some up with? remote_pipe? remote_transfer? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] WIP Patch: Selective binary conversion of CSV file foreign tables

2012-05-08 Thread David Fetter
On Tue, May 08, 2012 at 08:26:02PM +0900, Etsuro Fujita wrote: > I would like to propose to improve parsing efficiency of contrib/file_fdw by > selective parsing proposed by Alagiannis et al.[1], Is the patch they used against 9.0 published somewhere? Cheers, David. -- David Fetter http://fette

Re: [HACKERS] age(xid) on hot standby

2012-05-08 Thread Simon Riggs
On 8 May 2012 20:01, Peter Eisentraut wrote: > On ons, 2012-01-18 at 14:55 -0500, Tom Lane wrote: >> BTW, it strikes me that maybe the coding should work about like this: >> >>       if (!TransactionIdIsValid(age_reference_xid)) >>       { >>               age_reference_xid = GetTopTransactionIdIf

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Peter Geoghegan
On 9 May 2012 00:21, Peter Geoghegan wrote: > Yes, there is some checking of flags before the potential ResetLatch() > call, which may be acted on. The code there is almost identical to > that of the extant bgwriter code. I was under the impression that this > did not amount to a race, though it's

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Tom Lane
Peter Geoghegan writes: > On 8 May 2012 22:35, Tom Lane wrote: >> Now that I've actually read the patch, rather than just responding to >> your description of it, I find myself entirely unhappy with the proposed >> changes in the walwriter's sleep logic.  You have introduced race >> conditions (i

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Peter Geoghegan
On 8 May 2012 22:35, Tom Lane wrote: > Now that I've actually read the patch, rather than just responding to > your description of it, I find myself entirely unhappy with the proposed > changes in the walwriter's sleep logic.  You have introduced race > conditions (it is NOT okay to reset the latc

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-08 Thread Tom Lane
Peter Geoghegan writes: > Attached patch removes the questionable SetLatch() call, under the > assumption that it's okay if the WALWriter, having entered hibernation > due to sustained inactivity (10 seconds) subsequently takes up to 5 > seconds (2.5 on average) to notice that it has work to do. T

Re: [HACKERS] age(xid) on hot standby

2012-05-08 Thread Peter Eisentraut
On ons, 2012-01-18 at 14:55 -0500, Tom Lane wrote: > BTW, it strikes me that maybe the coding should work about like this: > > if (!TransactionIdIsValid(age_reference_xid)) > { > age_reference_xid = GetTopTransactionIdIfAny(); > if (!TransactionIdIsValid(age

Re: [HACKERS] more possible dead ports cleanup

2012-05-08 Thread Peter Eisentraut
On mån, 2012-05-07 at 10:50 -0400, Robert Haas wrote: > > - (Side point, the definition of endof() in the same place isn't used > > anywhere, and doesn't look safe to me, because it can go one past the > > end of memory.) > > That I think we could remove. I decided to keep it because it was used

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-05-08 Thread Fujii Masao
On Tue, May 8, 2012 at 12:59 AM, Robert Haas wrote: > On Sat, May 5, 2012 at 12:41 PM, Fujii Masao wrote: >> On Sat, Apr 28, 2012 at 4:00 AM, Robert Haas wrote: >>> On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane wrote: I'm not necessarily opposed to commandeering the name "smart" for the n

Re: [HACKERS] problem/bug in drop tablespace?

2012-05-08 Thread Michael Nolan
The last portion of my original post got edited out by mistake. The tests I ran were on version 9.1.3, running Fedora 14, kernel 2.6.35.14-106.fc14-i686. It seems to me that DROP TABLESPACE should check to see if there are references in the system catalog to the tablespace before dropping it, not

[HACKERS] WIP Patch: Selective binary conversion of CSV file foreign tables

2012-05-08 Thread Etsuro Fujita
I would like to propose to improve parsing efficiency of contrib/file_fdw by selective parsing proposed by Alagiannis et al.[1], which means that for a CSV/TEXT file foreign table, file_fdw performs binary conversion only for the columns needed for query processing. Attached is a WIP patch impleme

Re: [HACKERS] Temporary tables under hot standby

2012-05-08 Thread Noah Misch
On Mon, May 07, 2012 at 09:04:28AM -0500, Merlin Moncure wrote: > On Mon, May 7, 2012 at 8:52 AM, Michael Nolan wrote: > > To cross-pollinate with another thread, if temporary tables (and > > insert/delete/update transactions to them) are to be supported on a slave, > > will the applications using