Re: [HACKERS] Online base backup from the hot-standby

2011-06-22 Thread Jun Ishiduka
> 1) Today you can do a backup by just calling pg_start_backup('x'); copy > the data directory and > pg_stop_backup(); You do not need to use pg_basebackup to create a > backup. The solution you are proposing would require pg_basebackup to be > used to build backups from standby servers. YES. >

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Jeff Davis
On Wed, 2011-06-22 at 21:53 -0400, Robert Haas wrote: > On Wed, Jun 22, 2011 at 8:53 PM, Jeff Davis wrote: > > On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: > >> 2. In the words of a comment added by the patch: > >> * The critical integrity requirement here is that we must never end up >

Re: [HACKERS] [COMMITTERS] pgsql: Make the visibility map crash-safe.

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 9:12 AM, Robert Haas wrote: > On Wed, Jun 22, 2011 at 6:55 AM, Heikki Linnakangas > wrote: >> On 22.06.2011 06:05, Robert Haas wrote: >>> >>> Second, when inserting, updating, or deleting >>> a tuple, we can no longer get away with clearing the visibility map >>> bit after

Re: [HACKERS] pg_upgrade version check improvements and small fixes

2011-06-22 Thread Bruce Momjian
Dan McGee wrote: > Not sure what the normal process is for patches, but I put together a > few small patches for pg_upgrade after trying to use it earlier today > and staring a non-helpful error message before I finally figured out > what was going on. Thanks for the detailed report and patches.

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 8:53 PM, Jeff Davis wrote: > On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: >> 2. In the words of a comment added by the patch: >>  * The critical integrity requirement here is that we must never end up with >>  * a situation where the visibility map bit is set, and t

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Jeff Davis
On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: > 2. In the words of a comment added by the patch: > * The critical integrity requirement here is that we must never end up with > * a situation where the visibility map bit is set, and the page-level > * PD_ALL_VISIBLE bit is clear. If that

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Peter Geoghegan
On 23 June 2011 00:37, Greg Stark wrote: > Hm. I have to say the single most confusing thing about the Postgres > source that took me a *long* time to get over was remembering that > some of the typedefs were already pointers and some weren't. It seems > silly now but when I was trying to understa

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Tom Lane
Greg Stark writes: > On Wed, Jun 22, 2011 at 5:41 PM, Tom Lane wrote: >> If we were thinking of moving in that direction, I would argue that >> we should get rid of typedef'd pointers altogether, ie, change >> "Relation" to be a typedef for the struct and write "Relation *rel" >> not "Relation re

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Greg Stark
On Wed, Jun 22, 2011 at 5:41 PM, Tom Lane wrote: > If we were thinking of moving in that direction, I would argue that > we should get rid of typedef'd pointers altogether, ie, change > "Relation" to be a typedef for the struct and write "Relation *rel" > not "Relation rel". Hm. I have to say the

Re: [HACKERS] SYNONYMS (again)

2011-06-22 Thread Alvaro Herrera
Excerpts from Joshua D. Drake's message of mié jun 22 15:37:17 -0400 2011: > Per: > > http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php > > It seems we did come up with a use case in the procpid discussion. The > ability to change the names of columns/databases etc, to handle the

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Dan Ports wrote: > Note that this hadn't been a reasonable option until last week > when we added the check for non-MVCC snapshots, since there are > lots of things that use heap scans but SeqScan is the only > (currently-existing) one we want to lock. That is the sort of thing that I tended t

Re: [HACKERS] lazy vxid locks, v1

2011-06-22 Thread Florian Pflug
On Jun12, 2011, at 23:39 , Robert Haas wrote: > So, the majority (60%) of the excess spinning appears to be due to > SInvalReadLock. A good chunk are due to ProcArrayLock (25%). Hm, sizeof(LWLock) is 24 on X86-64, making sizeof(LWLockPadded) 32. However, cache lines are 64 bytes large on recent I

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Dan Ports
On Wed, Jun 22, 2011 at 12:07:04PM +0300, Heikki Linnakangas wrote: > Hmm, I wonder if we should move this logic to heapam.c. The optimization > to acquire a relation lock straight away should apply to all heap scans, > not only those coming from ExecSeqScan. The distinction is academic at > the

[HACKERS] fixing PQsetvalue()

2011-06-22 Thread Merlin Moncure
On Jun 6 (http://archives.postgresql.org/pgsql-hackers/2011-06/msg00272.php), Pavel discovered an issue with PQsetvalue that could cause libpq to wander off into unallocated memory that was present in 9.0.x. A fairly uninteresting fix was quickly produced, but Tom indicated during subsequent revie

[HACKERS] SYNONYMS (again)

2011-06-22 Thread Joshua D. Drake
Per: http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php It seems we did come up with a use case in the procpid discussion. The ability to change the names of columns/databases etc, to handle the fixing of bad decision decisions during development over time. Thoughts? JD -- Co

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Bruce Momjian's message of mi?? jun 22 15:24:40 -0400 2011: > > Alvaro Herrera wrote: > > > Excerpts from Bruce Momjian's message of mi jun 22 14:31:51 -0400 2011: > > > > > I am going to try to add descriptions for system tables for \d+ so maybe > > > > I can

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 03:24 PM, Bruce Momjian wrote: I am going to try to add descriptions for system tables for \d+ so maybe I can put it there. That would be OK with me, I guess -- something like "shared system catalog for databases", you mean? I was thinking "(shared)" as part of the description.

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié jun 22 15:24:40 -0400 2011: > Alvaro Herrera wrote: > > Excerpts from Bruce Momjian's message of mi jun 22 14:31:51 -0400 2011: > > > I am going to try to add descriptions for system tables for \d+ so maybe > > > I can put it there. > > > > That would

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mié jun 22 11:25:43 -0400 2011: > On Fri, Jun 10, 2011 at 21:07, Tom Lane wrote: > > Magnus Hagander writes: > >> I came across a situation today with a pretty bad crash of pg_dump, > >> due to not checking the return code from malloc(). When looking > >

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Bruce Momjian's message of mi?? jun 22 14:31:51 -0400 2011: > > Alvaro Herrera wrote: > > > Excerpts from Jeff MacDonald's message of mi jun 22 09:27:36 -0400 2011: > > > > Greetings, > > > > > > > > On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote:

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié jun 22 14:31:51 -0400 2011: > Alvaro Herrera wrote: > > Excerpts from Jeff MacDonald's message of mi jun 22 09:27:36 -0400 2011: > > > Greetings, > > > > > > On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: > > > > I assumed it was important

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 22.06.2011 17:28, Tom Lane wrote: >> Why is the call in ExecSeqScan at all, and not in the node >> startup function? > > It makes sense to delay it until the scan is actually started, so > that you don't get unnecessary serialization failures if the scan > is neve

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 12:41 PM, Tom Lane wrote: > Merlin Moncure writes: >> On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane wrote: >>> Declarations like "const structtype *param" are fine, because those >>> create a real, enforced contract on what the function can do to data >>> that is visible to i

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Jeff MacDonald's message of mi?? jun 22 09:27:36 -0400 2011: > > Greetings, > > > > On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: > > > I assumed it was important to indicate if someone was looking at > > > per-database or per-cluster data, like

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 17:28, Tom Lane wrote: Dan Ports writes: I was looking at ExecSeqScan today and noticed that it invokes PredicateLockRelation each time it's called, i.e. for each tuple returned. Any reason we shouldn't skip that call if rs_relpredicatelocked is already set, as in the attached pat

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 12:51 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié jun 22 08:56:02 -0400 2011: > >> Another option might be to leave heap_openrv() and relation_openrv() >> alone and add a missing_ok argument to try_heap_openrv() and >> try_relation_openrv().  Pass

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Jeff MacDonald's message of mié jun 22 09:27:36 -0400 2011: > Greetings, > > On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: > > I assumed it was important to indicate if someone was looking at > > per-database or per-cluster data, like pg_tablespace. The issue comes >

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié jun 22 08:56:02 -0400 2011: > Another option might be to leave heap_openrv() and relation_openrv() > alone and add a missing_ok argument to try_heap_openrv() and > try_relation_openrv(). Passing true would give the same behavior as > presently; passing f

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Tom Lane
Merlin Moncure writes: > On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane wrote: >> Declarations like "const structtype *param" are fine, because those >> create a real, enforced contract on what the function can do to data >> that is visible to its caller. But I don't see any value at all in >> const-

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Tom Lane wrote: > Why is the call in ExecSeqScan at all, and not in the node startup > function? Because when I asked about the placement of such calls in January of 2010 I didn't get any advice which suggested that, and this was a place I was able to find which worked correctly. If there's a

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Merlin Moncure
On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane wrote: > Declarations like "const structtype *param" are fine, because those > create a real, enforced contract on what the function can do to data > that is visible to its caller.  But I don't see any value at all in > const-ifying the parameter itself. >

[HACKERS] Re: Coding style point: "const" in function parameter declarations

2011-06-22 Thread Kevin Grittner
Tom Lane wrote: > 2. In cases such as "const Relation foo" where the parameter type > is a typedeffed pointer, it is easy for readers to arrive at the > false conclusion that this guarantees the function doesn't change > the pointed-to structure. So easy that in fact that was my belief when I

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Magnus Hagander
On Wed, Jun 22, 2011 at 17:48, Tom Lane wrote: > Magnus Hagander writes: >> Something along the line of this? > > I think this is a seriously, seriously bad idea: > >> +#define strdup(x) pg_strdup(x) >> +#define malloc(x) pg_malloc(x) >> +#define calloc(x,y) pg_calloc(x, y) >> +#define realloc(x,

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Peter Geoghegan
On 22 June 2011 16:25, Magnus Hagander wrote: > Something along the line of this? IMHO the redefinition of malloc() looks a bit hairy...can't you just make the callers use the functions directly? -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Trainin

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Tom Lane
Magnus Hagander writes: > Something along the line of this? I think this is a seriously, seriously bad idea: > +#define strdup(x) pg_strdup(x) > +#define malloc(x) pg_malloc(x) > +#define calloc(x,y) pg_calloc(x, y) > +#define realloc(x,y) pg_realloc(x, y) as it will render the code unreadable

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Tom Lane
I wrote: > That previous approach of adding extra expected files isn't going to > scale nicely if there are multiple places at risk ... but do we need > multiple places selecting the sequence contents? I remain of the > opinion that just omitting the value isn't good testing policy. Actually, on

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2011-06-22 Thread Simon Riggs
On Tue, Jun 21, 2011 at 4:12 PM, Tom Lane wrote: > Simon Riggs writes: >> On Mon, Jun 20, 2011 at 10:56 PM, Tom Lane wrote: >>> The ALTER TABLE patch >>> has greatly expanded the scope of the issue, and that *is* a regression >>> compared to prior releases. > >> I agree the scope for RELOID erro

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Tom Lane
Josh Kupershmidt writes: > Hmph, looks like buildfarm members koi and jaguar are failing make check now: > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=koi&dt=2011-06-22%2008%3A06%3A00 > due to a difference in sequence.out. I didn't muck with the part about > SELECT * FROM foo_seq_new

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Magnus Hagander
On Fri, Jun 10, 2011 at 21:07, Tom Lane wrote: > Magnus Hagander writes: >> I came across a situation today with a pretty bad crash of pg_dump, >> due to not checking the return code from malloc(). When looking >> through the code, it seems there are a *lot* of places in pg_dump that >> doesn't c

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Tom Lane
Dan Ports writes: > I was looking at ExecSeqScan today and noticed that it invokes > PredicateLockRelation each time it's called, i.e. for each tuple > returned. Any reason we shouldn't skip that call if > rs_relpredicatelocked is already set, as in the attached patch? Why is the call in ExecSeqS

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Tom Lane
Robert Haas writes: > Another option might be to leave heap_openrv() and relation_openrv() > alone and add a missing_ok argument to try_heap_openrv() and > try_relation_openrv(). +1 for that, although the try_ prefix might be inappropriate naming now; how about cond_relation_openrv?

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Bruce Momjian
Andrew Dunstan wrote: > >> Further research shows that C89 explicitly dropped support for the old > >> K&R "=-" operator, so we probably *should* remove this in case it > >> introduces an unintended bug. > > Well, the point is if someone does use that, it isn't going to generate > > a pgindent erro

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 09:08 AM, Bruce Momjian wrote: Andrew Dunstan wrote: On 06/22/2011 08:35 AM, Andrew Dunstan wrote: On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this rep

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Jeff MacDonald
Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: > I assumed it was important to indicate if someone was looking at > per-database or per-cluster data, like pg_tablespace. The issue comes > up when I do admin training about the system tables. +1 I favor features that mak

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 9:14 AM, Josh Kupershmidt wrote: > On Tue, Jun 21, 2011 at 10:58 PM, Robert Haas wrote: >> Committed the main patch, and your regression tests. > > Hmph, looks like buildfarm members koi and jaguar are failing make check now: >  http://www.pgbuildfarm.org/cgi-bin/show_log.

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Josh Kupershmidt
On Tue, Jun 21, 2011 at 10:58 PM, Robert Haas wrote: > Committed the main patch, and your regression tests. Hmph, looks like buildfarm members koi and jaguar are failing make check now: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=koi&dt=2011-06-22%2008%3A06%3A00 due to a difference in s

Re: [HACKERS] [COMMITTERS] pgsql: Make the visibility map crash-safe.

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 6:55 AM, Heikki Linnakangas wrote: > On 22.06.2011 06:05, Robert Haas wrote: >> >> Second, when inserting, updating, or deleting >> a tuple, we can no longer get away with clearing the visibility map >> bit after releasing the lock on the corresponding heap page, because >>

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Greg Sabino Mullane's message of mi?? jun 22 03:24:34 UTC 2011: > > > > Hash: RIPEMD160 > > > > > Do we do enough to show which tables are db shared, e.g. pg_database? I > > > don't see any indication from psql \dS. Are our docs clear enough? > > > > I don

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Bruce Momjian
Andrew Dunstan wrote: > > > On 06/22/2011 08:35 AM, Andrew Dunstan wrote: > > > > > > On 06/22/2011 02:03 AM, David Christensen wrote: > >>> # Avoid bug that converts 'x =- 1' to 'x = -1' > >>> $source =~ s!=- !-= !g; > >> > >> I haven't looked at the shell script this replaces, but is t

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 08:35 AM, Andrew Dunstan wrote: On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct substitution pattern? (BTW, I'm not s

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 6:18 AM, Noah Misch wrote: > On Tue, Jun 21, 2011 at 11:11:41PM -0400, Robert Haas wrote: >> On Tue, Jun 21, 2011 at 11:04 PM, Tom Lane wrote: >> > Robert Haas writes: >> >> Some of the refactoring you've done here seems likely to break things, >> >> because you're basica

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct substitution pattern? (BTW, I'm not seeing the token =- anywhere except in the Ma

Re: [HACKERS] Hugetables question

2011-06-22 Thread Radosław Smogura
On Wed, 22 Jun 2011 14:24:17 +0300, Marti Raudsepp wrote: On Sun, Jun 19, 2011 at 12:56, Radosław Smogura wrote: I want to implement hugepages for shared memory Hi, Have you read this post by Tom Lane about the performance estimation and a proof-of-concept patch with hugepages? http://archiv

Re: [HACKERS] Latch implementation that wakes on postmaster death on both win32 and Unix

2011-06-22 Thread Peter Geoghegan
Attached patch addresses Fujii's more recent concerns. On 22 June 2011 04:54, Fujii Masao wrote: > +WaitLatch(volatile Latch *latch, int wakeEvents, long timeout) > +WaitLatchOrSocket(volatile Latch *latch, int wakeEvents, pgsocket > sock, long timeout) > > If 'wakeEvent' is zero, we cannot get

Re: [HACKERS] Hugetables question

2011-06-22 Thread Marti Raudsepp
On Sun, Jun 19, 2011 at 12:56, Radosław Smogura wrote: > I want to implement hugepages for shared memory Hi, Have you read this post by Tom Lane about the performance estimation and a proof-of-concept patch with hugepages? http://archives.postgresql.org/pgsql-hackers/2010-11/msg01842.php It's p

Re: [HACKERS] [COMMITTERS] pgsql: Make the visibility map crash-safe.

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 06:05, Robert Haas wrote: Second, when inserting, updating, or deleting a tuple, we can no longer get away with clearing the visibility map bit after releasing the lock on the corresponding heap page, because an intervening crash might leave the visibility map bit set and the page-l

[HACKERS] Re: [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Noah Misch
On Tue, Jun 21, 2011 at 11:11:41PM -0400, Robert Haas wrote: > On Tue, Jun 21, 2011 at 11:04 PM, Tom Lane wrote: > > Robert Haas writes: > >> Some of the refactoring you've done here seems likely to break things, > >> because you're basically making the relation locking happen later than > >> it

Re: [HACKERS] Coding style point: "const" in function parameter declarations

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 02:58, Dan Ports wrote: On Tue, Jun 21, 2011 at 06:51:20PM -0400, Tom Lane wrote: I find this to be poor style, and would like to see if there's any support for getting rid of the "const" keywords. I'm in favor of removing them too. Ok, I've removed all the useless const qualif

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 07:58, Dan Ports wrote: I was looking at ExecSeqScan today and noticed that it invokes PredicateLockRelation each time it's called, i.e. for each tuple returned. Any reason we shouldn't skip that call if rs_relpredicatelocked is already set, as in the attached patch? That would sav