Re: [HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-29 Thread Andres Freund
On 2013-05-29 23:01:31 -0400, Robert Haas wrote: > On Wed, May 29, 2013 at 9:57 AM, Andres Freund wrote: > >> Thought about that, but given that 9.3's visibilitymap_set already will > >> already FPI heap pages I concluded it wouldn't really be an improvement > >> since it's only one ||log_heap_pag

Re: [HACKERS] MVCC catalog access

2013-05-29 Thread Michael Paquier
On Tue, May 28, 2013 at 10:39 PM, Robert Haas wrote: > IMHO, we should press forward with this approach. Considering that > these are pretty extreme test cases, I'm inclined to view the > performance loss as acceptable. We've never really viewed DDL as > something that needs to be micro-optimiz

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-29 Thread Jeff Davis
On Wed, 2013-05-29 at 22:46 -0400, Robert Haas wrote: > Again independently of Josh's proposal, we could eliminate > PD_ALL_VISIBLE. This would require either surrendering the > optimization whereby sequential scans can skip visibility checks on > individual tuples within the page, or referring to

Re: [HACKERS] Running pgindent

2013-05-29 Thread Bruce Momjian
On Wed, May 29, 2013 at 10:08:10PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Wow, uh, yeah, I guess we could do that. I will await more feedback. > > Please don't. I'm already rather concerned by this one. It looks like > there's a rule to pull a line in to mee

Re: [HACKERS] units in postgresql.conf comments

2013-05-29 Thread Bruce Momjian
On Wed, May 29, 2013 at 09:59:10PM -0400, Peter Eisentraut wrote: > I think these sort of entries don't make much sense: > > #wal_sender_timeout = 60s # in milliseconds; 0 disables > > I think we should remove units from the comments when it's clear from > the name or the default value that

Re: [HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-29 Thread Robert Haas
On Wed, May 29, 2013 at 9:57 AM, Andres Freund wrote: >> Thought about that, but given that 9.3's visibilitymap_set already will >> already FPI heap pages I concluded it wouldn't really be an improvement >> since it's only one ||log_heap_page or so there. Not sure what's >> better. Will write the

[HACKERS] Behavior of a pg_trgm index for 2 (or < 3) character LIKE queries

2013-05-29 Thread Amit Langote
Hello, I have been trying to understand how pg_trgm works. As part of that, I was looking at gin_extract_query_trgm(), which I think, extracts trigrams from a search query string. So, I debugged for 3 cases: 1) column_name LIKE '%緊急%' in this case, inside gin_extract_query_trgm(), after a call t

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-29 Thread Robert Haas
On Wed, May 29, 2013 at 1:11 PM, Jeff Davis wrote: > On Tue, 2013-05-28 at 19:51 -0400, Robert Haas wrote: >> > If we just wanted to reduce read cost, why not just take a simpler >> > approach and give the visibility map a "isfrozen" bit? Then we'd know >> > which pages didn't need rescanning wit

Re: [HACKERS] Running pgindent

2013-05-29 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > Wow, uh, yeah, I guess we could do that. I will await more feedback. Please don't. I'm already rather concerned by this one. It looks like there's a rule to pull a line in to meet the max-column requirement even when that makes things line up 'funny',

[HACKERS] units in postgresql.conf comments

2013-05-29 Thread Peter Eisentraut
I think these sort of entries don't make much sense: #wal_sender_timeout = 60s # in milliseconds; 0 disables I think we should remove units from the comments when it's clear from the name or the default value that time units are accepted. -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] XLogInsert scaling, revisited

2013-05-29 Thread Ants Aasma
On Wed, May 29, 2013 at 8:40 PM, Heikki Linnakangas wrote: > Thanks for asking :-). I've been fixing bitrot throughout the winter. I just > started cleaning it up again last week, and also continued with performance > testing. > > Unfortunately I lost the 8-core box I used earlier to test this, to

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/29/2013 03:31 PM, Joe Conway wrote: > On 05/29/2013 07:43 AM, Andres Freund wrote: > Couldn't ALTER EXTENSION ... ADD ...; be brought up to > speed to support this? >> Sounds better to me than manually fiddling with pg_depend... We >> c

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/29/2013 07:43 AM, Andres Freund wrote: > On 2013-05-29 07:35:42 -0700, Joe Conway wrote: >> On 05/29/2013 05:52 AM, Dimitri Fontaine wrote: >>> Andres Freund writes: On 2013-05-29 09:30:43 +0200, Dimitri Fontaine wrote: >> 2) How should

Re: [HACKERS] Running pgindent

2013-05-29 Thread Bruce Momjian
On Wed, May 29, 2013 at 05:56:32PM -0400, Alvaro Herrera wrote: > Bruce Momjian escribió: > > > Done. This was the first run of the Perl-based pgindent script. There > > was a lot of code churn in this run as the paragraphs are slightly > > wider. Also, I saw some outdenting of long lines, rath

Re: [HACKERS] streaming replication, "frozen snapshot backup on it" and missing relfile (postgres 9.2.3 on xfs + LVM)

2013-05-29 Thread David Powers
It's another possibility, but I think it's still somewhat remote given how long we've been using this method with this code. It's sadly hard to test because taking the full backup without the hard linking is fairly expensive (the databases comprise multiple terabytes). As a possibly unsatisfying

Re: [HACKERS] Running pgindent

2013-05-29 Thread Alvaro Herrera
Bruce Momjian escribió: > Done. This was the first run of the Perl-based pgindent script. There > was a lot of code churn in this run as the paragraphs are slightly > wider. Also, I saw some outdenting of long lines, rather than allowing > them to go past 80 characters, but it seemed minimal.

Re: [HACKERS] Running pgindent

2013-05-29 Thread Bruce Momjian
On Tue, May 28, 2013 at 09:56:03AM -0400, Bruce Momjian wrote: > On Tue, May 28, 2013 at 09:49:32AM -0400, Magnus Hagander wrote: > > On Tue, May 28, 2013 at 9:48 AM, Robert Haas wrote: > > > On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian wrote: > > >> On Wed, May 22, 2013 at 01:52:28PM -0400, Br

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Clark C. Evans
On Wed, May 29, 2013, at 10:08 AM, Stephen Frost wrote: > This capability might well come with a real way to have per-database > roles in general, which has been asked for quite often as well. You > would then be able to have an 'auditor' role in each database and have > them actually be different

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-29 Thread Daniel Farina
On Mon, May 27, 2013 at 9:41 AM, Simon Riggs wrote: > On 27 May 2013 15:36, Tom Lane wrote: >> Bruce Momjian writes: >>> On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote: That said, many discussions and ideas do get shut down, perhaps too early, because of pg_upgrade consi

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Clark C. Evans
On Wed, May 29, 2013, at 09:45 AM, Stephen Frost wrote: > * Albe Laurenz (laurenz.a...@wien.gv.at) wrote: > > Maybe the db_user_namespace parameter can help: > > http://www.postgresql.org/docs/9.2/static/runtime-config-connection.html#GUC-DB-USER-NAMESPACE > > I doubt it and I wouldn't encourage a

Re: [HACKERS] pg_stat_replication when standby is unreachable

2013-05-29 Thread Abhishek Rai
I looked a bit more into the code and it appears to me that the following are true: - A separate wal sender process is created on the primary side for each connected standby. - The wal sender process terminates (walsender.c / WalSndLoop) when there is an error to write to the standby's socket. - I

Re: [HACKERS] Unsigned integer types

2013-05-29 Thread Fabien COELHO
I agree that extensions are undermarketed. Although pgxn is a good step, I could not find it from "postgresql.org":-( I propose to not integrate the unsigned types into existing promotion hierarchy, and behave just like gcc would with -Werror: require explicit cast. Between them, the unsigned

Re: [HACKERS] Unsigned integer types

2013-05-29 Thread Hannu Krosing
On 05/29/2013 11:33 AM, Maciej Gajewski wrote: > I will implement it as an extension then. > > My feeling is that PostgreSQL extensions tend to fall into obscurity. > As an ordinary user it took me really long time to find out that > interesting features are available in form of extensions; they ar

Re: [HACKERS] getting rid of freezing

2013-05-29 Thread Jeff Davis
On Tue, 2013-05-28 at 09:29 -0700, Josh Berkus wrote: > - it would prevent us from getting rid of allvisible, which has a > documented and known write overhead It would? I don't think these proposals are necessarily in conflict. It's not entirely clear to me how they fit together in detail, but it

Re: [HACKERS] getting rid of freezing

2013-05-29 Thread Jeff Davis
On Tue, 2013-05-28 at 19:51 -0400, Robert Haas wrote: > > If we just wanted to reduce read cost, why not just take a simpler > > approach and give the visibility map a "isfrozen" bit? Then we'd know > > which pages didn't need rescanning without nearly as much complexity. > > That would break pg_

Re: [HACKERS] visibilitymap_set and checksums

2013-05-29 Thread Jeff Davis
On Fri, 2013-05-24 at 22:16 +0100, Simon Riggs wrote: > I think its perfectly understandable. Robert, Jeff and I discussed > that for a while before we passed it. I'm still not happy with it, and > think its a pretty confusing section of code with multiple paths > through it, but I just can't see

Re: [HACKERS] pg_stat_replication when standby is unreachable

2013-05-29 Thread Abhishek Rai
On Wed, May 29, 2013 at 9:14 AM, Dimitri Fontaine wrote: > Abhishek Rai writes: > > SELECT * from pg_stat_replication(); > > > > I've noticed that when I terminate the standby (cleanly or through kill > > -9), the result of above function goes from 1 row to zero rows. The > result > > comes ba

Re: [HACKERS] pg_stat_replication when standby is unreachable

2013-05-29 Thread Abhishek Rai
On Wed, May 29, 2013 at 9:16 AM, Peter Eisentraut wrote: > On 5/28/13 9:42 PM, Abhishek Rai wrote: > > Detecting primary health is easy. But what is the best way to know if > > the standby is live? Since this is not a hot-standby, I cannot send > > queries to it. > > Then how do you define "liv

Re: [HACKERS] pg_stat_replication when standby is unreachable

2013-05-29 Thread Peter Eisentraut
On 5/28/13 9:42 PM, Abhishek Rai wrote: > Detecting primary health is easy. But what is the best way to know if > the standby is live? Since this is not a hot-standby, I cannot send > queries to it. Then how do you define "live" for your use case? > Currently, I'm sending the following query to

Re: [HACKERS] pg_stat_replication when standby is unreachable

2013-05-29 Thread Dimitri Fontaine
Abhishek Rai writes: > SELECT * from pg_stat_replication(); > > I've noticed that when I terminate the standby (cleanly or through kill > -9), the result of above function goes from 1 row to zero rows. The result > comes back to 1 row when the standby restarts and reconnects. I was > wondering

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions "make install"

2013-05-29 Thread Cédric Villemain
> I just took time to inspect our contribs, USE_PGXS is not supported by all > of them atm because of SHLIB_PREREQS (it used submake) I have a patch > pending here to fix that. Once all our contribs can build with USE_PGXS I > fix the VPATH. I've added 'most' of the patches to the commitfest... (I

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Alvaro Herrera
Robert Haas escribió: > On Tue, May 28, 2013 at 10:08 PM, Andres Freund > wrote: > > On 2013-05-28 21:26:49 -0400, Robert Haas wrote: > >> > I am all for adding a comment why this is safe though. We thought about > >> > it for some while before we were convinced... > >> > >> I'm fine with that,

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-29 Thread Peter Eisentraut
On 5/29/13 10:42 AM, Andres Freund wrote: > On 2013-05-29 10:36:07 -0400, Stephen Frost wrote: >> I *really* hope that the Linux kernel, and other, folks are smart enough >> to realize that they can't just re-use random blocks from an I/O device >> without cleaning it first. > > FWIW, posix' descr

Re: [HACKERS] [GENERAL] pg_upgrade -u

2013-05-29 Thread Bruce Momjian
On Wed, May 29, 2013 at 08:59:42AM -0400, Ray Stell wrote: > > [ moved to hacker ] > > > > Wow, I never realized other tools used -U for user, instead of -u. > > Should I change pg_upgrade to use -U for 9.4? I can keep supporting > > -u as an undocumented option. > > That would make for consistenc

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Andres Freund
On 2013-05-29 07:35:42 -0700, Joe Conway wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 05/29/2013 05:52 AM, Dimitri Fontaine wrote: > > Andres Freund writes: > >> On 2013-05-29 09:30:43 +0200, Dimitri Fontaine wrote: > 2) How should we handle already installed extensions, w

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-29 Thread Andres Freund
On 2013-05-29 10:36:07 -0400, Stephen Frost wrote: > * Peter Eisentraut (pete...@gmx.net) wrote: > > On 5/28/13 11:36 AM, Greg Smith wrote: > > > Outside of the run for performance testing, I think it would be good at > > > this point to validate that there is really a 16MB file full of zeroes > >

Re: [HACKERS] Patch to .gitignore

2013-05-29 Thread Dimitri Fontaine
Christopher Browne writes: > You could hide your own favorite patterns by putting this into your > ~/.gitignore that isn't part of the repo, configuring this globally, thus: > git config --global core.excludesfile '~/.gitignore' You can also put per-project setup in .git/info/exclude, works well.

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-29 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: > On 5/28/13 11:36 AM, Greg Smith wrote: > > Outside of the run for performance testing, I think it would be good at > > this point to validate that there is really a 16MB file full of zeroes > > resulting from these operations. I am not really concerned

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/29/2013 05:52 AM, Dimitri Fontaine wrote: > Andres Freund writes: >> On 2013-05-29 09:30:43 +0200, Dimitri Fontaine wrote: 2) How should we handle already installed extensions, which will still lack dependency records after this bugfix

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Stephen Frost
Clark, * Clark C. Evans (c...@clarkevans.com) wrote: > I apologize for posting to -hackers; it was probably the wrong list. I don't know about that.. It's a new feature request, not sure where else you'd email about it. That said, it's also a non-trivial thing to change and it would have to be

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-29 Thread Peter Eisentraut
On 5/28/13 11:36 AM, Greg Smith wrote: > Outside of the run for performance testing, I think it would be good at > this point to validate that there is really a 16MB file full of zeroes > resulting from these operations. I am not really concerned that > posix_fallocate might be slower in some case

Re: [HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-29 Thread Andres Freund
On 2013-05-29 03:56:38 +0200, Andres Freund wrote: > On 2013-05-28 21:36:17 -0400, Robert Haas wrote: > > On Tue, May 28, 2013 at 1:58 PM, Andres Freund > > wrote: > > > Guessing around I looked and noticed the following problematic pattern: > > > 1) A: wants to do an update, doesn't have enough

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Peter Eisentraut
On 5/28/13 8:00 PM, Andres Freund wrote: >> - Various procedural languages use the combination of TID and XMIN to >> > determine whether a function needs to be recompiled. > Hm. As previously said, I am less than convinced of those adhoc > mechanisms and I think this should get properly integrated

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Stephen Frost
* Albe Laurenz (laurenz.a...@wien.gv.at) wrote: > Maybe the db_user_namespace parameter can help: > http://www.postgresql.org/docs/9.2/static/runtime-config-connection.html#GUC-DB-USER-NAMESPACE I doubt it and I wouldn't encourage anyone to use it even if it happened to help in this situation..

Re: [HACKERS] [GENERAL] pg_upgrade -u

2013-05-29 Thread Peter Eisentraut
On 5/28/13 10:55 PM, Bruce Momjian wrote: > Wow, I never realized other tools used -U for user, instead of -u. > Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u > as an undocumented option. It seems to me that that option shouldn't be necessary anyway. pg_upgrade should so

Re: [HACKERS] [GENERAL] pg_upgrade -u

2013-05-29 Thread Ray Stell
On May 28, 2013, at 10:55 PM, Bruce Momjian wrote: > On Wed, May 22, 2013 at 03:05:57PM -0400, Ray Stell wrote: >>> However, if we pass these items into the scripts, we then force >>> these values to be used, even if the user wants to use a different >>> value. It is a balance between supplying

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Dimitri Fontaine
Andres Freund writes: > On 2013-05-29 09:30:43 +0200, Dimitri Fontaine wrote: >> > 2) How should we handle already installed extensions, which will still >> >lack dependency records after this bugfix? >> >> I don't really see any other way here than providing an upgrade script >> that will so

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Albe Laurenz
Clark C. Evans wrote: >>> I'd really love the ability to grant a *user* >>> role-based privileges database by database. >> >> The only cluster-wide role permissions are the options >> SUPERUSER, CREATEDB, CREATEROLE, INHERIT, >> LOGIN and REPLICATION. > > Incorrect; role-to-role membership (differ

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Robert Haas
On Tue, May 28, 2013 at 10:08 PM, Andres Freund wrote: > On 2013-05-28 21:26:49 -0400, Robert Haas wrote: >> On Tue, May 28, 2013 at 8:00 PM, Andres Freund >> wrote: >> > I only suggested MOVED_IN/OFF because I didn't remember >> > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unl

Re: [HACKERS] FIX: auto_explain docs

2013-05-29 Thread Robert Haas
On Tue, May 28, 2013 at 5:43 PM, Tomas Vondra wrote: > Hi, > > I've just noticed that this patch in 2012-01 commitfest > > https://commitfest.postgresql.org/action/patch_view?id=729 > > added log_timing option to auto_explain, but it never actually made it > to the docs. Attached is a patch for

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Clark C. Evans
On Wed, May 29, 2013, at 04:26 AM, Albe Laurenz wrote: > Clark C. Evans wrote: > > I'd really love the ability to grant a *user* > > role-based privileges database by database. > > The only cluster-wide role permissions are the options > SUPERUSER, CREATEDB, CREATEROLE, INHERIT, > LOGIN and REPLIC

Re: [HACKERS] Unsigned integer types

2013-05-29 Thread Robert Haas
On Wed, May 29, 2013 at 4:33 AM, Maciej Gajewski wrote: > I propose to not integrate the unsigned types into existing promotion > hierarchy, and behave just like gcc would with -Werror: require > explicit cast. Between them, the unsigned types would be automatically > converted up (uint2 > uint4 >

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Andres Freund
On 2013-05-29 09:30:43 +0200, Dimitri Fontaine wrote: > > 2) How should we handle already installed extensions, which will still > >lack dependency records after this bugfix? > > I don't really see any other way here than providing an upgrade script > that will somehow re-attach those objects,

Re: [HACKERS] Unsigned integer types

2013-05-29 Thread Maciej Gajewski
I will implement it as an extension then. My feeling is that PostgreSQL extensions tend to fall into obscurity. As an ordinary user it took me really long time to find out that interesting features are available in form of extensions; they are certainly under-marketed. But this is a topic for sepa

Re: [HACKERS] GRANT role_name TO role_name ON database_name

2013-05-29 Thread Albe Laurenz
Clark C. Evans wrote: > I'd really love the ability to grant a *user* > role-based privileges database by database. > > For background, I have several databases running > in a single cluster, one database per business unit. > Each database has the same core schema with the same > basic role permis

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-29 Thread Dimitri Fontaine
Joe Conway writes: > The attached one-liner seems to do the trick. It should probably be > backpatched to 9.1. Remaining questions: Thanks for the patch (and testing, etc, that it entails)! > 1) Are there other database object types, likely to be included in >extension scripts, that are also