Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Fri, May 10, 2013 at 2:47 AM, Peter Eisentraut wrote: > On Thu, 2013-05-09 at 10:11 -0400, Peter Eisentraut wrote: >> On 5/9/13 3:25 AM, Dave Page wrote: >> > BTW - it's always worked fine for us on 64 bit machines with the past >> > few major releases of both PG and Python - are you saying tha

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Simon Riggs
On 9 May 2013 23:13, Greg Stark wrote: > On Thu, May 9, 2013 at 10:45 PM, Simon Riggs wrote: >> On 9 May 2013 22:39, Tom Lane wrote: >>> Simon Riggs writes: If the current WAL record is corrupt and the next WAL record is in every way valid, we can potentially continue. >>> >>> That se

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 09:22:55PM -0400, Evan D. Hoffman wrote: > Hmm... the database itself predates me, so I can't say for sure what > encoding it was created with, but when I did a "pg_dumpall -s" it > showed every database in the cluster uses "SET client_encoding = > 'UTF8';" OK, that's good

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On Thu, 2013-05-09 at 10:11 -0400, Peter Eisentraut wrote: > On 5/9/13 3:25 AM, Dave Page wrote: > > BTW - it's always worked fine for us on 64 bit machines with the past > > few major releases of both PG and Python - are you saying that's pure > > chance? > > It works because ActiveState Python h

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Robins Tharakan
One workaround is to use DROP COLLATION IF EXISTS, that conveys the message without UTF8 in the message string. But three other tests use ALTER COLLATION and I see no way but to comment / disable them. Currently have them disabled (with comments as to what they do, and why they're disabled). This

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Hmm... the database itself predates me, so I can't say for sure what encoding it was created with, but when I did a "pg_dumpall -s" it showed every database in the cluster uses "SET client_encoding = 'UTF8';" On Thu, May 9, 2013 at 7:25 PM, Bruce Momjian wrote: > On Thu, May 9, 2013 at 05:41:39PM

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 05:41:39PM -0400, Evan D. Hoffman wrote: > I believe the history of this cluster is that it started on 9.0 and > was upgraded to 9.1 via pg_upgrade. The instance I'm working on was > created as a streaming replica, then I broke the replication to make > it a standalone master

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
On Thu, 2013-05-09 at 23:13 +0100, Greg Stark wrote: > However it is possible to reduce the window... Sounds reasonable. It's fairly limited though -- the window is already a checkpoint (typically 5-30 minutes), and we'd bring that down an order of magnitude (10s). I speculate that, if it got cor

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 06:05:14PM -0400, Alvaro Herrera wrote: > Greg Stark escribió: > > On Thu, May 9, 2013 at 10:11 PM, Tom Lane wrote: > > > In any case, it seems like pg_upgrade ought to have a strategy for > > > dealing with tables acquiring toast tables like this, > > > > Acquiring toast

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
On Thu, 2013-05-09 at 14:28 -0500, Jim Nasby wrote: > What about moving some critical data from the beginning of the WAL > record to the end? That would make it easier to detect that we don't > have a complete record. It wouldn't necessarily replace the CRC > though, so maybe that's not good enough

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Greg Stark
On Thu, May 9, 2013 at 10:45 PM, Simon Riggs wrote: > On 9 May 2013 22:39, Tom Lane wrote: >> Simon Riggs writes: >>> If the current WAL record is corrupt and the next WAL record is in >>> every way valid, we can potentially continue. >> >> That seems like a seriously bad idea. > > I agree. But

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Alvaro Herrera
Greg Stark escribió: > On Thu, May 9, 2013 at 10:11 PM, Tom Lane wrote: > > In any case, it seems like pg_upgrade ought to have a strategy for > > dealing with tables acquiring toast tables like this, > > Acquiring toast tables seems pretty trivial to deal with. *Losing* a > toast table might be

[HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Greg Stark
On Thu, May 9, 2013 at 10:11 PM, Tom Lane wrote: > In any case, it seems like pg_upgrade ought to have a strategy for > dealing with tables acquiring toast tables like this, Acquiring toast tables seems pretty trivial to deal with. *Losing* a toast table might be a bit more involved... Neither s

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Simon Riggs
On 9 May 2013 22:39, Tom Lane wrote: > Simon Riggs writes: >> If the current WAL record is corrupt and the next WAL record is in >> every way valid, we can potentially continue. > > That seems like a seriously bad idea. I agree. But if you knew that were true, is stopping a better idea? Any cor

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
I believe the history of this cluster is that it started on 9.0 and was upgraded to 9.1 via pg_upgrade. The instance I'm working on was created as a streaming replica, then I broke the replication to make it a standalone master specifically for testing pg_upgrade to 9.2. On May 9, 2013, at 5:2

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Tom Lane
Simon Riggs writes: > If the current WAL record is corrupt and the next WAL record is in > every way valid, we can potentially continue. That seems like a seriously bad idea. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 10:06 PM, Peter Eisentraut wrote: > On 5/9/13 10:52 AM, Dave Page wrote: >> On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut wrote: >>> On 5/9/13 3:09 AM, Dave Page wrote: I assume you'll fix the configure script so it actually errors out if plpython cannot be bui

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 05:11:43PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > OK, that's progress. Having received the table schema privately via > > email, I see several 'character varying(40)' fields in the schema. So > > the question is how was this table able to get away without a TO

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Simon Riggs
On 9 May 2013 20:28, Jim Nasby wrote: >> Unfortunately, it seems that doing any kind of validation to determine >> that we have a valid end-of-the-WAL inherently requires some kind of >> separate durable write somewhere. It would be a tiny amount of data (an >> LSN and maybe some extra crosscheck

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Tom Lane
Bruce Momjian writes: > OK, that's progress. Having received the table schema privately via > email, I see several 'character varying(40)' fields in the schema. So > the question is how was this table able to get away without a TOAST > table in 9.1, while 9.2 created one for an empty table? Ide

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 10:52 AM, Dave Page wrote: > On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut wrote: >> On 5/9/13 3:09 AM, Dave Page wrote: >>> I assume you'll fix the configure script so it actually errors out if >>> plpython cannot be built, but is explicitly requested? >> >> That is ancient behavior,

[HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 04:21:05PM -0400, Evan D. Hoffman wrote: > Looks like your guess was correct: > > [ehoffman@dev-db2 ~]$ psql -Upostgres db -p 5433 > psql (9.2.4) > Type "help" for help. > > db=# SELECT oid, relname, reltoastrelid, reltoastidxid FROM pg_class > db-# WHERE r

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Here it is with the interesting field names mangled for paranoia reasons: db=# \d+ bpm.setupinfo; Table "bpm.setupinfo" Column| Type | Modifiers | Storage | Stats target | Description -

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Tom Lane
"Evan D. Hoffman" writes: > Looks like your guess was correct: Could we see the full schema (eg psql \d+) for setupinfo? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Looks like your guess was correct: [ehoffman@dev-db2 ~]$ psql -Upostgres db -p 5433 psql (9.2.4) Type "help" for help. db=# SELECT oid, relname, reltoastrelid, reltoastidxid FROM pg_class db-# WHERE reltoastrelid = 299749; oid | relname | reltoastrelid | reltoastidxid ---

[HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 03:52:42PM -0400, Evan D. Hoffman wrote: > That's correct. Here's what substitutionlist_pkey looks like in the new > cluster. From this, it looks like it's actually correct (the oid for > substitutionlist_pkey is correct) but pg_upgrade thinks it's wrong and dies. > I'll

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
That's correct. Here's what substitutionlist_pkey looks like in the new cluster. From this, it looks like it's actually correct (the oid for substitutionlist_pkey is correct) but pg_upgrade thinks it's wrong and dies. I'll look for the logs you requested and send them separately db=# SELECT rel

[HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 03:23:20PM -0400, Evan D. Hoffman wrote: > I just did the whole process over from the beginning. here's the full output: > > Copying user relation files > /var/lib/pgsql/9.1/data/base/16406/3016054 > Mismatch of relation OID in database "db": old OID 29

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jim Nasby
On 5/8/13 7:34 PM, Jeff Davis wrote: On Wed, 2013-05-08 at 17:56 -0500, Jim Nasby wrote: Apologies if this is a stupid question, but is this mostly an issue due to torn pages? IOW, if we had a way to ensure we never see torn pages, would that mean an invalid CRC on a WAL page indicated there rea

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
I just did the whole process over from the beginning. here's the full output: -bash-4.1$ date ; time /usr/pgsql-9.2/bin/pg_upgrade -b /usr/pgsql-9.1/bin/ -B /usr/pgsql-9.2/bin/ -d /var/lib/pgsql/9.1/data/ -D /var/lib/pgsql/9.2/data/ -p 50432 -P 50433 ; date Thu May 9 14:31:07 EDT 2013 Performing

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Darren Duncan
On 2013.05.09 11:22 AM, Alvaro Herrera wrote: Darren Duncan wrote: On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric.

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Misa Simic
2013/5/9 Pavel Stehule > Hello > > I am writing a article about 9.3. I found so event trigger functions is > not complete. We have only pg_event_trigger_dropped_objects() function. It > looks really strange and asymmetric. Can we implement similar function for > CREATE as minimum to 9.3? > > I am

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Alvaro Herrera
Darren Duncan wrote: > On 2013.05.09 10:40 AM, Pavel Stehule wrote: > >I am writing a article about 9.3. I found so event trigger functions is not > >complete. We have only pg_event_trigger_dropped_objects() function. It looks > >really strange and asymmetric. Can we implement similar function for

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Pavel Stehule
2013/5/9 Dimitri Fontaine > Darren Duncan writes: > > On 2013.05.09 10:40 AM, Pavel Stehule wrote: > >> I am expecting so this function should not be too complex - and can be > moved to > >> contrib maybe (if it is too late now). > > My understanding is that it's too late even for contrib now. >

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Dimitri Fontaine
Darren Duncan writes: > On 2013.05.09 10:40 AM, Pavel Stehule wrote: >> I am expecting so this function should not be too complex - and can be moved >> to >> contrib maybe (if it is too late now). My understanding is that it's too late even for contrib now. > Really, the touted new event trigge

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Darren Duncan
On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE as minimum to 9.3? I a

[HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Pavel Stehule
Hello I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE as minimum to 9.3? I am expecting so this function should not

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-05-09 Thread Robert Haas
On Sat, May 4, 2013 at 3:59 PM, Tom Lane wrote: > Given the lack of any good alternative proposals, I think we should > press ahead with this approach. We still need doc updates and fixes > for the affected contrib module(s), though. Also, in view of point 2, > it seems like the extensions code

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut wrote: > On 5/9/13 3:09 AM, Dave Page wrote: >> I assume you'll fix the configure script so it actually errors out if >> plpython cannot be built, but is explicitly requested? > > That is ancient behavior, which I'm not fond of, but now is not the t

[HACKERS] Re: [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 10:20:12AM -0400, Evan D. Hoffman wrote: > >From the 9.1 cluster (port 5432): > > > db=# SELECT relname, relfilenode, relkind from pg_class where oid = 2938685; > relname| relfilenode | relkind > ---+-+- > substitut

Re: [HACKERS] [GENERAL] pg_upgrade fails, "mismatch of relation OID" - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
>From the 9.1 cluster (port 5432): db=# SELECT relname, relfilenode, relkind from pg_class where oid = 2938685; relname| relfilenode | relkind ---+-+- substitutionlist_pkey |21446253 | i (1 row) db=# >From the 9.2 cluster (port 5433)

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 3:25 AM, Dave Page wrote: > BTW - it's always worked fine for us on 64 bit machines with the past > few major releases of both PG and Python - are you saying that's pure > chance? It works because ActiveState Python has PIC inside a static library. But we have no straightforward way of k

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 3:09 AM, Dave Page wrote: > I assume you'll fix the configure script so it actually errors out if > plpython cannot be built, but is explicitly requested? That is ancient behavior, which I'm not fond of, but now is not the time to change that. -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] Graph datatype addition

2013-05-09 Thread Merlin Moncure
On Wed, May 8, 2013 at 2:16 PM, Atri Sharma wrote: >> >> Your second drawing didn't really make any sense to me. :( >> >> I do think it would be most productive to focus on what the API for dealing >> with graph data would look like before trying to handle the storage aspect. >> The storage is pot

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Amit Kapila
On Thursday, May 09, 2013 2:14 PM Kyotaro HORIGUCHI wrote: > With printing some additinal logs, the situation should be more > clear.. > > It seems that Sby-B failes to promote to TLI= 2; nevertheless the > history file for TLI = 2 is somehow sent to sby-C. So sby-B > remains on TLI=1 but sby-C s

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Michael Paquier
On Thu, May 9, 2013 at 4:19 PM, Fabien COELHO wrote: > However, it is not clear whether these tests run automatically or only if > they are explicitely called. The README seems to suggest that it is the > later. If so, maybe having them invoked automatically if possible would > ensure that they

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-09 Thread Michael Paquier
On Thu, May 9, 2013 at 9:12 AM, David Fetter wrote: > On Wed, May 08, 2013 at 06:08:28PM -0500, Jim Nasby wrote: > > I believe that makes it significantly harder for them to actually > > contribute code back that doesn't give them a business advantage, as > > well as making it harder to justify h

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Kyotaro HORIGUCHI
With printing some additinal logs, the situation should be more clear.. It seems that Sby-B failes to promote to TLI= 2; nevertheless the history file for TLI = 2 is somehow sent to sby-C. So sby-B remains on TLI=1 but sby-C solely switches onto TLI=2. # Come to think of this, I suspect that the

Re: [HACKERS] Add regression tests for ROLE (USER)

2013-05-09 Thread Fabien COELHO
This updated version works for me and addresses previous comments. I think that such tests are definitely valuable, especially as many corner cases which must trigger errors are covered. I recommend to apply it. Please find an updated patch as per comments on Commitfest (comments replicated

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Kyotaro HORIGUCHI
Hello, > I think it can so happen that last checkpoint is with old timeline and there > are operations with new timeline which might have caused the problem Heikki > has seen. I suppose to have seen that. After adding an SQL command to modify the DB on standby-B after passive(propagated?) promot

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 8:09 AM, Dave Page wrote: > On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut wrote: >> On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: >>> It's failing on Linux. Even worse, it configures fine and then builds >>> without error. There is a message spewed out by configure

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Fabien COELHO
Maybe some tests could be applied under some condition, say a given locale is indeed available, but ISTM that it would require to change the test infrastructure in a portable way to add such feature. I just noticed that there is a "src/test/locale" directory with some stuff about german, gre

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut wrote: > On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: >> It's failing on Linux. Even worse, it configures fine and then builds >> without error. There is a message spewed out by configure, but it >> doesn't contain the words warning or error.