Re: Identity columns should own only one sequence

2019-05-02 Thread Laurenz Albe
On Thu, 2019-05-02 at 22:43 +0200, Peter Eisentraut wrote: > On 2019-04-29 18:28, Laurenz Albe wrote: > > I still think thatthat there is merit to Michael's idea of removing > > sequence "ownership" (which is just a dependency) when the DEFAULT > > on the column is dropped, but this approach is pos

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-05-02 Thread John Naylor
On Thu, May 2, 2019 at 4:57 PM Amit Kapila wrote: > > On Thu, May 2, 2019 at 12:39 PM John Naylor > wrote: > > > Okay, I have updated the patch to incorporate your changes and call > relcache invalidation at required places. I have updated comments at a > few places as well. The summarization

Re: POC: Cleaning up orphaned files using undo logs

2019-05-02 Thread Dilip Kumar
On Thu, May 2, 2019 at 7:00 PM Robert Haas wrote: > > On Thu, May 2, 2019 at 5:32 AM Dilip Kumar wrote: > > Yeah, at least in this patch it looks silly. Actually, I added that > > index to make the qsort stable when execute_undo_action sorts them in > > block order. But, as part of this patch w

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-02 16:54:11 -0400, Tom Lane wrote: >> I just finished a successful run of the core regression tests with CCA. >> Given the calendar, I think that's about as much CCA testing as I should >> do personally. I'll make a cleanup pass on this patch and try to get it >

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-02 16:54:11 -0400, Tom Lane wrote: >> How do you feel about the other patch to rejigger the order of operations >> in CommandCounterIncrement? I think that's a bug, but it's probably >> noncritical for most people. What I'm leaning towards for that one is >> wa

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 16:54:11 -0400, Tom Lane wrote: > I just finished a successful run of the core regression tests with CCA. > Given the calendar, I think that's about as much CCA testing as I should > do personally. I'll make a cleanup pass on this patch and try to get it > pushed within a few ho

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-02 12:59:55 -0400, Tom Lane wrote: >> One interesting thing that turns up in check-world is that if wal_level >> is minimal, we have to manually force an XID to be assigned, else >> reindexing pg_class fails with "cannot commit a transaction that deleted >> files

Re: Heap lock levels for REINDEX INDEX CONCURRENTLY not quite right?

2019-05-02 Thread Peter Eisentraut
On 2019-05-02 16:33, Andres Freund wrote: > And RangeVarCallbackForReindexIndex() pretty clearly sets it *heapOid: > >* Lock level here should match reindex_index() heap lock. If > the OID >* isn't valid, it means the index as concurrently dropped, > which is >

Re: Heap lock levels for REINDEX INDEX CONCURRENTLY not quite right?

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 22:39:15 +0200, Peter Eisentraut wrote: > On 2019-05-02 16:33, Andres Freund wrote: > > And RangeVarCallbackForReindexIndex() pretty clearly sets it *heapOid: > > > > * Lock level here should match reindex_index() heap lock. If > > the OID > > * isn't

Re: Identity columns should own only one sequence

2019-05-02 Thread Peter Eisentraut
On 2019-04-29 18:28, Laurenz Albe wrote: > I still think thatthat there is merit to Michael's idea of removing > sequence "ownership" (which is just a dependency) when the DEFAULT > on the column is dropped, but this approach is possibly cleaner. I think the proper way to address this would be to

Re: New vacuum option to do only freezing

2019-05-02 Thread Alvaro Herrera
Pushed. I added one comment to explain xmin_frozen also, which otherwise seemed a bit mysterious. I did not backpatch, though, so 9.6-11 are a bit different, but I'm not sure it's a good idea at this point, though it should be pretty innocuous. -- Álvaro Herrerahttps://www.2ndQu

Re: ERROR: failed to add item to the index page

2019-05-02 Thread Andreas Joseph Krogh
På torsdag 02. mai 2019 kl. 21:38:02, skrev Peter Geoghegan : > Pushed, though final version does the test a little differently. It > adds the required heap TID space to itupsz, rather than subtracting it > from pgspc. This is actually representative of the underlying logic, > and avoids unsign

Re: ERROR: failed to add item to the index page

2019-05-02 Thread Peter Geoghegan
On Tue, Apr 30, 2019 at 6:28 PM Peter Geoghegan wrote: > Attached is a much more polished version of the same patch. I tried to > make clear how the "page full" test (the test that has been fixed to > take heap TID space for high key into account) is related to other > close-by code, such as the t

Re: How to estimate the shared memory size required for parallel scan?

2019-05-02 Thread Thomas Munro
On Thu, May 2, 2019 at 8:06 PM Thomas Munro wrote: > I was pinged off-list by someone who is working on a parallel-aware > FDW, who asked if I still had the test code I mentioned up-thread. > While digging that out, I couldn't resist hacking it a bit more until > it gave the right answers, only so

Re: pg_upgrade --clone error checking

2019-05-02 Thread Jeff Janes
On Thu, May 2, 2019 at 12:28 PM Alvaro Herrera wrote: > On 2019-May-02, Jeff Janes wrote: > > > > > When something is doomed to fail, we should report the failure as early > as > > feasibly detectable. > > I agree -- this check should be done before checking the database > contents. Maybe even b

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 12:59:55 -0400, Tom Lane wrote: > I wrote: > > Andres Freund writes: > >> On 2019-05-02 11:41:28 -0400, Tom Lane wrote: > >>> But don't we need to worry about resetting relfrozenxid? > > >> Indexes don't have that though? We couldn't do it for pg_class itself, > >> but that's n

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
I wrote: > Andres Freund writes: >> On 2019-05-02 11:41:28 -0400, Tom Lane wrote: >>> But don't we need to worry about resetting relfrozenxid? >> Indexes don't have that though? We couldn't do it for pg_class itself, >> but that's not a problem here. > Hmm. Again, that seems like the sort of as

Re: pgbench - add option to show actual builtin script code

2019-05-02 Thread Fabien COELHO
Now the patch is good now. The new status of this patch is: Ready for Committer Ok, thanks. -- Fabien.

Re: pg_upgrade --clone error checking

2019-05-02 Thread Alvaro Herrera
On 2019-May-02, Jeff Janes wrote: > I think the error message wording is OK, I think it should be thrown > earlier, before the "Creating dump of database schemas" (which can take a > long time), and preferably before either database is even started. So > ideally it would be something like: > >

Re: pg_upgrade --clone error checking

2019-05-02 Thread Jeff Janes
On Thu, May 2, 2019 at 11:57 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-05-01 22:10, Jeff Janes wrote: > > With the new pg_upgrade --clone, if we are going to end up throwing the > > error "file cloning not supported on this platform" (which seems to > > depend only o

Re: Attempt to consolidate reading of XLOG page

2019-05-02 Thread Antonin Houska
Antonin Houska wrote: > Alvaro Herrera wrote: > > > Not sure about the walsize; maybe it can be a member in XLogReadPos, and > > given to XLogReadInitPos()? (Maybe rename XLogReadPos as > > XLogReadContext or something like that, indicating it's not just the > > read position.) > > As pointed

Re: New vacuum option to do only freezing

2019-05-02 Thread Alvaro Herrera
On 2019-May-01, Andres Freund wrote: > Alvaro, could we perhaps clean this up a bit? This is pretty confusing > looking. I think this probably could just be changed to > > boolxmin_frozen = false; > > xid = HeapTupleHeaderGetXmin(tuple); > > if (xid == Froze

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Thu, May 2, 2019 at 10:28 AM Andres Freund wrote: > It'd be good if somebody could make a pass over the safety mechanisms in > heap_prepare_freeze_tuple(). I added them at some point, after a data > corrupting bug related to freezing, but they really were more intended > as a secondary layer of

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-02 11:41:28 -0400, Tom Lane wrote: >> But don't we need to worry about resetting relfrozenxid? > Indexes don't have that though? We couldn't do it for pg_class itself, > but that's not a problem here. Hmm. Again, that seems like the sort of assumption that cou

Re: pg_upgrade --clone error checking

2019-05-02 Thread Peter Eisentraut
On 2019-05-01 22:10, Jeff Janes wrote: > With the new pg_upgrade --clone, if we are going to end up throwing the > error "file cloning not supported on this platform" (which seems to > depend only on ifdefs) I think we should throw it first thing, before > any other checks are done and certainly be

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 11:41:28 -0400, Tom Lane wrote: > Andres Freund writes: > > ISTM that if we go down this path, we should split (not now, but either > > still in v12, or *early* in v13), the sets of indexes that are intended > > to a) not being used for catalog queries b) may be skipped for inde

Re: Why is infinite_recurse test suddenly failing?

2019-05-02 Thread Tom Lane
Andres Freund writes: > Hm, I just noticed: >'HEAD' => [ >'force_parallel_mode = > regress' > ] Oooh, I didn't see that. > on all those animals. So it's n

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > ISTM that if we go down this path, we should split (not now, but either > still in v12, or *early* in v13), the sets of indexes that are intended > to a) not being used for catalog queries b) may be skipped for index > insertions. It seems pretty likely that somebody will o

Re: Why is infinite_recurse test suddenly failing?

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 11:02:03 -0400, Tom Lane wrote: > In the past week, four different buildfarm members have shown > non-reproducing segfaults in the "select infinite_recurse()" > test case, rather than the expected detection of stack overrun > before we get to the point of a segfault. I was just

Re: New vacuum option to do only freezing

2019-05-02 Thread Andres Freund
On 2019-05-02 11:09:10 -0400, Robert Haas wrote: > If I understand correctly, your first proposal amounts to redefining > tups_vacuumed to count #2 rather than #1, and your second proposal > amounts to making tups_vacuumed count #1 + #2 rather than #1. I > suggest a third option: have two counters

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 10:49:00 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-01 22:01:53 -0400, Tom Lane wrote: > >> I think that argument is pretty pointless considering that "REINDEX TABLE > >> pg_class" does it this way, and that code is nearly old enough to > >> vote. > > > IMO th

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Tue, Apr 16, 2019 at 4:00 PM Tom Lane wrote: > I wrote: > > I'm thinking that we really need to upgrade vacuum's reporting totals > > so that it accounts in some more-honest way for pre-existing dead > > line pointers. The patch as it stands has made the reporting even more > > confusing, rath

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-01 00:43:34 -0400, Tom Lane wrote: >> ... What it's really currently doing at the >> moment of the deadlock is cleaning out its temporary schema after the >> client disconnected. > I'm inclined to remove the tests from the backbranches, once we've > committed a

Why is infinite_recurse test suddenly failing?

2019-05-02 Thread Tom Lane
In the past week, four different buildfarm members have shown non-reproducing segfaults in the "select infinite_recurse()" test case, rather than the expected detection of stack overrun before we get to the point of a segfault. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bonito&dt=2019

Re: pgbench - add option to show actual builtin script code

2019-05-02 Thread Ibrar Ahmed
Now the patch is good now. The new status of this patch is: Ready for Committer

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Andres Freund
Hi, On 2019-05-01 00:43:34 -0400, Tom Lane wrote: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=locust&dt=2019-05-01%2003%3A12%3A13 > > The relevant bit of log is > > 2019-05-01 05:24:47.527 CEST [97690:429] pg_regress/create_view LOG: > statement: DROP SCHEMA temp_view_test CASCAD

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-05-02 Thread Tom Lane
Andres Freund writes: > On 2019-05-01 22:01:53 -0400, Tom Lane wrote: >> I think that argument is pretty pointless considering that "REINDEX TABLE >> pg_class" does it this way, and that code is nearly old enough to >> vote. > IMO the reindex_relation() case isn't comparable. IMV it's the exact

Re: Heap lock levels for REINDEX INDEX CONCURRENTLY not quite right?

2019-05-02 Thread Andres Freund
On 2019-05-02 10:44:44 +0200, Peter Eisentraut wrote: > On 2019-04-30 17:17, Andres Freund wrote: > > indOid = RangeVarGetRelidExtended(indexRelation, > > > > concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock, > >

Re: New vacuum option to do only freezing

2019-05-02 Thread Andres Freund
Hi, On 2019-05-02 10:20:25 -0400, Robert Haas wrote: > On Tue, Apr 16, 2019 at 12:44 PM Tom Lane wrote: > > So after thinking about this a bit more ... > > 2. Act as if the tuple were still live, just as would've happened if the > > state didn't change till just after we looked instead of just be

Re: pgbench - add option to show actual builtin script code

2019-05-02 Thread Fabien COELHO
Hello, Patch looks good to me, and work fine on my machine. One minor observation is option 'list' mostly used to list the elements like "pgbench -b list" shows the available builtin scripts. Therefore we should use a word which seems to be more relevant like --show-script. Thanks for the r

Re: New vacuum option to do only freezing

2019-05-02 Thread Robert Haas
On Tue, Apr 16, 2019 at 12:44 PM Tom Lane wrote: > So after thinking about this a bit more ... > > ISTM that what we have here is a race condition (ie, tuple changed state > since heap_page_prune), and that ideally we want the code to resolve it > as if no race had happened. That is, either of th

Re: Fixing order of resowner cleanup in 12, for Windows

2019-05-02 Thread Tom Lane
Thomas Munro writes: > A while back I posted a patch[1] to change the order of resowner > cleanup so that DSM handles are released last. That's useful for the > error cleanup path on Windows, when a SharedFileSet is cleaned up (a > mechanism that's used by parallel CREATE INDEX and parallel hash

Inconsistent error message wording for REINDEX CONCURRENTLY

2019-05-02 Thread Tom Lane
In view of the REINDEX-on-pg_class kerfuffle that we're currently sorting through, I was very glad to see that the concurrent reindex code doesn't even try: regression=# reindex index concurrently pg_class_oid_index; psql: ERROR: concurrent reindex is not supported for catalog relations regressio

Re: walsender vs. XLogBackgroundFlush during shutdown

2019-05-02 Thread Simon Riggs
On Wed, 1 May 2019 at 01:28, Tomas Vondra wrote: > Now, this situation is apparently expected, because WalSndWaitForWal() > does this: > > /* > * If we're shutting down, trigger pending WAL to be written out, > * otherwise we'd possibly end up waiting for WAL that never gets >

Re: pgbench - add option to show actual builtin script code

2019-05-02 Thread Ibrar Ahmed
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Patch looks good to me, and work fine on my machine. One mino

Re: POC: Cleaning up orphaned files using undo logs

2019-05-02 Thread Robert Haas
On Thu, May 2, 2019 at 5:32 AM Dilip Kumar wrote: > Yeah, at least in this patch it looks silly. Actually, I added that > index to make the qsort stable when execute_undo_action sorts them in > block order. But, as part of this patch we don't have that processing > so either we can remove this s

Re: Bad canonicalization for dateranges with 'infinity' bounds

2019-05-02 Thread Laurenz Albe
I wrote: > I propose the attached patch which fixes the problem. I forgot to attach the patch. Here it is. Yours, Laurenz Albe From 6bbad0acf3baae3a08d1f911b7017642c8a8afe9 Mon Sep 17 00:00:00 2001 From: Laurenz Albe Date: Thu, 2 May 2019 14:32:27 +0200 Subject: [PATCH] Don't canonicalize dater

Bad canonicalization for dateranges with 'infinity' bounds

2019-05-02 Thread Laurenz Albe
https://www.postgresql.org/docs/current/rangetypes.html#RANGETYPES-INFINITE has: Also, some element types have a notion of “infinity”, but that is just another value so far as the range type mechanisms are concerned. For example, in timestamp ranges, [today,] means the same thing as [toda

Re: walsender vs. XLogBackgroundFlush during shutdown

2019-05-02 Thread Tomas Vondra
On Wed, May 01, 2019 at 07:12:52PM +0200, Alexander Kukushkin wrote: Hi, On Wed, 1 May 2019 at 17:02, Tomas Vondra wrote: OK, so that seems like a separate issue, somewhat unrelated to the issue I reported. And I'm not sure it's a walsender issue - it seems it might be a psycopg2 issue in not

Fixing order of resowner cleanup in 12, for Windows

2019-05-02 Thread Thomas Munro
Hi all, A while back I posted a patch[1] to change the order of resowner cleanup so that DSM handles are released last. That's useful for the error cleanup path on Windows, when a SharedFileSet is cleaned up (a mechanism that's used by parallel CREATE INDEX and parallel hash join, for spilling fi

Re: speeding up planning with partitions

2019-05-02 Thread Amit Langote
On Wed, May 1, 2019 at 4:08 AM Tom Lane wrote: > OK, I tweaked that a bit and pushed both versions. Thank you. Regards, Amit

Re: POC: Cleaning up orphaned files using undo logs

2019-05-02 Thread Dilip Kumar
On Tue, Apr 30, 2019 at 8:44 PM Robert Haas wrote: > UndoRecInfo looks a bit silly, I think. Isn't index just the index of > this entry in the array? You can always figure that out by ptr - > array_base. Instead of having UndoRecPtr urp in this structure, how > about adding that to UnpackedUndo

Re: Volatile function weirdness

2019-05-02 Thread Julien Rouhaud
On Thu, May 2, 2019 at 11:05 AM Vik Fearing wrote: > > Why do these two queries produce different results? See https://www.postgresql.org/message-id/30382.1537932...@sss.pgh.pa.us

Volatile function weirdness

2019-05-02 Thread Vik Fearing
Why do these two queries produce different results? vik=# select random(), random(), random() from generate_series(1, 5); random | random | random ---+---+--- 0.47517032455653 | 0.631991865579039 | 0.985628996044397 0.3

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-05-02 Thread Amit Kapila
On Thu, May 2, 2019 at 12:39 PM John Naylor wrote: > > On Thu, May 2, 2019 at 2:31 PM Amit Kapila wrote: > > @@ -776,7 +776,10 @@ fsm_extend(Relation rel, BlockNumber fsm_nblocks) > > if ((rel->rd_smgr->smgr_fsm_nblocks == 0 || > > rel->rd_smgr->smgr_fsm_nblocks == InvalidBlockNumber) && > >

Re: Heap lock levels for REINDEX INDEX CONCURRENTLY not quite right?

2019-05-02 Thread Peter Eisentraut
On 2019-04-30 17:17, Andres Freund wrote: > indOid = RangeVarGetRelidExtended(indexRelation, > > concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock, >

Re: How to estimate the shared memory size required for parallel scan?

2019-05-02 Thread Thomas Munro
On Mon, Aug 20, 2018 at 11:09 AM Thomas Munro wrote: > 2. Teach your GetForeignPath function to do something like this: > [blah blah blah] I was pinged off-list by someone who is working on a parallel-aware FDW, who asked if I still had the test code I mentioned up-thread. While digging that out

Re: Failure in contrib test _int on loach

2019-05-02 Thread Heikki Linnakangas
(resending, previous attempt didn't make it to pgsql-hackers) On 29/04/2019 16:16, Anastasia Lubennikova wrote: In previous emails, I have sent two patches with test and bugfix (see attached). After Heikki shared his concerns, I've rechecked the algorithm and haven't found any potential error. S

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-05-02 Thread John Naylor
On Thu, May 2, 2019 at 2:31 PM Amit Kapila wrote: > @@ -776,7 +776,10 @@ fsm_extend(Relation rel, BlockNumber fsm_nblocks) > if ((rel->rd_smgr->smgr_fsm_nblocks == 0 || > rel->rd_smgr->smgr_fsm_nblocks == InvalidBlockNumber) && > !smgrexists(rel->rd_smgr, FSM_FORKNUM)) > + { > smgrcreate(r