Re: [HACKERS] Runtime Partition Pruning

2018-04-12 Thread Amit Langote
On 2018/04/13 14:48, Amit Langote wrote: > On 2018/04/13 14:38, Amit Langote wrote: >> About the specific relation_open(.., NoLock) under question, I think there >> might be a way to address this by opening the tables with the appropriate >> lock mode in partitioned_rels list in ExecLockNonleafAppe

Re: [HACKERS] Runtime Partition Pruning

2018-04-12 Thread Amit Langote
On 2018/04/13 14:38, Amit Langote wrote: > About the specific relation_open(.., NoLock) under question, I think there > might be a way to address this by opening the tables with the appropriate > lock mode in partitioned_rels list in ExecLockNonleafAppendTables That may have sounded a bit confusin

Re: [HACKERS] Runtime Partition Pruning

2018-04-12 Thread Amit Langote
On 2018/04/13 1:57, Robert Haas wrote: >> It might be possible to do something better in each module by keeping >> an array indexed by RTI which have each entry NULL initially then on >> first relation_open set the element in the array to that pointer. > > I'm not sure that makes a lot of sense in

Re: MinIndexTupleSize seems slightly wrong

2018-04-12 Thread Kyotaro HORIGUCHI
At Thu, 12 Apr 2018 17:26:33 -0700, Peter Geoghegan wrote in > itup.h says of MinIndexTupleSize/MaxIndexTuplesPerPage: > > /* > * MaxIndexTuplesPerPage is an upper bound on the number of tuples that can > * fit on one index page. An index tuple must have either data or a null > * bitmap, so

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Kyotaro HORIGUCHI
At Fri, 13 Apr 2018 08:31:02 +0530, Amit Kapila wrote in > On Fri, Apr 13, 2018 at 6:59 AM, Michael Paquier wrote: > > On Thu, Apr 12, 2018 at 02:55:53PM -0400, Robert Haas wrote: > >> I think it may actually be confusing. If you run pg_ctl reload and it > >> reports that the value has changed

Overcoming SELECT ... FOR UPDATE permission restrictions

2018-04-12 Thread Alexander Lakhin
Hello hackers, Can you please explain, is this a bug or intended behaviour? Running as non-privileged user: postgres=> SELECT datid, datname FROM pg_stat_database FOR UPDATE; ERROR: permission denied for view pg_stat_database (SQLState: 42501) But: postgres=> CREATE VIEW pgsd AS SELECT * FROM

Re: Make description of heap records more talkative for flags

2018-04-12 Thread Michael Paquier
On Thu, Apr 12, 2018 at 08:49:03PM -0700, Andres Freund wrote: > OTOH, that also kinda bloats the output noticeably... I'm somewhat > inclined to just put the hex value or such there? That would do as well for me. -- Michael signature.asc Description: PGP signature

Re: Make description of heap records more talkative for flags

2018-04-12 Thread Andres Freund
On 2018-04-13 12:47:34 +0900, Michael Paquier wrote: > Hi all, > > I was just playing with the WAL consistency issue with rows moved across > partitions when I noticed that heapdesc.c is not really talkative about > the different flag records set. > > What about something like the patch attached?

Make description of heap records more talkative for flags

2018-04-12 Thread Michael Paquier
Hi all, I was just playing with the WAL consistency issue with rows moved across partitions when I noticed that heapdesc.c is not really talkative about the different flag records set. What about something like the patch attached? I found that useful for debugging. (One comment of heapam_xlog.h

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread amul sul
Will look into this, thanks. Regards, Amul Sent from a mobile device. Please excuse brevity and tpyos. On Fri, Apr 13, 2018, 9:06 AM Andres Freund wrote: > On 2018-04-13 12:29:21 +0900, Amit Lan

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread Andres Freund
On 2018-04-13 12:29:21 +0900, Amit Langote wrote: > On 2018/04/13 7:36, Peter Geoghegan wrote: > > On Thu, Apr 12, 2018 at 11:47 AM, Peter Geoghegan wrote: > >> In short, it looks like the tests added to update.sql by commit > >> 2f178441 ("Allow UPDATE to move rows between partitions") lead to th

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread Ashutosh Bapat
On Fri, Apr 13, 2018 at 7:45 AM, Amit Langote wrote: > On 2018/04/13 1:47, Robert Haas wrote: >> On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera >> wrote: >>> Here's an idea. Why don't we move the function/opclass creation lines >>> to insert.sql, without the DROPs, and use the same functions/o

Re: Instability in partition_prune test?

2018-04-12 Thread David Rowley
On 13 April 2018 at 14:41, David Rowley wrote: > I'll just need to go think about how we can make the test stable now. Thomas and I discussed this a bit off-list. The attached basically adds: set max_parallel_workers = 0; before the Parallel Append tests. All those tests were intended to do w

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread Amit Langote
On 2018/04/13 7:36, Peter Geoghegan wrote: > On Thu, Apr 12, 2018 at 11:47 AM, Peter Geoghegan wrote: >> In short, it looks like the tests added to update.sql by commit >> 2f178441 ("Allow UPDATE to move rows between partitions") lead to this >> failure, since I always hit a problem when update.sq

Re: Instability in the postgres_fdw regression test

2018-04-12 Thread Etsuro Fujita
(2018/04/13 3:49), Robert Haas wrote: On Thu, Apr 12, 2018 at 12:49 PM, Tom Lane wrote: We've been seeing $subject since commit 1bc0100d2, with little clue as to the cause. Previous attempts to fix it by preventing autovacuum from running on the relevant tables didn't seem to help. I have now

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread Michael Paquier
On Thu, Apr 12, 2018 at 03:36:12PM -0700, Peter Geoghegan wrote: > Without having looked at it in much detail, this seems rather more > likely to be the fault of 2f178441. That was recent enough that it's > easy to believe that I'd be the first to notice it, and actually has > on-disk changes, in t

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread David Rowley
On 13 April 2018 at 14:15, Amit Langote wrote: > On 2018/04/13 1:47, Robert Haas wrote: >> On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera >> wrote: >>> Here's an idea. Why don't we move the function/opclass creation lines >>> to insert.sql, without the DROPs, and use the same functions/opclass

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Amit Kapila
On Fri, Apr 13, 2018 at 6:59 AM, Michael Paquier wrote: > On Thu, Apr 12, 2018 at 02:55:53PM -0400, Robert Haas wrote: >> I think it may actually be confusing. If you run pg_ctl reload and it >> reports that the value has changed, you'll expect it to have taken >> effect. But really, it will tak

Re: Instability in partition_prune test?

2018-04-12 Thread David Rowley
On 13 April 2018 at 14:37, Thomas Munro wrote: > I think it might be working as designed! The participants are allowed > to run other subplans, because they're parallel sub-plans: it's just a > question of whether any backend manages to complete its subplan and > then go looking for another subpl

Re: Instability in partition_prune test?

2018-04-12 Thread Thomas Munro
On Fri, Apr 13, 2018 at 1:21 PM, David Rowley wrote: > On 13 April 2018 at 10:29, Thomas Munro wrote: >> This is a Parallel Append with three processes working on three >> subplans. It looks like one of the subplans got executed twice? > > Hi Thomas, > > Thanks for the report. If you're able to

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread Amit Langote
On 2018/04/13 1:47, Robert Haas wrote: > On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera > wrote: >> Here's an idea. Why don't we move the function/opclass creation lines >> to insert.sql, without the DROPs, and use the same functions/opclasses >> in the three tests insert.sql, alter_table.sql,

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Michael Paquier
On Thu, Apr 12, 2018 at 02:55:53PM -0400, Robert Haas wrote: > I think it may actually be confusing. If you run pg_ctl reload and it > reports that the value has changed, you'll expect it to have taken > effect. But really, it will take effect at some later time. It is true that sometimes some p

Re: Instability in partition_prune test?

2018-04-12 Thread David Rowley
On 13 April 2018 at 10:29, Thomas Munro wrote: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=elver&dt=2018-04-12%2018%3A18%3A05 > > partition_prune ... FAILED > >Subplans Removed: 6 >-> Parallel Seq Scan on ab_a2_b1 (ac

MinIndexTupleSize seems slightly wrong

2018-04-12 Thread Peter Geoghegan
itup.h says of MinIndexTupleSize/MaxIndexTuplesPerPage: /* * MaxIndexTuplesPerPage is an upper bound on the number of tuples that can * fit on one index page. An index tuple must have either data or a null * bitmap, so we can safely assume it's at least 1 byte bigger than a bare * IndexTupleD

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Michael Paquier
On Thu, Apr 12, 2018 at 03:39:19PM -0400, Robert Haas wrote: > Well I don't object to updating the documentation, but just because > something isn't what the user expects doesn't make it a bug. Users > can have arbitrary expectations. Yes, I agree that this is not a bug, and should not be categor

Re: Instability in partition_prune test?

2018-04-12 Thread Tom Lane
Thomas Munro writes: > Apologies if this was already discussed, I didn't see it. One of my > animals elver had a one-off failure this morning: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=elver&dt=2018-04-12%2018%3A18%3A05 Yeah, this looks very much like my recent complaint: https:

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread Peter Geoghegan
On Thu, Apr 12, 2018 at 11:47 AM, Peter Geoghegan wrote: > In short, it looks like the tests added to update.sql by commit > 2f178441 ("Allow UPDATE to move rows between partitions") lead to this > failure, since I always hit a problem when update.sql is reached. I > haven't gone to the trouble of

Instability in partition_prune test?

2018-04-12 Thread Thomas Munro
Hi, Apologies if this was already discussed, I didn't see it. One of my animals elver had a one-off failure this morning: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=elver&dt=2018-04-12%2018%3A18%3A05 partition_prune ... FAILED Subplans Remo

Re: [HACKERS] Runtime Partition Pruning

2018-04-12 Thread David Rowley
On 13 April 2018 at 04:57, Robert Haas wrote: > BTW, looking at ExecSetupPartitionPruneState: > > /* > * Create a sub memory context which we'll use when making calls to > the > * query planner's function to determine which partitions will > match. The > * plan

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
I wonder what prompted people to #include "catalog/partition.h" in executor.h. Amit Langote wrote: > Anyway, after reading your replies, I thought of taking a stab at unifying > the partitioning information that's cached by relcache.c. After going over your patch, I think you went slightly overb

Re: psql leaks memory on query cancellation

2018-04-12 Thread Tom Lane
I wrote: > I imagine that this indicates that control-C processing allocates some > memory it doesn't free, resulting in an "island" up at the end of memory > that prevents glibc from releasing all the free memory it's got. Whether > that's an actual leak, or just memory we're holding onto in hope

Re: submake-errcodes

2018-04-12 Thread Christoph Berg
Re: Tom Lane 2018-04-12 <10354.1523558...@sss.pgh.pa.us> > ... or then again, maybe I do. Is it possible that your build > recipe involves invoking our makefiles from an outer "make" run? > If so, maybe you need to explicitly set MAKELEVEL=0 when invoking > our build, to keep it from thinking it i

Re: Covering GiST indexes

2018-04-12 Thread Komяpa
> Another thing that could be done for PostGIS geometries is just another > opclass which > stores geometries "as is" in leafs. As I know, geometries contain MBRs > inside their > own, so there is no need to store extra MBR. I think the reason why > PostGIS > doesn't have such opclass yet is that

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > Please revert the part of this commit that changed the lock level. Done. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Alvaro Herrera
Keith Fiske wrote: > I'm also not sure that we should have this mindset of partitioning working > as inheritance does either. Inheritance was only used before because it was > the only mechanism available. And while you do still use it under the hood > for parts of partitioning, I don't see any re

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Keith Fiske
On Thu, Apr 12, 2018 at 3:39 PM, Robert Haas wrote: > On Thu, Apr 12, 2018 at 3:15 PM, Jonathan S. Katz > wrote: > > Behavior-wise it’s certainly a bug: you add a TABLESPACE on the parent > > table, and that property is not passed down to the children, which is not > > what the user expects. At

Re: Partitioned tables and covering indexes

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 6:14 AM, Alexander Korotkov wrote: > I'm OK with collation of included columns to be the same as collation > of underlying table columns. But I still think we should throw an error > when user is trying to specify his own collation of included columns. I agree. The colla

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 3:15 PM, Jonathan S. Katz wrote: > Behavior-wise it’s certainly a bug: you add a TABLESPACE on the parent > table, and that property is not passed down to the children, which is not > what the user expects. At a minimum, if we don’t back patch it, we probably > need to upd

Re: submake-errcodes

2018-04-12 Thread Andres Freund
On 2018-04-12 11:22:45 -0700, Jacob Champion wrote: > On Thu, Apr 12, 2018 at 11:00 AM, Tom Lane wrote: > > Andrew Gierth writes: > >> Is it worth exploring the idea of changing to a non-recursive style of > >> makefile? > > > > Not post-feature-freeze, for sure. Whether it's worth the work as a

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Keith Fiske
On Thu, Apr 12, 2018 at 3:15 PM, Jonathan S. Katz < jonathan.k...@excoventures.com> wrote: > > > On Apr 12, 2018, at 3:10 PM, Alvaro Herrera > wrote: > > > > Robert Haas wrote: > >> On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz > >> wrote: > >>> If there are no strong objections I am going t

Proposal: Remove "no" from the default english.stop word list

2018-04-12 Thread Peter Marreck
I recently ran into an issue where (after implementing fulltext search on my site) a user searching real estate listings for "no pets" also got results for "pets OK"! This was obviously a problem. After investigating, it seems the word "no" is considered a stopword by default (it's in the english.s

Native partitioning tablespace inheritance

2018-04-12 Thread David G. Johnston
On Thursday, April 12, 2018, Robert Haas wrote: > On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz > wrote: > > If there are no strong objections I am going to add this to the “Older > Bugs” > > section of Open Items in a little bit. > > I strongly object. This is not a bug. The TABLESPACE cl

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Jonathan S. Katz
> On Apr 12, 2018, at 3:10 PM, Alvaro Herrera wrote: > > Robert Haas wrote: >> On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz >> wrote: >>> If there are no strong objections I am going to add this to the “Older Bugs” >>> section of Open Items in a little bit. >> >> I strongly object. This

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 3:10 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz >> wrote: >> > If there are no strong objections I am going to add this to the “Older >> > Bugs” >> > section of Open Items in a little bit. >> >> I strongly object.

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz > wrote: > > If there are no strong objections I am going to add this to the “Older Bugs” > > section of Open Items in a little bit. > > I strongly object. This is not a bug. The TABLESPACE clause doing > exactly what it was

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz wrote: > If there are no strong objections I am going to add this to the “Older Bugs” > section of Open Items in a little bit. I strongly object. This is not a bug. The TABLESPACE clause doing exactly what it was intended to do, which is determi

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Robert Haas
On Wed, Apr 11, 2018 at 7:09 AM, Heikki Linnakangas wrote: > I think the new behavior where the GUC only takes effect at next checkpoint > is OK. It seems quite intuitive. I think it may actually be confusing. If you run pg_ctl reload and it reports that the value has changed, you'll expect it t

Re: Instability in the postgres_fdw regression test

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 12:49 PM, Tom Lane wrote: > We've been seeing $subject since commit 1bc0100d2, with little clue > as to the cause. Previous attempts to fix it by preventing autovacuum > from running on the relevant tables didn't seem to help. > > I have now managed to reproduce the issue

wal_consistency_checking reports an inconsistency on master branch

2018-04-12 Thread Peter Geoghegan
Running "make installcheck" with wal_consistency_checking='all' on the master branch shows the follow failure on a streaming replica: 19696/2018-04-12 11:35:29 PDT FATAL: inconsistent page found, rel 1663/50192/66636, forknum 0, blkno 0 19696/2018-04-12 11:35:29 PDT CONTEXT: WAL redo at 2/6D8411

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Jonathan S. Katz
> On Apr 12, 2018, at 2:36 PM, Christophe Pettus wrote: > > >> On Apr 12, 2018, at 09:17, Robert Haas wrote: >> Hmm, that's interesting. So you want the children to inherit the >> parent's tablespace when they are created, but if the parent's >> tablespace is later changed, the existing child

Re: submake-errcodes

2018-04-12 Thread Andrew Gierth
> "Alvaro" == Alvaro Herrera writes: Alvaro> I'm altogether ignorant on how might we use it for the headers Alvaro> problem, mind. I only tried to tackle the main executable. Solving the headers problem would seem to require making things non-recursive at the topmost level rather than tryi

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Christophe Pettus
> On Apr 12, 2018, at 09:17, Robert Haas wrote: > Hmm, that's interesting. So you want the children to inherit the > parent's tablespace when they are created, but if the parent's > tablespace is later changed, the existing children don't move? +1 to that behavior. While it's always possible t

Re: submake-errcodes

2018-04-12 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > ... still same error. Easy to reproduce on F-27 box. I don't have F27 at hand, but I tried F26 and F28, and I can't reproduce on either one. I tried various combinations of python2 versus python3, in-tree build versus VPATH from bare checkout versus

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Apr 12, 2018 at 8:55 AM, Alvaro Herrera > wrote: > > Amit Langote wrote: > >> Anyway, after reading your replies, I thought of taking a stab at unifying > >> the partitioning information that's cached by relcache.c. > > > > Wow. Now that's one large patch. I'm going

Re: psql leaks memory on query cancellation

2018-04-12 Thread Teodor Sigaev
I imagine that this indicates that control-C processing allocates some memory it doesn't free, resulting in an "island" up at the end of memory that prevents glibc from releasing all the free memory it's got. Whether that's an actual leak, or just memory we're holding onto in hopes of reusing it,

Re: submake-errcodes

2018-04-12 Thread Jacob Champion
On Thu, Apr 12, 2018 at 11:00 AM, Tom Lane wrote: > Andrew Gierth writes: >> Is it worth exploring the idea of changing to a non-recursive style of >> makefile? > > Not post-feature-freeze, for sure. Whether it's worth the work as a > long-term project, I dunno. I've been taking a look at this

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'm dealing with this now -- will push shortly. The sane thing to do is > backpatch my previous memcxt fixes, since your patch introduces a > problem that we discussed with that other patch, namely that you would > leak the whole memory context if there is a problem while

Re: submake-errcodes

2018-04-12 Thread Alvaro Herrera
Andrew Gierth wrote: > > "Tom" == Tom Lane writes: > > Tom> I'm beginning to get dissatisfied with this approach of expecting > Tom> the topmost Make run to do the generated-headers work > > Is it worth exploring the idea of changing to a non-recursive style of > makefile? I looked onto t

Re: submake-errcodes

2018-04-12 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> I'm beginning to get dissatisfied with this approach of expecting > Tom> the topmost Make run to do the generated-headers work > Is it worth exploring the idea of changing to a non-recursive style of > makefile? Not post-feature-freeze,

Re: submake-errcodes

2018-04-12 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> I'm beginning to get dissatisfied with this approach of expecting Tom> the topmost Make run to do the generated-headers work Is it worth exploring the idea of changing to a non-recursive style of makefile? -- Andrew (irc:RhodiumToad)

Re: psql leaks memory on query cancellation

2018-04-12 Thread Tom Lane
Craig Ringer writes: > On 12 April 2018 at 18:26, Darafei "Komяpa" Praliaskouski > wrote: >> Is it expected behavior (so I can have a look at something server returned >> somehow and it's kept there for me), or a plain leak? > This is totally normal behaviour for any C program. Well, it depend

Re: Covering GiST indexes

2018-04-12 Thread Peter Geoghegan
On Thu, Apr 12, 2018 at 4:00 AM, Andrey Borodin wrote: > I have two concerns. > First one is about INDEX_AM_RESERVED_BIT. > B-tree uses it as a base for prefix truncation (I'm not quite sure why it is > usually called suffix truncation, but this is a matter for other thread). Since you brought i

Re: [HACKERS] Runtime Partition Pruning

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 12:40 AM, David Rowley wrote: > I guess the problem there would be there's nothing to say that parse > analysis will shortly be followed by a call to the planner, and a call > to the planner does not mean the plan is about to be executed. So I > don't think it would be poss

Instability in the postgres_fdw regression test

2018-04-12 Thread Tom Lane
We've been seeing $subject since commit 1bc0100d2, with little clue as to the cause. Previous attempts to fix it by preventing autovacuum from running on the relevant tables didn't seem to help. I have now managed to reproduce the issue reliably enough to study it. (It turns out that on longfin's

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread Robert Haas
On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera wrote: > Here's an idea. Why don't we move the function/opclass creation lines > to insert.sql, without the DROPs, and use the same functions/opclasses > in the three tests insert.sql, alter_table.sql, hash_part.sql and > partition_prune.sql, i.e. n

Re: WIP: Covering + unique indexes.

2018-04-12 Thread Teodor Sigaev
Peter Geoghegan wrote: On Tue, Apr 10, 2018 at 5:45 PM, Peter Geoghegan wrote: I did find another problem, though. Looks like the idea to explicitly represent the number of attributes directly has paid off already: pg@~[3711]=# create table covering_bug (f1 int, f2 int, f3 text); create uniq

Re: Covering GiST indexes

2018-04-12 Thread Alexander Korotkov
Hi, Andrey! On Thu, Apr 12, 2018 at 2:00 PM, Andrey Borodin wrote: > Looks like we finally have covering indexes! And that's cool! > Thank you! So I decided to create a thread to discuss covering GiST indexes. > Here's a prototype patch implementing this functionality. > It is quite small (+80

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Robert Haas
On Wed, Apr 11, 2018 at 9:55 PM, David Rowley wrote: > I imagine the correct thing to do is properly record the TABLESPACE > option for the partitioned table then make child tables use that if > nothing else was specified. > > This would allow the parent partition's tablespace to be changed from >

Re: crash with sql language partition support function

2018-04-12 Thread Robert Haas
On Thu, Apr 12, 2018 at 8:55 AM, Alvaro Herrera wrote: > Amit Langote wrote: >> Anyway, after reading your replies, I thought of taking a stab at unifying >> the partitioning information that's cached by relcache.c. > > Wow. Now that's one large patch. I'm going to run with this for HEAD, > but

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > I don't think it was a good idea to change this without a lot more > discussion, as part of another commit that really was about something > else, and after feature freeze. > Please revert the part of this commit that changed the lock level. You're right, that was too hasty.

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Amit Langote wrote: > Since this bug also exists in the released PG 10 branch, I also created a > patch for that. It's slightly different than the one for PG 11dev, > because there were some changes recently to how the memory context is > manipulated in RelationBuildPartitionKey. That's > v1-PG1

Re: WARNING in parallel index creation.

2018-04-12 Thread Robert Haas
On Wed, Apr 11, 2018 at 2:34 PM, Tom Lane wrote: > That test is, therefore, wrong. Otherwise, no non-builtin function > could ever be marked parallel safe, for fear that the shlib it lives > in might try to set up a custom variable at load time. I don't follow that logic. If the check is more s

Re: submake-errcodes

2018-04-12 Thread Devrim Gündüz
Hi Tom, On Wed, 2018-04-11 at 10:16 -0400, Tom Lane wrote: > You sure you're on 31f1f0bb4fd642643994d35c35ecb5b929711a99 or later? To make sure, I am using latest git snapshot: https://download.postgresql.org/pub/snapshot/dev/postgresql-snapshot.tar.bz2 and still same error. Easy to reproduce

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Jonathan S. Katz
> On Apr 12, 2018, at 9:24 AM, Ashutosh Bapat > wrote: > > On Thu, Apr 12, 2018 at 6:52 PM, Alvaro Herrera > wrote: >> >> Now, maybe what you suggest for open items is to create a separate view >> using much of the commitfest app code, say >> https://commitfest.postgresql.org/openitems/NNN >

Re: submake-errcodes

2018-04-12 Thread Tom Lane
Michael Paquier writes: > But this does not work: > ./configure blah > cd src/pl/plpython/ > make -j 4 check Hm. That shows yet another parallel-safety hazard, which can be resolved like this: diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile index 653fe64..c17015b 100644 --- a/s

Re: Partitioned tables and covering indexes

2018-04-12 Thread Teodor Sigaev
pushed. Hope, second try will be successful. Teodor Sigaev wrote: Thank you, pushed Amit Langote wrote: Hi. On 2018/04/11 0:36, Teodor Sigaev wrote: Does the attached fix look correct?  Haven't checked the fix with ATTACH PARTITION though. Attached patch seems to fix the problem.

Re: Reopen logfile on SIGHUP

2018-04-12 Thread Alexander Kuzmenkov
On 11.04.2018 00:00, Tom Lane wrote: So we need a mechanism that's narrowly targeted to reopening the logfile, without SIGHUP'ing the entire database. We can send SIGUSR1 to the syslogger process. To make its pid easier to find out, it can be published in "$PGDATA/logging_collector.pid", as s

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-12 Thread Robert Haas
On Mon, Apr 2, 2018 at 3:11 PM, Alvaro Herrera wrote: > Why do we need AccessExclusiveLock on all children of a relation that we > want to scan to search for rows not satisfying the constraint? I think > it should be enough to get ShareLock, which prevents INSERT, no? I have > a feeling I'm miss

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-12 Thread Tom Lane
Pavan Deolasee writes: > On Thu, Apr 12, 2018 at 1:58 AM, Tom Lane wrote: >> So while looking at this, it suddenly occurred to me that probing with >> SnapshotDirty isn't that safe for regular (non-TOAST) Oid assignment >> either. > Yeah it occurred to me as well, but when I looked at the code,

Re: submake-errcodes

2018-04-12 Thread Tom Lane
Christoph Berg writes: > Most of these work for me as well. The actual incantation via > debian/rules fails, but I couldn't really narrow down what the > difference is. Manually invoking the debian/rules targets: For a moment I thought that this might be the critical difference: > preparing build

Re: Partitioned tables and covering indexes

2018-04-12 Thread Teodor Sigaev
Thanks to everyone, this part is pushed. I will waiting a bit before pushing topic-stater patch to have a time to look on buildfarm. Alvaro, could you add a comment to CompareIndexInfo() to clarify why it doesn't compare indoptions (like DESC/ASC etc)? It doesn't matter for uniqueness of index

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
On Thu, Apr 12, 2018 at 6:52 PM, Alvaro Herrera wrote: > > Now, maybe what you suggest for open items is to create a separate view > using much of the commitfest app code, say > https://commitfest.postgresql.org/openitems/NNN > I think that would probably work well. Yes, that's what I had in mind

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Alvaro Herrera
Ashutosh Bapat wrote: > On Thu, Apr 12, 2018 at 6:27 PM, Alvaro Herrera > wrote: > > Ashutosh Bapat wrote: > > > >> We use commitfest app for tracking the patches submitted. It has done > >> well. Can we re-purpose the same for tracking open items? > > > > I think the rules are too different to c

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-12 Thread Pavan Deolasee
On Thu, Apr 12, 2018 at 5:21 AM, Tom Lane wrote: > Michael Paquier writes: > > I have not really checked this thread in details, but one thing that > > strikes me is that it would be rather easy to add a TAP test based on > > the initial script that Pavan has sent. Would that be worth testing >

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
On Thu, Apr 12, 2018 at 6:27 PM, Alvaro Herrera wrote: > Ashutosh Bapat wrote: > >> We use commitfest app for tracking the patches submitted. It has done >> well. Can we re-purpose the same for tracking open items? > > I think the rules are too different to cram both in the same place. What do yo

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-12 Thread Pavan Deolasee
On Thu, Apr 12, 2018 at 1:58 AM, Tom Lane wrote: > So while looking at this, it suddenly occurred to me that probing with > SnapshotDirty isn't that safe for regular (non-TOAST) Oid assignment > either. > Yeah it occurred to me as well, but when I looked at the code, I couldn't find a case that

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Alvaro Herrera
Ashutosh Bapat wrote: > We use commitfest app for tracking the patches submitted. It has done > well. Can we re-purpose the same for tracking open items? I think the rules are too different to cram both in the same place. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Amit Langote wrote: > Anyway, after reading your replies, I thought of taking a stab at unifying > the partitioning information that's cached by relcache.c. Wow. Now that's one large patch. I'm going to run with this for HEAD, but I think we should do a minimal fix for PG10. Did you detect any

Re: Boolean partitions syntax

2018-04-12 Thread Jonathan S. Katz
Hi, > On Apr 12, 2018, at 12:12 AM, Kyotaro HORIGUCHI > wrote: > > Hello. > > At Wed, 11 Apr 2018 09:43:37 -0400, "Jonathan S. Katz" > mailto:jonathan.k...@excoventures.com>> > wrote in > >>case EXPR_KIND_PARTITIO

Re: psql leaks memory on query cancellation

2018-04-12 Thread Komяpa
> > > > Is it expected behavior (so I can have a look at something server > returned > > somehow and it's kept there for me), or a plain leak? > > This is totally normal behaviour for any C program. > Thanks Konstantin and Craig for the help. To mitigate the issue I've changed the allocator on my

Re: Covering GiST indexes

2018-04-12 Thread Aleksander Alekseev
Hello Andrey, > So I decided to create a thread to discuss covering GiST indexes. > Here's a prototype patch implementing this functionality. It is quite > small (+80 -30) and lacks tests and docs. But it creates a context. I'm glad you got interested in this area. It would be great to have cove

Re: Covering GiST indexes

2018-04-12 Thread Teodor Sigaev
Interesting work. I don't have a time now to learn deep your patch, so, add it to next commitfest, pls. First of all I'd like to see more tests in patch, not only CREATE INDEX. Andrey Borodin wrote: Hi, hackers! Looks like we finally have covering indexes! And that's cool! So I decided to c

Re: psql leaks memory on query cancellation

2018-04-12 Thread Craig Ringer
On 12 April 2018 at 18:26, Darafei "Komяpa" Praliaskouski wrote: > Hi, > > psql (PostgreSQL) 10.3 > > Here are the steps to reproduce a leak: > > 1. connect to 10.3 server, perform the query similar to: > > select 'message' || generate_series(1,10); > > 2. monitoring psql memory usage in

Re: psql leaks memory on query cancellation

2018-04-12 Thread Konstantin Knizhnik
On 12.04.2018 13:26, Darafei "Komяpa" Praliaskouski wrote: Hi, psql (PostgreSQL) 10.3 Here are the steps to reproduce a leak: 1. connect to 10.3 server, perform the query similar to: select 'message' || generate_series(1,10); 2. monitoring psql memory usage in htop or similar tool,

Covering GiST indexes

2018-04-12 Thread Andrey Borodin
Hi, hackers! Looks like we finally have covering indexes! And that's cool! So I decided to create a thread to discuss covering GiST indexes. Here's a prototype patch implementing this functionality. It is quite small (+80 -30) and lacks tests and docs. But it creates a context. I have two concer

psql leaks memory on query cancellation

2018-04-12 Thread Komяpa
Hi, psql (PostgreSQL) 10.3 Here are the steps to reproduce a leak: 1. connect to 10.3 server, perform the query similar to: select 'message' || generate_series(1,10); 2. monitoring psql memory usage in htop or similar tool, press ctrl+c at some point where you can clearly distinguish a

Re: Partitioned tables and covering indexes

2018-04-12 Thread Alexander Korotkov
On Thu, Apr 12, 2018 at 1:14 AM, Teodor Sigaev wrote: > Peter Geoghegan wrote: > >> On Wed, Apr 11, 2018 at 2:29 PM, Peter Eisentraut >> wrote: >> >>> But in this case it doesn't even do equality comparison, it just returns >>> the value. >>> >> >> That's the idea that I tried to express. The po

Re: Partitioned tables and covering indexes

2018-04-12 Thread Alexander Korotkov
On Thu, Apr 12, 2018 at 1:14 AM, Teodor Sigaev wrote: > That's the idea that I tried to express. The point is that we need to >>> tell the user that there is no need to worry about it, rather than >>> that they're wrong to ask about it. Though we should probably actually >>> just throw an error.

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Michael Paquier
On Thu, Apr 12, 2018 at 04:59:10PM +0900, Kyotaro HORIGUCHI wrote: > At Thu, 12 Apr 2018 14:07:53 +0900, Michael Paquier > wrote in <20180412050753.ga19...@paquier.xyz> >> I have been able to spend a couple of hours on your patch, wrapping my >> mind on your stuff. So what I had in mind was some

Re: Problem while setting the fpw with SIGHUP

2018-04-12 Thread Kyotaro HORIGUCHI
Hello. At Thu, 12 Apr 2018 14:07:53 +0900, Michael Paquier wrote in <20180412050753.ga19...@paquier.xyz> > On Thu, Apr 12, 2018 at 10:34:30AM +0900, Kyotaro HORIGUCHI wrote: > > Checkpointer never calls CreateCheckPoint while > > RecoveryInProgress() == true. In other words, checkpointer is not

  1   2   >