Re: WIP: Covering + unique indexes.

2018-04-07 Thread Alvaro Herrera
I didn't like rel.h being included in itup.h. Do you really need a Relation as argument to index_truncate_tuple? It looks to me like you could pass the tupledesc instead; indnatts could be passed as a separate argument instead of IndexRelationGetNumberOfAttributes. -- Álvaro Herrera

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Alvaro Herrera
David Rowley wrote: > On 8 April 2018 at 01:18, Alvaro Herrera wrote: > > Amit had it as "indexes" also in his original. I wanted to avoid using > > the "indexes" word alone, whose meaning is so overloaded. > > hmm, good point. > > > How about

Re: WIP: a way forward on bootstrap data

2018-04-07 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > John Naylor wrote: > >> Commit 9fdb675fc added a symbol to pg_opfamily.h > >> where there were none before, so I went ahead and wrapped it with an > >> EXPOSE_TO_CLIENT_CODE macro. > > > Actually, after p

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-07 Thread Alvaro Herrera
amul sul wrote: > On Sat, Apr 7, 2018 at 9:08 AM, Andres Freund wrote: > >> +test: partition-key-update-1 > >> +test: partition-key-update-2 > >> +test: partition-key-update-3 > > > > Can you give these more descriptive names please (or further combine them)? > > As I explained above further com

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-07 Thread Alvaro Herrera
Andreas Seltenreich wrote: > Hi, > > sqlsmith found a query that triggers the following assertion in master > as of 039eb6e92f: > > TRAP: FailedAssertion("!(subpath->parallel_safe)", File: "pathnode.c", Line: > 1813) > > Backtrace and recipe against the regression database below. Uh, that's pr

Re: [HACKERS] path toward faster partition pruning

2018-04-07 Thread Alvaro Herrera
Andres Freund wrote: > On 2018-04-07 08:13:23 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > I've also attempted to fix rhinoceros's failure I remarked upon a couple > > > hours ago in > > > https://postgr.es/m/20180406210330.wmqw42wqgii

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Alvaro Herrera
I pushed this patch -- 0001, 0002 and 0003 only. I did not include anything from 0004 and 0005; I didn't even get to the point of reading them, so that I could focus on the first part. I did not find anything to complain about. I made a few adjustments and asked David to supply a paragraph for p

Re: pgsql: Support partition pruning at execution time

2018-04-07 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Support partition pruning at execution time > > Buildfarm member lapwing doesn't like this. I can reproduce the > failures here by setting force_parallel_mode = regress. Kind > of looks like instrumentation counts aren&#

Re: [sqlsmith] Failed assertion on pfree() via perform_pruning_combine_step

2018-04-09 Thread Alvaro Herrera
Hello, Amit Langote wrote: > I have reproduced this and found that the problem is that > perform_pruning_combine_step forgets to *copy* the bitmapset of the first > step in the handling of an COMBINE_INTERSECT step. Pushed, thanks Amit and Andreas! -- Álvaro Herrerahttps://www.

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-04-09 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Remember, the purpose of predicate locks is to lock key ranges, not physical > pages or tuples in the index. We use leaf pages as handy shortcut for "any > key value that would belong on this page", but it is just an implementation > detail. Hmm ... so, thinking about

Re: [HACKERS] Runtime Partition Pruning

2018-04-09 Thread Alvaro Herrera
Robert Haas wrote: > On Sat, Apr 7, 2018 at 5:13 PM, Alvaro Herrera > wrote: > > I had reservations about a relation_open() in the new executor code. It > > seemed a bit odd; we don't have any other relation_open in the executor > > anywhere. However, setting up t

Re: [HACKERS] Runtime Partition Pruning

2018-04-09 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Apr 9, 2018 at 2:28 PM, Alvaro Herrera > wrote: > > Robert Haas wrote: > >> I don't get this. The executor surely had to (and did) open all of > >> the relations somewhere even before this patch. > > > > I was wor

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-09 Thread Alvaro Herrera
Andres Freund wrote: > Another approach, that's simpler to implement, is to simply have a > second selfpipe, just for WL_POSTMASTER_DEATH. Would it work to use this second pipe, to which each child writes a byte that postmaster never reads, and then rely on SIGPIPE when postmaster dies? Then we

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-09 Thread Alvaro Herrera
Andres Freund wrote: > > On April 9, 2018 6:31:07 PM PDT, Alvaro Herrera > wrote: > >Would it work to use this second pipe, to which each child writes a > >byte that postmaster never reads, and then rely on SIGPIPE when > >postmaster dies? Then we never need to

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-04-10 Thread Alvaro Herrera
Nikolay Shaplov wrote: > В письме от 25 января 2018 11:29:34 пользователь Tom Lane написал: > > Alvaro Herrera writes: > > > Actually, Nikolay provided a possible solution: if you execute ALTER > > > TABLE SET (toast.foobar = xyz), and a toast table doesn't exist

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-04-10 Thread Alvaro Herrera
Nikolay Shaplov wrote: > But I need some confirmation, in order not to write patch in vain again :-) Don't worry, rest assured that you will still write *many* patches in vain, not just this one. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: crash with sql language partition support function

2018-04-10 Thread Alvaro Herrera
Ashutosh Bapat wrote: > On Tue, Apr 10, 2018 at 1:44 PM, Amit Langote > wrote: > > > > Attached fixes it. It teaches RelationBuildPartitionKey() to use > > fmgr_info_cxt and pass rd_partkeycxt to it. > > The patch is using partkeycxt and not rd_partkeycxt. Probably a typo > in the mail. But a wi

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2018-04-10 Thread Alvaro Herrera
Mark Rofail wrote: > Hello David, > > On Tue, Apr 10, 2018 at 3:47 PM, David Steele wrote: > > > You should produce a new version by then that addresses Alvaro's > > concerns and I imagine that will require quite a bit of discussion and > > work. > > I'll get working. > I'll produce a patch wit

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2018-04-10 Thread Alvaro Herrera
Mark Rofail wrote: > On Tue, Apr 10, 2018 at 3:59 PM, Alvaro Herrera > wrote: > > > > documentation to it and a few extensive tests to ensure it works well); > > I think the existing regression tests verify that the patch works as > expectations, correct? I meant for

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Changed CC to pgsql-hackers. Tom Lane wrote: > Alvaro Herrera writes: > > Frankly, I don't like this. I would rather have an instrument->ntuples2 > > rather than these "divide this by nloops, sometimes" schizoid counters. > > This is already being misu

Re: 2018-03 CF Cleanup

2018-04-10 Thread Alvaro Herrera
David Steele wrote: > Hackers, > > I have gone through all the remaining non-bug entries in CF 2018-03 and > pushed them or closed them as appropriate. The majority of the patches > were in Needs Review or Ready for Committer status and I did a brief > review of each to be sure the state was reas

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Apr 10, 2018 at 11:14 AM, Alvaro Herrera > wrote: > > Questions: > > 1. Do we want to back-patch this to 10? I suppose (without checking) > > that EXPLAIN ANALYZE is already reporting bogus numbers for parallel > > index-only scans

Re: pgsql: Support partition pruning at execution time

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > > I don't have a strong opinion on that. Since we know how many tuples > were processed by each worker, knowing how many heap fetches we have > on a per-worker basis seems like a good thing to have, too. On the > other hand, maybe EXPLAIN (ANALYZE, VERBOSE) would give us tha

Re: submake-errcodes

2018-04-10 Thread Alvaro Herrera
Tom Lane wrote: > $(OBJS): | submake-generated-headers > > but I took it out thinking it was no longer needed. > > The short-term solution seems to be to put that back, but that's sort > of annoying because it means this isn't a bulletproof solution. It > will only work for builds started in on

Re: [HACKERS] Runtime Partition Pruning

2018-04-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Apr 9, 2018 at 2:28 PM, Alvaro Herrera > wrote: > >> I don't get this. The executor surely had to (and did) open all of > >> the relations somewhere even before this patch. > > I was worried that this coding could be seen as br

Re: [HACKERS] path toward faster partition pruning

2018-04-11 Thread Alvaro Herrera
Here's an idea. Why don't we move the function/opclass creation lines to insert.sql, without the DROPs, and use the same functions/opclasses in the three tests insert.sql, alter_table.sql, hash_part.sql and partition_prune.sql, i.e. not recreate what are essentially the same objects three times?

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2018-04-11 Thread Alvaro Herrera
Mark Rofail wrote: > > In particular: it seemed to me that you decided to throw away the idea > > of the new GIN operator without sufficient evidence that it was > > unnecessary. > > I have to admit to that. But in my defence @> is also GIN indexable so the > only difference in performance betwee

Re: relispartition for index partitions

2018-04-11 Thread Alvaro Herrera
Hello Andres Freund wrote: > On 2018-01-26 18:57:03 +0900, Amit Langote wrote: > > I noticed that relispartition isn't set for index's partitions. > > Is that intentional? > > This appears to be a question about > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=8b08f7d4820fd7a8ef

Re: 'make check' fails

2018-04-11 Thread Alvaro Herrera
Tom Lane wrote: > We can fix this by making submake-generated-headers be a recursive > prerequisite for "check" as well as "all" and "install". I wonder > whether anybody is expecting any other shortcuts to work. check-world certainly, but presumably that depends on check? -- Álvaro Herrera

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Alvaro Herrera
Jonathan S. Katz wrote: > During some RMT discussions I had proposed formatting the open items > into a table on the Wiki page with some useful info to help track the status > and surface the necessary info to track down the open item. The other proposal was that we could have a simple web app to

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Alvaro Herrera
Andres Freund wrote: > On 2018-04-11 13:54:34 -0300, Alvaro Herrera wrote: > > The other proposal was that we could have a simple web app to track open > > items. After all, we now know what we need from it. A wiki page seems > > more laborious. (The commitfest app also sp

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Alvaro Herrera
Magnus Hagander wrote: > (And of course, if we want to go in *any* direction away from the wiki, > it's not going to happen in time for *this* release..) Absolutely. But if we never start, it'll never get done. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Developmen

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-11 Thread Alvaro Herrera
Thanks for the discussion. Per your suggestions, I changed the check for default partition OID to an assert instead of the 'if' condition, and removed the code that attempted vainly to verify the constraint when attaching a foreign table as a partition. And pushed. I think we're done here, so ma

Re: Partitioned tables and covering indexes

2018-04-11 Thread Alvaro Herrera
Teodor Sigaev wrote: > Patch attached. I wonder why this is a problem in opfamilies but not collations. If we don't compare collations, wouldn't it make more sense to break out of the loop once the number of keys is reached? When this code was written, there was no question as to what length the

Re: relispartition for index partitions

2018-04-11 Thread Alvaro Herrera
Amit Langote wrote: > Hi. > > I noticed that relispartition isn't set for index's partitions. This patch should fix it. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/src/backend/catalog/index.c b/

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Amit Langote wrote: > Anyway, after reading your replies, I thought of taking a stab at unifying > the partitioning information that's cached by relcache.c. Wow. Now that's one large patch. I'm going to run with this for HEAD, but I think we should do a minimal fix for PG10. Did you detect any

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Alvaro Herrera
Ashutosh Bapat wrote: > We use commitfest app for tracking the patches submitted. It has done > well. Can we re-purpose the same for tracking open items? I think the rules are too different to cram both in the same place. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Alvaro Herrera
Ashutosh Bapat wrote: > On Thu, Apr 12, 2018 at 6:27 PM, Alvaro Herrera > wrote: > > Ashutosh Bapat wrote: > > > >> We use commitfest app for tracking the patches submitted. It has done > >> well. Can we re-purpose the same for tracking open items? > >

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Amit Langote wrote: > Since this bug also exists in the released PG 10 branch, I also created a > patch for that. It's slightly different than the one for PG 11dev, > because there were some changes recently to how the memory context is > manipulated in RelationBuildPartitionKey. That's > v1-PG1

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > I don't think it was a good idea to change this without a lot more > discussion, as part of another commit that really was about something > else, and after feature freeze. > Please revert the part of this commit that changed the lock level. You're right, that was too hasty.

Re: submake-errcodes

2018-04-12 Thread Alvaro Herrera
Andrew Gierth wrote: > > "Tom" == Tom Lane writes: > > Tom> I'm beginning to get dissatisfied with this approach of expecting > Tom> the topmost Make run to do the generated-headers work > > Is it worth exploring the idea of changing to a non-recursive style of > makefile? I looked onto t

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'm dealing with this now -- will push shortly. The sane thing to do is > backpatch my previous memcxt fixes, since your patch introduces a > problem that we discussed with that other patch, namely that you would > leak the whole memory context if there i

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Apr 12, 2018 at 8:55 AM, Alvaro Herrera > wrote: > > Amit Langote wrote: > >> Anyway, after reading your replies, I thought of taking a stab at unifying > >> the partitioning information that's cached by relcache.c. > > &

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Apr 12, 2018 at 2:40 PM, Jonathan S. Katz > wrote: > > If there are no strong objections I am going to add this to the “Older Bugs” > > section of Open Items in a little bit. > > I strongly object. This is not a bug. The TABLESPACE clause doing > exactly what it was

Re: Native partitioning tablespace inheritance

2018-04-12 Thread Alvaro Herrera
Keith Fiske wrote: > I'm also not sure that we should have this mindset of partitioning working > as inheritance does either. Inheritance was only used before because it was > the only mechanism available. And while you do still use it under the hood > for parts of partitioning, I don't see any re

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-12 Thread Alvaro Herrera
Robert Haas wrote: > Please revert the part of this commit that changed the lock level. Done. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: crash with sql language partition support function

2018-04-12 Thread Alvaro Herrera
I wonder what prompted people to #include "catalog/partition.h" in executor.h. Amit Langote wrote: > Anyway, after reading your replies, I thought of taking a stab at unifying > the partitioning information that's cached by relcache.c. After going over your patch, I think you went slightly overb

Re: crash with sql language partition support function

2018-04-13 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/13 6:58, Alvaro Herrera wrote: > > After going over your patch, I think you went slightly overboard here. > > Or maybe not, but this patch is so large that it's hard to form an > > opinion about it. > > It's mostly code movem

Re: Instability in partition_prune test?

2018-04-13 Thread Alvaro Herrera
David Rowley wrote: > On 13 April 2018 at 14:41, David Rowley wrote: > > I'll just need to go think about how we can make the test stable now. > > Thomas and I discussed this a bit off-list. > > The attached basically adds: > > set max_parallel_workers = 0; > > before the Parallel Append tests

Re: Instability in partition_prune test?

2018-04-13 Thread Alvaro Herrera
Tom Lane wrote: > David Rowley writes: > > The attached basically adds: > > set max_parallel_workers = 0; > > It seems quite silly to be asking for a parallel plan and then insisting > it not run in parallel. The idea is to use the parallel append code, but run it in the leader. Now that you me

Re: [HACKERS] path toward faster partition pruning

2018-04-13 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera > wrote: > > Here's an idea. Why don't we move the function/opclass creation lines > > to insert.sql, without the DROPs, and use the same functions/opclasses > > in the three tests insert.s

Re: crash with sql language partition support function

2018-04-13 Thread Alvaro Herrera
I think this is a good improvement. On top of that, I propose a new file partitioning/partdefs.h with the following approximate contents. This reduces cross-inclusion of headers to the minimum. I'm dealing with the fallout from this now, will post a complete patch shortly. /*---

Re: partitioning code reorganization

2018-04-13 Thread Alvaro Herrera
Further thought: compute_hash_value and satisfies_hash_partition both belong in partbounds.c. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: partitioning code reorganization

2018-04-14 Thread Alvaro Herrera
Amit Langote wrote: > On Sat, Apr 14, 2018 at 11:48 PM, Amit Langote > wrote: > > Hi. > > > > Thanks for taking care of few things I left like those PartitionKey > > accessors in rel.h. > > Forgot to mention -- there are some files that still include > catalog/partition.h but no longer need to.

Re: Guidelines on dropping objects in regression tests, sqlsmith

2018-04-14 Thread Alvaro Herrera
Peter Geoghegan wrote: > Commit 975ad4e6 fixed a nasty bug in BRIN that was > originally found by Andreas' fuzz testing. Perhaps that bug would > still be around if Alvaro had included any DROP statements within > brin.sql. Presumably his choice to not to do so was completely > arbitrary. Actually

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-16 Thread Alvaro Herrera
Amit Langote wrote: > The solution I came up with is to call map_variable_attnos() directly, > instead of going through map_partition_varattnos() every time, after first > creating the attribute map ourselves. Yeah, sounds good. I added a tweak: if the tupledescs are equal, there should be no ne

Re: Instability in partition_prune test?

2018-04-16 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> It seems quite silly to be asking for a parallel plan and then insisting > >> it not run in parallel. > > > Now that you mention it, this probably decreases coverage for the > > choose_n

Re: Instability in partition_prune test?

2018-04-16 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Yeah, loss of executor code coverage was what concerned me. > > > Here's a proposed patch for this. > > Seems reasonable. I'm still uncomfortable with the assumption > that if we as

Re: [HACKERS] Runtime Partition Pruning

2018-04-16 Thread Alvaro Herrera
David Rowley wrote: > For a while, during my review of the faster partition pruning patch I > was asking Amit to add pfree() calls in various places for this exact > reason, but in the end, I gave up and decided it was easier to just > create a new memory context to call the planner function from.

Re: Proposal: Adding json logging

2018-04-17 Thread Alvaro Herrera
One issue I haven't seen mentioned in this thread is the translation status of the server message (as well as its encoding): it's possible to receive messages in some random language if the lc_message setting is changed. Requiring that lc_messages must always be set to some English locale seems li

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-17 Thread Alvaro Herrera
Amit Langote wrote: > Attached find a patch that does that. When working on this, I noticed > that when recursing for inheritance children, ATPrepAlterColumnType() > would use a AlterTableCmd (cmd) that's already scribbled on as if it were > the original. While I agree that the code here is in p

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-17 Thread Alvaro Herrera
Amit Langote wrote: > I just confirmed my hunch that this wouldn't somehow do the right thing > when the OID system column is involved. Like this case: This looks too big a patch to pursue now. I'm inclined to just remove the equalTupdesc changes. -- Álvaro Herrerahttps://www.

Re: Test coverage for mark_invalid_subplans_as_finished

2018-04-17 Thread Alvaro Herrera
Thanks for that, pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Deadlock in multiple CIC.

2018-04-17 Thread Alvaro Herrera
Tom Lane wrote: > It's still not entirely clear what's happening on okapi, but in the > meantime I've thought of an easily-reproducible way to cause similar > failures in any branch. That is to run CREATE INDEX CONCURRENTLY > with default_transaction_isolation = serializable. Then, snapmgr.c > w

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2018-04-17 Thread Alvaro Herrera
Tom Lane wrote: > Fujii Masao writes: > > When VACUUM tries to truncate the trailing empty pages, it scans > > shared_buffers > > to invalidate the pages-to-truncate during holding an AccessExclusive lock > > on > > the relation. So if shared_buffers is huge, other transactions need to wait > >

Append's first_partial_plan

2018-04-17 Thread Alvaro Herrera
David Rowley wrote in https://postgr.es/m/CAKJS1f8o2Yd=rOP=et3a0fwgf+gsaokfsu6enhngztpv7nn...@mail.gmail.com > I've made another pass over the nodeAppend.c code and I'm unable to > see what might cause this, although I did discover a bug where > first_partial_plan is not set taking into account t

Re: partitioning code reorganization

2018-04-17 Thread Alvaro Herrera
Amit Langote wrote: > 0001-Make-copying-of-cached-partitioning-info-more-con.patch > 0002-Cache-all-partitioning-info-under-one-memory-cont.patch > 0003-Cache-partsupfunc-separately-from-PartitionKey.patch I'd rather not do these patches now, unless there is some pressing reason to (eg. some bug

Re: Append's first_partial_plan

2018-04-17 Thread Alvaro Herrera
David Rowley wrote: > On 18 April 2018 at 07:52, Alvaro Herrera wrote: > > While looking at this patch I became curious as to why do we even have > > first_partial_plan in the first place; it seems to require some strange > > contortions in the code. Wouldn't it be simp

Re: pruning disabled for array, enum, record, range type partition keys

2018-04-17 Thread Alvaro Herrera
Amit Langote wrote: > Ah, I think I got it after staring at the (btree) index code for a bit. > > What pruning code got wrong is that it's comparing the expression type > (type of the constant arg that will be compared with partition bound > datums when pruning) with the partopcintype to determin

Re: VM map freeze corruption

2018-04-18 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Wed, Apr 18, 2018 at 7:37 AM, Wood, Dan wrote: > > My analysis is that heap_prepare_freeze_tuple->FreezeMultiXactId() > > returns FRM_NOOP if the MultiXACT locked rows haven't committed. This > > results in changed=false and totally_frozen=true(as initialized). When >

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-18 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/18 0:04, Alvaro Herrera wrote: > > Amit Langote wrote: > > > >> I just confirmed my hunch that this wouldn't somehow do the right thing > >> when the OID system column is involved. Like this case: > > > > This

Re: pruning disabled for array, enum, record, range type partition keys

2018-04-18 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/18 7:11, Alvaro Herrera wrote: > > @@ -1546,8 +1546,8 @@ match_clause_to_partition_key(RelOptInfo *rel, >case PARTITION_STRATEGY_HASH: > cmpfn = get_opfamily_proc(part_scheme->partopfamily[partkeyidx], > -

Re: Deadlock in multiple CIC.

2018-04-18 Thread Alvaro Herrera
Tom Lane wrote: > Anyway, at this point I'm going to give up on the debug logging, revert > 9.4 to its prior state, and then see if the transaction-restart patch > makes the problem go away. Agreed, thanks. > >> (A couple of the other isolation tests do fail reliably under this > >> scenario; is

Re: Deadlock in multiple CIC.

2018-04-18 Thread Alvaro Herrera
Tom Lane wrote: > *** /home/postgres/pgsql/src/test/isolation/expected/lock-update-delete_1.out > Mon Feb 12 14:53:46 2018 > --- > /home/postgres/pgsql/src/test/isolation/output_iso/results/lock-update-delete.out >Wed Apr 18 11:30:23 2018 > *** > *** 150,156 > > t

Re: Proposal: Adding json logging

2018-04-18 Thread Alvaro Herrera
John W Higgins wrote: > On Sun, Apr 15, 2018 at 11:08 AM, David Arnold wrote: > > > >This would appear to solve multiline issues within Fluent. > > >https://docs.fluentd.org/v0.12/articles/parser_multiline > > > > I definitely looked at that, but what guarantees do I have that the > > sequenc

Re: pruning disabled for array, enum, record, range type partition keys

2018-04-18 Thread Alvaro Herrera
Amit Langote wrote: > On Thu, Apr 19, 2018 at 12:01 AM, Alvaro Herrera > wrote: > > Makes sense. Still, I was expecting that pruning of hash partitioning > > would also work for pseudotypes, yet it doesn't. > > It does? Aha, so it does. While staring at this n

Re: initdb fails to initialize data directory

2018-04-19 Thread Alvaro Herrera
Nikhil Sontakke wrote: > Intrigued, on digging down further, this is happening because we are > not using a long enough buffer to accept the output of "postgres -V" > in the find_other_exec() function. In our case, we had used > --with-extra-version option with configure which caused the output of

Re: pruning disabled for array, enum, record, range type partition keys

2018-04-19 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/19 6:45, Alvaro Herrera wrote: > > Please give this version another look. I also rewrote a couple of > > comments. > > Thanks, your rewritten version looks much better. Thanks! Pushed now. -- Álvaro Herrerahttps:

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-19 Thread Alvaro Herrera
Amit Langote wrote: > Yeah, I too have wondered in the past what it would take to make > equalTupDescs() return true for parent and partitions. Maybe we can make > it work by looking a bit harder than I did then. How about simply relaxing the tdtypeid test from equalTupleDescs? I haven't looked

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-04-19 Thread Alvaro Herrera
Alvaro Herrera wrote: > Amit Langote wrote: > > > Yeah, I too have wondered in the past what it would take to make > > equalTupDescs() return true for parent and partitions. Maybe we can make > > it work by looking a bit harder than I did then. > > How about sim

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2018-04-19 Thread Alvaro Herrera
Michael Paquier wrote: > Then, let's consider the beginning of the first commit fest of v12 as > judgement. Implementing radix tree for shared buffers is a long-term > project, which has no guarantee to get merged, while a visibly-simple > reloptions which helps in some cases... In the scenario

Re: initdb fails to initialize data directory

2018-04-19 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Nitpick: using MAXPGPATH seems for the buffer size seems to wrong to me. > We're not storing a path here. MAXPGPATH is 1024 by default, which seems > fine, but I would've spelled it out directly as "line[1000]". Hmm ... yeah, kinda. Do you care about it strongly enoug

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2018-04-19 Thread Alvaro Herrera
Andres Freund wrote: > On 2018-04-19 16:56:59 -0300, Alvaro Herrera wrote: > > Michael Paquier wrote: > > > > > Then, let's consider the beginning of the first commit fest of v12 as > > > judgement. Implementing radix tree for shared buffers is a long-term

Re: Repeated crashes in GENERATED ... AS IDENTITY tests

2018-04-19 Thread Alvaro Herrera
Tom Lane wrote: > The test case that's failing, in identity.sql, has been there since > early February; the lack of any crashes till more recently suggests > that something committed in mid-to-late March broke it. > > I have no idea what's going on there, but I think this is clearly > something w

Re: Event trigger bugs (was Re: Repeated crashes in GENERATED ... AS IDENTITY tests)

2018-04-19 Thread Alvaro Herrera
Tom Lane wrote: > Hence, two questions: > > * Should EventTriggerTableRewrite do > > if (!currentEventTriggerState || > currentEventTriggerState->commandCollectionInhibited) > return; > > like most of the other functions, or should it just check for null > currentEventTrigge

Re: [HACKERS] path toward faster partition pruning

2018-04-20 Thread Alvaro Herrera
Amit Langote wrote: > PS: git grep "partition by hash\|PARTITION BY HASH" on src/test indicates > that there are hash partitioning related tests in create_table, > foreign_key, and partition_join files as well. Do we want to use the > custom opclass in those files as well? By the way, let me sug

Re: Postgres stucks in deadlock detection

2018-04-20 Thread Alvaro Herrera
Konstantin Knizhnik wrote: > As I have mentioned at the beginning of this thread the same problem with > deadlock detection timeout expiration we have with YSCB benchmark with zipf > distribution. > Here the source of contention are tuple locks. And as far as I understand > from the discussion in

Re: Foreign keys and partitioned tables

2018-04-20 Thread Alvaro Herrera
Alvaro Herrera wrote: > After wasting some time trying to resolve > "minor last minute issues", I decided to reduce the scope for now: in > the current patch, it's allowed to have foreign keys in partitioned > tables, but it is not possible to have foreign keys that po

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-20 Thread Alvaro Herrera
Amit Langote wrote: > Sorry, I should have said what I said after quoting only the last sentence > of what you had said. That is, I want to the new GUC to be the only > determiner of whether the pruning occurs or not for partitioned tables. > To implement that behavior, it will have to override t

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-20 Thread Alvaro Herrera
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index fa92ce2e68..c51a9270e4 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -951,6 +951,15 @@ static struct config_bool ConfigureNamesBool[] = true, NULL, NULL,

Re: minor fix for CloneRowTriggersToPartition

2018-04-23 Thread Alvaro Herrera
Amit Langote wrote: > Hi. > > I think we should apply the attached patch so that a CreateTriggerStmt > that CloneRowTriggersToPartition creates for a partition doesn't contain > pointers that point to the information in the parent table's relcache, > which may go stale before the pointers in quest

Re: minor fix for acquire_inherited_sample_rows

2018-04-23 Thread Alvaro Herrera
Hello Amit Amit Langote wrote: > acquire_inherited_sample_rows() currently uses equalTupleDescs() being > false as the condition for going to tupconv.c to determine whether tuple > conversion is needed. But equalTupleDescs() will always return false if > it's passed TupleDesc's of two different

Re: Oddity in tuple routing for foreign partitions

2018-04-23 Thread Alvaro Herrera
Robert, I think this is your turf, per 3d956d9562aa. Are you looking into it? Thanks, -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Make description of heap records more talkative for flags

2018-04-23 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Apr 12, 2018 at 08:49:03PM -0700, Andres Freund wrote: > > OTOH, that also kinda bloats the output noticeably... I'm somewhat > > inclined to just put the hex value or such there? > > That would do as well for me. Me too. Should we consider this for pg11? My vot

Re: perltidy version

2018-04-23 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 3/5/18 09:02, Magnus Hagander wrote: > > I think we should just pick some recent one and use it for X years; use > > that one for all backbranches.  I propose X=3.  I propose 20170521 > > (newer ones seem to cater for stuff that I think we mostly don't use).

Re: Make description of heap records more talkative for flags

2018-04-23 Thread Alvaro Herrera
Hello Andres Freund wrote: > On 2018-04-23 12:37:20 -0300, Alvaro Herrera wrote: > > Michael Paquier wrote: > > > On Thu, Apr 12, 2018 at 08:49:03PM -0700, Andres Freund wrote: > > > > OTOH, that also kinda bloats the output noticeably... I'm > > > &

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-23 Thread Alvaro Herrera
Hi, I just pushed David patch, with some pretty minor changes. I hope not to have broken anything. Amit Langote wrote: > Your proposed changes to inheritance_planner() look fine to me. In the > comment added by the patch in relation_excluded_by_constraints(): > > + /* > + * When constraint_ex

Re: [HACKERS] Runtime Partition Pruning

2018-04-23 Thread Alvaro Herrera
Anybody wanna argue against pushing this patch now? I'm inclined to push it on the grounds of being closure for already committed work, but there are possible arguments about this being new development after feature freeze. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Amit Langote wrote: > That's neat! Definitely agree that we should call ExecInitExpr just once > here. The patch looks good too, except the long line. How about this as a small tweak? Determine the array index using a macro, which serves as documentation. -- Álvaro Herrerahtt

Re: [HACKERS] Runtime Partition Pruning

2018-04-24 Thread Alvaro Herrera
Pushed. Thanks! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

<    4   5   6   7   8   9   10   11   12   13   >