Re: [HACKERS] Aggregates push-down to partitions

2017-11-10 Thread Ashutosh Bapat
g/message-id/CAJ3gD9dy0K_E8r727heqXoBmWZ83HwLFwdcaSSmBQ1%2BS%2BvRuUQ%40mail.gmail.com You may want to review [2] and [3] as well. [2] https://www.postgresql.org/message-id/9666.1491295317@localhost [3] https://www.postgresql.org/message-id/CAM2+6=V64_xhstVHie0Rz=kpeqnljmzt_e314p0jat_oj9m...@mail.gmail.com -

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2017-11-08 Thread Ashutosh Bapat
e can order all clauses in largest set once and use that order every time. Albeit we will have to remember the order somewhere OR make the separator routine retain the order in the larger set, which I guess is true about all separator functions. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2017-11-06 Thread Ashutosh Bapat
ould also benefit > (having perhaps already ordered the qual clauses earlier). +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-11-05 Thread Ashutosh Bapat
g partitions by their bounds somehow, DEFAULT partition listing would be tackled anyway. So, may be we should leave it outside the scope of this patch. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From 398003b2d5f6b54e6cdd8542f653786987ef3bfe Mon Sep 17

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-11-03 Thread Ashutosh Bapat
ch and show the number of partitions. While your suggestion is a valid request, it's kind of beyond the scope of this patch. Someone might want to extend this request and say that partitions should be listed in the order of their bounds (I do feel that we should do some effort in that directio

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Ashutosh Bapat
PARTITION BY to change partition keys (albeit at huge cost of data movement). If we do that, we will have to remember this one-off instance of code which assumes that the partition keys are immutable. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent

Re: [HACKERS] Account for cost and selectivity of HAVING quals

2017-11-01 Thread Ashutosh Bapat
On Wed, Nov 1, 2017 at 8:41 PM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Wed, Nov 1, 2017 at 3:15 AM, Tom Lane wrote: >>> here's a patch to fix the planner so that eval costs and selectivity of >>> HAVING quals are factored into the appropriate plan node n

Re: [HACKERS] Account for cost and selectivity of HAVING quals

2017-11-01 Thread Ashutosh Bapat
d may be we should try to explain why can we ignore selectivity. Similarly for the changes in create_minmaxagg_path(). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-30 Thread Ashutosh Bapat
rocess hangs in a connection to a foreign server etc. I am not sure that the complexity is worth the half-guarantee. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-30 Thread Ashutosh Bapat
On Fri, Oct 27, 2017 at 1:36 AM, Tom Lane wrote: > David Rowley writes: >> On 27 October 2017 at 01:44, Ashutosh Bapat >> wrote: >>> I think Antonin has a point. The join processing may deem some base >>> relations dummy (See populate_joinrel_with_paths()). So

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-29 Thread Ashutosh Bapat
On Fri, Oct 27, 2017 at 12:49 AM, David Rowley wrote: > On 27 October 2017 at 01:44, Ashutosh Bapat > wrote: >> I think Antonin has a point. The join processing may deem some base >> relations dummy (See populate_joinrel_with_paths()). So an Append >> relation which had

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-26 Thread Ashutosh Bapat
h_paths()). So an Append relation which had multiple child alive at the end of set_append_rel_size() might ultimately have only one child after partition-wise join has worked on it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-26 Thread Ashutosh Bapat
iour but I'd like to > share the current status of the patch. The patch includes regression > tests but not includes fully documentation. Any background worker, backend should be child of the postmaster, so we should not let a backend start a resolver process. It should be the job of the post

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-26 Thread Ashutosh Bapat
join. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Is anything preventing us from allowing write to foreign tables from standby?

2017-10-26 Thread Ashutosh Bapat
stency, atomicity of transactions writing to foreign server, we will need to create local transactions. Will that be possible on standby? Obviously, we could add a restriction that no consistency and atomic commit is guaranteed when foreign servers are written from a standby. I am not sure how ea

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-16 Thread Ashutosh Bapat
Sorry for not noticing this problem earlier. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From eca9e03410b049bf79d767657ad4d90fb974d19c Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 16 Oct 2017 13:23:49 +0530 Subject: [PATCH] Ignore d

Re: [HACKERS] [POC] hash partitioning

2017-10-16 Thread Ashutosh Bapat
On Mon, Oct 16, 2017 at 2:36 PM, Ashutosh Bapat wrote: > > Probably we should move changes to partition_bounds_copy() in 0003 to > 0001, whose name suggests so. > We can't do this, hash partition strategy is introduced by 0002. Sorry for the noise. -- Best Wishes, Ashutosh B

Re: [HACKERS] [POC] hash partitioning

2017-10-16 Thread Ashutosh Bapat
On Tue, Oct 10, 2017 at 4:37 PM, amul sul wrote: > On Tue, Oct 10, 2017 at 3:42 PM, Ashutosh Bapat > wrote: >> On Tue, Oct 10, 2017 at 3:32 PM, amul sul wrote: >> >>>> +hash_part? tr

Re: [HACKERS] How does postgres store the join predicate for a relation in a given query

2017-10-15 Thread Ashutosh Bapat
nts are applicable in your case and also see how the callers handle the return values. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2017-10-12 Thread Ashutosh Bapat
On Thu, Oct 12, 2017 at 9:46 PM, Robert Haas wrote: > On Wed, Oct 11, 2017 at 7:08 AM, Ashutosh Bapat > wrote: >> Here's updated patch set based on the basic partition-wise join >> committed. The patchset applies on top of the patch to optimize the >> case o

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-12 Thread Ashutosh Bapat
On Thu, Oct 12, 2017 at 10:49 PM, Robert Haas wrote: > On Wed, Oct 11, 2017 at 10:43 PM, Ashutosh Bapat > wrote: >> You are suggesting that a dummy partitioned table be treated as an >> un-partitioned table and apply above suggested optimization. A join >> between a parti

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-11 Thread Ashutosh Bapat
On Wed, Oct 11, 2017 at 7:47 PM, Robert Haas wrote: > On Mon, Oct 9, 2017 at 2:05 AM, Ashutosh Bapat > wrote: >> On Sat, Oct 7, 2017 at 1:04 AM, Robert Haas wrote: >>> Committed. I hope that makes things less red rather than more, >>> because I'm goi

Re: [HACKERS] How does postgres store the join predicate for a relation in a given query

2017-10-10 Thread Ashutosh Bapat
tInfo associated with it. Predicates applicable to it are stored in this RelOptInfo as a list. For base relations (simple tables) it's in baserestrictinfo. The join predicates applicable are in joininfo. You can get RelOptInfo of a given simple table using find_base_rel(). HTH. --

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread Ashutosh Bapat
hink the dimension(dim) is also unfit here. Any suggestions? > I think natts is ok, since we are dealing with the number of attributes in the pack of datums; esp. when ndatums is already taken. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread Ashutosh Bapat
Unless I am missing something, I don't think we should worry about parttyplen > because in the datumCopy() when the datatype is pass-by-value then typelen > is ignored. That's true, but it's ugly, passing typbyvalue of one type and len of other. -- Best Wishes, Ashutosh Bapat Ente

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-10 Thread Ashutosh Bapat
gt; additional cost is consistent, but it's probably not worth being too > critical here. > This looks good to me. I think it should be a separate, yet very small patch. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via p

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-09 Thread Ashutosh Bapat
spent in Append node as a fraction of say time spent in child seq scans. That might give us a clue as to how Append processing can be charged in terms of costing. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-10-09 Thread Ashutosh Bapat
e length of type of modulus and remainder. Is that correct? Probably we need some special handling wherever parttyplen and parttypbyval is used e.g. in call to partition_bounds_equal() from build_joinrel_partition_info(). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Data

Re: [HACKERS] PoC: full merge join on comparison clause

2017-10-09 Thread Ashutosh Bapat
ro.com > The Russian Postgres Company > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] separate serial_schedule useful?

2017-10-09 Thread Ashutosh Bapat
nforce the rule. > > Concretely, how about the attached? (Obviously we'd have to fix > parallel_schedule before committing this.) > Thanks, this will help. May be we should set default to 20 instead of unlimited. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Databa

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-10-09 Thread Ashutosh Bapat
> command followed by ALTER INDEX abstract_index ATTACH PARTITION > concrete_index or something of that sort. That means you can't > absolutely count on the parent index to have all of the children it's > supposed to have but maybe that's OK. +1. How about CREATE I

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-08 Thread Ashutosh Bapat
of a partitioned join can be considered partitioned. (This wasn't the case earlier when dummy relation was involved.). So, we can allocate the child-join RelOptInfo array in build_joinrel_partition_info(), instead of waiting for an appropriate pair to arrive in try_partition_wise_join(). --

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-06 Thread Ashutosh Bapat
on. But I think, the bug could cause crash as well. The fix is to copy the relevant partitioning information from relcache into PartitionSchemeData and RelOptInfo. Here's a quick patch with that fix. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Compa

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-06 Thread Ashutosh Bapat
On Fri, Oct 6, 2017 at 8:45 PM, Robert Haas wrote: > On Fri, Oct 6, 2017 at 8:40 AM, Ashutosh Bapat > wrote: >> Sorry. I sent a wrong file. Here's the real v37. > > Committed 0001-0006. I made some assorted comment and formatting > changes and two small sub

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-05 Thread Ashutosh Bapat
> > +* Strictly speaking, partition keys of an OUTER join should include > +* partition key expressions from the OUTER side only. Consider a join > > I would join this with the previous sentence instead of repeating > strictly speaking: ...and thus the partition keys should include > partition key expressions from the OUTER side only. After that > sentence, I'd skip a lot of the intermediate words here and continue > this way: However, because all commonly-used comparison operators are > strict, the presence of nulls on the outer side doesn't cause any > problem; they can't match anything at future join levels anyway. > Therefore, we track two sets of expressions: those that authentically > partition the relation (partexprs) and those that partition the > relation with the exception that extra nulls may be present > (nullable_partexprs). When the comparison operator is strict, the > latter is just as good as the former. > > Then, I think you can omit the rest of what you have; it should be > clear enough what's going on for the full and right cases given that > explanation. I liked this version. Changed the comments as per your suggestions. > > + * being joined. partexprs and nullable_partexprs are arrays > containing part_scheme->partnatts > > Long line, needs reflowing. Done. Also fixed a grammatical mistake: contains -> contain in the last line of that paragraph. > > I don't think this is too far from being committable. You've done > some nice work here! > Thanks a lot for your detailed reviews and guidance. I will post the updated patchset with my next reply. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-05 Thread Ashutosh Bapat
"_" when syntax doesn't allow the first two. I am not against "partitionwise" but I don't see any real reason why we should move away from popular usage of this term. [1] https://users.cs.duke.edu/~shivnath/papers/sigmod295-herodotou.pdf -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-05 Thread Ashutosh Bapat
On Wed, Oct 4, 2017 at 9:01 PM, Robert Haas wrote: > On Thu, Sep 21, 2017 at 8:52 AM, Robert Haas wrote: >> On Thu, Sep 21, 2017 at 8:21 AM, Ashutosh Bapat >> wrote: >>> About your earlier comment of making build_joinrel_partition_info() >>> simpler. Right now

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-04 Thread Ashutosh Bapat
On Wed, Oct 4, 2017 at 5:32 PM, Robert Haas wrote: > On Wed, Oct 4, 2017 at 6:40 AM, Ashutosh Bapat > wrote: >> Just like the local constraints on a foreign table are not ensured on >> remote table (unless user takes steps to make that sure), WCO defined >> locally need

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-04 Thread Ashutosh Bapat
like it, adapt if > you sorta like it, or replace if you dislike it. The table names at > least should be changed to something less likely to duplicate other > tests. > There are tests for multi-level partitioned table in the file. They test whole partition hierarchy join, part of i

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-04 Thread Ashutosh Bapat
On Wed, Oct 4, 2017 at 3:45 PM, Etsuro Fujita wrote: > On 2017/10/03 18:16, Ashutosh Bapat wrote: >> >> Enforcing WCO constraints imposed by the local server on the row/DML >> being passed to the foreign server is fine, but trying to impose them >> on the row being inse

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-03 Thread Ashutosh Bapat
ks odd. May be we should just leave this case as it is. I am comparing this case with the way we handle constraints on a foreign table. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-02 Thread Ashutosh Bapat
want to do. > +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: full merge join on comparison clause

2017-10-02 Thread Ashutosh Bapat
al_cost_mergejoin(). Not sure if that > approximation makes any sense, but this is the best I could think of. > > Style problems are fixed. > > Attached please find the new version of the patch that addresses all the > review comments except mergeopfamilies. > > The current co

Re: [HACKERS] Partition-wise aggregation/grouping

2017-09-28 Thread Ashutosh Bapat
On Wed, Sep 27, 2017 at 3:42 PM, Jeevan Chalke wrote: > Thanks Ashutosh for reviewing. > > Attached new patch-set with following changes: > > 1. Removed earlier 0007 and 0008 patches which were PoC for supporting > partial aggregation over fdw. I removed them as it will be a different > issue alto

Re: [HACKERS] PartitionSchemaData & partcollation (Re: [COMMITTERS] pgsql: Associate partitioning information with each RelOptInfo.)

2017-09-28 Thread Ashutosh Bapat
r partition pruning without constraint >> exclusion. >> >> Ashutosh Bapat, reviewed by Amit Langote and with quite a few changes, >> mostly cosmetic, by me. Additional review and testing of this patch >> series by Antonin Houska, Amit Khandekar, Rafia Sabih, Rajkumar >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-09-27 Thread Ashutosh Bapat
remote server. But even if that's not the >> scenario, I think we should assume that the DBA or some other system >> resolved it and therefore we don't need to do anything further. If we >> assume anything else, then we just go into an infinite error loop, >> w

Re: [HACKERS] Partition-wise aggregation/grouping

2017-09-26 Thread Ashutosh Bapat
- > Jeevan Chalke > Principal Software Engineer, Product Development > EnterpriseDB Corporation > The Enterprise PostgreSQL Company > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > ht

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-22 Thread Ashutosh Bapat
On Fri, Sep 15, 2017 at 5:29 PM, Ashutosh Bapat wrote: > >> >> Apart from these there is a regression case on a custom table, on head >> query completes in 20s and with this patch it takes 27s. Please find >> the attached .out and .sql file for the output and

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-22 Thread Ashutosh Bapat
when the tables are partitioned. Am I interpreting the results correctly? There may be some other way of partitioning, which may give better results, but I think what we have now shows the importance of partitioning in case of very large data e.g. scale 300 TPCH. -- Best Wishes, Ashutosh Bapat E

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-21 Thread Ashutosh Bapat
Worker 3: actual time=0.927..3130.497 rows=475545 loops=1 If we sum the rows returned by each worker they don't add up to (actual rows) * (actual loops). So I assumed that the unreported number of rows were processed by the leader. Is that right? I might be m

Re: [HACKERS] [Proposal] Make the optimiser aware of partitions ordering

2017-09-21 Thread Ashutosh Bapat
the sort keys match partition keys by creating an Append path by picking sorted paths from partition RelOptInfos in order. You may use slightly modified version of add_paths_to_append_rel(). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-20 Thread Ashutosh Bapat
On Tue, Sep 19, 2017 at 3:17 PM, Ashutosh Bapat wrote: >> >>>> - I'm not entirely sure whether maintaining partexprs and >>>> nullable_partexprs is the right design. If I understand correctly, >>>> whether or not a partexpr is nullable is reall

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-20 Thread Ashutosh Bapat
ningful out of this. May be there's some faster way by looking at the lines that are covered by B but not A. BTW, I checked those lines to see if there could be any bug there. But I don't see what could go wrong with those lines. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation T

Re: [HACKERS] Varying results when using merge joins over postgres_fdw vs hash joins

2017-09-20 Thread Ashutosh Bapat
_en_us_utf8_encoding e > (cost=100.00..110.67 rows=1 width=1548) (actual time=8.289..144.210 > rows=33418 loops=1) >Output: e.str1, e.str2, e.str3 >Remote SQL: SELECT str1, str2, str3 FROM > public.table_with_en_us_utf8_encoding WHERE ((str4 = '2'::text)) > Planning time: 0.153 ms > Execution time: 156.557 ms > (13 rows) > > In this case, both tables use same collation while comparing the rows, so result is different from the merge join result. Hash join executed on local server and the same executed on foreign server (by importing local table to the foreign server) would also differ. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: full merge join on comparison clause

2017-09-19 Thread Ashutosh Bapat
ot add {} around a single statement block. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw bug in 9.6

2017-09-19 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 1:10 PM, Etsuro Fujita wrote: > > > I think Tom is reviewing this patch [1]. > I am marking this as ready for committer as I don't have any new comments and possibly other reviewers have also done reviewing it. -- Best Wishes, Ashutosh Bapat EnterpriseD

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-18 Thread Ashutosh Bapat
On Sat, Sep 16, 2017 at 2:53 AM, Robert Haas wrote: > On Fri, Sep 15, 2017 at 6:11 AM, Ashutosh Bapat > wrote: >> Thanks a lot Robert. >> >> Here are rebased patches. > > I didn't get quite as much time to look at these today as I would have > liked, but her

[HACKERS] Re: [COMMITTERS] pgsql: Expand partitioned table RTEs level by level, without flattening

2017-09-18 Thread Ashutosh Bapat
true. The array can contain "dead" relations as well. I have removed the assertion and instead fixed the code to skip anything which is not "base" or "other member rel". I have also added a test to cover dead relations and lateral references in join.sql. -- Best Wishe

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-15 Thread Ashutosh Bapat
/CAFjFpRedUZPa7tKbCLEGK3u5UWdDNQoN=eyfb7ieg5d0d1p...@mail.gmail.com [3] https://www.postgresql.org/message-id/cafjfprejksdcfaeuzjgd79hoetzpz5bkdxljgxr7qznrxx+...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-14 Thread Ashutosh Bapat
lright, done. Thanks for reviewing. > LGTM. The patch applies cleanly on the current HEAD, compiles (small bit in regress.c requires compilation), and make check passes. Marking this as "ready for committer". -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Datab

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-14 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 6:58 AM, Thomas Munro wrote: > On Tue, Sep 12, 2017 at 11:23 PM, Ashutosh Bapat > wrote: >> On Wed, Aug 16, 2017 at 11:13 AM, Ashutosh Bapat >> wrote: >>> On Wed, Aug 16, 2017 at 8:44 AM, Alvaro Herrera >>> wrote: >>>>

Re: [HACKERS] <> join selectivity estimate question

2017-09-13 Thread Ashutosh Bapat
On Thu, Sep 14, 2017 at 4:30 AM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Sep 6, 2017 at 11:14 PM, Ashutosh Bapat >> wrote: >>> I added some "stable" tests to your patch taking inspiration from the >>> test SQL file. I think those will be stab

Re: [HACKERS] <> join selectivity estimate question

2017-09-13 Thread Ashutosh Bapat
On Thu, Sep 14, 2017 at 4:19 AM, Thomas Munro wrote: > On Wed, Sep 6, 2017 at 11:14 PM, Ashutosh Bapat > wrote: >> On Fri, Jul 21, 2017 at 4:10 AM, Thomas Munro >> wrote: >>> That just leaves the question of whether we should try to handle the >>> empty

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 12:51 PM, Ashutosh Bapat wrote: > On Wed, Sep 13, 2017 at 12:39 AM, Robert Haas wrote: >> On Tue, Sep 12, 2017 at 3:46 AM, Amit Langote >> wrote: >>> In this case, AcquireExecutorLocks will lock all the relations in >>> PlannedSt

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 1:27 PM, Amit Langote wrote: > On 2017/09/13 16:42, Ashutosh Bapat wrote: >> On Wed, Sep 13, 2017 at 7:49 AM, Amit Langote wrote: >>> In the attached updated patch, I created separate .source files in >>> src/test/regress/input an

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 7:49 AM, Amit Langote wrote: > On 2017/09/12 20:17, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 2:27 PM, Amit Langote >> wrote: >>> Thanks Ashutosh for taking a look at this. >>> >>> On 2017/09/05 21:16, Ashutosh

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
t; > All the TPCH results are posted in the same above mail thread. Can you please check if the attached patch fixes the issue. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company commit 203b3083318e9da41ad614a2ccec532025877c3b Author: Ashutosh Bapat Date:

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 11:29 AM, Amit Langote wrote: > On 2017/09/12 19:56, Ashutosh Bapat wrote: >> I think the code here expects the original parent_rte and not the one >> we set around line 1169. >> >> This isn't a bug right now, since both the parent_rte s ha

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
ts? -- Amit Langote agrees with this. It kind of makes the assertion lame but keeps the code sane. What do you think? [1] https://www.postgresql.org/message-id/d2f1cdcb-ebb4-76c5-e471-79348ca5d...@lab.ntt.co.jp [2] https://www.postgresql.org/message-id/CAFjFpRfJ3GRRmmOugaMA-q4i=se5p6yjz_c6a6hdrdqqtgx...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Constraint exclusion for partitioned tables

2017-09-13 Thread Ashutosh Bapat
exclusion always. For a partitioned table, this patch saves the time to run constraint exclusion on all the partitions if constraint exclusion succeeds on the partitioned table. If constraint exclusion fails, we have wasted CPU cycles on one run of constraint exclusion. The difference between the

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-12 Thread Ashutosh Bapat
for rebasing and improving the patch. Looks >> good to me too. > > Patch needed to be rebased after the default partitions patch went in, so > done. Per build status on http://commitfest.cputube.org :) > > Thanks, > Amit -- Best Wishes, Ashutosh Bapat EnterpriseDB Co

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-12 Thread Ashutosh Bapat
The patch needs to address some comments in the previous mails, so marking it as "waiting for author". On Wed, Aug 16, 2017 at 11:13 AM, Ashutosh Bapat wrote: > On Wed, Aug 16, 2017 at 8:44 AM, Alvaro Herrera > wrote: >> Christoph Berg wrote: >>> Re: Thomas Munro

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 2:27 PM, Amit Langote wrote: > Thanks Ashutosh for taking a look at this. > > On 2017/09/05 21:16, Ashutosh Bapat wrote: >> The patch needs a rebase. > > Attached rebased patch. Thanks for rebased patch. We could annotate each ERROR with an explana

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 2:35 PM, Amit Langote wrote: > On 2017/09/12 17:53, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 1:42 PM, Amit Langote wrote: >>> So, we can remove partitioned_rels from (Merge)AppendPath and >>> (Merge)Append nodes and remove ExecLockNonLeaf

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
/* Include child's rowmark type in parent's allMarkTypes */ parentrc->allMarkTypes |= childrc->allMarkTypes; } so that we push allMarkTypes up the hierarchy. I like the second way, since every intermediate parent records allMarkTypes of its descendants.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 1:42 PM, Amit Langote wrote: > On 2017/09/12 16:55, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 1:16 PM, Amit Langote wrote: >>> So I looked at this a bit closely and came to the conclusion that we may >>> not need to keep partitio

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 1:16 PM, Amit Langote wrote: > On 2017/09/11 21:07, Ashutosh Bapat wrote: >> On Mon, Sep 11, 2017 at 5:19 PM, Robert Haas wrote: >>> On Mon, Sep 11, 2017 at 6:45 AM, Ashutosh Bapat >>> wrote: >>>> So, all partitioned partit

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 7:31 AM, Amit Langote wrote: > On 2017/09/11 19:45, Ashutosh Bapat wrote: >> On Mon, Sep 11, 2017 at 12:16 PM, Amit Langote wrote: >>> IMHO, we should make it the responsibility of the future patch to set a >>> child PlanRowMark's prti to th

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 5:19 PM, Robert Haas wrote: > On Mon, Sep 11, 2017 at 6:45 AM, Ashutosh Bapat > wrote: >> So, all partitioned partitions are getting locked correctly. Am I >> missing something? > > That's not a valid test. In that scenario, you're goin

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Ashutosh Bapat
> with automatic messages to often. I believe that sending such messages > once a day would be enough. > > Unless there are any objections to give this idea a try I'm willing to > write and host a corresponding script. > +1 that would help. -- Best Wishes, Ashutosh Bapat E

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 2:16 PM, Amit Langote wrote: > On 2017/09/11 16:23, Ashutosh Bapat wrote: >> On Sat, Sep 9, 2017 at 6:28 AM, Robert Haas wrote: >>> I'm a bit suspicious about the fact that there are now executor >>> changes related to the PlanRowMarks.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 12:16 PM, Amit Langote wrote: > On 2017/09/09 2:38, Ashutosh Bapat wrote: >> On Fri, Sep 8, 2017 at 11:17 AM, Amit Langote wrote: >>> I updated the patch to include just those changes. I'm not sure about >>> one of the Ashutosh's cha

Re: [HACKERS] WIP: Aggregation push-down

2017-09-11 Thread Ashutosh Bapat
On Fri, Sep 8, 2017 at 7:04 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska wrote: >> > Antonin Houska wrote: >> > >> >> Antonin Houska wrote: >> >> >> >> > This

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Ashutosh Bapat
against this commit & renamed to 0001, PFA. Given that we have default partition support now, I am wondering whether hash partitioned tables also should have default partitions. The way we have structured hash partitioning syntax, there can be "holes" in partitions. Default part

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2017-09-11 Thread Ashutosh Bapat
On Thu, Sep 7, 2017 at 7:34 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> I have fixed the issues which were marked as TODOs in the attached >> patches. Also, I have included your test change patch in my series of >> patches. > > I've noticed that

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Sat, Sep 9, 2017 at 6:28 AM, Robert Haas wrote: > On Fri, Sep 8, 2017 at 1:38 PM, Ashutosh Bapat > wrote: >>> I also confirmed that the partition-pruning patch set works fine with this >>> patch instead of the patch on that thread with the same functionality, >&g

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-08 Thread Ashutosh Bapat
On Fri, Sep 8, 2017 at 12:34 AM, Robert Haas wrote: > On Tue, Sep 5, 2017 at 7:01 AM, Ashutosh Bapat > wrote: >> accumulate_append_subpath() is executed for every path instead of >> every relation, so changing it would collect the same list multiple >> times. Instead

Re: [HACKERS] Adding support for Default partition in partitioning

2017-09-07 Thread Ashutosh Bapat
ed only from DefineRelation(), > and not for alter command. I am not really sure how can we use > work queue for create command. No, we shouldn't use work queue for CREATE command. We should extract the common code into a function to be called from check_default_allows_bound() and ValidatePartitionConstraints(). To that function we pass a flag (or the work queue argument itself), which decides whether to add a work queue item or scan the relation directly. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-07 Thread Ashutosh Bapat
On Thu, Sep 7, 2017 at 4:32 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> On Fri, Sep 1, 2017 at 6:05 PM, Antonin Houska wrote: >> > >> > >> > >> > * get_partitioned_child_rels_for_join() >> > >> > I think the Assert

Re: [HACKERS] WIP: Aggregation push-down

2017-09-07 Thread Ashutosh Bapat
aggregates. 2. Following query throws error with these patches, but not without the patches. explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a = t2.a group by t2.a, t1.a; ERROR: ORDER/GROUP BY expression not found in targetlist [1] https://www.postgresql.org/message-id/CAFj

Re: [HACKERS] <> join selectivity estimate question

2017-09-06 Thread Ashutosh Bapat
; > Please find attached a new version, and a test script I used, which > shows a bunch of interesting cases. I'll add this to the commitfest. I added some "stable" tests to your patch taking inspiration from the test SQL file. I think those will be stable across machines and

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-06 Thread Ashutosh Bapat
27;s a partitioned table. So can you > directly > add the "is partitioned" in place of "has partitions"? > > Did those change in the attached patch and update regression expected > output. > Looks better. > Also run pgindent on the patch. > Thanks fo

Re: [HACKERS] paths in partitions of a dummy partitioned table

2017-09-06 Thread Ashutosh Bapat
On Fri, Aug 25, 2017 at 10:46 PM, Robert Haas wrote: > On Thu, Jul 6, 2017 at 11:35 AM, Ashutosh Bapat > wrote: >> If a partitioned table is proven dummy, set_rel_pathlist() doesn't mark the >> partition relations dummy and thus doesn't set any (dummy) paths in the

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-09-05 Thread Ashutosh Bapat
On Wed, Sep 6, 2017 at 1:32 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, Sep 5, 2017 at 4:02 PM, Jeevan Chalke >> wrote: >>> Attached patch for replacing linitial() with linital_node() appropriately in >>> planner.c > >> Ok. The patch lo

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-05 Thread Ashutosh Bapat
only see one small oversight, which is >> the >> successful case of ALTER FOREIGN DATA WRAPPER ... HANDLER ... is still >> not tested. I added one line for that in the attached modification of your >> patch. > > Ah right, thanks. > > Adding this

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-09-05 Thread Ashutosh Bapat
2a54a03ec90206c7e5579a562a121c Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Tue, 5 Sep 2017 16:50:58 +0530 Subject: [PATCH 1/2] Use lfirst_node() instead of lfirst() wherever suitable in planner.c Ashutosh Bapat, reviewed by Jeevan Chalke --- src/backend/optimizer/plan/planner.c | 94

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-05 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 1:16 PM, Amit Langote wrote: > On 2017/09/05 15:43, Ashutosh Bapat wrote: >> Ok. Can you please answer my previous questions? >> >> AFAIU, the list contained RTIs of the relations, which didnt' have >> corresponding AppendRelInfos to l

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-04 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 12:06 PM, Amit Langote wrote: > On 2017/09/05 15:30, Ashutosh Bapat wrote: >> Those changes are already part of my updated 0001 patch. Aren't they? >> May be you should just review those and see if those are suitable for >> you? > > Yeah, I

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-04 Thread Ashutosh Bapat
e partition-pruning > patch, because the other code in that patch relies on the same and I know > Ashuotsh has wanted that for a long time. :) Those changes are already part of my updated 0001 patch. Aren't they? May be you should just review those and see if those are suitable for you

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-04 Thread Ashutosh Bapat
translatability. I think the > original is fine. > We have used this style in the "else" case of if (!verbose). So, I just copied it. I have removed that change in the attached patch. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From c0a153

  1   2   3   4   5   6   7   8   9   10   >