Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-02-13 Thread Alvaro Herrera
David Rowley wrote: > On 19 January 2018 at 16:00, Kyotaro HORIGUCHI > wrote: > > And I'd like to ask David to check out his mail environment so > > that SPF record is available for his message. > > Will investigate This should be fixed now. Please let us know if you still see problems. -- Ál

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-23 Thread David Rowley
On 23 January 2018 at 23:22, Amit Langote wrote: > On 2018/01/23 15:44, David Rowley wrote: >> Attached is what I had in mind about how to do this. > > Thanks for the delta patch. I will start looking at it tomorrow. Thanks. I've been looking more at this and I've made a few more adjustments in

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-23 Thread Amit Langote
Hi David. On 2018/01/23 15:44, David Rowley wrote: > On 19 January 2018 at 04:03, David Rowley > wrote: >> On 18 January 2018 at 23:56, Amit Langote >> wrote: >>> So, I've been assuming that the planner changes in the run-time pruning >>> patch have to do with selecting clauses (restriction cl

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-22 Thread David Rowley
Hi Amit , On 19 January 2018 at 04:03, David Rowley wrote: > On 18 January 2018 at 23:56, Amit Langote > wrote: >> So, I've been assuming that the planner changes in the run-time pruning >> patch have to do with selecting clauses (restriction clauses not >> containing Consts and/or join clauses)

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-19 Thread David Rowley
On 19 January 2018 at 16:00, Kyotaro HORIGUCHI wrote: > And I'd like to ask David to check out his mail environment so > that SPF record is available for his message. Will investigate -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-18 Thread Kyotaro HORIGUCHI
Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender > Address Forgery]Re: [HACKERS] path toward faster partition pruning > as its subject, whereas the mail you're replying to only had > Subject: Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: > [HACKER

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-18 Thread Andres Freund
Hi Amit, It seems your mail system continually adds "[Sender Address Forgery]" prefixes to messages. E.g. this mail now has Subject: Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning as its subject, w

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-18 Thread David Rowley
On 18 January 2018 at 23:56, Amit Langote wrote: >> I've not fully worked out how run-time pruning >> will use this as it'll need another version of >> get_partitions_from_clauses but passes in a PartScanClauseInfo >> instead, and does not call extract_partition_key_clauses. That area >> probably

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-17 Thread David Rowley
On 18 January 2018 at 00:13, David Rowley wrote: > On 17 January 2018 at 23:48, Amit Langote wrote: >> I'm concerned that after your patch to remove >> match_clauses_to_partkey(), we'd be doing more work than necessary in >> some cases. For example, consider the case of using run-time pruning >>

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-17 Thread David Rowley
On 17 January 2018 at 23:48, Amit Langote wrote: > I'm concerned that after your patch to remove > match_clauses_to_partkey(), we'd be doing more work than necessary in > some cases. For example, consider the case of using run-time pruning > for nested loop where the inner relation is a partition

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-17 Thread Amit Langote
Hi David. On Wed, Jan 17, 2018 at 6:19 PM, David Rowley wrote: > On 17 January 2018 at 17:05, David Rowley > wrote: >> 6. Which brings me to; why do we need match_clauses_to_partkey at all? >> classify_partition_bounding_keys seems to do all the work >> match_clauses_to_partkey does, plus more.

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-17 Thread David Rowley
On 17 January 2018 at 17:05, David Rowley wrote: > 6. Which brings me to; why do we need match_clauses_to_partkey at all? > classify_partition_bounding_keys seems to do all the work > match_clauses_to_partkey does, plus more. Item #3 above is caused by > an inconsistency between these functions. W

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-16 Thread Amit Langote
Hi David. On Wed, Jan 17, 2018 at 12:32 PM, David Rowley wrote: > On 16 January 2018 at 21:08, Amit Langote > wrote: >> On 2018/01/12 12:30, David Rowley wrote: >>> 8. The code in get_partitions_from_ne_clauses() does perform quite a >>> few nested loops. I think a more simple way to would be t

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-16 Thread David Rowley
On 16 January 2018 at 21:08, Amit Langote wrote: > Attached v20. Thanks again. Thanks for working on v20. I've had a look over part of it and I've written down the following: 1. The following comment is not correct /* * Equality look up key. Values in the following array appear in no * partic

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-16 Thread David Rowley
On 16 January 2018 at 21:08, Amit Langote wrote: > On 2018/01/12 12:30, David Rowley wrote: >> 8. The code in get_partitions_from_ne_clauses() does perform quite a >> few nested loops. I think a more simple way to would be to track the >> offsets you've seen in a Bitmapset. This would save you hav

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-12 Thread Robert Haas
On Thu, Jan 11, 2018 at 10:30 PM, David Rowley wrote: > Instead, can you make it: > > if (keys->n_eqkeys > 0 || keys->n_minkeys > 0 || > keys->n_maxkeys > 0 || n_keynullness > 0) > return true; > > return false; Or better yet: return (keys->n_eqkeys > 0 || keys->n_minkeys > 0 || keys

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-12 Thread David Rowley
On 12 January 2018 at 22:51, Amit Langote wrote: > On 2018/01/12 18:09, David Rowley wrote: >> On 10 January 2018 at 17:18, David Rowley >> wrote: >>> Basically, the changes to add_paths_to_append_rel() are causing >>> duplication in partition_rels. > >> I also noticed earlier that this is still

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-12 Thread Amit Langote
Hi. On 2018/01/12 18:09, David Rowley wrote: > On 10 January 2018 at 17:18, David Rowley > wrote: >> Basically, the changes to add_paths_to_append_rel() are causing >> duplication in partition_rels. [ ... ] > I also noticed earlier that this is still broken in v19. I cannot see the duplicatio

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-11 Thread Amit Langote
David, On 2018/01/12 12:30, David Rowley wrote: > Can you also perform a self-review of the patch? Some of the things > I'm picking up are leftovers from a previous version of the patch. We > might never get through this review if you keep leaving those around! Sorry, I will look more closely bef

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-11 Thread David Rowley
On 12 January 2018 at 15:27, Amit Langote wrote: > On 2018/01/11 19:23, David Rowley wrote: >> ERROR: operator 531 is not a member of opfamily 1976 > > You'll be able to see that the error no longer appears with the attached > updated set of patches, but I'm now seeing that the resulting plan wi

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-11 Thread David Rowley
> On 11 January 2018 at 22:52, Amit Langote > wrote: >> Can you point me to the email where Beena reported the problem in question? > > https://www.postgresql.org/message-id/CAOG9ApERiop7P=grkqqka82aubkjxn3qvixie3wk4wqqpej...@mail.gmail.com To save you from having to look at the run-time prune p

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-11 Thread David Rowley
Thanks for addressing that list. Just one thing to reply on before I look at the updated version: On 11 January 2018 at 22:52, Amit Langote wrote: > On 2018/01/10 10:55, David Rowley wrote: >> One more thing I discovered while troubleshooting a bug Beena reported >> in the run-time partition pru

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-12-13 Thread Amit Langote
Hi David. On 2017/12/13 18:48, David Rowley wrote: > On 12 December 2017 at 22:13, Amit Langote > wrote: >> Attached updated patches. > > Thanks for sending the updated patches. > > I don't have a complete review at the moment, but the following code > in set_append_rel_pathlist() should be rem

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-12-07 Thread Amit Langote
Hi David. On 2017/12/07 19:48, David Rowley wrote: > On 30 November 2017 at 11:15, Robert Haas wrote: >> Committed 0004 after reviewing the code and testing that it seems to >> work as advertised. >> >> 0005 looks like it might need to be split into smaller patches. More >> broadly, the commit m

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-20 Thread Amit Langote
Hi David. On 2017/11/17 21:44, David Rowley wrote: > On 17 November 2017 at 23:01, Amit Langote > wrote: > >> Please find attached updated patch set. There are significant changes in >> this version as described below, including the support for hash >> partitioned tables. >> > > Hi Amit, > >

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-14 Thread Amit Langote
Hi David. On 2017/11/14 13:00, David Rowley wrote: > On 13 November 2017 at 22:46, Amit Langote wrote: >> On 2017/11/10 12:30, Kyotaro HORIGUCHI wrote: >>> The following uses a bit tricky bitmap operation but >>> is straightforward as a whole. >>> >>> = >>> /* fill the bits upper from BITNUM(l