Re: [HACKERS]WIP: Covering + unique indexes.

2016-01-31 Thread David Rowley
all that comfortable that all the code that needs to be updated has been updated. I'm not quite sure of a good way to find all these places. I wondering if hacking the code so that each btree index which is created with > 1 column puts all but the first column into the

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2016-02-08 Thread David Rowley
On 7/02/2016 4:14 am, "Tom Lane" wrote: > > David Rowley writes: > [ timestamp_out_speedup_2015-11-05.patch ] > > Pushed with a bunch of cosmetic tweaks. Great. Thanks for pushing this. David

Re: [HACKERS] Removing Functionally Dependent GROUP BY Columns

2016-02-14 Thread David Rowley
On 12/02/2016 12:01 am, "Tom Lane" wrote: > > David Rowley writes: > > [ prune_group_by_clause_ab4f491_2016-01-23.patch ] > > [ check_functional_grouping_refactor.patch ] > > I've committed this with mostly-cosmetic revisions (principally, rewriting >

Re: [HACKERS] Removing Functionally Dependent GROUP BY Columns

2016-02-14 Thread David Rowley
On 14/02/2016 5:11 pm, "Tom Lane" wrote: > > David Rowley writes: > > On 12/02/2016 12:01 am, "Tom Lane" wrote: > > I can't quite understand what you're seeing here. > > The second loop is iterating through the original GROUP BY list, so it

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-02-18 Thread David Rowley
On 23/01/2016 12:42 am, "David Rowley" wrote: > > On 23 January 2016 at 05:36, Tomas Vondra wrote: > > Otherwise I think the patch is ready for committer - I think this is a > > valuable optimization and I see nothing wrong with the code. > > > > >

Re: [HACKERS] Random note of encouragement

2016-02-24 Thread David Rowley
any changes to any NUMERIC aggregate functions. It would be interesting to see the explain analyze buffers for both. Perhaps 9.5 just read more buffers from disk than 9.6 did. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Se

Re: [HACKERS] JIT compiling - v4.0

2017-10-05 Thread David Rowley
iorated to some degree, but JITing obviously isn't always going to > be a win. It's pretty exciting to see thing being worked on. I've not looked at the code, but I'm thinking, could you not just JIT if the total cost of the plan is estimated to be > X ? Where

Re: [HACKERS] Discussion on missing optimizations

2017-10-07 Thread David Rowley
and finding performance is not quite what they need. It's a bit sad that often the people with the loudest voices are always so fast to stomp on the ideas for improvements. It would be much nicer if you'd at least wait for benchmarks before shooting. -- David Rowley h

Re: [HACKERS] Discussion on missing optimizations

2017-10-08 Thread David Rowley
t that as relations without unique indexes are pretty uncommon (at least in my world). Thoughts? -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services remove_left_join_distinct.patch Description: Binary data -- Sent via

Re: [HACKERS] Discussion on missing optimizations

2017-10-08 Thread David Rowley
On 9 October 2017 at 17:41, David Rowley wrote: > Thoughts? Actually, I was a little inconsistent with my List NULL/NIL checks in that last one. I've attached an updated patch. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-09 Thread David Rowley
comment in create_append_path() which claims the zero cost is a bit optimistic. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

[HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-09 Thread David Rowley
Removed by Filter: 333300 Planning time: 2.175 ms Execution time: 106.326 ms (8 rows) -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services allow_relabelled_vars_in_dependency_stats.patch Description: Binary data -- Sent via pgsq

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-10 Thread David Rowley
- 2.400209476818 (1 row) Maybe it's worth trying with different row counts to see if the additional cost is consistent, but it's probably not worth being too critical here. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-12 Thread David Rowley
your code assumed the OpExpr was a Var. The reason Tomas coded it the way it was coded is due to the fact that there's already code that works exactly the same way in clauselist_selectivity(). Personally, I don't particularly like that code, but I'd rather not invent a n

Re: [HACKERS] Discussion on missing optimizations

2017-10-12 Thread David Rowley
SELECTs too. I had imagined this would be some backend local cache that saved MRU plans up to some size of memory defined by a GUC, where 0 would disable the feature. I never got any further than those thoughts. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Deve

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-12 Thread David Rowley
On 13 October 2017 at 04:56, Alvaro Herrera wrote: > I pushed your original fix. Thanks for committing -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] Aggregate transition state merging vs. hypothetical set functions

2017-10-12 Thread David Rowley
f the user defines their normal aggregate as not safe for merging, then surely it'll not be suitable to be used as a window function either, since the final function will also be called there multiple times per state. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Aggregate transition state merging vs. hypothetical set functions

2017-10-12 Thread David Rowley
On 13 October 2017 at 12:41, Tom Lane wrote: > David Rowley writes: >> If the user defines their normal aggregate as not safe for merging, >> then surely it'll not be suitable to be used as a window function >> either, since the final function will also be called t

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-13 Thread David Rowley
I don't really think the DEFAULT_APPEND_COST_FACTOR adds much. it means very little by itself. It also seems that most of the other cost functions just use the magic number. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-13 Thread David Rowley
On 14 October 2017 at 09:04, Robert Haas wrote: > On Mon, Oct 9, 2017 at 11:03 PM, David Rowley > wrote: >> -- Unpatched >> Planning time: 0.184 ms >> Execution time: 105.878 ms >> >> -- Patched >> Planning time: 2.175 ms >> Execution time: 1

Re: [HACKERS] Discussion on missing optimizations

2017-10-13 Thread David Rowley
ust speed up the planner. Perhaps I missed some? It looks like there's plenty we could do in there, just nobody seems interested enough to go and do it, everyone who cares about performance is too busy trying to make execution run faster. -- David Rowley http://www.2ndQuadran

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-15 Thread David Rowley
On 15 October 2017 at 06:49, Robert Haas wrote: > On Fri, Oct 13, 2017 at 4:49 PM, David Rowley > wrote: >> tps = 8282.481310 (including connections establishing) >> tps = 8282.750821 (excluding connections establishing) > > vs. > >> tps = 8520.822410 (including

[HACKERS] A handful of typos in allpaths.c

2017-10-17 Thread David Rowley
A small patch to fix these is attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services allpath_typos_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

[HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-25 Thread David Rowley
which would be much cleaner. [1] https://www.postgresql.org/message-id/CAKJS1f9UXdk6ZYyqbJnjFO9a9hyHKGW7B%3DZRh-rxy9qxfPA5Gw%40mail.gmail.com -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-26 Thread David Rowley
On 26 October 2017 at 23:30, Robert Haas wrote: > On Wed, Oct 25, 2017 at 11:59 PM, David Rowley > wrote: >> As of today, because we include this needless [Merge]Append node, we >> cannot parallelise scans below the Append. > > Without disputing your general notion th

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-26 Thread David Rowley
On 26 October 2017 at 23:42, Antonin Houska wrote: > David Rowley wrote: > >> Method 1: >> >> In set_append_rel_size() detect when just a single subpath would be >> added to the Append path. > > I spent some time reviewing the partition-wise join patch during

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-26 Thread David Rowley
| | | plain | | b | integer | | | | plain | | Partition key: RANGE (a) Partitions: ab_p1 FOR VALUES FROM (1) TO (1), ab_p2 FOR VALUES FROM (10000) TO (2) -- David Rowley http:/

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread David Rowley
On 30 October 2017 at 22:34, Andres Freund wrote: > Hi, > > On 2015-12-20 02:49:13 +1300, David Rowley wrote: >> Alternatively, if there was some way to mark the path as cold from within >> the path, rather than from the if() condition before the path, then we >> could

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread David Rowley
On 30 October 2017 at 22:44, Andres Freund wrote: > On 2017-10-30 22:39:01 +1300, David Rowley wrote: >> Today I was thinking, to get around that issue, we might be able to >> generate another thin wrapper around elog_start() and mark that as >> __attribute__((cold)) and fud

[HACKERS] Removing LEFT JOINs in more cases

2017-10-31 Thread David Rowley
andling. A patch to do this is attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001-Support-removing-LEFT-JOINs-with-DISTINCT-GROUP-BY.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsq

[HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
xecutor) Comments on the design are welcome, but I was too late to the commitfest, so there are other priorities. However, if you have a strong opinion, feel free to voice it. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 00

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
of those places could be changed to lappend(), but I bet there's plenty that need prepend. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:38, Tom Lane wrote: > David Rowley writes: >> On 3 November 2017 at 03:17, Tom Lane wrote: >>> We've jacked up the List API and driven a new implementation underneath >>> once before. Maybe it's time to do that again. > >>

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:27, Stephen Frost wrote: > * David Rowley (david.row...@2ndquadrant.com) wrote: >> We could get around a few of these problems if Lists were implemented >> internally as arrays, however, arrays are pretty bad if we want to >> delete an item out the m

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
ab13bf. Can you send patches rebased on current master? Thanks -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
PARTITION_MAX_KEYS * sizeof(NullTestType *)); should be PARTITION_MAX_KEYS * sizeof(NullTestType). It might have worked on your machine if you're compiling as 32 bit. I'll continue on with the review in the next few days. -- David Rowley http://www.2ndQuadrant.com/

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:26, Craig Ringer wrote: > On 2 November 2017 at 22:22, David Rowley > wrote: >> Maybe, but the new implementation is not going to do well with places >> where we perform lcons(). Probably many of those places could be >> changed to lappend(),

Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
On 3 November 2017 at 17:32, David Rowley wrote: > 2. This code is way more complex than it needs to be. > > if (num_parts > 0) > { > int j; > > all_indexes = (int *) palloc(num_parts * sizeof(int)); > j = 0; > if (min_part_idx >= 0 && max_part

Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
owed too. This works slightly differently. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
On 6 November 2017 at 17:30, Amit Langote wrote: > On 2017/11/03 13:32, David Rowley wrote: >> On 31 October 2017 at 21:43, Amit Langote >> wrote: >> 1. This comment seem wrong. >> >> /* >> * Since the clauses in rel->baserestrictinfo should all

[HACKERS] Removing useless DISTINCT clauses

2017-11-06 Thread David Rowley
resql.org/gitweb/?p=postgresql.git;a=commit;h=d4c3a156cb46dcd1f9f97a8011bd94c544079bb5 [2] https://www.postgresql.org/message-id/flat/CAKJS1f9q0j3BgMUsDbtf9%3DecfVLnqvkYB44MXj0gpVuamcN8Xw%40mail.gmail.com#CAKJS1f9q0j3BgMUsDbtf9=ecfvlnqvkyb44mxj0gpvuamcn...@mail.gmail.com -- David Rowley

Re: [HACKERS] path toward faster partition pruning

2017-11-06 Thread David Rowley
the fix to make the new pruning > code handle Boolean partitioning. Thanks. I'll look over it all again starting my Tuesday morning. (UTC+13) -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pg

Re: [HACKERS] path toward faster partition pruning

2017-11-06 Thread David Rowley
On 7 November 2017 at 01:52, David Rowley wrote: > Thanks. I'll look over it all again starting my Tuesday morning. (UTC+13) I have a little more review to share: 1. Missing "in" in comment. Should be "mentioned in" * get_append_rel_partitions * Return the list of

Re: [HACKERS] path toward faster partition pruning

2017-11-07 Thread David Rowley
On 7 November 2017 at 01:52, David Rowley wrote: > Thanks. I'll look over it all again starting my Tuesday morning. (UTC+13) Hi Amit, I had another look over this today. Apologies if any of the review seems petty. Here goes: 1. If test seems to be testing for a child that's

Re: [HACKERS] path toward faster partition pruning

2017-11-09 Thread David Rowley
gt;> (BITS_PER_BITMAPWORD - (BITNUM(upper) - 1)); > = No objections here for making bms_add_range() perform better, but this is not going to work when lwordnum == uwordnum. You'd need to special case that. I didn't think it was worth the trouble, but maybe it is... I assume

[HACKERS] No mention of CREATE STATISTICS in event trigger docs

2017-11-10 Thread David Rowley
A patch to fix this is attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services event_trigger_statistics_doc.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] UPDATE of partition key

2017-11-13 Thread David Rowley
traints() and have it validate all remaining checks. 28. For table WITH OIDs, the OID should probably follow the new tuple for partition-key-UPDATEs. CREATE TABLE p (a BOOL NOT NULL, b INT NOT NULL) PARTITION BY LIST (a) WITH OIDS; CREATE TABLE P_true PARTITION OF p FOR VALUES IN('t'

Re: [HACKERS] Relids in upper relations

2016-10-09 Thread David Rowley
ushed down version. Of course, as mentioned by Robert, both versions would have to be costed in case the join condition was highly selective There's a good paper which goes into a bit more details on this http://www.vldb.org/conf/1995/P345.PDF -- David Rowley http://www.2ndQ

Re: [HACKERS] Improving RLS planning

2016-10-25 Thread David Rowley
one field as a set of bits. [1] https://www.postgresql.org/message-id/cakjs1f9fpdlkm6%3dsuzagwuch3otbspk6k0yt8-a1hgjfapl...@mail.gmail.com -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hac

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-10-30 Thread David Rowley
On 8 April 2016 at 06:49, Tom Lane wrote: > David Rowley writes: > Just had a thought about this, which should have crystallized a long > time ago perhaps. Where I'd originally imagined you were going with > this idea is to do what the thread title actually says, and check for

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-11-01 Thread David Rowley
On 31 October 2016 at 18:37, David Rowley wrote: > I've rebased the changes I made to address this back in April to current > master. Please note that I went ahead and marked this as "Ready for committer". It was previously marked as such in a previous commitfest. The ch

[HACKERS] Making clausesel.c Smarter

2017-02-24 Thread David Rowley
;t stretch far enough to imagine a OpExpr which passes with a NULL operand. If it did my logic is broken, and if that's the case then I think limiting "Others" to mean F_EQSEL and F_NEQSEL would be the workaround. Before I do any more work on this, I'd like to know, is this

[HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-01 Thread David Rowley
Hackers, I've attached a small patch which aims to improve the performance of AccessExclusiveLock when there are many AccessExclusiveLock stored. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services recovery_lock_s

Re: [HACKERS] Possible spelling fixes

2017-03-01 Thread David Rowley
tive (O(N^2)) know should be known. Perhaps you can include if you have a followup patch. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-03-05 Thread David Rowley
reused. I also ended up shifting out the code which processes the table options so that it is consistent. Reviews from people a bit closer to the foreign join pushdown code are welcome. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-03-05 Thread David Rowley
On 6 March 2017 at 18:51, Etsuro Fujita wrote: > On 2017/03/06 11:05, David Rowley wrote: >> The attached patch, based on 9.6, fixes the problem by properly >> processing the foreign server options in >> postgresGetForeignJoinPaths(). > > I think the fix would work we

[HACKERS] Small fix to postgresql.conf.sample's comment on max_parallel_workers

2017-03-06 Thread David Rowley
While scanning over postgresql.conf I happened to notice something that didn't ring quite true about max_parallel_workers. The comment confuses worker_processes with parallel workers. The attached aims to put this right. -- David Rowley http://www.2ndQuadrant.com/ Postg

Re: [HACKERS] Small fix to postgresql.conf.sample's comment on max_parallel_workers

2017-03-06 Thread David Rowley
max_parallel_workers", maybe. You can't "take" more than what's there, so perhaps the extra text is not required. Patch attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services postgresql.c

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-06 Thread David Rowley
On 2 March 2017 at 16:06, Amit Kapila wrote: > On Wed, Mar 1, 2017 at 5:32 PM, David Rowley > wrote: >> Hackers, >> >> I've attached a small patch which aims to improve the performance of >> AccessExclusiveLock when there are many AccessExclusiveLock stored. &

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread David Rowley
On 7 March 2017 at 17:31, David Rowley wrote: > On 2 March 2017 at 16:06, Amit Kapila wrote: >> Few comments on the patch: >> 1. >> +/* >> + * Number of buckets to split RecoveryLockTable into. >> + * This must be a power of two. >> + */ >> >>

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread David Rowley
On 7 March 2017 at 23:20, Simon Riggs wrote: > On 7 March 2017 at 10:01, David Rowley wrote: >> On 2 March 2017 at 16:06, Amit Kapila wrote: >>> On Wed, Mar 1, 2017 at 5:32 PM, David Rowley >>> wrote: >>>> Hackers, >>>> >>>> I

Re: [HACKERS] Small fix to postgresql.conf.sample's comment on max_parallel_workers

2017-03-07 Thread David Rowley
On 8 March 2017 at 09:32, Robert Haas wrote: > Committed. Thanks! -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread David Rowley
help with that today. Do you think I should introduce some new global variable for that? Or do you propose calling GetRunningTransactionLocks() again while generating the Commit/Abort record? -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Trai

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-03-07 Thread David Rowley
> if generation.c is a too generic filename. > > And pushed slab and its usage. Will have a look at generation.c > tomorrow. Attached is a patch to fix the compiler warning for compilers that don't understand elog(ERROR) does not return. -- David Rowley http:/

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-03-08 Thread David Rowley
On 7 March 2017 at 01:22, Ashutosh Bapat wrote: > On Mon, Mar 6, 2017 at 1:29 PM, David Rowley > wrote: >> On 6 March 2017 at 18:51, Etsuro Fujita wrote: >>> On 2017/03/06 11:05, David Rowley wrote: >> It seems like a much better idea to keep the server option pr

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-03-09 Thread David Rowley
gt;> The changes from yours are mostly cosmetic, but I've also added a >> regression test too. > > Thanks a lot for the test. > > PFA updated patch. Thanks for making the chances. I'd say it's ready to go, pending updating the out of date comment: + /* We mu

[HACKERS] Parallel Bitmap scans a bit broken

2017-03-09 Thread David Rowley
} } it fails, due to idxpages pointing to random memory Probably this is a simple fix for the authors, so passing it along. I'm a bit unable to see how the part above is meant to work. [1] https://www.postgresql.org/message-id/attachment/50164/brin-correlation-v3.patch -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-09 Thread David Rowley
ator. > > I don't know if this is the only problem -- it would be good if David > could retest -- but it's certainly *a* problem, so committed. > Thanks for committing, and generally parallelising more stuff. I confirm that my test case is now working again. I'll be in this general area today, so will mention if I stumble over anything that looks broken. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] BRIN cost estimate

2017-03-12 Thread David Rowley
e index is unlikely to help. So +1 for fixing this issue. Will Emre be around to make the required changes to the patch? I see it's been a while since it was originally posted. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] [NOVICE] Why is there a doubtful copyObject call in add_vars_to_targetlist

2017-03-12 Thread David Rowley
ly wouldn't buy that much performance wise is likely the reason it's still there. Probably if someone came up with a realistic enough case to prove that it was worth removing, then someone might take some time to go and check if it was safe to remove. There's a good chance that it&

Re: [HACKERS] multivariate statistics (v25)

2017-03-13 Thread David Rowley
finition of the system "extended statistic" relation (pg_statistic_ext) + * along with the relation's initial contents. + * + * + * Portions Copyright (c) 1996-2014, PostgreSQL Global Development Group 2017 + * stats.h + * Multivariate statistics and selectivity estimation funct

Re: [HACKERS] Performance improvement for joins where outer side is unique

2017-03-13 Thread David Rowley
On 14 March 2017 at 07:50, Tom Lane wrote: > [ getting back to this patch finally... ] > > David Rowley writes: > > I've attached a patch which implements this, though only for > > MergeJoin, else I'd imagine we'd also need to ensure all proofs used > >

Re: [HACKERS] Performance improvement for joins where outer side is unique

2017-03-13 Thread David Rowley
On 14 March 2017 at 11:35, David Rowley wrote: > On 14 March 2017 at 07:50, Tom Lane wrote: > >> [ getting back to this patch finally... ] >> >> David Rowley writes: >> > I've attached a patch which implements this, though only for >> > MergeJoin

Re: [HACKERS] multivariate statistics (v25)

2017-03-14 Thread David Rowley
On 13 March 2017 at 23:00, David Rowley wrote: > > 0003: > > No more time today. Will try and get to those soon. > 0003: I've now read this patch. My main aim here was to learn what it does and how it works. I need to spend much longer understanding how your calcul

Re: [HACKERS] multivariate statistics (v25)

2017-03-14 Thread David Rowley
;s a bug and should be fixed. I found a few of these and listed them during my review. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-15 Thread David Rowley
On 16 March 2017 at 13:31, Simon Riggs wrote: > On 8 March 2017 at 10:02, David Rowley > wrote: > > On 8 March 2017 at 01:13, Simon Riggs wrote: > >> Don't understand this. I'm talking about setting a flag on > >> commit/abort WAL records, like the att

Re: [HACKERS] multivariate statistics (v25)

2017-03-16 Thread David Rowley
at in the attached patch. I followed how pg_dump works for indexes, and created pg_get_statisticsextdef() in ruleutils.c. I was unsure if I should be naming this pg_get_statisticsdef() instead. I also noticed there's no COMMENT ON support either, so I added that too. -- David Rowle

Re: [HACKERS] multivariate statistics (v25)

2017-03-16 Thread David Rowley
pg_statisticsextdef(), but I didn't see a suitable location. stats_ext.sql would have been a good spot. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-19 Thread David Rowley
robably I misunderstood this again. Let me know if that's the case. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-19 Thread David Rowley
On 17 March 2017 at 00:04, Amit Kapila wrote: > On Thu, Mar 16, 2017 at 7:22 AM, David Rowley > wrote: >> On 16 March 2017 at 13:31, Simon Riggs wrote: >>> >>> On 8 March 2017 at 10:02, David Rowley >>> wrote: >>> > On 8 March 2017 at 01:1

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-20 Thread David Rowley
unction will actually be called with an invalid xid. Patch series is attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001-Speed-up-replay-of-Access-Exclusive-Locks-on-hot-sta.patch Description: Binary data

Re: [HACKERS] extended statistics: n-distinct

2017-03-20 Thread David Rowley
ot;select", insert) Statistics: "public.mytable_stats" WITH (ndistinct) ON (select, insert) notice lack of quoting on 'select'. List *keys; /* String nodes naming referenced columns */ Are these really keys? Sounds more like something you might call it if it wer

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Mon, Jun 9, 2014 at 11:20 PM, Marti Raudsepp wrote: > On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT > EXISTS > > queries and leaves NOT IN alone. The reason for this is because the > val

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
query to see if there's any not null quals. I'm about to put this to the test, but if it works then I think it should cover many more cases for using NOT IN(), I guess we're only leaving out function calls and calculations in the target list. Regards David Rowley

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Wed, Jun 11, 2014 at 9:32 PM, Marti Raudsepp wrote: > On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT > EXISTS > > queries and leaves NOT IN alone. The reason for this is because the > val

Re: [HACKERS] Window function optimisation, allow pushdowns of items matching PARTITION BY clauses

2014-06-17 Thread David Rowley
#x27;ve updated the expected results to remove the extra sort from the explain output Regards David Rowley wfunc_pushdown_partitionby_v0.3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Allowing join removals for more join types

2014-06-17 Thread David Rowley
lue,COUNT(*) FROM t2 GROUP BY value) t2 ON t1.id = t2.value; Regards David Rowley subquery_leftjoin_removal_v1.1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Window function optimisation, allow pushdowns of items matching PARTITION BY clauses

2014-06-21 Thread David Rowley
On 21 June 2014 01:38, Vik Fearing wrote: > I've had a chance to look at this and here is my review. > > On 04/14/2014 01:19 PM, David Rowley wrote: > > I've included the updated patch with some regression tests. > > The first thing I noticed is there is no docume

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-24 Thread David Rowley
On Wed, Jun 11, 2014 at 9:32 PM, Marti Raudsepp wrote: > On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT > EXISTS > > queries and leaves NOT IN alone. The reason for this is because the > val

Re: [HACKERS] Allowing join removals for more join types

2014-06-26 Thread David Rowley
On Sun, Jun 22, 2014 at 11:51 PM, Simon Riggs wrote: > On 17 June 2014 11:04, David Rowley wrote: > > On Wed, Jun 4, 2014 at 12:50 AM, Noah Misch wrote: > >> > >> As a point of procedure, I recommend separating the semijoin support > into > >> its >

Re: [HACKERS] Allowing join removals for more join types

2014-06-26 Thread David Rowley
On Wed, Jun 25, 2014 at 10:03 AM, Simon Riggs wrote: > On 23 June 2014 12:06, David Rowley wrote: > > >> It's not clear to me where you get the term "sortclause" from. This is > >> either the groupclause or distinctclause, but in the test cases you >

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-26 Thread David Rowley
that the target entry comes from a left join, then would it be a bit too quirky to just do the NOT NULL checking when list_length(query->rtable) == 1 ? or maybe even loop over query->rtable and abort if we find an outer join type? it seems a bit hackish, but perhaps it would please more people

Re: [HACKERS] Window function optimisation, allow pushdowns of items matching PARTITION BY clauses

2014-06-28 Thread David Rowley
On 28 June 2014 18:12, Tom Lane wrote: > David Rowley writes: > > [ wfunc_pushdown_partitionby_v0.4.patch ] > > I've committed this with the addition of a volatility check and some > other basically-cosmetic adjustments. > > Great, thank you for making the required

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-29 Thread David Rowley
On Fri, Jun 27, 2014 at 6:14 AM, Tom Lane wrote: > David Rowley writes: > > If there's no way to tell that the target entry comes from a left join, > > then would it be a bit too quirky to just do the NOT NULL checking when > > list_length(query->rtable) == 1 ?

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-07-02 Thread David Rowley
likely delay posting the updated patch until I have the other fix in place. Regards David Rowley Thanks > > -- > Jeevan B Chalke > Principal Software Engineer, Product Development > EnterpriseDB Corporation > The Enterprise PostgreSQL Company > >

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-07-05 Thread David Rowley
On Wed, Jul 2, 2014 at 9:25 PM, Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > > On Sun, Jun 29, 2014 at 4:18 PM, David Rowley > wrote: > >> I think I'm finally ready for a review again, so I'll update the >> commitfest app. >> >> >

Re: [HACKERS] Allowing join removals for more join types

2014-07-05 Thread David Rowley
On 6 July 2014 03:20, Tom Lane wrote: > David Rowley writes: > > Attached is a delta patch between version 1.2 and 1.3, and also a > > completely updated patch. > > Just to note that I've started looking at this, and I've detected a rather > significant omissi

Re: [HACKERS] Allowing join removals for more join types

2014-07-07 Thread David Rowley
On Mon, Jul 7, 2014 at 4:15 AM, Tom Lane wrote: > David Rowley writes: > > On 6 July 2014 03:20, Tom Lane wrote: > >> Just to note that I've started looking at this, and I've detected a > rather > >> significant omission: there's no check that the

[HACKERS] query_is_distinct_for does not take into account set returning functions

2014-07-08 Thread David Rowley
s_distinct_for returns true, therefore the outer query think's it's ok to perform an INNER JOIN rather than a SEMI join, which is this case produces an extra record. I think we should probably include the logic to test for set returning functions into query_is_distinct_for. The attach

Re: [HACKERS] Allowing join removals for more join types

2014-07-08 Thread David Rowley
On Tue, Jul 8, 2014 at 4:28 AM, Tom Lane wrote: > David Rowley writes: > > On Mon, Jul 7, 2014 at 4:15 AM, Tom Lane wrote: > >> I poked around to see if we didn't have some code already for that, and > >> soon found that not only do we have such code > (eq

<    1   2   3   4   5   6   7   8   9   10   >