Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-05-01 Thread David Rowley
aimed return type of pg_logical_slot_peek_changes.location was text, when it was pg_lsn (maybe apply separately?) 7. Change some places in the func.sgml where it was referring to the lsn as a "position" rather than "location". (We want consistency here) Is this touching all

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-05-04 Thread David Rowley
On 2 May 2017 at 00:10, David Rowley wrote: > On 20 April 2017 at 07:29, Euler Taveira wrote: >> 2017-04-19 1:32 GMT-03:00 Michael Paquier : >>> >>> I vote for "location" -> "lsn". I would expect complains about the >>> current in

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
batches too big as we might end up having to wait on slow workers at the end of a scan. Anyway. I don't want to hi-jack this thread with discussions on this. I just wanted to mark that I plan to work on this in order to avoid any repeat developments or analysis. I'll probably start a new thre

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 5 May 2017 at 13:37, Andres Freund wrote: > On 2017-05-02 15:13:58 -0400, Robert Haas wrote: >> Multiple people (including David Rowley >> as well as folks here at EnterpriseDB) have demonstrated that for >> certain queries, we can actually use a lot more workers and ever

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
On 3 May 2017 at 07:13, Robert Haas wrote: > Multiple people (including David Rowley > as well as folks here at EnterpriseDB) have demonstrated that for > certain queries, we can actually use a lot more workers and everything > works great. The problem is that for other queries, us

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread David Rowley
lower by doing it that way. I didn't manage to think of a way to get around the wrapping the position back to 0 when synch-scans are involved. i.e parallel_scan->phs_cblock++; if (parallel_scan->phs_cblock >= scan->rs_nblocks) parallel_scan->phs_cblock = 0; -- David R

Re: [HACKERS] CTE inlining

2017-05-05 Thread David Rowley
he docs. If anything that'll buy us a bit more wiggle room to change this in v11. I've attached a proposed patch. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services doc_caution_about_cte_changes_in_the_future

[HACKERS] Atomics for heap_parallelscan_nextpage()

2017-05-06 Thread David Rowley
e work. [1] https://www.postgresql.org/message-id/CAKJS1f-XhfQ2-%3D85wgYo5b3WtEs%3Dys%3D2Rsq%3DNuvnmaV4ZsM1XQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/20170505023646.3uhnmf2hbwtm63lc%40alap3.anarazel.de -- David Rowley http://www.2ndQuadrant.com/ Postg

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-07 Thread David Rowley
nce used for benchmarking! I'm glad we're all using a couple more rows these days. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-07 Thread David Rowley
#x27;d want to scan a whole 1GB extent per worker. I did post a patch to have heap_parallelscan_nextpage() use atomics instead of locking over in [1], but I think doing atomics there does not rule out also adding batching later. In fact, I think it structures things so batching would

Re: [HACKERS] Compiler warning in costsize.c

2017-05-07 Thread David Rowley
I would > personally consider the use of a variable, but not here. Any more thoughts on what is acceptable for fixing this? beta1 is looming and it seems a bit messy to be shipping that with these warnings, however harmless they are. -- David Rowley http://www.2ndQuadrant.

Re: [HACKERS] CTE inlining

2017-05-15 Thread David Rowley
m cte WHERE cte.value > 5; We'd be certain to receive complaints from disgruntled users about "Why is this not inlined when I specified INLINE?" #2 does not suffer from that. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Suppor

Re: [HACKERS] Pulling up more complicated subqueries

2017-05-17 Thread David Rowley
something around this in [1]. [1] https://www.postgresql.org/message-id/CAKJS1f_jRki1PQ4X-9UGKa-wnBhECQLnrxCX5haQzu4SDR_r2Q%40mail.gmail.com -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent vi

[HACKERS] Regression in join selectivity estimations when using foreign keys

2017-05-18 Thread David Rowley
go and see if they can Not because I'm super confident, more just because I want this to be correct. Thoughts? -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001-Minimal-fix-for-foreign-key-join-estimations.patch Des

Re: [HACKERS] Regression in join selectivity estimations when using foreign keys

2017-05-19 Thread David Rowley
On 18 May 2017 at 20:28, David Rowley wrote: > A vastly simplified example case is: > > create table fkest (a int, b int, c int unique, primary key(a,b)); > create table fkest1 (a int, b int, primary key(a,b)); > > insert into fkest select x/10,x%10, x from generate_Series(1,400

Re: [HACKERS] Regression in join selectivity estimations when using foreign keys

2017-05-21 Thread David Rowley
27;s version of clauselist_selectivity() will ignore these join conditions, so nothing can be done to assist the underestmations by adding extended stats yet. I also just noticed that I don't think I've got ANTI join cases correct in the patch I sent. I'll look at that now. --

Re: [HACKERS] Regression in join selectivity estimations when using foreign keys

2017-05-23 Thread David Rowley
On 22 May 2017 at 16:10, David Rowley wrote: > I also just noticed that I don't think I've got ANTI join cases > correct in the patch I sent. I'll look at that now. I've attached an updated patch. This one is much less invasive than my original attempt. There are tw

Re: [HACKERS] Perfomance bug in v10

2017-05-31 Thread David Rowley
to generating a plan that'll blow up like this. Is this a realistic enough one to bother accounting for? Did it come from a real world case? else, how did you stumble upon it? -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] Perfomance bug in v10

2017-06-01 Thread David Rowley
". A nested loop unique join is costed more cheaply than a non-unique one since we can skip to the next outer tuple once we've matched the current outer tuple to an inner tuple. In theory that's half as many comparisons for a non-parameterised nested loop. -- David Rowl

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-08-25 Thread David Rowley
this and I don't think anyone was in favour of queries randomly working sometimes. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Making clausesel.c Smarter

2017-09-05 Thread David Rowley
would see some resistance, so won't have time to look at this soon. If the possibility of this increasing planning time in corner cases is going to be a problem, then it might be best to return this with feedback for now and I'll resubmit if I get time later in the cycle. -- Da

Re: [HACKERS] path toward faster partition pruning

2017-09-14 Thread David Rowley
x27;s not fully cooked yet though. I'm interested in seeing improvements in this area, so I've put my name down to review this. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql

Re: [HACKERS] [GENERAL] Remove useless joins (VARCHAR vs TEXT)

2017-09-16 Thread David Rowley
this being a bug. The attached fixes. (CC'd -hackers since we're starting to discuss code changes. Further discussion which includes -hackers should drop the general list) -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &a

Re: [HACKERS] path toward faster partition pruning

2017-09-26 Thread David Rowley
it;h=9140cf826 Yeah, I see 0001 conflicts with that. I'm going to set this to waiting on author while you're busy rebasing this. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-h

Re: [HACKERS] path toward faster partition pruning

2017-09-28 Thread David Rowley
e of that function is not so pretty and that would likely tidy it up a bit. The matching partition indexes could be returned as a Bitmapset, yet, I don't really see any code which handles adding the NULL and DEFAULT partition in get_rel_partitions() either, maybe I've just not looked hard

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-22 Thread David Rowley
on that blocking process we could see the query that it's running and various other properties that are existing columns in pg_stat_activity. Obviously this is blue-skies stuff, but if we had a few to provide that information it would be a great step forward towards that. Regards David Rowl

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread David Rowley
roduced by the sub-query. >> > > Any chance you could implement that in the planner? > > Yes! I'm actually working on it now and so far have it partially working. Quite likely I'll be able to submit for CF2. There's still some costing tweaks to do. So far it just wo

Re: [HACKERS] Combining Aggregates

2015-07-26 Thread David Rowley
eAgg.c to use these, and EXPLAIN to give a better description of what its doing. > > Here are various academic research, for example, below is the good starting > point to clarify aggregate queries that we can run with 2-phase approach. > > http://www.researchgate.net/publication/2715288_Performing_Group-By_before_Join > > Thanks, I've been using that very paper. Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-26 Thread David Rowley
On 27 July 2015 at 03:24, Heikki Linnakangas wrote: > On 07/09/2015 12:44 PM, David Rowley wrote: > >> On 15 June 2015 at 12:05, David Rowley >> wrote: >> >> >>> This basically allows an aggregate's state to be shared between other >>>

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 18:15, Haribabu Kommi wrote: > On Thu, Jul 9, 2015 at 7:44 PM, David Rowley > wrote: > > On 15 June 2015 at 12:05, David Rowley > wrote: > >> > >> > >> This basically allows an aggregate's state to be shared between other

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 20:11, Heikki Linnakangas wrote: > On 07/27/2015 08:34 AM, David Rowley wrote: > > In this function I also wasn't quite sure if it was with comparing both >> non-NULL INITCOND's here. I believe my code comments may slightly >> contradict what

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread David Rowley
;-'; str = pg_int2str_pad(str, tm->tm_mon, 2); *str++ = '-'; str = pg_int2str_pad(str, tm->tm_mday, 2); etc I've used this method before and found it to be about 10 times faster than snprintf(), but I was reversing the string, so quite likely it be more than 10 times. I

Re: [HACKERS] Optimization idea: merging multiple EXISTS(...) with constraint-based join removal

2015-07-27 Thread David Rowley
tion and replace all Vars from the removed relation with the one from the other table, mark relation as REL_DEAD. I think 1 is quite cheap to perform, so normal queries wouldn't suffer much of a slow-down from these extra checks, as most queries won't have self joins. Are you thinking of working on it? Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 20:11, Heikki Linnakangas wrote: > On 07/27/2015 08:34 AM, David Rowley wrote: > >> - * agg_input_types, agg_state_type, agg_result_type identify the input, >> - * transition, and result types of the aggregate. These should all be >> - * resolved t

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-28 Thread David Rowley
the rant, but I just feel like we're painting ourselves into a corner by parallel enabling the executor node by node. Apologies if I've completely misunderstood things. Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-28 Thread David Rowley
On 28 July 2015 at 19:10, Andres Freund wrote: > On 2015-07-28 10:59:15 +1200, David Rowley wrote: > > It won't be quite as fast as what you've written, but I think it will be > > much neater and more likely to be used in other places if we invent a > > function

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-28 Thread David Rowley
On 29 July 2015 at 03:25, Andres Freund wrote: > On 2015-07-29 03:10:41 +1200, David Rowley wrote: > > Have you thought about what to do when HAVE_INT64_TIMESTAMP is not > defined? > > I don't think it's actually important. The only difference vs float > timestamp

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-08-02 Thread David Rowley
On 29 July 2015 at 03:45, Heikki Linnakangas wrote: > On 07/28/2015 04:14 AM, David Rowley wrote: > >> On 27 July 2015 at 20:11, Heikki Linnakangas wrote: >> >> On 07/27/2015 08:34 AM, David Rowley wrote: >>> >>> In this function I also wasn't qu

[HACKERS] cost_agg() with AGG_HASHED does not account for startup costs

2015-08-03 Thread David Rowley
-zero startup cost. The attached changes cost_agg() to include the startup cost, but does nothing for the changed plans in the regression tests. Is this really intended? Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/>

[HACKERS] GROUP BY before JOIN

2015-08-04 Thread David Rowley
work to do here, aggregates need to be supported, functional dependencies and transitive closures will also need to be detected in a more complete implementation. Here's what I have so far (attached) Comments are most welcome. Thanks. Regards David Rowley -- David Rowley http:

Re: [HACKERS] GROUP BY before JOIN

2015-08-04 Thread David Rowley
27;s certainly going to increase that, but I'm not sure if 'explode' is the best word as we at least require all of the rels seen in the GROUP BY expressions and aggregate function parameters to be joined before we can consider a GroupingPath. The patch uses bms_is_subset(root->

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-08-04 Thread David Rowley
On 5 August 2015 at 03:03, Heikki Linnakangas wrote: > On 08/03/2015 08:53 AM, David Rowley wrote: > >> Attached is a delta patched which is based >> on sharing_aggstate-heikki-2.patch to fix up the out-dated comments and >> also a few more test scenarios which tes

Re: [HACKERS] cost_agg() with AGG_HASHED does not account for startup costs

2015-08-04 Thread David Rowley
On 5 August 2015 at 01:54, Tom Lane wrote: > David Rowley writes: > > During working on allowing the planner to perform GROUP BY before joining > > I've noticed that cost_agg() completely ignores input_startup_cost > > when aggstrategy == AGG_HASHED. > >

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-08-04 Thread David Rowley
On 29 July 2015 at 03:25, Andres Freund wrote: > On 2015-07-29 03:10:41 +1200, David Rowley wrote: > > timestamp_out() = 2015-07-29 02:24:33.34 in 3.506000 > > timestamp_out_old() = 2015-07-29 02:24:33.034 in 64.518000 > > timestamp_out_af() = 2015-07-29 02:

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-08-05 Thread David Rowley
On 5 August 2015 at 12:51, David Rowley wrote: > On 29 July 2015 at 03:25, Andres Freund wrote: > >> On 2015-07-29 03:10:41 +1200, David Rowley wrote: >> > timestamp_out() = 2015-07-29 02:24:33.34 in 3.506000 >> > timestamp_out_old() = 2015-07-29 02:24:33.034 in 6

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread David Rowley
on of windows which supports GetSystemTimePreciseAsFileTime(); Master: postgres=# select now(); now --- 2015-08-09 01:14:01.959645+12 (1 row) 9.4.4 postgres=# select now(); now ---- 2015-08-09 01:15:09.783+12 (1 row) Reg

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-08-08 Thread David Rowley
On 29 July 2015 at 03:25, Andres Freund wrote: > On 2015-07-29 03:10:41 +1200, David Rowley wrote: > > timestamp_out() = 2015-07-29 02:24:33.34 in 3.506000 > > timestamp_out_old() = 2015-07-29 02:24:33.034 in 64.518000 > > timestamp_out_af() = 2015-07-29 02:

Re: [HACKERS] Moving SS_finalize_plan processing to the end of planning

2015-08-10 Thread David Rowley
n up to functions like recurse_set_operations() so that they have the option of choosing GroupAggregate / MergeAppend to implement UNION. If I'm right on this, then maybe there's a few things you can copy and paste from the patch I posted here: http://www.postgresql.org/message-id/CAKJS1f

[HACKERS] Small improvement to get_base_rel_indexes()

2015-08-15 Thread David Rowley
thought I'd post anyway as: 1. It removes 5 lines of code. 2. It's a better example to leave in the code. Is it worth applying? Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 S

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
it can't be. lbuckets = 1 << my_log2(hash_table_bytes / bucket_size); I think one day soon we'll need to allow larger work_mem sizes, but I think there's lots more to do than this change. Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
On 19 August 2015 at 12:23, Kouhei Kaigai wrote: > > -Original Message- > > From: David Rowley [mailto:david.row...@2ndquadrant.com] > > Sent: Wednesday, August 19, 2015 9:00 AM > > The size of your hash table is 101017630802 bytes, which is: > > >

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
On 19 August 2015 at 12:38, Tom Lane wrote: > David Rowley writes: > > david=# set work_mem = '94GB'; > > ERROR: 98566144 is outside the valid range for parameter "work_mem" (64 > .. > > 2097151) > > Apparently you're testing on a 32-b

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-19 Thread David Rowley
1073741824 Setting work_mem to 1GB or less gets the query running. I've patched the code with your patch Kohei, and it's now working. Thought I'd better post this just in case this gets forgotten about. Thanks David -- David Rowley http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-08-23 Thread David Rowley
ain output, there should probably be a space before the >'(inner unique)' text, so > > Hash Join (inner unique) ... > >instead of > > Hash Join(inner unique) > > but that's just nitpicking at this point. Otherwise the patch seems quite >

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-08-23 Thread David Rowley
On 24 August 2015 at 12:19, Tom Lane wrote: > David Rowley writes: > > On 24 August 2015 at 07:31, Tomas Vondra > > wrote: > >> 2) in the explain output, there should probably be a space before the > >> '(inner unique)' text, so > >> &g

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-08-24 Thread David Rowley
On 24 August 2015 at 14:29, Tom Lane wrote: > David Rowley writes: > > I have to admit I don't much like it either, originally I had this as an > > extra property that was only seen in EXPLAIN VERBOSE. > > Seems like a reasonable design from here. The attached patch

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2015-08-29 Thread David Rowley
On 28 August 2015 at 22:20, Etsuro Fujita wrote: > On 2015/07/22 15:25, Etsuro Fujita wrote: > >> On 2015/07/10 21:59, David Rowley wrote: > > I just glanced at this and noticed that the method for determining if >>> there's any system columns could be made a bit

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-02 Thread David Rowley
On 3 September 2015 at 05:10, Andres Freund wrote: > On 2015-08-09 12:47:53 +1200, David Rowley wrote: > > I took a bit of weekend time to finish this one off. Patch attached. > > > > A quick test shows a pretty good performance increase: > > > > creat

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

2015-09-02 Thread David Rowley
ww.naftaliharris.com/blog/2x-speedup-with-one-line-of-code/ Peter, would you be able to share the test case which you saw the speedup on. So far I've been unable to see much of an improvement. Regards David Rowley -- David Rowley http://www.2ndQuadrant.com/ <http://www

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-03 Thread David Rowley
On 3 September 2015 at 22:17, Andres Freund wrote: > On 2015-09-03 16:28:40 +1200, David Rowley wrote: > > I experimented with finding the fastest way to convert an int to a string > > at the weekend, I did happen to be testing with int64's but likely int32 > > will be

Re: [HACKERS] Memory prefetching while sequentially fetching from SortTuple array, tuplestore

2015-09-03 Thread David Rowley
On 3 September 2015 at 16:50, Peter Geoghegan wrote: > On Wed, Sep 2, 2015 at 9:43 PM, David Rowley > wrote: > > Peter, would you be able to share the test case which you saw the speedup > > on. So far I've been unable to see much of an improvement. > > The case

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-07 Thread David Rowley
On 3 September 2015 at 22:17, Andres Freund wrote: > On 2015-09-03 16:28:40 +1200, David Rowley wrote: > > > Wouldn't it be better to just normalize fsec to an integer in that > case? > > > Afaics that's the only remaining reason for the alternative path? > &

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-11 Thread David Rowley
grade hardware, and the performance increase is looking a bit better. create table ts (ts timestamp not null); insert into ts select generate_series('2010-01-01 00:00:00', '2011-01-01 00:00:00', '1 sec'::interval); vacuum freeze ts; Master tpch=# copy ts to '/de

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-09-11 Thread David Rowley
n't see any speedup, so I assume I'm not getting enough contention. As soon as our 4 socket machine is free I'll try a pgbench run with that. Just for fun, what's the results if you use -M prepared ? Regards David Rowley -- David Rowley http://www.2

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-13 Thread David Rowley
On Tue, Jan 14, 2014 at 2:00 PM, Florian Pflug wrote: > On Jan10, 2014, at 22:27 , David Rowley wrote: > > As the patch stands at the moment, I currently have a regression test > > which currently fails due to these extra zeros after the decimal point: > > > > -- Th

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-14 Thread David Rowley
with resetting numeric scale will turn a possible 9.4 patch into a 9.5/10.0 patch. I see no reason why what's there so far, minus sum(numeric), can't go in... If so then there's 36 aggregate functions ticked off my "create an inverse transition function for" list! I personally think that's a pretty good win. I'd rather do this than battle and miss deadlines for 9.4. I'd find that pretty annoying. Regards David Rowley

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-14 Thread David Rowley
On Tue, Jan 14, 2014 at 9:09 PM, David Rowley wrote: > I think unless anyone has some objections I'm going to remove the inverse > transition for SUM(numeric) and modify the documents to tell the user how > to build their own FAST_SUM(numeric) using the built in functions t

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-14 Thread David Rowley
On Tue, Jan 14, 2014 at 9:09 PM, David Rowley wrote: > I think unless anyone has some objections I'm going to remove the inverse > transition for SUM(numeric) and modify the documents to tell the user how > to build their own FAST_SUM(numeric) using the built in functions t

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-15 Thread David Rowley
to be per aggregate rather than per window. If you think it would be worth it I can modify the patch to work that way. Regards David Rowley > best regards, > Florian Pflug >

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-15 Thread David Rowley
in the number > of trailing zeros) depends on values outside of the frame. Which, I guess, > is a box that best stays closed... > > Yeah, I can understand the temptation but I agree we can't go changing results. > I'm currently thinking the best way forward is to get a basic

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-15 Thread David Rowley
On Tue, Jan 14, 2014 at 2:29 PM, Tom Lane wrote: > > One reason I'm thinking this is that whatever we do to ameliorate > the semantic issues is going to slow down the forward transition > function --- to no benefit unless the aggregate is being used as a > window function in a moving window. So

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread David Rowley
t would be a good optimisation to detect this and throw away the previous results and start fresh. Thanks for all of your reviewing on this so far. Regards David Rowley > best regards, > Florian Pflug > >

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread David Rowley
nce to look at your updated v2.2 patch yet sorry. I'll try and get some time tomorrow evening. Regards David Rowley > best regards, > Florian Pflug > > inverse_transition_functions_v2.3.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
there's much more hacking to do we could start pushing to a guthub repo to make it easier to work together on this. Regards David Rowley inverse_transition_functions_v2.6.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

[HACKERS] currawong is not a happy animal

2014-01-17 Thread David Rowley
I've not been able to build master for a few days now on visual studios due some problems in the new test_shm_mq contrib module. I'm not too familiar with how the visual studio project files are generated so I've not yet managed to come up with a fix just yet. However I have attached a fix for the

Re: [HACKERS] currawong is not a happy animal

2014-01-17 Thread David Rowley
only one of these >>> symbols we might need to make visible? >>> >> We've generally relied on the buildfarm to cue us to add PGDLLIMPORT. >> Add that one and see what happens ... >> >> >> > > > OK, done. > > Great, that fixes it on my end. Thanks for fixing those. Regards David Rowley > cheers > > andrew > >

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
e transition with a expectedscale as -1 we'll return null, otherwise we can perform the inverse transition... Thoughts? Regards David Rowley

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
On Sat, Jan 18, 2014 at 10:17 AM, Florian Pflug wrote: > On Jan17, 2014, at 20:34 , David Rowley wrote: > > On Fri, Jan 17, 2014 at 3:05 PM, Florian Pflug wrote: > > > >> I've now shuffled things around so that we can use inverse transition > functions > >

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
On Sat, Jan 18, 2014 at 10:42 AM, David Rowley wrote: > >> You could do better than that - the numeric problem amounts to tracking >> the maximum >> scale AFAICS, so it'd be sufficient to restart if we remove a value whose >> scale equals >> the current max

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
t; We I guess you're both now, but it's a bit weird to be author and reviewer so I've put your name against author too, hopefully Dean can review our combined results and we can review each other's work at the same time. > On Jan17, 2014, at 23:34 , David Rowley wrote:

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-17 Thread David Rowley
On Sat, Jan 18, 2014 at 6:15 PM, David Rowley wrote: > On Sat, Jan 18, 2014 at 2:20 PM, Florian Pflug wrote: > >> * Don't we need to check for volatile function in the filter expression >> too? >> > >> > I did manual testing on this before and the vola

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-18 Thread David Rowley
it for the bool and/or aggregates. We > could just > >> keep track of the values aggregated and the count of values as "true" > and return > >> true if those are the same in the case of "AND", then check the true > count > >> is > 0 in the

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-19 Thread David Rowley
On Sun, Jan 19, 2014 at 5:27 PM, David Rowley wrote: > >> >> It's probably far more worth it for the bool and/or aggregates. We >> could just >> >> keep track of the values aggregated and the count of values as "true" >> and return >&

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-19 Thread David Rowley
On Sat, Jan 18, 2014 at 11:34 AM, David Rowley wrote: > > The latest version of the patch is attached. > > I've attached an updated version of the patch. I'm now using github to track the changes on the patch, so I've included the commit sha in the file name of the la

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-19 Thread David Rowley
On Mon, Jan 20, 2014 at 5:53 AM, Florian Pflug wrote: > On Jan19, 2014, at 05:27 , David Rowley wrote: > >> I just finished implementing the inverse transition functions for > bool_and > >> and bool_or, these aggregates had a sort operator which I assume would > have

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-20 Thread David Rowley
On Mon, Jan 20, 2014 at 8:42 PM, David Rowley wrote: > On Mon, Jan 20, 2014 at 2:45 PM, Florian Pflug wrote: > >> * EXPLAIN VERBOSE ANALYZE now shows the max. number of forward aggregate >> transitions per row and aggregate. It's a bit imprecise, because it >> doe

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-20 Thread David Rowley
On Mon, Jan 20, 2014 at 2:45 PM, Florian Pflug wrote: > On Jan19, 2014, at 20:00 , David Rowley wrote: > > I've applied that patch again and put in the sort operators. > > I've push a new version to https://github.com/fgp/postgres/tree/invtrans > which inclu

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-21 Thread David Rowley
e all cases and give the exact same results as it would without inverse transitions, but it certainly fixes the error case which was presented Using the attached on HEAD of https://github.com/david-rowley/postgres/commits/invtrans explain (analyze, verbose) select mysum(v) over (order by i row

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-22 Thread David Rowley
On Tue, Jan 21, 2014 at 3:20 AM, Florian Pflug wrote: > On Jan20, 2014, at 08:42 , David Rowley wrote: > >> On Mon, Jan 20, 2014 at 2:45 PM, Florian Pflug wrote: > >> * I've also renamed INVFUNC to INVSFUNC. That's a pretty invasive > change, and > >&g

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-22 Thread David Rowley
On Wed, Jan 22, 2014 at 12:46 AM, Florian Pflug wrote: > On Jan21, 2014, at 10:53 , David Rowley wrote: > > It came to me that it might be possible to implement inverse transitions > > for floating point aggregates by just detecting if precision has been > > lost durin

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-25 Thread David Rowley
On Thu, Jan 23, 2014 at 1:57 PM, Florian Pflug wrote: > On Jan23, 2014, at 01:17 , David Rowley wrote: > > On Wed, Jan 22, 2014 at 12:46 AM, Florian Pflug wrote: > >> If you want to play with > >> this, I think the first step has to be to find a set of guarant

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-25 Thread David Rowley
to test every datatype, but a few would be nice. > > I've added a few regression tests for min, min and bool_or and bool_and. I've pushed these up to here: https://github.com/david-rowley/postgres/commits/invtrans_minmax I did wonder if you'd want to see uses of FILTER in there as

[HACKERS] Patch: Allow formatting in log_line_prefix

2013-08-26 Thread David Rowley
eople. If you apply this patch you can do things like have %10u %-10d in your log_line_prefix which will include spaces to give the option the number you specify as the minimum width. Left and right aligning is supported. Let this be the thread to collect consensus to whether this needed and use

Re: [HACKERS] Patch: Allow formatting in log_line_prefix

2013-08-27 Thread David Rowley
On Tue, Aug 27, 2013 at 7:55 PM, Vik Fearing wrote: > On 08/27/2013 05:06 AM, David Rowley wrote: > > Heikki wrote that it might be useful to allow formatting in the > > log_line_prefix here > > http://www.postgresql.org/message-id/5187cadb.50...@vmware.com > > >

[HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-19 Thread David Rowley
(Forgot to copy list) From: David Rowley [mailto:dgrowle...@gmail.com] Sent: 19 September 2013 22:35 To: Albe Laurenz Subject: Re: REVIEW: Allow formatting in log_line_prefix Hi Laurenz, Thanks for the review. Please see my comments below and the updated patch attached. On Thu, Sep

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-19 Thread David Rowley
On Fri, Sep 20, 2013 at 12:48 AM, Peter Eisentraut wrote: > Something is weird in your latest patch. The header is: > > diff -u -r b/postgresql/doc/src/sgml/config.sgml > a/postgresql/doc/src/sgml/config.sgml > --- b/postgresql/doc/src/sgml/config.sgml 2013-09-09 > 23:40:52.356371191 +1200

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-20 Thread David Rowley
On Sat, Sep 21, 2013 at 7:18 AM, Robert Haas wrote: > On Fri, Sep 20, 2013 at 3:15 AM, Albe Laurenz > wrote: > > David Rowley wrote: > >> I moved the source around and I've patched against it again. New patch > attached. > > > > Thank you, marked as

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-24 Thread David Rowley
On Tue, Sep 24, 2013 at 5:16 AM, Robert Haas wrote: > On Fri, Sep 20, 2013 at 11:28 PM, David Rowley > wrote:\ > > I put the above results into the attached spreadsheet. On my intel i5 > laptop > > I'm seeing a slow down of about 1 second per million queries for the &g

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-24 Thread David Rowley
On Wed, Sep 25, 2013 at 6:25 AM, Andres Freund wrote: > On 2013-09-24 19:56:32 +0200, Andres Freund wrote: > > On 2013-09-24 13:51:04 -0300, Alvaro Herrera wrote: > > > David Rowley escribió: > > > > > > > I do see a 15-18% slow down with the patched ver

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-25 Thread David Rowley
On Wed, Sep 25, 2013 at 1:20 AM, Robert Haas wrote: > On Tue, Sep 24, 2013 at 5:04 AM, David Rowley > wrote: > >> So... I guess the question that I'd ask is, if you write a PL/pgsql > >> function that does RAISE NOTICE in a loop a large number of times, can >

<    1   2   3   4   5   6   7   8   9   10   >