Re: [PERFORM] Pageinspect bt_metap help

2017-09-18 Thread Peter Geoghegan
n instead just pretend that it's shorter, knowing that upper levels don't contain useful information. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Pageinspect bt_metap help

2017-09-17 Thread Peter Geoghegan
out the B-Tree code, I suggest that you start by looking at the code for contrib/amcheck. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] 10x faster sort performance on Skylake CPU vs Ivy Bridge

2017-08-25 Thread Peter Geoghegan
t > some non-C locale on the slower. strcoll() is pretty darn expensive > compared to strcmp() :-( strcoll() is very noticeably slower on macOS, too. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Odd sudden performance degradation related to temp object churn

2017-08-14 Thread Peter Geoghegan
this job has been running for years with no issue remotely approaching > this one. We are also using pgpool. Did you happen to notice that this occurred when you upgrading point release? If so, what version did you move from/to? -- Peter Geoghegan -- Sent via pgsql-performance mail

Re: [PERFORM] Slow query after 9.3 to 9.6 migration

2017-07-26 Thread Peter Geoghegan
se I think it's possible that this is explained by a regression in 9.5's handling of index bloat, described here: http://postgr.es/m/CAH2-Wz=sfakvmv1x9jh19ej8am8tzn9f-yecips9hrrrqss...@mail.gmail.com I'm trying to track down cases where this could be an issue, to get a bette

Re: estimate correlation of index separately from table (Re: [PERFORM] index fragmentation on insert-only table with non-unique column)

2017-07-07 Thread Peter Geoghegan
ut going to great lengths. These optimizations are closely intertwined things, and the lack of clarity on how they all fit together is probably holding back an implementation of any one of them. [1] https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914

Re: [PERFORM] Efficiently merging and sorting collections of sorted rows

2017-06-23 Thread Peter Geoghegan
ong here? Is there a way to get Postgres to not do a > quick sort here? I would like that too. There is a patch that does what I think you're describing, but it seems to be in limbo: https://commitfest.postgresql.org/11/409/ -- Peter Geoghegan -- Sent via pgsql-performance mailing list

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Peter Geoghegan
more sophisticated approach, but it's probably better to focus on duplicate storage or even leaf page compression, as Stephen mentioned. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Peter Geoghegan
arguing for the idea that B-Trees should reliably keep tuples in order by a tie-break condition, that seems difficult to implement, and likely not worth it in practice. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Adding a ROLLUP switches to GroupAggregate unexpectedly

2016-04-09 Thread Peter Geoghegan
I bet that was a factor here. -- Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Slow Index Creation, why is it not consuming more memory.

2015-12-07 Thread Peter Geoghegan
to help unless they result in a fully internal sort. There is evidence that the heap that tuple sorting uses benefits from *lower* settings. Sometimes as low as 64MB. We're working to make this better in 9.6. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsq

Re: [PERFORM] Index creation running now for 14 hours

2015-08-26 Thread Peter Geoghegan
ace to do the sort, you're not going to be able to complete it. That much is very clear. If you're really worried about these costs, I suggest enabling trace_sort locally, and monitoring the progress of this sort in the logs. -- Regards, Peter Geoghegan -- Sent via pgsql-perfo

Re: [PERFORM] working around JSONB's lack of stats?

2015-01-28 Thread Peter Geoghegan
be fairly homogeneous in structure within a table, just like with systems like MongoDB. Strings within arrays are keys for our purposes, and these are often used for tags and so on. But Strings that are the key of an object/pair are much less useful to index, in my estimation. -- Regards, Peter Ge

Re: [PERFORM] working around JSONB's lack of stats?

2015-01-28 Thread Peter Geoghegan
work alright for arrays of "tags", and things like that. I tend to think that that's a common enough use-case. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
'm suggesting that we give up on the idea of sampling (which would be crazy). Streaming algorithms like HyperLogLog are very recent ideas, as these things go. I wouldn't be all that discouraged by the fact that it might not have been put to use in this way (for database statistics) by somebody b

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
ming of HLL. Believe it or not, the way I use HLL for estimating cardinality is virtually free. Hashing is really cheap when the CPU is bottlenecked on memory bandwidth. If you're interested, download the patch, and enable the debug traces. You'll see HyperLogLog accurately indicate the

Re: [PERFORM] Turn off Hyperthreading! WAS: 60 core performance with 9.3

2014-08-20 Thread Peter Geoghegan
ally important to consider power saving features in most benchmarks these days, where that might not have been true a few years ago. The CPU scaling governor can alter the outcome of many benchmarks quite significantly. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgs

Re: [PERFORM] Lack of index usage when doing array casts

2014-02-21 Thread Peter Geoghegan
x27;re using standard packages, I'm afraid. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Performance Benchmarking for data-warehousing instance?

2014-02-07 Thread Peter Geoghegan
On Fri, Feb 7, 2014 at 7:36 PM, Huy Nguyen wrote: > I think pgtune is optimized more for OLTP application. Is there something > similar to pgtune/pgbench for OLAP? IIRC pgtune can be told to give out an OLAP-optimized postgresql.conf. Maybe that's only recent versions? -- Reg

Re: [PERFORM] Postgres Query Plan Live Lock

2014-02-04 Thread Peter Geoghegan
s immediately after the queue is > cleared). Are you aware of hint bits? https://wiki.postgresql.org/wiki/Hint_Bits -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Why is n_distinct always -1 for range types?

2013-09-25 Thread Peter Geoghegan
lower and upper bounds for range types, and the fraction of empty ranges. -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] pg_stat_statements behavior in crash recovery

2013-06-19 Thread Peter Geoghegan
lized to disk when there's a clean shutdown. pg_stat_statements is similar to the statistics collector here. Why are you posting this to the -performance list? -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to

Re: [PERFORM] pg_stat_statements query normalization

2013-06-17 Thread Peter Geoghegan
s later representation (essentially, the internal representation that the rewriter stage processes). -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Poor performance on an aggregate query

2013-04-15 Thread Peter Geoghegan
nctional index on length(discussion_post.id)? -- Regards, Peter Geoghegan -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] serious under-estimation of n_distinct for clustered distributions

2013-01-14 Thread Peter Geoghegan
the street is that it may be worth pursuing some of the ideas described by the literature in just the last few years. I've often thought that this would be an interesting problem to work on. I haven't had time to pursue it, though. You may wish to propose a patch on the pgsql-hackers mailing

Re: [PERFORM] serious under-estimation of n_distinct for clustered distributions

2012-12-29 Thread Peter Geoghegan
thin src/backend/commands/analyze.c (IBM Research Report RJ 10025 is referenced there). The general advice here is: 1) Increase default_statistics_target for the column. 2) If that doesn't help, consider using the following DDL: alter table foo alter column bar set ( n_distinct = 5.0); -- Pe

Re: [PERFORM] Poor performance using CTE

2012-11-21 Thread Peter Geoghegan
On 21 November 2012 15:21, Andrew Dunstan wrote: > And I continue to think that spelling it "OFFSET 0" is horribly obscure. I'm not sure that it's any more obscure than the very idea of an optimisation fence. -- Peter Geoghegan http://www.2ndQuadrant.com/ Postg

Re: [PERFORM] Poor performance using CTE

2012-11-21 Thread Peter Geoghegan
On 21 November 2012 13:04, Heikki Linnakangas wrote: > Yes, I strongly feel that we should. Writing a query using WITH often makes > it more readable. It would be a shame if people have to refrain from using > it, because the planner treats it as an optimization fence. +1 -- Peter

Re: SOLVED - RE: [PERFORM] Poor performance using CTE

2012-11-14 Thread Peter Geoghegan
een telling people to do that for years, so it's already something that we've effectively committed to. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@pos

Re: [PERFORM] PostreSQL v9.2 uses a lot of memory in Windows XP

2012-11-14 Thread Peter Geoghegan
h, particularly if you're not interested in *what* Postgres is doing with memory, but need to summarise it usefully. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgs

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
ndexes costs essentially nothing to maintain, and simply represent an ongoing obligation for ANALYZE to provide statistics for an expression? -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
expression into a column, analyzed and grouped by that. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://

Re: [PERFORM] [repost] Help me develop new commit_delay advice

2012-10-08 Thread Peter Geoghegan
On 6 September 2012 04:20, Greg Smith wrote: > On 08/02/2012 02:02 PM, Peter Geoghegan wrote: > I dug up what I wrote when trying to provide better advice for this circa > V8.3. That never really gelled into something worth publishing at the time. > But I see some similar patter

Re: [PERFORM] average query performance measuring

2012-08-21 Thread Peter Geoghegan
better way to monitor query execution costs on earlier versions, I think that I'll probably have new information about that for my talk at Postgres Open. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-perfo

[PERFORM] [repost] Help me develop new commit_delay advice

2012-08-02 Thread Peter Geoghegan
This has been reposted to this list from the pgsql-hackers list, at the request of Josh Berkus. Hopefully there will be more interest here. -- Forwarded message -- From: Peter Geoghegan Date: 29 July 2012 16:39 Subject: Help me develop new commit_delay advice To: PG Hackers

Re: [PERFORM] optimizing queries using IN and EXISTS

2012-07-18 Thread Peter Geoghegan
ly equivalent. http://www.postgresql.org/docs/current/static/functions-subquery.html See the notes about NULL under IN. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@pos

Re: [PERFORM] Update blocking a select count(*)?

2012-06-15 Thread Peter Geoghegan
a select statement's AccessShare lock. -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

Re: [PERFORM] random_page_cost = 2.0 on Heroku Postgres

2012-02-12 Thread Peter Geoghegan
very plan, and therefore doesn't pay the considerable overhead of that instrumentation across the board). -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@po

Re: [PERFORM] What about implementing a bitmap index? Any use cases?

2011-09-18 Thread Peter Geoghegan
ead about an unsuccessful attempt to introduce on-disk bitmap indexes to Postgres. -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make change

Re: [PERFORM] Hash index use presently(?) discouraged since 2005: revive or bury it?

2011-09-13 Thread Peter Geoghegan
improved since 2005 - their performance was improved quite a bit in 9.0. Here's a more recent analysis: http://www.depesz.com/index.php/2010/06/28/should-you-use-hash-index/ -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services