Re: [PERFORM] Slow Count-Distinct Query

2014-04-06 Thread Varadharajan Mukundan
Hi Jeff, Instead what I get is the index only scan (to provide order) feeding into a > Group. > That's interesting. We tested out in two versions of Postgres (9.2 and 9.3) in different Mac machines and ended up with index-only scan only after the partial index. I remember doing a vacuum full anal

[PERFORM] Slow Count-Distinct Query

2014-04-06 Thread Jeff Janes
On Friday, April 4, 2014, Varadharajan Mukundan wrote: > Hi Jeff, > > It looks like the original emailer wrote a query that the planner is not >> smart enough to plan properly (A known limitation of that kind of query). >> He then made a bunch of changes, none of which worked. He then re-wrote

Re: [PERFORM] Slow Count-Distinct Query

2014-04-01 Thread Christopher Jackson
Hi Bricklen, Thanks again for the feedback. The concurrent refresh sounds cool. I just saw the 9.4 release is tentatively scheduled for later this year. Do you know what people have been doing for view refreshes in the meantime? Thanks On Tue, Apr 1, 2014 at 11:48 PM, Michael Paqui

Re: [PERFORM] Slow Count-Distinct Query

2014-04-01 Thread Michael Paquier
On Wed, Apr 2, 2014 at 1:22 PM, Christopher Jackson wrote: > > Hi Bricklen, > > Thanks for the feedback. I'll play around with materialized views. My > understanding is they have to be manually triggered for refresh Yep. > and there's an exclusive lock on the view while the refresh is t

Re: [PERFORM] Slow Count-Distinct Query

2014-04-01 Thread Christopher Jackson
Hi Bricklen, Thanks for the feedback. I'll play around with materialized views. My understanding is they have to be manually triggered for refresh and there's an exclusive lock on the view while the refresh is taking place. Is this your understanding as well? I'm using PG 9.3.3. If th

Re: [PERFORM] Slow Count-Distinct Query

2014-04-01 Thread bricklen
On Sun, Mar 30, 2014 at 12:45 PM, Christopher Jackson wrote: > Hi all, > > tl;dr - How can I speed up my count-distinct query? > Depending on how often you need to run that query and how important it is to you, if you are willing to accept a performance hit on INSERT/UPDATE/DELETE of the "par

Re: [PERFORM] Slow Count-Distinct Query

2014-03-31 Thread Christopher Jackson
Tom and Shawn, Thanks for the feedback. This has been helpful. It's worth noting that I was spiking this out on my local box using default memory utilization settings. I'll revisit this once we get our production box set up. It's good to know what the best practices are around the enable

Re: [PERFORM] Slow Count-Distinct Query

2014-03-31 Thread Tom Lane
Christopher Jackson writes: > tl;dr - How can I speed up my count-distinct query? EXPLAIN doesn't provide a lot of visibility into what the Aggregate plan node is doing, but in this case what it's doing is an internal sort/uniq operation to implement the DISTINCT. You didn't say what value of

Re: [PERFORM] Slow Count-Distinct Query

2014-03-31 Thread Shaun Thomas
>  tl;dr - How can I speed up my count-distinct query?   You can't. Doing a count(distinct x) is much different than a count(1), which can simply scan available indexes. To build a distinct, it has to construct an in-memory hash of every valid email, and count the distinct values therein. This

[PERFORM] Slow Count-Distinct Query

2014-03-30 Thread Christopher Jackson
Hi all, tl;dr - How can I speed up my count-distinct query? I apologize in advance if this question has been asked already. I'm finding the mailing list hard to navigate. I'm trying to speed up a query that will find a count of distinct emails with in a table using Postgres 9.3.3. The na