Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-18 Thread Scott Marlowe
On Thu, Oct 18, 2012 at 1:50 PM, Jeff Janes wrote: > On Wed, Oct 10, 2012 at 10:36 PM, Scott Marlowe > wrote: >> On Wed, Oct 10, 2012 at 4:37 PM, Claudio Freire >> wrote: >>> >>> In my case, if I set it too high, I get impossibly suboptimal plans >>> when an index scan over millions of rows hi

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-18 Thread Claudio Freire
On Thu, Oct 18, 2012 at 4:23 PM, Jeff Janes wrote: >> @Claudio So you are basically saying that if I have set effective_cache_size >> to 10GB and I have 10 concurrent processes which are using 10 different >> indices which are for example 2GB, > > It is the size of the table, not the index, which

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-18 Thread Jeff Janes
On Wed, Oct 10, 2012 at 10:36 PM, Scott Marlowe wrote: > On Wed, Oct 10, 2012 at 4:37 PM, Claudio Freire > wrote: >> >> In my case, if I set it too high, I get impossibly suboptimal plans >> when an index scan over millions of rows hits the disk way too often >> way too randomly. The difference

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-18 Thread Jeff Janes
On Wed, Oct 10, 2012 at 1:12 PM, Strahinja Kustudić wrote: > @Claudio So you are basically saying that if I have set effective_cache_size > to 10GB and I have 10 concurrent processes which are using 10 different > indices which are for example 2GB, It is the size of the table, not the index, whi

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-18 Thread Jeff Janes
On Thu, Oct 11, 2012 at 11:17 AM, Josh Berkus wrote: > >> Does anyone see effective_cache_size make a difference anyway? If so, >> in what circumstances? > > E_C_S, together with random_page_cost, the table and index sizes, the > row estimates and the cpu_* costs, form an equation which estimates

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
On 18 October 2012 17:52, Tom Lane wrote: > Thom Brown writes: >> On 18 October 2012 17:44, Tom Lane wrote: >>> Thom Brown writes: And as a side note, how come it's impossible to get the planner to use an index-only scan to satisfy the query (disabling sequential and regular inde

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Tom Lane
Peter Geoghegan writes: > Is there a case to be made for a index access method whose > pseudo-indexes costs essentially nothing to maintain, and simply > represent an ongoing obligation for ANALYZE to provide statistics for > an expression? If we were going to support it, I think we'd be better o

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Tom Lane
Thom Brown writes: > On 18 October 2012 17:44, Tom Lane wrote: >> Thom Brown writes: >>> And as a side note, how come it's impossible to get the planner to use >>> an index-only scan to satisfy the query (disabling sequential and >>> regular index scans)? >> Implementation restriction - we don'

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
On 18 October 2012 18:00, Peter Geoghegan wrote: > On 18 October 2012 17:52, Tom Lane wrote: >> I forgot to mention that there is a klugy workaround: add the required >> variable(s) as extra index columns. That is, >> >> create index i on t (foo(x), x); > > Is there a case to be made for

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
On 18 October 2012 17:52, Tom Lane wrote: > I forgot to mention that there is a klugy workaround: add the required > variable(s) as extra index columns. That is, > > create index i on t (foo(x), x); Is there a case to be made for a index access method whose pseudo-indexes costs essential

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
On 18 October 2012 17:44, Tom Lane wrote: > Thom Brown writes: >> And as a side note, how come it's impossible to get the planner to use >> an index-only scan to satisfy the query (disabling sequential and >> regular index scans)? > > Implementation restriction - we don't yet have a way to match

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
On 18 October 2012 17:24, Peter Geoghegan wrote: > On 18 October 2012 17:11, Thom Brown wrote: >> The estimate is down to almost a 10th of what it was before. What's going >> on? > > Even though the index isn't used, the pg_statistic entries that the > expression index would have made available

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Tom Lane
Thom Brown writes: > I've created a test table containing 21 million random dates and > times, but I get wildly different results when I introduce a > functional index then ANALYSE again, even though it doesn't use the > index: As Peter said, the existence of the index causes ANALYZE to gather st

Re: [PERFORM] Two identical systems, radically different performance

2012-10-18 Thread Craig James
On Wed, Oct 17, 2012 at 11:57 PM, Andrea Suisani wrote: > On 10/17/2012 06:35 PM, Scott Marlowe wrote: >> >> On Wed, Oct 17, 2012 at 9:45 AM, Andrea Suisani >> wrote: >>> >>> On 10/15/2012 05:34 PM, Scott Marlowe wrote: I'd recommend more synthetic benchmarks when trying to compare syst

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Peter Geoghegan
On 18 October 2012 17:11, Thom Brown wrote: > The estimate is down to almost a 10th of what it was before. What's going on? Even though the index isn't used, the pg_statistic entries that the expression index would have made available are. It's as if you materialised the expression into a column

Re: [PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
On 18 October 2012 17:11, Thom Brown wrote: > Hi all, > > I've created a test table containing 21 million random dates and > times, but I get wildly different results when I introduce a > functional index then ANALYSE again, even though it doesn't use the > index: > > postgres=# CREATE TABLE test

[PERFORM] Unused index influencing sequential scan plan

2012-10-18 Thread Thom Brown
Hi all, I've created a test table containing 21 million random dates and times, but I get wildly different results when I introduce a functional index then ANALYSE again, even though it doesn't use the index: postgres=# CREATE TABLE test (id serial, sampledate timestamp); CREATE TABLE postgres=#

Re: [PERFORM] LIKE op with B-Tree Index?

2012-10-18 Thread Albe Laurenz
Sam Wong wrote: > I am investigating a performance issue involved with LIKE '%' > on an index in a complex query with joins. > Q1. > SELECT * FROM shipments WHERE shipment_id LIKE '12345678%' > > Q2. > SELECT * FROM shipments WHERE shipment_id >= '12345678' AND