Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Kevin Grittner
Greg Smith wrote: > The idea that any of this will run automatically is a dream at > this point, so saying you want to automatically recover from > problems with the mechanism that doesn't even exist yet is a bit > premature. Well, I certainly didn't mean it to be a reason not to move forward

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Greg Smith
Kevin Grittner wrote: But its not hard to imagine an application mix where this feature could cause a surprising ten-fold performance drop after someone does a table scan which could persist indefinitely. I'm not risking that in production without a clear mechanism to automatically recover from

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Kevin Grittner
Greg Smith wrote: > On 06/19/2011 06:15 PM, Kevin Grittner wrote: >> I think the point is that if, on a fresh system, the first access >> to a table is something which uses a tables scan -- like select >> count(*) -- that all indexed access would then tend to be >> suppressed for that table. Aft

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Robert Haas
On Sun, Jun 19, 2011 at 3:32 PM, Cédric Villemain wrote: > 2nd, I provided the patches on the last CF, exactly to allow to go to > the exciting part: the cost-estimates changes. (after all, we can work > on the cost estimate, and if later we find a way to use ALTER TABLE or > pg_class_ng, just do

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Smith
On 06/19/2011 06:15 PM, Kevin Grittner wrote: I think the point is that if, on a fresh system, the first access to a table is something which uses a tables scan -- like select count(*) -- that all indexed access would then tend to be suppressed for that table. After all, for each individual que

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Kevin Grittner
Greg Smith wrote: > I'm not too concerned about the specific case you warned about > because I don't see how sequential scan vs. index costing will be > any different on a fresh system than it is now. I think the point is that if, on a fresh system, the first access to a table is something whi

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Smith
On 06/19/2011 09:38 AM, Greg Stark wrote: There's another problem which I haven't seen mentioned. Because the access method will affect the cache there's the possibility of feedback loops. e.g. A freshly loaded system prefers sequential scans for a given table because without the cache the seeks

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Cédric Villemain
2011/6/19 Robert Haas : > On Sun, Jun 19, 2011 at 9:38 AM, Greg Stark wrote: >> On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas wrote: >>> 1. ANALYZE happens far too infrequently to believe that any data taken >>> at ANALYZE time will still be relevant at execution time. >>> 2. Using data gathered b

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Robert Haas
On Sun, Jun 19, 2011 at 9:38 AM, Greg Stark wrote: > On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas wrote: >> 1. ANALYZE happens far too infrequently to believe that any data taken >> at ANALYZE time will still be relevant at execution time. >> 2. Using data gathered by ANALYZE will make plans less

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Cédric Villemain
2011/6/19 Greg Stark : > On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas wrote: >> 1. ANALYZE happens far too infrequently to believe that any data taken >> at ANALYZE time will still be relevant at execution time. >> 2. Using data gathered by ANALYZE will make plans less stable, and our >> users com

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Stark
On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas wrote: > 1. ANALYZE happens far too infrequently to believe that any data taken > at ANALYZE time will still be relevant at execution time. > 2. Using data gathered by ANALYZE will make plans less stable, and our > users complain not infrequently about

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 6:17 PM, Greg Smith wrote: > Who said anything about this being a commit candidate?  The "WIP" in the > subject says it's not intended to be.  The community asks people to submit > design ideas early so that ideas around them can be explored publicly.  One > of the things t

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 07:08 PM, Tom Lane wrote: I concur with Robert's desire to not push experimental code into the main repository, but we need to have *some* way of working with it. Maybe a separate repo where experimental branches could hang out would be helpful? Well, this one is sitting aroun

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Tom Lane
Greg Smith writes: > On 06/14/2011 01:16 PM, Robert Haas wrote: >> But there's no reason that code (which may or may not eventually prove >> useful) has to be incorporated into the main tree. We don't commit >> code so people can go benchmark it; we ask for the benchmarking to be >> done first, a

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Bruce Momjian
Greg Smith wrote: > On 06/14/2011 01:16 PM, Robert Haas wrote: > > But there's no reason that code (which may or may not eventually prove > > useful) has to be incorporated into the main tree. We don't commit > > code so people can go benchmark it; we ask for the benchmarking to be > > done first,

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 01:16 PM, Robert Haas wrote: But there's no reason that code (which may or may not eventually prove useful) has to be incorporated into the main tree. We don't commit code so people can go benchmark it; we ask for the benchmarking to be done first, and then if the results are favor

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: >> 0001-Add-reloscache-column-to-pg_class.patch > Hmm, do you really need this to be a new column? Would it work to have > it be a reloption? If it's to be updated in the same way as ANALYZE up

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Alvaro Herrera
Excerpts from Cédric Villemain's message of mar jun 14 17:10:20 -0400 2011: > If we can have ALTER TABLE running on heavy workload, why not. > I am bit scared by the effect of such reloption, it focus on HINT > oriented strategy when I would like to allow a dynamic strategy from > the server. This

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Alvaro Herrera : > Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: > >> Attached are updated patches without the plugin itself. I've also >> added the cache_page_cost GUC, this one is not per tablespace, like >> others page_cost. >> >> There are 6 patches: >> >

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Alvaro Herrera
Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: > Attached are updated patches without the plugin itself. I've also > added the cache_page_cost GUC, this one is not per tablespace, like > others page_cost. > > There are 6 patches: > > 0001-Add-reloscache-column-to-pg_

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Robert Haas : > On Tue, Jun 14, 2011 at 12:06 PM, Cédric Villemain > wrote: >>> 1. ANALYZE happens far too infrequently to believe that any data taken >>> at ANALYZE time will still be relevant at execution time. >> >> ANALYZE happens when people execute it, else it is auto-analyze and I

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 12:06 PM, Cédric Villemain wrote: >> 1. ANALYZE happens far too infrequently to believe that any data taken >> at ANALYZE time will still be relevant at execution time. > > ANALYZE happens when people execute it, else it is auto-analyze and I > am not providing auto-analyze

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 1:10 PM, Greg Smith wrote: > On 06/14/2011 11:04 AM, Robert Haas wrote: >> Even if the data were accurate and did not cause plan stability, we >> have no evidence that using it will improve real-world performance. > > That's the dependency Cédric has provided us a way to fi

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 11:04 AM, Robert Haas wrote: Even if the data were accurate and did not cause plan stability, we have no evidence that using it will improve real-world performance. That's the dependency Cédric has provided us a way to finally make progress on. Everyone says there's no evide

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Robert Haas : > On Tue, Jun 14, 2011 at 10:29 AM, Cédric Villemain > wrote: >> 0001-Add-reloscache-column-to-pg_class.patch >> 0002-Add-a-function-to-update-the-new-pg_class-cols.patch >> 0003-Add-ANALYZE-OSCACHE-VERBOSE-relation.patch >> 0004-Add-a-Hook-to-handle-OSCache-stats.patch >>

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 10:29 AM, Cédric Villemain wrote: > 0001-Add-reloscache-column-to-pg_class.patch > 0002-Add-a-function-to-update-the-new-pg_class-cols.patch > 0003-Add-ANALYZE-OSCACHE-VERBOSE-relation.patch > 0004-Add-a-Hook-to-handle-OSCache-stats.patch > 0005-Add-reloscache-to-Index-Rel-

[HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/5/16 Greg Smith : > Cédric Villemain wrote: >> >> >> http://git.postgresql.org/gitweb?p=users/c2main/postgres.git;a=shortlog;h=refs/heads/analyze_cache >> > > This rebases easily to make Cedric's changes move to the end; I just pushed > a version with that change to > https://github.com/greg2n