Re: [HACKERS] Overhauling GUCS

2008-08-19 Thread Peter Eisentraut
Am Monday, 18. August 2008 schrieb Tom Lane: > The impression I get every time this comes up is that various people > have different problems they want to solve that (they think) require > redesign of the way GUC works.  Those complicated solutions arise from > attempting to satisfy N different dem

Re: [HACKERS] Overhauling GUCS

2008-08-19 Thread Peter Eisentraut
Am Monday, 18. August 2008 schrieb Josh Berkus: > Right now, if you want to survey > your databases, tables, approx disk space, query activity, etc., you can > do that all through port 5432.  You can't manage most of your server > settings that way, and definitely can't manage the *persistent* sett

Re: [HACKERS] Overhauling GUCS

2008-08-19 Thread Magnus Hagander
Alvaro Herrera wrote: > Tom Lane escribió: >> Gregory Stark <[EMAIL PROTECTED]> writes: >>> The entire target market for such a thing is DBAs stuck on hosted databases >>> which don't have shell access to their machines. Perhaps the overlap between >>> that and the people who can write a server-sid

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Alvaro Herrera
Tom Lane escribió: > Gregory Stark <[EMAIL PROTECTED]> writes: > > The entire target market for such a thing is DBAs stuck on hosted databases > > which don't have shell access to their machines. Perhaps the overlap between > > that and the people who can write a server-side module which dumps out

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread dpage
On 8/18/08, Magnus Hagander <[EMAIL PROTECTED]> wrote: > Dave Page wrote: >> On Mon, Aug 18, 2008 at 8:51 PM, Gregory Stark <[EMAIL PROTECTED]> >> wrote: >>> The main problem that I've seen described is what I mentioned before: >>> allowing >>> adjusting the postgresql.conf GUC settings by remote u

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Magnus Hagander
Dave Page wrote: > On Mon, Aug 18, 2008 at 8:51 PM, Gregory Stark <[EMAIL PROTECTED]> wrote: >> The main problem that I've seen described is what I mentioned before: >> allowing >> adjusting the postgresql.conf GUC settings by remote users who don't have >> shell access. > > Which pgAdmin has don

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Greg Smith
On Mon, 18 Aug 2008, Gregory Stark wrote: Because coping with free-form user-edited text is a losing game. People don't consider it because it's a dead-end. Right, that's impossible technology to build, which is why I had to plan all these screen shots showing tools that handle that easily fo

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Dave Page
On Mon, Aug 18, 2008 at 8:51 PM, Gregory Stark <[EMAIL PROTECTED]> wrote: > The main problem that I've seen described is what I mentioned before: allowing > adjusting the postgresql.conf GUC settings by remote users who don't have > shell access. Which pgAdmin has done perfectly well for years, as

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Josh Berkus
Greg, > The main problem that I've seen described is what I mentioned before: > allowing adjusting the postgresql.conf GUC settings by remote users who > don't have shell access. Oh, ok. I think we're in agreement, though. I don't think that's the *1st* problem to be solved, but it's definitel

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Gregory Stark
"Josh Berkus" <[EMAIL PROTECTED]> writes: > Greg, > >> The entire target market for such a thing is DBAs stuck on hosted >> databases which don't have shell access to their machines. > > That's incorrect. The main reason for having a port-based API (such as the > SQL command line) for managing

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > The entire target market for such a thing is DBAs stuck on hosted databases > which don't have shell access to their machines. Perhaps the overlap between > that and the people who can write a server-side module which dumps out a > config file according t

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Josh Berkus
Greg, > The entire target market for such a thing is DBAs stuck on hosted > databases which don't have shell access to their machines. That's incorrect. The main reason for having a port-based API (such as the SQL command line) for managing your server is that it makes it much easier to manag

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Gregory Stark
"Greg Smith" <[EMAIL PROTECTED]> writes: > On Mon, 18 Aug 2008, Michael Nacos wrote: > >> Having done a SELECT * FROM pg_settings, all the information you need seems >> to be there... > > See http://archives.postgresql.org/pgsql-hackers/2008-06/msg00209.php You > sound > like you're at rung 2 on

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Greg Smith
On Mon, 18 Aug 2008, Michael Nacos wrote: Having done a SELECT * FROM pg_settings, all the information you need seems to be there... See http://archives.postgresql.org/pgsql-hackers/2008-06/msg00209.php You sound like you're at rung 2 on the tool author ladder I describe there, still thinkin

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Magnus Hagander
Steve Atkins wrote: > > On Aug 18, 2008, at 1:05 AM, Magnus Hagander wrote: > >> Josh Berkus wrote: >>> Steve, >>> First pass is done. Needs a little cleanup before sharing. I spent a fair while down OS-specific-hardware-queries rathole, but I'm better now. >>> >>> Gods, I hope you

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Steve Atkins
On Aug 18, 2008, at 1:05 AM, Magnus Hagander wrote: Josh Berkus wrote: Steve, First pass is done. Needs a little cleanup before sharing. I spent a fair while down OS-specific-hardware-queries rathole, but I'm better now. Gods, I hope you gave up on that. You want to use SIGAR or somet

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Michael Nacos
What I'm interested in is auto-tuning, not necessarily overhauling GUCS, which happens to be the subject of this thread :-) Having done a SELECT * FROM pg_settings, all the information you need seems to be there... Maybe I'm being over-simplistic here, but the important bit is knowing how you shoul

Re: [HACKERS] Overhauling GUCS

2008-08-18 Thread Magnus Hagander
Josh Berkus wrote: > Steve, > >> First pass is done. Needs a little cleanup before sharing. I spent a >> fair while down OS-specific-hardware-queries rathole, but I'm better now. > > Gods, I hope you gave up on that. You want to use SIGAR or something. If it's going to be C++, and reasonably cr

Re: [HACKERS] Overhauling GUCS

2008-08-17 Thread Josh Berkus
Steve, First pass is done. Needs a little cleanup before sharing. I spent a fair while down OS-specific-hardware-queries rathole, but I'm better now. Gods, I hope you gave up on that. You want to use SIGAR or something. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Overhauling GUCS

2008-08-17 Thread Steve Atkins
On Aug 17, 2008, at 1:48 PM, Greg Smith wrote: On Wed, 13 Aug 2008, Michael Nacos wrote: Hi there... Configuration autotuning is something I am really interested in. I have seen this page: http://wiki.postgresql.org/wiki/ GUCS_Overhaul and a couple of emails mentioning this, so I wanted to

Re: [HACKERS] Overhauling GUCS

2008-08-17 Thread Greg Smith
On Sun, 17 Aug 2008, Tom Lane wrote: What we have now was named Grand Unified Configuration for a reason: it centralized the handling of what had been a mess of different things configured in different ways. I'm not eager to go backwards on that. No need to change anything related to how the

Re: [HACKERS] Overhauling GUCS

2008-08-17 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > Josh Berkus and I have been exchanging some ideas for the GUC internals > overhaul and had a quick discussion about that in person last month. > We've been gravitating toward putting all the extra information we'd like > to push into there in an extra cat

Re: [HACKERS] Overhauling GUCS

2008-08-17 Thread Greg Smith
On Wed, 13 Aug 2008, Michael Nacos wrote: Hi there... Configuration autotuning is something I am really interested in. I have seen this page: http://wiki.postgresql.org/wiki/GUCS_Overhaul and a couple of emails mentioning this, so I wanted to ask is someone already on it? If yes, I'd like to con

Re: [HACKERS] Overhauling GUCS

2008-08-13 Thread Michael Nacos
Hi there... Configuration autotuning is something I am really interested in. I have seen this page: http://wiki.postgresql.org/wiki/GUCS_Overhaul and a couple of emails mentioning this, so I wanted to ask is someone already on it? If yes, I'd like to contribute. Ideally, an external little app sho

Re: [HACKERS] Overhauling GUCS

2008-07-16 Thread Bruce Momjian
Added to TODO: o Add external tool to auto-tune some postgresql.conf parameters http://archives.postgresql.org/pgsql-hackers/2008-06/msg0.php --- Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> w

Re: [HACKERS] Overhauling GUCS

2008-06-18 Thread Magnus Hagander
Greg Smith wrote: > On Thu, 5 Jun 2008, Magnus Hagander wrote: > >> We really need a "proper API" for it, and the stuff in pgAdmin isn't >> even enough to base one on. > > I would be curious to hear your opinion on whether the GUC overhaul > discussed in this thread is a useful precursor to build

Re: [HACKERS] Overhauling GUCS

2008-06-13 Thread Bruce Momjian
Robert Treat wrote: > On Wednesday 11 June 2008 16:54:23 Bruce Momjian wrote: > > Dave Page wrote: > > > On Wed, Jun 11, 2008 at 6:56 PM, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > > > Dave Page wrote: > > > >> On Mon, Jun 2, 2008 at 5:46 AM, Joshua D. Drake <[EMAIL PROTECTED]> > wrote: > > > >>

Re: [HACKERS] Overhauling GUCS

2008-06-13 Thread Robert Treat
On Wednesday 11 June 2008 16:54:23 Bruce Momjian wrote: > Dave Page wrote: > > On Wed, Jun 11, 2008 at 6:56 PM, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > > Dave Page wrote: > > >> On Mon, Jun 2, 2008 at 5:46 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > > >> > pg_ctl -D data check? > > >> >

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread James William Pye
You guys call this "simplification"? You're out of your minds. This proposal is ridiculously complicated, and yet it still fails even to consider adjusting non-numeric parameters. And what about things that require more than a trivial arithmetic expression to compute? It's not hard at all to im

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 >> Really? I'm the opposite: I never leave a client's setting at 10, that's >> just asking for trouble. Making it 100 *after* you encounter problem >> queries is reactive; I prefer being proactive. > Have you ever measured the system speed befo

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Tom Lane
Decibel! <[EMAIL PROTECTED]> writes: > On Jun 11, 2008, at 9:34 PM, Bruce Momjian wrote: >> I am kind of lost how this would work logically and am willing to >> think >> about it some more, but I do think we aren't going to simplify >> postgresql.conf without such a facility. > Agreed. And I thi

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Decibel!
On Jun 11, 2008, at 9:34 PM, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: Tom Lane wrote: The idea has a fundamental logical flaw, which is that it's not clear which parameter wins if the user changes both. Yes, you could get into problems by having variab

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Greg Smith
On Thu, 12 Jun 2008, Bruce Momjian wrote: I am thinking a web-based wizard would make the most sense. I have not a single customer I work with who could use an external web-based wizard. Way too many companies have privacy policy restrictions that nobody dare cross by giving out any info ab

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Steve Atkins
On Jun 12, 2008, at 11:21 AM, Bruce Momjian wrote: Josh Berkus wrote: Bruce, I am concerned that each wizard is going to have to duplicate the same logic each time, and adjust to release-based changes. I think that's a feature, not a bug. Right now, I'm not at all convinced that my al

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Bruce Momjian
Josh Berkus wrote: > Bruce, > > > I am thinking a web-based wizard would make the most sense. > > I'd prefer command-line, so that people could run it on their own servers. > For one thing, we need to generate at least two files on many platforms; a > postgresql.conf and a sysctl. They can ju

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Josh Berkus
Bruce, > I am thinking a web-based wizard would make the most sense. I'd prefer command-line, so that people could run it on their own servers. For one thing, we need to generate at least two files on many platforms; a postgresql.conf and a sysctl. -- Josh Berkus PostgreSQL @ Sun San Francis

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Bruce Momjian
Josh Berkus wrote: > Bruce, > > > I am concerned that each wizard is going to have to duplicate the same > > logic each time, and adjust to release-based changes. > > I think that's a feature, not a bug. Right now, I'm not at all convinced > that > my algorithms for setting the various major

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Josh Berkus
Bruce, > I am concerned that each wizzard is going to have to duplicate the same > logic each time, and adjust to release-based changes. I think that's a feature, not a bug. Right now, I'm not at all convinced that my algorithms for setting the various major dials are great (I just think that

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Gregory Stark
"Greg Sabino Mullane" <[EMAIL PROTECTED]> writes: >> Also, I'd actually assert that "10" seems to be perfectly adequate for >> the majority of users. That is, the number of users where I've >> recommended increasing d_s_t for the whole database is smaller than the >> number where I don't, and of

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Tom Lane
"Greg Sabino Mullane" <[EMAIL PROTECTED]> writes: > The orders of magnitude speed up of certain queries when the d_s_t goes > above 98 is what spawned my original thread proposing a change to 100: > http://markmail.org/message/tun3a3juxlsyjbsw That was a pretty special case (LIKE/regex estimation)

Re: [HACKERS] Overhauling GUCS

2008-06-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > Also, I'd actually assert that "10" seems to be perfectly adequate for > the majority of users. That is, the number of users where I've > recommended increasing d_s_t for the whole database is smaller than the > number where I don't, and of c

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Greg Smith wrote: > On Wed, 11 Jun 2008, Tom Lane wrote: > > > Who said anything about loops? What I am talking about is what happens > > during > > set memory_usage = X; // implicitly sets work_mem = X/100, say > > set work_mem = Y; > > set memory_usage = Z; > > What is work_mem now

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> The idea has a fundamental logical flaw, which is that it's not clear > >> which parameter wins if the user changes both. > > > Yes, you could get into problems by having variable dependency loops, > > Who said a

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Greg Smith
On Wed, 11 Jun 2008, Tom Lane wrote: Who said anything about loops? What I am talking about is what happens during set memory_usage = X; // implicitly sets work_mem = X/100, say set work_mem = Y; set memory_usage = Z; What is work_mem now, and what's your excuse for say

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> I agree with that for pg_clog and friends, but I'm much more leery of >> folding WAL into the same framework. > Well it may still be worthwhile stealing buffers from shared_buffers even if > we set a special flag

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Gregory Stark
"Josh Berkus" <[EMAIL PROTECTED]> writes: > Greg, > >> At least that way we could always steal more if we want or return some, as >> long as we're careful about when we do it. That would open the door to having >> these parameters be dynamically adjustable. That alone would be worthwhile >> even i

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Bruce Momjian <[EMAIL PROTECTED]> writes: >>> The second idea is the idea of having one parameter depend on another. >> We have tried to do that in the past, and it didn't work well *at all*. > We have? When? Just a couple months a

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Magnus Hagander wrote: > Bruce Momjian wrote: > > Joshua D. Drake wrote: > >> > >> On Wed, 2008-06-11 at 11:45 -0400, Bruce Momjian wrote: > >>> Greg Sabino Mullane wrote: > * The word 'paramters' is still misspelled. :) > >>> Corrected for 8.4. > >> Technically this is a bug fix... why not ba

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Josh Berkus
Greg, At least that way we could always steal more if we want or return some, as long as we're careful about when we do it. That would open the door to having these parameters be dynamically adjustable. That alone would be worthwhile even if we bypass all bells and whistles of the buffer manager

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Alvaro Herrera <[EMAIL PROTECTED]> writes: >> Josh Berkus wrote: >>> Ideally, of course, there would be no wal_buffers setting, and WAL >>> buffers would be allocated from shared_buffers pool on demand... > >> Same for pg_subtrans, pg_clog, etc (as previo

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > The second idea is the idea of having one parameter depend on another. > > Not only could we do that for some of our existing parameters, but we > > could have pseudo-parameters like concurrent_queries, memory_usage, and > > extra_dis

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > The second idea is the idea of having one parameter depend on another. > Not only could we do that for some of our existing parameters, but we > could have pseudo-parameters like concurrent_queries, memory_usage, and > extra_disk_space that could be at t

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> * Can we build a "configuration wizard" to tell newbies what settings > >> they need to tweak? > > > That would trump all the other suggestions conclusively. Anyone good at > > expert systems? > > How far could

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Josh Berkus wrote: >> Ideally, of course, there would be no wal_buffers setting, and WAL >> buffers would be allocated from shared_buffers pool on demand... > Same for pg_subtrans, pg_clog, etc (as previously discussed) I agree with that for pg_clog

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Dave Page wrote: > On Wed, Jun 11, 2008 at 6:56 PM, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > Dave Page wrote: > >> On Mon, Jun 2, 2008 at 5:46 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > >> > >> > pg_ctl -D data check? > >> > > >> > I would +1 that. > >> > >> I would also really like to se

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Dave Page
On Wed, Jun 11, 2008 at 6:56 PM, Bruce Momjian <[EMAIL PROTECTED]> wrote: > Dave Page wrote: >> On Mon, Jun 2, 2008 at 5:46 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: >> >> > pg_ctl -D data check? >> > >> > I would +1 that. >> >> I would also really like to see that - though I'd also like to se

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Dave Page wrote: > On Mon, Jun 2, 2008 at 5:46 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > > >> I've seen people not doing so more often > >> than you would think. Perhaps because they are DBAs and not sysadmins? I > >> also > >> meant a tool to do things like verify that the changes are vali

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Jignesh K. Shah
Josh Berkus wrote: Heikki, Ideally, of course, there would be no wal_buffers setting, and WAL buffers would be allocated from shared_buffers pool on demand... +1 --Josh +1 -Jignesh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Magnus Hagander
Bruce Momjian wrote: > Joshua D. Drake wrote: >> >> On Wed, 2008-06-11 at 11:45 -0400, Bruce Momjian wrote: >>> Greg Sabino Mullane wrote: * The word 'paramters' is still misspelled. :) >>> Corrected for 8.4. >> Technically this is a bug fix... why not backpatch it too? > > That might show up

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Joshua D. Drake wrote: > > > On Wed, 2008-06-11 at 11:45 -0400, Bruce Momjian wrote: > > Greg Sabino Mullane wrote: > > > * The word 'paramters' is still misspelled. :) > > > > Corrected for 8.4. > > Technically this is a bug fix... why not backpatch it too? That might show up as a diff for pe

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Robert Lor wrote: > Robert Treat wrote: > > On Wednesday 04 June 2008 22:04:54 Greg Smith wrote: > > > >> I was just talking to someone today about building a monitoring tool for > >> this. Not having a clear way to recommend people monitor use of work_mem > >> and its brother spilled to disk s

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Alvaro Herrera
Josh Berkus wrote: > Heikki, > >> Ideally, of course, there would be no wal_buffers setting, and WAL >> buffers would be allocated from shared_buffers pool on demand... Same for pg_subtrans, pg_clog, etc (as previously discussed) -- Alvaro Herrerahttp://www.Comm

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Josh Berkus
Tom Lane wrote: Josh Berkus <[EMAIL PROTECTED]> writes: Oh, and wal_buffers, the default for which we should just change if it weren't for SHMMAX. Uh, why? On a workload of mostly small transactions, what value is there in lots of wal_buffers? Actually, it's also useful for any workload wit

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Josh Berkus
Heikki, Ideally, of course, there would be no wal_buffers setting, and WAL buffers would be allocated from shared_buffers pool on demand... +1 --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Joshua D. Drake
On Wed, 2008-06-11 at 11:45 -0400, Bruce Momjian wrote: > Greg Sabino Mullane wrote: > > * The word 'paramters' is still misspelled. :) > > Corrected for 8.4. Technically this is a bug fix... why not backpatch it too? > > -- > Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us >

Re: [HACKERS] Overhauling GUCS

2008-06-11 Thread Bruce Momjian
Greg Sabino Mullane wrote: > * The word 'paramters' is still misspelled. :) Corrected for 8.4. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + --

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Heikki Linnakangas
Tom Lane wrote: Josh Berkus <[EMAIL PROTECTED]> writes: Oh, and wal_buffers, the default for which we should just change if it weren't for SHMMAX. Uh, why? On a workload of mostly small transactions, what value is there in lots of wal_buffers? None. But there's also little to no harm in hav

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Ron Mayer
Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: ...default_statistics_target?"..."Uhh 10." Ah, but we only ever hear about the cases where it's wrong of course. In other words even if we raised it to some optimal value we would still have precisely the same experience of see

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Oh, and wal_buffers, the default for which we should just change if it > weren't for SHMMAX. Uh, why? On a workload of mostly small transactions, what value is there in lots of wal_buffers? regards, tom lane -- Sent via pgsql-ha

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Josh Berkus
On Tuesday 10 June 2008 09:37, Josh Berkus wrote: > Robert, > > > shared_buffers > > effective_cache_size > > default_stats_target > > work_mem > > maintainance_work_mem > > listen_address > > max_connections > > the fsm parameters > > checkpoint_segements > > random_page_cost > > My list is very s

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Josh Berkus
Robert, > shared_buffers > effective_cache_size > default_stats_target > work_mem > maintainance_work_mem > listen_address > max_connections > the fsm parameters > checkpoint_segements > random_page_cost My list is very similar, execept that I drop random_page_cost and add synchronous_commit, au

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Josh Berkus
Ron, > I wonder if the fastest way to generate the configurator > would be to simply ask everyone to post their tuned > postgresql.conf files along with a brief description of > the use case for that file. The we could group the > use-cases into various classes; and average the values > of the su

Re: [HACKERS] Overhauling GUCS

2008-06-10 Thread Gregory Stark
"Josh Berkus" <[EMAIL PROTECTED]> writes: > Greg, > >> Speak to the statisticians. Our sample size is calculated using the same >> theory behind polls which sample 600 people to learn what 250 million >> people are going to do on election day. You do NOT need (significantly) >> larger samples for

Re: [HACKERS] Overhauling GUCS

2008-06-09 Thread Josh Berkus
Greg, > Speak to the statisticians. Our sample size is calculated using the same > theory behind polls which sample 600 people to learn what 250 million > people are going to do on election day. You do NOT need (significantly) > larger samples for larger populations. Your analogy is bad. For ele

Re: [HACKERS] Overhauling GUCS

2008-06-09 Thread Gregory Stark
"Hakan Kocaman" <[EMAIL PROTECTED]> writes: > On 6/9/08, Gregory Stark <[EMAIL PROTECTED]> wrote: >> >> n_distinct. For that Josh is right, we *would* need a sample size >> proportional to the whole data set which would practically require us to >> scan the whole table (and have a technique for s

Re: [HACKERS] Overhauling GUCS

2008-06-09 Thread Hakan Kocaman
On 6/9/08, Gregory Stark <[EMAIL PROTECTED]> wrote: > > "Josh Berkus" <[EMAIL PROTECTED]> writes: > > > Where analyze does systematically fall down is with databases over 500GB > in > > size, but that's not a function of d_s_t but rather of our tiny sample > size. > > > n_distinct. For that Josh is

Re: [HACKERS] Overhauling GUCS

2008-06-09 Thread Gregory Stark
"Josh Berkus" <[EMAIL PROTECTED]> writes: > Where analyze does systematically fall down is with databases over 500GB in > size, but that's not a function of d_s_t but rather of our tiny sample size. Speak to the statisticians. Our sample size is calculated using the same theory behind polls which

Re: [HACKERS] Overhauling GUCS

2008-06-09 Thread Josh Berkus
Tom, Actually, the reason it's still 10 is that the effort expended to get it changed has been *ZERO*. I keep asking for someone to make some measurements, do some benchmarking, anything to make a plausible case for a specific higher value as being a reasonable place to set it. The silence has

Re: [HACKERS] Overhauling GUCS

2008-06-08 Thread Robert Treat
On Sunday 08 June 2008 19:07:21 Gregory Stark wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > On Fri, 2008-06-06 at 20:19 -0400, Tom Lane wrote: > >> Robert Treat <[EMAIL PROTECTED]> writes: > >> > >> Actually, the reason it's still 10 is that the effort expended to get it > >> changed h

Re: [HACKERS] Overhauling GUCS

2008-06-08 Thread Gregory Stark
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > On Fri, 2008-06-06 at 20:19 -0400, Tom Lane wrote: >> Robert Treat <[EMAIL PROTECTED]> writes: > >> Actually, the reason it's still 10 is that the effort expended to get it >> changed has been *ZERO*. I keep asking for someone to make some >> measur

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > Not surprising really. It is a simple adjustment to make and it also is > easy to spot when its a problem. However it is not trivial to test for > (in terms of time and effort). I know 10 is wrong and so do you. Sure. But what is right? I'm afraid

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Joshua D. Drake
On Fri, 2008-06-06 at 20:19 -0400, Tom Lane wrote: > Robert Treat <[EMAIL PROTECTED]> writes: > Actually, the reason it's still 10 is that the effort expended to get it > changed has been *ZERO*. I keep asking for someone to make some > measurements, do some benchmarking, anything to make a pla

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Greg Smith
On Fri, 6 Jun 2008, Tom Lane wrote: Well, you can't see the default or reset values in pg_settings, only the current value. However, I fail to see the use of either of those for a configure wizard. I'm under the impression that the primary reason to put the default in there is to make it eas

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > On Fri, 6 Jun 2008, Gregory Stark wrote: >> "Greg Smith" <[EMAIL PROTECTED]> writes: >>> 1) Is it worthwhile to expand the information stored in the GUC structure to >>> make it better capable of supporting machine generation and to provide more >>> informat

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Tom Lane
Robert Treat <[EMAIL PROTECTED]> writes: > There is a saying, something like "The accumulation of annecdotes is not > data". Well, we seem to have a high bar on what proof we need to actually > change a default GUC settings. default_statistics_target is a prime example, > where almost no one i

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Joshua D. Drake
On Sat, 2008-06-07 at 01:30 +0200, Andreas Pflug wrote: > Gregory Stark wrote: > > "Andreas Pflug" <[EMAIL PROTECTED]> writes: > I think I made my point very clear when stating "not a file, but via > SQL". Though I'm not a native English speaker, and I'm sure you > understood. I must assume you'

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Andreas Pflug
Gregory Stark wrote: > "Andreas Pflug" <[EMAIL PROTECTED]> writes: > >> I personally wouldn't even think about starting such a wizard, unless I have >> an >> idea how to push the result into the database. No, not a file, but via SQL! >> So >> your statement you won't react unless a wizard is alm

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Chris Browne
[EMAIL PROTECTED] (Greg Smith) writes: > On Fri, 6 Jun 2008, Heikki Linnakangas wrote: > >> Or perhaps we should explicitly mark the settings the tool has >> generated, and comment out: >> >> #shared_buffers = 32MB # commented out by wizard on 2008-06-05 >> shared_buffers = 1024MB # automaticall

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Greg Smith
On Fri, 6 Jun 2008, Gregory Stark wrote: "Greg Smith" <[EMAIL PROTECTED]> writes: 1) Is it worthwhile to expand the information stored in the GUC structure to make it better capable of supporting machine generation and to provide more information for tool authors via pg_settings? The exact fi

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Robert Lor
Robert Treat wrote: On Wednesday 04 June 2008 22:04:54 Greg Smith wrote: I was just talking to someone today about building a monitoring tool for this. Not having a clear way to recommend people monitor use of work_mem and its brother spilled to disk sorts is an issue right now, I'll whack t

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Andreas 'ads' Scherbaum
On Thu, 05 Jun 2008 12:53:55 -0700 Ron Mayer wrote: > Steve Atkins wrote: > > ... cross-platform (Windows, Linux, Solaris, OS X as a bare > > minimum) > > I wonder how cross-platform the tuning algorithm itself is. > > I could also imagine that decisions like "do I let the OS page > cache, or p

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Gregory Stark
"Greg Smith" <[EMAIL PROTECTED]> writes: > 1) Is it worthwhile to expand the information stored in the GUC structure to > make it better capable of supporting machine generation and to provide more > information for tool authors via pg_settings? The exact fields that should or > shouldn't be incl

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Ron Mayer
Joshua D. Drake wrote: Tom Lane wrote: Peter Eisentraut <[EMAIL PROTECTED]> writes: - If we know better values, why don't we set them by default? The problem is: better for what? That is where some 80% solution sample config files come in. +1. At work I use 3 templates. * One for salespeo

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Alvaro Herrera
Robert Treat wrote: > One idea I have been kicking around is having every guc have a anchor in the > website (rahter than anchoring on parameter family). It might be enough to > just populate the search bot with every guc anchored to family though... +1 on the anchor per variable. -- Alvaro

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Greg Smith
On Fri, 6 Jun 2008, Tom Lane wrote: I grow weary of this thread. If we keep it up for, oh, another three years, then maybe you'll be as weary as I am of struggling with problems in this area. Strinking a balance between the wants and needs of people who want a fancy GUI tool for configurin

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Robert Treat
On Friday 06 June 2008 08:35:00 Peter Eisentraut wrote: > Am Mittwoch, 4. Juni 2008 schrieb Tom Lane: > > * Can we present the config options in a more helpful way (this is 99% > > a documentation problem, not a code problem)? > > ack > > > * Can we build a "configuration wizard" to tell newbies wh

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Robert Treat
On Thursday 05 June 2008 15:15:14 Greg Smith wrote: > (1) is in that proposal but is strictly optional as something to put in > the configuration file itself. The idea behind (2) is to enable tool > authors to have an easier way to suggest where to head for more > information. I'd like for it to

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Greg Smith
On Fri, 6 Jun 2008, Heikki Linnakangas wrote: Or perhaps we should explicitly mark the settings the tool has generated, and comment out: #shared_buffers = 32MB # commented out by wizard on 2008-06-05 shared_buffers = 1024MB # automatically set by wizard on 2008-06-05 What I would like to

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Gregory Stark
"Andreas Pflug" <[EMAIL PROTECTED]> writes: > I personally wouldn't even think about starting such a wizard, unless I have > an > idea how to push the result into the database. No, not a file, but via SQL! So > your statement you won't react unless a wizard is almost ready is prohibitive, > apart

Re: [HACKERS] Overhauling GUCS

2008-06-06 Thread Steve Atkins
On Jun 6, 2008, at 12:22 PM, Greg Smith wrote: On Fri, 6 Jun 2008, Peter Eisentraut wrote: - What settings do "newbies" (or anyone else) typically need to change? Please post a list. - What values would you set those settings to? Please provide a description for arriving at a value, whic

  1   2   3   >