Re: [HACKERS] proposal: a validator for configuration files

2011-07-25 Thread Alexey Klyukin
On Jul 16, 2011, at 9:55 PM, Tom Lane wrote: > I wrote: >> I think that it might be sensible to have the following behavior: > >> 1. Parse the file, where "parse" means collect all the name = value >> pairs. Bail out if we find any syntax errors at that level of detail. >> (With this patch, we

Re: [HACKERS] proposal: a validator for configuration files

2011-07-23 Thread Dimitri Fontaine
Florian Pflug writes: > A variant of this would be to allow extensions (in the CREATE EXTENSION > sense) to declare custom GUCs in their control file. Then we'd only > need to load those files, which seems better than loading a shared > library. The original patch for the extensions had that feat

Re: [HACKERS] proposal: a validator for configuration files

2011-07-19 Thread Tom Lane
Josh Berkus writes: > Hmmm. As someone who often deploys pg.conf changes as part of a > production code rollout, I actually like the "atomic" nature of updating > postgresql.conf -- that is, all your changes succeed, or they all fail. If we actually *had* that, I'd agree with you. The problem i

Re: [HACKERS] proposal: a validator for configuration files

2011-07-19 Thread Tom Lane
Peter Eisentraut writes: > On sön, 2011-07-17 at 00:59 -0400, Tom Lane wrote: >> Well, we *do* have a C API for that, of a sort. The problem is, what >> do you do in processes that have not loaded the relevant extension? > Those processes that have the extension loaded check the parameter > set

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Peter Eisentraut
On sön, 2011-07-17 at 00:59 -0400, Tom Lane wrote: > Well, we *do* have a C API for that, of a sort. The problem is, what > do you do in processes that have not loaded the relevant extension? Those processes that have the extension loaded check the parameter settings in their namespace, those tha

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Josh Berkus
Tom, Florian, >> On the downside, the current behaviour prevents problems if someone changes >> two interrelated GUCs, but makes a mistake at one of them. For example, >> someone might drastically lower bgwriter_delay but might botch the matching >> adjustment of bgwriter_lru_maxpages. > > That's

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Tom Lane
Robert Haas writes: > On Sun, Jul 17, 2011 at 12:59 AM, Tom Lane wrote: >> I agree custom_variable_classes is conceptually messy, but it's a >> reasonably lightweight compromise that gives some error checking without >> requiring a lot of possibly-irrelevant extensions to be loaded into >> every

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Pavel Stehule
2011/7/18 Florian Pflug : > On Jul18, 2011, at 01:29 , Robert Haas wrote: >> Hmm.  Maybe what we need is a mechanism that allows the configuration >> to be associated a loadable module, and whenever that module is >> loaded, we also load the associated configuration settings.  This is >> probably t

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Florian Pflug
On Jul18, 2011, at 01:29 , Robert Haas wrote: > Hmm. Maybe what we need is a mechanism that allows the configuration > to be associated a loadable module, and whenever that module is > loaded, we also load the associated configuration settings. This is > probably terribly syntax, but something li

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Robert Haas
On Sun, Jul 17, 2011 at 12:59 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane wrote: >>> Robert Haas writes: Is there any way that we could get *rid* of custom_variable_classes? > >>> Well, we could just drop it and say you can set any dotted-name GU

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Robert Haas writes: > On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane wrote: >> Robert Haas writes: >>> Is there any way that we could get *rid* of custom_variable_classes? >> Well, we could just drop it and say you can set any dotted-name GUC >> you feel like. > ...and the fact that we've made the

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Robert Haas
On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane wrote: >>> 2. Tentatively apply the new custom_variable_classes setting if any. > >> Is there any way that we could get *rid* of custom_variable_classes? >> The idea of using a GU

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane wrote: >> 2. Tentatively apply the new custom_variable_classes setting if any. > Is there any way that we could get *rid* of custom_variable_classes? > The idea of using a GUC to define the set of valid GUCs seems > intrinsically pr

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Robert Haas
On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane wrote: > 2. Tentatively apply the new custom_variable_classes setting if any. Is there any way that we could get *rid* of custom_variable_classes? The idea of using a GUC to define the set of valid GUCs seems intrinsically problematic. -- Robert Haas En

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 22:55 , Tom Lane wrote: > Florian Pflug writes: >> Btw, if we touch that, I think we should think about providing some way >> to detect when a backend fails to apply a value. > > Hm, maybe, but keep in mind that there are valid reasons for a backend > to ignore a postgresql.con

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Florian Pflug writes: > Btw, if we touch that, I think we should think about providing some way > to detect when a backend fails to apply a value. Hm, maybe, but keep in mind that there are valid reasons for a backend to ignore a postgresql.conf setting --- in particular, it might have a local ov

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 21:23 , Tom Lane wrote: > Florian Pflug writes: >> On the downside, the current behaviour prevents problems if someone changes >> two interrelated GUCs, but makes a mistake at one of them. For example, >> someone might drastically lower bgwriter_delay but might botch the matchi

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Florian Pflug writes: > On the downside, the current behaviour prevents problems if someone changes > two interrelated GUCs, but makes a mistake at one of them. For example, > someone might drastically lower bgwriter_delay but might botch the matching > adjustment of bgwriter_lru_maxpages. That's

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 20:55 , Tom Lane wrote: >> The original argument for the current behavior was to avoid applying >> settings from a thoroughly munged config file, but I think that the >> checks involved in steps 1-3 would be sufficient to reject files that >> had major problems. It's possible th

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
I wrote: > I think that it might be sensible to have the following behavior: > 1. Parse the file, where "parse" means collect all the name = value > pairs. Bail out if we find any syntax errors at that level of detail. > (With this patch, we could report some or all of the syntax errors > first.)

Re: [HACKERS] proposal: a validator for configuration files

2011-07-15 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Alexey Kluykin's message of jue jul 14 09:18:15 -0400 2011: >> This is happening because a check for total number of errors so far >> is happening only after coming across at least one non-recognized >> configuration option. What about adding one more check

Re: [HACKERS] proposal: a validator for configuration files

2011-07-15 Thread Alvaro Herrera
Excerpts from Alexey Kluykin's message of jue jul 14 09:18:15 -0400 2011: > > On Jul 14, 2011, at 4:38 AM, Alvaro Herrera wrote: > > >> On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: > This is happening because a check for total number of errors so far is > happening only after coming across

Re: [HACKERS] proposal: a validator for configuration files

2011-07-14 Thread Alexey Klyukin
On Jul 14, 2011, at 4:38 AM, Alvaro Herrera wrote: > Excerpts from Florian Pflug's message of mié jul 13 20:12:28 -0400 2011: >> On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: >>> One strange thing here is that you could get two such messages; say if a >>> file has 100 parse errors and there ar

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Alvaro Herrera
Excerpts from Florian Pflug's message of mié jul 13 20:12:28 -0400 2011: > On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: > > One strange thing here is that you could get two such messages; say if a > > file has 100 parse errors and there are also valid lines that contain > > bogus settings (foo

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Florian Pflug
On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: > One strange thing here is that you could get two such messages; say if a > file has 100 parse errors and there are also valid lines that contain > bogus settings (foo = bar). I don't find this to be too problematic, > and I think fixing it would b

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Alvaro Herrera
Excerpts from Alexey Kluykin's message of mar jun 21 07:43:02 -0400 2011: > > Another benefit of removing the check is that it reduces code complexity. > > Maybe > > not when measured in line counts, but it's one less outside factor that > > changes > > ProcessConfigFiles()'s behaviour and thus

Re: [HACKERS] proposal: a validator for configuration files

2011-06-21 Thread Alexey Klyukin
On Jun 20, 2011, at 6:22 PM, Florian Pflug wrote: > On Jun20, 2011, at 17:02 , Alexey Klyukin wrote: >> >> I don't think it has changed at all. Previously, we did goto cleanup_list (or >> cleanup_exit in ParseConfigFp) right after the first error, no matter whether >> that was a postmaster or it

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Tom Lane
Florian Pflug writes: > On Jun20, 2011, at 18:16 , Tom Lane wrote: >> This is already known to happen: there are cases where the postmaster >> and a backend can come to different conclusions about whether a setting >> is valid (eg, because it depends on database encoding). Whether that's >> a bug

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Florian Pflug
On Jun20, 2011, at 18:16 , Tom Lane wrote: > Florian Pflug writes: >> The code the actually implements the "check settings first, apply later" >> logic >> isn't easy to read. Now, assume that this code has a bug. Then, with your >> patch applied, we might end up with the postmaster applying a set

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Tom Lane
Florian Pflug writes: > The code the actually implements the "check settings first, apply later" logic > isn't easy to read. Now, assume that this code has a bug. Then, with your > patch applied, we might end up with the postmaster applying a setting (because > it didn't abort early) but the backe

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Florian Pflug
On Jun20, 2011, at 17:02 , Alexey Klyukin wrote: > On Jun 18, 2011, at 5:40 PM, Florian Pflug wrote: >> On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: >>> Attached is the v2 of the patch to show all parse errors at postgresql.conf. >>> Changes (per review and suggestions from Florian): >>> >>> -

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Alexey Klyukin
Florian, On Jun 18, 2011, at 5:40 PM, Florian Pflug wrote: > On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: >> Attached is the v2 of the patch to show all parse errors at postgresql.conf. >> Changes (per review and suggestions from Florian): >> >> - do not stop on the first error during postma

Re: [HACKERS] proposal: a validator for configuration files

2011-06-18 Thread Florian Pflug
On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: > Attached is the v2 of the patch to show all parse errors at postgresql.conf. > Changes (per review and suggestions from Florian): > > - do not stop on the first error during postmaster's start. > - fix errors in processing files from include direc

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
Hi, On Jun 16, 2011, at 9:18 PM, Florian Pflug wrote: > On Jun16, 2011, at 20:14 , Alexey Klyukin wrote: >> >> Well, while thinking about this I decided to leave the counter for the >> ParseConfigFp, but >> drop it in ProcessConfigFile. The case we are protecting against is a single >> file f

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 20:14 , Alexey Klyukin wrote: > On Jun 16, 2011, at 8:01 PM, Florian Pflug wrote: >> On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: >>> I just recalled a reason for counting the total number of errors. There is >>> a condition that >>> checks that the total number of errors is

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
On Jun 16, 2011, at 8:01 PM, Florian Pflug wrote: > On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: >> On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: >>> Hm, wouldn't a test for "context == PGC_POSTMASTER" be more appropriate? >> >> In such a case the errors caused by command-line arguments

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: > On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: >> Hm, wouldn't a test for "context == PGC_POSTMASTER" be more appropriate? > > In such a case the errors caused by command-line arguments won't stop the > postmaster. > PGC_S_FILE seems to handle

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: > On Jun16, 2011, at 17:23 , Alexey Klyukin wrote: >> On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: >>> The first problem I ran into when I tried to test this is that it *only* >>> reports multiple errors during config file reload on SIHUP, no

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 17:23 , Alexey Klyukin wrote: > On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: >> The first problem I ran into when I tried to test this is that it *only* >> reports multiple errors during config file reload on SIHUP, not during >> postmaster startup. I guess it's been done th

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
Florian, On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: > Hi > > On May14, 2011, at 00:49 , Alexey Klyukin wrote: >> The patch forces the parser to report all errors (max 100) from the >> ProcessConfigFile/ParseConfigFp. Currently, only the first parse error or an >> invalid directive is repo

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
Hi On May14, 2011, at 00:49 , Alexey Klyukin wrote: > The patch forces the parser to report all errors (max 100) from the > ProcessConfigFile/ParseConfigFp. Currently, only the first parse error or an > invalid directive is reported. Reporting all of them is crucial to automatic > validation of po

Re: [HACKERS] proposal: a validator for configuration files

2011-05-13 Thread Alexey Klyukin
Hi, On Apr 14, 2011, at 9:50 PM, Robert Haas wrote: > On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin > wrote: >> Here's the update of Selena's patch, which also shows all errors in >> configuration parameters (as well as parser errors) during reload. > > You should add this here: > > https://

Re: [HACKERS] proposal: a validator for configuration files

2011-04-14 Thread Robert Haas
On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin wrote: > Here's the update of Selena's patch, which also shows all errors in > configuration parameters (as well as parser errors) during reload. You should add this here: https://commitfest.postgresql.org/action/commitfest_view/open On a quick gla

Re: [HACKERS] proposal: a validator for configuration files

2011-04-04 Thread Alexey Klyukin
On Apr 1, 2011, at 12:08 AM, Alexey Klyukin wrote: > Hi Selena, > > On Mar 30, 2011, at 11:42 PM, Selena Deckelmann wrote: > >> Hi! >> >> On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin >> wrote: >> >> >> I did a little bit of work on this, and we discussed it here: >> >> http://archives.

Re: [HACKERS] proposal: a validator for configuration files

2011-03-31 Thread Alexey Klyukin
Hi Selena, On Mar 30, 2011, at 11:42 PM, Selena Deckelmann wrote: > Hi! > > On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin > wrote: > > > I did a little bit of work on this, and we discussed it here: > > http://archives.postgresql.org/pgsql-hackers/2009-03/msg00345.php > http://archives.po

Re: [HACKERS] proposal: a validator for configuration files

2011-03-30 Thread Selena Deckelmann
Hi! On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin wrote: > I'd like to add an ability to validate the corectness of PostgreSQL > configuration files, i.e. postgresql.conf, pg_hba.conf, pg_ident.conf without > actually applying them. The idea is that it would be a command-line option to > postg

[HACKERS] proposal: a validator for configuration files

2011-03-30 Thread Alexey Klyukin
Hi, I'd like to add an ability to validate the corectness of PostgreSQL configuration files, i.e. postgresql.conf, pg_hba.conf, pg_ident.conf without actually applying them. The idea is that it would be a command-line option to postgres for a stand-alone case, or a user-callable function when post