Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-28 Thread Michael Paquier
On Mon, Nov 28, 2022 at 04:12:40PM +0900, Michael Paquier wrote: > Attached is a rebased patch of the rest. With everything we have > dealt with in this CF, perhaps it would be better to mark this entry > as committed and switch to a new thread where the negative TAP tests > could be discussed? F

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 07:04:46PM +0800, Julien Rouhaud wrote: > And here's the rebased patch for the TAP tests. I will switch the CF entry to > Needs Review. I have been looking at that, and applied the part 1 of the test for the positive tests to cover the basic ground I'd like to see covered

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 09:49:31PM +0900, Ian Lawrence Barwick wrote: > Thanks for the quick update! FWIW, I do intend to tackle this last part ASAP, as the last piece to commit to get the full picture in the tree. -- Michael signature.asc Description: PGP signature

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Ian Lawrence Barwick
2022年11月27日(日) 20:04 Julien Rouhaud : > > On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > > > > I'm trying to reconcile open CommitFest entries with their actual > > > status; the entry for this: > > > > > > https://

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Julien Rouhaud
On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > I'm trying to reconcile open CommitFest entries with their actual > > status; the entry for this: > > > > https://commitfest.postgresql.org/40/3558/ > > > > shows as "Wa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Julien Rouhaud
Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > I'm trying to reconcile open CommitFest entries with their actual > status; the entry for this: > > https://commitfest.postgresql.org/40/3558/ > > shows as "Waiting on Author", but looks like it's all been committed; > is there anything > le

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Ian Lawrence Barwick
2022年11月25日(金) 11:25 Julien Rouhaud : > > On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached > > > v22 > > > that fixes it in all the plac

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > > that fixes it in all the places I found. > > Sounds fine. Added one comment, fixed

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Michael Paquier
On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > that fixes it in all the places I found. Sounds fine. Added one comment, fixed one comment, and applied. Thanks! -- Michael signature.asc Description:

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
Hi, On Thu, Nov 24, 2022 at 02:37:23PM +0800, Julien Rouhaud wrote: > On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > > The depth 0 is getting used quite a lot now, maybe we should have a > > > define for > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Julien Rouhaud
On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > The depth 0 is getting used quite a lot now, maybe we should have a define > > for > > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Michael Paquier
On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > The depth 0 is getting used quite a lot now, maybe we should have a define for > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like that? > And also add a define for the magical 10 for the max inclusion depth, f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Julien Rouhaud
Hi, Sorry for the very late answer, I had quite a lot of other things going on recently. And thanks for taking care of the patchset! On Wed, Nov 23, 2022 at 03:05:18PM +0900, Michael Paquier wrote: > On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > > + /* XXX: this should s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > + /* XXX: this should stick to elevel for some cases? */ > + ereport(LOG, > + (errmsg("skipping missing authentication file \"%s\"", > + inc_fullname))); > Should we always issue a LOG here?

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
On Thu, Nov 17, 2022 at 11:33:05AM +0900, Michael Paquier wrote: > By the way, I am wondering whether process_included_authfile() is > the most intuitive interface here. The only thing that prevents a > common routine to process the include commands is the failure on > GetConfFilesInDir(), where w

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-16 Thread Michael Paquier
On Wed, Nov 16, 2022 at 10:53:02AM +0800, Julien Rouhaud wrote: > While being the same inclusion infrastructure, it's likely that people will > have different usage. I'm assuming that for GUCs the main usage is to have > your automation tool put one of your template conf for instance > (small_vm.

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
Le mer. 16 nov. 2022 à 13:01, Ian Lawrence Barwick a écrit : > 2022年11月14日(月) 14:41 Michael Paquier : > > > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > > It's looks good to me. I agree that file name and line number should > be enough > > > to diagnose any unexpected er

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Ian Lawrence Barwick
2022年11月14日(月) 14:41 Michael Paquier : > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have looked at 0001 and 0002 again wi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
On Tue, Nov 15, 2022 at 08:46:55AM +0900, Michael Paquier wrote: > On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > > > > If you have an include_dir directive and multiple files have wrong > > permission > > (or maybe broken symlink or something like that), you will get multiple

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: >> If the caller passes NULL for *linectx as the initial line context, >> just create it as we do now. If *linectx is not NULL, just reuse it. >> That may be cleaner t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Julien Rouhaud
Hi, On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have loo

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Michael Paquier
On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > It's looks good to me. I agree that file name and line number should be > enough > to diagnose any unexpected error. Thanks for checking. I have looked at 0001 and 0002 again with a fresh mind, and applied both of them this morni

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-12 Thread Julien Rouhaud
On Thu, Nov 10, 2022 at 10:29:40AM +0900, Michael Paquier wrote: > > FWIW, I have been playing with the addition of a ErrorContextCallback > in tokenize_auth_file(), and this addition leads to a really nice > result. With this method, it is possible to know the full chain of > events leading to a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-09 Thread Michael Paquier
On Wed, Nov 09, 2022 at 12:09:01PM +0800, Julien Rouhaud wrote: > On Wed, Nov 09, 2022 at 09:51:17AM +0900, Michael Paquier wrote: >> Julien, please note that this is waiting on author for now. What do >> you think about the now-named v18-0001 and the addition of an >> ErrorContextCallback to prov

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Julien Rouhaud
Hi, On Wed, Nov 09, 2022 at 09:51:17AM +0900, Michael Paquier wrote: > On Tue, Nov 08, 2022 at 10:04:16AM +0900, Michael Paquier wrote: > > CF bot unhappy as I have messed up with rules.out. Rebased. I have > > removed the restriction on MAXPGPATH in AbsoluteConfigLocation() in > > 0001, while o

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Michael Paquier
On Tue, Nov 08, 2022 at 10:04:16AM +0900, Michael Paquier wrote: > CF bot unhappy as I have messed up with rules.out. Rebased. I have > removed the restriction on MAXPGPATH in AbsoluteConfigLocation() in > 0001, while on it. The absolute paths built on GUC or ident > inclusions are the same. Re

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-07 Thread Michael Paquier
On Mon, Nov 07, 2022 at 03:07:15PM +0900, Michael Paquier wrote: > Attached is a set of three patches: > - 0001 changes tokenize_inc_file() to use AbsoluteConfigLocation(). > AbsoluteConfigLocation() uses a static buffer and a MAXPGPATH, but > we'd rather change it to use a palloc()+strcpy() instea

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-06 Thread Michael Paquier
On Thu, Nov 03, 2022 at 08:55:05PM +0900, Michael Paquier wrote: > On Wed, Nov 02, 2022 at 09:06:02PM +0800, Julien Rouhaud wrote: >>> The addition of a check for the depth in two places seems unnecessary, >>> and it looks like we should do this kind of check in only one place. >> >> I usually pre

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-03 Thread Michael Paquier
On Wed, Nov 02, 2022 at 09:06:02PM +0800, Julien Rouhaud wrote: > Maybe one alternative approach would be to keep modifying the current test, > and > only do the split when committing the first part? The split itself should be > easy and mechanical: just remove everything unneeded (different file

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Julien Rouhaud
On Wed, Nov 02, 2022 at 04:46:48PM +0900, Michael Paquier wrote: > On Fri, Oct 28, 2022 at 11:49:54AM +0800, Julien Rouhaud wrote: > > To be honest I'd rather not to. It's excessively annoying to work on those > > tests (I spent multiple days trying to make it as clean and readable as > > possible

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Michael Paquier
On Fri, Oct 28, 2022 at 11:49:54AM +0800, Julien Rouhaud wrote: > To be honest I'd rather not to. It's excessively annoying to work on those > tests (I spent multiple days trying to make it as clean and readable as > possible), and splitting it to only test the current infrastructure will need > s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Julien Rouhaud
On Fri, Oct 28, 2022 at 10:24:23AM +0900, Michael Paquier wrote: > On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > > I am still not completely sure what's the best way to do things here, > so let's do the following: let's keep the patch the way you think is > better for now (I may

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: >> >> Putting things afresh, there are two different things here (sorry I >> need to see that typed ;p): >> 1) How do we want to check reliably the loading of the HBA a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
Hi, On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: > > Putting things afresh, there are two different things here (sorry I > need to see that typed ;p): > 1) How do we want to check reliably the loading of the HBA and ident > files on errors? I guess you meant the failure to loa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > Have you already done a rebase while working on the patch or are you intending > to take care of it, or should I? Let's no both do the work :) Spoiler alert: I have not done a rebase yet ;) -- Michael signature.asc Description: P

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > I don't mind taking care of that, but before doing so I'd like to have some > feedback on whether you're ok with my approach (per my initial email about it > at [1]) or if you had some different > ideas on how to do it. Putting thin

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
On Wed, Oct 26, 2022 at 03:56:07PM +0900, Michael Paquier wrote: > > So, I have spent a good portion of today looking at what you have > here, applying 0001 and 0003 while fixing, rebasing and testing the > whole, discarding 0002 (we could do more for the line number and > source file in terms of t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? So, I have spent a good portion of today looking at what you have here, applying 0001 and 0003 while fixing, reba

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? Yeah, but there is a twist: HbaLine or IdentLine can be passed as NULL when entering in fill_hba_line() or fill_i

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 08:59:57PM +0900, Michael Paquier wrote: > > Hmm. I would be tempted to keep track of the file name and the line > number as well in IdentLine. One reason is that this can become > useful for debugging. A second is that this can reduce a bit the > arguments of fill_ident_

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Tue, Oct 25, 2022 at 03:08:59PM +0800, Julien Rouhaud wrote: > On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: >> Another advantage is that it minimizes the presence of the hardcoded >> HbaFileName and IdentFileName in hba.c, which is one thing we are >> trying to achieve here f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: > > Another advantage is that it minimizes the presence of the hardcoded > HbaFileName and IdentFileName in hba.c, which is one thing we are > trying to achieve here for the inclusion of more files. I found a bit > strange that Ident

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Michael Paquier
On Mon, Oct 24, 2022 at 04:03:03PM +0800, Julien Rouhaud wrote: > It would also require to bring HbaLine->sourcefile. I'm afraid it would be > weird to introduce such a refactoring in a separate commit just to pass a > constant down multiple level of indirection, as all the macro will remain > spe

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Julien Rouhaud
Hi, On Mon, Oct 24, 2022 at 04:13:51PM +0900, Michael Paquier wrote: > On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > > v12 attached, fixing multiple conflicts with recent activity. > > typedef struct TokenizedAuthLine > { > List *fields; /* List of lists of

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Michael Paquier
On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > v12 attached, fixing multiple conflicts with recent activity. typedef struct TokenizedAuthLine { List *fields; /* List of lists of AuthTokens */ + char *file_name; /* File name * Hmm. While putting

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-23 Thread Julien Rouhaud
On Mon, Oct 10, 2022 at 10:51:32AM +0800, Julien Rouhaud wrote: > On Sun, Sep 18, 2022 at 01:06:12AM +0800, Julien Rouhaud wrote: > > On Tue, Aug 16, 2022 at 02:10:30PM +0800, Julien Rouhaud wrote: > > > > > > On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > > > > > > > > > > > - 0

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-09 Thread Julien Rouhaud
Hi, On Sun, Sep 18, 2022 at 01:06:12AM +0800, Julien Rouhaud wrote: > On Tue, Aug 16, 2022 at 02:10:30PM +0800, Julien Rouhaud wrote: > > > > On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > > > > > > > > - 0001: the rule_number / mapping_number addition in the views in a > > > s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-09-17 Thread Julien Rouhaud
On Tue, Aug 16, 2022 at 02:10:30PM +0800, Julien Rouhaud wrote: > Hi, > > On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > > > > > - 0001: the rule_number / mapping_number addition in the views in a separate > > commit > > - 0002: the main file inclusion patch. Only a few mino

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-15 Thread Julien Rouhaud
Hi, On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > > - 0001: the rule_number / mapping_number addition in the views in a separate > commit > - 0002: the main file inclusion patch. Only a few minor bugfix since > previous version discovered thanks to the tests (a bit more a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-06 Thread Julien Rouhaud
Hi, On Fri, Aug 05, 2022 at 09:56:29AM +0900, Michael Paquier wrote: > On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > > As a quick update from my side, I intend to look and apply 0001~0003 > > (not double-checked yet) shortly. > > And a couple of days later, these look fine so

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-04 Thread Michael Paquier
On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > As a quick update from my side, I intend to look and apply 0001~0003 > (not double-checked yet) shortly. And a couple of days later, these look fine so done as of 47ab1ac and 718fe0a. 0002 and 0003 have been merged together. -- Mi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-02 Thread Michael Paquier
On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > I've been working on all of that and came up with the attached v8. > > - 0001: I modified things as discussed previously to report the real auth file > names rather than the hardcoded "pg_ident.conf" and "pg_hba.conf" in the > v

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-31 Thread Julien Rouhaud
On Tue, Jul 26, 2022 at 1:14 PM Michael Paquier wrote: > > On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > > It doesn't have much impact most of the time. The filename is reported if > > there's an IO error while reading the already opened correct file. The real > > problem is

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-30 Thread Julien Rouhaud
Hi, On Tue, Jul 19, 2022 at 03:13:12PM +0900, Michael Paquier wrote: > On Mon, Jul 18, 2022 at 03:11:51PM +0800, Julien Rouhaud wrote: > > > I'm not really sure what should be done here. The best compromise I can > > think > > of is to split the tests in 3 parts: > > > > 1) view reporting with v

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Michael Paquier
On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > It doesn't have much impact most of the time. The filename is reported if > there's an IO error while reading the already opened correct file. The real > problem is if the hba_file and ident_file are stored in different directory,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:22:32PM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > See the attached, for reference, but it would fail with EXEC_BACKEND > > on WIN32. > > Ditto. While working on the full regression test coverage for the fil

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-18 Thread Michael Paquier
On Mon, Jul 18, 2022 at 03:11:51PM +0800, Julien Rouhaud wrote: > So first, even if we can test 99% of the features with just testing the views > output, I think it's should use the TAP framework since the tests will have to > mess with the pg_ident/pg_hba files. It's way easier to modify the auth

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-18 Thread Julien Rouhaud
Hi, On Mon, Jul 11, 2022 at 10:16:44AM +0900, Michael Paquier wrote: > On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: > > My apologies for the late reply. > > > I don't have an extensive knowledge of all the user facing error messages, > > but > > after a quick grep I see multi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-10 Thread Michael Paquier
On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: My apologies for the late reply. > I don't have an extensive knowledge of all the user facing error messages, but > after a quick grep I see multiple usage of OID, PID, GIN and other defined > acronyms. I also see multiple occurrenc

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-07 Thread Julien Rouhaud
Hi, On Thu, Jun 02, 2022 at 10:08:15AM +0900, Michael Paquier wrote: > On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > > > After a bit more digging, I think that this comes from the fact that > > there's no > > "official" name for this file. Even the documentation just says "th

Re: Allow file inclusion in pg_hba and pg_ident files

2022-06-01 Thread Michael Paquier
On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > So you mean having an error message like that (having an "include myconf" > in the HBA file): > > LOG: could not open authentication file "myconf" as "/path/to/myconf": No > such file or directory > LOG: pg_hba.conf was not reloa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-26 Thread Julien Rouhaud
On Tue, May 24, 2022 at 10:44:05AM +0900, Michael Paquier wrote: > On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > > That being said, I'd gladly drop that enum and only handle a single error > > message, as the rest of the error context (including the owning file name > > and > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Michael Paquier
On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > That being said, I'd gladly drop that enum and only handle a single error > message, as the rest of the error context (including the owning file name and > line) should provide enough information to users. > > If so, should I use "i

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Julien Rouhaud
Hi, On Mon, May 23, 2022 at 03:53:06PM +0900, Michael Paquier wrote: > > + switch (kind) > + { > + case SecondaryAuthFile: > + msglog = "could not open secondary authentication file > \"@%s\" as \"%s\": %m"; > + msgview = "could not open secondary

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-22 Thread Michael Paquier
On Wed, May 18, 2022 at 03:12:45PM +0800, Julien Rouhaud wrote: > The cfbot reports that the patch doesn't apply anymore, rebased v7 attached. + switch (kind) + { + case SecondaryAuthFile: + msglog = "could not open secondary authentication file \"@%s\" as \"%s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-18 Thread Julien Rouhaud
Hi, On Tue, Mar 29, 2022 at 04:14:54PM +0800, Julien Rouhaud wrote: > > I'm attaching a rebase of the rest of the patchset. Note that I also added > support for an "include_dir" directive, as it's also something that would be > quite helpful (and very welcome for my $work use case). The cfbot r

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-29 Thread Julien Rouhaud
Hi, On Tue, Mar 29, 2022 at 10:21:36AM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:33:30PM +0800, Julien Rouhaud wrote: > > Ok, v5 attached without the TAP tests and updated sysviews tests. > > The update of the query related to pg_hba_file_rules in the regression > tests was indep

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:33:30PM +0800, Julien Rouhaud wrote: > Ok, v5 attached without the TAP tests and updated sysviews tests. The update of the query related to pg_hba_file_rules in the regression tests was independant, so I have split and applied that first, as of 091a971. Now, for the res

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
On Mon, Mar 28, 2022 at 05:02:14PM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 03:43:41PM +0800, Julien Rouhaud wrote: > > > > Ok. We could still keep the tests for the valid lines part though? > > With the SQLs modified as below, this part is less interesting. Ok. > > Do you mean so

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 03:43:41PM +0800, Julien Rouhaud wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: >> We could use a failure path for each psql command rather than a SKIP >> block, as you told me, if the psql fails and check that we get some >> error strings related

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > > Note that those tests fail on Windows (and I'm assuming on EXEC_BACKEND > > builds), as they're testing invalid files which by definition prevent any > > further connection attempt. I'm not sure what would be best to do he

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > See the attached, for reference, but it would fail with EXEC_BACKEND > on WIN32. Ditto. -- Michael From 69e02734fd0199ba02cc34bc468b04584bdf0efd Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 28 Mar 2022 16:20:40 +0900

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Sun, Mar 27, 2022 at 05:52:22PM +0800, Julien Rouhaud wrote: > I didn't like the various suggestions, as it would mean to scatter the tests > all over the place. The whole point of those views is indeed to check the > current content of a file without applying the configuration change (not on >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-27 Thread Julien Rouhaud
Hi, On Fri, Mar 25, 2022 at 08:18:31PM +0900, Michael Paquier wrote: > > Now looking at 0002. The changes in hba.c are straight-forward, > that's a nice read. Thanks! > if (!field) { \ > - ereport(LOG, \ > + ereport(elevel, \ > (errcod

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-25 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:08:38PM +0800, Julien Rouhaud wrote: > On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: >> And so, this first part has been applied as of d4781d8. I'll try to >> look at 0002 shortly. > > Thanks! Now looking at 0002. The changes in hba.c are straight-f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Julien Rouhaud
On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > >> Yeah, I thought about it but didn't rename it given your concerns about git > >> history. I'm

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Michael Paquier
On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: >> Yeah, I thought about it but didn't rename it given your concerns about git >> history. I'm fine either way. > > Oh, OK. The amount of diffs created by 0001 is st

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > Yeah, I thought about it but didn't rename it given your concerns about git > history. I'm fine either way. Oh, OK. The amount of diffs created by 0001 is still fine to grab even with the struct rename, so let's stick with it. --

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 11:03:46AM +0900, Michael Paquier wrote: > > Pushing forward with 0001 by the end of the CF is the part that has no > controversy IMO, and I have no objections to it. Now, after looking > at this part, I found a few things, as of: > - HbaToken, the set of elements in

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Tue, Mar 22, 2022 at 09:38:00PM +0800, Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: >> Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also >> increase CATALOG_VERSION_NO? Not sure if we generally do this in the >> patches or expe

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: > > The v3-0001 patch LGTM. > > Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also > increase CATALOG_VERSION_NO? Not sure if we generally do this in the > patches or expect the committer to make the chang

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > It passes `make installcheck` ... `make installcheck-world`, of course. Sorry for the confusion. -- Best regards, Aleksander Alekseev

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > The cfbot says that the patch doesn't apply anymore, so here's a v3 with the > changes mentioned below. I came across this thread while looking for the patches that need review. The v3-0001 patch LGTM. Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also increase

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-15 Thread Julien Rouhaud
Hi, The cfbot says that the patch doesn't apply anymore, so here's a v3 with the changes mentioned below. On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > > If you prefer to interleave static and non static function I can change it. Change the split to not reorder functions. >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Michael Paquier
On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: >> Hmm. The diffs of 0001 are really hard to read. Do you know why this >> is happening? Is that because some code has been moved around? > > Yes, I followed the fi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Julien Rouhaud
Hi, On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: > > Hmm. The diffs of 0001 are really hard to read. Do you know why this > is happening? Is that because some code has been moved around? Yes, I followed the file convention to put the static functions first and then the expo

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Mon, Feb 28, 2022 at 07:42:17PM +0800, Julien Rouhaud wrote: > Done in attached v2. I did the split in a separate commit, as the diff is > otherwise unreadable. While at it I also fixed a few minor issues (I missed a > MemoryContextDelete, and now avoid relying on inet_net_pton which apparentl

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Julien Rouhaud
Hi, On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > > Finally I also added 0003, which is a POC for a new pg_hba_matches() > > function, > > that can help DBA to understand why their configuration isn't working as > > they > > expect. This only to start the discussion on tha

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:50:33PM +0800, Julien Rouhaud wrote: > Of course. I was thinking using "auth" for something that's common to pg_hba > and pg_ident (like e.g. TokenizeAuthFile()), and otherwise keep the current > hba/ident prefix. Okay, thanks. > Unless someone object or suggest better

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
On Sat, Feb 26, 2022 at 03:36:19PM +0900, Michael Paquier wrote: > On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > > > Note that in order to do so we would need to expose quite a lot more about > > hba > > internals, like tokenize_file() and parse_hba_line(), along with structs

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > I'm fine with it. Assuming that you meant to move also the underlying > functions that goes with it (fill_hba_line and such), that would end up > removing about 15% of hba.c (after applying 0001, 0002 and 0003). Cool. Thanks for t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
Hi, On Sat, Feb 26, 2022 at 03:04:43PM +0900, Michael Paquier wrote: > On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as > >> pg_hba_fi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as >> pg_hba_file_rules view but for mappings. It's probably already useful, for >> instance

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-23 Thread Nathan Bossart
On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > To address that, I'd like to propose the possibility to include files in hba > and ident configuration files. This was already discussed in the past, and in > my understanding this is mostly wanted, while some people expressed conce