Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-28 Thread Michael Paquier
On Mon, Nov 28, 2022 at 04:12:40PM +0900, Michael Paquier wrote: > Attached is a rebased patch of the rest. With everything we have > dealt with in this CF, perhaps it would be better to mark this entry > as committed and switch to a new thread where the negative TAP tests > could be discussed? F

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 07:04:46PM +0800, Julien Rouhaud wrote: > And here's the rebased patch for the TAP tests. I will switch the CF entry to > Needs Review. I have been looking at that, and applied the part 1 of the test for the positive tests to cover the basic ground I'd like to see covered

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 09:49:31PM +0900, Ian Lawrence Barwick wrote: > Thanks for the quick update! FWIW, I do intend to tackle this last part ASAP, as the last piece to commit to get the full picture in the tree. -- Michael signature.asc Description: PGP signature

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Ian Lawrence Barwick
2022年11月27日(日) 20:04 Julien Rouhaud : > > On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > > > > I'm trying to reconcile open CommitFest entries with their actual > > > status; the entry for this: > > > > > > https://

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Julien Rouhaud
On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > I'm trying to reconcile open CommitFest entries with their actual > > status; the entry for this: > > > > https://commitfest.postgresql.org/40/3558/ > > > > shows as "Wa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Julien Rouhaud
Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > I'm trying to reconcile open CommitFest entries with their actual > status; the entry for this: > > https://commitfest.postgresql.org/40/3558/ > > shows as "Waiting on Author", but looks like it's all been committed; > is there anything > le

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Ian Lawrence Barwick
2022年11月25日(金) 11:25 Julien Rouhaud : > > On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached > > > v22 > > > that fixes it in all the plac

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > > that fixes it in all the places I found. > > Sounds fine. Added one comment, fixed

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Michael Paquier
On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > that fixes it in all the places I found. Sounds fine. Added one comment, fixed one comment, and applied. Thanks! -- Michael signature.asc Description:

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
Hi, On Thu, Nov 24, 2022 at 02:37:23PM +0800, Julien Rouhaud wrote: > On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > > The depth 0 is getting used quite a lot now, maybe we should have a > > > define for > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Julien Rouhaud
On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > The depth 0 is getting used quite a lot now, maybe we should have a define > > for > > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Michael Paquier
On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > The depth 0 is getting used quite a lot now, maybe we should have a define for > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like that? > And also add a define for the magical 10 for the max inclusion depth, f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Julien Rouhaud
Hi, Sorry for the very late answer, I had quite a lot of other things going on recently. And thanks for taking care of the patchset! On Wed, Nov 23, 2022 at 03:05:18PM +0900, Michael Paquier wrote: > On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > > + /* XXX: this should s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
,7 +549,7 @@ fill_ident_view(Tuplestorestate *tuple_store, TupleDesc tupdesc) } /* Free tokenizer memory */ - MemoryContextDelete(linecxt); + free_auth_file(file, 0); /* Free parse_ident_line memory */ MemoryContextSwitchTo(oldcxt); MemoryContextDelete(identcxt); -- 2.38.1 From 1173e0badf1572816b30a251106a6d

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
ne should be used only when we have no err_msg, meaning that these have been consumed in some TokenizedAuthLines already. -- Michael From c585c8d4dffb99f6abb4129cd9adf27849fa0ce0 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 14 Nov 2022 13:47:15 +0900 Subject: [PATCH v20 1/2] Allow file i

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-16 Thread Michael Paquier
On Wed, Nov 16, 2022 at 10:53:02AM +0800, Julien Rouhaud wrote: > While being the same inclusion infrastructure, it's likely that people will > have different usage. I'm assuming that for GUCs the main usage is to have > your automation tool put one of your template conf for instance > (small_vm.

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
Le mer. 16 nov. 2022 à 13:01, Ian Lawrence Barwick a écrit : > 2022年11月14日(月) 14:41 Michael Paquier : > > > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > > It's looks good to me. I agree that file name and line number should > be enough > > > to diagnose any unexpected er

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Ian Lawrence Barwick
2022年11月14日(月) 14:41 Michael Paquier : > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have looked at 0001 and 0002 again wi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
On Tue, Nov 15, 2022 at 08:46:55AM +0900, Michael Paquier wrote: > On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > > > > If you have an include_dir directive and multiple files have wrong > > permission > > (or maybe broken symlink or something like that), you will get multiple

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: >> If the caller passes NULL for *linectx as the initial line context, >> just create it as we do now. If *linectx is not NULL, just reuse it. >> That may be cleaner t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Julien Rouhaud
Hi, On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have loo

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Michael Paquier
On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > It's looks good to me. I agree that file name and line number should be > enough > to diagnose any unexpected error. Thanks for checking. I have looked at 0001 and 0002 again with a fresh mind, and applied both of them this morni

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-12 Thread Julien Rouhaud
On Thu, Nov 10, 2022 at 10:29:40AM +0900, Michael Paquier wrote: > > FWIW, I have been playing with the addition of a ErrorContextCallback > in tokenize_auth_file(), and this addition leads to a really nice > result. With this method, it is possible to know the full chain of > events leading to a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-09 Thread Michael Paquier
= filename; + callback_arg.linenum = line_number; + + tokenerrcontext.callback = tokenize_error_callback; + tokenerrcontext.arg = (void *) &callback_arg; + tokenerrcontext.previous = error_context_stack; + error_context_stack = &tokenerrcontext; linecxt = AllocSetContextCreate(CurrentMemoryConte

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Julien Rouhaud
Hi, On Wed, Nov 09, 2022 at 09:51:17AM +0900, Michael Paquier wrote: > On Tue, Nov 08, 2022 at 10:04:16AM +0900, Michael Paquier wrote: > > CF bot unhappy as I have messed up with rules.out. Rebased. I have > > removed the restriction on MAXPGPATH in AbsoluteConfigLocation() in > > 0001, while o

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Michael Paquier
t open usermap file \"%s\": %m", - IdentFileName))); + file = open_auth_file(IdentFileName, ERROR, 0, NULL); - linecxt = tokenize_auth_file(IdentFileName, file, &ident_lines, DEBUG3); + linecxt = tokenize_auth_file(IdentFileName, file, &ident_lines, DEBUG3, 0);

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-07 Thread Michael Paquier
lt in a message in a view * entry.) */ - file = AllocateFile(IdentFileName, "r"); - if (file == NULL) - ereport(ERROR, -(errcode_for_file_access(), - errmsg("could not open usermap file \"%s\": %m", - IdentFileName))); + file = open_auth_file(Id

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-06 Thread Michael Paquier
orestate *tuple_store, TupleDesc tupdesc) * (Most other error conditions should result in a message in a view * entry.) */ - file = AllocateFile(IdentFileName, "r"); - if (file == NULL) - ereport(ERROR, -(errcode_for_file_access(), - errmsg("could not open user

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-03 Thread Michael Paquier
On Wed, Nov 02, 2022 at 09:06:02PM +0800, Julien Rouhaud wrote: > Maybe one alternative approach would be to keep modifying the current test, > and > only do the split when committing the first part? The split itself should be > easy and mechanical: just remove everything unneeded (different file

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Julien Rouhaud
On Wed, Nov 02, 2022 at 04:46:48PM +0900, Michael Paquier wrote: > On Fri, Oct 28, 2022 at 11:49:54AM +0800, Julien Rouhaud wrote: > > To be honest I'd rather not to. It's excessively annoying to work on those > > tests (I spent multiple days trying to make it as clean and readable as > > possible

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Michael Paquier
8f Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Wed, 2 Nov 2022 16:42:53 +0900 Subject: [PATCH v15 1/2] Allow file inclusion in pg_hba and pg_ident files. pg_hba.conf file now has support for "include", "include_dir" and "include_if_exists" directives, whic

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Julien Rouhaud
On Fri, Oct 28, 2022 at 10:24:23AM +0900, Michael Paquier wrote: > On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > > I am still not completely sure what's the best way to do things here, > so let's do the following: let's keep the patch the way you think is > better for now (I may

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: >> >> Putting things afresh, there are two different things here (sorry I >> need to see that typed ;p): >> 1) How do we want to check reliably the loading of the HBA a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
Hi, On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: > > Putting things afresh, there are two different things here (sorry I > need to see that typed ;p): > 1) How do we want to check reliably the loading of the HBA and ident > files on errors? I guess you meant the failure to loa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > Have you already done a rebase while working on the patch or are you intending > to take care of it, or should I? Let's no both do the work :) Spoiler alert: I have not done a rebase yet ;) -- Michael signature.asc Description: P

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > I don't mind taking care of that, but before doing so I'd like to have some > feedback on whether you're ok with my approach (per my initial email about it > at [1]) or if you had some different > ideas on how to do it. Putting thin

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
On Wed, Oct 26, 2022 at 03:56:07PM +0900, Michael Paquier wrote: > > So, I have spent a good portion of today looking at what you have > here, applying 0001 and 0003 while fixing, rebasing and testing the > whole, discarding 0002 (we could do more for the line number and > source file in terms of t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? So, I have spent a good portion of today looking at what you have here, applying 0001 and 0003 while fixing, reba

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? Yeah, but there is a twist: HbaLine or IdentLine can be passed as NULL when entering in fill_hba_line() or fill_i

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 08:59:57PM +0900, Michael Paquier wrote: > > Hmm. I would be tempted to keep track of the file name and the line > number as well in IdentLine. One reason is that this can become > useful for debugging. A second is that this can reduce a bit the > arguments of fill_ident_

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Tue, Oct 25, 2022 at 03:08:59PM +0800, Julien Rouhaud wrote: > On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: >> Another advantage is that it minimizes the presence of the hardcoded >> HbaFileName and IdentFileName in hba.c, which is one thing we are >> trying to achieve here f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: > > Another advantage is that it minimizes the presence of the hardcoded > HbaFileName and IdentFileName in hba.c, which is one thing we are > trying to achieve here for the inclusion of more files. I found a bit > strange that Ident

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Michael Paquier
lse { /* no parsing result, so set relevant fields to nulls */ - memset(&nulls[1], true, (NUM_PG_IDENT_FILE_MAPPINGS_ATTS - 2) * sizeof(bool)); + memset(&nulls[2], true, (NUM_PG_IDENT_FILE_MAPPINGS_ATTS - 3) * sizeof(bool)); } /* error */ @@ -495,6 +516,7 @@ fill_ident_view(Tuplestore

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Julien Rouhaud
Hi, On Mon, Oct 24, 2022 at 04:13:51PM +0900, Michael Paquier wrote: > On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > > v12 attached, fixing multiple conflicts with recent activity. > > typedef struct TokenizedAuthLine > { > List *fields; /* List of lists of

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Michael Paquier
On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > v12 attached, fixing multiple conflicts with recent activity. typedef struct TokenizedAuthLine { List *fields; /* List of lists of AuthTokens */ + char *file_name; /* File name * Hmm. While putting

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-23 Thread Julien Rouhaud
_number, line_number, type, database, user_name, address, netmask, auth_method, options, error); +pg_ident_file_mappings| SELECT a.mapping_number, +a.line_number, a.map_name, a.sys_name, a.pg_username, a.error - FROM pg_ident_file_mappings() a(line_number, map_name, sys_n

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-09 Thread Julien Rouhaud
ype, database, user_name, address, netmask, auth_method, options, error); -pg_ident_file_mappings| SELECT a.line_number, + FROM pg_hba_file_rules() a(rule_number, line_number, type, database, user_name, address, netmask, auth_method, options, error); +pg_ident_file_mappings| SELECT a.mapping

Re: Allow file inclusion in pg_hba and pg_ident files

2022-09-17 Thread Julien Rouhaud
ngs| SELECT a.line_number, + FROM pg_hba_file_rules() a(rule_number, line_number, type, database, user_name, address, netmask, auth_method, options, error); +pg_ident_file_mappings| SELECT a.mapping_number, +a.line_number, a.map_name, a.sys_name, a.pg_username, a.error

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-15 Thread Julien Rouhaud
th_method, options, error); +pg_ident_file_mappings| SELECT a.mapping_number, +a.line_number, a.map_name, a.sys_name, a.pg_username, a.error - FROM pg_ident_file_mappings() a(line_number, map_name, sys_name, pg_username, error); + FROM pg_ident_file_mappings() a(mapping_nu

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-06 Thread Julien Rouhaud
Hi, On Fri, Aug 05, 2022 at 09:56:29AM +0900, Michael Paquier wrote: > On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > > As a quick update from my side, I intend to look and apply 0001~0003 > > (not double-checked yet) shortly. > > And a couple of days later, these look fine so

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-04 Thread Michael Paquier
On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > As a quick update from my side, I intend to look and apply 0001~0003 > (not double-checked yet) shortly. And a couple of days later, these look fine so done as of 47ab1ac and 718fe0a. 0002 and 0003 have been merged together. -- Mi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-02 Thread Michael Paquier
On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > I've been working on all of that and came up with the attached v8. > > - 0001: I modified things as discussed previously to report the real auth file > names rather than the hardcoded "pg_ident.conf" and "pg_hba.conf" in the > v

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-31 Thread Julien Rouhaud
On Tue, Jul 26, 2022 at 1:14 PM Michael Paquier wrote: > > On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > > It doesn't have much impact most of the time. The filename is reported if > > there's an IO error while reading the already opened correct file. The real > > problem is

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-30 Thread Julien Rouhaud
T a.line_number, +pg_hba_file_rules| SELECT a.rule_number, +a.line_number, a.type, a.database, a.user_name, @@ -1346,13 +1347,14 @@ pg_hba_file_rules| SELECT a.line_number, a.auth_method, a.options, a.error - FROM pg_hba_file_rules() a(line_number, type, datab

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Michael Paquier
On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > It doesn't have much impact most of the time. The filename is reported if > there's an IO error while reading the already opened correct file. The real > problem is if the hba_file and ident_file are stored in different directory,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:22:32PM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > See the attached, for reference, but it would fail with EXEC_BACKEND > > on WIN32. > > Ditto. While working on the full regression test coverage for the fil

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-18 Thread Michael Paquier
On Mon, Jul 18, 2022 at 03:11:51PM +0800, Julien Rouhaud wrote: > So first, even if we can test 99% of the features with just testing the views > output, I think it's should use the TAP framework since the tests will have to > mess with the pg_ident/pg_hba files. It's way easier to modify the auth

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-18 Thread Julien Rouhaud
Hi, On Mon, Jul 11, 2022 at 10:16:44AM +0900, Michael Paquier wrote: > On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: > > My apologies for the late reply. > > > I don't have an extensive knowledge of all the user facing error messages, > > but > > after a quick grep I see multi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-10 Thread Michael Paquier
On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: My apologies for the late reply. > I don't have an extensive knowledge of all the user facing error messages, but > after a quick grep I see multiple usage of OID, PID, GIN and other defined > acronyms. I also see multiple occurrenc

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-07 Thread Julien Rouhaud
Hi, On Thu, Jun 02, 2022 at 10:08:15AM +0900, Michael Paquier wrote: > On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > > > After a bit more digging, I think that this comes from the fact that > > there's no > > "official" name for this file. Even the documentation just says "th

Re: Allow file inclusion in pg_hba and pg_ident files

2022-06-01 Thread Michael Paquier
On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > So you mean having an error message like that (having an "include myconf" > in the HBA file): > > LOG: could not open authentication file "myconf" as "/path/to/myconf": No > such file or directory > LOG: pg_hba.conf was not reloa

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-26 Thread Julien Rouhaud
On Tue, May 24, 2022 at 10:44:05AM +0900, Michael Paquier wrote: > On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > > That being said, I'd gladly drop that enum and only handle a single error > > message, as the rest of the error context (including the owning file name > > and > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Michael Paquier
On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > That being said, I'd gladly drop that enum and only handle a single error > message, as the rest of the error context (including the owning file name and > line) should provide enough information to users. > > If so, should I use "i

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Julien Rouhaud
Hi, On Mon, May 23, 2022 at 03:53:06PM +0900, Michael Paquier wrote: > > + switch (kind) > + { > + case SecondaryAuthFile: > + msglog = "could not open secondary authentication file > \"@%s\" as \"%s\": %m"; > + msgview = "could not open secondary

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-22 Thread Michael Paquier
On Wed, May 18, 2022 at 03:12:45PM +0800, Julien Rouhaud wrote: > The cfbot reports that the patch doesn't apply anymore, rebased v7 attached. + switch (kind) + { + case SecondaryAuthFile: + msglog = "could not open secondary authentication file \"@%s\" as \"%s

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-18 Thread Julien Rouhaud
r my $work use case). The cfbot reports that the patch doesn't apply anymore, rebased v7 attached. >From 04943f3ff8dfb8efa5e538e0d9524fb041c3b39b Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 21 Feb 2022 15:45:26 +0800 Subject: [PATCH v7 1/2] Allow file inclusion in pg_hba a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-29 Thread Julien Rouhaud
se case). >From e5dc352cd2eb8cff24714930338adb0b2d20ff94 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 21 Feb 2022 15:45:26 +0800 Subject: [PATCH v6 1/2] Allow file inclusion in pg_hba and pg_ident files. Catversion is bumped. Author: Julien Rouhaud Reviewed-by: FIXME Discussion: htt

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:33:30PM +0800, Julien Rouhaud wrote: > Ok, v5 attached without the TAP tests and updated sysviews tests. The update of the query related to pg_hba_file_rules in the regression tests was independant, so I have split and applied that first, as of 091a971. Now, for the res

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
lect count(*) = 0 as ok from pg_cursors; select count(*) >= 0 as ok from pg_file_settings; -- There will surely be at least one rule -select count(*) > 0 as ok from pg_hba_file_rules; +select count(*) > 0 as ok, count(*) FILTER (WHERE error IS NOT NULL) = 0 AS no_err + from pg_hba_fil

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 03:43:41PM +0800, Julien Rouhaud wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: >> We could use a failure path for each psql command rather than a SKIP >> block, as you told me, if the psql fails and check that we get some >> error strings related

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > > Note that those tests fail on Windows (and I'm assuming on EXEC_BACKEND > > builds), as they're testing invalid files which by definition prevent any > > further connection attempt. I'm not sure what would be best to do he

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > See the attached, for reference, but it would fail with EXEC_BACKEND > on WIN32. Ditto. -- Michael From 69e02734fd0199ba02cc34bc468b04584bdf0efd Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 28 Mar 2022 16:20:40 +0900

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Sun, Mar 27, 2022 at 05:52:22PM +0800, Julien Rouhaud wrote: > I didn't like the various suggestions, as it would mean to scatter the tests > all over the place. The whole point of those views is indeed to check the > current content of a file without applying the configuration change (not on >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-27 Thread Julien Rouhaud
pg_indexes| SELECT n.nspname AS schemaname, c.relname AS tablename, i.relname AS indexname, diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out index 442eeb1e3f..31ba549883 100644 --- a/src/test/regress/expected/sysviews.out +++ b/src/test/

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-25 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:08:38PM +0800, Julien Rouhaud wrote: > On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: >> And so, this first part has been applied as of d4781d8. I'll try to >> look at 0002 shortly. > > Thanks! Now looking at 0002. The changes in hba.c are straight-f

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Julien Rouhaud
On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > >> Yeah, I thought about it but didn't rename it given your concerns about git > >> history. I'm

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Michael Paquier
On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: >> Yeah, I thought about it but didn't rename it given your concerns about git >> history. I'm fine either way. > > Oh, OK. The amount of diffs created by 0001 is st

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > Yeah, I thought about it but didn't rename it given your concerns about git > history. I'm fine either way. Oh, OK. The amount of diffs created by 0001 is still fine to grab even with the struct rename, so let's stick with it. --

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 11:03:46AM +0900, Michael Paquier wrote: > > Pushing forward with 0001 by the end of the CF is the part that has no > controversy IMO, and I have no objections to it. Now, after looking > at this part, I found a few things, as of: > - HbaToken, the set of elements in

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Tue, Mar 22, 2022 at 09:38:00PM +0800, Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: >> Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also >> increase CATALOG_VERSION_NO? Not sure if we generally do this in the >> patches or expe

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: > > The v3-0001 patch LGTM. > > Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also > increase CATALOG_VERSION_NO? Not sure if we generally do this in the > patches or expect the committer to make the chang

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > It passes `make installcheck` ... `make installcheck-world`, of course. Sorry for the confusion. -- Best regards, Aleksander Alekseev

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > The cfbot says that the patch doesn't apply anymore, so here's a v3 with the > changes mentioned below. I came across this thread while looking for the patches that need review. The v3-0001 patch LGTM. Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also increase

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-15 Thread Julien Rouhaud
Hi, The cfbot says that the patch doesn't apply anymore, so here's a v3 with the changes mentioned below. On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > > If you prefer to interleave static and non static function I can change it. Change the split to not reorder functions. >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Michael Paquier
On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: >> Hmm. The diffs of 0001 are really hard to read. Do you know why this >> is happening? Is that because some code has been moved around? > > Yes, I followed the fi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Julien Rouhaud
Hi, On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: > > Hmm. The diffs of 0001 are really hard to read. Do you know why this > is happening? Is that because some code has been moved around? Yes, I followed the file convention to put the static functions first and then the expo

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Mon, Feb 28, 2022 at 07:42:17PM +0800, Julien Rouhaud wrote: > Done in attached v2. I did the split in a separate commit, as the diff is > otherwise unreadable. While at it I also fixed a few minor issues (I missed a > MemoryContextDelete, and now avoid relying on inet_net_pton which apparentl

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Julien Rouhaud
Hi, On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > > Finally I also added 0003, which is a POC for a new pg_hba_matches() > > function, > > that can help DBA to understand why their configuration isn't working as > > they > > expect. This only to start the discussion on tha

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:50:33PM +0800, Julien Rouhaud wrote: > Of course. I was thinking using "auth" for something that's common to pg_hba > and pg_ident (like e.g. TokenizeAuthFile()), and otherwise keep the current > hba/ident prefix. Okay, thanks. > Unless someone object or suggest better

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
On Sat, Feb 26, 2022 at 03:36:19PM +0900, Michael Paquier wrote: > On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > > > Note that in order to do so we would need to expose quite a lot more about > > hba > > internals, like tokenize_file() and parse_hba_line(), along with structs

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > I'm fine with it. Assuming that you meant to move also the underlying > functions that goes with it (fill_hba_line and such), that would end up > removing about 15% of hba.c (after applying 0001, 0002 and 0003). Cool. Thanks for t

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
Hi, On Sat, Feb 26, 2022 at 03:04:43PM +0900, Michael Paquier wrote: > On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as > >> pg_hba_fi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as >> pg_hba_file_rules view but for mappings. It's probably already useful, for >> instance

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-23 Thread Nathan Bossart
On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > To address that, I'd like to propose the possibility to include files in hba > and ident configuration files. This was already discussed in the past, and in > my understanding this is mostly wanted, while some people expressed conce

Allow file inclusion in pg_hba and pg_ident files

2022-02-22 Thread Julien Rouhaud
ews.out @@ -55,6 +55,13 @@ select count(*) > 0 as ok from pg_hba_file_rules; t (1 row) +-- We expect no user mapping in this test +select count(*) = 0 as ok from pg_ident_file_mappings; + ok + + t +(1 row) + -- There will surely be at least one active lock select count(*) > 0 a