Re: New Object Access Type hooks

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 13:47:27 -0400, Andrew Dunstan wrote: > OK, I have pushed that. Seems like it might actually be good to test that object access hooks work well in a parallel worker. How about going the other way and explicitly setting force_parallel_mode = disabled for parts of the test and to

Re: SQL/JSON: functions

2022-03-22 Thread Tom Lane
Andrew Dunstan writes: > I have committed the first of these. That will break the cfbot for this > and the json_table patches. The remainder should be committed in the > following days. That patch is 0-for-three on my buildfarm animals. regards, tom lane

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Thu, 26 Aug 2021 at 14:54, Andy Fan wrote: > > On Thu, Aug 19, 2021 at 2:35 PM David Rowley wrote: > > > > On Thu, 19 Aug 2021 at 00:20, Andy Fan wrote: > > > In the current master, the result is: > > > > > > empno | salary | c | dr > > > ---++---+ > > > 8 | 6000 | 4 |

Re: SQL/JSON: functions

2022-03-22 Thread Tom Lane
I wrote: > That patch is 0-for-three on my buildfarm animals. ... the reason being that they use -Werror, and this patch spews a bunch of warnings. This is not acceptable, especially not in the middle of a CF when other people are trying to get work done. Please revert. r

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Wed, 16 Mar 2022 at 10:24, Greg Stark wrote: > > This looks like an awesome addition. Thanks > I have one technical questions... > > Is it possible to actually transform the row_number case into a LIMIT > clause or make the planner support for this case equivalent to it (in > which case we ca

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
> On Mar 22, 2022, at 3:20 PM, Andres Freund wrote: > > Seems like it might actually be good to test that object access hooks work > well in a parallel worker. How about going the other way and explicitly > setting > force_parallel_mode = disabled for parts of the test and to enabled for > ot

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Thu, 17 Mar 2022 at 17:04, Corey Huinker wrote: > It seems like this effort would aid in implementing what some other databases > implement via the QUALIFY clause, which is to window functions what HAVING is > to aggregate functions. > example: > https://cloud.google.com/bigquery/docs/refere

Re: New Object Access Type hooks

2022-03-22 Thread Tom Lane
Mark Dilger writes: >> On Mar 22, 2022, at 3:20 PM, Andres Freund wrote: >> Seems like it might actually be good to test that object access hooks work >> well in a parallel worker. How about going the other way and explicitly >> setting >> force_parallel_mode = disabled for parts of the test and

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-03-22 Thread Jacob Champion
On Tue, 2022-03-22 at 14:48 -0700, samay sharma wrote: > Thank you for porting this on top of the pluggable auth methods API. > I've addressed the feedback around other backend changes in my latest > patch, but the client side changes still remain. I had a few > questions to understand them better.

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 16:55:49 -0400, Tom Lane wrote: > 4. I find each of the above ideas to be far more attractive than > optimizing SELECT-DISTINCT-that-matches-an-index, so I don't really > understand why the current patchsets seem to be driven largely > by that single use-case. It's something cau

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-03-22 Thread Nathan Bossart
On Thu, Mar 17, 2022 at 04:45:28PM -0700, Nathan Bossart wrote: > I think this one requires some more work, and it needn't be a priority for > v15, so I've adjusted the commitfest entry to v16 and moved it to the next > commitfest. Here is a new patch. The main differences from v3 are in heapam_v

Re: New Object Access Type hooks

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 18:41:45 -0400, Tom Lane wrote: > Mark Dilger writes: > >> On Mar 22, 2022, at 3:20 PM, Andres Freund wrote: > >> Seems like it might actually be good to test that object access hooks work > >> well in a parallel worker. How about going the other way and explicitly > >> settin

Re: New Object Access Type hooks

2022-03-22 Thread Justin Pryzby
If I'm not wrong, this is still causing issues at least on cfbot/windows, even since f0206d99. https://cirrus-ci.com/task/5266352712712192 https://cirrus-ci.com/task/5061218867085312 https://cirrus-ci.com/task/5663822005403648 https://cirrus-ci.com/task/5744257246953472 https://cirrus-ci.com/task

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 16:06:40 -0700, Nathan Bossart wrote: > On Thu, Mar 17, 2022 at 04:45:28PM -0700, Nathan Bossart wrote: > > I think this one requires some more work, and it needn't be a priority for > > v15, so I've adjusted the commitfest entry to v16 and moved it to the next > > commitfest. >

Re: SQL/JSON: functions

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 18:31:39 -0400, Tom Lane wrote: > I wrote: > > That patch is 0-for-three on my buildfarm animals. > > ... the reason being that they use -Werror, and this patch spews > a bunch of warnings. This is not acceptable, especially not in > the middle of a CF when other people are try

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Tom Lane
Andres Freund writes: > On 2022-03-22 16:55:49 -0400, Tom Lane wrote: >> BTW, I've had a bee in my bonnet for a long time about whether some of >> nbtree's scan setup work could be done once during planning, rather than >> over again during each indexscan start. > It does show up in simple-index-

Re: New Object Access Type hooks

2022-03-22 Thread Tom Lane
Justin Pryzby writes: > If I'm not wrong, this is still causing issues at least on cfbot/windows, even > since f0206d99. That's probably a variant of the encoding dependency I described upthread. regards, tom lane

Re: Probable CF bot degradation

2022-03-22 Thread Thomas Munro
On Mon, Mar 21, 2022 at 12:46 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro wrote: > > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > > wrote: > > > Additionally, are there plans to validate commits of the main branch > > > before using them as a base for CF ent

Re: SQL/JSON: functions

2022-03-22 Thread Tom Lane
Andres Freund writes: > There's also > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jabiru&dt=2022-03-22%2022%3A25%3A26 > that started failing with > ../../preproc/ecpg --regression -I./../../include -I. -o test1.c test1.pgc > test1.pgc:12: ERROR: syntax error at or near "int" > with th

Re: Window Function "Run Conditions"

2022-03-22 Thread Zhihong Yu
On Tue, Mar 22, 2022 at 3:24 PM David Rowley wrote: > On Thu, 26 Aug 2021 at 14:54, Andy Fan wrote: > > > > On Thu, Aug 19, 2021 at 2:35 PM David Rowley > wrote: > > > > > > On Thu, 19 Aug 2021 at 00:20, Andy Fan > wrote: > > > > In the current master, the result is: > > > > > > > > empno | s

Re: New Object Access Type hooks

2022-03-22 Thread Andrew Dunstan
On 3/22/22 18:18, Tom Lane wrote: > Andrew Dunstan writes: >> Fixed > Now that we got past the hard failures, we can see that the test > falls over with (some?) non-default encodings, as for instance > here: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2022-03-22%2020%3A2

Re: SQL/JSON: functions

2022-03-22 Thread Andrew Dunstan
On 3/22/22 18:31, Tom Lane wrote: > I wrote: >> That patch is 0-for-three on my buildfarm animals. > ... the reason being that they use -Werror, and this patch spews > a bunch of warnings. This is not acceptable, especially not in > the middle of a CF when other people are trying to get work don

Re: New Object Access Type hooks

2022-03-22 Thread Tom Lane
Andrew Dunstan writes: > On 3/22/22 18:18, Tom Lane wrote: >> Now, if our attitude to the OAT hooks is that we are going to >> sprinkle some at random and whether they are useful is someone >> else's problem, then maybe these are not interesting concerns. > So this was a pre-existing problem that

Re: New Object Access Type hooks

2022-03-22 Thread Andrew Dunstan
On 3/22/22 20:07, Tom Lane wrote: > Andrew Dunstan writes: >> On 3/22/22 18:18, Tom Lane wrote: >>> Now, if our attitude to the OAT hooks is that we are going to >>> sprinkle some at random and whether they are useful is someone >>> else's problem, then maybe these are not interesting concerns.

cpluspluscheck vs ICU

2022-03-22 Thread Andres Freund
Hi, I was about to propose adding headerscheck / cpluspluscheck to the CI file so that cfbot can catch future issues. Unfortunately running cpluspluscheck with ICU enabled is, um, not fun: There's 30k lines of error output. /home/andres/src/postgresql/src/tools/pginclude/cpluspluscheck /home/and

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-03-22 Thread Japin Li
On Wed, 23 Mar 2022 at 01:22, Maxim Orlov wrote: > Hi! > > Here is v24. Changes are: > - correct commit messages for 0001 and 0002 > - use uint64 for SLRU page numbering instead of int64 in v23 > - fix code formatting and indent > - and minor fixes in slru.c > > Reviews are very welcome! > Than

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-03-22 Thread Nathan Bossart
On Tue, Mar 22, 2022 at 04:13:47PM -0700, Andres Freund wrote: > Just skimming this thread quickly, I really have no idea what this is trying > to achieve and the commit message doesn't help either... I didn't read the > referenced thread, but I shouldn't have to, to get a basic idea. Ah, my bad.

Re: shared-memory based stats collector - v66

2022-03-22 Thread Melanie Plageman
On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > I've attached a substantially improved version of the shared memory stats > patch. ... > - lot of the pg_stat_ views like bgwriter, pg_stat_database have zero > coverage today Attached are some tests including tests that reset of stats wor

Re: Probable CF bot degradation

2022-03-22 Thread Justin Pryzby
On Wed, Mar 23, 2022 at 12:44:09PM +1300, Thomas Munro wrote: > On Mon, Mar 21, 2022 at 12:46 PM Thomas Munro wrote: > > On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro > > wrote: > > > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > > > wrote: > > > > Additionally, are there plans to va

Re: SQL/JSON: functions

2022-03-22 Thread Tom Lane
I wrote: > Andres Freund writes: >> There's also >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jabiru&dt=2022-03-22%2022%3A25%3A26 >> that started failing with >> ../../preproc/ecpg --regression -I./../../include -I. -o test1.c test1.pgc >> test1.pgc:12: ERROR: syntax error at or near

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Tue, Mar 22, 2022 at 09:38:00PM +0800, Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: >> Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also >> increase CATALOG_VERSION_NO? Not sure if we generally do this in the >> patches or expe

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 11:03:46AM +0900, Michael Paquier wrote: > > Pushing forward with 0001 by the end of the CF is the part that has no > controversy IMO, and I have no objections to it. Now, after looking > at this part, I found a few things, as of: > - HbaToken, the set of elements in

Re: Column Filtering in Logical Replication

2022-03-22 Thread Amit Kapila
On Wed, Mar 23, 2022 at 12:54 AM Alvaro Herrera wrote: > > On 2022-Mar-19, Tomas Vondra wrote: > > > @@ -174,7 +182,13 @@ ALTER PUBLICATION noinsert SET (publish = 'update, > > delete'); > > > > Add some tables to the publication: > > > > -ALTER PUBLICATION mypublication ADD TABLE users

Re: cpluspluscheck vs ICU

2022-03-22 Thread Andres Freund
Hi, On 2022-03-22 17:20:24 -0700, Andres Freund wrote: > I was about to propose adding headerscheck / cpluspluscheck to the CI file so > that cfbot can catch future issues. The attached patch does so, with ICU disabled to avoid the problems discussed in the thread. Example run: https://cirrus-ci.

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-22 Thread Kyotaro Horiguchi
At Tue, 22 Mar 2022 11:00:06 -0700, Andres Freund wrote in > Hi, > > On 2022-03-22 21:57:51 +0530, Bharath Rupireddy wrote: > > > This is probably close to an order of magnitude slower than pg_waldump > > > --stats. Which imo renders this largely useless. > > > > Yeah that's true. Do you sugges

Re: cpluspluscheck vs ICU

2022-03-22 Thread Thomas Munro
On Wed, Mar 23, 2022 at 3:23 PM Andres Freund wrote: > On 2022-03-22 17:20:24 -0700, Andres Freund wrote: > > I was about to propose adding headerscheck / cpluspluscheck to the CI file > > so > > that cfbot can catch future issues. > > The attached patch does so, with ICU disabled to avoid the pr

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-22 Thread Kyotaro Horiguchi
At Wed, 23 Mar 2022 11:51:25 +0900 (JST), Kyotaro Horiguchi wrote in > The two places emit different outputs but the only difference is the > delimiter between two blockrefs. (By the way, the current code forgets > to insert a delimiter there). So even if the function took "bool > is_waldump",

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Wed, 23 Mar 2022 at 12:50, Zhihong Yu wrote: > The following code seems to be common between if / else blocks (w.r.t. > wfunc_left) of find_window_run_conditions(). > It would be nice if this code can be shared. I remember thinking about that and thinking that I didn't want to overcomplicat

Re: Window Function "Run Conditions"

2022-03-22 Thread David Rowley
On Wed, 23 Mar 2022 at 11:24, David Rowley wrote: > I think it's safer to just disable the optimisation when there are > multiple window clauses. Multiple matching clauses are merged > already, so it's perfectly valid to have multiple window functions, > it's just they must share the same window

Re: [PATCH] Add native windows on arm64 support

2022-03-22 Thread Thomas Munro
On Tue, Mar 22, 2022 at 11:30 PM Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 09:37:46AM +, Niyas Sait wrote: > > Yes, we could look into providing a build machine. Do you have any > > reference to what the CI system looks like now for PostgresSQL and how to > > add new workers etc.? > > It

Re: freeing bms explicitly

2022-03-22 Thread Amit Kapila
On Tue, Mar 22, 2022 at 3:39 AM Zhihong Yu wrote: > > On Mon, Mar 21, 2022 at 3:05 PM Tom Lane wrote: >> >> Zhihong Yu writes: >> >> I was looking at calls to bms_free() in PG code. >> >> e.g. src/backend/commands/publicationcmds.c line 362 >> >> bms_free(bms); >> >> The above is just an ex

Re: Temporary tables versus wraparound... again

2022-03-22 Thread Greg Stark
Here's a rebased patch. I split the test into a separate patch that I would lean to dropping. But at least it applies now. I did look into pg_stat_get_backend_pid() and I guess it would work but going through the stats mechanism does seem like going the long way around since we're already looking

Re: freeing bms explicitly

2022-03-22 Thread Zhihong Yu
On Tue, Mar 22, 2022 at 8:45 PM Amit Kapila wrote: > On Tue, Mar 22, 2022 at 3:39 AM Zhihong Yu wrote: > > > > On Mon, Mar 21, 2022 at 3:05 PM Tom Lane wrote: > >> > >> Zhihong Yu writes: > >> >> I was looking at calls to bms_free() in PG code. > >> >> e.g. src/backend/commands/publicationcmds

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > Yeah, I thought about it but didn't rename it given your concerns about git > history. I'm fine either way. Oh, OK. The amount of diffs created by 0001 is still fine to grab even with the struct rename, so let's stick with it. --

Re: Adding CI to our tree

2022-03-22 Thread Justin Pryzby
On Fri, Mar 18, 2022 at 03:45:03PM -0700, Andres Freund wrote: > Pushed 0001, 0002. Only change I made was to add Thanks - is there any reason not to do the MSVC compiler warnings one, too ? I see that it'll warn about issues with at least 3 patches (including one of yours). -- Justin

Re: Optimize external TOAST storage

2022-03-22 Thread Michael Paquier
On Tue, Mar 22, 2022 at 02:42:53PM -0700, Nathan Bossart wrote: > On Tue, Mar 22, 2022 at 04:34:05PM -0400, Robert Haas wrote: >> Then, too, I'm not very confident about the usefulness of EXTENDED, >> EXTERNAL, and MAIN. I think it's useful to be able to categorically >> disable compression (as EXT

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-22 Thread Bharath Rupireddy
On Tue, Mar 22, 2022 at 8:12 PM Andres Freund wrote: > > Do you mean like this? > > ereport(LOG, > > /* translator: the placeholders show checkpoint options */ > > (errmsg("%s starting:%s%s%s%s%s%s%s%s", > > restartpoint ? _("restartpoint") : _("checkpoint")

Re: make MaxBackends available in _PG_init

2022-03-22 Thread Julien Rouhaud
Hi, Sorry for showing up this late, but I'm a bit confused with the new situation. Unless I'm missing something, the new situation is that the system is supposed to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally agree with, but without doing anything for extensions that

Re: Window Function "Run Conditions"

2022-03-22 Thread David G. Johnston
On Tue, Mar 22, 2022 at 3:39 PM David Rowley wrote: > On Thu, 17 Mar 2022 at 17:04, Corey Huinker > wrote: > > It seems like this effort would aid in implementing what some other > databases implement via the QUALIFY clause, which is to window functions > what HAVING is to aggregate functions. >

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-22 Thread Kyotaro Horiguchi
At Tue, 22 Mar 2022 20:42:37 +, Jacob Champion wrote in > Thanks, looks like I had some old header dependencies left over from > several versions ago. Fixed in v9. Thanks! Looks perfect. > v9 contains the bare minimum but I don't think it's quite enough. How > much of the behavior (and ed

Re: Add pg_freespacemap extension sql test

2022-03-22 Thread Michael Paquier
On Mon, Mar 21, 2022 at 09:12:37PM +0900, Dong Wook Lee wrote: > 2022년 3월 20일 (일) 03:13, Fabrízio de Royes Mello 님이 > 작성: >> On Sat, Mar 19, 2022 at 1:18 PM Dong Wook Lee wrote: Well, my guess is that you basically just care about being able to detect if there is free space in the map or

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-22 Thread Michael Paquier
On Mon, Mar 21, 2022 at 06:28:28PM -0700, Andres Freund wrote: > Doesn't apply cleanly anymore: http://cfbot.cputube.org/patch_37_2377.log > > Marked as waiting-on-author. FWIW, per my review the bit of the patch set that I found the most relevant is the addition of a note in the docs of pg_stat_

Re: Estimating HugePages Requirements?

2022-03-22 Thread Michael Paquier
On Tue, Mar 15, 2022 at 03:44:39PM -0700, Nathan Bossart wrote: > A simple approach could be to just set log_min_messages to PANIC before > exiting. I've attached a patch for this. With this patch, we'll still see > a FATAL if we try to use 'postgres -C' for a runtime-computed GUC on a > running

<    1   2