Re: Parallelize correlated subqueries that execute within each worker

2022-03-21 Thread Andres Freund
Hi, On 2022-01-22 20:25:19 -0500, James Coleman wrote: > On the other hand this is a dramatically simpler patch series. > Assuming the approach is sound, it should much easier to maintain than > the previous version. > > The final patch in the series is a set of additional checks I could > imagin

RE: logical replication empty transactions

2022-03-21 Thread houzj.f...@fujitsu.com
On Monday, March 21, 2022 6:01 PM Amit Kapila wrote: > > On Sat, Mar 19, 2022 at 9:10 AM Ajin Cherian wrote: > > > > On Thu, Mar 17, 2022 at 10:43 PM Amit Kapila > wrote: > > > > > 3. Can we add a simple test for it in one of the existing test > > > files(say in 001_rep_changes.pl)? > > > > add

Re: Partial aggregates pushdown

2022-03-21 Thread Andres Freund
On 2022-01-17 15:27:53 +0300, Alexander Pyhalov wrote: > Alexander Pyhalov писал 2022-01-17 15:26: > > Updated patch. > > Sorry, missed attachment. Needs another update: http://cfbot.cputube.org/patch_37_3369.log Marked as waiting on author. - Andres

Re: Pluggable toaster

2022-03-21 Thread Andres Freund
Hi, On 2022-03-22 02:31:21 +0300, Nikita Malakhov wrote: > Hi Hackers, > Because of 3 months have passed since Pluggable Toaster presentation and a > lot of > commits were pushed into v15 master - we would like to re-introduce > this patch > rebased onto actual master. Last commit being used - > c

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-03-21 Thread Andres Freund
On 2022-01-24 14:44:10 -0500, Robert Haas wrote: > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > > no more listed as unused OIDs. > > Thanks

Re: [PATCH] Proof of concept for GUC improvements

2022-03-21 Thread Tom Lane
Andres Freund writes: > My impression is that there's not a lot of enthusiasm for the concept? If > that's true we maybe ought to mark the CF entry as rejected? Yeah, I'm kind of leaning that way too. I don't see how we can incorporate the symbolic values into any existing display paths without

Re: Reducing power consumption on idle servers

2022-03-21 Thread Andres Freund
On 2022-02-21 21:04:19 +, Simon Riggs wrote: > On Mon, 21 Feb 2022 at 16:49, Chapman Flack wrote: > > > Shouldn't the comment be "with work_done=false" ? > > Good catch, thanks. > > I've also added docs to say that "promote_trigger_file" is now > deprecated. There were no tests for that fun

Re: Removing unneeded self joins

2022-03-21 Thread Andres Freund
Hi, On 2022-03-04 15:47:47 +0500, Andrey Lepikhov wrote: > Also, in new version of the patch I fixed one stupid bug: checking a > self-join candidate expression operator - we can remove only expressions > like F(arg1) = G(arg2). This CF entry currently fails tests: https://cirrus-ci.com/task/463

Re: WIN32 pg_import_system_collations

2022-03-21 Thread Andres Freund
Hi, On 2022-01-25 15:49:01 +0100, Juan José Santamaría Flecha wrote: > So, I'm doing that in the attached version, just changing the object name. Currently fails to apply, please rebase: http://cfbot.cputube.org/patch_37_3450.log Marked as waiting-on-author. - Andres

Re: logical replication restrictions

2022-03-21 Thread Andres Freund
On 2022-03-20 21:40:40 -0300, Euler Taveira wrote: > On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: > > Long time, no patch. Here it is. I will provide documentation in the next > > version. I would appreciate some feedback. > This patch is broken since commit 705e20f8550c0e8e47c0b6b20b5f5f

Re: Temporary tables versus wraparound... again

2022-03-21 Thread Andres Freund
Hi, On 2021-10-12 18:04:35 -0400, Greg Stark wrote: > Here's an updated patch. Unfortunately it doesn't apply anymore these days: http://cfbot.cputube.org/patch_37_3358.log Marked as waiting-on-author. Greetings, Andres Freund

Re: Window Function "Run Conditions"

2022-03-21 Thread Andres Freund
Hi, On 2021-08-19 18:35:27 +1200, David Rowley wrote: > Anyway, I'll take a few more days to think about it before posting a fix. The patch in the CF entry doesn't apply: http://cfbot.cputube.org/patch_37_3234.log The quoted message was ~6 months ago. I think this CF entry should be marked as r

Re: WaitLatchOrSocket seems to not count to 4 right...

2022-03-21 Thread Andres Freund
Hi, On 2022-02-11 10:47:45 +1300, Thomas Munro wrote: > I'd originally sketched this out for another project, but I don't > think I need it for that anymore. After this exchange I couldn't > resist fleshing it out for a commitfest, just on useability grounds. > Thoughts? This currently doesn't a

Re: logical replication restrictions

2022-03-21 Thread Euler Taveira
On Mon, Mar 21, 2022, at 10:04 PM, Andres Freund wrote: > On 2022-03-20 21:40:40 -0300, Euler Taveira wrote: > > On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: > > > Long time, no patch. Here it is. I will provide documentation in the next > > > version. I would appreciate some feedback. >

Re: LogwrtResult contended spinlock

2022-03-21 Thread Andres Freund
Hi, On 2021-11-22 18:56:43 -0300, Alvaro Herrera wrote: > There was an earlier comment by Andres that asyncXactLSN should also be > atomic, to avoid an ugly spinlock interaction with the new atomic-based > logwrtResult. The 0002 here is an attempt at doing that; I found that > it also needed to c

Re: pg14 psql broke \d datname.nspname.relname

2022-03-21 Thread Andres Freund
On 2022-01-26 09:04:15 -0800, Mark Dilger wrote: > Also, rebased as necessary: Needs another one: http://cfbot.cputube.org/patch_37_3367.log Marked as waiting-on-author. Greetings, Andres Freund

Re: Separate the result of \watch for each query execution (psql)

2022-03-21 Thread Andres Freund
Hi, On 2022-02-25 13:23:31 +0900, Noboru Saito wrote: > I have created a patch that allows you to turn it on and off in \pset. The patch unfortunately causes tests to fail: https://cirrus-ci.com/task/5932406812180480 diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/psql.out /tmp/cirrus-ci

Re: [PATCH] pg_stat_toast v8

2022-03-21 Thread Andres Freund
Hi, On 2022-03-08 19:32:03 +0100, Gunnar "Nick" Bluth wrote: > v8 (applies cleanly to today's HEAD/master) attached. This doesn't apply anymore, likely due to my recent pgstat changes - which you'd need to adapt to... http://cfbot.cputube.org/patch_37_3457.log Marked as waiting on author. Gree

Re: Allow batched insert during cross-partition updates

2022-03-21 Thread Amit Langote
On Tue, Mar 22, 2022 at 9:30 AM Andres Freund wrote: > On 2021-08-24 12:03:59 +0900, Amit Langote wrote: > > Tomas committed the bug-fix, so attaching a rebased version of the > > patch for $subject. > > This patch is in the current CF, but doesn't apply: > http://cfbot.cputube.org/patch_37_2992.

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2022-03-21 Thread Andres Freund
Hi, On 2022-02-23 00:51:24 +0100, Tomas Vondra wrote: > And here's the slightly simplified patch, without the part adding the > unnecessary GetServerVersion() function. Doesn't apply anymore: http://cfbot.cputube.org/patch_37_3535.log Marked as waiting-on-author. Greetings, Andres Freund

Re: a misbehavior of partition row movement (?)

2022-03-21 Thread Amit Langote
Hi Alvaro, On Mon, Mar 21, 2022 at 2:58 AM Alvaro Herrera wrote: > On 2022-Mar-20, Amit Langote wrote: > > On Sun, Mar 20, 2022 at 5:13 AM Alvaro Herrera > > wrote: > > > On 2022-Mar-18, Zhihong Yu wrote: > > > > > + if (!partRel->rd_rel->relispartition) > > > > + elog(ERROR, "cannot fi

Re: [PATCH] sort leaf pages by ctid for gist indexes built using sorted method

2022-03-21 Thread Andres Freund
Hi, On 2021-12-16 14:49:25 +0500, Andrey Borodin wrote: > > > With the current implementation, for GiST indexes created by doing multiple > > inserts, index tuples match heap tuples order, but it doesn't work that way > > for sorted method where index tuples on all levels are ordered using > >

Re: speed up a logical replica setup

2022-03-21 Thread Andres Freund
Hi, On 2022-02-21 09:09:12 -0300, Euler Taveira wrote: > A new tool called pg_subscriber does this conversion and is tightly integrated > with Postgres. Given that this has been submitted just before the last CF and is a patch of nontrivial size, has't made significant progress ISTM it should be

Re: Temporary tables versus wraparound... again

2022-03-21 Thread Greg Stark
No problem, I can update the patch and check on the fuzz. But the actual conflict is just in the test and I'm not sure it's really worth having a test at all. It's testing a pretty low level detail. So I'm leaning toward fixing the conflict by just ripping the test out. Nathan also pointed out th

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-21 Thread Andres Freund
Hi, On 2022-03-09 10:50:45 -0600, Justin Pryzby wrote: > Rebased over 9e9858389 (Michael may want to look at the tuplestore part?). Doesn't apply cleanly anymore: http://cfbot.cputube.org/patch_37_2377.log Marked as waiting-on-author. Greetings, Andres Freund

Re: pg14 psql broke \d datname.nspname.relname

2022-03-21 Thread Mark Dilger
> On Mar 21, 2022, at 6:12 PM, Andres Freund wrote: > > Needs another one: http://cfbot.cputube.org/patch_37_3367.log > > Marked as waiting-on-author. v6-0001-Reject-patterns-with-too-many-parts-or-wrong-db.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.

Re: MDAM techniques and Index Skip Scan patch

2022-03-21 Thread Andres Freund
Hi, On 2022-01-22 22:37:19 +0100, Dmitry Dolgov wrote: > > On Fri, Jan 14, 2022 at 04:03:41PM +0800, Julien Rouhaud wrote: > > Hi, > > > > On Fri, Jan 14, 2022 at 08:55:26AM +0100, Dmitry Dolgov wrote: > > > > > > FYI, I've attached this thread to the CF item as an informational one, > > > but as

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-03-21 Thread Andres Freund
Hi, On 2022-03-08 21:44:37 +0800, Andy Fan wrote: > I have finished the PoC for planning timing improvement and joinrel rows > estimation. This currently crashes on cfbot: https://api.cirrus-ci.com/v1/task/6158455839916032/logs/cores.log https://cirrus-ci.com/task/6158455839916032 As this is cle

Re: speed up a logical replica setup

2022-03-21 Thread Fabrízio de Royes Mello
On Fri, 18 Mar 2022 at 19:34 Andrew Dunstan wrote: > > On 3/15/22 09:51, Peter Eisentraut wrote: > > On 21.02.22 13:09, Euler Taveira wrote: > >> A new tool called pg_subscriber does this conversion and is tightly > >> integrated > >> with Postgres. > > > > Are we comfortable with the name pg_sub

Re: Commitfest manager for 2022-03

2022-03-21 Thread Andres Freund
Hi, On 2022-02-26 16:12:27 -0500, Greg Stark wrote: > I do have the time available. What I don't necessarily have is insight > into everything that needs to be done, especially behind the scenes. One thing that desperately needs doing, particularly during the last commitfest, is looking through C

Re: Frontend error logging style

2022-03-21 Thread Andres Freund
Hi, On 2022-02-25 12:15:25 -0500, Tom Lane wrote: > The attached revision does that, standardizes on pg_fatal() as the > abbreviation for pg_log_error() + exit(1), and invents detail/hint > features as per previous discussion. This unfortunately has had some bitrot: http://cfbot.cputube.org/patc

Re: Fast COPY FROM based on batch insert

2022-03-21 Thread Etsuro Fujita
On Fri, Jun 4, 2021 at 5:26 PM Andrey Lepikhov wrote: > We still have slow 'COPY FROM' operation for foreign tables in current > master. > Now we have a foreign batch insert operation And I tried to rewrite the > patch [1] with this machinery. I’d been reviewing the previous version of the patch

Re: Per-table storage parameters for TableAM/IndexAM extensions

2022-03-21 Thread Andres Freund
Hi, On 2022-02-24 12:26:08 +0530, Sadhuprasad Patro wrote: > I have added a dummy test module for table AM and did the document > change in the latest patch attached... The test module doesn't build on windows, unfortunately... Looks like you need to add PGDLLIMPORT to a few variables: [01:26:18.

RE: Logical replication timeout problem

2022-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 21, 2022 at 1:31 PM Amit Kapila wrote: > Thanks for your comments. > On Fri, Mar 18, 2022 at 4:20 PM Amit Kapila wrote: > > > > On Fri, Mar 18, 2022 at 10:43 AM wangw.f...@fujitsu.com > > wrote: > > > > > > On Thu, Mar 17, 2022 at 7:52 PM Masahiko Sawada > wrote: > > > > > > > > >

RE: logical replication empty transactions

2022-03-21 Thread houzj.f...@fujitsu.com
> On Monday, March 21, 2022 6:01 PM Amit Kapila > wrote: > > > > On Sat, Mar 19, 2022 at 9:10 AM Ajin Cherian wrote: > > > > > > On Thu, Mar 17, 2022 at 10:43 PM Amit Kapila > > > > > wrote: > > > > > > > 3. Can we add a simple test for it in one of the existing test > > > > files(say in 001_rep

Re: Frontend error logging style

2022-03-21 Thread Tom Lane
Andres Freund writes: > On 2022-02-25 12:15:25 -0500, Tom Lane wrote: >> The attached revision does that, standardizes on pg_fatal() as the >> abbreviation for pg_log_error() + exit(1), and invents detail/hint >> features as per previous discussion. > This unfortunately has had some bitrot: > ht

Re: Frontend error logging style

2022-03-21 Thread Andres Freund
Hi, On 2022-03-21 22:00:37 -0400, Tom Lane wrote: > Andres Freund writes: > > This unfortunately has had some bitrot: > > http://cfbot.cputube.org/patch_37_3574.log > > Yeah, I know. That patch touches enough places that it's sure to > break every few days during a commitfest. I'm not real ex

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-21 Thread Thomas Munro
I have a new socket abstraction patch that should address the known Windows socket/event bugs, but it's a little bigger than I thought it would be, not quite ready, and now too late to expect people to review for 15, so I think it should go into the next cycle. I've bounced https://commitfest.post

Re: [RFC] building postgres with meson -v8

2022-03-21 Thread Andres Freund
Hi, Attached is v8. It's just a rebase to resolve conflicts with recent changes. - Andres v8-0001-meson-prereq-output-and-depencency-tracking-work.patch.gz Description: application/patch-gzip binfm5f_PMrcn.bin Description: application/patch-gzip binkuFB75lqSa.bin Description: application/pa

Re: Make mesage at end-of-recovery less scary.

2022-03-21 Thread Kyotaro Horiguchi
At Mon, 21 Mar 2022 17:01:19 -0700, Andres Freund wrote in > Patch currently fails to apply, needs a rebase: > http://cfbot.cputube.org/patch_37_2490.log Thanks for noticing me of that. Rebased to the current HEAD. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From a7c9f36e6

Re: Fast COPY FROM based on batch insert

2022-03-21 Thread Etsuro Fujita
On Tue, Mar 22, 2022 at 8:58 AM Andres Freund wrote: > > On 2021-06-07 16:16:58 +0500, Andrey Lepikhov wrote: > > Second version of the patch fixes problems detected by the FDW regression > > tests and shows differences of error reports in tuple-by-tuple and batched > > COPY approaches. > > Patch

Broken make dependency somewhere near win32ver.o?

2022-03-21 Thread Thomas Munro
Hi, Here's a strange one-off failure seen on CI[1], in the CompilerWarnings task where we check that mingw cross-compile works: [10:48:29.045] time make -s -j${BUILD_JOBS} world-bin [10:48:38.705] x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or directory [10:48:38.705] make[3]: *** [Ma

Re: On login trigger: take three

2022-03-21 Thread Andres Freund
Hi, On 2022-03-18 17:03:39 +0100, Daniel Gustafsson wrote: > > On 18 Mar 2022, at 08:24, a.soko...@postgrespro.ru wrote: > > > - auth_extra => [ '--create-role', 'repl_role' ]); > > + auth_extra => [ '--create-role', 'repl_role,regress_user' ]); > > Looking at the test in

Re: shared-memory based stats collector - v67

2022-03-21 Thread Kyotaro Horiguchi
At Mon, 21 Mar 2022 14:30:17 -0700, Andres Freund wrote in > Hi, > > Attached is v67 of the patch. Changes: Thanks for the lot of work on this. > > This is also painful to maintain. Mostly kept separate from 0007 for easier > > reviewing: > > - 0009-pgstat-reorder-file-pgstat.c-pgstat.h-conte

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-21 Thread Tom Lane
Thomas Munro writes: > I have a new socket abstraction patch that should address the known > Windows socket/event bugs, but it's a little bigger than I thought it > would be, not quite ready, and now too late to expect people to review > for 15, so I think it should go into the next cycle. I've b

Re: Broken make dependency somewhere near win32ver.o?

2022-03-21 Thread Andres Freund
Hi, On 2022-03-22 15:47:13 +1300, Thomas Munro wrote: > Here's a strange one-off failure seen on CI[1], in the > CompilerWarnings task where we check that mingw cross-compile works: > > [10:48:29.045] time make -s -j${BUILD_JOBS} world-bin > [10:48:38.705] x86_64-w64-mingw32-gcc: error: win32ver.

Re: Add index scan progress to pg_stat_progress_vacuum

2022-03-21 Thread Masahiko Sawada
Sorry for the late reply. On Tue, Mar 15, 2022 at 1:20 AM Imseih (AWS), Sami wrote: > > >I'm still unsure the current design of 0001 patch is better than other > >approaches we’ve discussed. Even users who don't use parallel vacuum > >are forced to allocate shared memory for index vac

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-21 Thread Thomas Munro
On Tue, Mar 22, 2022 at 4:13 PM Tom Lane wrote: > Thomas Munro writes: > > I have a new socket abstraction patch that should address the known > > Windows socket/event bugs, but it's a little bigger than I thought it > > would be, not quite ready, and now too late to expect people to review > > f

Re: Add index scan progress to pg_stat_progress_vacuum

2022-03-21 Thread Andres Freund
Hi, On 2022-03-16 21:47:49 +, Imseih (AWS), Sami wrote: > From 85c47dfb3bb72f764b9052e74a7282c19ebd9898 Mon Sep 17 00:00:00 2001 > From: "Sami Imseih (AWS)" > Date: Wed, 16 Mar 2022 20:39:52 + > Subject: [PATCH 1/1] Add infrastructure for parallel progress reporting > > Infrastructure to

Re: A test for replay of regression tests

2022-03-21 Thread Masahiko Sawada
i, On Mon, Mar 21, 2022 at 5:45 AM Thomas Munro wrote: > > On Mon, Mar 21, 2022 at 2:34 AM Andrew Dunstan wrote: > > On 3/20/22 05:36, Thomas Munro wrote: > > > On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro > > > wrote: > > >> I'll try to come up with the perl needed to see the regression.diffs

Re: Per-table storage parameters for TableAM/IndexAM extensions

2022-03-21 Thread Sadhuprasad Patro
On Tue, Mar 22, 2022 at 7:24 AM Andres Freund wrote: > > Hi, > > On 2022-02-24 12:26:08 +0530, Sadhuprasad Patro wrote: > > I have added a dummy test module for table AM and did the document > > change in the latest patch attached... > > The test module doesn't build on windows, unfortunately... L

Re: A test for replay of regression tests

2022-03-21 Thread Thomas Munro
On Tue, Mar 22, 2022 at 4:31 PM Masahiko Sawada wrote: > SELECT pg_relation_size('vac_truncate_test') = 0; > ?column? > -- > - t > + f Thanks. Ahh, déjà vu... this probably needs the same treatment as b700f96c and 3414099c provided for the reloptions test. Well, at least the first

Re: simplifying foreign key/RI checks

2022-03-21 Thread Amit Langote
On Mon, Mar 14, 2022 at 6:28 PM Zhihong Yu wrote: > On Mon, Mar 14, 2022 at 1:33 AM Amit Langote wrote: >> On Tue, Jan 18, 2022 at 3:30 PM Amit Langote wrote: >> > v13 is attached. >> >> I noticed that the recent 641f3dffcdf's changes to >> get_constraint_index() made it basically unusable for t

Re: Commitfest manager for 2022-03

2022-03-21 Thread Greg Stark
On Mon, 21 Mar 2022 at 21:48, Andres Freund wrote: > > Hi, > > On 2022-02-26 16:12:27 -0500, Greg Stark wrote: > > I do have the time available. What I don't necessarily have is insight > > into everything that needs to be done, especially behind the scenes. > > One thing that desperately needs do

Re: Out-of-tree certificate interferes ssltest

2022-03-21 Thread Michael Paquier
On Fri, Mar 18, 2022 at 06:15:28PM -0400, Andrew Dunstan wrote: > On 3/17/22 21:02, Michael Paquier wrote: >> Another thing that Horiguchi-san has pointed out upthread (?) is 003, >> where it is also possible to trigger failures once the environment is >> hijacked. The attached allows the full tes

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-21 Thread Kyotaro Horiguchi
At Fri, 18 Mar 2022 16:38:57 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 17 Mar 2022 21:55:07 +, Jacob Champion > wrot> Thanks! .. and some nitpicks..(Sorry) > > fe-secure-common.c doesn't need netinet/in.h. > > > +++ b/src/include/utils/inet.h > .. > +#include "common/inet-comm

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-21 Thread Dilip Kumar
On Mon, Mar 21, 2022 at 11:53 PM Robert Haas wrote: > > On Mon, Mar 21, 2022 at 11:21 AM Dilip Kumar wrote: > > I tried to debug the case but I realized that somehow > > CHECK_FOR_INTERRUPTS() is not calling the > > AcceptInvalidationMessages() and I could not find the same while > > looking into

Re: New Object Access Type hooks

2022-03-21 Thread Thomas Munro
On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger wrote: > (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl when > testing v1-0001. I'm not sure yet what that is about.) Doesn't look like 0001 has anything to do with that... Are you on a Mac? Did it look like this recent fa

Re: Add sub-transaction overflow status in pg_stat_activity

2022-03-21 Thread Dilip Kumar
On Tue, Mar 22, 2022 at 5:15 AM Andres Freund wrote: > There seems to be some agreement on this (I certainly do agree). Thus it seems > we should mark the CF entry as rejected? > > It's been failing on cfbot for weeks... > https://cirrus-ci.com/task/5289336424890368?logs=docs_build#L347 I have

Re: Broken make dependency somewhere near win32ver.o?

2022-03-21 Thread Thomas Munro
On Tue, Mar 22, 2022 at 4:14 PM Andres Freund wrote: > The problem looks to be that pg_dumpall doesn't have a dependency on OBJs, > which in turn is what contains the dependency on WIN32RES, which in turn > contains win32ver.o. So the build succeeds if pg_dump/restores's dependencies > are built f

<    1   2