Re: [Proposal] Add foreign-server health checks infrastructure

Hi, Kuroda-san. At Thu, 17 Feb 2022 04:11:09 +, "kuroda.hay...@fujitsu.com" wrote in > Dear Horiguchi-san, > > Thank you for giving your suggestions. I want to confirm your saying. > > > FWIW, I'm not sure this feature necessarily requires core support > > dedicated to FDWs. The core hav

Re: [Proposal] Add foreign-server health checks infrastructure

At Thu, 17 Feb 2022 04:32:26 +, "kuroda.hay...@fujitsu.com" wrote in > > I understood here as removing following mechanism from core: > > > > * disable timeout at end of tx. > > While reading again and this part might be wrong. > Sorry for inconvenience. > But anyway some codes should be (

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

At Thu, 17 Feb 2022 15:50:09 +0800, Julien Rouhaud wrote in > On Thu, Feb 17, 2022 at 03:51:26PM +0900, Kyotaro Horiguchi wrote: > > So, the function doesn't return 63 for all registered names and wrong > > names. > > > > So other possibilities I can think of are.. > > - Someone had broken pg_e

Assert in pageinspect with NULL pages

Hi, hackers! If we trying to call pageinspect functions for pages which are filled with nulls, we will get core dump. It happens with null pages for all indexes in pageinspect and for page_checksum. This problem was founded firstly by Anastasia Lubennikova, and now I continue this task. For

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

On Thu, Feb 17, 2022 at 05:24:58PM +0900, Kyotaro Horiguchi wrote: > At Thu, 17 Feb 2022 15:50:09 +0800, Julien Rouhaud wrote > in > > On Thu, Feb 17, 2022 at 03:51:26PM +0900, Kyotaro Horiguchi wrote: > > > So, the function doesn't return 63 for all registered names and wrong > > > names. > > >

Re: Assert in pageinspect with NULL pages

On Thu, Feb 17, 2022 at 01:46:40PM +0500, Daria Lepikhova wrote: > INSERT INTO test1(y) SELECT 0 FROM generate_series(1,1E6) AS x; > SELECT page_checksum(repeat(E'\\000', 8192)::bytea, 1); Indeed. Good catch, and that seems pretty old at quick glance for the checksum part. I'll try to look at al

Re: Failed transaction statistics to measure the logical replication progress

On Wed, Jan 12, 2022 at 6:04 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, December 23, 2021 6:37 PM Wang, Wei/王 威 > wrote: > > On Wednesday, December 22, 2021 10:30 PM osumi.takami...@fujitsu.com > > wrote: > > > On Wednesday, December 22, 2021 8:38 PM I wrote: > > > > Do we expect th

Re: Assert in pageinspect with NULL pages

On Thu, Feb 17, 2022 at 05:57:49PM +0900, Michael Paquier wrote: > On Thu, Feb 17, 2022 at 01:46:40PM +0500, Daria Lepikhova wrote: > > INSERT INTO test1(y) SELECT 0 FROM generate_series(1,1E6) AS x; > > SELECT page_checksum(repeat(E'\\000', 8192)::bytea, 1); > > Indeed. Good catch, and that seem

Re: Failed transaction statistics to measure the logical replication progress

On Tue, Jan 4, 2022 at 5:22 PM osumi.takami...@fujitsu.com wrote: > > On Friday, December 31, 2021 10:12 AM Tang, Haiying/唐 海英 > wrote: > > 4) > > +void > > +pgstat_send_subworker_xact_stats(LogicalRepWorker *repWorker, bool > > +force) { > > + static TimestampTz last_report = 0; > > + P

Re: Failed transaction statistics to measure the logical replication progress

On Thu, Feb 17, 2022 at 3:13 PM Amit Kapila wrote: > > On Tue, Jan 4, 2022 at 5:22 PM osumi.takami...@fujitsu.com > wrote: > > > > On Friday, December 31, 2021 10:12 AM Tang, Haiying/唐 海英 > > wrote: > > > 4) > > > +void > > > +pgstat_send_subworker_xact_stats(LogicalRepWorker *repWorker, bool >

RE: [Proposal] Add foreign-server health checks infrastructure

Dear Horiguchi-san, > I think we just don't need to add the special timeout kind to the > core. postgres_fdw can use USER_TIMEOUT and it would be suffiction to > keep running health checking regardless of transaction state then fire > query cancel if disconnection happens. As I said in the previo

Re: logical replication empty transactions

On Mon, Jan 31, 2022 at 6:18 PM Ajin Cherian wrote: > Few comments: = 1. Is there any particular why the patch is not skipping empty xacts for streaming (in-progress) transactions as noted in the commit message as well? 2. +static void +pgoutput_begin(LogicalDecodingContext *ctx, Reo

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

Hi, On Thu, Feb 17, 2022 at 12:26:07PM +0530, Nitin Jadhav wrote: > > Thank you for sharing the information. 'triggering backend PID' (int) > - can be stored without any problem. There can be multiple processes triggering a checkpoint, or at least wanting it to happen or happen faster. > 'chec

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

On Thu, 17 Feb 2022 at 07:56, Nitin Jadhav wrote: > > > Progress parameters are int64, so all of the new 'checkpoint start > > location' (lsn = uint64), 'triggering backend PID' (int), 'elapsed > > time' (store as start time in stat_progress, timestamp fits in 64 > > bits) and 'checkpoint or resta

Re: logical replication empty transactions

On Wed, Feb 16, 2022 at 8:45 AM osumi.takami...@fujitsu.com wrote: [ideas to skip empty prepare/commit_prepare ] > > I feel if we don't want to change the protocol of commit_prepared, > we need to make the publisher solely judge whether the prepare was empty or > not, > after the restart. >

Re: Small and unaffected typo in pg_logical_slot_get_changes_guts()

Michael Paquier writes: > On Wed, Feb 16, 2022 at 01:25:09PM +0900, Kasahara Tatsuhito wrote: >> Remove all references to tuplestore_donestoring() except for the header. > > Looks fine, thanks. This has no impact on Melanie's patch posted on > [1], so applied after tweaking the comment in tuples

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

Em qui., 17 de fev. de 2022 às 05:25, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > At Thu, 17 Feb 2022 15:50:09 +0800, Julien Rouhaud > wrote in > > On Thu, Feb 17, 2022 at 03:51:26PM +0900, Kyotaro Horiguchi wrote: > > > So, the function doesn't return 63 for all registered names and

Re: Make mesage at end-of-recovery less scary.

On Thu, Feb 17, 2022 at 1:20 PM Kyotaro Horiguchi wrote: > > At Tue, 15 Feb 2022 20:17:20 +0530, Ashutosh Sharma > wrote in > > OK. The v13 patch looks good. I have marked it as ready to commit. > > Thank you for working on all my review comments. > > Thaks! But the recent xlog.c refactoring cra

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

Sorry for the break post... Em qui., 17 de fev. de 2022 às 05:25, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > At Thu, 17 Feb 2022 15:50:09 +0800, Julien Rouhaud > wrote in > > On Thu, Feb 17, 2022 at 03:51:26PM +0900, Kyotaro Horiguchi wrote: > > > So, the function doesn't return 63

[PATCH] Fix possible minor memory leak (src/backend/catalog/heap.c)

Hi, Per Coverity. Maybe this is a mistake, but, Is it necessary or not to free the memory allocated by nodeToString? If yes, the patch attached fixes this. regards, Ranier Vilela fix_minor_memory_leak_heap.patch Description: Binary data

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

> On 17 Feb 2022, at 13:19, Ranier Vilela wrote: > 1. One #ifdef with a mistake, the correct is _WIN32 and not WIN32. Can you elaborate on this, we are using WIN32 pretty extensively in the code: $ git grep "if[n]\{0,1\}def WIN32$"|wc -l 511 $ git grep "if[n]\{0,1\}def _WIN32$"|wc -l

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson escreveu: > > On 17 Feb 2022, at 13:19, Ranier Vilela wrote: > > > 1. One #ifdef with a mistake, the correct is _WIN32 and not WIN32. > > Can you elaborate on this, we are using WIN32 pretty extensively in the > code: > > $ git grep "if[n]

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

> On 17 Feb 2022, at 13:59, Ranier Vilela wrote: > > Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson > escreveu: > > On 17 Feb 2022, at 13:19, Ranier Vilela > > wrote: > > > 1. One #ifdef with a mistake, the correct is _WIN32 and not

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

Em qui., 17 de fev. de 2022 às 10:18, Daniel Gustafsson escreveu: > > On 17 Feb 2022, at 13:59, Ranier Vilela wrote: > > > > Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson > escreveu: > > > On 17 Feb 2022, at 13:19, Ranier Vilela ranier...@gmail.com>> wrote: >

Re: postgres_fdw and skip locked

On Wed, Feb 16, 2022 at 8:38 PM Alexander Pyhalov wrote: > > Ashutosh Bapat писал 2022-02-16 16:40: > > On Mon, Feb 14, 2022 at 4:23 PM Alexander Pyhalov > > wrote: > >> > >> Hi. > >> > >> Now select ... for update ... [skip locked|nowait] options are not > >> pushed down to remote servers. I see

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Wed, Feb 16, 2022 at 07:43:09PM -0800, Peter Geoghegan wrote: > > I also have a hard time making heads or tails out of the commit message of > > 44fa84881ff. It's quite long without being particularly descriptive. The > > commit just changes a lot of things at once, making it hard to precisely >

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Wed, Feb 16, 2022 at 10:43 PM Peter Geoghegan wrote: > How can you be surprised that I committed 44fa8488? It's essentially > the same patch as the first version, posted November 22 -- almost 3 > months ago. And it's certainly not a big patch (though it is > complicated). Let's back up a minut

Re: killing perl2host

On 2/16/22 21:17, Andres Freund wrote: > Hi, > > On 2022-02-16 14:42:30 -0800, Andres Freund wrote: >> On February 16, 2022 1:10:35 PM PST, Andrew Dunstan >> wrote: >>> So something like this in Utils.pm: >>> >>> >>> die "Msys targeted perl is unsupported for running TAP tests" if >>> $Config{o

Re: adding 'zstd' as a compression algorithm

On Wed, Feb 16, 2022 at 11:34 PM Michael Paquier wrote: > Thanks. This looks pretty much right, except for two things that I > have taken the freedom to fix as of the v3 attached. Ah, OK, cool. It seems like we have consensus on this being a good direction, so I plan to commit this later today u

Re: PGEventProcs must not be allowed to break libpq

Not really related to this complaint and patch, but as far as I can see, libpq events go completely untested in the core source. Maybe we should come up with a test module or something? -- Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/

Re: PGEventProcs must not be allowed to break libpq

Alvaro Herrera writes: > Not really related to this complaint and patch, but as far as I can see, > libpq events go completely untested in the core source. Maybe we should > come up with a test module or something? Yeah, I suppose. The libpq part of it is pretty simple, but still...

Re: do only critical work during single-user vacuum?

On Wed, Feb 16, 2022 at 10:08 PM Peter Geoghegan wrote: > > 6. Sometimes the user decides to run VACUUM FULL instead of plain > > VACUUM because it sounds better. > > It's a pity that the name suggests otherwise. If only we'd named it > something that suggests "option of last resort". Oh well. Un

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

Daniel Gustafsson writes: > Question remains though, should src/interfaces/ecpg/test/sql/sqlda.pgc really > be using WIN32 and not _WIN32, or doesn't it matter? (or does it only matter > for consistency?) WIN32 and _WIN32 aren't very informative searchterms to use > for finding more information.

Re: initdb / bootstrap design

On Wed, Feb 16, 2022 at 2:50 PM Andres Freund wrote: > > initdb is already plenty fast enough for any plausible production > > usage; it's cases like check-world where we wish it were faster. > > It's not just our own usage though. I've seen it be a noticable time in test > suites of applications

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

> On 17 Feb 2022, at 16:05, Tom Lane wrote: > > Daniel Gustafsson writes: >> Question remains though, should src/interfaces/ecpg/test/sql/sqlda.pgc really >> be using WIN32 and not _WIN32, or doesn't it matter? (or does it only matter >> for consistency?) WIN32 and _WIN32 aren't very informativ

Re: pgsql: pg_upgrade: Preserve relfilenodes and tablespace OIDs.

On Sun, Feb 13, 2022 at 6:51 AM Christoph Berg wrote: > Re: Robert Haas > > pg_upgrade: Preserve relfilenodes and tablespace OIDs. > > > src/bin/pg_dump/pg_dumpall.c | 3 + > > --- a/src/bin/pg_dump/pg_dumpall.c > +++ b/src/bin/pg_dump/pg_dumpall.c > @@ -1066,6 +1066,9 @@ du

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

On 2022/02/11 21:59, Etsuro Fujita wrote: I tweaked comments/docs a little bit as well. Thanks for updating the patches! I reviewed 0001 patch. It looks good to me except the following minor things. If these are addressed, I think that the 001 patch can be marked as ready for committer.

Re: initdb / bootstrap design

On 2/17/22 10:36, Robert Haas wrote: > On Wed, Feb 16, 2022 at 2:50 PM Andres Freund wrote: >>> initdb is already plenty fast enough for any plausible production >>> usage; it's cases like check-world where we wish it were faster. >> It's not just our own usage though. I've seen it be a noticabl

Re: buildfarm warnings

I wrote: > Justin Pryzby writes: >> pg_basebackup.c:1261:35: warning: storing the address of local variable >> archive_filename in progress_filename [-Wdangling-pointer=] >> => new in 23a1c6578 - looks like a real error > I saw that one a few days ago but didn't get around to looking > more clos

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

> > Thank you for sharing the information. 'triggering backend PID' (int) > > - can be stored without any problem. > > There can be multiple processes triggering a checkpoint, or at least wanting > it > to happen or happen faster. Yes. There can be multiple processes but there will be one checkp

Re: killing perl2host

Hi, On 2022-02-17 09:20:56 -0500, Andrew Dunstan wrote: > I don't think we have or have ever had a buildfarm animal targeting > msys. In general I think of msys as a build environment to create native > binaries. But if we want to support targeting msys we should have an > animal doing that. It's

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

Hi, On Thu, Feb 17, 2022 at 10:39:02PM +0530, Nitin Jadhav wrote: > > > Thank you for sharing the information. 'triggering backend PID' (int) > > > - can be stored without any problem. > > > > There can be multiple processes triggering a checkpoint, or at least > > wanting it > > to happen or ha

Re: [PATCH] Accept IP addresses in server certificate SANs

On Tue, 2022-02-15 at 15:16 +0900, Kyotaro Horiguchi wrote: > (This needs rebasing) Done in v6, attached. > # I forgot to mention that, the test fails for me even without the > # change. I didn't checked what is wrong there, though. Ah. We should probably figure that out, then -- what failures

Re: CREATEROLE and role ownership hierarchies

On Mon, Jan 31, 2022 at 1:57 PM Joshua Brindle wrote: > This is precisely the use case I am trying to accomplish with this > patchset, roughly: > > - An automated bot that creates users and adds them to the employees role > - Bot cannot access any employee (or other roles) table data > - Bot canno

Re: Per-table storage parameters for TableAM/IndexAM extensions

> On Sat, Feb 12, 2022 at 2:35 AM Robert Haas wrote: >> >> >> Imagine that I am using the "foo" tableam with "compression=lots" and >> I want to switch to the "bar" AM which does not support that option. >> If I remove the "compression=lots" option using a separate command, >> the "foo" table AM m

Re: buildfarm warnings

On Thu, Feb 17, 2022 at 12:08 PM Tom Lane wrote: > I wrote: > > Justin Pryzby writes: > >> pg_basebackup.c:1261:35: warning: storing the address of local variable > >> archive_filename in progress_filename [-Wdangling-pointer=] > >> => new in 23a1c6578 - looks like a real error > > > I saw that

Re: killing perl2host

On 2/17/22 12:12, Andres Freund wrote: > Hi, > > On 2022-02-17 09:20:56 -0500, Andrew Dunstan wrote: >> I don't think we have or have ever had a buildfarm animal targeting >> msys. In general I think of msys as a build environment to create native >> binaries. But if we want to support targeting

Re: Time to drop plpython2?

Hi, On 2022-02-16 23:14:46 -0800, Andres Freund wrote: > > > > Done. Curious how red the BF will turn out to be. Let's hope it's not > > too bad. > > I've pinged the owners of the animals failing so far: Now also pinged: - curculio - guaibasaurus - snapper - gadwall, takin > - snakefly, massasa

Re: killing perl2host

On 2022-02-17 13:08:01 -0500, Andrew Dunstan wrote: > >> perhaps something like: > >> > >> > >>     my $msystem = $ENV{MSYSTEM} || 'undef'; > >> > >>     die "incompatible perl" if $Config{osname} eq 'msys' && $msystem ne > >> 'MSYS'; > > Why tests MSYSTEM instead of $host_os? > Is that available

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

Hi, On 2022-02-17 09:17:04 -0500, Robert Haas wrote: > Commit messages need to describe what the commit actually changes. > Theoretical ideas are fine, but if I, as a committer who have done > significant work in this area in the past, can't read the commit > message and understand what is actuall

Re: O(n) tasks cause lengthy startups and checkpoints

On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: > On 2022-02-16 20:14:04 -0800, Nathan Bossart wrote: >> >> - while ((spc_de = ReadDirExtended(spc_dir, "pg_tblspc", LOG)) != NULL) >> >> + while (!ShutdownRequestPending && >> >> +(spc_de = ReadDirExtended(spc_dir, "pg_tbls

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

Hi, On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > 44fa8488 started off -- purely as refactoring work. The problem is that it didn't end up as that. You combined refactoring with substantial changes. And describe

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

So I think knowing what bad it is to have this feature is the key point to discussion now. > While I've only read your description of the patch not the patch itself, This comment applies to me also. Is the join selectivity properly calculated in all cases, e.g. in the n:m join case in parti

Isn't gist_page_items broken for opclasses with compress methods?

Hi, I've been looking at a report of a crash an in-place upgrade [1], related to a GiST index, and I tried to use the new GiST support added to pageinspect in 756ab29124 (so it's in 14). My knowledge of GiST is somewhat limited, but after struggling with it for a while I wonder if it's actually co

Re: support for CREATE MODULE

On Tue, Feb 15, 2022 at 12:29:54PM -0800, Swaha Miller wrote: > On Mon, Feb 14, 2022 at 4:58 PM Bruce Momjian wrote: > > I was working on a talk about microservices today and decided to create > > two schemas --- a public one that has USAGE permission for other > services > > with vie

Re: [PATCH] Support pg_ident mapping for LDAP

On Fri, 2021-10-29 at 17:38 +, Jacob Champion wrote: > v3 attached, which uses the above naming scheme and removes the stale > TODO. Changes in since-v2. v4 rebases over the recent TAP changes. --Jacob From e0f36725013610eade9bc83414c4d1f5adea17e2 Mon Sep 17 00:00:00 2001 From: Jacob Champion

Proposal: Support custom authentication methods using hooks

Hi all, I wanted to submit a patch to expose 2 new hooks (one for the authentication check and another one for error reporting) in auth.c. These will allow users to implement their own authentication methods for Postgres or add custom logic around authentication. A use case where this is useful a

Re: O(n) tasks cause lengthy startups and checkpoints

Hi, On 2022-02-17 10:23:37 -0800, Nathan Bossart wrote: > On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: > > They're accessed by xid. The LSN is just for cleanup. Accessing files > > left over from a previous transaction with the same xid wouldn't be > > good - we'd read wrong cata

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Tue, Feb 15, 2022 at 6:49 AM Dilip Kumar wrote: > Here is the updated version of the patch, the changes are 1) Fixed > review comments given by Robert and one open comment from Ashutosh. > 2) Preserved the old create db method. 3) As agreed upthread for now > we are using the new strategy only

Re: buildfarm warnings

Robert Haas writes: > If not, I think that your quick-and-dirty fix is about right, except > that we probably need to do it every place where we set > progress_filename, and we should arrange to free the memory later. > With -Ft, you won't have enough archives to matter, but with -Fp, you > might

Re: killing perl2host

On 2/17/22 13:10, Andres Freund wrote: > On 2022-02-17 13:08:01 -0500, Andrew Dunstan wrote: perhaps something like:     my $msystem = $ENV{MSYSTEM} || 'undef';     die "incompatible perl" if $Config{osname} eq 'msys' && $msystem ne 'MSYS'; >>> Why tests MSYSTEM

Re: killing perl2host

Hi, On 2022-02-17 14:40:01 -0500, Andrew Dunstan wrote: > Sure, that could be done, but what's the issue? Msys2 normally defines > MSYSTEM for you - see /etc/msystem which is sourced by /etc/profile. It seems not a great idea to me to use different sources of truth about build target. And I think

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

On Tue, Feb 1, 2022 at 10:08 AM Andy Fan wrote: > To address the row estimation issue, The most straightforward way to fix this > is to > ignore the derived clauses when figuring out the RelOptInfo->rows on base > relation. > To note which clause is derived from this patch, I added a new field

Re: buildfarm warnings

On Thu, Feb 17, 2022 at 2:36 PM Tom Lane wrote: > Yeah, I came to the same conclusion while out doing some errands. > There's no very convincing reason to believe that what's passed to > progress_update_filename has got adequate lifespan either, or that > that would remain true even if it's true t

Re: killing perl2host

On 2/17/22 15:09, Andres Freund wrote: > Hi, > > On 2022-02-17 14:40:01 -0500, Andrew Dunstan wrote: >> Sure, that could be done, but what's the issue? Msys2 normally defines >> MSYSTEM for you - see /etc/msystem which is sourced by /etc/profile. > It seems not a great idea to me to use different

Re: killing perl2host

On 2022-02-17 15:23:36 -0500, Andrew Dunstan wrote: > Very well. I think the easiest way will be to stash $host_os in the > environment and let the script pick it up similarly to what I suggested > with MSYSTEM. WFM.

Re: buildfarm warnings

On 2022-02-17 15:22:08 -0500, Robert Haas wrote: > OK, sounds good, thanks. I couldn't (and still can't) think of a good > way of testing the progress-reporting code either. I mean I guess if > you could convince pg_basebackup not to truncate the filenames, maybe > by convincing it that your termin

Re: buildfarm warnings

On Thu, Feb 17, 2022 at 3:51 PM Andres Freund wrote: > On 2022-02-17 15:22:08 -0500, Robert Haas wrote: > > OK, sounds good, thanks. I couldn't (and still can't) think of a good > > way of testing the progress-reporting code either. I mean I guess if > > you could convince pg_basebackup not to tru

Re: O(n) tasks cause lengthy startups and checkpoints

On Thu, Feb 17, 2022 at 11:27:09AM -0800, Andres Freund wrote: > On 2022-02-17 10:23:37 -0800, Nathan Bossart wrote: >> On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: >> > They're accessed by xid. The LSN is just for cleanup. Accessing files >> > left over from a previous transactio

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Thu, Feb 17, 2022 at 6:17 AM Robert Haas wrote: > Let's back up a minute and talk about the commit of $SUBJECT. The > commit message contains a Discussion link to this thread. This thread, > at the time you put that link in there, had exactly one post: from > you. That's not much of a discussio

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Hi, On 2022-02-17 14:27:09 -0500, Robert Haas wrote: > The other one is trickier, because AFAICT it's basically an opinion > question: is accessing pg_class in the template database from some backend > that is connected to another database too ugly to be acceptable? Several > people have expressed

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

On 2/17/22 21:15, Robert Haas wrote: > On Tue, Feb 1, 2022 at 10:08 AM Andy Fan wrote: >> To address the row estimation issue, The most straightforward way to fix >> this is to >> ignore the derived clauses when figuring out the RelOptInfo->rows on base >> relation. >> To note which clause is de

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Thu, Feb 17, 2022 at 4:10 PM Peter Geoghegan wrote: > > I would say it differently: I think the commit message does a poor job > > describing what the commit actually does. For example, it says nothing > > about changing VACUUM to always scan the last page of every heap > > relation. This whole

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

On Thu, Feb 17, 2022 at 4:17 PM Tomas Vondra wrote: > IMHO the whole problem is we're unable to estimate the join clause as a > conditional probability, i.e. > >P(A.x = B.x | (A.x < 42) & (B.x < 42)) > > so maybe instead of trying to generate additional RelOptInfo items we > should think about

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Thu, Feb 17, 2022 at 1:36 PM Robert Haas wrote: > > It's not that simple. As I said in the fix-up commit message, and in > > the opening email to this thread, it basically isn't a new behavior at > > all. It would be much more accurate to describe it as a behavior that > > originated with commi

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Thu, Feb 17, 2022 at 10:33 AM Andres Freund wrote: > On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > > 44fa8488 started off -- purely as refactoring work. > > The problem is that it didn't end up as that. You

Re: O(n) tasks cause lengthy startups and checkpoints

Hi, On 2022-02-17 13:00:22 -0800, Nathan Bossart wrote: > Okay. So IIUC the problem might already exist today, but offloading these > tasks to a separate process could make it more likely. Vastly more, yes. Before checkpoints not happening would be a (but not a great) form of backpressure. You c

Re: O(n) tasks cause lengthy startups and checkpoints

On Thu, Feb 17, 2022 at 02:28:29PM -0800, Andres Freund wrote: > As far as I understand, the primary concern are logical decoding serialized > snapshots, because a lot of them can accumulate if there e.g. is an old unused > / far behind slot. It should be easy to reduce the number of those snapshot

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Thu, Feb 17, 2022 at 4:13 PM Andres Freund wrote: > Could you or Dilip outline how it now works, and what exactly makes it safe > etc (e.g. around locking, invalidation processing, snapshots, xid horizons)? > > I just scrolled through the patchset without finding such an explanation, so > it's

Re: Use generation context to speed up tuplesorts

On Sun, 13 Feb 2022 at 09:56, Tomas Vondra wrote: > I'm not against pushing the generation context improvements, e.g. > something like the patches posted in [1], because those seem reasonable > in general. But I'm somewhat confused about the last patch (adjusting > allocChunkLimit) causing fairly

Re: O(n) tasks cause lengthy startups and checkpoints

Hi, On 2022-02-17 14:58:38 -0800, Nathan Bossart wrote: > On Thu, Feb 17, 2022 at 02:28:29PM -0800, Andres Freund wrote: > > As far as I understand, the primary concern are logical decoding serialized > > snapshots, because a lot of them can accumulate if there e.g. is an old > > unused > > / far

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Hi, On 2022-02-17 18:00:19 -0500, Robert Haas wrote: > Now you pointed out earlier one problem that it doesn't fix: unlike > the current method, this method involves reading buffers from the > template database into shared_buffers, and those buffers, once read, > stick around even after the operat

Silencing upcoming warning about stack_base_ptr

GCC 12, coming soon to a distro near you, complains like this: postgres.c: In function 'set_stack_base': postgres.c:3430:24: warning: storing the address of local variable 'stack_base' in 'stack_base_ptr' [-Wdangling-pointer=] 3430 | stack_base_ptr = &stack_base; |

Re: Silencing upcoming warning about stack_base_ptr

Hi, On 2022-02-17 18:44:27 -0500, Tom Lane wrote: > (that's visible now on buildfarm member caiman). We probably > should take some thought for silencing this before it starts > to be in people's faces during routine development. Agreed. One annoying thing I recently encountered, related to thi

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

Hi, On 2022-02-17 14:23:51 -0800, Peter Geoghegan wrote: > On Thu, Feb 17, 2022 at 10:33 AM Andres Freund wrote: > > On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > > > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > > > 44fa8488 started off -- purely as refact

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

On Thu, Feb 17, 2022 at 4:18 PM Andres Freund wrote: > > What substantial changes are you referring to? The one thing that did > > change was the commit message, which framed everything in terms of the > > later work. It really is true that the patch that I committed was > > essentially the same p

Re: Race conditions in 019_replslot_limit.pl

Hi, On 2022-02-16 20:03:00 -0800, Andres Freund wrote: > I'll run the test in a loop, perhaps I can reproduce... I've done this for a couple hundred iterations, under load, subsequently with additional assertions, without being able to reproduce. There've not been any new reports in the last 18

Re: Silencing upcoming warning about stack_base_ptr

Andres Freund writes: > On 2022-02-17 18:44:27 -0500, Tom Lane wrote: >> (that's visible now on buildfarm member caiman). We probably >> should take some thought for silencing this before it starts >> to be in people's faces during routine development. > We could try using __builtin_frame_addres

Re: adding 'zstd' as a compression algorithm

On Thu, Feb 17, 2022 at 09:40:13AM -0500, Robert Haas wrote: > Ah, OK, cool. It seems like we have consensus on this being a good > direction, so I plan to commit this later today unless objections or > additional review comments turn up. So, will there be a part of the system where we'll make use

Re: Small and unaffected typo in pg_logical_slot_get_changes_guts()

On Thu, Feb 17, 2022 at 11:33:55AM +, Dagfinn Ilmari Mannsåker wrote: > Would it be possible to make that macro only defined when building > extensions, but not when building Postgres itself? For example, Perl > has a PERL_CORE macro that's only defined when building Perl itself, but > not whe

Re: adding 'zstd' as a compression algorithm

Hi, On 2022-02-17 13:34:08 +0900, Michael Paquier wrote: > %define needs to include HAVE_LIBZSTD, HAVE_ZSTD_H and USE_ZSTD, so > this version fails the sanity check between pg_config.h.in and the > MSVC scripts checking that all flags exist. Do we really need all three defines? How about using AC

Re: Support escape sequence for cluster_name in postgres_fdw.application_name

On 2022/02/15 8:52, r.takahash...@fujitsu.com wrote: Hi Fujii san, Thank you for updating the patch. I have no additional comments. Thanks for the review! Pushed. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Fix overflow in DecodeInterval

Ok, so I've attached a patch with my final unprompted changes. It contains the following two changes: 1. I found some more overflows with the ISO8601 formats and have included some fixes. 2. I reverted the overflow checks for the seconds field. It's actually a bit more complicated than I thought.

Re: Logical replication timeout problem

On Tue, Feb 8, 2022 at 1:59 PM wangw.f...@fujitsu.com wrote: > > On Wed, Jan 26, 2022 at 11:37 AM I wrote: > > On Sat, Jan 22, 2022 at 7:12 PM Amit Kapila wrote: > > > Now, one idea to solve this problem could be that whenever we skip > > > sending any change we do try to update the plugin progre

Re: Assert in pageinspect with NULL pages

BRIN can also crash if passed a non-brin index. I've been sitting on this one for awhile. Feel free to include it in your patchset. commit 08010a6037fc4e24a9ba05e5386e766f4310d35e Author: Justin Pryzby Date: Tue Jan 19 00:25:15 2021 -0600 pageinspect: brin_page_items(): check that given

Re: Assert in pageinspect with NULL pages

On Thu, Feb 17, 2022 at 05:40:41PM +0800, Julien Rouhaud wrote: > About the patch, it's incorrectly using a hardcoded 8192 block-size rather > than > using the computed :block_size (or computing one when it's not already the > case). Well, the tests of pageinspect fail would already fail when usi

Fix formatting of Interval output

When formatting the output of an Interval, we call abs() on the hours field of the Interval. Calling abs(INT_MIN) returns back INT_MIN causing the output to contain two '-' characters. The attached patch fixes that issue by special casing INT_MIN hours. Here is an example of the issue: postgre

Re: Assert in pageinspect with NULL pages

On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: > BRIN can also crash if passed a non-brin index. > > I've been sitting on this one for awhile. Feel free to include it in your > patchset. Ugh. Thanks! I am keeping a note about this one. -- Michael signature.asc Description: PG

Re: Fix CheckIndexCompatible comment

On 2022/02/07 19:14, Yugo NAGATA wrote: Agreed. I updated the patch to add a comment about 'oldId'. Thanks for updating the patch! I slightly modified the patch and pushed it. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPOR

Re: Race conditions in 019_replslot_limit.pl

Hi, On 2022-02-17 16:34:34 -0800, Andres Freund wrote: > I've done this for a couple hundred iterations, under load, subsequently with > additional assertions, without being able to reproduce. Playing around with this further I did get into one interesting state: I started psql with replication=

  1   2   >