Re: Atomic rename feature for Windows.

2022-04-13 Thread Michael Paquier
On Fri, Apr 08, 2022 at 11:44:31AM -0400, Greg Stark wrote: > Generally I think supporting older systems that do funny things is > helpful in avoiding problems that either 1) Can happen on newer > systems but rarely 2) Can happen on other systems that people are > using but we don't know about and

Re: Add --{no-,}bypassrls flags to createuser

2022-04-13 Thread Michael Paquier
On Wed, Apr 13, 2022 at 03:46:25PM +0900, Kyotaro Horiguchi wrote: > It is sensible to rig createuser command with full capability of > CREATE ROLE is reasonable. > > Only --replication is added by commit 9b8aff8c19 (2010) since > 8ae0d476a9 (2005). BYPASSRLS and NOBYPASSRLS were introduced by > 4

Re: Skip partition tuple routing with constant partition key

2022-04-13 Thread Masahiko Sawada
On Wed, Apr 13, 2022 at 2:39 PM Amit Kapila wrote: > > On Tue, Apr 12, 2022 at 6:16 AM Masahiko Sawada wrote: > > > > Hi, > > > > On Thu, Apr 7, 2022 at 4:37 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2022-04-06 00:07:07 -0400, Tom Lane wrote: > > > > Amit Langote writes: > > > > >

Re: Skipping schema changes in publication

2022-04-13 Thread Peter Smith
On Wed, Apr 13, 2022 at 2:40 PM Amit Kapila wrote: > > On Wed, Apr 13, 2022 at 8:45 AM vignesh C wrote: > > > > On Tue, Apr 12, 2022 at 4:46 PM Amit Kapila wrote: > > > > > > I understand that part but what I pointed out was that it might be > > > better to avoid using ADD keyword in this syntax

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Michael Paquier
On Tue, Apr 12, 2022 at 05:43:17PM -0400, Tom Lane wrote: > It probably is. I'm just offering this as a solution if people want to > insist on a mechanism to prevent unsafe GUC changes. If we drop the > idea of trying to forcibly prevent extensions from Doing It Wrong, > then we don't need this,

Re: Column Filtering in Logical Replication

2022-04-13 Thread Masahiko Sawada
On Sun, Mar 20, 2022 at 3:23 PM Amit Kapila wrote: > > On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: > > > > On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra > > wrote: > > > > > So the question is why those two sync workers never complete - I guess > > > there's some sort of lock wait (deadlo

Re: Add --{no-,}bypassrls flags to createuser

2022-04-13 Thread Kyotaro Horiguchi
At Wed, 13 Apr 2022 16:10:01 +0900, Michael Paquier wrote in > On Wed, Apr 13, 2022 at 03:46:25PM +0900, Kyotaro Horiguchi wrote: > > It is sensible to rig createuser command with full capability of > > CREATE ROLE is reasonable. > > > > Only --replication is added by commit 9b8aff8c19 (2010) s

Re: Support logical replication of DDLs

2022-04-13 Thread Dilip Kumar
On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila wrote: > > > The *initial* DDL replication is a different problem than DDL replication. > > The > > former requires a snapshot to read the current catalog data and build a > > CREATE > > command as part of the subscription process. The subsequent DDLs

pgbench translation

2022-04-13 Thread Guillaume Lelarge
Hello, I've been working yesterday on the translation of v15 .po french files. Once again, I've found that there was no .po file for oid2name, pgbench, and vacuumlo. I suppose that's because they are contrib applications. But pgbench is no longer a contrib application. It's now located in the "sr

Re: pgbench translation

2022-04-13 Thread Alvaro Herrera
On 2022-Apr-13, Guillaume Lelarge wrote: > Once again, I've found that there was no .po file for oid2name, pgbench, > and vacuumlo. I suppose that's because they are contrib applications. But > pgbench is no longer a contrib application. It's now located in the > "src/bin/pgbench" directory. So I'

Re: Column Filtering in Logical Replication

2022-04-13 Thread Amit Kapila
On Wed, Apr 13, 2022 at 1:41 PM Masahiko Sawada wrote: > > I've looked at this issue and had the same analysis. Also, I could > reproduce this issue with the steps shared by Amit. > > As I mentioned in another thread[1], the fact that the tablesync > worker doesn't check the return value from > wa

Re: Support logical replication of DDLs

2022-04-13 Thread Amit Kapila
On Wed, Apr 13, 2022 at 2:38 PM Dilip Kumar wrote: > > On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila wrote: > > > > > The *initial* DDL replication is a different problem than DDL > > > replication. The > > > former requires a snapshot to read the current catalog data and build a > > > CREATE > >

Re: pgbench translation

2022-04-13 Thread Guillaume Lelarge
Le mer. 13 avr. 2022 à 11:29, Alvaro Herrera a écrit : > On 2022-Apr-13, Guillaume Lelarge wrote: > > > Once again, I've found that there was no .po file for oid2name, pgbench, > > and vacuumlo. I suppose that's because they are contrib applications. But > > pgbench is no longer a contrib applica

Re: [RFC] building postgres with meson -v8

2022-04-13 Thread Peter Eisentraut
Some feedback and patches for your branch at 3274198960c139328fef3c725cee1468bbfff469: 0001-Install-a-few-more-files.patch These are just some files that were apparently forgotten to be installed so far. 0002-Adjust-some-header-file-installation-paths.patch The installation of server header

Re: Logical replication timeout problem

2022-04-13 Thread Amit Kapila
On Mon, Apr 11, 2022 at 12:09 PM wangw.f...@fujitsu.com wrote: > > So I skip tracking lag during a transaction just like the current HEAD. > Attach the new patch. > Thanks, please find the updated patch where I have slightly modified the comments. Sawada-San, Euler, do you have any opinion on th

Re: CLUSTER on partitioned index

2022-04-13 Thread Justin Pryzby
On Wed, Apr 13, 2022 at 03:50:15PM +0900, Michael Paquier wrote: > > > That dates to a556549d7 (see also cbe24a6dd8 for an earlier commit in > > CLUSTER > > itself). The reason was to avoid blocking if an unprivileged user runs > > VACUUM > > FULL which would try to lock things (including share

Re: A qsort template

2022-04-13 Thread John Naylor
As promised, I've done another round of tests (script and spreadsheet attached) with - v15 with 6974924347 and cc58eecc5d reverted - v15 with Thomas' patch - v15 with David's patch - v15 as is ("std") ...where v15 is at 7b735f8b52ad. This time I limited it to int, bigint, and text types. Since m

Error logging messages

2022-04-13 Thread Daniel Gustafsson
When reviewing the patch in "Frontend error logging style" [0] I noticed that some messages could do with a little bit of touching up. The original review was posted and responded to in that thread, but to keep goalposts in place it was put off until that patch had landed. To avoid this getting b

Re: Unit tests for SLRU

2022-04-13 Thread Pavel Borisov
> > Hi hackers, > > > Here is version 3 of the patch. > > [...] > > I think the tests are about as good as they will ever get. > > Here is version 4. Same as v3, but with resolved conflicts against the > current `master` branch. > Hi, Alexander! The test seems good enough to be pushed. Only one th

Re: should frontend tools use syncfs() ?

2022-04-13 Thread Justin Pryzby
On Thu, Sep 30, 2021 at 12:49:36PM +0900, Michael Paquier wrote: > On Wed, Sep 29, 2021 at 07:43:41PM -0500, Justin Pryzby wrote: > > Forking this thread in which Thomas implemented syncfs for the startup > > process > > (61752afb2). > > https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BSG9jS

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Tue, Apr 12, 2022 at 6:26 PM Nathan Bossart wrote: > Okay. So maybe we only need the attached patches. 0001 is just 5ecd018 > un-reverted, and 0002 is Julien's patch from a few weeks ago with some > minor edits. Hmm. I suppose I was thinking that we'd go the other way around: move the call t

Re: Add --{no-,}bypassrls flags to createuser

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 4:35 AM Kyotaro Horiguchi wrote: > I don't think there's a definitive criteria (other than feasibility) > for whether each CREATE ROLE option should have the correspondent > option in the createuser command. I don't see a clear reason why > createuser command should not ha

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > Hmm. I suppose I was thinking that we'd go the other way around: move > the call to InitializeMaxBackends() earlier, as proposed previously, > and add a hook to allow extensions to get control before that point. > The reason that I like that approach is that I suspect that it

JSON docs: RFC7159 is now superceded

2022-04-13 Thread Simon Riggs
Minor doc patch to replace with latest RFC number Intended for PG15 -- Simon Riggshttp://www.EnterpriseDB.com/ json_docs_rfc8259.v1.patch Description: Binary data

Re: Atomic rename feature for Windows.

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 3:04 AM Michael Paquier wrote: > Agreed. I think that things can be usually helpful. Now, I am not > really convinced that there is a strong need in running a VAX if you > are worrying about timing issues so this is a matter of balance. You > could get down to the same l

Re: JSON docs: RFC7159 is now superceded

2022-04-13 Thread Tom Lane
Simon Riggs writes: > Minor doc patch to replace with latest RFC number Hmm, I'm a bit disinclined to claim compliance with a new RFC sight unseen. What were the changes? regards, tom lane

Re: JSON docs: RFC7159 is now superceded

2022-04-13 Thread Andrew Dunstan
On 2022-04-13 We 09:38, Simon Riggs wrote: > Minor doc patch to replace with latest RFC number > > Intended for PG15 Idea is fine, but -  data, as specified in https://tools.ietf.org/html/rfc7159";>RFC -  7159. Such data can also be stored as text, but +  data, as specified in https://tools.

Re: JSON docs: RFC7159 is now superceded

2022-04-13 Thread Simon Riggs
On Wed, 13 Apr 2022 at 14:53, Tom Lane wrote: > > Simon Riggs writes: > > Minor doc patch to replace with latest RFC number > > Hmm, I'm a bit disinclined to claim compliance with a new RFC > sight unseen. What were the changes? I checked... so I should have mentioned this before https://datat

Re: deparsing utility commands

2022-04-13 Thread Peter Eisentraut
On 13.04.22 06:29, Ajin Cherian wrote: This patch-set has not been rebased for some time. I see that there is interest in this patch from the logical replication of DDL thread [1]. I will take a stab at rebasing this patch-set, I have already rebased the first patch and will work on the other pa

Re: Atomic rename feature for Windows.

2022-04-13 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 13, 2022 at 3:04 AM Michael Paquier wrote: >> Agreed. I think that things can be usually helpful. Now, I am not >> really convinced that there is a strong need in running a VAX if you >> are worrying about timing issues so this is a matter of balance. You >> c

Building Postgres with lz4 on Visual Studio

2022-04-13 Thread Melih Mutlu
Hi all, I tried to build Postgres from source using Visual Studio 19. It worked all good. Then I wanted to build it with some dependencies, started with the ones listed here [1]. But I'm having some issues with lz4. First I downloaded the latest release of lz4 from this link [2]. Modified the src

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 9:27 AM Tom Lane wrote: > Robert Haas writes: > > Hmm. I suppose I was thinking that we'd go the other way around: move > > the call to InitializeMaxBackends() earlier, as proposed previously, > > and add a hook to allow extensions to get control before that point. > > The

Re: Atomic rename feature for Windows.

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 10:19 AM Tom Lane wrote: > Meh. I agree that it seems unlikely that anyone will come out with a > new processor design that lacks the ability to do spinlocks or atomics. > It's substantially more likely though that someone would want those > configure switches temporarily

GSoC 2022: Proposal of pgmoneta on-disk encryption

2022-04-13 Thread Solo Kyo
To whom it may concern: Hi, I am Jichen Xu. I am a first year master computer science student at Waseda University. Here is a link to my proposal: https://docs.google.com/document/d/1vdgPY5wvhjhrX9aSUw5mTDOnXwEQNcr4cxfzuSHMWlg Looking forward to working with you in next few months. Best regards,

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > I wouldn't say that approach is wrong. I think we're just prioritizing > different things. I think that the pattern of wanting to request > shared memory in _PG_init() is common, and there are lots of > extensions that already do it that way, and if we pursue this plan, > the

Re: Fixes for compression options of pg_receivewal and refactoring of backup_compression.{c,h}

2022-04-13 Thread gkokolatos
--- Original Message --- On Wednesday, April 13th, 2022 at 7:25 AM, Michael Paquier wrote: > > > On Tue, Apr 12, 2022 at 06:22:48PM +0900, Michael Paquier wrote: > > > On Mon, Apr 11, 2022 at 12:46:02PM +, gkokola...@pm.me wrote: > > > > > It looks good. If you choose to discard

Re: Atomic rename feature for Windows.

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 10:19:44 -0400, Tom Lane wrote: > Meh. I agree that it seems unlikely that anyone will come out with a > new processor design that lacks the ability to do spinlocks or atomics. > It's substantially more likely though that someone would want those > configure switches temporarily

Re: GSoC 2022: Proposal of pgmoneta on-disk encryption

2022-04-13 Thread Jesper Pedersen
Hi Jichen, On 4/13/22 10:40, Solo Kyo wrote: To whom it may concern: Hi, I am Jichen Xu. I am a first year master computer science student at Waseda University. Here is a link to my proposal: https://docs.google.com/document/d/1vdgPY5wvhjhrX9aSUw5mTDOnXwEQNcr4cxfzuSHMWlg Looking forward to work

Re: Atomic rename feature for Windows.

2022-04-13 Thread Tom Lane
Andres Freund writes: > On 2022-04-13 10:19:44 -0400, Tom Lane wrote: >> Next decade's hot new processor design might do things >> differently enough that it matters that we use SpinLockInit() >> not memset-to-zero. This is not academic either, as we've had >> exactly such bugs in the past. > FW

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 10:43 AM Tom Lane wrote: > Yeah, there is something to be said for preventing subtle interactions > between extensions. > > > So in the end I see basically four possibilities here: > > > A. No hard compatibility break, but invent a set-a-GUC hook that runs > > earlier. > >

Re: Atomic rename feature for Windows.

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 11:03 AM Andres Freund wrote: > > Next decade's hot new processor design might do things > > differently enough that it matters that we use SpinLockInit() > > not memset-to-zero. This is not academic either, as we've had > > exactly such bugs in the past. > > FWIW, I'l lik

Re: Atomic rename feature for Windows.

2022-04-13 Thread Andres Freund
Hi, On April 13, 2022 8:30:33 AM PDT, Robert Haas wrote: >On Wed, Apr 13, 2022 at 11:03 AM Andres Freund wrote: >> > Next decade's hot new processor design might do things >> > differently enough that it matters that we use SpinLockInit() >> > not memset-to-zero. This is not academic either, a

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > I guess my feeling about this is that _PG_init() is sort of a grab > bag, and that's not very good. You're supposed to register new GUCs > there, and request shared memory space, and install any hook functions > you want to use, and maybe some other stuff. But why is it appro

Re: Improving the "Routine Vacuuming" docs

2022-04-13 Thread Robert Haas
On Tue, Apr 12, 2022 at 5:53 PM Peter Geoghegan wrote: > My high level concerns are: > > * Instead of discussing FrozenTransactionId (and then explaining how > that particular magic value is not really used anymore anyway), why > not describe freezing in terms of the high level rules? > > Somethin

Re: Temporary file access API

2022-04-13 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > Robert Haas wrote: > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > > > > > > https://wiki.postgre

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 11:39 AM Tom Lane wrote: > Is there anything else typically done in _PG_init that has to be > conditional on process_shared_preload_libraries_in_progress? I recall > something about reserving LWLocks, which probably should get the same > treatment. If we can get to a poin

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > What's a little wonky right now is that it's fairly common for > extensions to just return straight-off without doing *anything at all* > if !process_shared_preload_libraries_in_progress. See > pg_stat_statements for example. It seems kind of strange to me to make > registeri

Re: row filtering for logical replication

2022-04-13 Thread Alvaro Herrera
On 2022-Apr-13, Amit Kapila wrote: > Thanks, this will work and fix the issue. I think this looks better > than the current code, Thanks for looking! Pushed. > however, I am not sure if the handling for the > concurrently dropped tables is better (both get_rel_relkind() and > get_rel_name() ca

timezones BCE

2022-04-13 Thread Dave Cramer
Can someone help me understand select '0101-01-01'::timestamptz; timestamptz 0101-01-01 00:00:00+00 (1 row) test=# set timezone to 'America/Toronto'; SET test=# select '0101-01-01'::timestamptz; timestamptz -- 0101-01-01 00:00:

Re: Improving the "Routine Vacuuming" docs

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 8:40 AM Robert Haas wrote: > > Something along the lines of the following seems more useful: "A tuple > > whose xmin is frozen (and xmax is unset) is considered visible to > > every possible MVCC snapshot. In other words, the transaction that > > inserted the tuple is treat

Re: timezones BCE

2022-04-13 Thread chap
On 2022-04-13 12:33, Dave Cramer wrote: test=# set timezone to 'America/Toronto'; SET test=# select '0101-01-01'::timestamptz; timestamptz -- 0101-01-01 00:00:00-05:17:32 Specifically why the -05:17:32 Timezones were regularized into their (typically hour-

Re: typos

2022-04-13 Thread Alvaro Herrera
On 2022-Apr-11, David Rowley wrote: > and also skipped: > 0016 (unsure if we should change these of pgindent is not touching it) > 0017 (unsure if we should change these of pgindent is not touching it) I verified that pgindent will indeed not touch these changes by running before and after. (I a

Re: typos

2022-04-13 Thread Justin Pryzby
On Wed, Apr 13, 2022 at 07:29:34PM +0200, Alvaro Herrera wrote: > On 2022-Apr-11, David Rowley wrote: > > > and also skipped: > > 0016 (unsure if we should change these of pgindent is not touching it) > > 0017 (unsure if we should change these of pgindent is not touching it) > > I verified that p

Re: timezones BCE

2022-04-13 Thread Tom Lane
c...@anastigmatix.net writes: > On 2022-04-13 12:33, Dave Cramer wrote: >> Specifically why the -05:17:32 > Timezones were regularized into their (typically hour-wide) chunks > during a period around the late nineteenth century IIRC. > If you decompile the zoneinfo database to look at America/Tor

Re: timezones BCE

2022-04-13 Thread Dave Cramer
On Wed, 13 Apr 2022 at 14:10, Tom Lane wrote: > c...@anastigmatix.net writes: > > On 2022-04-13 12:33, Dave Cramer wrote: > >> Specifically why the -05:17:32 > > > Timezones were regularized into their (typically hour-wide) chunks > > during a period around the late nineteenth century IIRC. > > >

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Nathan Bossart
On Wed, Apr 13, 2022 at 12:05:08PM -0400, Tom Lane wrote: > Robert Haas writes: >> It may be too much to hope that we're going to completely get rid of >> process_shared_preload_libraries_in_progress tests. > > Perhaps, but this is definitely an area that could stand to have some > serious design

Re: How about a psql backslash command to show GUCs?

2022-04-13 Thread Tom Lane
"Jonathan S. Katz" writes: > On 4/12/22 1:00 PM, Tom Lane wrote: >> Yeah, most of what shows up in a minimally-configured installation is >> postmaster-computed settings like config_file, rather than things >> that were actually set by the DBA. Personally I'd rather hide the >> ones that have sou

Re: Pluggable toaster

2022-04-13 Thread Nikita Malakhov
Hi, For a pluggable toaster - in previous patch set part 7 patch file contains invalid string. Fixup (v2 file should used instead of previous) patch: 7) 0007_fix_alignment_of_custom_toast_pointers.patch - fixes custom toast pointer's alignment required by bytea toaster by Nikita Glukhov; On Wed,

Re: PG DOCS - logical replication filtering

2022-04-13 Thread Euler Taveira
On Wed, Apr 13, 2022, at 12:24 AM, Peter Smith wrote: > PSA patch v10 which addresses the remaining review comments from Euler [1] Looks good to me. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Improving the "Routine Vacuuming" docs

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 12:34 PM Peter Geoghegan wrote: > What do you think of the idea of relating freezing to removing tuples > by VACUUM at this point? This would be a basis for explaining how > freezing and tuple removal are constrained by the same cutoff. A very > old snapshot can hold up cle

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-04-13 Thread Justin Pryzby
On Wed, Mar 30, 2022 at 05:58:24PM +0900, Kyotaro Horiguchi wrote: > I'm not still confident on this, but it should be better than the v1. +Andres as this seems to be related to 277692220. I added it as an Opened Item.

SQL JSON compliance

2022-04-13 Thread Andrew Dunstan
Simon has just pointed out to me that as a result of recent commits, a number of things now should move from the unsupported table to the supported table in features.sgml. In particular, it looks to me like all of these should move: T811           Basic SQL/JSON constructor functions       T812

Re: typos

2022-04-13 Thread David Rowley
On Mon, 11 Apr 2022 at 22:10, Justin Pryzby wrote: > Thanks for amending and pushing those. There's some more less obvious ones > attached. Here are my notes from yesterday that I made when reviewing and pushing many of the 2nd batch of patches. 0001: Pushed and back patched to v12 0002: Didn'

Re: CLUSTER on partitioned index

2022-04-13 Thread Michael Paquier
On Wed, Apr 13, 2022 at 05:52:14AM -0500, Justin Pryzby wrote: > Are you sure? The ownership re-check in cluster_rel() occurs after acquiring > locks. Yep, you are right. However, the SQL test does not check for this blocking scenario. In short, removing the new ACL check in get_tables_to_clust

Re: timezones BCE

2022-04-13 Thread chap
On 2022-04-13 14:13, Dave Cramer wrote: Oh please don't do something bespoke. I'm trying to make this work with the JDBC driver. So it has to be at least compatible with other libraries. Looks like Java agrees with the offset, prior to Toronto's 1895 adoption of the hour-wide zone: jshell>

Re: Fixes for compression options of pg_receivewal and refactoring of backup_compression.{c,h}

2022-04-13 Thread Michael Paquier
On Wed, Apr 13, 2022 at 02:58:28PM +, gkokola...@pm.me wrote: > It's really not hard to add compression level. However we had briefly > discussed it in the original thread [1] and decided against. That is why > I did not write that code. If the community thinks differently now, let > me know if

Re: Improving the "Routine Vacuuming" docs

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 1:25 PM Robert Haas wrote: > On Wed, Apr 13, 2022 at 12:34 PM Peter Geoghegan wrote: > > What do you think of the idea of relating freezing to removing tuples > > by VACUUM at this point? This would be a basis for explaining how > > freezing and tuple removal are constrain

allow specifying action when standby encounters incompatible parameter settings

2022-04-13 Thread Nathan Bossart
Hi hackers, As of 15251c0, when a standby encounters an incompatible parameter change, it pauses replay so that read traffic can continue while the administrator fixes the parameters. Once the server is restarted, replay can continue. Before this change, such incompatible parameter changes caused

Re: typos

2022-04-13 Thread David Rowley
(For the future, just to make discussions easier, it would be good if you could have git format-patch -v N to give a unique version number to these patches) On Thu, 14 Apr 2022 at 05:40, Justin Pryzby wrote: > There's (only) a few remaining. I've pushed 0001 and 0002 of the 3rd batch of patches.

Re: Improving the "Routine Vacuuming" docs

2022-04-13 Thread David G. Johnston
On Wed, Apr 13, 2022 at 2:19 PM Peter Geoghegan wrote: > On Wed, Apr 13, 2022 at 1:25 PM Robert Haas wrote: > > On Wed, Apr 13, 2022 at 12:34 PM Peter Geoghegan wrote: > > > What do you think of the idea of relating freezing to removing tuples > > > by VACUUM at this point? This would be a basi

Intermittent buildfarm failures on wrasse

2022-04-13 Thread Tom Lane
For the past five days or so, wrasse has been intermittently failing due to unexpectedly not using an Index Only Scan plan in the create_index test [1], eg @@ -1910,11 +1910,15 @@ SELECT unique1 FROM tenk1 WHERE unique1 IN (1,42,7) ORDER BY unique1; - QUERY PLAN

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 3:08 PM Tom Lane wrote: > I'm tempted to add something like > > SELECT relallvisible = relpages FROM pg_class WHERE relname = 'tenk1'; > > so that we can confirm or refute the theory that relallvisible is > the driving factor. It would be fairly straightforward to commit a

Re: Temporary file access API

2022-04-13 Thread Bruce Momjian
On Mon, Apr 11, 2022 at 04:34:18PM -0400, Robert Haas wrote: > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > > > (An

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Apr 13, 2022 at 3:08 PM Tom Lane wrote: >> I'm tempted to add something like >> SELECT relallvisible = relpages FROM pg_class WHERE relname = 'tenk1'; >> so that we can confirm or refute the theory that relallvisible is >> the driving factor. > It would be fairl

Re: Temporary file access API

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > I don't think we want to be encrypting pg_xact/, so they can get the > transaction commit rate from there. I think it would be a good idea to eventually encrypt SLRU data, including pg_xact. Maybe not in the first version. -- Robert Haas ED

Re: PG DOCS - logical replication filtering

2022-04-13 Thread Peter Smith
I've changed the CF entry [1] status to "ready for committer". -- [1] https://commitfest.postgresql.org/38/3605/ Kind Regards, Peter Smith. Fujitsu Australia

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread David Rowley
On Thu, 14 Apr 2022 at 10:54, Tom Lane wrote: > After a bit more navel-contemplation I see a way that the pgstats > work could have changed timing in this area. We used to have a > rate limit on how often stats reports would be sent to the > collector, which'd ensure half a second or so delay bef

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 3:54 PM Tom Lane wrote: > After a bit more navel-contemplation I see a way that the pgstats > work could have changed timing in this area. We used to have a > rate limit on how often stats reports would be sent to the > collector, which'd ensure half a second or so delay b

Re: Temporary file access API

2022-04-13 Thread Stephen Frost
Greetings, On Wed, Apr 13, 2022 at 18:54 Robert Haas wrote: > On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > > I don't think we want to be encrypting pg_xact/, so they can get the > > transaction commit rate from there. > > I think it would be a good idea to eventually encrypt SLRU data

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Andres Freund
Hi, On April 13, 2022 7:06:33 PM EDT, David Rowley wrote: >On Thu, 14 Apr 2022 at 10:54, Tom Lane wrote: >> After a bit more navel-contemplation I see a way that the pgstats >> work could have changed timing in this area. We used to have a >> rate limit on how often stats reports would be sent

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 4:13 PM Andres Freund wrote: > IIRC the problem in matter isn't skipped pages, but that the horizon simply > isn't new enough to mark pages as all visible. Sometimes OldestXmin can go backwards in VACUUM operations that are run in close succession against the same table,

Re: Temporary file access API

2022-04-13 Thread Bruce Momjian
On Wed, Apr 13, 2022 at 06:54:13PM -0400, Robert Haas wrote: > On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > > I don't think we want to be encrypting pg_xact/, so they can get the > > transaction commit rate from there. > > I think it would be a good idea to eventually encrypt SLRU data,

Re: shared-memory based stats collector - v70

2022-04-13 Thread David G. Johnston
On Tue, Apr 5, 2022 at 8:00 PM Andres Freund wrote: > Here comes v70: > > One thing I just noticed while peeking at pg_stat_slru: The stats_reset column for my newly initdb'd cluster is showing me "2000-01-01 00:00:00" (v15). I was expecting null, though a non-null value restriction does make s

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Apr 13, 2022 at 4:13 PM Andres Freund wrote: >> IIRC the problem in matter isn't skipped pages, but that the horizon simply >> isn't new enough to mark pages as all visible. > Sometimes OldestXmin can go backwards in VACUUM operations that are > run in close su

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 4:38 PM Tom Lane wrote: > So what seems to be happening on wrasse is that a background > autovacuum (or really autoanalyze?) is preventing pages from > being marked all-visible not only during test_setup.sql but > also create_index.sql; but it's gone by the time sanity_chec

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Apr 13, 2022 at 4:38 PM Tom Lane wrote: >> It seems like a reliable fix might require test_setup to wait >> for any background autovac to exit before it does its own >> vacuums. Ick. > This is hardly a new problem, really. I wonder if it's worth inventing > a c

Re: shared-memory based stats collector - v70

2022-04-13 Thread David G. Johnston
On Wed, Apr 13, 2022 at 4:34 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Tue, Apr 5, 2022 at 8:00 PM Andres Freund wrote: > >> Here comes v70: >> >> > One thing I just noticed while peeking at pg_stat_slru: > > The stats_reset column for my newly initdb'd cluster is showing me

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 4:51 PM Tom Lane wrote: > Yeah, we have band-aided around this type of problem repeatedly. > Making a fix that's readily accessible from any test script > seems like a good idea. We might even be able to consistently rely on this new option, given *any* problem involving t

VPath Build Errors

2022-04-13 Thread David G. Johnston
Hey, I've been building in the git repo just fine but wanted to use vpath builds so I could keep both "maked" v14 and v15 binaries around, ready to be installed. The attached log is result of (while in the versioned directory, a sibling of the git repo) `../postgresql/configure` `make` `tree` st

Re: typos

2022-04-13 Thread Justin Pryzby
On Thu, Apr 14, 2022 at 09:39:42AM +1200, David Rowley wrote: > On Thu, 14 Apr 2022 at 05:40, Justin Pryzby wrote: > > There's (only) a few remaining. > > I've pushed 0001 and 0002 of the 3rd batch of patches. I left 0003 as Thanks > I just didn't feel it was a meaningful enough improvement. >

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 16:45:44 -0700, Peter Geoghegan wrote: > On Wed, Apr 13, 2022 at 4:38 PM Tom Lane wrote: > > So what seems to be happening on wrasse is that a background > > autovacuum (or really autoanalyze?) is preventing pages from > > being marked all-visible not only during test_setup.sql

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Apr 13, 2022 at 4:51 PM Tom Lane wrote: >> Yeah, we have band-aided around this type of problem repeatedly. >> Making a fix that's readily accessible from any test script >> seems like a good idea. > We might even be able to consistently rely on this new option,

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 15:28:19 -0500, Justin Pryzby wrote: > On Wed, Mar 30, 2022 at 05:58:24PM +0900, Kyotaro Horiguchi wrote: > > I'm not still confident on this, but it should be better than the v1. > > +Andres as this seems to be related to 277692220. FWIW, that'd just mean it's an old bug that

Re: VPath Build Errors

2022-04-13 Thread Tom Lane
"David G. Johnston" writes: > The attached log is result of (while in the versioned directory, a sibling > of the git repo) > `../postgresql/configure` > `make` > `tree` The VPATH buildfarm members haven't been complaining, and I can't reproduce a failure here, so I'm inclined to suspect pilot er

Re: VPath Build Errors

2022-04-13 Thread David G. Johnston
On Wed, Apr 13, 2022 at 5:44 PM Tom Lane wrote: > "David G. Johnston" writes: > > The attached log is result of (while in the versioned directory, a > sibling > > of the git repo) > > `../postgresql/configure` > > `make` > > `tree` > > The VPATH buildfarm members haven't been complaining, and I

Re: shared-memory based stats collector - v70

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 16:56:45 -0700, David G. Johnston wrote: > On Wed, Apr 13, 2022 at 4:34 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: > Sorry, apparently this "2000-01-01" behavior only manifests after crash > recovery on v15 (didn't check v14); after a clean initdb on v15 I got th

RE: PG DOCS - logical replication filtering

2022-04-13 Thread houzj.f...@fujitsu.com
On Wednesday, April 13, 2022 11:25 AM Peter Smith wrote: > > PSA patch v10 which addresses the remaining review comments from Euler [1] Thanks for the patch, it looks good to me. Best regards, Hou zj

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 18:54:06 -0400, Tom Lane wrote: > We used to have a rate limit on how often stats reports would be sent > to the collector, which'd ensure half a second or so delay before a > transaction's change counts became visible to the autovac daemon. Just for posterity: That's not actual

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2022 at 5:35 PM Tom Lane wrote: > My guess is that you'd need both this new wait-for-horizon behavior > *and* DISABLE_PAGE_SKIPPING. But the two together ought to make > for pretty reproducible behavior. I noticed while scanning the > commit log that some patches have tried addin

Re: Intermittent buildfarm failures on wrasse

2022-04-13 Thread Andres Freund
Hi, On 2022-04-13 20:35:50 -0400, Tom Lane wrote: > Peter Geoghegan writes: > > On Wed, Apr 13, 2022 at 4:51 PM Tom Lane wrote: > >> Yeah, we have band-aided around this type of problem repeatedly. > >> Making a fix that's readily accessible from any test script > >> seems like a good idea. > >

  1   2   >