Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-06 Thread Peter Smith
Hi Kuroda-san. This is a WIP review. I'm yet to do more testing and more study of the POC patch's design. While reading the code I kept a local list of my review comments. Meanwhile, there is a long weekend coming up here, so I thought it would be better to pass these to you now rather than next

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2023-04-06 Thread Etsuro Fujita
On Tue, Apr 4, 2023 at 7:28 PM Etsuro Fujita wrote: > The parallel-abort patch received a review from David, and I addressed > his comments. Also, he tested with the patch, and showed that it > reduces time taken to abort remote transactions. So, if there are no > objections, I will commit the p

Re: doc: add missing "id" attributes to extension packaging page

2023-04-06 Thread Peter Eisentraut
On 04.04.23 21:52, Brar Piening wrote: The XSLT implementation looks sound to me.  It would be a touch better if it had some comments about which parts of the templates were copied from upstream stylesheets and which were changed.  There are examples of such commenting in the existing customizati

Re: meson documentation build open issues

2023-04-06 Thread Peter Eisentraut
On 05.04.23 16:45, Andres Freund wrote: I think it's still an issue that "make docs" builds html and man but "ninja docs" only builds html. For some reason the wiki page actually claims that ninja docs builds both, but this does not happen for me. It used to, but Tom insisted that it should no

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Thu, Apr 6, 2023 at 12:10 PM Amit Kapila wrote: > > On Wed, Apr 5, 2023 at 9:27 PM Drouvot, Bertrand > wrote: > > > > Another comment on 0001. > extern void CheckSlotRequirements(void); > extern void CheckSlotPermissions(void); > +extern void ResolveRecoveryConflictWithLogicalSlots(Oid dboid

Re: [BUG] Logical replica crash if there was an error in a function.

2023-04-06 Thread Anton A. Melnikov
On 05.04.2023 17:35, Tom Lane wrote: "Anton A. Melnikov" writes: On 03.04.2023 21:49, Tom Lane wrote: I did not think this case was worth memorializing in a test before, and I still do not. I'm inclined to reject this patch. Could you help me to figure out, please. The problem was an Ass

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread David Rowley
second On Thu, 6 Apr 2023 at 14:14, Melanie Plageman wrote: > > Attached is v14 which adds back in tests for the BUFFER_USAGE_LIMIT > option. I've spent quite a bit of time looking at this since you sent it. I've also made quite a few changes, mostly cosmetic ones, but there are a few things bel

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Thu, Apr 6, 2023 at 11:29 AM Amit Kapila wrote: > > > > > This doesn't seem to be addressed in the latest version. And today, I > think I see one more point about this doc change: > + > + A logical replication slot can also be created on a hot standby. > To prevent > + VACUUM from r

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 08:39, Masahiko Sawada wrote: > Also I agree with > where to put the log but I think the log message should start with > lower cases: > > +elog(DEBUG2, > + "Autovacuum VacuumUpdateCosts(db=%u, rel=%u, In principle I agree, but in this

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 8:40 AM, Amit Kapila wrote: On Wed, Apr 5, 2023 at 9:27 PM Drouvot, Bertrand After this, I think for backends that have active slots, it would simply cancel the current query. Will that be sufficient? Because we want the backend process should exit and release the slot so that the

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 11:55 AM, Amit Kapila wrote: On Thu, Apr 6, 2023 at 12:10 PM Amit Kapila wrote: On Wed, Apr 5, 2023 at 9:27 PM Drouvot, Bertrand wrote: Another comment on 0001. extern void CheckSlotRequirements(void); extern void CheckSlotPermissions(void); +extern void ResolveRecovery

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 7:59 AM, Amit Kapila wrote: On Wed, Apr 5, 2023 at 6:14 PM Drouvot, Bertrand wrote: On 4/5/23 12:28 PM, Amit Kapila wrote: On Wed, Apr 5, 2023 at 2:41 PM Drouvot, Bertrand wrote: Maybe we could change the doc with something among those lines instead? " Existing logical slots

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 2:23 PM, Amit Kapila wrote: On Thu, Apr 6, 2023 at 11:29 AM Amit Kapila wrote: Thinking some more on this, I think such a slot won't decode any other records. During CreateInitDecodingContext->ReplicationSlotReserveWal, for standby's, we use lastReplayedEndRecPtr as restart_lsn.

Re: Initial Schema Sync for Logical Replication

2023-04-06 Thread Masahiko Sawada
On Thu, Mar 30, 2023 at 10:11 PM Masahiko Sawada wrote: > > On Thu, Mar 30, 2023 at 12:18 AM Masahiko Sawada > wrote: > > > > On Wed, Mar 29, 2023 at 7:57 PM Kumar, Sachin wrote: > > > > > > > > > > From: Amit Kapila > > > > > > > > I think we won't be able to use same snapshot because the > >

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Thu, Apr 6, 2023 at 6:32 PM Drouvot, Bertrand wrote: > > Hi, > > On 4/6/23 11:55 AM, Amit Kapila wrote: > > On Thu, Apr 6, 2023 at 12:10 PM Amit Kapila wrote: > >> > >> On Wed, Apr 5, 2023 at 9:27 PM Drouvot, Bertrand > >> wrote: > >>> > >> > >> Another comment on 0001. > >> extern void Che

Re: doc: add missing "id" attributes to extension packaging page

2023-04-06 Thread Brar Piening
On 06.04.2023 at 11:06, Peter Eisentraut wrote: On 04.04.23 21:52, Brar Piening wrote: The XSLT implementation looks sound to me.  It would be a touch better if it had some comments about which parts of the templates were copied from upstream stylesheets and which were changed.  There are exampl

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 3:39 PM, Amit Kapila wrote: On Thu, Apr 6, 2023 at 6:32 PM Drouvot, Bertrand wrote: I don't think it could be possible to create logical walsenders on a standby if AllowCascadeReplication() is not true, or am I missing something? Right, so why to even traverse walsenders for

Re: doc: add missing "id" attributes to extension packaging page

2023-04-06 Thread Karl O. Pinc
On Thu, 6 Apr 2023 16:19:30 +0200 Brar Piening wrote: > Reviewer is Karl O. Pink "Karl O. Pinc" actually, with a "c". Regards, Karl Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

Re: Add index scan progress to pg_stat_progress_vacuum

2023-04-06 Thread Imseih (AWS), Sami
> As one thing, > for example, it introduces a dependency to parallel.h to do progress > reporting without touching at backend_progress.h. Containing the logic in backend_progress.h is a reasonable point from a maintenance standpoint. We can create a new function in backend_progress.h called p

Re: zstd compression for pg_dump

2023-04-06 Thread Tomas Vondra
On 4/5/23 21:42, Tomas Vondra wrote: > On 4/4/23 05:04, Justin Pryzby wrote: >> On Mon, Apr 03, 2023 at 11:26:09PM +0200, Tomas Vondra wrote: >>> On 4/3/23 21:17, Justin Pryzby wrote: On Sat, Apr 01, 2023 at 10:26:01PM +0200, Tomas Vondra wrote: >> Feel free to mess around with threads

Re: Schema variables - new implementation for Postgres 15

2023-04-06 Thread Julien Rouhaud
On Thu, Apr 6, 2023 at 1:58 AM Pavel Stehule wrote: > > st 5. 4. 2023 v 19:20 odesílatel Greg Stark napsal: >> >> On Sun, 26 Mar 2023 at 07:34, Julien Rouhaud wrote: >> > >> > This feature can significantly increase log size, so it's disabled by >> > default. >> > For testing or development env

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
On Wed, Apr 5, 2023 at 11:10 PM Melanie Plageman wrote: > On Wed, Apr 5, 2023 at 3:43 PM Melanie Plageman > wrote: > > On Wed, Apr 5, 2023 at 2:56 PM Robert Haas wrote: > > > > > > + /* > > > + * Balance and update limit values for autovacuum workers. We must > > > + * always do this in case th

Re: zstd compression for pg_dump

2023-04-06 Thread Justin Pryzby
On Thu, Apr 06, 2023 at 05:34:30PM +0200, Tomas Vondra wrote: > I looked at the long mode patch again, updated the commit message and > pushed it. I was wondering if long_mode should really be bool - > logically it is, but ZSTD_CCtx_setParameter() expects int. But I think > that's fine. Thanks! >

Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)

2023-04-06 Thread Justin Pryzby
On Tue, Mar 14, 2023 at 12:16:16AM +0100, Tomas Vondra wrote: > On 3/9/23 19:00, Tomas Vondra wrote: > > On 3/9/23 01:30, Michael Paquier wrote: > >> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote: > >>> IMO we should fix that. We have a bunch of buildfarm members running on > >>> Ubu

Re: Rethinking the implementation of ts_headline()

2023-04-06 Thread Alexander Lakhin
Hi, 19.01.2023 19:13, Tom Lane wrote: Alvaro Herrera writes: Anyway, I don't think this needs to stop your current patch. Many thanks for looking at it! I've found that starting from commit 5a617d75 this query: SELECT ts_headline('english', 'To be, or not to be', to_tsquery('english', 'or

Re: Schema variables - new implementation for Postgres 15

2023-04-06 Thread Kirk Wolak
On Wed, Apr 5, 2023 at 1:58 PM Pavel Stehule wrote: > > > st 5. 4. 2023 v 19:20 odesílatel Greg Stark napsal: > >> On Sun, 26 Mar 2023 at 07:34, Julien Rouhaud wrote: >> > >> > This feature can significantly increase log size, so it's disabled by >> default. >> > For testing or development envi

Re: Should vacuum process config file reload more often

2023-04-06 Thread Robert Haas
On Thu, Apr 6, 2023 at 11:52 AM Melanie Plageman wrote: > > Gah, I think I misunderstood you. You are saying that only calling > > AutoVacuumUpdateCostLimit() after napping while vacuuming a table may > > not be enough. The frequency at which the number of workers changes will > > likely be differ

Re: monitoring usage count distribution

2023-04-06 Thread Robert Haas
On Wed, Apr 5, 2023 at 4:16 PM Tom Lane wrote: > Nathan Bossart writes: > > On Wed, Apr 05, 2023 at 12:09:21PM -0700, Andres Freund wrote: > >> I would not mind having a separate function returning 6 rows, if we really > >> want that, but making pg_buffercache_summary() return 6 rows would imo >

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread Melanie Plageman
On Thu, Apr 06, 2023 at 11:34:44PM +1200, David Rowley wrote: > second On Thu, 6 Apr 2023 at 14:14, Melanie Plageman > wrote: > > > > Attached is v14 which adds back in tests for the BUFFER_USAGE_LIMIT > > option. > > I've spent quite a bit of time looking at this since you sent it. I've > also

Re: psql \watch 2nd argument: iteration count

2023-04-06 Thread Tom Lane
Kirk Wolak writes: > Marked as Ready for Committer. Pushed with a pretty fair number of cosmetic changes. One non-cosmetic change I made is that I didn't agree with your interpretation of the execution count. IMO this ought to produce three executions: regression=# select 1 \watch c=3 Thu Apr

Re: Schema variables - new implementation for Postgres 15

2023-04-06 Thread Pavel Stehule
> > >> example >> >> create variable a as int; >> create table foo(a int); >> >> select a from foo; -- the "a" is ambiguous, variable "a" is shadowed >> >> This is a basic case, and the unique names don't help. The variables are >> more aggressive in namespace than tables, because they don't requir

Re: monitoring usage count distribution

2023-04-06 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 5, 2023 at 4:16 PM Tom Lane wrote: >> Having two functions doesn't seem unreasonable to me either. >> Robert spoke against it to start with, does he still want to >> advocate for that? > My position is that if we replace the average usage count with > something

Re: Should vacuum process config file reload more often

2023-04-06 Thread Robert Haas
On Wed, Apr 5, 2023 at 4:59 PM Peter Geoghegan wrote: > I think that I agree. I think that the difficulty of tuning autovacuum > is the actual real problem. (Or maybe it's just very closely related > to the real problem -- the precise definition doesn't seem important.) I agree, and I think that

Re: psql \watch 2nd argument: iteration count

2023-04-06 Thread Andrey Borodin
On Thu, Apr 6, 2023 at 10:23 PM Tom Lane wrote: > > Kirk Wolak writes: > > Marked as Ready for Committer. > > Pushed with a pretty fair number of cosmetic changes. Great, thank you! > If you write a semicolon first, you get four, but it's the semicolon > producing the first result not \watch.

Re: monitoring usage count distribution

2023-04-06 Thread Nathan Bossart
On Thu, Apr 06, 2023 at 01:32:35PM -0400, Tom Lane wrote: > There seems to be enough support for the existing summary function > definition to leave it as-is; Andres likes it for one, and I'm not > excited about trying to persuade him he's wrong. But a second > slightly-less-aggregated summary fun

Re: cataloguing NOT NULL constraints

2023-04-06 Thread Justin Pryzby
On Thu, Apr 06, 2023 at 01:33:56AM +0200, Alvaro Herrera wrote: > - The forms ADD (without USING INDEX), > + The forms ADD (without USING INDEX, > and > + except for the NOT NULL > column_name > + form to add a table constraint), The "except" part seems pretty incoherent to me :( > +

Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 18:39, Justin Pryzby wrote: > *If* you wanted to do something to fix this, you could create a key > called files_to_remove_after_loading, and run unlink on those files > rather than running a shell command. Or maybe just remove the file > unconditionally at the start of the s

Re: what should install-world do when docs are not available?

2023-04-06 Thread Andrew Dunstan
On 2023-04-05 We 00:57, Andres Freund wrote: Hi, On 2023-04-04 21:46:11 -0700, Andres Freund wrote: Pushed the changes. This failed on crake - afaict because the meson buildfarm code disables all features. Because 'docs' is a feature now, the BF code building doc/src/sgml/html fails. I cha

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 19:18, Robert Haas wrote: > > On Thu, Apr 6, 2023 at 11:52 AM Melanie Plageman > wrote: >>> Gah, I think I misunderstood you. You are saying that only calling >>> AutoVacuumUpdateCostLimit() after napping while vacuuming a table may >>> not be enough. The frequency at which t

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
v17 attached does not yet fix the logging problem or variable naming problem. I have not changed where AutoVacuumUpdateCostLimit() is called either. This is effectively just a round of cleanup. I hope I have managed to address all other code review feedback so far, though some may have slipped th

Re: Rethinking the implementation of ts_headline()

2023-04-06 Thread Tom Lane
Alexander Lakhin writes: > I've found that starting from commit 5a617d75 this query: > SELECT ts_headline('english', 'To be, or not to be', to_tsquery('english', > 'or')); > invokes a valgrind-detected error: > ==00:00:00:03.950 3241424== Invalid read of size 1 On my machine, I also see PG-repor

Git sources doesn't contain the INSATLL file?

2023-04-06 Thread tison
Hi Hackers, I opened https://git.postgresql.org/gitweb/?p=postgresql.git;a=tree and clone the repo. The README file says that I can read the INSTALL file to understand how to build from source. But there is no such file in Git sources. Is it expected? If so, why? Best, tison.

Re: Git sources doesn't contain the INSATLL file?

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 22:13, tison wrote: > > Hi Hackers, > > I opened https://git.postgresql.org/gitweb/?p=postgresql.git;a=tree and clone > the repo. The README file says that I can read the INSTALL file to understand > how to build from source. But there is no such file in Git sources. Is it

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
I think attached v18 addresses all outstanding issues except a run through the docs making sure all mentions of the balancing algorithm are still correct. On Wed, Apr 5, 2023 at 9:10 AM Daniel Gustafsson wrote: > > On 4 Apr 2023, at 22:04, Melanie Plageman wrote: > >> +extern int VacuumCostLimit

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread David Rowley
On Fri, 7 Apr 2023 at 05:20, Melanie Plageman wrote: > > diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml > This still says that the default value is -1. Oops, I had this staged but didn't commit and formed the patch with "git diff master.." instead of "git diff master", so miss

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread Justin Pryzby
+VACUUM uses a ring like sequential scans, however, the size this ring +controlled by the vacuum_buffer_usage_limit GUC. Dirty pages are not removed should say: ".. the size OF this ring IS .." ?

Re: PostgreSQL 16 Release Management Team & Feature Freeze

2023-04-06 Thread Jonathan S. Katz
On 3/21/23 11:35 AM, Jonathan S. Katz wrote: Additionally, the RMT has set the feature freeze to be **April 8, 2023 at 0:00 AoE**[1]. This is the last time to commit features for PostgreSQL 16. In  other words, no new PostgreSQL 16 feature can be committed after April 8, 2023 at 0:00 AoE. Th

Re: Add SHELL_EXIT_CODE to psql

2023-04-06 Thread Tom Lane
Corey Huinker writes: > This is a follow up patch to apply the committed pattern to the various > piped output commands. Pushed with some changes: * You didn't update the documentation. * I thought this was way too many copies of the same logic. I made a subroutine to set these variables, an

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread Melanie Plageman
On Fri, Apr 07, 2023 at 09:12:32AM +1200, David Rowley wrote: > On Fri, 7 Apr 2023 at 05:20, Melanie Plageman > wrote: > > GUC name mentioned in comment is inconsistent with current GUC name. > > > > > +/* > > > + * Upper and lower hard limits for the buffer access strategy ring size > > > + * sp

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 6 Apr 2023, at 23:06, Melanie Plageman wrote: > Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost > limit or cost delay have been changed. If they have, they assert that > they don't already hold the AutovacuumLock, take it in shared mode, and > do the logging. Another id

Re: Should vacuum process config file reload more often

2023-04-06 Thread Melanie Plageman
On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: > > > On 6 Apr 2023, at 23:06, Melanie Plageman wrote: > > > Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost > > limit or cost delay have been changed. If they have, they assert that > > they don't already hold the Autov

Re: Temporary tables versus wraparound... again

2023-04-06 Thread Greg Stark
On Wed, 5 Apr 2023 at 13:42, Andres Freund wrote: > > Not if you determine a relation specific xmin, and the relation is not a > shared relation. > > ISTM that the problem here really is that you're relying on RecentXmin, rather > than computing something more accurate. Why not use > GetOldestNonR

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-06 Thread Yurii Rashkovskii
Hi Tom, On Wed, Mar 29, 2023 at 6:55 PM Tom Lane wrote: > Yurii Rashkovskii writes: > > I would like to suggest a patch against master (although it may be worth > > backporting it) that makes it possible to listen on any unused port. > > I think this is a bad idea, mainly because this: > > > In

Re: Improve logging when using Huge Pages

2023-04-06 Thread Justin Pryzby
On Tue, Mar 28, 2023 at 09:35:30AM +0900, Michael Paquier wrote: > On Thu, Mar 23, 2023 at 08:50:50PM -0500, Justin Pryzby wrote: > > You mean that you abused of it in some custom branch running the CI on > github? If I may ask, what did you do to make sure that huge pages > are set when re-attac

Re: cursor use vs pg_stat_statements

2023-04-06 Thread Michael Paquier
On Wed, Oct 20, 2021 at 09:53:38AM -0400, Andrew Dunstan wrote: > Try again with autocommit turned off. Sorry, I omitted that crucial > detail. Exact test code attached (name/password removed) For the same of the archives, this should be OK now under 1d477a9. See also this thread: https://www.post

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-04-06 Thread reid . thompson
Updated patches attached. Rebased to current master. Added additional columns to pg_stat_global_memory_allocation to summarize backend allocations by type. Updated documentation. Corrected some issues noted in review by John Morris. Added code re EXEC_BACKEND for dev-max-memory branch. From 3451

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-06 Thread Thomas Munro
On Thu, Apr 6, 2023 at 6:40 PM Richard Guo wrote: > Seems it wins if the parallel scan becomes part of a hash join in final > plan. I wonder if we have a way to know that in this early stage. I haven't tried but I'm not sure off the top of my head how to make a decision that early unless it's su

Re: Should vacuum process config file reload more often

2023-04-06 Thread Daniel Gustafsson
> On 7 Apr 2023, at 00:12, Melanie Plageman wrote: > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: >> >>> On 6 Apr 2023, at 23:06, Melanie Plageman wrote: >> >>> Autovacuum workers, at the end of VacuumUpdateCosts(), check if cost >>> limit or cost delay have been changed. If they

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-06 Thread Michael Paquier
On Thu, Apr 06, 2023 at 10:54:43AM +0900, Michael Paquier wrote: > 0002 can also be done before 0001, so I'd like to get that part > applied on HEAD before the feature freeze and close this thread. If > there are any objections, please feel free.. I was doing a pre-commit review of the patch, and

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-06 Thread Michael Paquier
On Fri, Apr 07, 2023 at 08:08:34AM +0900, Michael Paquier wrote: > So bumping mainrdata_len to uint64 is actually not entirely in line > with this code. Well, it will work because we'd still fail a couple > of lines down, but perhaps its readability should be improved so as > we have an extra chec

Re: pg_recvlogical prints bogus error when interrupted

2023-04-06 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Hello The patch applies and tests fine. I like the way to have b

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-06 Thread Matthias van de Meent
On Fri, 7 Apr 2023, 01:35 Michael Paquier, wrote: > On Fri, Apr 07, 2023 at 08:08:34AM +0900, Michael Paquier wrote: > > So bumping mainrdata_len to uint64 is actually not entirely in line > > with this code. Well, it will work because we'd still fail a couple > > of lines down, but perhaps its

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-06 Thread Michael Paquier
On Fri, Apr 07, 2023 at 01:50:00AM +0200, Matthias van de Meent wrote: > Yes, that was a bad oversight, which would've shown up in tests on a system > with an endianness that my computer doesn't have... I don't think that we have many bigendian animals in the buildfarm, either.. > That looks fin

Re: what should install-world do when docs are not available?

2023-04-06 Thread Andres Freund
Hi, On 2023-04-06 14:52:51 -0400, Andrew Dunstan wrote: > On 2023-04-05 We 00:57, Andres Freund wrote: > > Hi, > > > > On 2023-04-04 21:46:11 -0700, Andres Freund wrote: > > > Pushed the changes. > > This failed on crake - afaict because the meson buildfarm code disables all > > features. Because

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-04-06 Thread David Rowley
On Fri, 7 Apr 2023 at 09:44, Melanie Plageman wrote: > Otherwise, LGTM. Thanks for looking. I've also taken Justin's comments about the README into account and fixed that part. I've pushed the patch after a little more adjusting. I added some text to the docs that mention larger VACUUM_BUFFER_

Re: hio.c does visibilitymap_pin()/IO while holding buffer lock

2023-04-06 Thread Andres Freund
Hi, On 2023-04-03 12:00:30 -0700, Andres Freund wrote: > It's not great, I agree. I tried to make it easier to read in this version by > a) changing GetVisibilityMapPins() as I proposed > b) added a new variable "recheckVmPins", that gets set in >if (unlockedTargetBuffer) >and >if (oth

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-04-06 Thread Andres Freund
Hi, On 2023-04-05 18:46:16 -0700, Andres Freund wrote: > On 2023-04-04 17:39:45 -0700, Andres Freund wrote: > > After that I'm planning to wait for a buildfarm cycle, and push the changes > > necessary to use bulk extension in hio.c (the main win). > > Working on that. Might end up being tomorrow

Re: Minimal logical decoding on standbys

2023-04-06 Thread Andres Freund
Hi, On 2023-04-06 12:10:57 +0530, Amit Kapila wrote: > After this, I think for backends that have active slots, it would > simply cancel the current query. Will that be sufficient? Because we > want the backend process should exit and release the slot so that the > startup process can mark it inva

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Fri, Apr 7, 2023 at 6:55 AM Andres Freund wrote: > > On 2023-04-06 12:10:57 +0530, Amit Kapila wrote: > > After this, I think for backends that have active slots, it would > > simply cancel the current query. Will that be sufficient? Because we > > want the backend process should exit and relea

Re: Partial aggregates pushdown

2023-04-06 Thread Bruce Momjian
On Fri, Mar 31, 2023 at 05:49:21AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > Hi Mr.Momjian > > > First, am I correct? > Yes, you are correct. This patch uses new special aggregate functions for > partial aggregate > (then we call this partialaggfunc). First, my apologies for not ad

Re: zstd compression for pg_dump

2023-04-06 Thread Tom Lane
Justin Pryzby writes: > On Thu, Apr 06, 2023 at 05:34:30PM +0200, Tomas Vondra wrote: >> I think that's all for PG16 in this patch series. If there's more we want to >> do, it'll have to wait for PG17 - > Yes Shouldn't the CF entry be closed as committed? It's certainly making the cfbot unhap

Re: Minimal logical decoding on standbys

2023-04-06 Thread Andres Freund
Hi, TBH, I don't like the state of 0001 much. I'm working on polishing it now. A lot of the new functions in slot.h don't seem right to me: - ObsoleteSlotIsInvalid() - isn't an obsolete slot by definition invalid? - Why does ObsoleteSlotIsInvalid() sometime check invalidated_at and sometimes no

Re: pg_upgrade and logical replication

2023-04-06 Thread Julien Rouhaud
Hi, On Thu, Apr 06, 2023 at 04:49:59AM +, Hayato Kuroda (Fujitsu) wrote: > Dear Julien, > > > I'm attaching a v3 to fix a recent conflict with pg_dump due to > > a563c24c9574b7 > > (Allow pg_dump to include/exclude child tables automatically). > > Thank you for making the patch. > FYI - it co

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-06 Thread Julien Rouhaud
Hi, On Tue, Apr 04, 2023 at 07:00:01AM +, Hayato Kuroda (Fujitsu) wrote: > Dear hackers, > (CC: Amit and Julien) (thanks for the Cc) > This is a fork thread of Julien's thread, which allows to upgrade subscribers > without losing changes [1]. > > I briefly implemented a prototype for allowin

Re: Schema variables - new implementation for Postgres 15

2023-04-06 Thread Kirk Wolak
On Thu, Mar 30, 2023 at 4:06 AM Pavel Stehule wrote: > Hi > > ne 26. 3. 2023 v 19:44 odesílatel Dmitry Dolgov <9erthali...@gmail.com> > napsal: > >> > On Fri, Mar 24, 2023 at 08:04:08AM +0100, Pavel Stehule wrote: >> > čt 23. 3. 2023 v 19:54 odesílatel Pavel Stehule < >> pavel.steh...@gmail.com>

Re: Minimal logical decoding on standbys

2023-04-06 Thread Andres Freund
Hi, On 2023-04-06 12:10:57 +0530, Amit Kapila wrote: > Also, it seems you have removed the checks related to > slots, is it because PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT is only > used for logical slots? If so, do you think an Assert would make > sense? The asserts that have been added aren't cor

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Fri, Apr 7, 2023 at 8:43 AM Andres Freund wrote: > > On 2023-04-06 12:10:57 +0530, Amit Kapila wrote: > > Also, it seems you have removed the checks related to > > slots, is it because PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT is only > > used for logical slots? If so, do you think an Assert would

Re: Minimal logical decoding on standbys

2023-04-06 Thread Amit Kapila
On Thu, Apr 6, 2023 at 7:50 PM Drouvot, Bertrand wrote: > > Thanks! Will update 0005. > I noticed a few typos in the latest patches. 0004 1. + * Physical walsenders don't need to be wakon up during replay unless Typo. 0005 2. +# Check if all the slots on standby are dropped. These include the

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
On 4/7/23 3:59 AM, Amit Kapila wrote: On Fri, Apr 7, 2023 at 6:55 AM Andres Freund wrote: On 2023-04-06 12:10:57 +0530, Amit Kapila wrote: After this, I think for backends that have active slots, it would simply cancel the current query. Will that be sufficient? Because we want the backend pr

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/7/23 4:18 AM, Andres Freund wrote: Hi, TBH, I don't like the state of 0001 much. I'm working on polishing it now. Thanks Andres! A lot of the new functions in slot.h don't seem right to me: - ObsoleteSlotIsInvalid() - isn't an obsolete slot by definition invalid? bad naming, agr

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/6/23 4:20 PM, Drouvot, Bertrand wrote: Hi, On 4/6/23 3:39 PM, Amit Kapila wrote: On Thu, Apr 6, 2023 at 6:32 PM Drouvot, Bertrand wrote: I don't think it could be possible to create logical walsenders on a standby if AllowCascadeReplication() is not true, or am I missing something

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/7/23 5:47 AM, Amit Kapila wrote: On Thu, Apr 6, 2023 at 7:50 PM Drouvot, Bertrand wrote: Thanks! Will update 0005. I noticed a few typos in the latest patches. 0004 1. + * Physical walsenders don't need to be wakon up during replay unless Typo. Thanks! Fixed in V63 just posted

Re: CREATE SUBSCRIPTION -- add missing tab-completes

2023-04-06 Thread Amit Kapila
On Wed, Apr 5, 2023 at 5:58 AM Peter Smith wrote: > > There are some recent comment that added new options for CREATE SUBSCRIPTION > ... > PSA patches to add those tab completions. > LGTM, so pushed. BTW, while looking at this, I noticed that newly added options "password_required" and "run_as_ow

Re: Minimal logical decoding on standbys

2023-04-06 Thread Andres Freund
Hi, On 2023-04-07 07:02:04 +0200, Drouvot, Bertrand wrote: > Done in V63 attached and did change the associated comment a bit. Can you send your changes incrementally, relative to V62? I'm polishing them right now, and that'd make it a lot easier to apply your changes ontop. Greetings, Andres F

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-06 Thread Michael Paquier
On Fri, Apr 07, 2023 at 08:59:22AM +0900, Michael Paquier wrote: > Okay, cool! Done this one with 8fcb32d. -- Michael signature.asc Description: PGP signature

Re: Minimal logical decoding on standbys

2023-04-06 Thread Drouvot, Bertrand
Hi, On 4/7/23 7:56 AM, Andres Freund wrote: Hi, On 2023-04-07 07:02:04 +0200, Drouvot, Bertrand wrote: Done in V63 attached and did change the associated comment a bit. Can you send your changes incrementally, relative to V62? I'm polishing them right now, and that'd make it a lot easier to

Re: Should vacuum process config file reload more often

2023-04-06 Thread Masahiko Sawada
On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > On 7 Apr 2023, at 00:12, Melanie Plageman wrote: > > > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsson wrote: > >> > >>> On 6 Apr 2023, at 23:06, Melanie Plageman > >>> wrote: > >> > >>> Autovacuum workers, at the end of VacuumUp