Re: improving user.c error messages

2023-01-26 Thread Nathan Bossart
On Thu, Jan 26, 2023 at 05:41:32PM -0500, Tom Lane wrote: > Well, it's not a hint. I think the above is fine for non-password > cases, but for passwords maybe > > ERROR: permission denied to alter role password > DETAIL: To change another role's password, you must have CREATEROLE >

Re: Something is wrong with wal_compression

2023-01-26 Thread Andrey Borodin
On Thu, Jan 26, 2023 at 3:04 PM Tom Lane wrote: > > Indeed, it seems like this behavior makes pg_xact_status() basically > useless as things stand. > If we agree that xid allocation is not something persistent, let's fix the test? We can replace a check with select * from pg_class or, maybe, add

Re: Partition key causes problem for volatile target list query

2023-01-26 Thread Tom Lane
Bruce Momjian writes: > I have found an odd behavior --- a query in the target list that assigns > to a partitioned column causes queries that would normally be volatile > to return always zero. Well, if you looked further than the first few rows, it wouldn't be "always zero". But the select fro

Re: Something is wrong with wal_compression

2023-01-26 Thread Tom Lane
Andrey Borodin writes: > On Thu, Jan 26, 2023 at 3:04 PM Tom Lane wrote: >> Indeed, it seems like this behavior makes pg_xact_status() basically >> useless as things stand. > If we agree that xid allocation is not something persistent, let's fix > the test? If we're not going to fix this behavi

Re: Partition key causes problem for volatile target list query

2023-01-26 Thread Bruce Momjian
On Thu, Jan 26, 2023 at 07:21:16PM -0500, Tom Lane wrote: > Well, if you looked further than the first few rows, it wouldn't be > "always zero". But the select from the partitioned table will read > the first partition first, and that partition will have the rows > with d1=0, by definition. > > =

Re: Something is wrong with wal_compression

2023-01-26 Thread Michael Paquier
On Thu, Jan 26, 2023 at 04:14:57PM -0800, Andrey Borodin wrote: > If we agree that xid allocation is not something persistent, let's fix > the test? We can replace a check with select * from pg_class or, > maybe, add an amcheck run. > As far as I recollect, this test was introduced to test this new

Re: Record queryid when auto_explain.log_verbose is on

2023-01-26 Thread Michael Paquier
On Thu, Jan 26, 2023 at 10:00:04PM +0900, torikoshia wrote: > I'll work on this next. Cool, thanks! -- Michael signature.asc Description: PGP signature

Re: Something is wrong with wal_compression

2023-01-26 Thread Thomas Munro
On Fri, Jan 27, 2023 at 1:30 PM Michael Paquier wrote: > On Thu, Jan 26, 2023 at 04:14:57PM -0800, Andrey Borodin wrote: > > If we agree that xid allocation is not something persistent, let's fix > > the test? We can replace a check with select * from pg_class or, > > maybe, add an amcheck run. >

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-26 Thread Andres Freund
Hi, On 2023-01-26 14:27:53 -0500, Robert Haas wrote: > One idea that I've had about how to solve this problem is to try to > make vacuum try to aggressively freeze some portion of the table on > each pass, and to behave less aggressively on the rest of the table so > that, hopefully, no single vac

Re: Add LZ4 compression in pg_dump

2023-01-26 Thread Michael Paquier
On Thu, Jan 26, 2023 at 12:22:45PM -0600, Justin Pryzby wrote: > Yeah. But the original log_warning text was better, and should be > restored: > > - if (AH->compression != 0) > - pg_log_warning("archive is compressed, but this installation > does not support compression -- no

Re: wrong Append/MergeAppend elision?

2023-01-26 Thread Amit Langote
On Fri, Jan 27, 2023 at 5:43 AM Tom Lane wrote: > David Rowley writes: > > On Fri, 27 Jan 2023 at 01:30, Amit Langote wrote: > >> It seems that the planner currently elides an Append/MergeAppend that > >> has run-time pruning info (part_prune_index) set, but which I think is > >> a bug. > > > Th

Re: Something is wrong with wal_compression

2023-01-26 Thread Tom Lane
Thomas Munro writes: > On Fri, Jan 27, 2023 at 1:30 PM Michael Paquier wrote: >> My opinion would be to make this function more reliable, FWIW, even if >> that involves a performance impact when called in a close loop by >> forcing more WAL flushes to ensure its report durability and >> consisten

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-26 Thread Andres Freund
Hi, On 2023-01-26 15:36:52 -0800, Peter Geoghegan wrote: > On Thu, Jan 26, 2023 at 12:45 PM Andres Freund wrote: > > > Most of the overhead of FREEZE WAL records (with freeze plan > > > deduplication and page-level freezing in) is generic WAL record header > > > overhead. Your recent adversarial

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-26 Thread Michael Paquier
On Thu, Jan 26, 2023 at 09:37:13AM +0100, Peter Eisentraut wrote: > Ok, the documentation make sense now. I wonder what the performance impact > is. Probably, nobody cares about microoptimizing CREATE TABLE statements. > But BEGIN/COMMIT could matter. However, whatever you do in between the > BE

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-26 Thread Peter Geoghegan
On Thu, Jan 26, 2023 at 6:37 PM Andres Freund wrote: > I also don't really see how that is responsive to anything else in my > email. That's just as true for the current gating condition (the issuance of > an FPI during heap_page_prune() / HTSV()). > > What I was wondering about is whether we shou

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-26 Thread Michael Paquier
On Thu, Jan 26, 2023 at 09:39:05AM +0100, Peter Eisentraut wrote: > There are a couple of repetitive comments, like "typmod and collation > information are irrelevant for the query jumbling". This applies to all > nodes, so we don't need to repeat it for a number of nodes (and then not > mention i

Re: Something is wrong with wal_compression

2023-01-26 Thread Thomas Munro
On Fri, Jan 27, 2023 at 3:04 PM Tom Lane wrote: > Thomas Munro writes: > > On Fri, Jan 27, 2023 at 1:30 PM Michael Paquier wrote: > >> My opinion would be to make this function more reliable, FWIW, even if > >> that involves a performance impact when called in a close loop by > >> forcing more W

Re: Monotonic WindowFunc support for ntile(), percent_rank() and cume_dist()

2023-01-26 Thread David Rowley
On Wed, 25 Jan 2023 at 02:39, Melanie Plageman wrote: > > On Tue, Jan 24, 2023 at 02:00:33PM +1300, David Rowley wrote: > > If you feel strongly about that, then feel free to show me what you > > have in mind in more detail so I can think harder about it. > > Nah, I don't feel strongly. I think it

Re: Something is wrong with wal_compression

2023-01-26 Thread Tom Lane
Thomas Munro writes: > On Fri, Jan 27, 2023 at 3:04 PM Tom Lane wrote: >> I think we need to get the thing correct first and worry about >> performance later. What's wrong with simply making pg_xact_status >> write and flush a record of the XID's existence before returning it? >> Yeah, it will c

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-26 Thread Hayato Kuroda (Fujitsu)
Dear hackers, I have updated my patch for error handling and kqueue() support. Actually I do not have BSD-like machine, but I developed by using github CICD. I think at first we should focus on 0001-0003, and then work for 0004. Followings are change notes and my analysis. 0001 * Fix missed rep

Re: Add LZ4 compression in pg_dump

2023-01-26 Thread Justin Pryzby
On Mon, Jan 16, 2023 at 11:54:46AM +0900, Michael Paquier wrote: > On Sun, Jan 15, 2023 at 07:56:25PM -0600, Justin Pryzby wrote: > > On Mon, Jan 16, 2023 at 10:28:50AM +0900, Michael Paquier wrote: > >> The functions changed by 0001 are cfopen[_write](), > >> AllocateCompressor() and ReadDataFromA

Re: Improve GetConfigOptionValues function

2023-01-26 Thread Bharath Rupireddy
On Tue, Jan 24, 2023 at 8:43 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: > >> Also, I intentionally dropped the GUC_NO_SHOW_ALL check in > >> get_explain_guc_options, because it seems redundant given > >> the preceding GUC_EXPLAIN check.

RE: Exit walsender before confirming remote flush in logical replication

2023-01-26 Thread Hayato Kuroda (Fujitsu)
Dear Dilip, hackers, Thanks for giving your opinion. I analyzed the relation with the given commit, and I thought I could keep my patch. How do you think? # Abstract * Some modifications should be needed. * We cannot rollback the shutdown if walsenders are stuck * We don't have a good way to det

Re: Something is wrong with wal_compression

2023-01-26 Thread Laurenz Albe
On Fri, 2023-01-27 at 16:15 +1300, Thomas Munro wrote: > On Fri, Jan 27, 2023 at 3:04 PM Tom Lane wrote: > > Thomas Munro writes: > > > On Fri, Jan 27, 2023 at 1:30 PM Michael Paquier > > > wrote: > > > > My opinion would be to make this function more reliable, FWIW, even if > > > > that involv

Re: Inconsistency in reporting checkpointer stats

2023-01-26 Thread Bharath Rupireddy
On Wed, Dec 21, 2022 at 5:15 PM Nitin Jadhav wrote: > > I have modified the code accordingly and attached the new version of > patches. patch 0001 fixes the inconsistency in checkpointer stats and > patch 0002 separates main buffer and SLRU buffer count from checkpoint > complete log message. IMO

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-26 Thread Amit Kapila
On Thu, Dec 8, 2022 at 8:17 AM Masahiko Sawada wrote: > > The same assertion failure has been reported on another thread[1]. > Since I could reproduce this issue several times in my environment > I've investigated the root cause. > > I think there is a race condition of updating > procArray->repli

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-26 Thread Masahiko Sawada
On Thu, Jan 26, 2023 at 2:33 PM Bharath Rupireddy wrote: > > On Thu, Jan 26, 2023 at 2:45 AM Andres Freund wrote: > > > > Hi, > > > > On 2023-01-14 12:34:03 -0800, Andres Freund wrote: > > > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupired

Re: recovery modules

2023-01-26 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:34:21PM +0900, Michael Paquier wrote: > The loop part is annoying.. I've never been a fan of adding this > cross-value checks for the archiver modules in the first place, and it > would make things much simpler in the checkpointer if we need to think > about that as we w

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-26 Thread Andres Freund
Hi, On 2023-01-26 19:01:03 -0800, Peter Geoghegan wrote: > On Thu, Jan 26, 2023 at 6:37 PM Andres Freund wrote: > > I also don't really see how that is responsive to anything else in my > > email. That's just as true for the current gating condition (the issuance of > > an FPI during heap_page_pr

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-26 Thread Andres Freund
Hi, On 2023-01-27 14:24:51 +0900, Masahiko Sawada wrote: > If I'm understanding this result correctly, it seems to me that your > patch works well with the WAL DIO patch (WALDIO vs. WAL DIO & WAL > BUFFERS READ), but there seems no visible performance gain with only > your patch (HEAD vs. WAL BUFF

Re: Reducing power consumption on idle servers

2023-01-26 Thread Bharath Rupireddy
On Wed, Jan 25, 2023 at 2:10 AM Tom Lane wrote: > > Thomas Munro writes: > > Yeah, I definitely want to fix it. I just worry that 60s is so long > > that it also needs that analysis work to be done to explain that it's > > OK that we're a bit sloppy on noticing when to wake up, at which point >

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-26 Thread Masahiko Sawada
On Fri, Jan 27, 2023 at 3:17 PM Andres Freund wrote: > > Hi, > > On 2023-01-27 14:24:51 +0900, Masahiko Sawada wrote: > > If I'm understanding this result correctly, it seems to me that your > > patch works well with the WAL DIO patch (WALDIO vs. WAL DIO & WAL > > BUFFERS READ), but there seems no

Re: Considering additional sort specialisation functions for PG16

2023-01-26 Thread John Naylor
On Thu, Jan 26, 2023 at 7:15 PM David Rowley wrote: > > On Thu, 26 Jan 2023 at 23:29, John Naylor wrote: > > Coming back to this, I wanted to sketch out this idea in a bit more detail. > > > > Have two memtuple arrays, one for first sortkey null and one for first sortkey non-null: > > - Qsort the

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-26 Thread Hayato Kuroda (Fujitsu)
I found cfbot failure, PSA fixed version. Sorry for noise. Best Regards, Hayato Kuroda FUJITSU LIMITED v29-0003-add-test.patch Description: v29-0003-add-test.patch v29-0004-add-kqueue-support-for-PQconnCheck-and-PQcanConn.patch Description: v29-0004-add-kqueue-support-for-PQconnCheck-and-PQc

Re: Reducing power consumption on idle servers

2023-01-26 Thread Thomas Munro
On Fri, Jan 27, 2023 at 7:37 PM Bharath Rupireddy wrote: > On Wed, Jan 25, 2023 at 2:10 AM Tom Lane wrote: > > It's kind of moot until we've reached the point where we can > > credibly claim to have explicit wakeups for every event of > > interest. I don't think we're very close to that today, a

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-26 Thread Peter Geoghegan
On Thu, Jan 26, 2023 at 9:58 PM Andres Freund wrote: > It doesn't seem like a great proxy to me. ISTM that this means that how > aggressive vacuum is about opportunistically freezing pages depends on config > variables like checkpoint_timeout & max_wal_size (less common opportunistic > freezing),

Re: Syncrep and improving latency due to WAL throttling

2023-01-26 Thread Bharath Rupireddy
On Thu, Jan 26, 2023 at 9:21 PM Andres Freund wrote: > > > 7. I think we need to not let backends throttle too frequently even > > though they have crossed wal_throttle_threshold bytes. The best way is > > to rely on replication lag, after all the goal of this feature is to > > keep replication la

Re: drop postmaster symlink

2023-01-26 Thread Peter Eisentraut
On 26.01.23 19:36, Karl O. Pinc wrote: I see a possible problem at line 1,412 of runtime.sgml This says: in the postmaster's startup script just before invoking the postmaster. Depending on how this is read, it could be interpreted to mean that a "postmaster" binary is invoked. It might be

Re: Minimal logical decoding on standbys

2023-01-26 Thread Drouvot, Bertrand
Hi, On 1/26/23 9:13 PM, Andres Freund wrote: Hi, On 2023-01-26 18:56:10 +0100, Drouvot, Bertrand wrote: - I'm struggling to create a test for btree killtuples as there is a need for rows removal on the table (that could produce a conflict too): Do you've a scenario in mind for this one? (and

<    1   2