Re: RFC: Logging plan of the running query

2023-10-24 Thread Étienne BERSAC
> Hi, > > Yeah, and when we have a situation where we want to run > pg_log_query_plan(), we can run it in any environment as long as it > is bundled with the core. Is it possible to get the plan of a backend programmatically without access to the logs? Something like pg_get_query_plan(pid, forma

Re: Synchronizing slots from primary to standby

2023-10-24 Thread Peter Smith
Here are some review comments for v24-0001 == 1. GENERAL - failover slots terminology There is inconsistent terminology, such as below. Try to use the same wording everywhere. - failover logical slots - failover slots - logical failover slots - logical replication failover slots - etc. These

Re: Synchronizing slots from primary to standby

2023-10-24 Thread Peter Smith
Here are some review comments for patch v25-0002 (additional to v25-0002 review comments [1]) == src/backend/catalog/system_views.sql 1. @@ -1003,7 +1003,8 @@ CREATE VIEW pg_replication_slots AS L.safe_wal_size, L.two_phase, L.conflicting, -

RE: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2023-10-24 Thread Hayato Kuroda (Fujitsu)
Dear Horiguchi-san, Thanks for updates! I was quite not sure the Windows env, but I can post comments. (We need reviews by windows-friendly developers...) > Other error cases will fit to "shouldn't occur under normal > conditions" errors. Formatting of messages for write_stderr() seem different

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-24 Thread Bharath Rupireddy
On Tue, Oct 24, 2023 at 11:32 AM Hayato Kuroda (Fujitsu) wrote: > > > If we don't want to keep it generic then we should use something like > > 'contains_decodable_change'. 'is_change_decodable' could have suited > > here if we were checking a particular change. > > I kept the name for now. How do

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Andrei Zubkov
Hi, During last moving to the current commitfest this patch have lost its reviewers list. With respect to reviewers contribution in this patch, I think reviewers list should be fixed. regards, Andrei Zubkov Postgres Professional The Russian Postgres Company

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2023-10-24 Thread Kyotaro Horiguchi
At Tue, 24 Oct 2023 07:37:22 +, "Hayato Kuroda (Fujitsu)" wrote in > Dear Horiguchi-san, > > Thanks for updates! I was quite not sure the Windows env, but I can post > comments. > (We need reviews by windows-friendly developers...) Indeed, I haven't managed to successfully build using Mes

Re: Replace references to malloc() in libpq documentation with generic language

2023-10-24 Thread Daniel Gustafsson
> On 24 Oct 2023, at 07:13, Gurjeet Singh wrote: > The user does not benefit from knowing that libpq allocates some/all memory > using malloc(). Mentioning malloc() here has a few downsides, and almost no > benefits. I'm not entirely convinced that replacing "malloc" with "allocated on the heap"

Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Laurenz Albe
Try this as a user with NOBYPASSRLS: CREATE TABLE rlsbug (deleted boolean); INSERT INTO rlsbug VALUES (FALSE); CREATE POLICY p_sel ON rlsbug FOR SELECT TO laurenz USING (NOT deleted); CREATE POLICY p_upd ON rlsbug FOR UPDATE TO laurenz USING (TRUE); ALTER TABLE rlsbug ENABLE ROW LEVEL SECURIT

Re: Simplify create_merge_append_path a bit for clarity

2023-10-24 Thread Alena Rybakina
Hi! On 11.08.2023 05:31, Richard Guo wrote: As explained in the comments for generate_orderedappend_paths, we don't currently support parameterized MergeAppend paths, and it doesn't seem like going to change anytime soon.  Based on that,  we could simplify create_merge_append_path a bit, such as

Re: Synchronizing slots from primary to standby

2023-10-24 Thread Drouvot, Bertrand
Hi, On 10/24/23 7:44 AM, Ajin Cherian wrote: On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand wrote: @@ -602,6 +602,9 @@ CreateDecodingContext(XLogRecPtr start_lsn, SnapBuildSetTwoPhaseAt(ctx->snapshot_builder, start_lsn); } + /* set failover in the slot, as requested */

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Dean Rasheed
On Tue, 24 Oct 2023 at 09:36, Laurenz Albe wrote: > > I'd say that this error is wrong. The FOR SELECT policy should be applied > to the WHERE condition, but certainly not to check new rows. > Yes, I had the same thought recently. I would say that the SELECT policies should only be used to check

A case for GIST supporting ORDER BY

2023-10-24 Thread Michał Kłeczek
Hi, Some time ago I’ve provided some details with the issues we face when trying to use GIST and partitioning at the same time in the postgresql-general mailing list: https://www.postgresql.org/message-id/3FA1E0A9-8393-41F6-88BD-62EEEA1EC21F%40kleczek.org GIST index and ORDER BY postgresql.org

RE: logical decoding and replication of sequences, take 2

2023-10-24 Thread Zhijie Hou (Fujitsu)
On Thursday, October 12, 2023 11:06 PM Tomas Vondra wrote: > Hi, I have been reviewing the patch set, and here are some initial comments. 1. I think we need to mark the RBTXN_HAS_STREAMABLE_CHANGE flag for transactional sequence change in ReorderBufferQueueChange(). 2. ReorderBufferSequence

d844cd75a and postgres_fdw

2023-10-24 Thread Devrim Gündüz
Hi, I'm seeing an issue after upgrading from 12.13 to 15.4. This happens when we run a query against a foreign table (fdw on the same instance to a different database) -- but does not appear when we get rid of postgres_fdw: ERROR: cursor can only scan forward HINT: Declare it with SCROLL optio

Open a streamed block for transactional messages during decoding

2023-10-24 Thread Zhijie Hou (Fujitsu)
Hi, While reviewing the test_decoding code, I noticed that when skip_empty_xacts option is specified, it doesn't open the streaming block( e.g. pg_output_stream_start) before streaming the transactional MESSAGE even if it's the first change in a streaming block. It looks inconsistent with what we

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Peter Eisentraut
On 24.10.23 09:58, Andrei Zubkov wrote: During last moving to the current commitfest this patch have lost its reviewers list. With respect to reviewers contribution in this patch, I think reviewers list should be fixed. I don't think it's the purpose of the commitfest app to track how *has* re

Re: trying again to get incremental backup

2023-10-24 Thread Robert Haas
On Mon, Oct 23, 2023 at 7:56 PM David Steele wrote: > > I also think a lot of the use of the low-level API is > > driven by it being just too darn slow to copy the whole database, and > > incremental backup can help with that in some circumstances. > > I would argue that restore performance is *mo

Re: RFC: Logging plan of the running query

2023-10-24 Thread torikoshia
On 2023-10-24 16:12, Étienne BERSAC wrote: Hi, Yeah, and when we have a situation where we want to run pg_log_query_plan(), we can run it in any environment as long as it is bundled with the core. Is it possible to get the plan of a backend programmatically without access to the logs? Somethi

make pg_ctl start more friendly

2023-10-24 Thread Crisp Lee
I tried PITR recovery, and the 'recovery_target_action' guc is shutdown. I got a failure, and it told me to check the log, finally I found the result was due to guc. I think pg_ctl should print some information which told users recovery had been done. I developed a commit in my workspace. The steps

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Julien Rouhaud
On Tue, Oct 24, 2023 at 6:57 PM Peter Eisentraut wrote: > > On 24.10.23 09:58, Andrei Zubkov wrote: > > During last moving to the current commitfest this patch have lost its > > reviewers list. With respect to reviewers contribution in this patch, I > > think reviewers list should be fixed. > > I

Re: Query execution in Perl TAP tests needs work

2023-10-24 Thread Andrew Dunstan
On 2023-10-18 We 11:47, Tom Lane wrote: Robert Haas writes: On Wed, Oct 18, 2023 at 10:28 AM Tom Lane wrote: I did a bit of research on this on my favorite platforms, and did not like the results: Hmm. That's unfortunate. Is perl -MCPAN -e 'install Platypus::FFI' a viable alternative? Prob

Re: Various bugs if segment containing redo pointer does not exist

2023-10-24 Thread Robert Haas
On Mon, Oct 23, 2023 at 8:43 PM Andres Freund wrote: > The source of the emode=13=DEBUG2 is that that's hardcoded in > WaitForWALToBecomeAvailable(). I guess the error ought to come from > XLogPageRead(), but all that happens is this: > > case XLREAD_FAIL: >

Re: Add trailing commas to enum definitions

2023-10-24 Thread Andrew Dunstan
On 2023-10-23 Mo 17:04, Tom Lane wrote: Nathan Bossart writes: From a long-term perspective, I think standardizing on the trailing comma style will actually improve git-blame because patches won't need to add a comma to the previous line when adding a value. Yeah, that's a good point. I ha

Re: More new SQL/JSON item methods

2023-10-24 Thread Andrew Dunstan
On 2023-10-19 Th 02:06, Jeevan Chalke wrote: Thanks, Peter for the comments. On Fri, Oct 6, 2023 at 5:13 PM Peter Eisentraut wrote: On 29.08.23 09:05, Jeevan Chalke wrote: > v1-0001-Implement-jsonpath-.bigint-.integer-and-.number-m.patch > > This commit implements jsonpath .

Re: run pgindent on a regular basis / scripted manner

2023-10-24 Thread Andrew Dunstan
On 2023-10-17 Tu 09:52, Robert Haas wrote: On Tue, Oct 17, 2023 at 6:34 AM Jelte Fennema wrote: I think *it is* dead easy to comply. If you run the following commands before committing/after rebasing, then koel should always be happy: src/tools/pgindent/pgindent src # works always but a bit

Re: run pgindent on a regular basis / scripted manner

2023-10-24 Thread Bruce Momjian
On Tue, Oct 24, 2023 at 09:40:01AM -0400, Andrew Dunstan wrote: > Slightly off topic, but apropos your message, maybe we should recommend a > standard git commit template. I use this and then automatically remove any sections that are empty. ---

Re: [PATCH] Extend ALTER OPERATOR to support adding commutator, negator, hashes, and merges

2023-10-24 Thread Christoph Berg
Re: Tommy Pavlicek > I've added another patch (0002-require_unused_neg_com-v1.patch) that > prevents using a commutator or negator that's already part of a pair. Hmm. I agree with the general idea of adding sanity checks, but this might be overzealous: This change is breaking pgsphere which has <

Re: run pgindent on a regular basis / scripted manner

2023-10-24 Thread Andrew Dunstan
On 2023-10-18 We 05:07, Jelte Fennema wrote: I think --enable-indent-checks sounds like a good improvement to the status quo. But I'm not confident that it will help remove the cases where only a comment needs to be re-indented. Do commiters really always run check-world again when only changin

Should Explain show Parallel Hash node’s total rows?

2023-10-24 Thread Zhang Mingli
Hi, all Shall we show Parallel Hash node’s total rows of a Parallel-aware HashJoin? Ex: a non-parallel plan, table simple has 2 rows. zml=# explain select count(*) from simple r join simple s using (id); QUERY PLAN ---

Re: trying again to get incremental backup

2023-10-24 Thread Peter Eisentraut
On 04.10.23 22:08, Robert Haas wrote: - I would like some feedback on the generation of WAL summary files. Right now, I have it enabled by default, and summaries are kept for a week. That means that, with no additional setup, you can take an incremental backup as long as the reference backup was

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Peter Eisentraut
On 24.10.23 14:40, Julien Rouhaud wrote: On Tue, Oct 24, 2023 at 6:57 PM Peter Eisentraut wrote: On 24.10.23 09:58, Andrei Zubkov wrote: During last moving to the current commitfest this patch have lost its reviewers list. With respect to reviewers contribution in this patch, I think reviewer

Re: Replace references to malloc() in libpq documentation with generic language

2023-10-24 Thread Tom Lane
Daniel Gustafsson writes: >> On 24 Oct 2023, at 07:13, Gurjeet Singh wrote: >> The user does not benefit from knowing that libpq allocates some/all memory >> using malloc(). Mentioning malloc() here has a few downsides, and almost no >> benefits. > I'm not entirely convinced that replacing "mall

Re: [PATCH] Extend ALTER OPERATOR to support adding commutator, negator, hashes, and merges

2023-10-24 Thread Tom Lane
Christoph Berg writes: > This change is breaking pgsphere which has <@ @> operator pairs, but > for historical reasons also includes alternative spellings of these > operators (both called @ with swapped operand types) which now > explodes because we can't add them with the "proper" commutator and

RE: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-24 Thread José Neves
Hi there, hope to find you well. I have a follow-up question to this already long thread. Upon deploying my PostgreSQL logical replication fed application on a stale database, I ended up running out of space, as the replication slot is being held back till the next time that we receive a data-c

Re: [PATCH] Extend ALTER OPERATOR to support adding commutator, negator, hashes, and merges

2023-10-24 Thread Christoph Berg
Re: Tom Lane > > We might be able to simply delete the @ operators, but doesn't this > > new check break the general possibility to have more than one spelling > > for the same operator? > > You can have more than one operator atop the same function. > But why didn't you make the @ operators commu

Re: [PATCH] Extend ALTER OPERATOR to support adding commutator, negator, hashes, and merges

2023-10-24 Thread Tom Lane
Christoph Berg writes: > Anyway, if this doesn't raise any "oh we didn't think of this" > concerns, we'll just remove the old operators in pgsphere. Well, the idea was exactly to forbid that sort of setup. However, if we get sufficient pushback maybe we should reconsider --- for example, maybe it

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Tom Lane
Dean Rasheed writes: > On Tue, 24 Oct 2023 at 09:36, Laurenz Albe wrote: >> I'd say that this error is wrong. The FOR SELECT policy should be applied >> to the WHERE condition, but certainly not to check new rows. > Yes, I had the same thought recently. I would say that the SELECT > policies sh

Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Jelte Fennema
Many usages of the foreach macro in the Postgres codebase only use the ListCell variable to then get its value. This adds macros that simplify iteration code for that very common use case. Instead of passing a ListCell you can pass a variable of the type of its contents. This IMHO improves readabil

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Alvaro Herrera
On 2023-Oct-11, Dilip Kumar wrote: > In my last email, I forgot to give the link from where I have taken > the base path for dividing the buffer pool in banks so giving the same > here[1]. And looking at this again it seems that the idea of that > patch was from Andrey M. Borodin and the idea of

Re: trying again to get incremental backup

2023-10-24 Thread Robert Haas
On Tue, Oct 24, 2023 at 10:53 AM Peter Eisentraut wrote: > The easiest answer is to have it off by default. Let people figure out > what works for them. There are various factors like storage, network, > server performance, RTO that will determine what combination of full > backup, incremental b

Re: Adding a pg_get_owned_sequence function?

2023-10-24 Thread Nathan Bossart
On Tue, Sep 12, 2023 at 03:53:28PM +0100, Dagfinn Ilmari Mannsåker wrote: > Tom Lane writes: >> It's possible that we could get away with just summarily changing >> the argument type from text to regclass. ISTR that we did exactly >> that with nextval() years ago, and didn't get too much push-bac

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 06:03:48PM +0200, Jelte Fennema wrote: > Many usages of the foreach macro in the Postgres codebase only use the > ListCell variable to then get its value. This adds macros that > simplify iteration code for that very common use case. Instead of > passing a ListCell you can p

Re: LLVM 16 (opaque pointers)

2023-10-24 Thread Mark Wong
On Mon, Oct 23, 2023 at 10:47:24PM -0400, Tom Lane wrote: > Andres Freund writes: > > FC 29 is well out of support, so I don't think it makes sense to invest any > > further time in this. Personally, I don't think it's useful to have years > > old > > fedora in the buildfarm... > > +1. It's goo

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Jelte Fennema
On Tue, 24 Oct 2023 at 18:47, Nathan Bossart wrote: > BTW after applying your patches, initdb began failing with the following > for me: > > TRAP: failed Assert("n >= 0 && n < list->length"), File: "list.c", > Line: 770, PID: 902807 Oh oops... That was an off by one error in the modified

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 06:04:13PM +0200, Alvaro Herrera wrote: > Like everybody else, I like having less GUCs to configure, but going > this far to avoid them looks rather disastrous to me. IMO we should > just use Munro's older patches that gave one GUC per SLRU, and users > only need to increas

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Jeff Davis
On Tue, 2023-10-24 at 11:59 -0400, Tom Lane wrote: > I'm fairly sure that it was intentional, but I don't recall the > reasoning; perhaps Stephen does.  In any case, I grasp your point > that maybe we should distinguish RETURNING from not-RETURNING cases. Perhaps the idea is that if there are cons

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Robert Haas
On Tue, Oct 24, 2023 at 1:46 PM Jeff Davis wrote: > Perhaps the idea is that if there are constraints involved, the failure > or success of an INSERT/UPDATE/DELETE could leak information that you > don't have privileges to read. My recollection of this topic is pretty hazy, but like Tom, I seem t

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-24 Thread Stephen Frost
Greetings, On Tue, Oct 24, 2023 at 14:42 Robert Haas wrote: > On Tue, Oct 24, 2023 at 1:46 PM Jeff Davis wrote: > > Perhaps the idea is that if there are constraints involved, the failure > > or success of an INSERT/UPDATE/DELETE could leak information that you > > don't have privileges to read

Re: Row pattern recognition

2023-10-24 Thread Jacob Champion
On Sat, Oct 21, 2023 at 7:39 PM Tatsuo Ishii wrote: > Attached is the v10 patch. This version enhances the performance of > pattern matching. Nice! I've attached a couple of more stressful tests (window partitions of 1000 rows each). Beware that the second one runs my desktop out of memory fairly

Re: MERGE ... RETURNING

2023-10-24 Thread Jeff Davis
On Wed, 2023-08-23 at 11:58 +0100, Dean Rasheed wrote: > Updated version attached, fixing an uninitialized-variable warning > from the cfbot. I took another look and I'm still not comfortable with the special IsMergeSupportFunction() functions. I don't object necessarily -- if someone else wants t

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2023-10-24 Thread Alexander Korotkov
On Wed, Sep 28, 2022 at 11:44 AM Aleksander Alekseev wrote: > > I think, this patch was marked as "Waiting on Author", probably, by > > mistake. Since recent changes were done without any significant code > > changes and CF bot how happy again. > > > > I'm going to move it to RfC, could I? If no

Re: Replace references to malloc() in libpq documentation with generic language

2023-10-24 Thread Daniel Gustafsson
> On 24 Oct 2023, at 17:07, Tom Lane wrote: > Daniel Gustafsson writes: >> I do agree with this proposed change though: > >> - all the space that will be freed by . >> + all the memory that will be freed by . > > +1, seems harmless. I've pushed this part, skipping the rest. -- Dani

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-10-24 Thread Daniel Gustafsson
> On 13 Sep 2023, at 01:49, Andres Freund wrote: > My patch increased user/sys time a bit (likely due to a higher number of > futile psql forks), but Daniel's doesn't. And it does show a nice overall wall > clock time saving. I went ahead and applied this on master, thanks for review! Now to se

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-10-24 Thread Tom Lane
Daniel Gustafsson writes: > I went ahead and applied this on master, thanks for review! Now to see if > there will be any noticeable difference in resource usage. I think that tools like Coverity are likely to whine about your use of sprintf instead of snprintf. Sure, it's perfectly safe, but t

RFC: Pluggable TOAST

2023-10-24 Thread Nikita Malakhov
Hi hackers! We need community feedback on previously discussed topic [1]. There are some long-live issues in Postgres related to the TOAST mechanics, like [2]. Some time ago we already proposed a set of patches with an API allowing to plug in different TOAST implementations into a live database. T

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-24 Thread Bruce Momjian
On Tue, Dec 13, 2022 at 07:38:15PM -0700, David G. Johnston wrote: > All, > > The recent discussion surrounding aggregates and ORDER BY moved me to look > over > our existing documentation, especially now that we've reworked the function > tables, to see what improvements can be had by simply doc

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-10-24 Thread Daniel Gustafsson
> On 24 Oct 2023, at 22:34, Tom Lane wrote: > > Daniel Gustafsson writes: >> I went ahead and applied this on master, thanks for review! Now to see if >> there will be any noticeable difference in resource usage. > > I think that tools like Coverity are likely to whine about your > use of spri

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Nathan Bossart
On Fri, Oct 20, 2023 at 05:18:56PM +0900, Michael Paquier wrote: > On Fri, Oct 20, 2023 at 07:08:58AM +, Xiang Gao wrote: >> This patch uses a parallel computing optimization algorithm to >> improve crc32c computing performance on ARM. The algorithm comes >> from Intel whitepaper: >> crc-iscsi-

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 04:09:54PM -0500, Nathan Bossart wrote: > I'm able to reproduce the speedup with the provided benchmark on an Apple > M1 Pro (which appears to have the required instructions). There was almost > no change for the 512-byte case, but there was a ~60% speedup for the > 4096-by

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 06:58:04PM +0200, Jelte Fennema wrote: > On Tue, 24 Oct 2023 at 18:47, Nathan Bossart wrote: >> BTW after applying your patches, initdb began failing with the following >> for me: >> >> TRAP: failed Assert("n >= 0 && n < list->length"), File: "list.c", >> Line: 770

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Heikki Linnakangas
On 25/10/2023 00:18, Nathan Bossart wrote: On Tue, Oct 24, 2023 at 04:09:54PM -0500, Nathan Bossart wrote: I'm able to reproduce the speedup with the provided benchmark on an Apple M1 Pro (which appears to have the required instructions). There was almost no change for the 512-byte case, but th

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Michael Paquier
On Wed, Oct 25, 2023 at 12:37:45AM +0300, Heikki Linnakangas wrote: > On 25/10/2023 00:18, Nathan Bossart wrote: >> Actually, since the pg_waldump benchmark likely only involves very small >> WAL records, it would make sense that there isn't much difference. >> *facepalm* > > No need to guess, pg_

walwriter interacts quite badly with synchronous_commit=off

2023-10-24 Thread Andres Freund
Hi, I recently mentioned to Heikki that I was seeing latch related wakeups being frequent enough to prevent walwriter from doing a whole lot of work. He asked me to write that set of concerns up, which seems quite fair... Here's a profile of walwriter while the following pgbench run was ongoing:

Re: Row pattern recognition

2023-10-24 Thread Tatsuo Ishii
> On Sat, Oct 21, 2023 at 7:39 PM Tatsuo Ishii wrote: >> Attached is the v10 patch. This version enhances the performance of >> pattern matching. > > Nice! I've attached a couple of more stressful tests (window > partitions of 1000 rows each). Beware that the second one runs my > desktop out of m

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-10-24 Thread Jeff Davis
On Sat, 2023-10-21 at 23:59 +0530, Bharath Rupireddy wrote: > I'm attaching v12 patch set with just pgperltidy ran on the new TAP > test added in 0002. No other changes from that of v11 patch set. Thank you. Comments: * It would be good to document that this is partially an optimization (read fr

Re: Show version of OpenSSL in ./configure output

2023-10-24 Thread Michael Paquier
On Mon, Oct 23, 2023 at 08:44:01PM -0400, Tom Lane wrote: > OK by me. Cool, done down to 16 as this depends on c8e4030d1bdd. -- Michael signature.asc Description: PGP signature

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-10-24 Thread David Rowley
On Tue, 17 Oct 2023 at 20:39, David Rowley wrote: > > On Mon, 16 Oct 2023 at 05:56, Tom Lane wrote: > > I guess the next question is whether we want to stop here or > > try to relax the requirement about NUL-termination. I'd be inclined > > to call that a separate issue deserving a separate comm

Re: MERGE ... RETURNING

2023-10-24 Thread Merlin Moncure
On Tue, Oct 24, 2023 at 2:11 PM Jeff Davis wrote: > On Wed, 2023-08-23 at 11:58 +0100, Dean Rasheed wrote: > > Updated version attached, fixing an uninitialized-variable warning > > from the cfbot. > > I took another look and I'm still not comfortable with the special > IsMergeSupportFunction() f

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Nathan Bossart
On Wed, Oct 25, 2023 at 07:17:55AM +0900, Michael Paquier wrote: > If you are looking at computing the CRC of records with arbitrary > sizes, why not just generating a series with > pg_logical_emit_message() before doing a comparison with pg_waldump or > a custom replay loop to go through the recor

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-24 Thread David G. Johnston
On Tue, Oct 24, 2023 at 1:39 PM Bruce Momjian wrote: > On Tue, Dec 13, 2022 at 07:38:15PM -0700, David G. Johnston wrote: > > All, > > > > The recent discussion surrounding aggregates and ORDER BY moved me to > look over > > our existing documentation, especially now that we've reworked the > fun

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 05:15:19PM -0700, Jeff Davis wrote: > * Are you sure that reducing the number of calls to memcpy() is a win? > I would expect that to be true only if the memcpy()s are tiny, but here > they are around XLOG_BLCKSZ. I believe this was done based on a comment > from Nathan Boss

Re: Synchronizing slots from primary to standby

2023-10-24 Thread shveta malik
On Tue, Oct 24, 2023 at 3:35 PM Drouvot, Bertrand wrote: > > Hi, > > On 10/24/23 7:44 AM, Ajin Cherian wrote: > > On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand > > wrote: > >> > >> @@ -602,6 +602,9 @@ CreateDecodingContext(XLogRecPtr start_lsn, > >> SnapBuildSetTwoPhaseAt(ctx->snap

Re: post-recovery amcheck expectations

2023-10-24 Thread Peter Geoghegan
On Mon, Oct 23, 2023 at 7:28 PM Noah Misch wrote: > > That makes sense to me. I believe that it's not possible to have a > > string of consecutive sibling pages that are all half-dead (regardless > > of the BlockNumber order of sibling pages, even). But I'd probably > > have written the fix in rou

Re: Patch: Improve Boolean Predicate JSON Path Docs

2023-10-24 Thread David E. Wheeler
On Oct 23, 2023, at 20:20, Erik Wienhold wrote: > I thought that you may have missed that one because I saw this change > that removes the article: > >> -In the strict mode, the specified path must exactly match the structure >> of >> +In strict mode, the specified path must exactly mat

Re: Row pattern recognition

2023-10-24 Thread Tatsuo Ishii
> Great. I will look into this. I am impressed the simple NFA implementation. It would be nicer if it could be implemented without using recursion. > By the way, I tested my patch (v10) to handle more large data set and > tried to following query with pgbench database. On my laptop it works > wi

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread David Rowley
On Wed, 25 Oct 2023 at 06:00, Jelte Fennema wrote: > Attached is a fixed version. With foreach(), we commonly do "if (lc == NULL)" at the end of loops as a way of checking if we did "break" to terminate the loop early. Doing the equivalent with the new macros won't be safe as the list element's v

Re: Synchronizing slots from primary to standby

2023-10-24 Thread shveta malik
On Tue, Oct 24, 2023 at 11:54 AM Drouvot, Bertrand wrote: > > Hi, > > On 10/23/23 2:56 PM, shveta malik wrote: > > On Mon, Oct 23, 2023 at 5:52 PM Drouvot, Bertrand > > wrote: > > >> We are waiting for DEFAULT_NAPTIME_PER_CYCLE (3 minutes) before checking > >> if there > >> is new synced slot(s)

Re: post-recovery amcheck expectations

2023-10-24 Thread Noah Misch
On Tue, Oct 24, 2023 at 07:03:34PM -0700, Peter Geoghegan wrote: > On Mon, Oct 23, 2023 at 7:28 PM Noah Misch wrote: > > > That makes sense to me. I believe that it's not possible to have a > > > string of consecutive sibling pages that are all half-dead (regardless > > > of the BlockNumber order

RE: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Xiang Gao
Thanks for your suggestion, this is the modified patch and two test files. -Original Message- From: Michael Paquier Sent: Friday, October 20, 2023 4:19 PM To: Xiang Gao Cc: pgsql-hackers@lists.postgresql.org Subject: Re: CRC32C Parallel Computation Optimization on ARM On Fri, Oct 20, 20

Re: RFC: Logging plan of the running query

2023-10-24 Thread Ashutosh Bapat
On Wed, Oct 18, 2023 at 10:04 PM James Coleman wrote: > > While that decision as regards auto_explain has long since been made > (and there would be work to undo it), I don't believe that we should > repeat that choice here. I'm -10 on moving this into auto_explain. > Right. > However perhaps th

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-10-24 Thread John Naylor
On Tue, Oct 17, 2023 at 9:39 PM Robert Haas wrote: > > Cool. Seems we are all in agreement, so committed these. Thanks! Thank you for getting this across the finish line!

Adding facility for injection points (or probe points?) for more advanced tests

2023-10-24 Thread Michael Paquier
Hi all, I don't remember how many times in the last few years when I've had to hack the backend to produce a test case that involves a weird race condition across multiple processes running in the backend, to be able to prove a point or just test a fix (one recent case: 2b8e5273e949). Usually, I c

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-10-24 Thread Amul Sul
On Wed, Oct 25, 2023 at 9:43 AM Michael Paquier wrote: > Hi all, > > I don't remember how many times in the last few years when I've had to > hack the backend to produce a test case that involves a weird race > condition across multiple processes running in the backend, to be able > to prove a po

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-10-24 Thread Michael Paquier
On Wed, Oct 25, 2023 at 10:06:17AM +0530, Amul Sul wrote: > +1 for the feature. > > TWIMW, here[1] is an interesting talk from pgconf.in 2020 on the similar > topic. > > 1] https://pgconf.in/conferences/pgconfin2020/program/proposals/101 Right, this uses a shared hash table. There is a patch fr

Re: Synchronizing slots from primary to standby

2023-10-24 Thread Amit Kapila
On Tue, Oct 24, 2023 at 3:35 PM Drouvot, Bertrand wrote: > > On 10/24/23 7:44 AM, Ajin Cherian wrote: > > On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand > > wrote: > > 2) When we create a subscription, another slot is created during the > subscription synchronization, namely > like "pg_16397

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Dilip Kumar
On Tue, Oct 24, 2023 at 9:34 PM Alvaro Herrera wrote: > > On 2023-Oct-11, Dilip Kumar wrote: > > > In my last email, I forgot to give the link from where I have taken > > the base path for dividing the buffer pool in banks so giving the same > > here[1]. And looking at this again it seems that th

Re: [dynahash] do not refill the hashkey after hash_search

2023-10-24 Thread John Naylor
On Fri, Oct 13, 2023 at 10:07 AM Nathan Bossart wrote: > > On Thu, Sep 14, 2023 at 04:28:26PM +0800, Junwang Zhao wrote: > > Add a v2 with some change to fix warnings about unused-parameter. > > > > I will add this to Commit Fest. > > This looks reasonable to me. I've marked the commitfest entry

Re: [dynahash] do not refill the hashkey after hash_search

2023-10-24 Thread Tom Lane
John Naylor writes: > I'd prefer just adding "Assert(hentry->event == oldn);" and declaring > hentry PG_USED_FOR_ASSERTS_ONLY. I'm not aware of any other places where we have Asserts checking that hash_search() honored its contract. Why do we need one here? regards, tom

Re: Guiding principle for dropping LLVM versions?

2023-10-24 Thread Thomas Munro
Here are some systematic rules I'd like to propose to anchor this stuff to reality and avoid future doubt and litigation: 1. Build farm animals testing LLVM determine the set of OSes and LLVM versions we consider. 2. We exclude OSes that will be out of full vendor support when a release ships. 3

Re: [dynahash] do not refill the hashkey after hash_search

2023-10-24 Thread John Naylor
On Wed, Oct 25, 2023 at 12:21 PM Tom Lane wrote: > > John Naylor writes: > > I'd prefer just adding "Assert(hentry->event == oldn);" and declaring > > hentry PG_USED_FOR_ASSERTS_ONLY. > > I'm not aware of any other places where we have Asserts checking > that hash_search() honored its contract.

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-24 Thread Amit Kapila
On Tue, Oct 24, 2023 at 1:20 PM Bharath Rupireddy wrote: > > > I spent some time on the v57 patch and it looks good to me - tests are > passing, no complaints from pgindent and pgperltidy. I turned the CF > entry https://commitfest.postgresql.org/45/4273/ to RfC. > Thanks, the patch looks mostly

Re: Guiding principle for dropping LLVM versions?

2023-10-24 Thread Tom Lane
Thomas Munro writes: > Here are some systematic rules I'd like to propose to anchor this > stuff to reality and avoid future doubt and litigation: > 1. Build farm animals testing LLVM determine the set of OSes and LLVM > versions we consider. > 2. We exclude OSes that will be out of full vendor

Re: ALTER COLUMN ... SET EXPRESSION to alter stored generated column's expression

2023-10-24 Thread Amul Sul
Here is the rebase version for the latest master head(673a17e3120). I haven't done any other changes related to the ON UPDATE trigger since that seems non-trivial; need a bit of work to add trigger support in ATRewriteTable(). Also, I am not sure yet, if we were doing these changes, and the correc

Re: SQL:2011 application time

2023-10-24 Thread jian he
On Wed, Oct 11, 2023 at 12:47 PM Paul Jungwirth wrote: > > On 9/25/23 14:00, Peter Eisentraut wrote: > > Looking through the tests in v16-0001: > > > > +-- PK with no columns just WITHOUT OVERLAPS: > > +CREATE TABLE temporal_rng ( > > + valid_at tsrange, > > + CONSTRAINT temporal_rng_p

Re: A performance issue with Memoize

2023-10-24 Thread Richard Guo
On Fri, Oct 20, 2023 at 6:40 PM Richard Guo wrote: > I haven't thought thoroughly about the fix yet. But one way I'm > thinking is that in create_subqueryscan_plan() we can first add the > subquery's subplan_params to root->curOuterParams, and then replace > outer-relation Vars in scan_clauses a

Re: A performance issue with Memoize

2023-10-24 Thread Richard Guo
On Fri, Oct 20, 2023 at 7:43 PM Pavel Stehule wrote: > +1 > > it would be great to fix this problem - I've seen this issue a few times. > Thanks for the input. I guess this is not rare in the real world. If the subquery contains lateral reference to a Var that also appears in the subquery's jo

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Andrei Lepikhov
On 19/10/2023 19:40, Andrei Zubkov wrote: Hi hackers, New version 23 attached. It contains rebase to the current master. I discovered the patch and parameters you've proposed. In my opinion, the stats_since parameter adds valuable information and should definitely be included in the stats da