Re: Vacuum statistics

2024-06-06 Thread Dilip Kumar
On Thu, May 30, 2024 at 11:57 PM Alena Rybakina wrote: > > On 30.05.2024 10:33, Alena Rybakina wrote: > > > > I suggest gathering information about vacuum resource consumption for > > processing indexes and tables and storing it in the table and index > > relationships (for example, PgStat_StatTab

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Dilip Kumar
On Fri, Jun 7, 2024 at 11:53 AM Ashutosh Sharma wrote: > > On Fri, Jun 7, 2024 at 10:06 AM Dilip Kumar wrote: > > > > On Thu, Jun 6, 2024 at 7:39 PM Ashutosh Sharma > > wrote: > > > > > > On Thu, Jun 6, 2024 at 6:20 PM Dilip Kumar wrote: > > >> > > >> On Thu, Jun 6, 2024 at 5:59 PM Ashutosh Sh

Re: altering a column's collation leaves an invalid foreign key

2024-06-06 Thread jian he
On Sat, Apr 13, 2024 at 9:13 PM jian he wrote: > > > > > Here is a patch implementing this. It was a bit more fuss than I > > > > expected, so maybe someone has a > > > > better way. > I think I found a simple way. > > the logic is: > * ATExecAlterColumnType changes one column once at a time. > *

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-06 Thread Matthias van de Meent
On Fri, 7 Jun 2024 at 07:18, Dilip Kumar wrote: > > On Wed, Jun 5, 2024 at 10:59 PM Matthias van de Meent > wrote: >> >> On Wed, 5 Jun 2024 at 18:47, Ranier Vilela wrote: >>> >>> Why not use it too, if not binary_upgrade? >> >> Because in the normal case (not during binary_upgrade) you don't wan

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Ashutosh Sharma
On Fri, Jun 7, 2024 at 10:06 AM Dilip Kumar wrote: > > On Thu, Jun 6, 2024 at 7:39 PM Ashutosh Sharma wrote: > > > > On Thu, Jun 6, 2024 at 6:20 PM Dilip Kumar wrote: > >> > >> On Thu, Jun 6, 2024 at 5:59 PM Ashutosh Sharma > >> wrote: > >> > > >> > Hello everyone, > >> > > >> > At present, we

Add support to TLS 1.3 cipher suites and curves lists

2024-06-06 Thread Erica Zhang
Hi All, I’m a Postgres user and I’m looking into restricting the set of allowed ciphers on Postgres and configure a concrete set of curves on our postgres instances. I see in current Postgres doc mentioned that only TLS1.2 and below cipher lists can be configured. And there is no setting that c

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Thomas Munro
On Fri, Jun 7, 2024 at 3:06 PM Thomas Munro wrote: > On Fri, Jun 7, 2024 at 3:00 PM Alexander Lakhin wrote: > > My bisect run ended with: > > 210622c60e1a9db2e2730140b8106ab57d259d15 is the first bad commit > > > > Author: Thomas Munro > > Date: Wed Apr 3 00:03:08 2024 +1300 > > > > Provi

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-06 Thread Dilip Kumar
On Wed, Jun 5, 2024 at 10:59 PM Matthias van de Meent wrote: > > On Wed, 5 Jun 2024 at 18:47, Ranier Vilela wrote: > > > > Em ter., 4 de jun. de 2024 às 16:39, Nathan Bossart > > escreveu: > >> > >> I noticed that the "Restoring database schemas in the new cluster" part of > >> pg_upgrade can t

Re: cannot drop intarray extension

2024-06-06 Thread Michael Paquier
On Fri, Jun 07, 2024 at 11:32:14AM +0800, jian he wrote: > in deleteObjectsInList, under certain conditions trying to sort the to > be deleted object list > by just using sort_object_addresses seems to work, > but it looks like a hack. > maybe the proper fix would be in findDependentObjects. @@ -1

PgStat_KindInfo.named_on_disk not required in shared stats

2024-06-06 Thread Michael Paquier
Hi all, (Relevant folks in CC.) While hacking on the area of pgstat_*.c, I have noticed the existence of named_on_disk in PgStat_KindInfo, that is here to track the fact that replication slots are a particular case in the PgStat_HashKey for the dshash table of the stats because this kind of stats

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 7:39 PM Ashutosh Sharma wrote: > > On Thu, Jun 6, 2024 at 6:20 PM Dilip Kumar wrote: >> >> On Thu, Jun 6, 2024 at 5:59 PM Ashutosh Sharma wrote: >> > >> > Hello everyone, >> > >> > At present, we use MVCC snapshots to identify dependent objects. This >> > implies that if

Re: speed up a logical replica setup

2024-06-06 Thread Euler Taveira
On Wed, May 22, 2024, at 12:16 PM, Euler Taveira wrote: > I'll summarize all issues as soon as I finish the review of sync slot > support. I > think we should avoid new development if we judge that the item can be > documented as a limitation for this version. Nevertheless, I will share > patches

Re: relfilenode statistics

2024-06-06 Thread Andres Freund
Hi, On 2024-06-03 11:11:46 +, Bertrand Drouvot wrote: > The main argument is that we currently don’t have writes counters for > relations. > The reason is that we don’t have the relation OID when writing buffers out. > Tracking writes per relfilenode would allow us to track/consolidate writes

Re: cannot drop intarray extension

2024-06-06 Thread jian he
On Mon, Jun 3, 2024 at 12:14 PM jian he wrote: > > hi. > > setup > drop table if exist test__int cascade; > create extension intarray; > > CREATE TABLE test__int( a int[] ); > CREATE INDEX text_idx on test__int using gist (a gist__intbig_ops(siglen = > 1)); > drop extension intarray cascade;

Re: relfilenode statistics

2024-06-06 Thread Andres Freund
Hi, On 2024-06-06 12:27:49 -0400, Robert Haas wrote: > On Wed, Jun 5, 2024 at 1:52 AM Bertrand Drouvot > wrote: > > I think we should keep the stats in the relation during relfilenode changes. > > As a POC, v1 implemented a way to do so during TRUNCATE (see the changes in > > table_relation_set_n

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Thomas Munro
On Fri, Jun 7, 2024 at 3:00 PM Alexander Lakhin wrote: > My bisect run ended with: > 210622c60e1a9db2e2730140b8106ab57d259d15 is the first bad commit > > Author: Thomas Munro > Date: Wed Apr 3 00:03:08 2024 +1300 > > Provide vectored variant of ReadBuffer(). > > Other buildfarm failures wi

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Alexander Lakhin
Hello Noah, 06.06.2024 22:07, Noah Misch wrote: I don't know, but if the locks are really missing now, I feel like the first question is "which commit got rid of them?". It's a little hard to believe that they've never been there and somehow nobody has noticed. Then again, maybe we have; see

RE: Synchronizing slots from primary to standby

2024-06-06 Thread Zhijie Hou (Fujitsu)
On Thursday, June 6, 2024 12:21 PM Peter Smith > > Hi, here are some review comments for the docs patch v5-0001. Thanks for the comments! Here is the V6 patch that addressed the these. Best Regards, Hou zj v6-0001-Document-the-steps-to-check-if-the-standby-is-rea.patch Description: v6-0001-D

Re: Logical Replication of sequences

2024-06-06 Thread Masahiko Sawada
On Thu, Jun 6, 2024 at 6:40 PM Amit Kapila wrote: > > On Thu, Jun 6, 2024 at 11:10 AM Masahiko Sawada wrote: > > > > On Wed, Jun 5, 2024 at 9:30 PM Amit Kapila wrote: > > > > > > > > To achieve this, we can allow sequences to be copied during > > > the initial CREATE SUBSCRIPTION command similar

Re: question regarding policy for patches to out-of-support branches

2024-06-06 Thread Tom Lane
Joe Conway writes: > On 6/6/24 14:12, Tom Lane wrote: >> To get a sense of how much of a problem we have, I grepped the git >> history for comments mentioning project policies. Ignoring ones >> that are really talking about very localized issues, what I found >> is attached. It seems like it's l

Re: Revive num_dead_tuples column of pg_stat_progress_vacuum

2024-06-06 Thread Masahiko Sawada
On Wed, Jun 5, 2024 at 7:19 PM Andrey M. Borodin wrote: > > > > > On 4 Jun 2024, at 00:26, Masahiko Sawada wrote: > > Thank you! Vacuum enhancement is a really good step forward, and this small > change would help a lot of observability tools. > > > > On 4 Jun 2024, at 00:49, Peter Geoghegan wr

Re: report on not thread-safe functions

2024-06-06 Thread Jeff Davis
On Thu, 2024-06-06 at 16:34 +0200, Peter Eisentraut wrote: > - setlocale() > > The locale business needs to be reworked to use locale_t and _l > functions.  This is already being discussed for other reasons. I posted a few patches to do this for collation: https://commitfest.postgresql.org/48/50

Re: Injection points: preloading and runtime arguments

2024-06-06 Thread Michael Paquier
On Thu, Jun 06, 2024 at 03:47:47PM +0500, Andrey M. Borodin wrote: > Is it OK to detach() before wakeup()? Or, perhaps, can a detach() do a > wakeup() automatically? It is OK to do a detach before a wakeup. Noah has been relying on this behavior in an isolation test for a patch he's worked on.

Re: race condition in pg_class

2024-06-06 Thread Michael Paquier
On Thu, Jun 06, 2024 at 09:48:51AM -0400, Robert Haas wrote: > It's not this patch set's fault, but I'm not very pleased to see that > the injection point wait events have been shoehorned into the > "Extension" category - which they are not - instead of being a new > wait_event_type. That would hav

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Jeff Davis
On Fri, 2024-06-07 at 00:19 +0200, Jelte Fennema-Nio wrote: > Even by default making the search_path "pg_catalog, pg_temp" for > functions created by extensions would be very useful. Right now there's no syntax to override that. We'd need something to say "get the search_path from the session". R

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 20:10, Isaac Morland wrote: > > On Thu, 6 Jun 2024 at 12:53, Jeff Davis wrote: > >> >> > I didn't get you completely here. w.r.t extensions how will this have >> > an impact if we set the search_path for definer functions. >> >> If we only set the search path for SECURITY DE

Re: ssl tests fail due to TCP port conflict

2024-06-06 Thread Jelte Fennema-Nio
On Wed, 5 Jun 2024 at 23:37, Tom Lane wrote: > > Andrew Dunstan writes: > > On 2024-06-05 We 16:00, Alexander Lakhin wrote: > >> That is, psql from the test instance 001_ssltests_34 opened a > >> connection to > >> the test server with the client port 50072 and it made using the port by > >> the

Re: Things I don't like about \du's "Attributes" column

2024-06-06 Thread Pavel Luzanov
On 06.06.2024 17:29, Robert Haas wrote: I think the first of these special interpretations is unnecessary and should be removed. It seems pretty clear what 0 means. Agree. There is an additional technical argument for removing this replacement. I don't like explicit cast to text of the "Connect

Re: Postgresql OOM

2024-06-06 Thread Tom Lane
Radu Radutiu writes: > The query itself runs fine in a reasonable time with enable_parallel_hash = > 'off'. I see two problems - one is the wrong execution plan (right after > running analyze), the second and the most important is the huge memory > usage (far exceeding work_mem and shared buffers)

Re: small fix for llvm build

2024-06-06 Thread Peter Eisentraut
On 28.05.24 17:17, Peter Eisentraut wrote: I'm getting build failures when building with meson and llvm enabled, like this: [1/112] Generating src/backend/jit/llvm/llvmjit_types.bc with a custom command FAILED: src/backend/jit/llvm/llvmjit_types.bc /usr/local/bin/ccache /usr/local/Cellar/llvm

Re: Avoid orphaned objects dependencies, take 3

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 1:56 AM Bertrand Drouvot wrote: > v9 is more invasive (as it changes code in much more places) than v8 but it is > easier to follow (as it is now clear where the new lock is acquired). Hmm, this definitely isn't what I had in mind. Possibly that's a sign that what I had in

Re: question regarding policy for patches to out-of-support branches

2024-06-06 Thread Joe Conway
On 6/6/24 14:12, Tom Lane wrote: Robert Haas writes: On Thu, Jun 6, 2024 at 4:25 AM Hannu Krosing wrote: Not absolutely sure, but would at least adding a page to PostgreSQL Wiki about this make sense ? I feel like we need to do something. Tom says this is a policy, and he's made that comme

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-06 Thread Nathan Bossart
On Thu, Jun 06, 2024 at 03:31:53PM -0400, Robert Haas wrote: > I don't really like making this a GUC, but what's the other option? > It's reasonable for people to want to ask the server how many > resources it will need to start, and -C is the only tool we have for > that right now. So I feel like

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Julien Tachoires
Le jeu. 6 juin 2024 à 07:24, Alvaro Herrera a écrit : > > On 2024-Jun-06, Amit Kapila wrote: > > > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > > > When the content of a large transaction (size exceeding > > > logical_decoding_work_mem) and its sub-transactions has to be > > > r

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 3:21 PM Nathan Bossart wrote: > Here is a rebased version of the patch for v18 that adds a runtime-computed > GUC. As I noted earlier, there still isn't a consensus on this approach. I don't really like making this a GUC, but what's the other option? It's reasonable for pe

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Julien Tachoires
Le jeu. 6 juin 2024 à 06:40, Amit Kapila a écrit : > > On Thu, Jun 6, 2024 at 6:22 PM Julien Tachoires wrote: > > > > Le jeu. 6 juin 2024 à 04:13, Amit Kapila a écrit : > > > > > > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > > > > > When the content of a large transaction (si

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 18:01, Robert Haas wrote: > As I see it, the issue here is whether the default value would ever be > different from the latest value. If not, then using blank to mean the > latest seems fine, but if so, then I'd expect blank to mean the > default version and latest to mean th

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-06 Thread Nathan Bossart
Here is a rebased version of the patch for v18 that adds a runtime-computed GUC. As I noted earlier, there still isn't a consensus on this approach. -- nathan >From 74f638f7df9c51f5ab47b282bb7107c4ba6cb5b6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 21 May 2024 14:02:22 -0500 Subje

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Noah Misch
On Thu, Jun 06, 2024 at 12:36:32PM -0400, Robert Haas wrote: > On Thu, Jun 6, 2024 at 6:00 AM Alexander Lakhin wrote: > > Am I missing something or the the page buffer indeed lacks locking there? > > I don't know, but if the locks are really missing now, I feel like the > first question is "which

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2024-06-06 Thread Andrew Dunstan
On 2024-06-06 Th 04:15, Kyotaro Horiguchi wrote: At Thu, 06 Jun 2024 16:45:00 +0900 (JST), Kyotaro Horiguchi wrote in I have been thinking about this since then. At first, I thought it referred to FindFirstChangeNotification() and friends, and inotify on Linux. However, I haven't found a way

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Alexander Lakhin
Hello Robert, 06.06.2024 19:36, Robert Haas wrote: On Thu, Jun 6, 2024 at 6:00 AM Alexander Lakhin wrote: Am I missing something or the the page buffer indeed lacks locking there? I don't know, but if the locks are really missing now, I feel like the first question is "which commit got rid of

Re: tiny step toward threading: reduce dependence on setlocale()

2024-06-06 Thread Jeff Davis
On Wed, 2024-06-05 at 17:23 -0700, Jeff Davis wrote: > A brief test shows that there may be a performance regression for > libc > default collations. But if so, I'm not sure that's avoidable if the > goal is to take away setlocale. I'll see if removing the extra > branches > mitigates it. I redid

Re: question regarding policy for patches to out-of-support branches

2024-06-06 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 6, 2024 at 4:25 AM Hannu Krosing wrote: >> Not absolutely sure, but would at least adding a page to PostgreSQL >> Wiki about this make sense ? > I feel like we need to do something. Tom says this is a policy, and > he's made that comment before about other thing

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Isaac Morland
On Thu, 6 Jun 2024 at 12:53, Jeff Davis wrote: > > I didn't get you completely here. w.r.t extensions how will this have > > an impact if we set the search_path for definer functions. > > If we only set the search path for SECURITY DEFINER functions, I don't > think that solves the whole problem

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Jeff Davis
On Thu, 2024-06-06 at 21:17 +0530, Ashutosh Sharma wrote: > That can be controlled via some GUC if needed, I guess. That's a possibility, but it's easy to create a mess that way. I don't necessarily oppose it, but we'd need some pretty strong agreement that we are somehow moving users in a better

Re: Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 6:00 AM Alexander Lakhin wrote: > Am I missing something or the the page buffer indeed lacks locking there? I don't know, but if the locks are really missing now, I feel like the first question is "which commit got rid of them?". It's a little hard to believe that they've n

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-06-06 Thread Robert Haas
On Wed, Jun 5, 2024 at 3:48 AM Ashutosh Bapat wrote: > Here's planning time measurements. > num_joins | master (ms) | patched (ms) | change in planning time (ms) | > change in planning time > ---+-+--+--+

Re: relfilenode statistics

2024-06-06 Thread Robert Haas
On Wed, Jun 5, 2024 at 1:52 AM Bertrand Drouvot wrote: > I think we should keep the stats in the relation during relfilenode changes. > As a POC, v1 implemented a way to do so during TRUNCATE (see the changes in > table_relation_set_new_filelocator() and in pg_statio_all_tables): as you can > see

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 5:12 AM Jelte Fennema-Nio wrote: > Looking at ssl_max_protocol_version closer though, to stay really > consistent I'd have to change "latest" to be renamed to empty string > (i.e. there is no max_protocol_version). I think I might prefer > staying consistent over introducing

Re: ResourceOwner refactoring

2024-06-06 Thread Heikki Linnakangas
On 06/06/2024 18:27, Robert Haas wrote: On Thu, Jun 6, 2024 at 7:32 AM Heikki Linnakangas wrote: Barring objections, I'll commit this later today or tomorrow. Thanks for the report! Committed. I think you may have forgotten to push. Huh, you're right. I could swear I did... Pushed now th

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-06 Thread Ashutosh Sharma
Hi, On Thu, Jun 6, 2024 at 2:36 AM Jeff Davis wrote: > On Wed, 2024-06-05 at 14:36 +0530, Ashutosh Sharma wrote: > > Thank you, Ashutosh, for the quick response. I've drafted a patch > > aimed at addressing this issue. The patch attempts to solve this > > issue by configuring the search_path for

Re: question regarding policy for patches to out-of-support branches

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 4:25 AM Hannu Krosing wrote: > Not absolutely sure, but would at least adding a page to PostgreSQL > Wiki about this make sense ? I feel like we need to do something. Tom says this is a policy, and he's made that comment before about other things, but the fact that they're

Re: ResourceOwner refactoring

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 7:32 AM Heikki Linnakangas wrote: > > Barring objections, I'll commit this later today or tomorrow. Thanks for > > the report! > > Committed. I think you may have forgotten to push. -- Robert Haas EDB: http://www.enterprisedb.com

report on not thread-safe functions

2024-06-06 Thread Peter Eisentraut
In the context of the multithreaded-server project, I looked into potentially not thread-safe functions. (See proposed next steps at the end of this message.) Here is a list of functions in POSIX that are possibly not thread-safe: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_ch

Re: [multithreading] extension compatibility

2024-06-06 Thread Heikki Linnakangas
On 06/06/2024 17:23, Robert Haas wrote: On Thu, Jun 6, 2024 at 5:00 AM Heikki Linnakangas wrote: If there is some material harm from compiling with multithreading support even if you're not using it, we should try to fix that. I'm not dead set against having a compile-time option, but I don't s

Re: Things I don't like about \du's "Attributes" column

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 5:08 AM Pavel Luzanov wrote: > But now there are no changes in pg_roles. Just a special interpretation > of the two values of the "Connection limit" column: > 0 - Now allowed (changed from 'No connections') > -1 - empty string I think the first of these special interpret

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Alvaro Herrera
On 2024-Jun-06, Amit Kapila wrote: > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > When the content of a large transaction (size exceeding > > logical_decoding_work_mem) and its sub-transactions has to be > > reordered during logical decoding, then, all the changes are written >

Re: [multithreading] extension compatibility

2024-06-06 Thread Robert Haas
On Thu, Jun 6, 2024 at 5:00 AM Heikki Linnakangas wrote: > If there is some material harm from compiling with multithreading > support even if you're not using it, we should try to fix that. I'm not > dead set against having a compile-time option, but I don't see the need > for it at the moment.

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Ashutosh Sharma
On Thu, Jun 6, 2024 at 6:57 PM Bertrand Drouvot < bertranddrouvot...@gmail.com> wrote: > On Thu, Jun 06, 2024 at 05:59:00PM +0530, Ashutosh Sharma wrote: > > Hello everyone, > > > > At present, we use MVCC snapshots to identify dependent objects. This > > implies that if a new dependent object is

Re: Postgresql OOM

2024-06-06 Thread Radu Radutiu
> > >> I am not qualified to answer on the OOM issue but why are you joining the > same table (outputrequest) 4 times (using an identical join condition)? > This essentially does a cross join, if an input_sequence value has say, > 1000 related rows in outputrequest, you will be getting 1000^4 rows

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Ashutosh Sharma
On Thu, Jun 6, 2024 at 6:20 PM Dilip Kumar wrote: > On Thu, Jun 6, 2024 at 5:59 PM Ashutosh Sharma > wrote: > > > > Hello everyone, > > > > At present, we use MVCC snapshots to identify dependent objects. This > implies that if a new dependent object is inserted within a transaction > that is st

Re: race condition in pg_class

2024-06-06 Thread Robert Haas
On Wed, Jun 5, 2024 at 2:17 PM Noah Misch wrote: > Starting 2024-06-10, I plan to push the first seven of the ten patches: > > inplace005-UNEXPECTEDPASS-tap-meson-v1.patch > inplace010-tests-v1.patch > inplace040-waitfuncs-v1.patch > inplace050-tests-inj-v1.patch > inplace060-nodeModifyTable-comme

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Amit Kapila
On Thu, Jun 6, 2024 at 6:22 PM Julien Tachoires wrote: > > Le jeu. 6 juin 2024 à 04:13, Amit Kapila a écrit : > > > > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > > > When the content of a large transaction (size exceeding > > > logical_decoding_work_mem) and its sub-transactio

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Bertrand Drouvot
On Thu, Jun 06, 2024 at 05:59:00PM +0530, Ashutosh Sharma wrote: > Hello everyone, > > At present, we use MVCC snapshots to identify dependent objects. This > implies that if a new dependent object is inserted within a transaction > that is still ongoing, our search for dependent objects won't inc

Re: Postgresql OOM

2024-06-06 Thread Pantelis Theodosiou
On Thu, Jun 6, 2024 at 1:25 PM Radu Radutiu wrote: > Hello all, > > I have a query that forces an out of memory error, where the OS will kill > the postgresql process. > The query plan (run immediately after a vacuum analyze) is at > https://explain.depesz.com/s/ITQI#html . > > ... > > Any idea

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Julien Tachoires
Le jeu. 6 juin 2024 à 04:13, Amit Kapila a écrit : > > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > When the content of a large transaction (size exceeding > > logical_decoding_work_mem) and its sub-transactions has to be > > reordered during logical decoding, then, all the chan

Re: How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 5:59 PM Ashutosh Sharma wrote: > > Hello everyone, > > At present, we use MVCC snapshots to identify dependent objects. This implies > that if a new dependent object is inserted within a transaction that is still > ongoing, our search for dependent objects won't include th

Re: Logical Replication of sequences

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 9:34 AM Amit Kapila wrote: > > On Wed, Jun 5, 2024 at 3:17 PM Bharath Rupireddy > wrote: > > > > On Tue, Jun 4, 2024 at 5:40 PM Amit Kapila wrote: > > > > > > Even if we decode it periodically (say each time we decode the > > > checkpoint record) then also we need to send

Re: Proposal: Job Scheduler

2024-06-06 Thread Dmitry Dolgov
> On Thu, Jun 06, 2024 at 12:53:38PM GMT, Alvaro Herrera wrote: > On 2024-Jun-06, Dave Page wrote: > > > It's this kind of choice that means it's unlikely we'd include any one > > option in PostgreSQL, much like various other tools such as failover > > managers or poolers. > > TBH I see that more a

How about using dirty snapshots to locate dependent objects?

2024-06-06 Thread Ashutosh Sharma
Hello everyone, At present, we use MVCC snapshots to identify dependent objects. This implies that if a new dependent object is inserted within a transaction that is still ongoing, our search for dependent objects won't include this recently added one. Consequently, if someone attempts to drop the

Postgresql OOM

2024-06-06 Thread Radu Radutiu
Hello all, I have a query that forces an out of memory error, where the OS will kill the postgresql process. The query plan (run immediately after a vacuum analyze) is at https://explain.depesz.com/s/ITQI#html . PostgreSQL version 16.3, running on RHEL 8.9, 16 vCPU, 64 GB RAM, 32 GB swap shared

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 4:43 PM Amit Kapila wrote: > > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > > > When the content of a large transaction (size exceeding > > logical_decoding_work_mem) and its sub-transactions has to be > > reordered during logical decoding, then, all the chang

Re: Conflict Detection and Resolution

2024-06-06 Thread Nisha Moond
On Wed, Jun 5, 2024 at 7:29 PM Dilip Kumar wrote: > > On Tue, Jun 4, 2024 at 9:37 AM Amit Kapila wrote: > > > > Can you share the use case of "earliest_timestamp_wins" resolution > > method? It seems after the initial update on the local node, it will > > never allow remote update to succeed whic

Re: ResourceOwner refactoring

2024-06-06 Thread Heikki Linnakangas
On 05/06/2024 16:58, Heikki Linnakangas wrote: On 04/06/2024 01:49, Heikki Linnakangas wrote: A straightforward fix is to modify RelationFlushRelation() so that if !IsTransactionState(), it just marks the entry as invalid instead of calling RelationClearRelation(). That's what RelationClearRelat

Re: ssl tests fail due to TCP port conflict

2024-06-06 Thread Andrew Dunstan
On 2024-06-05 We 16:00, Alexander Lakhin wrote: Hello Andrew, 05.06.2024 21:10, Andrew Dunstan wrote: I think I see what's going on here. It looks like it's because we start the server in unix socket mode, and then switch to using TCP as well. Can you try your test with this patch applie

Re: Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Amit Kapila
On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote: > > When the content of a large transaction (size exceeding > logical_decoding_work_mem) and its sub-transactions has to be > reordered during logical decoding, then, all the changes are written > on disk in temporary files located in pg_repls

Re: Function and Procedure with same signature?

2024-06-06 Thread Hannu Krosing
Hi Peter and Tom Following up on our conversation art pgcon.dev If I understood correctly Peter has some old patch for splitting the namespaces which could be resurrected to try to move forward on this ? Can you share what you did there ? Also, while at it we should extend the function lookup t

Re: Logical Replication of sequences

2024-06-06 Thread Amit Kapila
On Thu, Jun 6, 2024 at 3:44 PM Ashutosh Bapat wrote: > > On Thu, Jun 6, 2024 at 9:22 AM Amit Kapila wrote: >> >> On Wed, Jun 5, 2024 at 6:01 PM Ashutosh Bapat >> wrote: >> > >> > On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila wrote: >> >> >> >> How about periodically sending this information? >> >

Compress ReorderBuffer spill files using LZ4

2024-06-06 Thread Julien Tachoires
Hi, When the content of a large transaction (size exceeding logical_decoding_work_mem) and its sub-transactions has to be reordered during logical decoding, then, all the changes are written on disk in temporary files located in pg_replslot/. Decoding very large transactions by multiple replicatio

Re: Proposal: Job Scheduler

2024-06-06 Thread Alvaro Herrera
On 2024-Jun-06, Dave Page wrote: > It's this kind of choice that means it's unlikely we'd include any one > option in PostgreSQL, much like various other tools such as failover > managers or poolers. TBH I see that more as a bug than as a feature, and I see the fact that there are so many schedul

Re: Injection points: preloading and runtime arguments

2024-06-06 Thread Andrey M. Borodin
> On 5 Jun 2024, at 03:52, Michael Paquier wrote: > > Another thing you could do is to define a > INJECTION_POINT_LOAD() in the code path you're stressing outside the > critical section where the point is run. This should save from a call > to the SQL function. This choice is up to the one

Re: Remove dependency on VacuumPage(Hit/Miss/Dirty) counters in do_analyze_rel

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 3:23 PM Anthonin Bonnefoy wrote: > > Hi, > > I sent a similar patch for this in > https://www.postgresql.org/message-id/flat/cao6_xqr__kttclkftqs0qscm-j7_xbrg3ge2rwhucxqjmjh...@mail.gmail.com Okay, I see, In that case, we can just discard mine, thanks for notifying me. -

Re: Conflict Detection and Resolution

2024-06-06 Thread Dilip Kumar
On Thu, Jun 6, 2024 at 3:43 PM Amit Kapila wrote: > > On Wed, Jun 5, 2024 at 7:29 PM Dilip Kumar wrote: > > > > On Tue, Jun 4, 2024 at 9:37 AM Amit Kapila wrote: > > > > > > Can you share the use case of "earliest_timestamp_wins" resolution > > > method? It seems after the initial update on the

Re: Logical Replication of sequences

2024-06-06 Thread Ashutosh Bapat
On Thu, Jun 6, 2024 at 9:22 AM Amit Kapila wrote: > On Wed, Jun 5, 2024 at 6:01 PM Ashutosh Bapat > wrote: > > > > On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila > wrote: > >> > >> How about periodically sending this information? > >> > > >> > >> Now, if we want to support some sort of failover th

Re: Conflict Detection and Resolution

2024-06-06 Thread Amit Kapila
On Wed, Jun 5, 2024 at 7:29 PM Dilip Kumar wrote: > > On Tue, Jun 4, 2024 at 9:37 AM Amit Kapila wrote: > > > > Can you share the use case of "earliest_timestamp_wins" resolution > > method? It seems after the initial update on the local node, it will > > never allow remote update to succeed whic

Assert in heapgettup_pagemode() fails due to underlying buffer change

2024-06-06 Thread Alexander Lakhin
Hello hackers, I tried to investigate a recent buildfarm test failure: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder&dt=2024-06-04%2003%3A27%3A47  29/295 postgresql:recovery / recovery/026_overwrite_contrecord ERROR    39.55s   exit status 32 log/026_overwrite_contrecor

Re: Remove dependency on VacuumPage(Hit/Miss/Dirty) counters in do_analyze_rel

2024-06-06 Thread Anthonin Bonnefoy
Hi, I sent a similar patch for this in https://www.postgresql.org/message-id/flat/cao6_xqr__kttclkftqs0qscm-j7_xbrg3ge2rwhucxqjmjh...@mail.gmail.com Regards, Anthonin On Thu, Jun 6, 2024 at 11:10 AM Dilip Kumar wrote: > As part of commit 5cd72cc0c5017a9d4de8b5d465a75946da5abd1d, the > dependen

Re: Logical Replication of sequences

2024-06-06 Thread Amit Kapila
On Thu, Jun 6, 2024 at 11:10 AM Masahiko Sawada wrote: > > On Wed, Jun 5, 2024 at 9:30 PM Amit Kapila wrote: > > > > > To achieve this, we can allow sequences to be copied during > > the initial CREATE SUBSCRIPTION command similar to what we do for > > tables. And then later by new/existing comma

Re: Proposal: Job Scheduler

2024-06-06 Thread Andrei Lepikhov
On 6/6/2024 16:04, Wang Cheng wrote: Noted. Thanks for suggestions. We will open-source it as an extension. It would be nice! `For me doesn't matter where to contribute: to PostgreSQL core or to its extension if it is published under BSD license. -- regards, Andrei Lepikhov

Incorrect matching of sql/json PASSING variable names

2024-06-06 Thread Amit Langote
Hi, Alvaro reported off-list that the following should really fail, because the jsonpath expression refers to a PASSING variable that doesn't exist: select json_query('"1"', jsonpath '$xy' passing 2 AS xyz); json_query 2 (1 row) This works because of a bug in GetJsonPathVar() wher

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-06-06 Thread Jelte Fennema-Nio
On Thu, 6 Jun 2024 at 03:03, Robert Haas wrote: > This makes some sense to me. I don't think that I believe > max_protocol_version=latest is future-proof: just because I want to > opt into this round of new features doesn't mean I feel the same way > about the next round. But it may still be a goo

Remove dependency on VacuumPage(Hit/Miss/Dirty) counters in do_analyze_rel

2024-06-06 Thread Dilip Kumar
As part of commit 5cd72cc0c5017a9d4de8b5d465a75946da5abd1d, the dependency on global counters such as VacuumPage(Hit/Miss/Dirty) was removed from the vacuum. However, do_analyze_rel() was still using these counters, necessitating the tracking of global counters alongside BufferUsage counters. The

Re: Things I don't like about \du's "Attributes" column

2024-06-06 Thread Pavel Luzanov
On 16.04.2024 09:15, Pavel Luzanov wrote: On 16.04.2024 01:06, David G. Johnston wrote: At this point I'm on board with retaining the \dr charter of simply being an easy way to access the detail exposed in pg_roles with some display formatting but without any attempt to convey how the system use

Re: Proposal: Job Scheduler

2024-06-06 Thread Wang Cheng
Noted. Thanks for suggestions. We will open-source it as an extension. Regards, Cheng   -- Original -- From: "Dave Page"

Re: [multithreading] extension compatibility

2024-06-06 Thread Heikki Linnakangas
On 06/06/2024 05:47, Robert Haas wrote: On Wed, Jun 5, 2024 at 10:09 PM Andres Freund wrote: Maybe. I think shipping a mode where users can fairly simply toggle between threaded and process mode will allow us to get this stable a *lot* quicker than if we distribute two builds. Most users don't

Re: Proposal: Job Scheduler

2024-06-06 Thread Dave Page
On Thu, 6 Jun 2024 at 09:47, Laurenz Albe wrote: > On Thu, 2024-06-06 at 16:27 +0800, Wang Cheng wrote: > > We are the PostgreSQL team in Tencent. We have recently developed a job > scheduler > > that runs inside the database to schedules and manages jobs similar to > Oracle > > DBMS_JOB package,

Re: Proposal: Job Scheduler

2024-06-06 Thread Laurenz Albe
On Thu, 2024-06-06 at 16:27 +0800, Wang Cheng wrote: > We are the PostgreSQL team in Tencent. We have recently developed a job > scheduler > that runs inside the database to schedules and manages jobs similar to Oracle > DBMS_JOB package, and we would like to contribute this feature to the > comm

Proposal: Job Scheduler

2024-06-06 Thread Wang Cheng
Hackers, We are the PostgreSQL team in Tencent. We have recently developed a job scheduler that runs inside the database to schedules and manages jobs similar to Oracle DBMS_JOB package, and we would like to contribute this feature to the community. Similar to autovacuum, the job scheduler c

Re: question regarding policy for patches to out-of-support branches

2024-06-06 Thread Hannu Krosing
On Wed, Jun 5, 2024 at 8:29 PM Tom Lane wrote: > > Joe Conway writes: > > I was having a discussion regarding out-of-support branches and effort > > to keep them building, but could not for the life of me find any actual > > documented policy (although I distinctly remember that we do something..

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2024-06-06 Thread Kyotaro Horiguchi
At Thu, 06 Jun 2024 17:15:15 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 06 Jun 2024 16:45:00 +0900 (JST), Kyotaro Horiguchi > wrote in > > I have been thinking about this since then. At first, I thought it > > referred to FindFirstChangeNotification() and friends, and inotify on > > Li

  1   2   >