Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Peter Smith
On Mon, Mar 6, 2023 at 5:44 PM Amit Kapila wrote: > > On Mon, Mar 6, 2023 at 10:12 AM Peter Smith wrote: > > > > 4. IdxIsRelationIdentityOrPK > > > > +/* > > + * Given a relation and OID of an index, returns true if the > > + * index is relation's replica identity index or relation's > > + * prim

Re: Add support for unit "B" to pg_size_pretty()

2023-03-06 Thread Peter Eisentraut
On 02.03.23 20:58, David Rowley wrote: On Mon, 27 Feb 2023 at 21:34, Peter Eisentraut wrote: On 22.02.23 03:39, David Rowley wrote: I think you'll need to find another way to make the aliases work. Maybe another array with the name and an int to reference the corresponding index in size_prett

Re: using memoize in in paralel query decreases performance

2023-03-06 Thread David Rowley
On Mon, 6 Mar 2023 at 20:34, Pavel Stehule wrote: > In one query I can see very big overhead of memoize node - unfortunately with > hits = 0 > > The Estimate is almost very good. See details in attachment Are you able to share the version number for this? Also, it would be good to see EXPLAIN A

Re: Simplify standby state machine a bit in WaitForWALToBecomeAvailable()

2023-03-06 Thread Bharath Rupireddy
On Mon, Mar 6, 2023 at 1:26 PM Michael Paquier wrote: > > On Sat, Mar 04, 2023 at 09:47:05AM +0530, Bharath Rupireddy wrote: > > Okay, here's a patch attached. > > Thanks. > > + * When source == XLOG_FROM_ANY, this function first searches for the segment > + * with a TLI in archive first, if not f

Re: Add support for unit "B" to pg_size_pretty()

2023-03-06 Thread David Rowley
On Mon, 6 Mar 2023 at 21:13, Peter Eisentraut wrote: > > On 02.03.23 20:58, David Rowley wrote: > > I think I'd prefer to see the size_bytes_unit_alias struct have an > > index into size_pretty_units[] array. i.e: > > Ok, done that way. (I had thought about that, but I was worried that > that wou

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-06 Thread Alexander Kukushkin
Hello Soumyadeep, The problem indeed exists, but IMO the "log" directory case must be handled differently: 1. We don't need or I would even say we don't want to sync log files from the new primary, because it destroys the actual logs, which could be very important to figure out what has happened w

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Mon, Mar 6, 2023 at 1:40 PM Peter Smith wrote: > > On Mon, Mar 6, 2023 at 5:44 PM Amit Kapila wrote: > > > > On Mon, Mar 6, 2023 at 10:12 AM Peter Smith wrote: > > > > > > 4. IdxIsRelationIdentityOrPK > > > > > > +/* > > > + * Given a relation and OID of an index, returns true if the > > > +

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Julien Rouhaud
On Wed, Mar 01, 2023 at 08:30:00PM +0530, Bharath Rupireddy wrote: > On Wed, Mar 1, 2023 at 1:00 PM Bharath Rupireddy > wrote: > > > > In a recent discussion [1], Michael Paquier asked if we can combine > > pg_walinspect till_end_of_wal functions with other functions > > pg_get_wal_records_info an

Re: using memoize in in paralel query decreases performance

2023-03-06 Thread Pavel Stehule
po 6. 3. 2023 v 9:16 odesílatel David Rowley napsal: > On Mon, 6 Mar 2023 at 20:34, Pavel Stehule > wrote: > > In one query I can see very big overhead of memoize node - unfortunately > with hits = 0 > > > > The Estimate is almost very good. See details in attachment > > Are you able to share th

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-06 Thread Daniel Gustafsson
> On 5 Mar 2023, at 00:04, Tom Lane wrote: > > Peter Eisentraut writes: >> [ v2-0001-Remove-incidental-md5-function-uses-from-main-reg.patch ] > > I've gone through this and have a modest suggestion: let's invent some > wrapper functions around encode(sha256()) to reduce the cosmetic diffs > an

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Önder Kalacı
Hi Amit, all > > > > Given Amit's suggestion on [1], I'm planning to drop this check > altogether, and > > rely on table storage parameters. > > > > This still seems to be present in the latest version. I think we can > just remove this and then add the additional check as suggested by you > as p

wrong results due to qual pushdown

2023-03-06 Thread tender wang
tender wang [image: 附件]14:51 (2小时前) 发送至 pgsql-hackers Hi hackers. This query has different result on 16devel and 15.2. select sample_3.n_regionkey as c0, ref_7.l_linenumber as c3, sample_4.l_quantity as c6, sample_5.n_nationkey as c7, sample_3.n_name as c8 from public.

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-03-06 Thread Kartyshov Ivan
Update patch to fix conflict with master -- Ivan Kartyshov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..422bb1ed82 100644 --- a/src/backend/access/t

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Fri, Mar 3, 2023 at 6:40 PM Önder Kalacı wrote: > > Hi Vignesh, > > Thanks for the review > >> >> 1) We are currently calling RelationGetIndexList twice, once in >> FindUsableIndexForReplicaIdentityFull function and in the caller too, >> we could avoid one of the calls by passing the indexlist

Re: [Proposal] Add foreign-server health checks infrastructure

2023-03-06 Thread Katsuragi Yuta
Hi Kuroda-san, Thank you for updating the patch! 4. the code of pqSocketPoll +#if defined(POLLRDHUP) + if (forConnCheck) + input_fd.events |= POLLRDHUP; +#endif I think it is better to use PQconnCheckable() to remove the macro. IIUC the macro is needed. In FreeBSD, macOS

RE: Support logical replication of DDLs

2023-03-06 Thread wangw.f...@fujitsu.com
On Mon, Mar 6, 2023 14:34 AM Ajin Cherian wrote: > Changes are in patch 1 and patch 2 Thanks for updating the patch set. Here are some comments: For v-75-0002* patch. 1. In the function deparse_AlterRelation. + if ((sub->address.objectId != relId && +sub->a

RE: Rework LogicalOutputPluginWriterUpdateProgress

2023-03-06 Thread wangw.f...@fujitsu.com
On Fri, Mar 3, 2023 8:18 AM Peter Smith wrote: > Thanks for your comments. > 1. > + > +static void UpdateProgressAndKeepalive(LogicalDecodingContext *ctx, > +bool finished_xact); > + > +static bool is_keepalive_threshold_exceeded(LogicalDecodingContext *ctx); > > 1a. > There is an unnecess

Re: Deduplicate logicalrep_read_tuple()

2023-03-06 Thread Amit Kapila
On Sun, Mar 5, 2023 at 1:22 PM Peter Smith wrote: > > On Fri, Mar 3, 2023 at 10:04 PM Amit Kapila wrote: > > > > > > > > Thanks. Done so in the attached v2. > > > > > > > LGTM. Unless Peter or someone has any comments on this, I'll push this > > early next week. > > > > No more comments. Patch v2

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Dilip Kumar
On Mon, Mar 6, 2023 at 2:38 PM Önder Kalacı wrote: > I was going through the thread and patch, I noticed that in the initial version, we were depending upon the planner to let it decide whether index scan is cheaper or not and which index to pick. But in the latest patch if a useful index exists

Re: [PATCH] Add CANONICAL option to xmlserialize

2023-03-06 Thread Jim Jones
On 06.03.23 00:32, Thomas Munro wrote: I couldn't reproduce that locally either, but I just tested on CI with your patch applied saw the failure, and then removed "PYTHONCOERCECLOCALE=0 LANG=C" and it's all green: https://github.com/macdice/postgres/commit/91999f5d13ac2df6f7237a301ed6cf73f2bb5b

Re: daitch_mokotoff module

2023-03-06 Thread Dag Lem
Dag Lem writes: > Tomas Vondra writes: > >> On 2/7/23 18:08, Paul Ramsey wrote: >>> >>> On Feb 7, 2023, at 6:47 AM, Dag Lem wrote: I just went by to check the status of the patch, and I noticed that you've added yourself as reviewer earlier - great! Please tell m

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Mon, Mar 6, 2023 at 4:18 PM Dilip Kumar wrote: > > On Mon, Mar 6, 2023 at 2:38 PM Önder Kalacı wrote: > > > I was going through the thread and patch, I noticed that in the > initial version, we were depending upon the planner to let it decide > whether index scan is cheaper or not and which i

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Önder Kalacı
Hi Amit, all > > > > > I think you've "simplified" this function in v28 but AFAICT now it has > > a different logic to v27. > > > > PREVIOUSLY it was coded like > > + return RelationGetReplicaIndex(rel) == idxoid || > > + RelationGetPrimaryKeyIndex(rel) == idxoid; > > > > You can see if 'idxoid'

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Önder Kalacı
Hi Peter, all > > 1. > A published table must have a replica identity configured in order to > be able to replicate UPDATE and DELETE operations, so that appropriate > rows to update or delete can be identified on the subscriber side. By > default, this is the primary key, if there is one. Anothe

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Önder Kalacı
Hi Amit, all Amit Kapila , 6 Mar 2023 Pzt, 12:40 tarihinde şunu yazdı: > On Fri, Mar 3, 2023 at 6:40 PM Önder Kalacı wrote: > > > > Hi Vignesh, > > > > Thanks for the review > > > >> > >> 1) We are currently calling RelationGetIndexList twice, once in > >> FindUsableIndexForReplicaIdentityFull

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Thu, Mar 2, 2023 at 2:45 PM Amit Kapila wrote: > > On Thu, Mar 2, 2023 at 1:37 PM shiy.f...@fujitsu.com > wrote: > > - results of `gprof` > > case1: > > master > > % cumulative self self total > > time seconds secondscalls ms/call ms/call name > > 1.37

RE: [Proposal] Add foreign-server health checks infrastructure

2023-03-06 Thread Hayato Kuroda (Fujitsu)
Dear Katsuragi-san, Thank you for reviewing! PSA new version. > >> 4. the code of pqSocketPoll > >> +#if defined(POLLRDHUP) > >> + if (forConnCheck) > >> + input_fd.events |= POLLRDHUP; > >> +#endif > >> > >> I think it is better to use PQconnCheckable() to remove the macro. > > > > IIU

Re: GUC for temporarily disabling event triggers

2023-03-06 Thread Daniel Gustafsson
> On 27 Jan 2023, at 15:00, Mikhail Gribkov wrote: > There is a complete framework for disabling various types of the event > triggers separately, but, the list of valid GUC values only include 'all' and > 'none'. Why not adding support for all the event trigger types separately? > Everything

Re: Normalization of utility queries in pg_stat_statements

2023-03-06 Thread Andrei Zubkov
Hi Michael! I'm rebasing a patch "Tracking statements entry timestamp in pg_stat_statements" for applying after this patch. I've noted that current tests are not quite independent one from another. There is two statements in the end of user_activity.sql test: DROP ROLE regress_stats_user1; DROP R

Re: Support logical replication of DDLs

2023-03-06 Thread vignesh C
On Mon, 6 Mar 2023 at 12:04, Ajin Cherian wrote: > > On Wed, Feb 15, 2023 at 3:33 PM Peter Smith wrote: > > > > > > > > > > 9. > > > > + > > > > +/* > > > > + * Append the parenthesized arguments of the given pg_proc row into > > > > the output > > > > + * buffer. force_qualify indicates whether

Re: [PATCH] Add CANONICAL option to xmlserialize

2023-03-06 Thread Jim Jones
On 06.03.23 11:50, I wrote: I guess this confusion is happening because xml_parse() was being called with the database encoding from GetDatabaseEncoding(). I added a condition before calling xml_parse() to check if the xml document has a different encoding than UTF-8 parse_xml_decl(xml_text2

Re: meson: Optionally disable installation of test modules

2023-03-06 Thread Nazir Bilal Yavuz
Hi, On Fri, 3 Mar 2023 at 22:43, Andrew Dunstan wrote: > These changes have broken the buildfarm adaptation work in different ways on > different platforms. > > On Windows (but not Linux), the install_test_files are apparently getting > installed under runpython in the build directory rather th

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Dilip Kumar
On Mon, Mar 6, 2023 at 4:45 PM Amit Kapila wrote: > > On Mon, Mar 6, 2023 at 4:18 PM Dilip Kumar wrote: > > > > On Mon, Mar 6, 2023 at 2:38 PM Önder Kalacı wrote: > > > > > I was going through the thread and patch, I noticed that in the > > initial version, we were depending upon the planner to

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-03-06 Thread Daniel Gustafsson
> On 28 Feb 2023, at 15:28, Damir Belyalov wrote: > Tested patch on all cases: CIM_SINGLE, CIM_MULTI, CIM_MULTI_CONDITION. As > expected it works. > Also added a description to copy.sgml and made a review on patch. > > I added 'ignored_errors' integer parameter that should be output after the

Re: wrong results due to qual pushdown

2023-03-06 Thread Ashutosh Bapat
On Mon, Mar 6, 2023 at 3:00 PM tender wang wrote: > tender wang > [image: 附件]14:51 (2小时前) > 发送至 pgsql-hackers > Hi hackers. >This query has different result on 16devel and 15.2. > select > sample_3.n_regionkey as c0, > ref_7.l_linenumber as c3, > sample_4.l_quantity as c6, >

Re: [BUG] pg_stat_statements and extended query protocol

2023-03-06 Thread Imseih (AWS), Sami
> Well, it is one of these areas where it seems to me we have never been > able to put a definition on what should be the correct behavior when > it comes to pg_stat_statements. What needs to be defined here is how pgss should account for # of rows processed when A) a select goes through extended

Re: Inaccurate comment for pg_get_partkeydef

2023-03-06 Thread Ashutosh Bapat
Looks good to me. Fixed according to the actual output. select pg_get_partkeydef('prt1'::regclass); pg_get_partkeydef --- RANGE (a) (1 row) bdrdemo@342511=#\d+ prt1 Partitioned table "public.prt1" Column | Type | Collation | Nullable | Def

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-06 Thread Jelte Fennema
Small update. Improved some wording in the docs. On Fri, 3 Mar 2023 at 15:37, Jelte Fennema wrote: > > > I want to note that the Fisher-Yates algorithm is implemented in a > > difficult to understand manner. > > +if (j < i) /* avoid fetching undefined data if j=i */ > > This stuff does not make s

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Bharath Rupireddy
On Thu, Mar 2, 2023 at 9:47 PM Melanie Plageman wrote: > > > However, I am mainly looking for feedback about whether or not others > > would find this useful, and, if so, what columns they would like to see > > in the returned tuplestore. IMO, pg_get_wal_records_extended_info as proposed doesn't

Re: wrong results due to qual pushdown

2023-03-06 Thread tender wang
Results on 16devel: c0 | c3 | c6 | c7 |c8 ++++--- 0 |||| ALGERIA 0 |||| ETHIOPIA 0 |||| KENYA 0 |||| MOROCCO 0 |||| MOZAMBIQUE 1 |||| ARGENTINA 1 |||

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-06 Thread Tom Lane
Daniel Gustafsson writes: >> On 5 Mar 2023, at 00:04, Tom Lane wrote: >> I've gone through this and have a modest suggestion: let's invent some >> wrapper functions around encode(sha256()) to reduce the cosmetic diffs >> and consequent need for closer study of patch changes. In the attached >> I

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Bharath Rupireddy
On Mon, Mar 6, 2023 at 2:22 PM Julien Rouhaud wrote: > > > > I'm attaching a patch doing the $subject with the following behavior: > > > 1. If start_lsn is NULL, error out/return NULL. > > Maybe naive and unrelated question, but is that really helpful? If for some > reason I want to see informati

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Matthias van de Meent
On Mon, 6 Mar 2023 at 15:40, Bharath Rupireddy wrote: > > On Thu, Mar 2, 2023 at 9:47 PM Melanie Plageman > wrote: > > > > > However, I am mainly looking for feedback about whether or not others > > > would find this useful, and, if so, what columns they would like to see > > > in the returned tu

Re: Fix comments in gistxlogDelete, xl_heap_freeze_page and xl_btree_delete

2023-03-06 Thread Robert Haas
On Sat, Mar 4, 2023 at 3:33 AM Drouvot, Bertrand wrote: > Indeed, for gistxlogDelete, that's the other way around (as > compare to what the commit message says). Woops. Good point. -- Robert Haas EDB: http://www.enterprisedb.com

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi wrote: > > At Mon, 06 Mar 2023 15:24:25 +0900 (JST), Kyotaro Horiguchi > wrote in > > In any case, I think we need to avoid such concurrent autovacuum/analyze. > > If it is correct, I believe the attached fix works. Thanks for investigating this!

Re: wrong results due to qual pushdown

2023-03-06 Thread Tom Lane
tender wang writes: > It looks wrong that the qual (e.g ((l_suppkey IS NOT NULL) OR > (getdatabaseencoding() !~~ CASE WHEN ($0 OR NULL::boolean) THEN NULL::text > ELSE NULL::text END))) is pushdown. I think this is the same issue reported at [1]. regards, tom lane [1] h

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Matthias van de Meent
On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy wrote: > If we try to make these functions figure out the oldest WAl file and > start from there, then it'll unnecessarily complicate the APIs and > functions. If we still think we need a better function for the users > to figure out the oldest WAL fi

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Bharath Rupireddy
On Mon, Mar 6, 2023 at 8:52 PM Matthias van de Meent wrote: > > On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy > wrote: > > If we try to make these functions figure out the oldest WAl file and > > start from there, then it'll unnecessarily complicate the APIs and > > functions. If we still think

Re: Add shared buffer hits to pg_stat_io

2023-03-06 Thread Melanie Plageman
Thanks for the review! On Tue, Feb 28, 2023 at 7:36 AM Drouvot, Bertrand wrote: > BufferDesc * > LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber > blockNum, > -bool *foundPtr, IOContext *io_context) > +bool *

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Matthias van de Meent
On Mon, 6 Mar 2023 at 16:37, Bharath Rupireddy wrote: > > On Mon, Mar 6, 2023 at 8:52 PM Matthias van de Meent > wrote: > > > > On Mon, 6 Mar 2023 at 16:06, Bharath Rupireddy > > wrote: > > > If we try to make these functions figure out the oldest WAl file and > > > start from there, then it'll

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-03-06 Thread John Naylor
On Mon, Mar 6, 2023 at 1:28 PM Masahiko Sawada wrote: > > Since the block-level measurement is likely overestimating quite a bit, I propose to simply reverse the order of the actions here, effectively reporting progress for the *last page* and not the current one: First update progress with the c

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 15:55, Tom Lane wrote: > Daniel Gustafsson writes: >> For readers without all context, wouldn't it be better to encode in the >> function name why we're not just calling a hash like md5? Something like >> fips_allowed_hash() or similar? > > I'd prefer shorter than that ---

Re: a very minor bug and a couple of comment changes for basebackup.c

2023-03-06 Thread Robert Haas
Thanks for the review. I have committed the patches. On Thu, Mar 2, 2023 at 2:59 AM Michael Paquier wrote: > Seems right, I think that you should backpatch that as > VERIFY_CHECKSUMS is the default. Done. > There is more to it: the page LSN is checked before its checksum. > Hence, if the page's

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-03-06 Thread Önder Kalacı
Hi all, Thanks for working on this. > I imagine that a typical use case would be to set min_send_delay to > several hours to days. I'm concerned that it could not be an > acceptable trade-off for many users that the system cannot collect any > garbage during that. > I'm not too worried about th

Re: Track IO times in pg_stat_io

2023-03-06 Thread Melanie Plageman
Thanks for the review! On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand wrote: > On 2/26/23 5:03 PM, Melanie Plageman wrote: > > As suggested in [1], the attached patch adds IO times to pg_stat_io; > > Thanks for the patch! > > I started to have a look at it and figured out that a tiny rebase wa

Re: Timeline ID hexadecimal format

2023-03-06 Thread Peter Eisentraut
On 03.03.23 16:52, Sébastien Lardière wrote: On 02/03/2023 09:12, Peter Eisentraut wrote: On 24.02.23 17:27, Sébastien Lardière wrote: diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index be05a33205..7e26b51031 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sg

Re: running logical replication as the subscription owner

2023-03-06 Thread Robert Haas
On Fri, Mar 3, 2023 at 6:57 PM Jelte Fennema wrote: > I'm definitely in favor of making it easier to use logical replication > in a safe manner. Cool. > In Citus we need to logically replicate and we're > currently using quite some nasty and undocumented hacks to do so: > We're creating a subscr

Re: running logical replication as the subscription owner

2023-03-06 Thread Jelte Fennema
> Yeah. As Andres pointed out somewhere or other, that also means you're > decoding the WAL once per user instead of just once. I'm surprised > that hasn't been cost-prohibitive. We'd definitely prefer to have one subscription and do the decoding only once. But we haven't run into big perf issues

Re: HOT chain validation in verify_heapam()

2023-03-06 Thread Robert Haas
On Thu, Feb 9, 2023 at 12:09 PM Himanshu Upadhyaya wrote: > Initially while implementing logic to identify the root of the HOT chain > I was getting crash and regression failure's that time I thought of having > this check along with a few other changes that were required, > but you are right, it'

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote: > That was mostly OK for me, so applied after tweaking a couple of > places in the tests (extra explanations, for one), the comments and > the code. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-06 Thread Soumyadeep Chakraborty
On Mon, Mar 6, 2023 at 12:28 AM Alexander Kukushkin wrote: > > Hello Soumyadeep, > > The problem indeed exists, but IMO the "log" directory case must be handled > differently: > 1. We don't need or I would even say we don't want to sync log files from the > new primary, because it destroys the a

Re: Commitfest 2023-03 starting tomorrow!

2023-03-06 Thread Gregory Stark (as CFM)
Sorry, I wasn't feeling very well since Friday. I'm still not 100% but I'm going to try to do some triage this afternoon. There are a few patches that need a rebase. And a few patches failing Meson builds or autoconf stages -- I wonder if there's something unrelated broken there? But what I think

Re: About default inBufSize (connection read buffer size) in libpq

2023-03-06 Thread Andres Freund
Hi, On 2023-03-05 05:42:06 +0300, Трофимов Иван wrote: > I was running some benchmarks for PG driver built on top of libpq async > functionality, > and noticed that recv syscalls issued by the application are limited by 16Kb, > which seems to > be inBufSize coming from makeEmptyPGconn in interface

Re: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Gregory Stark (as CFM)
It looks like this patch needs a quick rebase, there's a conflict in the meson.build. I'll leave the state since presumably this would be easy to resolve but it would be more likely to get attention if it's actually building cleanly. http://cfbot.cputube.org/patch_42_4023.log On Tue, 28 Feb 2023

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > wrote: > > > > At Mon, 06 Mar 2023 15:24:25 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > In any case, I think we need to avoid such concurrent autovacuum/analyze. > > > > If it is

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > Hi, > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > > wrote: > > > > > > At Mon, 06 Mar 2023 15:24:25 +0900 (JST), Kyotaro Horiguchi > > > wrote in > > > > In any ca

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 14:24:09 -0500, Melanie Plageman wrote: > On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > > > wrote: > > > > > > > > At Mon, 06 Mar 2023 15:24:25 +09

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 09:37:23AM -0800, Nathan Bossart wrote: > On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote: > > That was mostly OK for me, so applied after tweaking a couple of > > places in the tests (extra explanations, for one), the comments and > > the code. I noticed in

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-06 Thread Heikki Linnakangas
On 01/03/2023 12:21, Aleksander Alekseev wrote: Hi, I'm surprised that these patches extend the page numbering to 64 bits, but never actually uses the high bits. The XID "epoch" is not used, and pg_xact still wraps around and the segment names are still reused. I thought we could stop doing tha

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-06 Thread Gregory Stark (as CFM)
This patch too is conflicting on meson.build. Are these two patches interdependent? This one looks a bit more controversial. I'm not sure if Tom has been convinced and I haven't seen anyone else speak up. Perhaps this needs a bit more discussion of other options to solve this issue. Maybe it can

Re: Make mesage at end-of-recovery less scary.

2023-03-06 Thread Gregory Stark (as CFM)
It looks like this needs a rebase and at a quick glance it looks like more than a trivial conflict. I'll mark it Waiting on Author. Please update it back when it's rebased -- Gregory Stark As Commitfest Manager

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Andres Freund
Hi, On 2023-03-01 14:10:07 +0530, Amit Kapila wrote: > On Wed, Mar 1, 2023 at 12:09 AM Andres Freund wrote: > > > > > I see this as a way to provide this feature for users but I would > > > prefer to proceed with this if we can get some more buy-in from senior > > > community members (at least on

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-03-06 Thread Gregory Stark (as CFM)
I'm sorry, It seems this is failing again? It's Makefile and meson.build again :( Though there are also patching file contrib/pg_stat_statements/sql/oldextversions.sql can't find file to patch at input line 1833 and |--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql |+++ b/contrib/pg

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 17:02:38 +0530, Amit Kapila wrote: > Andres, do you have any thoughts on this? We seem to have figured out > the cause of regression in the case Shi-San has reported and others > also agree with it. We can't think of doing anything better than what > the patch currently is doing,

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-06 Thread Gregory Stark (as CFM)
The CFBot says this patch is failing but I find it hard to believe this is related to this patch... 2023-03-05 20:56:58.705 UTC [33902][client backend] [pg_regress/btree_index][18/750:0] STATEMENT: ALTER INDEX btree_part_idx ALTER COLUMN id SET (n_distinct=100); 2023-03-05 20:56:58.709 UTC [33902

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-03-06 Thread Andrei Zubkov
Hi Gregory, > patching file contrib/pg_stat_statements/sql/oldextversions.sql > can't find file to patch at input line 1833 > > > and > > > --- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql > > +++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql > -- >

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 2:34 PM Andres Freund wrote: > > Hi, > > On 2023-03-06 14:24:09 -0500, Melanie Plageman wrote: > > On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horigu

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is > called, 4211fbd84 changes the else into an else if [1]. I understand > after reading the commit and re-reading the code why that is now, but I > was initially

Re: Support logical replication of DDLs

2023-03-06 Thread Zheng Li
On Mon, Mar 6, 2023 at 5:17 AM wangw.f...@fujitsu.com wrote: > > For v-75-0003* patch. > 2. In the function deparse_CreateSeqStmt. > It seems that we are not deparsing the "AS data_type" clause (CREATE SEQUENCE > ... AS data_type). I think this causes all data_type to be default (bigint) > after e

Re: proposal: possibility to read dumped table's name from file

2023-03-06 Thread Gregory Stark (as CFM)
So This patch has been through a lot of commitfests. And it really doesn't seem that hard to resolve -- Pavel has seemingly been willing to go along whichever way the wind has been blowing but honestly it kind of seems like he's just gotten drive-by suggestions and he's put a lot of work into t

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart wrote: > > On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is > > called, 4211fbd84 changes the else into an else if [1]. I understand > > after reading the commi

Re: On login trigger: take three

2023-03-06 Thread Gregory Stark (as CFM)
It looks like Daniel Gustafsson, Andres, and Tom have all weighed in on this patch with at least a neutral comment (+-0 from Andres :) It looks like the main concern was breaking physical replicas and that there was consensus that as long as single-user mode worked that it was ok? So maybe it's t

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote: > On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart > wrote: >> On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: >> > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is >> > called, 4211fbd84 change

Re: proposal: possibility to read dumped table's name from file

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) wrote: > > So This patch has been through a lot of commitfests. And it really > doesn't seem that hard to resolve -- Pavel has seemingly been willing > to go along whichever way the wind has been blowing but honestly it > kind of seems like he

Re: On login trigger: take three

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 21:55, Gregory Stark (as CFM) wrote: > > It looks like Daniel Gustafsson, Andres, and Tom have all weighed in > on this patch with at least a neutral comment (+-0 from Andres :) I think the concept of a login event trigger has merits, even though it's kind of a niche use case

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-06 Thread Regina Obe
> This patch too is conflicting on meson.build. Are these two patches > interdependent? > > This one looks a bit more controversial. I'm not sure if Tom has been > convinced and I haven't seen anyone else speak up. > > Perhaps this needs a bit more discussion of other options to solve this issue.

RE: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Regina Obe
> It looks like this patch needs a quick rebase, there's a conflict in the > meson.build. > > I'll leave the state since presumably this would be easy to resolve but it > would > be more likely to get attention if it's actually building cleanly. > > http://cfbot.cputube.org/patch_42_4023.log >

Re: using memoize in in paralel query decreases performance

2023-03-06 Thread David Rowley
On Mon, 6 Mar 2023 at 21:55, Pavel Stehule wrote: > default https://explain.depesz.com/s/fnBe It looks like the slowness is coming from the Bitmap Index scan and Bitmap heap scan rather than Memoize. -> Bitmap Heap Scan on public.item i (cost=285.69..41952.12 rows=29021 width=16) (actua

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-06 Thread Nathan Bossart
+void +XLogReadFromBuffers(XLogRecPtr startptr, +TimeLineID tli, +Size count, +char *buf, +Size *read_bytes) Since this function presently doesn't return anything, can we have it return the number of bytes read instead

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 01:13:37PM -0800, Nathan Bossart wrote: > On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote: > > On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart > > wrote: > >> On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > >> > I noticed in vacuum_rel()

Re: On login trigger: take three

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 15:55:01 -0500, Gregory Stark (as CFM) wrote: > It looks like Daniel Gustafsson, Andres, and Tom have all weighed in > on this patch with at least a neutral comment (+-0 from Andres :) > > It looks like the main concern was breaking physical replicas and that > there was consens

Re: On login trigger: take three

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 23:10, Andres Freund wrote: > > Hi, > > On 2023-03-06 15:55:01 -0500, Gregory Stark (as CFM) wrote: >> It looks like Daniel Gustafsson, Andres, and Tom have all weighed in >> on this patch with at least a neutral comment (+-0 from Andres :) >> >> It looks like the main conce

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 13:29:50 +0900, Michael Paquier wrote: > On Mon, Feb 20, 2023 at 01:54:00PM +0530, Bharath Rupireddy wrote: > > I ran some tests on my dev system [1] and I don't see much difference > > between v3 and v4. So, +1 for v3 patch (+ argument order swap) from > > Andres to keep the cod

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 5:43 PM Nathan Bossart wrote: > > On Mon, Mar 06, 2023 at 05:09:58PM -0500, Melanie Plageman wrote: > > I would move this comment inside of the outer if statement since it is > > distinguishing between the two branches of the inner if statement. > > Oops, done. > > > Also, I

RE: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Regina Obe
> It looks like this patch needs a quick rebase, there's a conflict in the > meson.build. > > I'll leave the state since presumably this would be easy to resolve but it > would > be more likely to get attention if it's actually building cleanly. > > http://cfbot.cputube.org/patch_42_4023.log > -

Re: Record queryid when auto_explain.log_verbose is on

2023-03-06 Thread Imseih (AWS), Sami
I am wondering if this patch should be backpatched? The reason being is in auto_explain documentation [1], there is a claim of equivalence of the auto_explain.log_verbose option and EXPLAIN(verbose) ". it's equivalent to the VERBOSE option of EXPLAIN." This can be quite confusing for users o

Re: a very minor bug and a couple of comment changes for basebackup.c

2023-03-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > Thanks for the review. I have committed the patches. No objections to what was committed. > On Thu, Mar 2, 2023 at 2:59 AM Michael Paquier wrote: > > There is more to it: the page LSN is checked before its checksum. > > Hence, if the pag

Re: Normalization of utility queries in pg_stat_statements

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 03:50:55PM +0300, Andrei Zubkov wrote: > Those statements is not related to any WAL tests. It seems a little bit > incorrect to me. The intention is to have each file run in isolation, so this is incorrect as it stands. Thanks for the report! -- Michael signature.asc Des

Re: [PoC] Let libpq reject unexpected authentication requests

2023-03-06 Thread Jacob Champion
On Fri, Mar 3, 2023 at 6:35 PM Michael Paquier wrote: > I was refreshing my mind with 0001 yesterday, and except for the two > parts where we need to worry about AUTH_REQ_OK being sent too early > and the business with gssenc, this is a rather straight-forward. It > also looks like the the partic

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 06:12:36PM -0500, Melanie Plageman wrote: > LGTM. -* Do the actual work --- either FULL or "lazy" vacuum +* If PROCESS_MAIN is set (the default), it's time to vacuum the main +* relation. Otherwise, we can skip this part. If required, we'll process +* the

  1   2   >