Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-22 Thread Ashutosh Sharma
On Fri, Sep 23, 2022 at 12:24 PM Ashutosh Sharma wrote: > > PFA v2 patch. > > Changes in the v2 patch: > > - Reuse the existing get_controlfile function in > src/common/controldata_utils.c instead of adding a new one. > > - Set env variable PGDATA with the data directory specified by the user. >

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-22 Thread Ashutosh Sharma
PFA v2 patch. Changes in the v2 patch: - Reuse the existing get_controlfile function in src/common/controldata_utils.c instead of adding a new one. - Set env variable PGDATA with the data directory specified by the user. -- With Regards, Ashutosh Sharma.

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-09-22 Thread Bharath Rupireddy
On Wed, Sep 21, 2022 at 4:30 AM Nathan Bossart wrote: > > 0001 looks reasonable to me. Thanks for reviewing. > +errno = 0; > +rc = pg_pwritev_zeros(fd, pad_to_size); > > Do we need to reset errno? pg_pwritev_zeros() claims to set errno > appropriately. Right, pg_pwritev_zeros()

Re: making relfilenodes 56 bits

2022-09-22 Thread Dilip Kumar
On Tue, Sep 20, 2022 at 7:46 PM Amul Sul wrote: > Thanks for the review > Here are a few minor suggestions I came across while reading this > patch, might be useful: > > +#ifdef USE_ASSERT_CHECKING > + > + { > > Unnecessary space after USE_ASSERT_CHECKING. Changed > > + return

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-22 Thread Ashutosh Sharma
On Fri, Sep 23, 2022 at 6:05 AM Bharath Rupireddy wrote: > > On Thu, Sep 22, 2022 at 10:25 PM Ashutosh Sharma > wrote: > > > > PFA that enhances pg_waldump to show the latest LSN and the > > corresponding WAL file when the -l or --lastLSN option is passed an > > argument to pg_waldump. Below is

Re: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread Amit Kapila
On Thu, Sep 22, 2022 at 4:50 PM kuroda.hay...@fujitsu.com wrote: > > Hi Amit, > > > > I checked other flags that are set by signal handlers, their datatype > > > seemed to > > be sig_atomic_t. > > > Is there any reasons that you use normal bool? It should be changed if > > > not. > > > > > > > I

Re: CI and test improvements

2022-09-22 Thread Thomas Munro
On Fri, Sep 23, 2022 at 9:07 AM Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > > - # Workaround around performance issues due to 32KB block size > > > > > - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh > > > > >create_user_script:

Re: Removing unused parameter in SnapBuildGetOrBuildSnapshot

2022-09-22 Thread Amit Kapila
On Wed, Sep 21, 2022 at 8:11 PM Zhang Mingli wrote: > > On Sep 21, 2022, 22:22 +0800, Melih Mutlu , wrote: > > Hi hackers, > > Sharing a small patch to remove an unused parameter in > SnapBuildGetOrBuildSnapshot function in snapbuild.c > > With commit 6c2003f8a1bbc7c192a2e83ec51581c018aa162f, Sna

Re: libpq error message refactoring

2022-09-22 Thread Tom Lane
Andres Freund writes: > Heh, figured it out. It's inside #ifdef ENABLE_NLS. So it fails on all > platforms without NLS enabled. Argh, how simple! The question about va_start/va_end placement still stands, though. regards, tom lane

Re: libpq error message refactoring

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 19:27:27 -0700, Andres Freund wrote: > I just noticed it when trying to understand the linker failure - which I > still don't... Heh, figured it out. It's inside #ifdef ENABLE_NLS. So it fails on all platforms without NLS enabled. Greetings, Andres Freund

Re: libpq error message refactoring

2022-09-22 Thread Tom Lane
Andres Freund writes: > I suspect the appendPQExpBufferVA is orthogonal - most (all?) of the other > functions in pqexpbuffer.h are visible, so it feels weird/confusing to not > make appendPQExpBufferVA() available. I thought the same to start with, but if I'm right in my nearby reply that we'll

Re: libpq error message refactoring

2022-09-22 Thread Tom Lane
Peter Eisentraut writes: > On 22.09.22 17:42, Andres Freund wrote: >> It's not the cause of this failure, I think, but doesn't appendPQExpBufferVA >> need to be added to exports.txt? > I don't want to make that function available to users of libpq, just use > it inside libpq across .c files. Is

Re: libpq error message refactoring

2022-09-22 Thread Andres Freund
HHi, On 2022-09-22 22:00:00 -0400, Peter Eisentraut wrote: > On 22.09.22 17:42, Andres Freund wrote: > > This patch has been failing for a while: > > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3854 > > > > Interestingly, previously the error only happened when targetti

Re: libpq error message refactoring

2022-09-22 Thread Peter Eisentraut
On 22.09.22 17:42, Andres Freund wrote: This patch has been failing for a while: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3854 Interestingly, previously the error only happened when targetting windows, but meson also shows it on freebsd. It's not the cause of this

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-22 Thread Bharath Rupireddy
On Thu, Sep 22, 2022 at 10:25 PM Ashutosh Sharma wrote: > > PFA that enhances pg_waldump to show the latest LSN and the > corresponding WAL file when the -l or --lastLSN option is passed an > argument to pg_waldump. Below is an example: Thanks for the patch. I have some quick thoughts about it.

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-22 Thread Bharath Rupireddy
On Thu, Sep 22, 2022 at 3:17 PM Fujii Masao wrote: > > + MemSet(backup_state, 0, sizeof(BackupState)); > + MemSet(backup_state->name, '\0', sizeof(backup_state->name)); > > The latter MemSet() is not necessary because the former already > resets that with zero, is it?

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-22 Thread Bharath Rupireddy
On Thu, Sep 22, 2022 at 8:55 PM Andres Freund wrote: > > Due to the merge of the meson based build this patch needs some > adjustment. See > https://cirrus-ci.com/build/6146162607521792 > Looks like it just requires adding xlogbackup.c to > src/backend/access/transam/meson.build. Thanks! I will p

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-22 Thread Ashwin Agrawal
On Wed, Sep 21, 2022 at 5:34 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Wednesday, September 21, 2022, Ashwin Agrawal > wrote: > >> Currently, pg_basebackup has >> --create-slot option to create slot if not already exists or >> --slot to use existing slot >> >> Which means it

Re: [PoC] Let libpq reject unexpected authentication requests

2022-09-22 Thread Jacob Champion
On Thu, Sep 22, 2022 at 4:52 AM Peter Eisentraut wrote: > On 22.09.22 01:37, Jacob Champion wrote: > > I think this is potentially > > dangerous, but it mirrors the current behavior of libpq and I'm not > > sure that we should change it as part of this patch. > > It might be worth reviewing that b

Re: cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Tom Lane
Andres Freund writes: > On 2022-09-22 17:44:56 -0400, Tom Lane wrote: >> I was confused about how come the new patches I'd just posted in >> the 3848 CF item (write/read support for raw parse trees) are >> showing a mix of passes and fails in the cfbot. I eventually >> realized that the fail resu

Re: Making C function declaration parameter names consistent with corresponding definition names

2022-09-22 Thread Peter Geoghegan
On Thu, Sep 22, 2022 at 3:20 PM Tom Lane wrote: > WFM. Okay, pushed a minimally invasive commit to fix the inconsistencies in pg_dump related code just now. That's the last of them. Now the only remaining clang-tidy warnings about inconsistent parameter names are those that seem practically impos

Re: cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 17:44:56 -0400, Tom Lane wrote: > I was confused about how come the new patches I'd just posted in > the 3848 CF item (write/read support for raw parse trees) are > showing a mix of passes and fails in the cfbot. I eventually > realized that the fail results are all old and stal

Re: cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Tom Lane
Thomas Munro writes: > On Fri, Sep 23, 2022 at 10:17 AM Thomas Munro wrote: >> Ah, right. Morning here and I've just spotted that too after the >> first results of the Meson era rolled in overnight. It shows the >> latest result for each distinct task name, which now includes extinct >> tasks (

Re: cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Thomas Munro
On Fri, Sep 23, 2022 at 10:17 AM Thomas Munro wrote: > On Fri, Sep 23, 2022 at 9:44 AM Tom Lane wrote: > > I was confused about how come the new patches I'd just posted in > > the 3848 CF item (write/read support for raw parse trees) are > > showing a mix of passes and fails in the cfbot. I even

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Nathan Bossart
On Thu, Sep 22, 2022 at 01:28:09PM -0700, Andres Freund wrote: > On 2022-09-22 13:05:33 -0700, Nathan Bossart wrote: >> * I'm using an Ubuntu-based distribution, and the version of meson that apt >> installed was not new enough for Postgres. I ended up cloning meson [0] >> and using the newest tag

Re: Making C function declaration parameter names consistent with corresponding definition names

2022-09-22 Thread Tom Lane
Peter Geoghegan writes: > That makes it easy, then. I'll just take the least invasive approach > possible with pg_dump: treat the names from function definitions as > authoritative, and mechanically adjust the function declarations as > needed to make everything agree. WFM.

Re: cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Thomas Munro
On Fri, Sep 23, 2022 at 9:44 AM Tom Lane wrote: > I was confused about how come the new patches I'd just posted in > the 3848 CF item (write/read support for raw parse trees) are > showing a mix of passes and fails in the cfbot. I eventually > realized that the fail results are all old and stale,

Re: Making C function declaration parameter names consistent with corresponding definition names

2022-09-22 Thread Peter Geoghegan
On Thu, Sep 22, 2022 at 2:55 PM Tom Lane wrote: > Yeah. I'm not much on board with the AHX->A and AH->A changes you made; > those seem extremely invasive and it's not real clear that they add a > lot of value. That makes it easy, then. I'll just take the least invasive approach possible with pg_

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Peter Geoghegan
On Thu, Sep 22, 2022 at 2:50 PM Andres Freund wrote: > I'm likely the most biased person on this, but for me the reliable incremental > builds and the readability of the test output are big enough wins that the > answer is pretty clear... Doesn't hurt that running all tests is faster too. It's ni

Re: Making C function declaration parameter names consistent with corresponding definition names

2022-09-22 Thread Tom Lane
Peter Geoghegan writes: > I would like to give another 24 hours for anybody to lodge final > objections to what I've done in this patch. It seems possible that > there will be concerns about how this might affect backpatching, or > something like that. This patch goes relatively far in the directi

Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-09-22 Thread Jacob Champion
On 9/21/22 21:55, Andrey Chudnovsky wrote: > First, My message from corp email wasn't displayed in the thread, I see it on the public archives [1]. Your client is choosing some pretty confusing quoting tactics, though, which you may want to adjust. :D I have what I'll call some "skeptical curiosi

Re: CI and test improvements

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 16:07:02 -0500, Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > > @@ -71,8 +69,6 @@ task: > > > > > fingerprint_key: ccache/freebsd > > > > > reupload_on_changes: true > > > > > > > > > > - # Workaround around performance i

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 13:21:28 -0700, Peter Geoghegan wrote: > Is it generally recommended that individual hackers mostly switch over > to Meson for their day to day work soon? I'm guessing that this > question doesn't really have a clear answer yet, but thought I'd ask, > just in case. It'll probabl

cfbot vs. changes in the set of CI tasks

2022-09-22 Thread Tom Lane
I was confused about how come the new patches I'd just posted in the 3848 CF item (write/read support for raw parse trees) are showing a mix of passes and fails in the cfbot. I eventually realized that the fail results are all old and stale, because (for example) there's no longer a "FreeBSD - 13"

Re: Making C function declaration parameter names consistent with corresponding definition names

2022-09-22 Thread Peter Geoghegan
On Wed, Sep 21, 2022 at 6:58 PM Peter Geoghegan wrote: > Attached revision shows where I'm at with this. Would be nice to get > it all out of the way before too long. Attached is v6, which now consists of only one single patch, which fixes things up in pg_dump. (This is almost though not quite id

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Tom Lane
I wrote: > I left off the last one because it fails check-world: we now > get through the core regression tests okay, but then the pg_dump > tests fail on the new SQL function. To fix that, we would have > to extend ruleutils.c's get_utility_query_def() to be able to > fully reconstruct any legal

Re: Consider parallel for lateral subqueries with limit

2022-09-22 Thread James Coleman
On Mon, Sep 19, 2022 at 4:29 PM Robert Haas wrote: > > On Mon, Sep 19, 2022 at 3:58 PM James Coleman wrote: > > But in the case where there's correlation via LATERAL we already don't > > guarantee unique executions for a given set of params into the lateral > > subquery execution, right? For exam

Re: CI and test improvements

2022-09-22 Thread Justin Pryzby
On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > @@ -71,8 +69,6 @@ task: > > > > fingerprint_key: ccache/freebsd > > > > reupload_on_changes: true > > > > > > > > - # Workaround around performance issues due to 32KB block size > > > > - repartition_script: src/tool

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Tom Lane
Andres Freund writes: > On 2022-09-22 12:48:47 -0400, Tom Lane wrote: >> After staring at the code a bit, I think we don't need to touch >> pg_strtok() per se. I propose that this can be resolved with changes >> at the next higher level. Let's make outToken print NULL as <> as >> it always has,

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 13:05:33 -0700, Nathan Bossart wrote: > I gave the meson build system a try, and it seems to work nicely. It > didn't take long at all to adapt my workflow. > > A few notes from my experience: > > * I'm using an Ubuntu-based distribution, and the version of meson that apt > in

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Peter Geoghegan
On Thu, Sep 22, 2022 at 1:05 PM Nathan Bossart wrote: > Otherwise, all of my usual build options, ccache, etc. are working just > like before. Nice work! +1 Is it generally recommended that individual hackers mostly switch over to Meson for their day to day work soon? I'm guessing that this que

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Nathan Bossart
I gave the meson build system a try, and it seems to work nicely. It didn't take long at all to adapt my workflow. A few notes from my experience: * I'm using an Ubuntu-based distribution, and the version of meson that apt installed was not new enough for Postgres. I ended up cloning meson [0]

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Thu, 22 Sept 2022 at 15:16, Alvaro Herrera wrote: > > On 2022-Sep-22, Simon Riggs wrote: > > > On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > > > VACUUM was willing to remove a committed-dead tuple immediately if it > > > was > > > deleted by the same transaction that inserted i

Re: archive modules

2022-09-22 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 09:33:46PM +0200, Peter Eisentraut wrote: > Another question on this feature: Currently, if archive_library is set, > archive_command is ignored. I think if both are set, it should be an error. > Compare for example what happens if you set multiple recovery_target_xxx > set

Re: ICU for global collation

2022-09-22 Thread Marina Polyakova
On 2022-09-21 17:53, Peter Eisentraut wrote: Committed with that test, thanks. I think that covers all the ICU issues you reported for PG15 for now? I thought about the order of the ICU checks - if it is ok to check that the selected encoding is supported by ICU after printing all the locale

Re: Mingw task for Cirrus CI

2022-09-22 Thread Andres Freund
Hi Melih, On 2022-09-05 16:52:17 -0700, Andres Freund wrote: > I think we can convert this to meson soon, and that seems a *lot* faster at > configure than autoconf on mingw. Not even close to as fast as on a modern-ish > linux, but not that painful. Now that meson has been merged, could you try

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Andres Freund
On 2022-09-22 12:48:47 -0400, Tom Lane wrote: > I wrote: > > I think this is the issue Peter mentioned about needing to distinguish > > between empty strings and NULL strings. We're going to need to rethink > > the behavior of pg_strtok() a bit to fix that. > > After staring at the code a bit, I

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-22 Thread Ashutosh Sharma
On Thu, Sep 22, 2022 at 7:41 AM Bharath Rupireddy wrote: > > On Wed, Sep 21, 2022 at 9:53 PM Ashutosh Sharma wrote: > > > > Yeah, we can either add this functionality to pg_waldump or maybe add > > a new binary itself that would return this information. > > IMV, a separate tool isn't the way, sin

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-22 Thread Peter Geoghegan
On Wed, Sep 21, 2022 at 9:21 PM Nathan Bossart wrote: > I wouldn't mind giving this a try. Definitely seems worth experimenting with. Many WAL records generated during VACUUM (and during opportunistic pruning/index tuple deletion) have offset number arrays that can be assumed to be both sorted an

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Tom Lane
I wrote: > I think this is the issue Peter mentioned about needing to distinguish > between empty strings and NULL strings. We're going to need to rethink > the behavior of pg_strtok() a bit to fix that. After staring at the code a bit, I think we don't need to touch pg_strtok() per se. I propos

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Tom Lane
Andres Freund writes: > On 2022-08-24 17:25:31 +0200, Peter Eisentraut wrote: >> Here are patches for that. > These patches have been failing since they were posted, afaict: > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3848 > I assume that's known? I think this is th

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-09-22 Thread Önder Kalacı
Hii Wang wei, > > 1. In the function GetCheapestReplicaIdentityFullPath. > + if (rel->pathlist == NIL) > + { > + /* > +* A sequential scan could have been dominated by by an > index scan > +* during make_one_rel(). We should always have a >

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-09-22 Thread Önder Kalacı
Hi Peter, Kuroda kuroda.hay...@fujitsu.com , 21 Eyl 2022 Çar, 04:21 tarihinde şunu yazdı: > > One last thing - do you think there is any need to mention this > > behaviour in the pgdocs, or is OK just to be a hidden performance > > improvement? > > FYI - I put my opinion. > We have following sent

Re: Summary function for pg_buffercache

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 18:22:44 +0300, Melih Mutlu wrote: > Since header locks are removed again, I put some doc changes and comments > back. Due to the merge of the meson build system, this needs to adjust meson.build as well. > --- a/contrib/pg_buffercache/expected/pg_buffercache.out > +++ b/contr

Re: libpq error message refactoring

2022-09-22 Thread Andres Freund
Hi, On 2022-08-25 16:34:26 +0200, Peter Eisentraut wrote: > libpq now contains a mix of error message strings that end with newlines and > don't end with newlines, due to some newer code paths with new ways of > passing errors around. This has now gotten me confused a few too many times > both du

Re: Temporary file access API

2022-09-22 Thread John Morris
I’m a latecomer to the discussion, but as a word of introduction, I’m working with Stephen, and I have started looking over the temp file proposal with the idea of helping it move along. I’ll start by summarizing the temp file proposal and its goals. >From a high level, the proposed code: *

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-09-22 Thread Martin Kalcher
Am 22.09.22 um 17:23 schrieb Andres Freund: Hi, On 2022-08-04 07:46:10 +0200, Martin Kalcher wrote: Patch update without merge conflicts. Due to the merge of the meson based build, this patch needs to be adjusted. See https://cirrus-ci.com/build/6580671765282816 Looks like it'd just be adding

Re: Extending outfuncs support to utility statements

2022-09-22 Thread Andres Freund
Hi, On 2022-08-24 17:25:31 +0200, Peter Eisentraut wrote: > Here are patches for that. These patches have been failing since they were posted, afaict: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3848 I assume that's known? Most of the failures seem to be things like di

Re: Add last_vacuum_index_scans in pg_stat_all_tables

2022-09-22 Thread Andres Freund
Hi, On 2022-09-16 13:23:06 +0900, Ken Kato wrote: > Thank you for the review! > I fixed it and resubmitting the patch. cfbot flags that the docs aren't valid: https://cirrus-ci.com/task/5309377937670144?logs=docs_build#L295 [15:05:39.683] monitoring.sgml:4574: parser error : Opening and ending ta

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 16:43:19 +0900, Michael Paquier wrote: > I have put my hands on 0001, and finished with the attached, that > includes many fixes and tweaks. Due to the merge of the meson based build this patch needs some adjustment. See https://cirrus-ci.com/build/6146162607521792 Looks like it

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-09-22 Thread Andres Freund
Hi, On 2022-08-04 07:46:10 +0200, Martin Kalcher wrote: > Patch update without merge conflicts. Due to the merge of the meson based build, this patch needs to be adjusted. See https://cirrus-ci.com/build/6580671765282816 Looks like it'd just be adding user_prng.c to src/backend/utils/adt/meson.bu

Re: Summary function for pg_buffercache

2022-09-22 Thread Melih Mutlu
Hi, Since header locks are removed again, I put some doc changes and comments back. Thanks, Melih v10-0001-Added-pg_buffercache_summary-function.patch Description: Binary data

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Alvaro Herrera
On 2022-Sep-22, Tom Lane wrote: > Ah, right, the joys of maintaining multiple build systems. I wonder > if there's any way to avoid that by scraping file lists from one > group to the other. Or maybe we could have a file common to both, say OBJS, which both scrape in their own way. That could b

Re: Amcheck verification of GiST and GIN

2022-09-22 Thread Andres Freund
Hi, On 2022-08-17 17:28:02 +0500, Andrey Borodin wrote: > Here's v13. Changes: > 1. Fixed passing through downlink in GIN index > 2. Fixed GIN tests (one test case was not working) > > Thanks to Vitaliy Kukharik for trying this patches. Due to the merge of the meson based build, this patch needs

Re: pg_waldump: add test for coverage

2022-09-22 Thread Andres Freund
Hi, On 2022-08-23 10:50:08 +0900, Dong Wook Lee wrote: > I wrote a test for coverage. Unfortunately the test doesn't seem to pass on windows, and hasn't ever done so: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3834 Due to the merge of the meson patchset, you should al

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2022-09-22 Thread Andres Freund
Hi, On 2022-05-20 17:46:38 +0400, Pavel Borisov wrote: > CFbot says v12 patch does not apply. > Rebased. PFA v13. > Your reviews are very much welcome! Due to the merge of the meson based build this patch needs to be adjusted: https://cirrus-ci.com/build/6350479973154816 Looks like you need to a

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 11:46 AM John Naylor wrote: > One thing I want to try soon is storing fewer than 16/32 etc entries, so that the whole node fits comfortably inside a power-of-two allocation. That would allow us to use aset without wasting space for the smaller nodes, which would be faster a

Re: proposal: possibility to read dumped table's name from file

2022-09-22 Thread Andres Freund
Hi, On 2022-09-12 09:58:37 +0200, Daniel Gustafsson wrote: > > On 9 Sep 2022, at 11:00, Andrew Dunstan wrote: > > > >> On Sep 9, 2022, at 5:53 PM, John Naylor > >> wrote: > >> > >> Note that the grammar has shift-reduce conflicts. > > > Looks like the last rule for Filters should not be the

Re: Tab complete for CREATE SUBSCRIPTION ... CONECTION does not work

2022-09-22 Thread Japin Li
On Tue, 20 Sep 2022 at 00:19, Japin Li wrote: > Hi hacker, > > As $subject detailed, the tab-complete cannot work such as: > >CREATE SUBSCRIPTION sub CONNECTION 'dbname=postgres port=6543' \t > > It seems that the get_previous_words() could not parse the single quote. > > OTOH, it works for C

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 04:29:15 -0400, Andrew Dunstan wrote: > Great. Now I'll start on buildfarm support. Given my current > commitments, this will take me a while, but I hope to have a working > client by about the beginning of November. Great! Let me know if there's something I can do to help. Gre

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 16:56:57 +0200, Alvaro Herrera wrote: > On 2022-Sep-22, Tom Lane wrote: > > Initial reports from the cfbot are mostly promising, but there are a few > > patches where all the meson builds fail while all the autoconf ones pass, > > so there's something for you to look at. So far

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Sep-22, Tom Lane wrote: >> Initial reports from the cfbot are mostly promising, but there are a few >> patches where all the meson builds fail while all the autoconf ones pass, >> so there's something for you to look at. So far CF entries 3464, 3733, >> 3771, 3808

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Alvaro Herrera
On 2022-Sep-22, Tom Lane wrote: > Initial reports from the cfbot are mostly promising, but there are a few > patches where all the meson builds fail while all the autoconf ones pass, > so there's something for you to look at. So far CF entries 3464, 3733, > 3771, 3808 look that way. Hmm, but tho

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Tom Lane
Andres Freund writes: > On 2022-09-21 09:46:30 -0700, Andres Freund wrote: >> I'm planning to commit this today, unless somebody wants to argue against >> that. > And done! Yay! Initial reports from the cfbot are mostly promising, but there are a few patches where all the meson builds fail whil

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 7:52 PM John Naylor wrote: > > > On Thu, Sep 22, 2022 at 1:26 PM Masahiko Sawada wrote: > > Good point. While keeping the chunks in the small nodes in sorted > > order is useful for visiting all keys in sorted order, additional > > branches and memmove calls could be slow.

Re: Pruning never visible changes

2022-09-22 Thread Alvaro Herrera
On 2022-Sep-22, Simon Riggs wrote: > On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > VACUUM was willing to remove a committed-dead tuple immediately if it > > was > > deleted by the same transaction that inserted it. The idea is that > > such a > > tuple could never have been

Re: RFC: Logging plan of the running query

2022-09-22 Thread torikoshia
On 2022-09-21 17:30, Alena Rybakina wrote: Thanks for your reply! I also noticed it. However I also discovered that above function declarations to be aplied for explain command and used to be printed details of the executed query. We have a similar task to print the plan of an interrupted proc

Re: why can't a table be part of the same publication as its schema

2022-09-22 Thread Jonathan S. Katz
> On Sep 22, 2022, at 8:02 AM, Alvaro Herrera wrote: > > FWIW I put this to CI: > https://cirrus-ci.com/build/5823276948652032 (master) > > and everything appears to be OK. If anybody has reservations about this > grammar change, please speak up soon, as there's not much time before RC1. >

Re: Fix snapshot name for SET TRANSACTION documentation

2022-09-22 Thread Japin Li
On Thu, 22 Sep 2022 at 12:00, Fujii Masao wrote: > On 2022/09/21 14:40, Fujii Masao wrote: >> On 2022/09/21 12:01, Japin Li wrote: >>> >>> Hi hackers, >>> >>> In 6c2003f8a1bbc7c192a2e83ec51581c018aa162f, we change the snapshot name >>> when exporting snapshot, however, there is one place we miss

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > On Fri, 16 Sept 2022 at 10:27, Tom Lane wrote: > > > > Simon Riggs writes: > > > A user asked me whether we prune never visible changes, such as > > > BEGIN; > > > INSERT... > > > UPDATE.. (same row) > > > COMMIT; > > > > Didn't we just have t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 1:26 PM Masahiko Sawada wrote: > > On Thu, Sep 22, 2022 at 1:46 PM John Naylor > wrote: > > While on the subject, I wonder how important it is to keep the chunks in the small nodes in sorted order. That adds branches and memmove calls, and is the whole reason for the recen

Re: why can't a table be part of the same publication as its schema

2022-09-22 Thread Alvaro Herrera
FWIW I put this to CI: https://cirrus-ci.com/build/5823276948652032 (master) and everything appears to be OK. If anybody has reservations about this grammar change, please speak up soon, as there's not much time before RC1. The one for 15 just started running: https://cirrus-ci.com/build/4735322

Re: archive modules

2022-09-22 Thread Peter Eisentraut
On 17.09.22 11:49, Peter Eisentraut wrote: On 14.09.22 23:09, Nathan Bossart wrote: On Wed, Sep 14, 2022 at 09:31:04PM +0200, Peter Eisentraut wrote: Here is a patch that addresses this. My intent was to present archive_command as the built-in archive library, but I can see how this might cau

Re: [PoC] Let libpq reject unexpected authentication requests

2022-09-22 Thread Peter Eisentraut
On 22.09.22 01:37, Jacob Champion wrote: On Wed, Sep 21, 2022 at 3:36 PM Peter Eisentraut wrote: So let's look at the two TODO comments you have: * TODO: how should !auth_required interact with an incomplete * SCRAM exchange? What specific combination of events are you t

Re: pg_basebackup's --gzip switch misbehaves

2022-09-22 Thread Michael Paquier
On Thu, Sep 22, 2022 at 12:47:34AM -0400, Tom Lane wrote: > Sure. I'd say we have 48 hours to choose whether to put this in v15. > But not more than that. I have a window to be able to look at the buildfarm today, tomorrow being harder, so I have adjusted that now on both HEAD and REL_15_STABLE f

RE: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread kuroda.hay...@fujitsu.com
Hi Amit, > > I checked other flags that are set by signal handlers, their datatype > > seemed to > be sig_atomic_t. > > Is there any reasons that you use normal bool? It should be changed if not. > > > > It follows the logic similar to ParallelMessagePending. Do you see any > problem with it? H

Re: cannot to compile master in llvm_resolve_symbols

2022-09-22 Thread Pavel Stehule
čt 22. 9. 2022 v 12:54 odesílatel Thomas Munro napsal: > On Thu, Sep 22, 2022 at 10:43 PM Pavel Stehule > wrote: > > Today I found new bug > > > > -o llvmjit_wrap.o llvmjit_wrap.cpp -MMD -MP -MF .deps/llvmjit_wrap.Po > > llvmjit.c: In function ‘llvm_resolve_symbols’: > > llvmjit.c:1115:57: error

Re: cannot to compile master in llvm_resolve_symbols

2022-09-22 Thread Thomas Munro
On Thu, Sep 22, 2022 at 10:43 PM Pavel Stehule wrote: > Today I found new bug > > -o llvmjit_wrap.o llvmjit_wrap.cpp -MMD -MP -MF .deps/llvmjit_wrap.Po > llvmjit.c: In function ‘llvm_resolve_symbols’: > llvmjit.c:1115:57: error: ‘LLVMJITCSymbolMapPair’ undeclared (first use in > this function); d

cannot to compile master in llvm_resolve_symbols

2022-09-22 Thread Pavel Stehule
Hi Today I found new bug -o llvmjit_wrap.o llvmjit_wrap.cpp -MMD -MP -MF .deps/llvmjit_wrap.Po llvmjit.c: In function ‘llvm_resolve_symbols’: llvmjit.c:1115:57: error: ‘LLVMJITCSymbolMapPair’ undeclared (first use in this function); did you mean ‘LLVMOrcCSymbolMapPair’? 1115 | LLVMOrcCSy

Re: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread Amit Kapila
On Thu, Sep 22, 2022 at 8:59 AM wangw.f...@fujitsu.com wrote: > Few comments on v33-0001 === 1. + else if (data->streaming == SUBSTREAM_PARALLEL && + data->protocol_version < LOGICALREP_PROTO_STREAM_PARALLEL_VERSION_NUM) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VA

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-22 Thread Fujii Masao
On 2022/09/22 16:43, Michael Paquier wrote: Added that part before pg_backup_stop() now where it makes sense with the refactoring. I have put my hands on 0001, and finished with the attached, that includes many fixes and tweaks. Some of the variable renames felt out of place, while some com

Re: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread Amit Kapila
On Thu, Sep 22, 2022 at 1:37 PM kuroda.hay...@fujitsu.com wrote: > > === > applyparallelworker.c > > 03. declaration > > ``` > +/* > + * Is there a message pending in parallel apply worker which we need to > + * receive? > + */ > +volatile bool ParallelApplyMessagePending = false; > ``` > > I chec

RE: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread houzj.f...@fujitsu.com
On Thursday, September 22, 2022 4:08 PM Kuroda, Hayato/黒田 隼人 wrote: > > Thanks for updating the patch! Followings are comments for v33-0001. Thanks for the comments. > 04. HandleParallelApplyMessages() > > ``` > + if (winfo->error_mq_handle == NULL) > + con

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Andrew Dunstan
On 2022-09-22 Th 01:57, Andres Freund wrote: > Hi, > > On 2022-09-21 09:46:30 -0700, Andres Freund wrote: >> I'm planning to commit this today, unless somebody wants to argue against >> that. > And done! > > Changes: > - fixed a few typos (thanks Thomas) > - less duplication in the CI tasks > - r

RE: Perform streaming logical transactions by background workers and parallel apply

2022-09-22 Thread kuroda.hay...@fujitsu.com
Dear Wang, Thanks for updating the patch! Followings are comments for v33-0001. === libpqwalreceiver.c 01. inclusion ``` +#include "catalog/pg_subscription.h" ``` We don't have to include it because the analysis of parameters is done at caller. === launcher.c 02. logicalrep_worker_launch()

RE: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-09-22 Thread Andrey Chudnovskiy
We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one. Libpq implementing OAUTHBEARER is needed for community/3rd party tools to have user-friendly authentication experience: 1. For community client tools, like pg_admin, psql etc.

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-22 Thread Michael Paquier
On Wed, Sep 21, 2022 at 05:10:39PM +0530, Bharath Rupireddy wrote: >> deallocate_backup_variables() is the only part of xlogbackup.c that >> includes references of the tablespace map_and backup_label >> StringInfos. I would be tempted to fully decouple that from >> xlogbackup.c/h for the time bein

Re: Multi-insert related comment in CopyFrom()

2022-09-22 Thread Etsuro Fujita
On Wed, Sep 21, 2022 at 4:39 PM Etsuro Fujita wrote: > While working on the “Fast COPY FROM based on batch insert” patch, I > noticed this: > > else if (proute != NULL && resultRelInfo->ri_TrigDesc != NULL && > resultRelInfo->ri_TrigDesc->trig_insert_new_table) > { > /