Python-3.11 patch

2022-01-18 Thread Horvath, Miklos
Hi, This patch is important for postgresql 13.5 and 14.1 if you want to use them with Python-3.11. Miklos Description: Build fix on Python3-3.11 requires for Python > 3.10 because the eval.h header file was removed. These files must not be included directly, as they are already included in Pytho

Re: generic plans and "initial" pruning

2022-01-18 Thread Amit Langote
Hi Simon, On Tue, Jan 18, 2022 at 4:44 PM Simon Riggs wrote: > On Tue, 11 Jan 2022 at 16:22, Robert Haas wrote: > > This is just a relatively simple example and I think there are > > probably a bunch of others. There are a lot of kinds of DDL that could > > be performed on a partition that gets

Re: Push down time-related SQLValue functions to foreign server

2022-01-18 Thread Alexander Pyhalov
Hi. Tom Lane писал 2022-01-18 02:08: Alexander Pyhalov writes: Perhaps in a MACRO? Changed this check to a macro, also fixed condition in is_foreign_param() and added test for it. Also fixed comment in prepare_query_params(). I took a quick look at this. I'm unconvinced that you need the

[PATCH] allow src/tools/msvc/clean.bat script to be called from the root of the source tree

2022-01-18 Thread Juan José Santamaría Flecha
In [1] capacity for $SUBJECT was added to most of the batch scripts, but clean.bat was not included. I propose to do so with the attached patch. By the way, are pgbison.bat and pgflex.bat directly called anywhere? [1] https://www.postgresql.org/message-id/2b7a674b-5fb0-d264-75ef-ecc7a31e5...@post

Re: Extensible Rmgr for Table AMs

2022-01-18 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 12:42:06AM -0800, Jeff Davis wrote: > On Mon, 2021-11-08 at 15:36 -0800, Jeff Davis wrote: > > The attached patch (against v14, so it's easier to test columnar) is > > somewhat like a simplified version of [3] combined with refactoring > > to > > make decoding a part of

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-18 Thread Peter Eisentraut
On 09.12.21 14:31, Tom Lane wrote: Peter Eisentraut writes: On 02.12.21 08:20, Peter Eisentraut wrote: Buildfarm impact: gaur and prariedog use Python 2.6 and would need to be upgraded. Tom, are you planning to update the Python version on these build farm members? I realize these are very

Re: Python-3.11 patch

2022-01-18 Thread Peter Eisentraut
On 17.01.22 20:58, Horvath, Miklos wrote: This patch is important for postgresql 13.5 and 14.1 if you want to use them with Python-3.11. Thanks, this was already done and will be in the next minor releases for all branches.

Re: generic plans and "initial" pruning

2022-01-18 Thread Simon Riggs
On Tue, 18 Jan 2022 at 08:10, Amit Langote wrote: > > Hi Simon, > > On Tue, Jan 18, 2022 at 4:44 PM Simon Riggs > wrote: > > On Tue, 11 Jan 2022 at 16:22, Robert Haas wrote: > > > This is just a relatively simple example and I think there are > > > probably a bunch of others. There are a lot of

Re: Column Filtering in Logical Replication

2022-01-18 Thread Peter Eisentraut
On 15.01.22 04:45, Amit Kapila wrote: I think another issue w.r.t column filter patch is that even while creating publication (even for 'insert' publications) it should check that all primary key columns must be part of published columns, otherwise, it can fail while applying on subscriber as it

Re: support for MERGE

2022-01-18 Thread Peter Eisentraut
On 13.01.22 13:43, Alvaro Herrera wrote: Apologies, there was a merge failure and I failed to notice. Here's the correct patch. I looked through this a bit. I wonder why there is a check for "unreachable WHEN clause". I don't see this in the SQL standard. On the other hand, there is a req

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-18 Thread Takashi Menjo
Hi Justin, Thanks for your help. I'm making an additional patch for Cirrus CI. I'm also trying to reproduce the "make check-world" error you reported, on my Linux environment that has neither a real PMem nor an emulated one, with PMEM_IS_PMEM_FORCE=1. I'll keep you updated. Regards, Takashi On

Re: Showing applied extended statistics in explain

2022-01-18 Thread Tatsuro Yamada
Hi Tomas! >> What exactly use case do you have in mind? >Well, my goal was to help users investigating the plan/estimates, >because once you create multiple "candidate" statistics objects it may >get quite tricky to determine which of them were applied, in what order, >etc. It's not all that strai

Re: a misbehavior of partition row movement (?)

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-18, Amit Langote wrote: > Would you like me to update the patch with the above renumbering or > are you working on a new version yourself? I have a few very minor changes apart from that. Let me see if I can get this pushed soon; if I'm unable to (there are parts I don't fully grok y

Re: row filtering for logical replication

2022-01-18 Thread Amit Kapila
On Mon, Jan 17, 2022 at 8:58 PM houzj.f...@fujitsu.com wrote: > > Attach the V66 patch set which addressed Amit, Peter and Greg's comments. > Thanks, some more comments, and suggestions: 1. /* + * If no record in publication, check if the table is the partition + * of a published partitioned tab

Re: row filtering for logical replication

2022-01-18 Thread Amit Kapila
On Tue, Jan 18, 2022 at 6:05 PM Amit Kapila wrote: > > On Mon, Jan 17, 2022 at 8:58 PM houzj.f...@fujitsu.com > wrote: > > Spurious line removal. I have added back in the attached top-up patch. > > Apart from the above, I have made some modifications to other comments. > Sorry, attached the wron

Re: ICU for global collation

2022-01-18 Thread Peter Eisentraut
On 18.01.22 05:02, Julien Rouhaud wrote: If this is applied, then in my estimation all these hunks will completely disappear from the global ICU patch. So this would be a significant win. Agreed, the patch will be quite smaller and also easier to review. Are you planning to apply it on its own

Re: Adding CI to our tree

2022-01-18 Thread Peter Eisentraut
On 17.01.22 22:13, Andrew Dunstan wrote: Well, the logic we use for TAP tests is we run them for the following if they have a t subdirectory, subject to configuration settings like PG_TEST_EXTRA:  src/test/bin/*  contrib/*  src/test/modules/*  src/test/* As long as any new TAP test gets pla

Re: missing indexes in indexlist with partitioned tables

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-18, Julien Rouhaud wrote: > SET enable_partitionwise_join = on; > EXPLAIN (COSTS OFF) > SELECT * FROM fract_t x LEFT JOIN fract_t y USING (id) ORDER BY id ASC LIMIT > 10; > - QUERY PLAN > -

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-17, Robert Haas wrote: > On Mon, Jan 17, 2022 at 4:34 PM Alvaro Herrera > wrote: > > Maybe it would work to have a single LSN in shared memory, as an atomic > > variable, which uses monotonic advance[1] to be updated. Whether this is > > updated or not would depend on a new GUC, ma

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 12:14 AM Peter Geoghegan wrote: > I quite clearly said that you'll only get an anti-wraparound VACUUM > with the patch applied when the only factor that *ever* causes *any* > autovacuum worker to VACUUM the table (assuming the workload is > stable) is the anti-wraparound/au

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Robert Haas
On Mon, Jan 17, 2022 at 8:39 PM James Coleman wrote: > I wondered about that, but commit_ts already does more than commit > timestamps by recording the xid of the last commit. Well, if you're maintaining an SLRU, you do kind of need to know where the leading and lagging ends are. > For that matt

Re: refactoring basebackup.c

2022-01-18 Thread Jeevan Ladhe
Hi, Similar to LZ4 server-side compression, I have also tried to add a ZSTD server-side compression in the attached patch. I have done some initial testing and things seem to be working. Example run: pg_basebackup -t server:/tmp/data_zstd -Xnone --server-compression=zstd The patch surely needs s

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 9:25 AM Robert Haas wrote: > > On Mon, Jan 17, 2022 at 8:39 PM James Coleman wrote: > > I wondered about that, but commit_ts already does more than commit > > timestamps by recording the xid of the last commit. > > Well, if you're maintaining an SLRU, you do kind of need t

Re: generic plans and "initial" pruning

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 3:10 AM Amit Langote wrote: > Pursuing that kind of a project would perhaps have been more > worthwhile if the locking issue had affected more than just this > particular case, that is, the case of running prepared statements over > partitioned tables using generic plans.

Re: Pluggable toaster

2022-01-18 Thread Teodor Sigaev
Hi! Maybe doing that kind of compression in TOAST is somehow simpler, but I don't see it. Seems, in ideal world, compression should be inside toaster. 2 Current toast storage stores chunks in heap accesses method and to provide fast access by toast id it makes an index. Ideas:    - store c

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 9:47 AM James Coleman wrote: > > Well, if you're maintaining an SLRU, you do kind of need to know where > > the leading and lagging ends are. > > As far as I can tell the data in commitTsShared is used purely as an > optimization for the path looking up the timestamp for an

Re: Refactoring of compression options in pg_basebackup

2022-01-18 Thread Robert Haas
On Mon, Jan 17, 2022 at 8:36 PM Michael Paquier wrote: > On Mon, Jan 17, 2022 at 12:48:12PM -0500, Robert Haas wrote: > > Alvaro's proposal is fine with me. I don't see any value in replacing > > --compress with --compression. It's longer but not superior in any way > > that I can see. Having both

Re: [PATCH] Add reloption for views to enable RLS

2022-01-18 Thread Christoph Heiss
Hi Laurenz, thanks for the review! I've attached a v2 where I addressed the things you mentioned. On 1/11/22 19:59, Laurenz Albe wrote: [..] You made that an enum with only a single value. What other values could you imagine in the future? I think that this should be a boolean reloption, for

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-18 Thread Tom Lane
Peter Eisentraut writes: > Also, considering the failure on prairiedog, I do see now on > that the sysconfig > module is "New in version 3.2". I had interpreted the fact that it > exists in version 2.7 that that includes all higher versions, b

Re: 32TB relation size make mdnblocks overflow

2022-01-18 Thread Tom Lane
Julien Rouhaud writes: > On Tue, Jan 18, 2022 at 02:21:14PM +0800, 陈佳昕(步真) wrote: >> We know that PostgreSQL doesn't support a single relation size over 32TB, >> limited by the MaxBlockNumber. But if we just 'insert into' one relation over >> 32TB, it will get an error message 'unexpected data bey

Re: In-placre persistance change of a relation

2022-01-18 Thread Tom Lane
Julien Rouhaud writes: > The cfbot is failing on all OS with this version of the patch. Apparently > v16-0002 introduces some usage of "testtablespace" client-side variable that's > never defined, e.g. That test infrastructure got rearranged very recently, see d6d317dbf.

Re: [Proposal] Add foreign-server health checks infrastructure

2022-01-18 Thread Fujii Masao
On 2021/12/15 15:40, kuroda.hay...@fujitsu.com wrote: Yeah, remote-checking timeout will be enable even ifa local transaction is opened. In my understanding, postgres cannot distinguish whether opening transactions are using only local object or not. My first idea was that turning on the time

Re: 32TB relation size make mdnblocks overflow

2022-01-18 Thread 陈佳昕(步真)
​I think we must meet some corner cases about our storage. The relation has 32TB blocks, so 'mdnblocks' gets the unexpected value, we will check it again. Thanks a lot.

Re: New developer papercut - Makefile references INSTALL

2022-01-18 Thread Daniel Gustafsson
> On 17 Jan 2022, at 13:26, Daniel Gustafsson wrote: > >> On 17 Jan 2022, at 11:25, Magnus Hagander wrote: > >> That said, I'm not sure we're actually gaining anything by *not* >> referring to the website as well. TBH, I bet the majority of users >> will actually prefer to read them there. So I

Re: New developer papercut - Makefile references INSTALL

2022-01-18 Thread Peter Eisentraut
On 18.01.22 16:51, Daniel Gustafsson wrote: On 17 Jan 2022, at 13:26, Daniel Gustafsson wrote: On 17 Jan 2022, at 11:25, Magnus Hagander wrote: That said, I'm not sure we're actually gaining anything by *not* referring to the website as well. TBH, I bet the majority of users will actuall

Re: Pluggable toaster

2022-01-18 Thread Tomas Vondra
On 1/18/22 15:56, Teodor Sigaev wrote: > Hi! > >> Maybe doing that kind of compression in TOAST is somehow simpler, but >> I don't see it. > Seems, in ideal world, compression should be inside toaster. > I'm not convinced that's universally true. Yes, I'm sure certain TOAST implementations wo

Re: New developer papercut - Makefile references INSTALL

2022-01-18 Thread Tom Lane
Daniel Gustafsson writes: > I plan on applying the attached which address the feedback given. +1 regards, tom lane

Re: a misbehavior of partition row movement (?)

2022-01-18 Thread Tom Lane
Julien Rouhaud writes: > @@ -133,7 +133,7 @@ > SELECT pg_relation_size('reloptions_test') = 0; > ?column? > -- > - t > + f > (1 row) Some machines have been showing that on HEAD too, so I doubt it's the fault of this patch. That reloptions test isn't stable yet seemingly.

Re: Adding CI to our tree

2022-01-18 Thread Andrew Dunstan
On 1/18/22 08:06, Peter Eisentraut wrote: > On 17.01.22 22:13, Andrew Dunstan wrote: >> Well, the logic we use for TAP tests is we run them for the following if >> they have a t subdirectory, subject to configuration settings like >> PG_TEST_EXTRA: >> >>   src/test/bin/* >>   contrib/* >>   src/t

Re: Support for NSS as a libpq TLS backend

2022-01-18 Thread Joshua Brindle
On Tue, Jan 18, 2022 at 7:43 AM Daniel Gustafsson wrote: > > > On 18 Jan 2022, at 07:36, Julien Rouhaud wrote: > > > On Mon, Jan 17, 2022 at 03:09:11PM +0100, Daniel Gustafsson wrote: > >> > >> I must've fat-fingered the "git add -p" for v50 as the fix was in > >> configure.ac > >> but not confi

Re: psql - add SHOW_ALL_RESULTS option

2022-01-18 Thread Peter Eisentraut
On 15.01.22 10:00, Fabien COELHO wrote: The reason this test constantly fails on cfbot windows is a use-after-free bug. Indeed! Thanks a lot for the catch and the debug! The ClearOrSaveResult function is quite annoying because it may or may not clear the result as a side effect. Attached v

Re: Support for NSS as a libpq TLS backend

2022-01-18 Thread Jacob Champion
On Wed, 2021-12-15 at 23:10 +0100, Daniel Gustafsson wrote: > I've attached a v50 which fixes the issues found by Joshua upthread, as well > as > rebases on top of all the recent SSL and pgcrypto changes. I'm currently tracking down a slot leak. When opening and closing large numbers of NSS datab

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-18 Thread Peter Eisentraut
On 18.01.22 16:24, Tom Lane wrote: Peter Eisentraut writes: Also, considering the failure on prairiedog, I do see now on that the sysconfig module is "New in version 3.2". I had interpreted the fact that it exists in version 2.7 that that incl

Re: [PATCH] allow src/tools/msvc/clean.bat script to be called from the root of the source tree

2022-01-18 Thread Andrew Dunstan
On 1/18/22 04:41, Juan José Santamaría Flecha wrote: > In [1] capacity for $SUBJECT was added to most of the batch scripts, > but clean.bat was not included. I propose to do so with the attached > patch. That looks a bit ugly. How about this (untested) instead? > > By the way, are pgbison.bat

Re: Push down time-related SQLValue functions to foreign server

2022-01-18 Thread Tom Lane
Alexander Pyhalov writes: > [ updated patch ] Thanks for updating the patch. (BTW, please attach version numbers to new patch versions, to avoid confusion.) However, before we proceed any further with this patch, I think we really ought to stop and think about the question I raised last night:

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-18 Thread Tom Lane
Peter Eisentraut writes: > I don't have a clear idea how to fix this in the long run. We would > perhaps need to determine at which points the various platforms had > fixed this issue in their Python installations and select between the > old and the new approach based on that. Seems messy.

Re: Per-table storage parameters for TableAM/IndexAM extensions

2022-01-18 Thread Sadhuprasad Patro
On Mon, Jan 17, 2022 at 4:47 PM Jelte Fennema wrote: > > Big +1, this is a great addition! > > I think it would be very useful if there were some tests for this new > feature. Something similar to the tests for storage parameters for > index AMs in src/test/modules/dummy_index_am. > Sure, I will r

Re: Adding CI to our tree

2022-01-18 Thread Andres Freund
Hi, On 2022-01-18 11:20:08 -0500, Andrew Dunstan wrote: > Sure, very doable, although we will still need some special logic for > src/test - there are security implication from running the ssl, ldap and > kerberos tests by default. See its Makefile. ISTM that we should move the PG_TEST_EXTRA hand

Re: slowest tap tests - split or accelerate?

2022-01-18 Thread Robert Haas
On Mon, Jan 17, 2022 at 2:57 PM Andres Freund wrote: > > pg_basebackup's 010_pg_basebackup.pl looks like it could be split up, > > though. That one, at least to me, looks like people have just kept > > adding semi-related things into the same test file. > > Yea. Here's a patch that splits up that

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-17, James Coleman wrote: > I'd be happy to make it a separate GUC, though it seems adding an > additional atomic access is worse (assuming we can convince ourselves > putting this into the commit timestamps infrastructure is acceptable) > given here we're already under a lock. I was t

Re: docs: pg_replication_origin_oid() description does not match behaviour

2022-01-18 Thread Bossart, Nathan
On 1/17/22, 5:24 PM, "Michael Paquier" wrote: > On Tue, Jan 18, 2022 at 10:19:41AM +0900, Ian Lawrence Barwick wrote: >> Given that the code has remained unchanged since the function was >> introduced in 9.5, it seems reasonable to change the documentation >> to match the function behaviour rather

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2022-01-18 Thread Tom Lane
Jelte Fennema writes: > It seems the man page of TCP_USER_TIMEOUT does not align with > reality then. When I use it on my local machine it is effectively used > as a connection timeout too. Huh, I should have thought to try that. I confirm this behavior on RHEL8 (kernel 4.18.0). Not the first

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 12:50 PM Alvaro Herrera wrote: > > On 2022-Jan-17, James Coleman wrote: > > > I'd be happy to make it a separate GUC, though it seems adding an > > additional atomic access is worse (assuming we can convince ourselves > > putting this into the commit timestamps infrastructu

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-18 Thread Peter Geoghegan
On Tue, Jan 18, 2022 at 6:11 AM Robert Haas wrote: > On Tue, Jan 18, 2022 at 12:14 AM Peter Geoghegan wrote: > > I quite clearly said that you'll only get an anti-wraparound VACUUM > > with the patch applied when the only factor that *ever* causes *any* > > autovacuum worker to VACUUM the table (

Re: [PATCH] reduce page overlap of GiST indexes built using sorted method

2022-01-18 Thread Andrey Borodin
> 18 янв. 2022 г., в 03:54, Björn Harrtell > написал(а): > > There might be some deep reason in the architecture that I'm unaware of that > could make it difficult to affect the node size but regardless, I believe > there could be a substantial win if node size could be controlled. That's

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-18, James Coleman wrote: > Reading the code it seems the only usage (besides > the boolean activation status also stored there) is in > TransactionIdGetCommitTsData, and the only consumers of that in core > appear to be the SQL callable functions to get the latest commit info. > It is

Re: refactoring basebackup.c

2022-01-18 Thread Robert Haas
On Tue, Nov 16, 2021 at 4:47 PM Robert Haas wrote: > Here's a new patch set. And here's another one. I've committed the first two patches from the previous set, the second of those just today, and so we're getting down to the meat of the patch set. 0001 adds "server" and "blackhole" as backup t

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 1:52 PM Alvaro Herrera wrote: > > On 2022-Jan-18, James Coleman wrote: > > > Reading the code it seems the only usage (besides > > the boolean activation status also stored there) is in > > TransactionIdGetCommitTsData, and the only consumers of that in core > > appear to b

Re: Adding CI to our tree

2022-01-18 Thread Andrew Dunstan
On 1/18/22 12:44, Andres Freund wrote: > Hi, > > On 2022-01-18 11:20:08 -0500, Andrew Dunstan wrote: >> Sure, very doable, although we will still need some special logic for >> src/test - there are security implication from running the ssl, ldap and >> kerberos tests by default. See its Makefile.

Re: Push down time-related SQLValue functions to foreign server

2022-01-18 Thread Alexander Pyhalov
Tom Lane писал 2022-01-18 19:53: Alexander Pyhalov writes: [ updated patch ] Thanks for updating the patch. (BTW, please attach version numbers to new patch versions, to avoid confusion.) However, before we proceed any further with this patch, I think we really ought to stop and think about

Re: Push down time-related SQLValue functions to foreign server

2022-01-18 Thread Tom Lane
Alexander Pyhalov writes: > Tom Lane писал 2022-01-18 19:53: >> However, before we proceed any further with this patch, I think we >> really ought to stop and think about the question I raised last >> night: why are we building a one-off feature for SQLValueFunction? >> Wouldn't the same parameter

Re: O(n) tasks cause lengthy startups and checkpoints

2022-01-18 Thread Bossart, Nathan
On 1/14/22, 11:26 PM, "Bharath Rupireddy" wrote: > On Sat, Jan 15, 2022 at 12:46 AM Bossart, Nathan wrote: >> I'd personally like to avoid creating two code paths for the same >> thing. Are there really cases when this one extra auxiliary process >> would be too many? And if so, how would a us

Re: Push down time-related SQLValue functions to foreign server

2022-01-18 Thread Tom Lane
I wrote: > Alexander Pyhalov writes: >> This means we'll translate something like >> explain select * from t where d > now() - '1 day'::interval; >> to >> select * from t where d > $1; > Right. After thinking about that a bit more, I see that this will result in a major redefinition of what is "

Re: [PATCH] allow src/tools/msvc/clean.bat script to be called from the root of the source tree

2022-01-18 Thread Juan José Santamaría Flecha
On Tue, Jan 18, 2022 at 5:49 PM Andrew Dunstan wrote: > > On 1/18/22 04:41, Juan José Santamaría Flecha wrote: > > In [1] capacity for $SUBJECT was added to most of the batch scripts, > > but clean.bat was not included. I propose to do so with the attached > > patch. > > > That looks a bit ugly.

Re: [PATCH] reduce page overlap of GiST indexes built using sorted method

2022-01-18 Thread sergei sh.
Hi, I've addressed Andrey Borodin's concerns about v2 of this patch by Aliaksandr Kalenik in attached version. Change list: * Number of pages to collect moved to GUC parameter "gist_sorted_build_page_buffer_size". * GistSortedBuildPageState type renamed to GistSortedBuildLevelState. * Comment

Re: Allow root ownership of client certificate key

2022-01-18 Thread Tom Lane
David Steele writes: > [ client-key-perm-002.patch ] I took a quick look at this and agree with the proposed behavior change, but also with your self-criticisms: > We may want to do the same on the server side to make the code blocks > look more similar. > > Also, on the server side the S_ISREG

Re: [PATCH] reduce page overlap of GiST indexes built using sorted method

2022-01-18 Thread Komяpa
Hello hackers, On Tue, Jan 18, 2022 at 11:26 PM sergei sh. wrote: > Hi, > > I've addressed Andrey Borodin's concerns about v2 of this patch by > Aliaksandr > Kalenik in attached version. > [snip] This patchset got some attention in the PostGIS development channel, as it is important to really

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 1:51 PM Robert Haas wrote: > Modify pg_basebackup to use a new COPY subprotocol for base backups. Andres pointed out to me that longfin is sad: 2022-01-18 14:52:35.484 EST [82470:4] LOG: server process (PID 82487) was terminated by signal 4: Illegal instruction: 4 2022-0

Re: Adding CI to our tree

2022-01-18 Thread Justin Pryzby
On Mon, Jan 17, 2022 at 12:16:19PM -0800, Andres Freund wrote: > I think it might still be worth adding stopgap way of running all tap tests on > windows though. Having a vcregress.pl function to find all directories with t/ > and run the tests there, shouldn't be a lot of code... I started doing

Re: [PATCH] allow src/tools/msvc/clean.bat script to be called from the root of the source tree

2022-01-18 Thread Andrew Dunstan
On 1/18/22 15:08, Juan José Santamaría Flecha wrote: > > > > > By the way, are pgbison.bat and pgflex.bat directly called anywhere? > > Not to my knowledge. One of the things that's annoying about them is > that the processor names are hardcoded, so if you install winflexbison > a

Re: refactoring basebackup.c

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 9:43 AM Jeevan Ladhe wrote: > The patch surely needs some grooming, but I am expecting some initial > review, specially in the area where we are trying to close the zstd stream > in bbsink_zstd_end_archive(). We need to tell the zstd library to end the > compression by call

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Andres Freund
Hi, On 2022-01-17 18:34:16 -0300, Alvaro Herrera wrote: > Maybe it would work to have a single LSN in shared memory, as an atomic > variable, which uses monotonic advance[1] to be updated. That could be a reasonable approach. > Whether this is updated or not would depend on a new GUC, maybe > t

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Tom Lane
Robert Haas writes: > Andres pointed out to me that longfin is sad: > 2022-01-18 14:52:35.484 EST [82470:4] LOG: server process (PID 82487) > was terminated by signal 4: Illegal instruction: 4 > Tom, any chance you can get a stack trace? Hmm, I'd assumed that was just a cosmic ray or something

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 4:32 PM Andres Freund wrote: > > Hi, > > On 2022-01-17 18:34:16 -0300, Alvaro Herrera wrote: > > Maybe it would work to have a single LSN in shared memory, as an atomic > > variable, which uses monotonic advance[1] to be updated. > > That could be a reasonable approach. > >

Re: slowest tap tests - split or accelerate?

2022-01-18 Thread Andres Freund
Hi, On 2022-01-18 12:49:16 -0500, Robert Haas wrote: > Here's a patch that splits up that file. Ah, nice! The split seems sensible to me. > Sadly, we've gained about 2.5 seconds of runtime as the price of > splitting the test. Arguably the options part could be split up a lot > more finely than

Re: Allow root ownership of client certificate key

2022-01-18 Thread David Steele
On 1/18/22 15:41, Tom Lane wrote: David Steele writes: I took a quick look at this and agree with the proposed behavior change, but also with your self-criticisms: We may want to do the same on the server side to make the code blocks look more similar. Also, on the server side the S_ISREG()

Re: Allow root ownership of client certificate key

2022-01-18 Thread Tom Lane
David Steele writes: > On 1/18/22 15:41, Tom Lane wrote: >> The only other nitpick I have is that I'd make the cross-references be >> to the two file names, ie like "Note that similar checks are performed >> in fe-secure-openssl.c ..." References to the specific functions seem >> likely to bit-ro

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 4:36 PM Tom Lane wrote: > Robert Haas writes: > > Andres pointed out to me that longfin is sad: > > > 2022-01-18 14:52:35.484 EST [82470:4] LOG: server process (PID 82487) > > was terminated by signal 4: Illegal instruction: 4 > > > Tom, any chance you can get a stack tra

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Tom Lane
I wrote: >> Tom, any chance you can get a stack trace? > Hmm, I'd assumed that was just a cosmic ray or something. My mistake: it's failing because of -fsanitize=alignment. Here's the stack trace: * frame #0: 0x00010885dfd0 postgres`sendFile(sink=0x7fdedf071cb0, readfilename="./globa

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Tom Lane
Robert Haas writes: > Unfortunately, I can't reproduce this locally, even with COPT=-Wall > -Werror -fno-omit-frame-pointer -fsanitize-trap=alignment > -Wno-deprecated-declarations -DWRITE_READ_PARSE_PLAN_TREES > -DSTRESS_SORT_INT_MIN -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Now that I re-rea

Re: a misbehavior of partition row movement (?)

2022-01-18 Thread Alvaro Herrera
On 2022-Jan-18, Alvaro Herrera wrote: > On 2022-Jan-18, Amit Langote wrote: > > > Would you like me to update the patch with the above renumbering or > > are you working on a new version yourself? > > I have a few very minor changes apart from that. Let me see if I can > get this pushed soon; i

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Andres Freund
On 2022-01-18 16:40:25 -0500, James Coleman wrote: > If I read between the lines I imagine you'd see even e.g. every 2s as > not that big of a deal here, right? Right. Even every 0.2s wouldn't be a problem.

Synchronizing slots from primary to standby

2022-01-18 Thread Hsu, John
Hello, The doc mentions that standby_slot_names is to only be used for waiting on physical slots. However, it seems like we calculate the flush_pos for logical slots as well in wait_for_standby_confirmation? Re-posting some of my previous comment since it seems like it got lost... In wait_for_

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 4:32 PM Andres Freund wrote: > I wonder if a very different approach could make sense here. Presumably this > wouldn't need to be queried at a very high frequency, right? If so, what about > storing the latest commit LSN for each backend in PGPROC? That could be > maintaine

Re: is ErrorResponse possible on Sync?

2022-01-18 Thread Rafi Shamim
I used your example and tried it with prepared statements. I captured the traffic with Wireshark. My client sent Bind/Execute/Sync messages, and PostgreSQL 14 sent back BindComplete/CommandComplete/ErrorResponse messages, followed by ReadyForQuery after that. So yes, it looks like ErrorResponse is

Re: Extend compatibility of PostgreSQL::Test::Cluster

2022-01-18 Thread Andrew Dunstan
On 12/31/21 11:22, Andrew Dunstan wrote: > On 12/31/21 11:20, Dagfinn Ilmari Mannsåker wrote: >> Andrew Dunstan writes: >> >>> + my $subclass = __PACKAGE__ . "::V_$maj"; >>> + bless $node, $subclass; >>> + unless ($node->isa(__PACKAGE__)) >>> + { >>> +

tab-complete COMPLETE_WITH_ATTR can become confused by table-lists.

2022-01-18 Thread Peter Smith
Hi. I stumbled onto a small quirk/bug in the tab-complete code. There are some places that suggest tab completions using the current table columns. These are coded like: COMPLETE_WITH_ATTR(prev2_wd, ""); The assumption is that the prev2_wd represents the table to select from. Normally, this wor

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-18 Thread Tom Lane
I wrote: > Are we sure it's an issue within Python, rather than something we > could dodge by invoking sysconfig differently? It's hard to believe > that sysconfig could be totally unfit for the purpose of finding out > the include path and would remain so for multiple years. I dug up a Debian 9

Re: In-placre persistance change of a relation

2022-01-18 Thread Kyotaro Horiguchi
At Tue, 18 Jan 2022 10:37:53 -0500, Tom Lane wrote in > Julien Rouhaud writes: > > The cfbot is failing on all OS with this version of the patch. Apparently > > v16-0002 introduces some usage of "testtablespace" client-side variable > > that's > > never defined, e.g. > > That test infrastruct

Re: slowest tap tests - split or accelerate?

2022-01-18 Thread Andres Freund
Hi, On 2022-01-18 13:40:40 -0800, Andres Freund wrote: > Maybe we really should do at least the most simplistic caching for initdbs, by > doing one initdb as part of the creation of temp_install. Then Cluster::init > would need logic to only use that if $params{extra} is empty. I hacked this toge

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread Andres Freund
Hi, On 2022-01-18 18:31:42 -0500, James Coleman wrote: > One other question on this: if we went with this would you expect a > new function to parallel pg_last_committed_xact()? I don't think I have an opinion the user interface aspect. > Or allow the xid and lsn in the return of pg_last_commit

Re: [Ext:] Re: Stream Replication not working

2022-01-18 Thread Allie Crawford
I was able to figure out what the problem was that was preventing my PostgreSQL Stream replication from replicating the changes even though all the health checks queries were showing that the replication had no problems. I am a newbie and I am learning PostgreSQL on my own, so it took me a while

Re: tab-complete COMPLETE_WITH_ATTR can become confused by table-lists.

2022-01-18 Thread Tom Lane
Peter Smith writes: > If there are spaces in the table-list like "t1, t2" then the word is > recognized as "t2" and it works as expected. > But, if there are no spaces in the table-list like "t1,t2" then the > word is recognized as "t1,t2", and since that is no such table name > the COMPLETE_WITH_

Re: Adding CI to our tree

2022-01-18 Thread Andres Freund
Hi, On 2022-01-18 15:08:47 -0600, Justin Pryzby wrote: > On Mon, Jan 17, 2022 at 12:16:19PM -0800, Andres Freund wrote: > > I think it might still be worth adding stopgap way of running all tap tests > > on > > windows though. Having a vcregress.pl function to find all directories with > > t/ >

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 8:05 PM Andres Freund wrote: > > Hi, > > On 2022-01-18 18:31:42 -0500, James Coleman wrote: > > One other question on this: if we went with this would you expect a > > new function to parallel pg_last_committed_xact()? > > I don't think I have an opinion the user interface

Re: docs: pg_replication_origin_oid() description does not match behaviour

2022-01-18 Thread Michael Paquier
On Tue, Jan 18, 2022 at 06:20:22PM +, Bossart, Nathan wrote: > +1 And done. -- Michael signature.asc Description: PGP signature

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Andres Freund
On 2022-01-18 17:12:00 -0500, Tom Lane wrote: > Robert Haas writes: > > Unfortunately, I can't reproduce this locally, even with COPT=-Wall > > -Werror -fno-omit-frame-pointer -fsanitize-trap=alignment > > -Wno-deprecated-declarations -DWRITE_READ_PARSE_PLAN_TREES > > -DSTRESS_SORT_INT_MIN -DENFOR

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-18 Thread Robert Haas
On Tue, Jan 18, 2022 at 5:12 PM Tom Lane wrote: > Now that I re-read what you did, I believe you need both of > > -fsanitize=alignment -fsanitize-trap=alignment > > to enable those traps to happen. That seems to be the case with > Apple's clang, anyway. Ah, I guess I copied and pasted the option

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-18 Thread James Coleman
On Tue, Jan 18, 2022 at 4:32 PM Andres Freund wrote: > > I wonder if a very different approach could make sense here. Presumably this > wouldn't need to be queried at a very high frequency, right? If so, what about > storing the latest commit LSN for each backend in PGPROC? That could be > maintai

RE: row filtering for logical replication

2022-01-18 Thread houzj.f...@fujitsu.com
On Tues, Jan 18, 2022 9:27 AM Peter Smith wrote: > Here are some review comments for v66-0001 (review of updates since > v65-0001) Thanks for the comments! > ~~~ > > 1. src/backend/catalog/pg_publication.c - GetTopMostAncestorInPublication > > @@ -276,17 +276,45 @@ GetPubPartitionOptionRelatio

  1   2   >