Re: Incorrect include file order in guc-file.l

2022-11-01 Thread John Naylor
On Wed, Nov 2, 2022 at 1:01 PM Julien Rouhaud wrote: > > Hi, > > On Wed, Nov 02, 2022 at 02:29:50PM +0900, Michael Paquier wrote: > > > > While reviewing a different patch, I have noticed that guc-file.l > > includes sys/stat.h in the middle of the PG internal headers. The > > usual practice is t

remap the .text segment into huge pages at run time

2022-11-01 Thread John Naylor
It's been known for a while that Postgres spends a lot of time translating instruction addresses, and using huge pages in the text segment yields a substantial performance boost in OLTP workloads [1][2]. The difficulty is, this normally requires a lot of painstaking work (unless your OS does superp

Re: Improve description of XLOG_RUNNING_XACTS

2022-11-01 Thread Amit Kapila
On Tue, Nov 1, 2022 at 4:33 PM Amit Kapila wrote: > > On Tue, Nov 1, 2022 at 12:53 PM Michael Paquier wrote: > > > > On Mon, Oct 17, 2022 at 02:53:57PM +0900, Michael Paquier wrote: > > > No objections from here if you want to go ahead with v3 and print the > > > full set of subxids on top of the

Re: Incorrect include file order in guc-file.l

2022-11-01 Thread Julien Rouhaud
Hi, On Wed, Nov 02, 2022 at 02:29:50PM +0900, Michael Paquier wrote: > > While reviewing a different patch, I have noticed that guc-file.l > includes sys/stat.h in the middle of the PG internal headers. The > usual practice is to have first postgres[_fe].h, followed by the > system headers and fi

Re: A new strategy for pull-up correlated ANY_SUBLINK

2022-11-01 Thread Andy Fan
Hi Andrey: > > In this patch we distinguish the above case and try to pull-up it within > > one step if it is helpful, It looks to me that what we need to do is just > > transform it to EXIST-SUBLINK. > Maybe code [1] would be useful for your purposes/tests. > Looks like we are resolving the sam

Incorrect include file order in guc-file.l

2022-11-01 Thread Michael Paquier
Hi all, While reviewing a different patch, I have noticed that guc-file.l includes sys/stat.h in the middle of the PG internal headers. The usual practice is to have first postgres[_fe].h, followed by the system headers and finally the internal headers. That's a nit, but all the other files do t

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Corey Huinker
> > > SELECT $1, $2 \gp 'foo' 'bar' > > I think this is a great idea, but I foresee people wanting to send that output to a file or a pipe like \g allows. If we assume everything after the \gp is a param, don't we paint ourselves into a corner?

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-01 Thread Dilip Kumar
On Mon, Oct 31, 2022 at 6:54 PM Simon Riggs wrote: > > > > What is the behavior if "nested_transactions" value is changed within > > > a transaction execution, suppose the value was on and we have created > > > a few levels of nested subtransactions and within the same transaction > > > I switched

Re: A new strategy for pull-up correlated ANY_SUBLINK

2022-11-01 Thread Andrey Lepikhov
On 2/11/2022 09:02, Andy Fan wrote: In the past we pull-up the ANY-sublink with 2 steps, the first step is to pull up the sublink as a subquery, and the next step is to pull up the subquery if it is allowed.  The benefits of this method are obvious, pulling up the subquery has more requirements,

A new strategy for pull-up correlated ANY_SUBLINK

2022-11-01 Thread Andy Fan
In the past we pull-up the ANY-sublink with 2 steps, the first step is to pull up the sublink as a subquery, and the next step is to pull up the subquery if it is allowed. The benefits of this method are obvious, pulling up the subquery has more requirements, even if we can just finish the first s

Re: Prefetch the next tuple's memory during seqscans

2022-11-01 Thread Andres Freund
Hi, On 2022-10-31 16:52:52 +1300, David Rowley wrote: > As part of the AIO work [1], Andres mentioned to me that he found that > prefetching tuple memory during hot pruning showed significant wins. > I'm not proposing anything to improve HOT pruning here I did try and reproduce my old results, an

Re: PL/pgSQL cursors should get generated portal names by default

2022-11-01 Thread Pavel Stehule
st 2. 11. 2022 v 0:39 odesílatel Tom Lane napsal: > There's a complaint at [1] about how you can't re-use the same > cursor variable name within a routine called from another routine > that's already using that name. The complaint is itself a bit > under-documented, but I believe it is referring

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-01 Thread Masahiko Sawada
On Mon, Oct 24, 2022 at 8:42 PM Masahiko Sawada wrote: > > On Wed, Oct 12, 2022 at 3:04 PM Amit Kapila wrote: > > > > On Tue, Oct 11, 2022 at 5:52 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Oct 7, 2022 at 2:00 PM Amit Kapila > > > wrote: > > > > > > > > About your point that having dif

Re: Glossary and initdb definition work for "superuser" and database/cluster

2022-11-01 Thread David G. Johnston
On Tue, Nov 1, 2022 at 5:20 PM Justin Pryzby wrote: > On Tue, Nov 01, 2022 at 03:47:15PM -0700, David G. Johnston wrote: > > > I think this is wrong: > > | https://www.postgresql.org/docs/devel/app-initdb.html > | -U username > | --username=username > | > | Selects the user name of the databa

Re: missing indexes in indexlist with partitioned tables

2022-11-01 Thread Arne Roland
Hi! I mainly changed the comments, the Assert and some casing. > From: David Rowley > Sent: Monday, October 3, 2022 00:51 > > * In the header comment in get_relation_info(), I don't think we need > to mention join removals explicitly. At a stretch, maybe mentioning > "unique proofs" might be ok,

Re: [Refactor]Avoid to handle FORCE_NOT_NULL/FORCE_NULL options when COPY TO

2022-11-01 Thread Michael Paquier
On Tue, Nov 01, 2022 at 05:51:42PM +0800, Richard Guo wrote: > I'm OK with not having these assertions. I have to admit they look > somewhat redundant here, after what ProcessCopyOptions has done. Thanks, and done. While on it, I have noticed some gaps with the coverage of the code, where we did

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-01 Thread David Rowley
On Tue, 1 Nov 2022 at 23:19, David Rowley wrote: > If nobody has any objections, I plan to push this tomorrow morning New > Zealand time (around 10 hours from now). Pushed. Thank you both for reviewing this. David

Re: Direct I/O

2022-11-01 Thread Andres Freund
Hi, On 2022-11-01 15:54:02 -0700, Andres Freund wrote: > On 2022-11-02 09:44:30 +1300, Thomas Munro wrote: > > Oh, so BufFile is palloc'd and contains one of these. BufFile is not > > even using direct I/O, but by these rules it would need to be > > palloc_io_align'd. I will think about what to

Re: Glossary and initdb definition work for "superuser" and database/cluster

2022-11-01 Thread Justin Pryzby
On Tue, Nov 01, 2022 at 03:47:15PM -0700, David G. Johnston wrote: > Hey, > > Recent threads have pointed out some long-standing doc language in initdb > that could be made more precise, especially in light of the relatively > recent addition of a glossary. Toward this end I'm attaching a patch t

Re: Error for WITH options on partitioned tables

2022-11-01 Thread Simon Riggs
Apologies, I only just noticed these messages. I have set WoA until I have read, understood and can respond to your detailed input, thanks. On Fri, 28 Oct 2022 at 22:21, David Zhang wrote: > > Hi Karina, > > I am not very clear about why `build_reloptions` is removed in patch > `v2-0002-better-er

Re: Collation version tracking for macOS

2022-11-01 Thread Thomas Munro
On Wed, Nov 2, 2022 at 1:42 AM Thomas Munro wrote: > On Tue, Nov 1, 2022 at 11:33 PM Peter Eisentraut > wrote: > > What I'm wondering is where those ICU installations are going to come > > from. In order for this project to be viable, we would need to convince > > some combination of ICU maintai

Re: Allow single table VACUUM in transaction block

2022-11-01 Thread Simon Riggs
On Thu, 27 Oct 2022 at 21:07, Justin Pryzby wrote: > > On Thu, Oct 27, 2022 at 10:31:31AM +0100, Simon Riggs wrote: > > Allows both ANALYZE and vacuum of toast tables, but not VACUUM FULL. > > Maybe I misunderstood what you meant: you said "not VACUUM FULL", but > with your patch, that works: > >

PL/pgSQL cursors should get generated portal names by default

2022-11-01 Thread Tom Lane
There's a complaint at [1] about how you can't re-use the same cursor variable name within a routine called from another routine that's already using that name. The complaint is itself a bit under-documented, but I believe it is referring to this ancient bit of behavior: A bound cursor v

Re: Commit fest 2022-11

2022-11-01 Thread Ian Lawrence Barwick
2022年11月2日(水) 1:36 Magnus Hagander :> > On Tue, Nov 1, 2022 at 11:59 AM Michael Paquier wrote: >> >> On Tue, Nov 01, 2022 at 07:10:15PM +0900, Ian Lawrence Barwick wrote: >> > Yup, Monday by the looks of it. >> > >> > My community login is "barwick", in case needed. >> >> Adding Magnus in CC, in c

Re: User functions for building SCRAM secrets

2022-11-01 Thread Jacob Champion
On Mon, Oct 31, 2022 at 1:27 PM Jonathan S. Katz wrote: > Having a set of SCRAM secret building functions would help in a few areas: I have mixed-to-negative feelings about this. Orthogonality with other methods seems reasonable, except we don't really recommend that people use those other method

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Simon Riggs
On Tue, 1 Nov 2022 at 20:48, Peter Eisentraut wrote: > > On 01.11.22 10:10, Simon Riggs wrote: > > On Fri, 28 Oct 2022 at 07:53, Peter Eisentraut > > wrote: > >> > >> This adds a new psql command \gp that works like \g (or semicolon) but > >> uses the extended query protocol. Parameters can also

Re: Direct I/O

2022-11-01 Thread Andres Freund
Hi, On 2022-11-02 09:44:30 +1300, Thomas Munro wrote: > On Wed, Nov 2, 2022 at 2:33 AM Justin Pryzby wrote: > > On Tue, Nov 01, 2022 at 08:36:18PM +1300, Thomas Munro wrote: > > > io_data_direct = whether to use O_DIRECT for main data files > > > io_wal_direct = ... for WAL > > > io_wal_ini

Glossary and initdb definition work for "superuser" and database/cluster

2022-11-01 Thread David G. Johnston
Hey, Recent threads have pointed out some long-standing doc language in initdb that could be made more precise, especially in light of the relatively recent addition of a glossary. Toward this end I'm attaching a patch that defines three terms: "bootstrap superuser", "database superuser" and "sup

Re: warn if GUC set to an invalid shared library

2022-11-01 Thread Justin Pryzby
On Mon, Oct 31, 2022 at 08:31:20AM -0500, Justin Pryzby wrote: > On Sun, Oct 30, 2022 at 04:12:33PM -0700, Maciek Sakrejda wrote: > > On Sat, Oct 29, 2022 at 10:40 AM Justin Pryzby wrote: > > > On Fri, Sep 02, 2022 at 05:24:58PM -0500, Justin Pryzby wrote: > > > > It caused no issue when I changed

Re: doc: Fix description of how the default user name is chosen

2022-11-01 Thread David G. Johnston
This is the only sentence I claim is factually incorrect, with a suggested re-wording. diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index 9494f28063..f375a0fc11 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++ b/doc/src/sgml/ref/psql-ref.sgml @@ -660,7 +660,8 @@ EOF

Re: Pluggable toaster

2022-11-01 Thread Aleksander Alekseev
Hi Nikita, > Please don't forget to run `pgindent` before formatting the patches > with `git format-patch` next time. There are also some compiler warnings, please see the attachment. > I'm going to submit a more detailed code review soon. -- Best regards, Aleksander Alekseev /home/eax/project

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-11-01 Thread Peter Eisentraut
On 01.11.22 19:01, Andres Freund wrote: I don't know how much longer we can rely on headers being -Wdeclaration-after-statement clean, my impression is that people don't have a lot of patience for C89isms anymore. I wonder if we should try to use -isystem for a bunch of external dependencies.

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Peter Eisentraut
On 01.11.22 10:10, Simon Riggs wrote: On Fri, 28 Oct 2022 at 07:53, Peter Eisentraut wrote: This adds a new psql command \gp that works like \g (or semicolon) but uses the extended query protocol. Parameters can also be passed, like SELECT $1, $2 \gp 'foo' 'bar' +1 for the concept. T

Re: Direct I/O

2022-11-01 Thread Thomas Munro
On Wed, Nov 2, 2022 at 2:33 AM Justin Pryzby wrote: > On Tue, Nov 01, 2022 at 08:36:18PM +1300, Thomas Munro wrote: > > io_data_direct = whether to use O_DIRECT for main data files > > io_wal_direct = ... for WAL > > io_wal_init_direct = ... for WAL-file initialisation > > You added 3 boolea

Re: Check return value of pclose() correctly

2022-11-01 Thread Peter Eisentraut
On 01.11.22 06:52, Tom Lane wrote: I think there are two issues here. POSIX says Upon successful return, pclose() shall return the termination status of the command language interpreter. Otherwise, pclose() shall return -1 and set errno to indicate the error. That is, first you

Re: Check return value of pclose() correctly

2022-11-01 Thread Peter Eisentraut
On 01.11.22 06:35, Michael Paquier wrote: - if (WIFEXITED(exitstatus)) + if (exitstatus == -1) + { + snprintf(str, sizeof(str), "%m"); + } This addition in wait_result_to_str() looks inconsistent with the existing callers of pclose() and ClosePipeStream() that check for -1 as exit s

Re: Segfault on logical replication to partitioned table with foreign children

2022-11-01 Thread Tom Lane
Ilya Gladyshev writes: > [ v2-0001-check-relkind-of-subscription-target-recursively.patch ] Hmm. I like Shi yu's way better (formal patch attached). Checking at CREATE/ALTER SUBSCRIPTION is much more complicated, and it's really insufficient, because what if someone adds a new partition after s

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-11-01 Thread Dagfinn Ilmari Mannsåker
Andres Freund writes: > Hi, > > Tom pinged me privately because mylodon, an animal enforcing C89/C99 > compatibility, was failed. This is due to perl on the machine being upgraded > to perl 5.36. > > Mylodon was failing because of: > > configure:18839: ccache clang-13 -c -Wall -Wmissing-prototype

Re: pg15 inherited stats expressions: cache lookup failed for statistics object

2022-11-01 Thread Tom Lane
Justin Pryzby writes: > I added a CF entry and marked RFC. > This should be included in v15.1. Right, done. regards, tom lane

perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-11-01 Thread Andres Freund
Hi, Tom pinged me privately because mylodon, an animal enforcing C89/C99 compatibility, was failed. This is due to perl on the machine being upgraded to perl 5.36. Mylodon was failing because of: configure:18839: ccache clang-13 -c -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-

Re: pg_dump: Refactor code that constructs ALTER ... OWNER TO commands

2022-11-01 Thread Corey Huinker
On Mon, Oct 24, 2022 at 5:54 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > Avoid having to list all the possible object types twice. Instead, only > _getObjectDescription() needs to know about specific object types. It > communicates back to _printTocEntry() whether an owner

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-11-01 Thread Jacob Champion
On Tue, Nov 1, 2022 at 10:03 AM Jacob Champion wrote: > I'm not familiar with "unregistered scheme" in this context and will > need to dig in. Unfortunately I can't reproduce with 3.0.0 on Ubuntu :( I'm suspicious that it may be related to [1], in which case the problem might be fixed by upgradi

Re: heavily contended lwlocks with long wait queues scale badly

2022-11-01 Thread Andres Freund
Hi, On 2022-11-01 11:19:02 -0400, Jonathan S. Katz wrote: > This is the type of fix that would make headlines in a major release > announcement (10x TPS improvement w/4096 connections?!). That is also part > of the tradeoff of backpatching this, is that we may lose some of the higher > visibility

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-11-01 Thread Jacob Champion
On Tue, Nov 1, 2022 at 5:30 AM wrote: > Sweet. I just created an account with username `habets`. Added! OpenSSL 3.0.0 doesn't get along with one of my new tests: # Failed test 'sslrootcert=system does not connect with private CA: matches' # at /Users/admin/pgsql/src/test/ssl/t/001_sslte

Re: Commit fest 2022-11

2022-11-01 Thread Magnus Hagander
On Tue, Nov 1, 2022 at 11:59 AM Michael Paquier wrote: > On Tue, Nov 01, 2022 at 07:10:15PM +0900, Ian Lawrence Barwick wrote: > > Yup, Monday by the looks of it. > > > > My community login is "barwick", in case needed. > > Adding Magnus in CC, in case you need the admin permissions on the CF > a

Re: BUG #17663:Connect to the database through jdbc, call the stored procedure containing the rollback statement,the database triggers an assertion, and the database is in recovery mode.

2022-11-01 Thread Tom Lane
=?gb2312?B?1dTG5Lnw?= writes: > The patch submitted addresses #17663 in the pgsql-b...@lists.postgresql.org > list. > Problem: Add the parameters --enable-debug and --enable-cassert when the > database is compiled. Driven by jdbc, the stored procedure containing rollbck > is called, and an asse

Re: heavily contended lwlocks with long wait queues scale badly

2022-11-01 Thread Andres Freund
Hi, On 2022-11-01 08:37:39 -0400, Robert Haas wrote: > On Tue, Nov 1, 2022 at 3:17 AM Bharath Rupireddy > wrote: > > Below are test results with v3 patch. +1 for back-patching it. > > The problem with back-patching stuff like this is that it can have > unanticipated consequences. I think that th

Re: [Refactor]Avoid to handle FORCE_NOT_NULL/FORCE_NULL options when COPY TO

2022-11-01 Thread Mingli Zhang
Hi, > > The point about cleaning up the attribute handling of FORCE_NOT_NULL > and FORCE_NULL in the COPY TO path is a good catch, though, so let's > remove all that. I'll go apply this part of the patch in a bit, or > tomorrow. > -- > Michael Thanks for review!

Re: ResourceOwner refactoring

2022-11-01 Thread Andres Freund
Hi, On 2022-11-01 12:39:39 +0200, Heikki Linnakangas wrote: > > 1) As far as I can tell, the way ResourceOwnerReleaseAll() now works seems > > to > > assume that within a phase the reset order does not matter. I don't > > think > > that's a good assumption. I e.g. have a patch to replace

Re: heavily contended lwlocks with long wait queues scale badly

2022-11-01 Thread Jonathan S. Katz
On 11/1/22 8:37 AM, Robert Haas wrote: On Tue, Nov 1, 2022 at 3:17 AM Bharath Rupireddy wrote: Below are test results with v3 patch. +1 for back-patching it. First, awesome find and proposed solution! The problem with back-patching stuff like this is that it can have unanticipated consequen

BUG #17663:Connect to the database through jdbc, call the stored procedure containing the rollback statement,the database triggers an assertion, and the database is in recovery mode.

2022-11-01 Thread 赵其桂
Dear developer: The patch submitted addresses #17663 in the pgsql-b...@lists.postgresql.org list. Problem: Add the parameters --enable-debug and --enable-cassert when the database is compiled. Driven by jdbc, the stored procedure containing rollbck is called, and an assertion occurs. Cause of th

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-11-01 Thread thomas
On Mon, 31 Oct 2022 23:36:16 +, Jacob Champion said: >> I wanted to get feedback on the approach before wordsmithing too >> much. > > I've added this to tomorrow's CF [1]. Thomas, if you get (or already > have) a PG community username, I can add you as an author. Sweet. I just created an acco

Re: Direct I/O

2022-11-01 Thread Justin Pryzby
On Tue, Nov 01, 2022 at 08:36:18PM +1300, Thomas Munro wrote: > Hi, > > Here is a patch to allow PostgreSQL to use $SUBJECT. It is from the > AIO patch-set[1]. It adds three new settings, defaulting to off: > > io_data_direct = whether to use O_DIRECT for main data files > io_wal_direct = .

Re: Privileges on PUBLICATION

2022-11-01 Thread Peter Eisentraut
On 20.06.22 16:01, Antonin Houska wrote: On Wed, May 18, 2022, at 6:44 AM, Antonin Houska wrote: ok, please see the next version. The new paragraph looks good to me. I'm not sure if the CREATE PUBLICATION is the right place to provide such information. As I suggested in a previous email [1], yo

Re: real/float example for testlibpq3

2022-11-01 Thread Tom Lane
Peter Eisentraut writes: > If we want to start documenting the binary format for all data types, it > should go into the "Data Types" chapter. There, we already document the > text format in great detail, so putting the binary format in there as > well would make sense. Agreed that the libpq

Re: pg15 inherited stats expressions: cache lookup failed for statistics object

2022-11-01 Thread Justin Pryzby
On Tue, Nov 01, 2022 at 05:33:24PM +0800, Richard Guo wrote: > On Mon, Oct 31, 2022 at 1:33 PM Michael Paquier wrote: > > > On Mon, Oct 31, 2022 at 01:12:09PM +0800, Richard Guo wrote: > > > BTW, I noticed a micro-optimization opportunity in examine_variable that > > > we can fetch the RangeTblEn

Re: pg_upgrade allows itself to be run twice

2022-11-01 Thread Justin Pryzby
On Tue, Nov 01, 2022 at 01:54:35PM +0100, Peter Eisentraut wrote: > On 07.07.22 08:22, Justin Pryzby wrote: > > > This one comes from NextOID in the control data file after a fresh > > > initdb, and GetNewObjectId() would enforce that in a postmaster > > > environment to be FirstNormalObjectId when

Re: pg_upgrade allows itself to be run twice

2022-11-01 Thread Peter Eisentraut
On 07.07.22 08:22, Justin Pryzby wrote: This one comes from NextOID in the control data file after a fresh initdb, and GetNewObjectId() would enforce that in a postmaster environment to be FirstNormalObjectId when assigning the first user OID. Would you imply an extra step at the end of initdb t

Re: Collation version tracking for macOS

2022-11-01 Thread Thomas Munro
On Tue, Nov 1, 2022 at 11:33 PM Peter Eisentraut wrote: > What I'm wondering is where those ICU installations are going to come > from. In order for this project to be viable, we would need to convince > some combination of ICU maintainers, OS packagers, and PGDG packagers to > provide and mainta

Re: ResourceOwner refactoring

2022-11-01 Thread Robert Haas
On Tue, Nov 1, 2022 at 6:39 AM Heikki Linnakangas wrote: > However, I feel that trying to enforce a particular order moves the > goalposts. If we need that, let's add it as a separate patch later. I don't really see it that way, because with the current implementation, we do all resources of a pa

Re: [patch] \g with multiple result sets and \watch with copy queries

2022-11-01 Thread Daniel Verite
Corey Huinker wrote: > I think that because it is more complicated than usual psql, we may want to > comment on the intention of the tests and some of the less-than-common psql > elements (\set concatenation, resetting \o, etc). If you see value in that > I can amend the patch. If the int

Re: heavily contended lwlocks with long wait queues scale badly

2022-11-01 Thread Robert Haas
On Tue, Nov 1, 2022 at 3:17 AM Bharath Rupireddy wrote: > Below are test results with v3 patch. +1 for back-patching it. The problem with back-patching stuff like this is that it can have unanticipated consequences. I think that the chances of something like this backfiring are less than for a pa

Re: Refactor to introduce pg_strcoll().

2022-11-01 Thread Peter Eisentraut
On 15.10.22 01:00, Jeff Davis wrote: On Thu, 2022-10-13 at 10:57 +0200, Peter Eisentraut wrote: It's a bit confusing that arguments must be NUL-terminated, but the length is still specified.  Maybe another sentence to explain that would be helpful. Added a comment. It was a little frustrating

Re: Prefetch the next tuple's memory during seqscans

2022-11-01 Thread David Rowley
On Wed, 2 Nov 2022 at 00:09, Andy Fan wrote: > I just have a different platforms at hand, Here is my test with > Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz. > shared_buffers has been set to big enough to hold all the data. Many thanks for testing that. Those numbers look much better than the one

Re: Prefetch the next tuple's memory during seqscans

2022-11-01 Thread Thomas Munro
On Wed, Nov 2, 2022 at 12:09 AM Andy Fan wrote: > By theory, Why does the preferch make thing better? I am asking this > because I think we need to read the data from buffer to cache line once > in either case (I'm obvious wrong in face of the test result.) CPUs have several different kinds of 'h

Tests for psql \g and \o

2022-11-01 Thread Daniel Verite
Hi, Here's a patch adding regression tests for \g and \o, and TAP tests for \g | program, It's a follow up to the discussion at [1]. Since this discussion already has a slot in the CF [2] with a committed patch, let's start a new separate thread. [1] https://www.postgresql.org/message-id/4333844

Re: [PATCH] Improve tab completion for ALTER TABLE on identity columns

2022-11-01 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 14.10.22 16:31, Dagfinn Ilmari Mannsåker wrote: >>> I noticed that psql has no tab completion around identity columns in >>> ALTER TABLE, so here's some patches for that. >> Added to the next commit fest: >> https://commitfest.postgresql.org/40/3947/ > > Committed.

Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-01 Thread David Rowley
Part of the work that Thomas mentions in [1], regarding Direct I/O, has certain requirements that pointers must be page-aligned. I've attached a patch which implements palloc_aligned() and MemoryContextAllocAligned() which accept an 'alignto' parameter which must be a power-of-2 value. The memory

Re: [PATCH] Improve tab completion for ALTER TABLE on identity columns

2022-11-01 Thread Peter Eisentraut
On 26.10.22 00:45, Matheus Alcantara wrote: I noticed that psql has no tab completion around identity columns in ALTER TABLE, so here's some patches for that. In passing, I also added completion for ALTER SEQUECNE … START, which was missing for some reason. I've tested all 4 of your patches,

Re: [PATCH] Improve tab completion for ALTER TABLE on identity columns

2022-11-01 Thread Peter Eisentraut
On 14.10.22 16:31, Dagfinn Ilmari Mannsåker wrote: I noticed that psql has no tab completion around identity columns in ALTER TABLE, so here's some patches for that. Added to the next commit fest: https://commitfest.postgresql.org/40/3947/ Committed.

Re: Latches vs lwlock contention

2022-11-01 Thread Thomas Munro
On Fri, Oct 28, 2022 at 4:56 PM Thomas Munro wrote: > See attached sketch patches. I guess the main thing that may not be > good enough is the use of a fixed sized latch buffer. Memory > allocation in don't-throw-here environments like the guts of lock code > might be an issue, which is why it j

Re: Prefetch the next tuple's memory during seqscans

2022-11-01 Thread Andy Fan
Hi: > Different platforms would be good. Certainly, 1 platform isn't a good > enough indication that this is going to be useful. I just have a different platforms at hand, Here is my test with Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz. shared_buffers has been set to big enough to hold all the da

Re: Improve description of XLOG_RUNNING_XACTS

2022-11-01 Thread Amit Kapila
On Tue, Nov 1, 2022 at 12:53 PM Michael Paquier wrote: > > On Mon, Oct 17, 2022 at 02:53:57PM +0900, Michael Paquier wrote: > > No objections from here if you want to go ahead with v3 and print the > > full set of subxids on top of the information about these > > overflowing. > > While browsing th

Re: Commit fest 2022-11

2022-11-01 Thread Michael Paquier
On Tue, Nov 01, 2022 at 07:10:15PM +0900, Ian Lawrence Barwick wrote: > Yup, Monday by the looks of it. > > My community login is "barwick", in case needed. Adding Magnus in CC, in case you need the admin permissions on the CF app. (I have no idea how to do it, and I likely lack the permissions

Re: Commit fest 2022-11

2022-11-01 Thread Michael Paquier
On Tue, Nov 01, 2022 at 07:15:34PM +0900, Ian Lawrence Barwick wrote: > From casual observation it's been like that for the last few CFs, i.e. there > have been effectively two CFMs. > > I am on record as volunteering earlier in the thread, with the caveat that > there > will be a limit on what I

Re: ResourceOwner refactoring

2022-11-01 Thread Heikki Linnakangas
On 01/11/2022 02:15, Andres Freund wrote: On 2022-10-31 10:51:36 +0100, Heikki Linnakangas wrote: These are functions where quite a lot of things happen between the ResourceOwnerEnlarge and ResourceOwnerRemember calls. It's important that there are no unrelated ResourceOwnerRemember() calls in t

Re: real/float example for testlibpq3

2022-11-01 Thread Peter Eisentraut
On 01.07.22 19:18, Mark Wong wrote: Just wanted to do another quick check in before I go too far. How do does this start look? Extending the libpq section with a code snippet and description per data type? If we want to start documenting the binary format for all data types, it should go int

Re: [PATCH] Backport perl tests for pg_upgrade from 322becb60

2022-11-01 Thread Anton A. Melnikov
Add backport to REL_14_STABLE. Unlike to the 13th version's one there are still some differences in the final dumps, eg during upgrade test 12->14. The similar differences present during upgrade test 12->master. Achieving zero dump diffs needs additional work, now in progress. With best regards

Re: Collation version tracking for macOS

2022-11-01 Thread Peter Eisentraut
On 22.10.22 03:22, Thomas Munro wrote: Suppose your pgdata encounters a PostgreSQL linked against a later ICU library, most likely after an OS upgrade or migratoin, a pg_upgrade, or via streaming replication. You might get a new error "can't find ICU collation 'en' with version '153.14'; HINT: i

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-11-01 Thread Etsuro Fujita
Hi David, On Sat, Oct 1, 2022 at 5:54 AM David Zhang wrote: > After rebase the file `postgres_fdw.out` and applied to master branch, > make and make check are all ok for postgres_fdw. Thanks for testing! Attached is a rebased version of the patch. Best regards, Etsuro Fujita v10-postgres-fdw

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-01 Thread David Rowley
On Tue, 1 Nov 2022 at 20:55, Bharath Rupireddy wrote: > I took another look at v3 patch today and it looked good to me, hence > marked it RfC - https://commitfest.postgresql.org/40/3967/ Many thanks for reviewing this. If nobody has any objections, I plan to push this tomorrow morning New Zealan

Re: Commit fest 2022-11

2022-11-01 Thread Ian Lawrence Barwick
2022年11月1日(火) 17:59 Simon Riggs : > > On Mon, 31 Oct 2022 at 05:42, Michael Paquier wrote: > > > As per the world clock, the next commit fest will begin in 30 hours > > (11/1 0:00 AoE time). I may have missed something, but it looks like > > we have no CFM for this one yet. > > > > Opinions, thou

Re: Commit fest 2022-11

2022-11-01 Thread Ian Lawrence Barwick
2022年11月1日(火) 13:40 Michael Paquier : > > On Tue, Nov 01, 2022 at 10:01:05AM +0900, Ian Lawrence Barwick wrote: > > This is on my bucket list of things to do some day, so I guess now is as > > bad a > > time as any :). Caveat is that this will have to be a personal > > free-time project, > > so wo

Re: Pluggable toaster

2022-11-01 Thread Aleksander Alekseev
Hi Nikita, > Aleksander, thanks for the discussion! It seems to me that I have to add some > parts > of it to API documentation, to clarify the details on API purpose and > use-cases. I'm in the process of testing and reviewing the v23 patchset. So far I just wanted to report that there seems t

Re: [Refactor]Avoid to handle FORCE_NOT_NULL/FORCE_NULL options when COPY TO

2022-11-01 Thread Richard Guo
On Tue, Nov 1, 2022 at 3:41 PM Michael Paquier wrote: > On Tue, Aug 02, 2022 at 04:13:30PM +0800, Zhang Mingli wrote: > > On Aug 2, 2022, 12:30 +0800, Kyotaro Horiguchi , > wrote: > >> There are some other option combinations that are rejected > >> by ProcessCopyOptions. On the other hand *re*che

Re: pg15 inherited stats expressions: cache lookup failed for statistics object

2022-11-01 Thread Richard Guo
On Mon, Oct 31, 2022 at 1:33 PM Michael Paquier wrote: > On Mon, Oct 31, 2022 at 01:12:09PM +0800, Richard Guo wrote: > > BTW, I noticed a micro-optimization opportunity in examine_variable that > > we can fetch the RangeTblEntry for 'onerel' outside the foreach loop > > when iterating the extend

Re: psql: Add command to use extended query protocol

2022-11-01 Thread Simon Riggs
On Fri, 28 Oct 2022 at 07:53, Peter Eisentraut wrote: > > This adds a new psql command \gp that works like \g (or semicolon) but > uses the extended query protocol. Parameters can also be passed, like > > SELECT $1, $2 \gp 'foo' 'bar' +1 for the concept. The patch looks simple and complete.

Re: Commit fest 2022-11

2022-11-01 Thread Simon Riggs
On Mon, 31 Oct 2022 at 05:42, Michael Paquier wrote: > As per the world clock, the next commit fest will begin in 30 hours > (11/1 0:00 AoE time). I may have missed something, but it looks like > we have no CFM for this one yet. > > Opinions, thoughts or volunteers? If we have no volunteers, ma

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-01 Thread Julien Tachoires
Hi, Le mar. 1 nov. 2022 à 09:39, Dilip Kumar a écrit : > > On Fri, Oct 28, 2022 at 10:55 PM Julien Tachoires wrote: > > > > Hi, > > > > Le lun. 26 sept. 2022 à 15:57, Simon Riggs > > a écrit : > > > > > > On Fri, 16 Sept 2022 at 13:20, Simon Riggs > > > wrote: > > > > > > > > Thanks for the r

Re: Lockless queue of waiters in LWLock

2022-11-01 Thread Alexander Korotkov
Hi Andres, Thank you for your feedback. On Tue, Nov 1, 2022 at 2:15 AM Andres Freund wrote: > On 2022-10-31 14:38:23 +0400, Pavel Borisov wrote: > > If the lock state contains references to the queue head and tail, we can > > implement a lockless queue of waiters for the LWLock. Adding new item

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-01 Thread Dilip Kumar
On Fri, Oct 28, 2022 at 10:55 PM Julien Tachoires wrote: > > Hi, > > Le lun. 26 sept. 2022 à 15:57, Simon Riggs > a écrit : > > > > On Fri, 16 Sept 2022 at 13:20, Simon Riggs > > wrote: > > > > > > Thanks for the review. > > > > > > v10 attached > > > > v11 attached, corrected for recent commit

Re: Adding doubly linked list type which stores the number of items in the list

2022-11-01 Thread Bharath Rupireddy
On Mon, Oct 31, 2022 at 8:28 PM Bharath Rupireddy wrote: > > On Mon, Oct 31, 2022 at 6:58 PM Aleksander Alekseev > wrote: > > > > Hi hackers, > > > > > I will take another look at v3 tomorrow and probably mark it RfC. > > > > I very much like the patch. While on it: I took another look at v3 pat

Re: [Refactor]Avoid to handle FORCE_NOT_NULL/FORCE_NULL options when COPY TO

2022-11-01 Thread Michael Paquier
On Tue, Aug 02, 2022 at 04:13:30PM +0800, Zhang Mingli wrote: > On Aug 2, 2022, 12:30 +0800, Kyotaro Horiguchi , > wrote: >> There are some other option combinations that are rejected >> by ProcessCopyOptions. On the other hand *re*checking all >> combinations that the function should have rejecte

Direct I/O

2022-11-01 Thread Thomas Munro
Hi, Here is a patch to allow PostgreSQL to use $SUBJECT. It is from the AIO patch-set[1]. It adds three new settings, defaulting to off: io_data_direct = whether to use O_DIRECT for main data files io_wal_direct = ... for WAL io_wal_init_direct = ... for WAL-file initialisation O_DIRECT

Re: Improve description of XLOG_RUNNING_XACTS

2022-11-01 Thread Michael Paquier
On Mon, Oct 17, 2022 at 02:53:57PM +0900, Michael Paquier wrote: > No objections from here if you want to go ahead with v3 and print the > full set of subxids on top of the information about these > overflowing. While browsing the CF entries, this was still listed. Amit, any updates? -- Michael

Re: heavily contended lwlocks with long wait queues scale badly

2022-11-01 Thread Bharath Rupireddy
On Tue, Nov 1, 2022 at 5:21 AM Andres Freund wrote: > > On 2022-10-31 16:21:06 +0530, Bharath Rupireddy wrote: > > BTW, I've seen a sporadic crash (SEGV) with the patch in bg writer > > with the same set up [1], I'm not sure if it's really because of the > > patch. I'm unable to reproduce it now a

Re: ICU for global collation

2022-11-01 Thread Michael Paquier
On Fri, Oct 21, 2022 at 05:32:38PM +0300, Marina Polyakova wrote: > Hello! > > I discovered an interesting behaviour during installcheck runs when the > cluster was initialized with ICU locale provider: > > $ initdb --locale-provider icu --icu-locale en-US -D data && > 2) The option --no-locale i