Re: On disable_cost

2024-10-10 Thread David Rowley
On Fri, 11 Oct 2024 at 02:02, Alena Rybakina wrote: > On 10.10.2024 15:43, David Rowley wrote: > > > If anyone wants to take a look at the attached, please do so. > > Otherwise, I'm pretty happy with it and will likely push it on New > > Zealand Friday (aka later today). > > I think you missed som

Re: general purpose array_sort

2024-10-10 Thread jian he
tricky case: should we allow array element type to be composite/domain? currently seems to work fine. create table t(b int[]); insert into t values ('{{1,3}}'), ('{{1,2}}'); select array_sort((select array_agg(t) from t), 'desc'); array_sort --- {"(\

Doc of typmod arg perhaps deserves an update

2024-10-10 Thread Steve Lau
# Context I hit the issue described in this thread when building my toy pgvector implementation: https://www.postgresql.org/message-id/162867790903161317q178e11e6ie276d1254c279dd1%40mail.gmail.com, TLDR of the thread is that: When you define a new custom type that needs a type modifier, it is

Missing deconstruct_array_builtin usage

2024-10-10 Thread Bertrand Drouvot
Hi hackers, While working on [1], I noticed that we missed using deconstruct_array_builtin() in 062a8444242. Indeed, d746021de1 added construct_array_builtin and deconstruct_array_builtin but , later on, 062a8444242 made use of deconstruct_array for TEXTOID. Please find attached a tiny patch to

Re: Enhance file_fdw to report processed and skipped tuples in COPY progress

2024-10-10 Thread Fujii Masao
On 2024/10/04 2:12, Masahiko Sawada wrote: Hi, On Thu, Oct 3, 2024 at 2:23 AM Fujii Masao wrote: Hi, Currently, file_fdw updates several columns in the pg_stat_progress_copy view, like relid and bytes_processed, but it doesn't track tuples_processed or tuples_skipped. Monitoring these wou

Re: Add contrib/pg_logicalsnapinspect

2024-10-10 Thread Peter Smith
Hi, Here are a few comments for patch set v13* // Patch v13-0001 == Commit message 1.1 /were no use case/was no use case/ ~~~ 1.2 It seemed a bit odd that the switch cases for 'construct_array_builtin' are not the same as those for 'deconstruct_array_builtin'. For example, all th

Re: sunsetting md5 password support

2024-10-10 Thread Heikki Linnakangas
On 11/10/2024 00:03, Bruce Momjian wrote: On Wed, Oct 9, 2024 at 10:30:15PM +0200, Jelte Fennema-Nio wrote: On Wed, 9 Oct 2024 at 21:55, Nathan Bossart wrote: In this message, I propose a multi-year, incremental approach to remove MD5 password support from Postgres. +many for the general id

Re: Function for listing pg_wal/summaries directory

2024-10-10 Thread Fujii Masao
On 2024/10/08 23:36, Nathan Bossart wrote: On Tue, Oct 08, 2024 at 01:19:52PM +0900, Michael Paquier wrote: On Tue, Oct 08, 2024 at 12:41:16PM +0900, Fujii Masao wrote: One benefit of supporting something like pg_ls_summariesdir() is that it allows us to view the last modification time of ea

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Furthermore, the queries in this case are 10667 bytes in total, so should fit inside ten-ish MTUs in an AWS VPC with the pg wire protocol overhead. So I'm not too worried about the time spent sending the queries. Still, it would be interesting to measure the impact of storing them in the DB and pre

Re: sunsetting md5 password support

2024-10-10 Thread Tom Lane
Andrew Dunstan writes: > Hmm, yeah. It would be easy enough to prevent MD5 passwords in things > like CREATE ROLE / ALTER ROLE, but harder to check for MD5 if there are > direct updates to pg_authid. Maybe we need to teach pg_dumpall a way to > do that as a workaround? That seems like a pretty

Re: GUC names in messages

2024-10-10 Thread Peter Smith
On Wed, Oct 9, 2024 at 8:54 PM Michael Paquier wrote: > > Applied the previous patch after looking at it again. > -- > Michael AFAIK this brings my year-long "GUC names" thread to a conclusion. Thanks, Michael for helping to get these last couple of patches pushed, and thanks to everybody else w

Remove unlogged materialized view persistence handling

2024-10-10 Thread px shi
Hi, Since materialized views do not support the unlogged type, I think that we should remove the logic and handling related to unlogged. Therefore, I have submitted a patch. Regards, Pixian Shi 0001-Remove-unlogged-materialized-view-persistence-handli.patch Description: Binary data

Re: Enhance file_fdw to report processed and skipped tuples in COPY progress

2024-10-10 Thread Yugo Nagata
On Fri, 11 Oct 2024 10:53:10 +0900 Fujii Masao wrote: > > > On 2024/10/04 2:12, Masahiko Sawada wrote: > > Hi, > > > > On Thu, Oct 3, 2024 at 2:23 AM Fujii Masao > > wrote: > >> > >> Hi, > >> > >> Currently, file_fdw updates several columns in the pg_stat_progress_copy > >> view, > >> like

Re: On disable_cost

2024-10-10 Thread Laurenz Albe
On Fri, 2024-10-11 at 17:24 +1300, David Rowley wrote: > On Fri, 11 Oct 2024 at 02:02, Alena Rybakina > wrote: > > On 10.10.2024 15:43, David Rowley wrote: > > > > > If anyone wants to take a look at the attached, please do so. > > > Otherwise, I'm pretty happy with it and will likely push it on

Re: Remove unlogged materialized view persistence handling

2024-10-10 Thread px shi
> > +1 > > Materialized view is introduced by 3bf3ab8c563 and at that UNLOGGED was > allowed, and it is disallowed by another commit 3223b25ff73. However, > it seems that the tab-complement is missed to fixed. > > The Assert for RELKIND_MATVIEW in heapam_hander.c was introduced in > d25f519107b > a

Re: sunsetting md5 password support

2024-10-10 Thread Laurenz Albe
On Thu, 2024-10-10 at 18:39 -0400, Tom Lane wrote: > Jesper Pedersen writes: > > On 10/10/24 5:45 PM, Heikki Linnakangas wrote: > > > Note that some authentication methods like LDAP and Radius use > > > "password" authentication on the wire. > > > Please, deprecate - aka remove - old methods. >

Fix attributes of consume_xids and consume_xids_until

2024-10-10 Thread Yushi Ogiwara
Hi, I found the attributes of the functions consume_xids and consume_xids_until are incorrectly specified: - Both functions are marked as IMMUTABLE attribute, but they advance the transaction ID, which modifies the system state. Thus, they should be marked as VOLATILE. - Additionally, both

Re: Doc: typo in config.sgml

2024-10-10 Thread Tatsuo Ishii
> We can check non-ASCII letters SGML/XML files by preparing "allowlist" > that contains lines which are allowed to have non-ascii characters, > although this list will need to be maintained when lines in it are modified. > I've attached a patch to add a simple Perl script to do this. I doubt it r

Re: Remove unlogged materialized view persistence handling

2024-10-10 Thread Yugo Nagata
On Fri, 11 Oct 2024 10:35:17 +0800 px shi wrote: > Hi, > > Since materialized views do not support the unlogged type, I think that we > should remove the logic and handling related to unlogged. Therefore, I have > submitted a patch. +1 Materialized view is introduced by 3bf3ab8c563 and at that

Re: Doc: typo in config.sgml

2024-10-10 Thread Yugo NAGATA
On Fri, 11 Oct 2024 12:16:50 +0900 (JST) Tatsuo Ishii wrote: > > We can check non-ASCII letters SGML/XML files by preparing "allowlist" > > that contains lines which are allowed to have non-ascii characters, > > although this list will need to be maintained when lines in it are modified. > > I've

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Tom Lane
Mikael Sand writes: > I wonder if it would be possible to link pgcommon_shlib and pgport_shlib > statically into pq and do LTO already at that point, such that consumers of > libpq could merely link against libpq.a and that would be enough? What would be the point? A typical build will still hav

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-10-10 Thread Masahiko Sawada
On Tue, Oct 8, 2024 at 8:34 PM Michael Paquier wrote: > > On Mon, Oct 07, 2024 at 03:23:08PM -0700, Masahiko Sawada wrote: > > In the benchmark, I've applied the v20 patch set and 'master' in the > > result refers to a19f83f87966. And I disabled CPU turbo boost where > > possible. Overall, v20 pat

Re: Add contrib/pg_logicalsnapinspect

2024-10-10 Thread Masahiko Sawada
On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot wrote: > > Hi, > > On Thu, Oct 10, 2024 at 12:05:10AM -0700, Masahiko Sawada wrote: > > On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot > > wrote: > > > So I think that having > > > construct_array_builtin()/deconstruct_array_builtin() > > > takin

Re: Set query_id for query contained in utility statement

2024-10-10 Thread jian he
On Wed, Oct 9, 2024 at 4:49 PM Anthonin Bonnefoy wrote: > > Here is a new version of the patchset. > > 0001: Add tests to cover the current behaviour: Missing nested > statements for CreateTableAs, DeclareCursor and MaterializedViews, > nested statements reported by explain including the whole str

Re: allowing extensions to control planner behavior

2024-10-10 Thread Andrei Lepikhov
On 10/10/24 23:51, Robert Haas wrote: On Wed, Sep 18, 2024 at 11:48 AM Robert Haas wrote: 1. If you want to specify in-query hints using comments, how does your extension get access to the comments? Having designed two features [1,2] that do the stuff mostly similar to pg_hint_plan but based on

Re: sunsetting md5 password support

2024-10-10 Thread Tom Lane
Jesper Pedersen writes: > On 10/10/24 5:45 PM, Heikki Linnakangas wrote: >> Note that some authentication methods like LDAP and Radius use >> "password" authentication on the wire. > Please, deprecate - aka remove - old methods. > All client libraries have caught up, and if they havn't then it t

RE: Improve EXPLAIN output for multicolumn B-Tree Index

2024-10-10 Thread Masahiro.Ikeda
> The CF bot is red for some time now, please provide a rebase. Thanks. I have attached the rebased patch. Regards, -- Masahiro Ikeda NTT DATA CORPORATION v3-0001-Support-Non-Key-Filter-for-multicolumn-B-Tree-Ind.patch Description: v3-0001-Support-Non-Key-Filter-for-multicolumn-B-Tree-Ind.pat

Re: Inconsistent RestrictInfo serial numbers

2024-10-10 Thread Ashutosh Bapat
On Thu, Oct 10, 2024 at 7:37 AM Richard Guo wrote: > > > > > > So may > > > > be we should do this processing elsewhere and replace the original > > > > clause itself? > > > > > > I’m not sure about this. Different versions of the same qual clause > > > can lead to different conclusions about whe

Re: sunsetting md5 password support

2024-10-10 Thread Christoph Moench-Tegeder
## Heikki Linnakangas (hlinn...@iki.fi): > This is a bit weird state. What exactly is "upgrading"? I guess you > mean pg_upgrade, but lots of people use pg_dump & restore or logical > replication or something else entirely for upgrading. That's > indistinguishable from setting a pre-hashed MD5 pas

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Aleksander Alekseev
Hi Mikael, > This is for compiling a c++ application that uses libpq with the -static > flag, the server compiles fine. OK. I couldn't quite do this because the only Linux machine I have at the moment runs Raspbian and there doesn't seem to be a static glibc available for it. But here is how to

Re: On disable_cost

2024-10-10 Thread David Rowley
On Tue, 8 Oct 2024 at 04:14, Robert Haas wrote: > I think you have adequate consensus to proceed with this. I'd just ask > that you don't disappear completely if it turns out that there are > problems. I accept that my commit created this problem and I'm > certainly willing to be involved too if w

RE: incorrect wal removal due to max_slot_wal_keep_size

2024-10-10 Thread Hayato Kuroda (Fujitsu)
Dear Jeff, Thanks for reporting the issue. I've tried to reproduce the issue (by adding delay on worker-side and immediate shut-down), but not done yet. If possible, could you please share a script to reproduce? It is helpful to analyze. > I'm going to try to reproduce this on 17.0, but in the

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-10-10 Thread Ashutosh Bapat
Hi hackers, Here's next set of patch with following changes 1. Addressed some of Alvaro's comments which he gave offlist. 2. One of Alvaro's comments made me realise that there is opportunity to save planning time. Patch 0003 added for the same. 3. Alvaro suggested to try simplehash.h instead of d

Re: Add contrib/pg_logicalsnapinspect

2024-10-10 Thread Bertrand Drouvot
Hi, On Thu, Oct 10, 2024 at 12:05:10AM -0700, Masahiko Sawada wrote: > On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot > wrote: > > So I think that having construct_array_builtin()/deconstruct_array_builtin() > > taking care of XIDOID is the way to go. If that makes sense to you then I'll > > sub

Re: Doc: typo in config.sgml

2024-10-10 Thread Tatsuo Ishii
> Bruce Momjian writes: >> Can we use Unicode in the SGML files? > > I believe we've been doing it for contributors' names that require > non-ASCII letters, but not in any other places. We have non-ASCII letters in charset.sgml too, to show some examples of collation. Best reagards, -- Tatsuo I

Re: [Bug Fix]standby may crash when switching-over in certain special cases

2024-10-10 Thread px shi
> > Although I've not tested it because I don't have good way to reproduce > the problem I use GDB to reproduce the issue by killing the primary node with kill -9 when the standby’s flushedlsn was at the begining of a WAL segment. However, this causes to request the primary to stream data fro

Re: Add contrib/pg_logicalsnapinspect

2024-10-10 Thread Masahiko Sawada
On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Oct 09, 2024 at 10:21:31AM -0700, Masahiko Sawada wrote: > > On Wed, Oct 9, 2024 at 1:12 AM Bertrand Drouvot > > wrote: > > > One option could be (did not test it) to add this switch in > > > construct_array_builtin(): >

Re: sunsetting md5 password support

2024-10-10 Thread Andrew Dunstan
On 2024-10-09 We 7:11 PM, Heikki Linnakangas wrote: On 09/10/2024 22:55, Nathan Bossart wrote: In this message, I propose a multi-year, incremental approach to remove MD5 password support from Postgres. +1   2.  In v19, allow upgrading with MD5 passwords and allow authenticating   wi

Re: replace strtok()

2024-10-10 Thread Ranier Vilela
Hi Alexander, Em qui., 10 de out. de 2024 às 02:00, Alexander Lakhin escreveu: > Hello Peter, > > 23.07.2024 15:38, Peter Eisentraut wrote: > > This has been committed. Thanks. > > Please look at the SCRAM secret, which breaks parse_scram_secret(), > perhaps because strsep() doesn't return NULL

Re: On disable_cost

2024-10-10 Thread Alena Rybakina
Hi! On 10.10.2024 15:43, David Rowley wrote: On Tue, 8 Oct 2024 at 04:14, Robert Haas wrote: I think you have adequate consensus to proceed with this. I'd just ask that you don't disappear completely if it turns out that there are problems. I accept that my commit created this problem and I'm

Re: sunsetting md5 password support

2024-10-10 Thread Jesper Pedersen
On 10/10/24 5:45 PM, Heikki Linnakangas wrote: On 11/10/2024 00:03, Bruce Momjian wrote: On Wed, Oct  9, 2024 at 10:30:15PM +0200, Jelte Fennema-Nio wrote: On Wed, 9 Oct 2024 at 21:55, Nathan Bossart wrote: In this message, I propose a multi-year, incremental approach to remove MD5 password

Re: sunsetting md5 password support

2024-10-10 Thread Jelte Fennema-Nio
On Thu, 10 Oct 2024 at 23:45, Heikki Linnakangas wrote: > I wouldn't recommend it if SCRAM is available, but yeah, with TLS and > sslmode=verify-full, it's secure enough. Agreed, I'd definitely still recommend SCRAM over password. A big downside of "password" auth over TLS is that plaintext passw

Re: sunsetting md5 password support

2024-10-10 Thread Daniel Gustafsson
> On 11 Oct 2024, at 00:28, Tom Lane wrote: > On the whole I agree with Heikki's comment that we should just > do it (disallow MD5, full stop) whenever we feel that enough > time has passed. These intermediate states are mostly going to > add headaches. Maybe we could do something with an inter

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Well for getting the potential benefits of link-time optimization into more places, and slightly improved developer ergonomics, and perhaps mostly to eliminate potential confusion if one needs pg_common and pg_ports or the shared library version when compiling statically. Would you happen to know i

Re: Using per-transaction memory contexts for storing decoded tuples

2024-10-10 Thread Masahiko Sawada
On Thu, Oct 10, 2024 at 8:26 AM Masahiko Sawada wrote: > > On Thu, Oct 10, 2024 at 8:04 AM Fujii Masao > wrote: > > > > > > > > On 2024/10/04 3:32, Masahiko Sawada wrote: > > > Yes, but as for this macro specifically, I thought that it might be > > > better to keep it, since it avoids breaking e

Re: pgbench: Improve result outputs related to failed transactinos

2024-10-10 Thread Tatsuo Ishii
>>> Thanks for the fix. Here is the new run with the v2 patch. The result >>> looks good to me. >>> >>> src/bin/pgbench/pgbench -p 11002 -c1 -t 1 -f c.sql -f d.sql >>> --failures-detailed -r test >>> pgbench (18devel) >>> starting vacuum...end. >>> transaction type: multiple scripts >>> scaling

Re: Doc: typo in config.sgml

2024-10-10 Thread Yugo Nagata
On Thu, 10 Oct 2024 16:00:41 +0900 (JST) Tatsuo Ishii wrote: > > Bruce Momjian writes: > >> Can we use Unicode in the SGML files? > > > > I believe we've been doing it for contributors' names that require > > non-ASCII letters, but not in any other places. > > We have non-ASCII letters in char

Re: Converting tab-complete.c's else-if chain to a switch

2024-10-10 Thread Anthonin Bonnefoy
Hi, bd1276a3c9 seems to have introduced a segfault when trying to complete a word that doesn't have any match. For example, 'postgres=# z\t' will yield the following backtrace: #0: psql`pg_strcasecmp(s1="", s2="ACCESS METHOD") at pgstrcasecmp.c:40:39 #1: psql`psql_completion(text=":pgss-", start=

Re: Doc: typo in config.sgml

2024-10-10 Thread Daniel Gustafsson
> On 9 Oct 2024, at 04:49, Tatsuo Ishii wrote: > Besides nbsp, there are tons of confusing Unicode > characters out there. For example there are many "hyphen like > characters". Using characters which look alike is in the field of internet security known as homograph attacks, where for example a

Re: Converting tab-complete.c's else-if chain to a switch

2024-10-10 Thread Daniel Gustafsson
> On 10 Oct 2024, at 09:22, Anthonin Bonnefoy > wrote: > psql tries to read the previous word when matches are NULL. However, > there's no previous word set here, leading to the segfault. > > I've attached a patch that checks if the previous word does exist > before trying to use it, along with

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Aleksander Alekseev
Hi Mikael, Please use the "Reply to All" button - send your emails to the mailing list, not to me directly. > The official packages and the original source tar both fail to compile, the > patch enables it to compile. I just run: ``` docker run -e POSTGRES_PASSWORD=foo postgres:17.0-alpine3.20

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Hello Aleksander This is for compiling a c++ application that uses libpq with the -static flag, the server compiles fine. Br Mikael On Thu, Oct 10, 2024 at 12:16 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Mikael, > > Please use the "Reply to All" button - send your emails to

Re: Wrong results with grouping sets

2024-10-10 Thread Richard Guo
On Thu, Oct 10, 2024 at 4:06 PM Richard Guo wrote: > While we can fix this issue by propagating the hasGroupRTE mark from > the EXISTS subquery to the parent, a better fix might be to remove the > subquery's RTE_GROUP entry, since we have dropped the subquery's > groupClause before the pull-up (se

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Sorry for not having a properly minimal reproduction to begin with, seems I can't reduce this any further at the moment at least: # syntax=docker/dockerfile:1 > FROM chainguard/git:latest-dev AS builder > USER root > WORKDIR /app > RUN apk update && apk add --no-cache --update-cache \ > clang-

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Aleksander Alekseev
Hi Mikael, > Sorry for not having a properly minimal reproduction to begin with, seems I > can't reduce this any further at the moment at least: > [...] Is there any particular reason for using pg.patch in your Dockerfile or not using well regarded / official [1][2] Docker images [3] ? It seems

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
E.g. this fails to compile: FROM chainguard/git:latest-dev AS builder USER root RUN apk update && apk add --no-cache --update-cache clang-19 postgresql-17-dev openssl-dev COPY < int main(){return PQconnectdb("")==NULL;} EOF RUN clang++ -static -o main main.cpp -lpq -lpgcommon -lpgport -lssl -lcryp

Re: RFC: Additional Directory for Extensions

2024-10-10 Thread Ebru Aydin Gol
Thanks for your efforts, a secondary directory for extensions is a very useful feature. Is there any updates on the patch? -Ebru On Thu, Aug 29, 2024 at 6:55 PM David E. Wheeler wrote: > On Aug 27, 2024, at 22:24, Craig Ringer > wrote: > > > `pg_config` only cares about compile-time settings

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-10 Thread Fujii Masao
On 2024/10/08 23:16, Anton A. Melnikov wrote: On 08.10.2024 15:42, Fujii Masao wrote: On 2024/09/30 12:26, Fujii Masao wrote: In 0002.patch, I also modified the description of num_requested from "Number of backend requested checkpoints" to remove "backend," as it can be confusing since num_

Re: overflow bug for inhcounts

2024-10-10 Thread Alvaro Herrera
On 2024-Oct-09, jian he wrote: > CreateConstraintEntry(const char *constraintName, > we can change > int conInhCount, > to > int16 conInhCount. > doing that, meaning we also need to refactor some of the caller functions. Good thought, thanks. > we can also change > node Constraint field inhcoun

Re: allowing extensions to control planner behavior

2024-10-10 Thread Robert Haas
On Wed, Sep 18, 2024 at 11:48 AM Robert Haas wrote: > Still, I think it's a pretty useful starting point. It is mostly > enough to give you control over join planning, and if combined with > similar work for scan planning, I think it would be enough for > pg_hint_plan. If we also got control over

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Aleksander Alekseev
Mikael, On Thu, Oct 10, 2024 at 8:49 PM Tom Lane wrote: > > Mikael Sand writes: > > RUN clang++ -fno-common -static -o main main.cpp \ > > -L/usr/local/lib -lpq -lpgcommon -lpgport \ > > -lldap -lsasl2 -lssl -lcrypto -llber \ > > -lgssapi_krb5 \ > > -lkrb5 -lk5crypto -lcom_err -l

Re: Annoying build warnings from latest Apple toolchain

2024-10-10 Thread Mikael Sand
Good day! I'm encountering a build issue with postgresql 17, I wonder if this was an intentional consequence of this commit: https://github.com/postgres/postgres/commit/b6c7cfac88c47a9194d76f3d074129da3c46545a Or if this was unintentional. Or is there any way to compile pgcommon with the correct

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-10-10 Thread Peter Geoghegan
On Tue, Oct 8, 2024 at 12:52 PM Peter Geoghegan wrote: > The code in v3-0001-Avoid-unneeded-nbtree-backwards-scan-buffer-locks.patch > looks reasonable to me. I don't think that there are any impediments > to committing it sometime this week. I see significant benefits for parallel index-only bac

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Tom Lane
Mikael Sand writes: > RUN clang++ -fno-common -static -o main main.cpp \ > -L/usr/local/lib -lpq -lpgcommon -lpgport \ > -lldap -lsasl2 -lssl -lcrypto -llber \ > -lgssapi_krb5 \ > -lkrb5 -lk5crypto -lcom_err -lkrb5support \ > -lgdbm The short answer here is that your link reci

Re: Converting tab-complete.c's else-if chain to a switch

2024-10-10 Thread Jacob Champion
Hi Tom, Meson's coverage generation is unhappy with the new hardcoded #line references emitted by gen_tabcomplete.pl: genhtml: ERROR: cannot read /home/jacob/src/postgres/worktree-oauth/build-dev/tab-complete.in.c I've attached a potential fix which references $ARGV[0] instead, but it's not

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
We use static linking and link time optimization to squeeze the last bits of performance out of the code in our most performance-critical queries, and it simplifies our security audits to have a static binary running inside chainguard/static as the data we handle is sensitive/business critical. Wo

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Just for reference in case anyone else who utilizes static linking for any reason hits upon this issue, here is a working Dockerfile for libpq / postgresql 17 FROM postgres:17.0-alpine3.20 AS builder USER root WORKDIR /app RUN apk update && apk add --no-cache --update-cache \ openssl-libs-stat

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Aleksander, do you have something against static linking or am I reading you wrong? I've seen this sentiment in several places but never understood why anyone would hold this position. Could you elaborate? At least for executables intended to be run in production inside containers, they usually ru

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Tom Lane
Mikael Sand writes: > Wonderful, the build works when using -lpgcommon_shlib -lpgport_shlib > So this appears to imply that the output of `pkg-config -libs -static > libpq`is incorrect? >> /app # pkg-config -libs -static libpq >> -L/usr/local/lib -lpq -L/usr/lib/llvm15/lib -L/usr/local/lib -lpgco

Re: Wrong results with grouping sets

2024-10-10 Thread Richard Guo
On Thu, Oct 10, 2024 at 2:39 PM David Rowley wrote: > create table a(a int); > explain select * from a where exists(Select 1 from a a2 where a.a = > a2.a group by a); > CREATE TABLE > server closed the connection unexpectedly > > TRAP: failed Assert("parse->hasGroupRTE"), File: > "../src/backend/

Re: Pg17 Crash in Planning (Arrays + Casting + UDF)

2024-10-10 Thread Fredrik Widlert
As the original reporter on the PostGIS mailing list, I want to thank everyone both on that list and on the Postgres list for the very quick response. Having the problem confirmed and a ticket opened in 2 minutes is really impressive. My report contained a very simplified version of our crashing

Re: Should CSV parsing be stricter about mid-field quotes?

2024-10-10 Thread Daniel Verite
Joel Jacobson wrote: > - No Headers or Metadata: It's not clear why it's necessary to disable the HEADER option for this format? > The format does not support header rows or end-of-data markers; > every line is treated as data. With COPY FROM STDIN followed by inline data in a script,

Re: ALTER TABLE ONLY .. DROP CONSTRAINT on partitioned tables

2024-10-10 Thread jian he
On Mon, Sep 30, 2024 at 6:01 PM Alvaro Herrera wrote: > > Hello, > > On 2024-Sep-27, Amit Langote wrote: > > > On Fri, Sep 27, 2024 at 2:52 AM Alvaro Herrera > > wrote: > > > While studying a review note from Jian He on not-null constraints, I > > > came across some behavior introduced by commit

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Here is a simpler Dockerfile for making a static build of a c++ application using libpq from postgresql 17 # syntax=docker/dockerfile:1 > FROM alpine:3.20 AS builder > USER root > WORKDIR /app > RUN apk update && apk add --no-cache --update-cache \ > openssl-libs-static \ > e2fsprogs-dev \

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
This seems to work: # syntax=docker/dockerfile:1 FROM alpine:3.20 AS builder # Fails with 17.0 succeeds with 16.4 ARG PG=17.0 USER root RUN apk update && apk add --no-cache --update-cache \ ca-certificates-bundle \ util-linux-dev \ execline-dev \ libedit-dev \ libxml2-dev \ clang17-d

Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Hello dear Hackers I'm trying to upgrade to v17 and encountering a strange issue. I've made a minimal reproduction that works with 16.4 but fails with 17.0, it also works without the "-static" compile flag: # syntax=docker/dockerfile:1 > FROM alpine:3.20 AS builder > > # Fails with 17.0 succeed

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Hmm, so is static linking of all applications that include libpq intentionally broken? At least googling around, it seems like a relatively common practice. At least it seems that 2ndquadrant builds and uses libpq statically. https://www.postgresql.org/message-id/20327.1501536978%40sss.pgh.pa.us

Re: [PATCH] Move clause_sides_match_join() into pathnode.h

2024-10-10 Thread James Hunter
On Wed, Oct 9, 2024 at 5:26 PM David Rowley wrote: > > On Thu, 10 Oct 2024 at 08:38, James Hunter wrote: > > We had two almost-identical copies of the utility function > > clause_sides_match_join() -- one in joinpath.c, and one in > > analyzejoins.c. Both copies were marked "inline," so we might

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
I don't mind having this patch in use too much, I have a functional build and nothing to worry about, no luck necessary :) But, I expect a lot of unnecessary churn in the community if this is not fixed properly. Not all applications run inside docker, and not all can use dynamic linking. People ca

Re: Using per-transaction memory contexts for storing decoded tuples

2024-10-10 Thread Masahiko Sawada
On Thu, Oct 10, 2024 at 8:04 AM Fujii Masao wrote: > > > > On 2024/10/04 3:32, Masahiko Sawada wrote: > > Yes, but as for this macro specifically, I thought that it might be > > better to keep it, since it avoids breaking extension unnecessarily > > and it seems to be natural to have it as an opti

Re: type cache cleanup improvements

2024-10-10 Thread Artur Zakirov
Hi all, On Fri, 13 Sept 2024 at 01:38, Alexander Korotkov wrote: > > 0001 - adds comment about concurrent invalidation handling > 0002 - revised c14d4acb8. Now we track type oids, whose > TypeCacheEntry's filing is in-progress. Add entry to > RelIdToTypeIdCacheHash at the end of lookup_type_cac

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
E.g. this works with 16.4 but fails in 17.0: FROM postgres:16.4-alpine3.20 AS builder USER root WORKDIR /app RUN apk update && apk add --no-cache --update-cache \ openssl-libs-static \ cyrus-sasl-static \ libevent-static \ libxml2-static \ libedit-static \ libxslt-static \

Re: BF mamba failure

2024-10-10 Thread Bertrand Drouvot
Hi, On Thu, Oct 10, 2024 at 06:19:30AM +, Kouber Saparev wrote: > Am I correct to believe this patch is fixing the "can only drop stats once" > issue? > > It just happened to us, one of our streaming replicas decided to shut down. Does the error message looks like (added in [1]): " trying

Re: ECPG Refactor: move sqlca variable in ecpg_log()

2024-10-10 Thread Alvaro Herrera
On 2024-Oct-10, Yuto Sasaki (Fujitsu) wrote: > The sqlca variable in the ecpg_log() was declared with an unnecessarily wide > scope, so I moved to the appropriate place. Hmm, I'm not sure we want that, because if we do this, then ECPGget_sqlca() gets run with the debug_mutex held. In the origina

Re: Converting tab-complete.c's else-if chain to a switch

2024-10-10 Thread Tom Lane
Anthonin Bonnefoy writes: > bd1276a3c9 seems to have introduced a segfault when trying to complete > a word that doesn't have any match. For example, 'postgres=# z\t' will > yield the following backtrace: Interesting --- it seems to depend on which readline version you're using, because I see a c

Re: Using per-transaction memory contexts for storing decoded tuples

2024-10-10 Thread Fujii Masao
On 2024/10/04 3:32, Masahiko Sawada wrote: Yes, but as for this macro specifically, I thought that it might be better to keep it, since it avoids breaking extension unnecessarily and it seems to be natural to have it as an option for slab context. If the macro has value, I'm okay with leavin

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
This seems to be the commit that caused this: https://github.com/postgres/postgres/commit/b6c7cfac88c47a9194d76f3d074129da3c46545a So it seems the commit is related to static linking. Is it somehow possible to statically link in a version of pgcommon with correct function names into libpq?

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-10 Thread Anton A. Melnikov
On 10.10.2024 18:14, Fujii Masao wrote: Thanks for the review! Pushed. Thanks a lot! With the best regards, -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Statistics Import and Export

2024-10-10 Thread Corey Huinker
> > This seems like a reasonable refactoring exercise that we could take care > of before the rest of the patch set goes in. I added one new reference to > dopt.schemaOnly in commit bd15b7d, so that should probably be revised to > !dumpData, too. I also noticed a few references to dataOnly/schema

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Aleksander Alekseev
Hi Mikael, > We use static linking and link time optimization to squeeze the last bits of > performance out of the code in our most performance-critical queries, and it > simplifies our security audits to have a static binary running inside > chainguard/static as the data we handle is sensitive

Re: sunsetting md5 password support

2024-10-10 Thread Nathan Bossart
On Thu, Oct 10, 2024 at 02:11:53AM +0300, Heikki Linnakangas wrote: > My feeling is that it would be less confusing to users to just disallow md5 > passwords in one release. I'm not sure these intermediate steps are really > doing anyone any favors. As I'm reading the various responses in this thr

Re: generic plans and "initial" pruning

2024-10-10 Thread Robert Haas
Hi Amit, This is not a full review (sorry!) but here are a few comments. In general, I don't have a problem with this direction. I thought Tom's previous proposal of abandoning ExecInitNode() in medias res if we discover that we need to replan was doable and I still think that, but ISTM that this

Re: Statistics Import and Export

2024-10-10 Thread Nathan Bossart
On Thu, Oct 10, 2024 at 03:49:16PM -0400, Corey Huinker wrote: >> One other question I had when looking at this patch is whether we could >> remove dataOnly/schemaOnly from DumpOptions and RestoreOptions. Once 0007 >> is applied, those variables become particularly hazardous, so we really >> want

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Tom Would this be an acceptable patch? Subject: [PATCH] Fix static build --- Index: src/Makefile.shlib <+>UTF-8 === diff --git a/src/Makefile.shlib b/src/Makefile.shlib --- a/src/Makefile.shlib (revision fd64ed60b62697984bb69a09a3ae1

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Hi Aleksander Ok. So no actual benefit from using dynamic? Well, it seems postgresql and all dependencies already support it, no? Doesn't go do static linking by default / prefer it? Unless you use some part that uses CGO, in which case many go developers appear to disable CGO anyway and use the p

Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

2024-10-10 Thread Mikael Sand
Btw Aleksander To give some context, this is a component intended to solve our last scaling issue / most demanding query, which happens at page load and thus need to minimize latency over all else. We've used libpq in pipelined binary mode, serialized the results it in a custom binary format, comp

RFC: Extension Packaging & Lookup

2024-10-10 Thread David E. Wheeler
Hackers, Back at the end of August, I promised[1]: > I’ll try to put some thought into a more formal proposal in a new thread next > week. Unless your Gabriele beats me to it 😂. I guess I should get off my butt and do it. So let’s do this. Here’s what I propose. * When an extension is insta

Re: Allow default \watch interval in psql to be configured

2024-10-10 Thread Daniel Gustafsson
> On 10 Oct 2024, at 02:01, Michael Paquier wrote: > > On Wed, Oct 09, 2024 at 04:24:27PM +0200, Daniel Gustafsson wrote: >> Fixed. > > -doublesleep = 2; > +doublesleep = pset.watch_interval; > > This forces the use of seconds as unit. The interval values I have

Re: sunsetting md5 password support

2024-10-10 Thread Bruce Momjian
On Wed, Oct 9, 2024 at 10:30:15PM +0200, Jelte Fennema-Nio wrote: > On Wed, 9 Oct 2024 at 21:55, Nathan Bossart wrote: > > In this message, I propose a multi-year, incremental approach to remove MD5 > > password support from Postgres. > > +many for the general idea > > I think it makes sense to

Re: Converting tab-complete.c's else-if chain to a switch

2024-10-10 Thread Tom Lane
Jacob Champion writes: > Meson's coverage generation is unhappy with the new hardcoded #line > references emitted by gen_tabcomplete.pl: > genhtml: ERROR: cannot read > /home/jacob/src/postgres/worktree-oauth/build-dev/tab-complete.in.c > I've attached a potential fix which references $ARGV[0]

  1   2   >