Re: Disallow redundant indexes

2025-04-28 Thread Nathan Bossart
On Mon, Apr 28, 2025 at 01:12:18PM -0500, Sami Imseih wrote: > If I had a dollar for every time I asked a user to check for duplicate > indexes :) So > I think it's a good idea to provide some way to detect these ( besides > a query in the > Wiki ), but I don't think we should prevent it. > > Whil

Re: teach pg_upgrade to handle in-place tablespaces

2025-04-28 Thread Nathan Bossart
On Mon, Apr 28, 2025 at 04:07:16PM -0500, Nathan Bossart wrote: > On Tue, Mar 25, 2025 at 04:03:57PM -0500, Nathan Bossart wrote: >> I also wanted to draw attention to this note in 0003: >> >> /* >> * XXX: The below line is a hack to

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-28 Thread Nathan Bossart
hat would be even harder to miss. -- nathan >From 39f870e69aa0cb08c0ef653b60ff07fa020c6bba Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 28 Apr 2025 16:27:33 -0500 Subject: [PATCH v6 1/1] Remove pg_replication_origin's TOAST table. Reviewed-by: Michael Paquier Reviewed-by: Amit

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-28 Thread Nathan Bossart
On Wed, Apr 23, 2025 at 04:44:51PM -0500, Nathan Bossart wrote: > On Wed, Apr 23, 2025 at 05:33:41PM -0400, Tom Lane wrote: >> I don't see any comments in this patch that capture the real >> reason for removing pg_replication_origin's TOAST table, >> namely (IIU

teach pg_upgrade to handle in-place tablespaces

2025-04-28 Thread Nathan Bossart
(Creating new thread from https://postgr.es/m/Z-MaPREQvH5YB0af%40nathan.) On Tue, Mar 25, 2025 at 04:03:57PM -0500, Nathan Bossart wrote: > I also wanted to draw attention to this note in 0003: > > /* >* XXX: The below line is a hack to deal with the

Re: optimize file transfer in pg_upgrade

2025-04-28 Thread Nathan Bossart
On Mon, Apr 28, 2025 at 09:00:01PM +0300, Alexander Lakhin wrote: > Thank you for the references! Unfortunately I still can't see where the > lack of upgrade log files is discussed. That was briefly discussed here: https://postgr.es/m/644cf995-e3a5-4f69-9398-7db500e2673d%40dunslane.net O

Re: optimize file transfer in pg_upgrade

2025-04-28 Thread Nathan Bossart
On Sun, Apr 27, 2025 at 05:00:01PM +0300, Alexander Lakhin wrote: > Both happened on Windows, but what's worse is that the failure logs > contain no information on the exact reason. We can see: > #   Failed test 'pg_upgrade with transfer mode --swap: stdout matches' > #   at > C:/tools/xmsys64/hom

Re: allow changing autovacuum_max_workers without restarting

2025-04-28 Thread Nathan Bossart
On Mon, Apr 28, 2025 at 09:14:54AM -0400, Tom Lane wrote: > Peter Eisentraut writes: >> This initdb output seems, well, kinda fake, which it is by its own >> admission. > > Agreed. > >> Could we do this less fake maybe like this: >> selecting default "max_connections", "autovacuum_worker_sl

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-23 Thread Nathan Bossart
On Wed, Apr 23, 2025 at 05:33:41PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Here is a new version of the patch with this change. > > I don't see any comments in this patch that capture the real > reason for removing pg_replication_origin's TOAST table, >

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-23 Thread Nathan Bossart
On Tue, Apr 22, 2025 at 12:21:01PM -0500, Nathan Bossart wrote: > On Tue, Apr 22, 2025 at 05:17:17PM +0530, Nisha Moond wrote: >> 5) As Euler pointed out, logical replication already has a built-in >> restriction for internally assigned origin names, limiting them to >> NA

Re: Doc: fix the rewrite condition when executing ALTER TABLE ADD COLUMN

2025-04-23 Thread Nathan Bossart
This one was briefly discussed in an RMT meeting. On Mon, Mar 24, 2025 at 11:37:20AM +0100, Álvaro Herrera wrote: > On 2025-Mar-24, jian he wrote: > >> In the current development branch, >> virtual generated columns still do not support the domain. >> you can not change the generation expression

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-23 Thread Nathan Bossart
This one was briefly discussed in an RMT meeting. On Wed, Apr 09, 2025 at 01:16:20PM +0800, jian he wrote: > attached patch is for address pg_dump inconsistency > when parent is "not null not valid" while child is "not null". I see an open item [0] that points to this patch, but there's no owner

Re: vacuumdb --missing-stats-only and pg_upgrade from PG13

2025-04-23 Thread Nathan Bossart
Here is what I have staged for commit. I'll aim to commit these patches sometime next week to give time for additional feedback. -- nathan >From 6d1e608edb17d9bbaaf7d57ace35fee68ff869c0 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 23 Apr 2025 10:11:46 -0500 Subject: [PATC

Re: vacuumdb --missing-stats-only and pg_upgrade from PG13

2025-04-23 Thread Nathan Bossart
On Wed, Apr 23, 2025 at 04:01:33PM +0200, Christoph Berg wrote: > If I create a table in a PG13-or-earlier cluster, never ANALYZE it, > and then pg_upgrade to 18 and run vacuumdb --analyze-only > --missing-stats-only, the table will not get analyzed. The only table > visited there is pg_largeobject

Re: pgsql: Update guidance for running vacuumdb after pg_upgrade.

2025-04-22 Thread Nathan Bossart
On Tue, Apr 22, 2025 at 11:03:29PM +0200, Christoph Berg wrote: > Re: Nathan Bossart >> In any case, IMO it's unfortunate >> that we might end up recommending roughly the same post-upgrade steps as >> before even though the optimizer statistics are carried over. &g

Re: pgsql: Update guidance for running vacuumdb after pg_upgrade.

2025-04-22 Thread Nathan Bossart
On Tue, Apr 22, 2025 at 09:43:56PM +0200, Christoph Berg wrote: > Re: Nathan Bossart >> Update guidance for running vacuumdb after pg_upgrade. >> >> Now that pg_upgrade can carry over most optimizer statistics, we >> should recommend using vacuumdb's new --miss

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-22 Thread Nathan Bossart
On Tue, Apr 22, 2025 at 05:17:17PM +0530, Nisha Moond wrote: > Thanks for the patch! I tested it for functionality and here are a few > comments: Thank you for reviewing. > 1) While testing, I noticed an unexpected behavior with the new 512 > byte length restriction in place. Since there´s no ex

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-21 Thread Nathan Bossart
Apparently replication origins in tests are supposed to be prefixed with "regress_". Here is a new patch with that fixed. -- nathan >From fa6a8a5a155a7beb50e54bfe42c81be26ffd428a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Apr 2025 10:53:15 -0500 Subject: [PATCH v

Re: ci: Allow running mingw tests by default via environment variable

2025-04-14 Thread Nathan Bossart
On Mon, Apr 14, 2025 at 09:26:59PM +0200, Andres Freund wrote: > On April 14, 2025 9:22:40 PM GMT+02:00, Nathan Bossart > wrote: >>On Sat, Apr 12, 2025 at 08:18:35PM +1200, Thomas Munro wrote: >>> Is it safe to assume that CI changes come under the same code-freeze >&

Re: ci: Allow running mingw tests by default via environment variable

2025-04-14 Thread Nathan Bossart
On Sat, Apr 12, 2025 at 08:18:35PM +1200, Thomas Munro wrote: > Is it safe to assume that CI changes come under the same code-freeze > exception as tests and docs? I would assume so, it's just scripting > glue in .cirrus.XXX files not affecting one bit of the software, but I > don't recall that we

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-11 Thread Nathan Bossart
p -E "^#define\s[A-Z_]+\s\([0-9]+\)$" src/* -rI | wc -l 23 $ grep -E "^#define\s[A-Z_]+\s[0-9]+$" src/* -rI | wc -l 861 > Should we document this maximum length? I've added a note. -- nathan >From 6762a3786b76379c82ccfa4c9da1812e92

Re: New committer: Jacob Champion

2025-04-11 Thread Nathan Bossart
On Fri, Apr 11, 2025 at 01:26:04PM -0700, Jonathan S. Katz wrote: > The Core Team would like to extend our congratulations to Jacob Champion, > who has accepted an invitation to become our newest PostgreSQL committer. > > Please join us in wishing Jacob much success and few reverts! Congrats! --

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-11 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 12:22:00PM -0500, Nathan Bossart wrote: > On Tue, Apr 08, 2025 at 01:07:09PM -0400, Tom Lane wrote: >> Nathan Bossart writes: >>> I do think it's worth considering going back to copying >>> pg_largobject_metadata's files for upgrades

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-04-11 Thread Nathan Bossart
On Thu, Apr 10, 2025 at 11:58:41PM +0530, Mahendra Singh Thalor wrote: > As per above discussions, for v18, we will not do any change to server > side to fix the issue of \n\r in database names. But as a cleanup > patch, we can give an alert to the user by "pg_upgrade --check". As > per current cod

Re: Improve a few appendStringInfo calls new to v18

2025-04-11 Thread Nathan Bossart
On Fri, Apr 11, 2025 at 10:40:57AM +1200, David Rowley wrote: > On Fri, 11 Apr 2025 at 02:51, Nathan Bossart wrote: >> This probably isn't v18 material, but this reminds me of my idea to change >> appendStringInfoString() into a macro for appendBinaryStringInfo() so that

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-10 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:42:20PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Unless I'm missing something, we don't seem to have had any dependency >> handling before commit 12a53c7. Was that broken before we moved to SQL >> commands? > > Sounds lik

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-10 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:07:09PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I do think it's worth considering going back to copying >> pg_largobject_metadata's files for upgrades from v16 and newer. > > (If we do this) I don't see why we'd nee

Re: Add os_page_num to pg_buffercache

2025-04-10 Thread Nathan Bossart
On Thu, Apr 10, 2025 at 03:35:24PM +0200, Tomas Vondra wrote: > This seems like a good idea in principle, but at this point it has to > wait for PG19. Please add it to the July commitfest. +1. From a glance, this seems to fall in the "new feature" bucket and should likely wait for v19. -- natha

Re: Improve a few appendStringInfo calls new to v18

2025-04-10 Thread Nathan Bossart
On Thu, Apr 10, 2025 at 11:24:36AM +0300, Heikki Linnakangas wrote: > On 10/04/2025 06:51, David Rowley wrote: >> Any objections to doing this soonish? Or in a few weeks? > > Sure, let's do it. Why would we wait? +1. You did something similar for v17 (commit 8461424), and it seems like an entire

Re: PostgreSQL 18 Release Management Team & Feature Freeze

2025-04-10 Thread Nathan Bossart
On Thu, Apr 10, 2025 at 03:11:00PM +0300, Heikki Linnakangas wrote: > On 10/04/2025 14:57, Aleksander Alekseev wrote: >> Do you think that it's worth merging SLRU refactoring into PG18, or is >> it considered a feature? [1] This is arguably not the top priority and >> we could wait for another year

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-09 Thread Nathan Bossart
hem. But this'll need to wait for a couple of months until v19 development begins. -- nathan >From 932a7ff2623c3185f6f98194233d3d99807909fe Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 9 Apr 2025 14:00:31 -0500 Subject: [PATCH v1 1/1] Remove pg_replication_origin's TOAST tab

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 09:41:06PM +0200, Hannu Krosing wrote: > On Tue, Apr 8, 2025 at 8:39 PM Nathan Bossart > wrote: >> I've also verified that the dependency information is carried over in >> upgrades to later versions (AFAICT all the supported ones). > > If I

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:51:22PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Tue, Apr 08, 2025 at 01:36:58PM -0400, Tom Lane wrote: >>> Hmm ... one annoying thing for this project is that AFAICS pg_upgrade >>> does *not* preserve database OIDs, which is pro

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 05:37:50PM -0400, Jan Wieck wrote: > I remember an incident where large amounts of LOs ran pg_upgrade into a > transaction-ID wrap around because the restore part would create individual > single statement transactions per LO to create, then change permissions and > ownershi

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 04:44:02PM +0530, Amit Kapila wrote: > On Fri, Apr 4, 2025 at 7:58 PM Nathan Bossart > wrote: >> On Fri, Apr 04, 2025 at 05:16:43PM +0530, Amit Kapila wrote: >> > Can we dodge adding this push call if we restrict the length of the >> > origin

Re: Feature freeze

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 05:13:15PM +0200, Daniel Gustafsson wrote: >> On 8 Apr 2025, at 16:59, Bruce Momjian wrote: >> Frankly, I think the name "anywhere on Earth" is confusing, since it >> really is "everywhere on Earth": > > I find both of the above needlessly confusing when we instead could u

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:36:58PM -0400, Tom Lane wrote: > Hmm ... one annoying thing for this project is that AFAICS pg_upgrade > does *not* preserve database OIDs, which is problematic for using > COPY to load pg_shdepend rows. I think it does; see commit aa01051. -- nathan

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 12:37:43PM -0400, Tom Lane wrote: > Hannu Krosing writes: >> I think we do preserve role oids > > Oh ... I'd been looking for mentions of "role" in > pg_upgrade_support.c, but what I should have looked for was > "pg_authid". So yeah, we do preserve role OIDs, and maybe th

Re: Feature freeze

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 06:00:27PM +0200, Peter Eisentraut wrote: > On 08.04.25 16:59, Bruce Momjian wrote: >> Anywhere on Earth (AoE) is a calendar designation that indicates >> that a period expires when the date passes everywhere on Earth. > > Yes, that works intuitively when you spec

Re: prevent 006_transfer_modes.pl from leaving files behind

2025-04-08 Thread Nathan Bossart
Committed. -- nathan

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-08 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 09:35:24AM +0200, Hannu Krosing wrote: > On Tue, Apr 8, 2025 at 12:17 AM Nathan Bossart > wrote: >> That being said, I >> regularly hear about slow upgrades with many LOs, so I think it'd be >> worthwhile to try to improve matters in v19. &

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Nathan Bossart
On Mon, Apr 07, 2025 at 10:33:47PM +0200, Hannu Krosing wrote: > The obvious solution would be to handle the table > `pg_largeobject_metadata` the same way as we currently handle > `pg_largeobject `by not doing anything with it in `pg_dump > --binary-upgrade` and just handle the contents it like we

prevent 006_transfer_modes.pl from leaving files behind

2025-04-07 Thread Nathan Bossart
rom a349a98b19708984dc677e5809038d343d67bf17 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 7 Apr 2025 16:23:44 -0500 Subject: [PATCH v1 1/1] Fix the 006_transfer_modes.pl test. This test was leaving files like delete_old_cluster.sh in the source directory for VPATH and meson builds. To fix, change the directory

Re: Update Unicode data to Unicode 16.0.0

2025-04-06 Thread Nathan Bossart
On Sun, Apr 06, 2025 at 09:16:17AM -0500, Nathan Bossart wrote: > Coverity is unhappy about these. I think we should at least do something > like the following. I'll commit this when I have an opportunity. Committed. -- nathan

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-04-06 Thread Nathan Bossart
On Sun, Apr 06, 2025 at 10:24:58AM -0400, Andrew Dunstan wrote: > On 2025-03-28 Fr 10:43 AM, Nathan Bossart wrote: >> Taking a step back, are we sure that 1) this is the right place to do these >> checks and 2) we shouldn't apply the same restrictions to all names? I'm &g

Re: Update Unicode data to Unicode 16.0.0

2025-04-06 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 02:33:42PM -0600, Nathan Bossart wrote: > + /* > + * The builtin provider did not exist prior to version 17. While there > are > + * still problems that could potentially be caught from earlier > versions, > + * such as an index on NOR

Re: optimize file transfer in pg_upgrade

2025-04-05 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 12:44:38PM -0400, Andres Freund wrote: > On 2025-03-19 12:28:33 -0400, Tom Lane wrote: >> Nathan Bossart writes: >> > I'm currently planning to commit this sometime early-ish next week. One >> > notable loose end is the lack of a pg

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 07:32:48PM -0400, Corey Huinker wrote: > This patch shrinks the array size to 1 for versions < 9.4, which keeps the > modern code fairly elegant. Committed. -- nathan

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 03:58:53PM -0500, Nathan Bossart wrote: > I pushed commit 8ec0aae to fix this. And now I'm seeing cross-version test failures due to our use of WITH ORDINALITY, which wasn't added until v9.4. Looking into it... -- nathan

Re: Improve CRC32C performance on SSE4.2

2025-04-04 Thread Nathan Bossart
On Wed, Apr 02, 2025 at 02:10:40PM +0700, John Naylor wrote: > Thanks for looking, I plan to commit this over the weekend unless > there are objections. I noticed that autoconf is defining USE_AVX512_CRC_WITH_RUNTIME_CHECK, but everywhere else expects USE_AVX512_CRC32C_WITH_RUNTIME_CHECK (with the

Re: vacuum_truncate configuration parameter and isset_offset

2025-04-04 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 11:27:15PM +0300, Nikolay Shaplov wrote: > В письме от понедельник, 24 марта 2025 г. 23:04:39 MSK пользователь Nathan > Bossart написал: >> But again, I don't see any strong reason why we must change all such >> reloptions. > > Because

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 03:06:45PM -0500, Nathan Bossart wrote: > I see the buildfarm failure and am working on a fix. I pushed commit 8ec0aae to fix this. -- nathan

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 02:56:54PM -0500, Nathan Bossart wrote: > Committed. I see the buildfarm failure and am working on a fix. -- nathan

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Thu, Apr 03, 2025 at 09:19:51PM -0500, Nathan Bossart wrote: > Great. I'm planning to commit the attached patch set tomorrow morning. Committed. -- nathan

Re: Disabling vacuum truncate for autovacuum

2025-04-04 Thread Nathan Bossart
On Thu, Mar 20, 2025 at 02:18:33PM -0700, David G. Johnston wrote: > So my concern about dump/restore seems to be alleviated but then, why can > we not just do whatever pg_dump is doing to decide whether the current > value for vacuum_truncate is its default (and thus would not be dumped) or > not

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Tue, Apr 01, 2025 at 10:44:19PM -0700, Jeff Davis wrote: > On Tue, 2025-04-01 at 22:21 -0500, Nathan Bossart wrote: >> We might be able to improve this by inventing a new callback that fails for >> all formats except for custom with feesko() available.  That would at least

Re: vacuum_truncate configuration parameter and isset_offset

2025-04-04 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 10:12:05PM +0300, Nikolay Shaplov wrote: > And second general idea: changing engine is bad, at least when you can manage > without changing it. You have asserted this a couple of times without providing any reasons why. I know of no general project policy about changing th

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 05:16:43PM +0530, Amit Kapila wrote: > Can we dodge adding this push call if we restrict the length of the > origin name to some reasonable limit like 256 or 512 and avoid the > need of toast altogether? We did consider just removing pg_replication_origin's TOAST table earl

Re: Extend ALTER DEFAULT PRIVILEGES for large objects

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 07:18:11PM +0900, Fujii Masao wrote: > I've pushed the patch. Thanks! Just a heads up, I fixed a pgindent issue in this commit (see commits e1a8b1ad58 and 742317a80f). I'd ordinarily just report it, but since we're nearing feature freeze, I just fixed it because my workflo

Re: Statistics Import and Export

2025-04-03 Thread Nathan Bossart
ars. -- nathan >From a5a31f2754bf69a81fdc48769c1ee950317a2cf0 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 3 Apr 2025 13:20:28 -0500 Subject: [PATCH v12n6 1/4] Skip second WriteToc() call for custom-format dumps without data. Presently, "pg_dump --format=custom" calls WriteTo

Re: Statistics Import and Export

2025-04-02 Thread Nathan Bossart
On Wed, Apr 02, 2025 at 10:34:58PM -0400, Corey Huinker wrote: >> >> > Also, why do we need the clause "WHERE s.tablename = ANY($2)"? Isn't >> > that already implied by "JOIN unnest($1, $2) ... s.tablename = >> > u.tablename"? >> >> Good question. Corey, do you recall why this was needed? >> > >

Re: Statistics Import and Export

2025-04-01 Thread Nathan Bossart
right, so our usual tests should also offer some assurance. > Also, why do we need the clause "WHERE s.tablename = ANY($2)"? Isn't > that already implied by "JOIN unnest($1, $2) ... s.tablename = > u.tablename"? Good question. Corey, do you recall why this was neede

Re: [PATCH v1] parallel pg_restore: avoid disk seeks when jumping short distance forward

2025-04-01 Thread Nathan Bossart
On Tue, Apr 01, 2025 at 09:33:32PM +0200, Dimitrios Apostolou wrote: > It didn't break any test, but I also don't see any difference, the > performance boost is noticeable only when restoring a huge archive that is > missing offsets. This seems generally reasonable to me, but how did you decide on

Re: Statistics Import and Export

2025-04-01 Thread Nathan Bossart
On Mon, Mar 31, 2025 at 09:33:15PM -0500, Nathan Bossart wrote: > My goal is to commit the attached patches on Friday morning, but of course > that is subject to change based on any feedback or objections that emerge > in the meantime. I spent some more time polishing these patches thi

Re: Statistics Import and Export

2025-04-01 Thread Nathan Bossart
On Tue, Apr 01, 2025 at 01:20:30PM -0500, Nathan Bossart wrote: > On Mon, Mar 31, 2025 at 09:33:15PM -0500, Nathan Bossart wrote: >> My goal is to commit the attached patches on Friday morning, but of course >> that is subject to change based on any feedback or objections that e

Re: Improve CRC32C performance on SSE4.2

2025-04-01 Thread Nathan Bossart
On Tue, Apr 01, 2025 at 05:33:02PM +0700, John Naylor wrote: > On Thu, Mar 27, 2025 at 2:55 AM Devulapalli, Raghuveer > wrote: >> (1) zmm_regs_available() in pg_crc32c_sse42_choose.c is a duplicate of >> pg_popcount_avx512.c but perhaps that’s fine for now. I will propose a >> consolidated SIMD r

Re: Statistics Import and Export

2025-03-31 Thread Nathan Bossart
it the attached patches on Friday morning, but of course that is subject to change based on any feedback or objections that emerge in the meantime. -- nathan >From 0256405dba245baa90c3fe35ee69e3cb1ce6253e Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 31 Mar 2025 10:44:26 -0500 Su

Re: [PATCH] SVE popcount support

2025-03-28 Thread Nathan Bossart
Committed. On Fri, Mar 28, 2025 at 10:25:26AM -0500, Nathan Bossart wrote: > On Thu, Mar 27, 2025 at 03:31:27PM +0700, John Naylor wrote: >> On Thu, Mar 27, 2025 at 10:38 AM Nathan Bossart >> wrote: >>> I also noticed a silly mistake in 0003 that would cause us to pote

Re: [PATCH] SVE popcount support

2025-03-28 Thread Nathan Bossart
On Thu, Mar 27, 2025 at 03:31:27PM +0700, John Naylor wrote: > On Thu, Mar 27, 2025 at 10:38 AM Nathan Bossart > wrote: >> I also noticed a silly mistake in 0003 that would cause us to potentially >> skip part of the tail. That should be fixed now. > > I'm not sur

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-03-28 Thread Nathan Bossart
On Fri, Mar 28, 2025 at 05:08:26PM +0530, Mahendra Singh Thalor wrote: > Here, I am attaching updated patches for review. > > v04_001* has the changes for CREATE DATABASE/ROLE/USER and > v04_002* has the changes into pg_upgrade to give ALERTS for invalid names. In general, +1 for these changes.

Re: Disabling vacuum truncate for autovacuum

2025-03-27 Thread Nathan Bossart
On Thu, Mar 20, 2025 at 09:59:45AM -0700, David G. Johnston wrote: > I get the need for this kind of logic, since we used a boolean for the > table option, but as a self-proclaimed hack it seems worth more comments > than provided here. Especially pertaining to whether this is indeed > generic or

Re: [PATCH] SVE popcount support

2025-03-26 Thread Nathan Bossart
On Wed, Mar 26, 2025 at 04:44:24PM -0500, Nathan Bossart wrote: > IMHO these are acceptable results, at least for the use-cases I see in the > tree. We might be able to minimize the difference between the Neon and SVE > implementations on the low end with some additional code, but I

Re: [PATCH] SVE popcount support

2025-03-26 Thread Nathan Bossart
with some additional code, but I'm really not sure if it's worth the effort. Barring feedback or objections, I'm planning to commit these on Friday. -- nathan >From 1a8d7b9552efa3de23be4b18b8031520150a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 24 Mar 2025 19

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-26 Thread Nathan Bossart
On Wed, Mar 26, 2025 at 08:09:53AM -0700, David G. Johnston wrote: > I'm willing to say "I don't know why this is so very important to Nikolay, > but I trust him that it is, and since my opinion isn't that strong and this > isn't a big deal, so I will accommodate the person screaming that adding >

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-26 Thread Nathan Bossart
On Wed, Mar 26, 2025 at 11:41:13AM -0400, Robert Haas wrote: > What I'm upset about is that it feels to me like Nikolay is trying to > win the argument by yelling. I don't want that to be the way we do > things around here. I admit that sometimes it is, and I think that is > bad, no matter who the

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-26 Thread Nathan Bossart
On Wed, Mar 26, 2025 at 03:30:49PM +0100, Álvaro Herrera wrote: > On 2025-Mar-26, David G. Johnston wrote: >> The argument being made is that the enum patch adheres to established >> practices; and when adding new code that new code is encouraged to adhere >> to how existing code is written. A vot

Re: Cannot find a working 64-bit integer type on Illumos

2025-03-25 Thread Nathan Bossart
On Tue, Mar 25, 2025 at 09:41:21PM +1300, Thomas Munro wrote: > On Wed, Mar 5, 2025 at 6:00 AM Peter Eisentraut wrote: >> I agree with your patch 0001-Deprecate-pg_int64.patch. I don't see a >> reason to keep the current arrangement around pg_int64. > > Thanks for looking! Pushed. anaconda see

Re: optimize file transfer in pg_upgrade

2025-03-25 Thread Nathan Bossart
On Thu, Mar 20, 2025 at 03:23:13PM -0500, Nathan Bossart wrote: > I'm still aiming to commit this sometime early next week. Committed. Thanks to everyone who chimed in on this thread. While writing the attributions, I noticed that nobody seems to have commented specifically on 00

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-25 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 08:57:27PM -0700, David G. Johnston wrote: > On Mon, Mar 24, 2025 at 11:45 AM Nathan Bossart > wrote: >> * I don't think this matches the parse_bool() behavior exactly. For >> example, parse_bool() appears to accept inputs like "t"

Re: Disabling vacuum truncate for autovacuum

2025-03-24 Thread Nathan Bossart
On Fri, Mar 21, 2025 at 08:54:55AM -0700, David G. Johnston wrote: > I'm still partial to mine but yours probably fits the overall style of the > codebase better. Committed with some light edits. -- nathan

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 10:35:41PM +0300, Nikolay Shaplov wrote: > We can have isset_offset, but then we have redesign all options with > custom unset behavior to use it, instead of unreachable default value. > This will make it consistent then. I don't see any reason why we are compelled to redes

Re: [PATCH] SVE popcount support

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 06:34:45PM +0700, John Naylor wrote: > On Sat, Mar 22, 2025 at 10:42 AM Nathan Bossart > wrote: >> * 0002 introduces the Neon implementation, which conveniently doesn't need >> configure-time checks or function pointers. I noticed that some >

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 09:03:11PM +0300, Nikolay Shaplov wrote: > В письме от понедельник, 24 марта 2025 г. 20:48:29 MSK пользователь Nathan > Bossart написал: > >> For the sake of discussion, here is what the enum approach would look like. > > In my point of view this

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
For the sake of discussion, here is what the enum approach would look like. -- nathan >From f108e6c7e07c4148f097fbfd612cf79db60c5acd Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 24 Mar 2025 12:46:26 -0500 Subject: [PATCH 1/1] change vacuum_truncate relopt to enum --- src/back

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 07:53:43PM +0300, Nikolay Shaplov wrote: > You've just changed the whole engine, for what is seems to be an exceptional > case, that can be solved via existing means. I have not changed the whole engine. I have added an optional integer field to a single struct. -- nath

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 09:40:24AM -0700, David G. Johnston wrote: > So, given the precedent of vacuum_index_cleanup and the above, we should > turn this into an enum that accepts all existing boolean literal inputs and > also has a undocumented "unset" default value that the user is not allowed >

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 05:00:51PM +0100, Álvaro Herrera wrote: > I don't understand why this shouldn't work exactly like > vacuum_index_cleanup (cf. vacuum_rel lines 2170ff). That would require > no new mechanism. I explained my reasons for not proceeding with that approach upthread [0]. I don'

Re: vacuum_truncate configuration parameter and isset_offset

2025-03-24 Thread Nathan Bossart
On Mon, Mar 24, 2025 at 03:27:35PM +0300, Nikolay Shaplov wrote: > I would suggest to use enum here to achieve same goal, or add some "trilean" > data type that can be "true/false/unset" I did first try making the vacuum_truncate reloption an enum, but I didn't proceed with that approach for a co

Re: [PATCH] SVE popcount support

2025-03-21 Thread Nathan Bossart
26196731aa2379babf535e698260f0066 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Fri, 21 Mar 2025 09:47:30 -0500 Subject: [PATCH v8 1/3] Rename TRY_POPCNT_FAST to POPCNT_X86_64. --- src/include/port/pg_bitutils.h | 6 +++--- src/port/pg_bitutils.c | 14 +++--- src/port/pg_popcount

Re: optimize file transfer in pg_upgrade

2025-03-20 Thread Nathan Bossart
On Thu, Mar 20, 2025 at 11:11:46AM -0500, Nathan Bossart wrote: > As promised, I've committed just 0001 for now. I'll watch closely for any > issues in the buildfarm. Seeing none, here's is a rebased patch set without 0001. The only changes are some fleshed-out comment

Re: optimize file transfer in pg_upgrade

2025-03-20 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 09:02:42PM -0500, Nathan Bossart wrote: > On Wed, Mar 19, 2025 at 04:28:23PM -0500, Nathan Bossart wrote: >> On Wed, Mar 19, 2025 at 02:32:01PM -0500, Nathan Bossart wrote: >>> In addition to testing with in-place tablespaces, we might also want to >

Re: Disabling vacuum truncate for autovacuum

2025-03-20 Thread Nathan Bossart
Committed. -- nathan

Re: optimize file transfer in pg_upgrade

2025-03-19 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 04:28:23PM -0500, Nathan Bossart wrote: > On Wed, Mar 19, 2025 at 02:32:01PM -0500, Nathan Bossart wrote: >> In addition to testing with in-place tablespaces, we might also want to >> teach the transfer modes test to do cross-version testing when possible.

Re: optimize file transfer in pg_upgrade

2025-03-19 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 02:32:01PM -0500, Nathan Bossart wrote: > In addition to testing with in-place tablespaces, we might also want to > teach the transfer modes test to do cross-version testing when possible. > In that case, we can test normal (non-in-place) tablespaces. However, tha

Re: Disabling vacuum truncate for autovacuum

2025-03-19 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 10:40:31AM -0500, Nathan Bossart wrote: > On Thu, Mar 20, 2025 at 12:34:59AM +0900, Fujii Masao wrote: >> +# - Default Behavior - >> + >> +#vacuum_truncate = on # enable truncation after vacuum >> >> Since there's

Re: Disabling vacuum truncate for autovacuum

2025-03-19 Thread Nathan Bossart
On Thu, Mar 20, 2025 at 12:34:59AM +0900, Fujii Masao wrote: > +# - Default Behavior - > + > +#vacuum_truncate = on# enable truncation after vacuum > > Since there's no existing GUC category that fits to vacuum_truncate, > I'm fine with adding a new category like "Vacuuming

Re: optimize file transfer in pg_upgrade

2025-03-19 Thread Nathan Bossart
On Tue, Mar 18, 2025 at 09:14:22PM -0500, Nathan Bossart wrote: > And here is a new version of the full patch set. I'm currently planning to commit this sometime early-ish next week. One notable loose end is the lack of a pg_upgrade test with a non-default tablespace, but that is an

Re: PATCH: warn about, and deprecate, clear text passwords

2025-03-19 Thread Nathan Bossart
On Wed, Mar 19, 2025 at 10:06:58AM -0400, Robert Haas wrote: > On Wed, Mar 19, 2025 at 9:24 AM Greg Sabino Mullane > wrote: >>> The user has no particular reason to care about the fact that the >>> password they just typed ended up in the log. That is a concern for the >>> DBA, not the user, and

Re: optimize file transfer in pg_upgrade

2025-03-19 Thread Nathan Bossart
On Tue, Mar 18, 2025 at 01:50:10PM -0400, Andres Freund wrote: > On 2025-03-18 12:47:01 -0500, Nathan Bossart wrote: >> On Tue, Mar 18, 2025 at 01:37:02PM -0400, Andres Freund wrote: >> > - Do we need a new old cluster for each of the modes? That seems like >> > wasted

Re: optimize file transfer in pg_upgrade

2025-03-18 Thread Nathan Bossart
platform." Each test verifies that some very simple objects make it to the new version, which we could of course expand on. Would something like this suffice? -- nathan >From 89b27b68194c2be0e1aebdee871e556028cdd5b5 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 18 Mar

  1   2   3   4   5   6   7   8   9   10   >