Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 04:39:00PM -0400, Corey Huinker wrote: > On Thu, Jun 12, 2025 at 4:22 PM Nathan Bossart > wrote: >> I do think this is useful functionality, I only suggested removing it >> because AFAICT it is redundant, i.e., you can accomplish the same thing >>

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 04:12:35PM -0400, Corey Huinker wrote: > The use case for --statistics-only is to extract the existing statistics > for the tables and indexes that are involved in a given query that is > giving you problems, allowing you to apply those statistics to an existing > QA/dev dat

Re: pg_get_multixact_members not documented

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 12:15:03AM -0500, Sami Imseih wrote: >> Attached patch removing extra comma. Otherwise LGTM. > > Thanks! For v4, the final comma in the list is grammatically correct, > and it´s the style we use throughout the docs. +1 > I marked the patch RFC. Barring comments/objection

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 10:07:05PM +0200, Laurenz Albe wrote: > I must be missing something, but I think --no-statistics is sorely needed. > How else can I get the effect of > > pg_dump --no-statistics mydb This was recently changed to be the default behavior (see commit 34eb2a8). -- nathan

Re: Use RELATION_IS_OTHER_TEMP where possible

2025-06-12 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 07:35:39PM +0800, Junwang Zhao wrote: > I have created a cf entry to track this. > > https://commitfest.postgresql.org/patch/5815/ I'll plan on committing this once v19 development begins. -- nathan

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 08:58:15AM -0700, Jeff Davis wrote: > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: >> If the idea is to remove all options for default behavior, we'd be >> removing >> --no-statistics, --with-data, and --with-schema at this poi

Re: add function for creating/attaching hash table in DSM registry

2025-06-12 Thread Nathan Bossart
ash [0]. I am ok with " DSA" because the DSA tranche > is created implicitly by the API. Okay, I've done this in the attached patch. -- nathan >From 345fa06de44c836c90ce26866a6b7f6b87d1ec6e Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 4 Jun 2025 14:21:14 -0500 Sub

Re: Replace some %llu remnants in the tree

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 07:16:37AM +0900, Michael Paquier wrote: > On Wed, Jun 11, 2025 at 09:58:00AM +0200, Peter Eisentraut wrote: >> On 09.06.25 05:59, Michael Paquier wrote: >>> That's not necessarily mandatory for v18, for sure, but as this is new >>> code we could as well clean it up before f

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 10:18:56AM -0400, Robert Haas wrote: > On Fri, Jun 6, 2025 at 11:40 AM Nathan Bossart > wrote: >> On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: >> > What is the purpose of the --with-data option? Dumping the data is the >

Re: CHECKPOINT unlogged data

2025-06-11 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 05:53:15PM +0200, Christoph Berg wrote: > Ack, done in v4. Thanks! The overall shape of these patches looks pretty good to me. I'll aim to give them a deeper review in the near future. -- nathan

Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward

2025-06-11 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 12:32:58AM +0200, Dimitrios Apostolou wrote: > Thank you for benchmarking! Before answering in more depth, I'm curious, > what read-seek pattern do you see on the system call level (as shown by > strace)? In pg_restore it was a constant loop of read(4K)-lseek(8-16K). For fs

Re: add function for creating/attaching hash table in DSM registry

2025-06-11 Thread Nathan Bossart
#x27;ve done in the test. -- nathan >From f08bfe5945ff2661cb9710dedff04237a215dc6b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 4 Jun 2025 14:21:14 -0500 Subject: [PATCH v6 1/1] simplify creating hash table in dsm registry --- src/backend/storage/ipc/dsm_regis

Re: add function for creating/attaching hash table in DSM registry

2025-06-11 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 05:11:54PM +0300, Florents Tselai wrote: >> On 11 Jun 2025, at 4:57 PM, Nathan Bossart wrote: >> I considered adding another function that would create/attach a DSA in the >> DSM registry, since that's already an intermediate step of dshash creatio

Re: CHECKPOINT unlogged data

2025-06-11 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 03:45:46PM +0200, Christoph Berg wrote: > Do we want to change the checkpoint log message (and the new options) > only, or include the CHECKPOINT_* flags? (I would guess there aren't > many external users of these flags, but mmmv.) IMO we should try to make the terminology

Re: add function for creating/attaching hash table in DSM registry

2025-06-11 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 07:15:56PM +0530, Rahila Syed wrote: >> How can one dsa_allocate in the same area as the returned dshash_table ? >> in other words: shouldn't the state->dsa_handle be returned somehow ? > > +1. FWIW, Having used the DSA apis in my code, I think having the registry > return >

Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward

2025-06-10 Thread Nathan Bossart
On Mon, Jun 09, 2025 at 10:09:57PM +0200, Dimitrios Apostolou wrote: > Fix by avoiding forward seeks for jumps of less than 1MB forward. > Do instead sequential reads. > > Performance gain can be significant, depending on the size of the dump > and the I/O subsystem. On my local NVMe drive, read s

Re: add function for creating/attaching hash table in DSM registry

2025-06-10 Thread Nathan Bossart
On Tue, Jun 10, 2025 at 02:05:16PM -0500, Sami Imseih wrote: > There is also that dynamic tranche named are stored in local backend > look-up table, so if you have some backends that attached some dynamic > hash table > and others that did not, only the ones that registered would be able to > resol

Re: Improve tab completion for COPY

2025-06-10 Thread Nathan Bossart
On Tue, Jun 10, 2025 at 12:37:48PM -0700, Masahiko Sawada wrote: >> > (1) adds tab completion support for the REJECT_LIMIT option, which was >> > introduced in v18 >> > (2) splits the tab completion logic between COPY FROM and COPY TO to >> > reflect their different options. > > [...] > > Given RE

Re: add function for creating/attaching hash table in DSM registry

2025-06-10 Thread Nathan Bossart
; > is it worth considering increasing this to 128 maybe? > > I´ve used DSMR extensively for namespacing keys etc, and I´ve come close to > 50-60 chars at times. > > I´m not too fixed on that though. Seems fine to me. -- nathan >From c5dc6135b5fe04257debe5ceb80601a8a11dd720 M

Re: Use RELATION_IS_OTHER_TEMP where possible

2025-06-10 Thread Nathan Bossart
On Wed, Jun 11, 2025 at 12:07:35AM +0800, Junwang Zhao wrote: > All other files perform this check using RELATION_IS_OTHER_TEMP. > Should we update tablecmds.c to do the same for consistency? Seems like a good idea. -- nathan

Re: add function for creating/attaching hash table in DSM registry

2025-06-10 Thread Nathan Bossart
On Tue, Jun 10, 2025 at 10:38:29AM -0500, Nathan Bossart wrote: > On Mon, Jun 09, 2025 at 07:14:28PM -0500, Sami Imseih wrote: >> Going back to the original point, DSMRegistryHash and DSMRegistryHash >> are built-in, and those names are well-defined and actually refer to >>

Re: add function for creating/attaching hash table in DSM registry

2025-06-10 Thread Nathan Bossart
hash table, so presumably any wait events for those locks should be named accordingly, right? -- nathan >From c04bcb37eb623375d334120dbc897b0237c78bf1 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 4 Jun 2025 14:21:14 -0500 Subject: [PATCH v3 1/1] simplify creating hash table in dsm

Re: Cleanup gcc trick with varattrib_1b_e in VARATT_EXTERNAL_GET_POINTER()

2025-06-09 Thread Nathan Bossart
On Mon, Jun 09, 2025 at 05:21:19PM +0900, Michael Paquier wrote: > Back in b89e151054a0, the following macro has been introduced to > retrieve the varatt_external of an on-disk external TOAST Datum, stuff > now in detoast.h: > /* > * Macro to fetch the possibly-unaligned contents of an EXTERNAL da

Re: add function for creating/attaching hash table in DSM registry

2025-06-09 Thread Nathan Bossart
On Mon, Jun 09, 2025 at 03:10:30PM -0500, Sami Imseih wrote: > One thing I noticed, and I´m not too fond of, is that the wait_event name > shows > up with the _dsh suffix: > > ``` > postgres=# select query, wait_event, wait_event_type from pg_stat_activity ; > query | wait_event > | wait_event_ty

Re: dsm_registry: Add detach and destroy features

2025-06-09 Thread Nathan Bossart
On Tue, Mar 18, 2025 at 10:05:07AM +0900, Sungwoo Chang wrote: > +/* > + * Attempt to destroy a named DSM segment > + * > + * This routine attempts to destroy the DSM segment. We unpin the dsm_segment > + * and delete the entry from dsm_registry_table. This may not destroy the > + * dsm_segment ins

Re: pg_rewind: Doc update for PostgreSQL 18

2025-06-09 Thread Nathan Bossart
On Mon, Jun 09, 2025 at 01:38:09PM -0400, Jesper Pedersen wrote: > pg_rewind requires that the target server > either has > the option enabled > in postgresql.conf or data checksums enabled when > - the cluster was initialized with initdb. > Neither of these > - are currently on

Re: a couple of small cleanup patches for DSM-related code

2025-06-06 Thread Nathan Bossart
Committed. -- nathan

Re: CHECKPOINT unlogged data

2025-06-06 Thread Nathan Bossart
On Fri, Jun 06, 2025 at 06:20:21PM +0200, Christoph Berg wrote: > Re: Nathan Bossart >> I don't understand why we need to add both FAST and IMMEDIATE. > > We have both: > > =# checkpoint ; > 2025-06-06 18:09:25.743 CEST [872379] LOG: checkpoint star

Re: pg_dump --with-* options

2025-06-06 Thread Nathan Bossart
On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: > We have > > -a, --data-only dump only the data, not the schema or statistics > --no-datado not dump data > --with-data dump the data # this one is new > > (and there is also --section=data), and t

Re: CHECKPOINT unlogged data

2025-06-06 Thread Nathan Bossart
On Fri, Jun 06, 2025 at 04:26:56PM +0200, Christoph Berg wrote: > Re: Fujii Masao >> Some users might want to trigger a spread checkpoint but not wait for >> it to finish, since it could take a long time? If that's a valid use case, >> maybe we should add a WAIT option to let users choose whether t

Re: Partitioned tables and [un]loggedness

2025-06-05 Thread Nathan Bossart
On Thu, Jun 05, 2025 at 01:57:30PM +0900, Michael Paquier wrote: > How does the RMT feel about this change? Nathan, would you be OK with > that? It's not a big problem, as well, if the code is kept as-is, but > as it's a simple change.. IMHO a case can be reasonably made that this is an oversigh

Re: postmaster uses more CPU in 18 beta1 with io_method=io_uring

2025-06-05 Thread Nathan Bossart
On Thu, Jun 05, 2025 at 12:47:52PM -0400, Tom Lane wrote: > Andres Freund writes: >> I think this is a big enough pitfall that it's, obviously assuming the patch >> has a sensible complexity, worth fixing this in 18. RMT, anyone, what do you >> think? > > Let's see the patch ... but yeah, I'd rat

Re: add function for creating/attaching hash table in DSM registry

2025-06-05 Thread Nathan Bossart
t; state->dsh_tranche_name); Done. > 3/ It will be good to "Assert(dsh)" before "return dsh;" for safety? > > MemoryContextSwitchTo(oldcontext); > LWLockRelease(DSMRegistryLock); > > return dsh; Eh, I would expect the tests to start failing horri

Re: like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-06-04 Thread Nathan Bossart
On Wed, Jun 04, 2025 at 09:03:03PM +0300, Florents Tselai wrote: > On 4 Jun 2025, at 7:19 PM, Nathan Bossart wrote: >> Would you mind combining the patches into just one patch? That's how I >> reviewed it, and that's how it would be committed. > > This works ? Yes, thanks. -- nathan

add function for creating/attaching hash table in DSM registry

2025-06-04 Thread Nathan Bossart
alled GetNamedDSMHash() that makes creating/attaching a hash table in the DSM registry much easier. -- nathan >From cf29232fbc7e6702f15dbc4f0c238fd69ec648a9 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 4 Jun 2025 14:21:14 -0500 Subject: [PATCH v1 1/1] simplify creating hash table in dsm

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-04 Thread Nathan Bossart
On Wed, Jun 04, 2025 at 04:46:52PM -0400, Bruce Momjian wrote: > On Wed, Jun 4, 2025 at 10:15:49PM +0200, Peter Eisentraut wrote: >> I tend think pg_upgrade should stick to checking things that are necessary >> for the upgrade to succeed. It shouldn't start being an interactive portal >> to the r

a couple of small cleanup patches for DSM-related code

2025-06-04 Thread Nathan Bossart
y objections? -- nathan >From 1fa264cbc3ee7775adaf4a682e2edd229b5b16db Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 4 Jun 2025 14:24:38 -0500 Subject: [PATCH v1 1/2] fix sign mismatches in test_dsm_registry --- src/test/modules/test_dsm_registry/test_dsm_registry.c | 4 ++--

Re: fix notes about password encryption in pg_authid docs

2025-06-04 Thread Nathan Bossart
On Wed, Jun 04, 2025 at 02:11:55PM +0900, Michael Paquier wrote: > Looks fine to me. Committed, thanks. -- nathan

Re: like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-06-04 Thread Nathan Bossart
On Wed, Jun 04, 2025 at 11:17:42AM +0300, Florents Tselai wrote: > Thanks; let's wait a bit to see if there're any objections. Would you mind combining the patches into just one patch? That's how I reviewed it, and that's how it would be committed. -- nathan

Re: Partitioned tables and [un]loggedness

2025-06-04 Thread Nathan Bossart
On Wed, Jun 04, 2025 at 07:05:20PM +0900, Shinya Kato wrote: > On Wed, Jun 4, 2025 at 6:55 PM Shinya Kato wrote: >> Should we consider preventing tab completion for PARTITION BY >> immediately after CREATE TABLE name (...)? Or is it fine to leave it >> as is, given that it's syntactically correct?

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-06-04 Thread Nathan Bossart
I have marked the commitfest entry for this [0] as waiting-on-author because the patch needs to be rebased. [0] https://commitfest.postgresql.org/patch/5538/ -- nathan

Re: like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-06-03 Thread Nathan Bossart
This latest patch set looks pretty good to me. -- nathan

Re: like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 10:39:25PM +0300, Florents Tselai wrote: > Thanks for the detailed review Nathan Thanks for the updated patch! +if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo)) +ereport(ERROR, (errmsg("pg_get_dsm_registry_allocations must be used in a SRF context"))); Init

Re: Add log_autovacuum_{vacuum|analyze}_min_duration

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 10:57:11AM +0200, Michael Banck wrote: > On Tue, Jun 03, 2025 at 05:25:40PM +0900, Shinya Kato wrote: >> I surely think adding log_autoanalyze_min_duration is simpler and >> shorter, but the reason I chose this GUC name is for consistency with >> other autovacuum parameters.

Re: fix notes about password encryption in pg_authid docs

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 01:43:15PM +0900, Michael Paquier wrote: > You are missing one reference in doc/src/sgml/system-views.sgml for > pg_shadow.passwd, no? Yup. Here is an updated patch. -- nathan >From 2d41fa2cff14b548905e2dfdd98b992976137e61 Mon Sep 17 00:00:00 2001 From: Natha

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 09:43:59AM -0500, Nathan Bossart wrote: > On Tue, Jun 03, 2025 at 10:34:06AM -0400, Tom Lane wrote: >> If we really want to be in peoples' face about this, the thing >> to do is to print a warning every time they log in with an MD5 >> password.

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 10:34:06AM -0400, Tom Lane wrote: > If we really want to be in peoples' face about this, the thing > to do is to print a warning every time they log in with an MD5 > password. Also, to Michael's point, that really would be exactly > the same place where the eventual "sorry,

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-03 Thread Nathan Bossart
On Tue, Jun 03, 2025 at 01:38:49PM +0900, Michael Paquier wrote: > I'm not sure that this is necessary. Only requiring one to use > --retain sounds kind of enough to me. Yeah, maybe we should just leave it alone for now. > Saying that, warning users if they have MD5 passwords is a good idea, > b

Re: like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-06-03 Thread Nathan Bossart
On Sat, Mar 15, 2025 at 10:41:15AM +0200, Florents Tselai wrote: > Here's an updated v3 that fixes some white spaces v2 had-no other changes. Thanks for the patch. > I'm wondering though if it also makes sense to expose: > - backend_pid (who created the segment) > - is_pinned bool > - created_at

Re: pg_get_multixact_members not documented

2025-06-02 Thread Nathan Bossart
On Mon, Jun 02, 2025 at 12:46:51PM -0500, Sami Imseih wrote: > v1-0001 is the documentation only patch. I improved upon the description > suggested in [0] Your patch adds an entry to the "Transaction ID and Snapshot Information Functions" table, while Álvaro's introduced a new "Multixact Functions

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-02 Thread Nathan Bossart
On Mon, Jun 02, 2025 at 12:41:47PM -0700, Jeff Davis wrote: > On Mon, 2025-06-02 at 12:04 -0500, Nathan Bossart wrote: >> That seems worth considering.  Another option could be to just >> document >> that files generated for warnings will be lost without --retain.  >>

Re: pg_upgrade: warn about roles with md5 passwords

2025-06-02 Thread Nathan Bossart
On Mon, Jun 02, 2025 at 09:45:55AM -0700, Jeff Davis wrote: > On Mon, 2025-06-02 at 10:32 -0500, Nathan Bossart wrote: >> The one thing I don't like about this check is that it's probably not >> great >> from a security standpoint to effectively announce which

pg_upgrade: warn about roles with md5 passwords

2025-06-02 Thread Nathan Bossart
t know how worried to be about this, either, but I did run into it while testing the attached patch, so it seemed worth bringing up. -- nathan >From 011fe65c172d8788b1b6f2447b658717b84db6f0 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 2 Jun 2025 10:10:03 -0500 Subject: [PATCH v1 1/1]

Re: fix notes about password encryption in pg_authid docs

2025-06-02 Thread Nathan Bossart
On Mon, Jun 02, 2025 at 09:16:10AM -0500, Nathan Bossart wrote: > I noticed that the docs for the pg_authid catalog still indicate that > passwords might be stored "unencrypted," which hasn't been possible since > commit eb61136. The attached patch attempts to fix that. I

fix notes about password encryption in pg_authid docs

2025-06-02 Thread Nathan Bossart
I noticed that the docs for the pg_authid catalog still indicate that passwords might be stored "unencrypted," which hasn't been possible since commit eb61136. The attached patch attempts to fix that. If acceptable, I'd back-patch it to all supported versions. -- nathan

Re: C11 / VS 2019

2025-06-02 Thread Nathan Bossart
On Mon, Jun 02, 2025 at 05:44:12AM +0200, Peter Eisentraut wrote: > Summary: > 1. require VS 2019 > 2. use C11 > > Thoughts? +1 -- nathan

Re: pg_get_multixact_members not documented

2025-05-30 Thread Nathan Bossart
On Fri, May 30, 2025 at 02:23:30PM -0500, Sami Imseih wrote: > While looking into another multixact related topic, I realised that > pg_get_multixact_members > is not documented. I saw the lack of documentation was mentioned here [0], but > this was never committed. > > Any reason it should not be

Re: pg_get_multixact_members not documented

2025-05-30 Thread Nathan Bossart
On Fri, May 30, 2025 at 04:24:43PM -0500, Sami Imseih wrote: >> Want to put together a patch? > > Yes, will do For extra credit, maybe we could add a test or two, too... -- nathan

Re: regdatabase

2025-05-30 Thread Nathan Bossart
On Fri, May 30, 2025 at 04:55:58PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> For now, I've just added another case block for REGDATABASEOID to match the >> others. If there are problems with non-pinned objects being considered >> shippable, it's not really

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-05-30 Thread Nathan Bossart
On Wed, May 28, 2025 at 11:43:52AM +0900, Michael Paquier wrote: > Thanks for the refreshed versions. Looks sensible to me overall. Committed. Thanks for reviewing. -- nathan

Re: regdatabase

2025-05-30 Thread Nathan Bossart
On Mon, May 19, 2025 at 04:17:26PM -0500, Nathan Bossart wrote: > On Mon, May 19, 2025 at 10:37:29AM +0800, jian he wrote: >> I saw REGROLEOID in foreign_expr_walker, >> I'm wondering whether REGDATABASEOID is reachable within foreign_expr_walker. >> >> not fam

Re: CHECKPOINT unlogged data

2025-05-30 Thread Nathan Bossart
On Fri, May 30, 2025 at 12:39:02PM -0400, Andres Freund wrote: > On 2025-05-30 18:17:45 +0200, Christoph Berg wrote: >> A customer reported to use CHECKPOINT before shutdowns to make the >> shutdowns themselves faster and asked if it was possible to make >> CHECKPOINT optionally also write out unlo

Re: Add comment explaining why queryid is int64 in pg_stat_statements

2025-05-29 Thread Nathan Bossart
On Thu, May 29, 2025 at 01:53:07PM +0900, Michael Paquier wrote: > On Thu, May 22, 2025 at 01:01:14PM +0900, Michael Paquier wrote: >> I have added an open item about the plan ID part as it applies to v18, >> adding the RMT in CC to get an opinion. If we cannot get a consensus >> on all that, lett

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-05-27 Thread Nathan Bossart
On Thu, May 08, 2025 at 08:55:08AM +0900, Michael Paquier wrote: > On Wed, May 07, 2025 at 02:55:49PM -0500, Nathan Bossart wrote: >> Committed with that change. That takes care of a good chunk of these TOAST >> snapshot problems. I think there are about 4 others left, unless s

Re: PG 18 release notes draft committed

2025-05-27 Thread Nathan Bossart
For the "Deprecate MD5 password authentication" item, IMHO we should emphasize that support for MD5 passwords will be removed in a future major release, as we did for the 18beta1 announcement. Here's an attempt: Deprecate MD5 password authentication (Nathan Bossart)

Re: Statistics Import and Export

2025-05-22 Thread Nathan Bossart
On Thu, May 22, 2025 at 03:29:38PM -0400, Greg Sabino Mullane wrote: > On Thu, May 22, 2025 at 2:52 PM Jeff Davis wrote: >> * The default for pg_restore is --no-statistics. That could cause a minor >> surprise if the user specifies --with-statistics for pg_dump and >> not for pg_restore. An argum

Re: Statistics Import and Export

2025-05-22 Thread Nathan Bossart
On Wed, May 21, 2025 at 04:53:17PM -0700, Jeff Davis wrote: > On Wed, 2025-05-21 at 16:29 -0500, Nathan Bossart wrote: >> I don't know precisely where that line might be, but in this case, >> the >> dumped stats have no hope of restoring into anything older than >>

Re: Statistics Import and Export

2025-05-22 Thread Nathan Bossart
On Thu, May 22, 2025 at 10:20:16AM -0400, Robert Haas wrote: > It also sort > of looks like we might have a consensus anyway: Jeff said "I lean > towards making it opt-in for pg_dump and opt-out for pg_upgrade" and I > agree with that and it seems you do, too. So perhaps Jeff should make > it so?

Re: Statistics Import and Export

2025-05-21 Thread Nathan Bossart
On Wed, May 21, 2025 at 02:14:55PM -0700, Jeff Davis wrote: > Originally, one of the reasons we added a version field during dump is > so that some future version could reinterpret stats in older dump files > during import. > > This patch is using a newer version of pg_dump to interpret stats from

Re: Statistics Import and Export

2025-05-21 Thread Nathan Bossart
2da392d5f Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 21 May 2025 10:53:29 -0500 Subject: [PATCH v3 1/1] pg_dump: Adjust reltuples from 0 to -1 for dumps on older versions. Before v14, a reltuples value of 0 was ambiguous: it could either mean the relation is empty, or it could mea

Re: Remove Instruction Synchronization Barrier in spin_delay() for ARM64 architecture

2025-05-19 Thread Nathan Bossart
On Mon, May 19, 2025 at 11:38:39AM -0500, Nathan Bossart wrote: > On Tue, May 13, 2025 at 11:37:45AM -0700, Salvatore Dipietro wrote: >> Based on your findings Nathan, what is the best way to proceed for this >> change? >> Do we need more validation for it? If yes, which

Re: Statistics Import and Export

2025-05-19 Thread Nathan Bossart
On Mon, May 19, 2025 at 02:13:45PM -0700, Hari Krishna Sunder wrote: > I think it would be better to revert 9879105 since there can be a > considerable number of true empty tables that we don´t need to process. I'm not sure that's a use-case we really need to optimize. Even with 100,000 empty tab

Re: regdatabase

2025-05-19 Thread Nathan Bossart
On Mon, May 19, 2025 at 10:37:29AM +0800, jian he wrote: > I saw REGROLEOID in foreign_expr_walker, > I'm wondering whether REGDATABASEOID is reachable within foreign_expr_walker. > > not familiar with this area, also there are no coverage tests for > other reg*Const, > per > https://coverage.pos

Re: Statistics Import and Export

2025-05-19 Thread Nathan Bossart
On Wed, May 14, 2025 at 01:30:48PM -0700, Hari Krishna Sunder wrote: > Here is the patch with a comment. Thanks. > On Wed, May 14, 2025 at 8:53 AM Nathan Bossart > wrote: >> There was a similar report for vacuumdb's new --missing-stats-only option. >> We fixed that in

Re: Remove Instruction Synchronization Barrier in spin_delay() for ARM64 architecture

2025-05-19 Thread Nathan Bossart
On Tue, May 13, 2025 at 11:37:45AM -0700, Salvatore Dipietro wrote: > On Thu, 1 May 2025 at 14:50, Nathan Bossart wrote: >> So... >> >> * The ISB does seem to have a positive effect without commit 3d0b4b1 >> applied. >> >> * With commit 3d0b4b1 applied, re

Re: Statistics Import and Export

2025-05-14 Thread Nathan Bossart
On Tue, May 13, 2025 at 05:01:02PM -0700, Hari Krishna Sunder wrote: > We found a minor issue when testing statistics import with upgrading from > versions older than v14. (We have VACUUM and ANALYZE disabled) > 3d351d916b20534f973eda760cde17d96545d4c4 >

Re: [PATCH] Fix references in comments, and sync up heap_page_is_all_visible() with heap_page_prune_and_freeze()

2025-05-09 Thread Nathan Bossart
On Tue, May 06, 2025 at 03:39:07PM +, Gregory Burd wrote: > 0001: Updates that comment so future authors know that this "stripped > down function" should retain the logic in heap_page_prune_and_freeze(), > not lazy_scan_prune() as was the case before 6dbb490. Hm. It certainly had some resembl

Re: regdatabase

2025-05-08 Thread Nathan Bossart
On Thu, May 08, 2025 at 10:38:04PM +0900, Ian Lawrence Barwick wrote: > Revised patch attached which adds coverage of that and also for the > "constant of the type reg(role|database) cannot be used here" error. LGTM. I've marked it as ready-for-committer [0] and will plan on committing it as soon

Re: regdatabase

2025-05-07 Thread Nathan Bossart
On Wed, May 07, 2025 at 10:41:23PM -0500, Nathan Bossart wrote: > The attached patch should address everything I've mentioned. And this version of the patch should be free of my embarrassing copy/paste mistakes. -- nathan >From c5460d594a225d3443f901e79c6c9bc7501bd9af Mon Sep 17 00

Re: regdatabase

2025-05-07 Thread Nathan Bossart
-')::oid; > 0 Yeah, I'm not sure there's any coverage of that for the existing reg* types, either. -- nathan >From f104b98532302ed534c33ea2e15ff9ae9a37e5d7 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 May 2025 22:24:31 -0500 Subject: [PATCH v3 1/1] Add regda

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-05-07 Thread Nathan Bossart
On Wed, May 07, 2025 at 08:00:39AM +0900, Michael Paquier wrote: > On Tue, May 06, 2025 at 11:41:49AM -0500, Nathan Bossart wrote: >> I brought this up with the RMT, and everyone seemed okay with committing it >> for v18. > > Cool, thanks for the update. >

Re: add --sequence-data to pg_dumpall

2025-05-07 Thread Nathan Bossart
On Tue, May 06, 2025 at 11:39:33AM -0500, Nathan Bossart wrote: > I brought this up in the RMT meeting today, and everyone was fine with > committing it for v18. I'll plan on doing so later this week. Committed. -- nathan

Re: PostgreSQL 18 Beta 1 release announcement draft

2025-05-07 Thread Nathan Bossart
On Tue, May 06, 2025 at 11:24:44PM -0400, Jonathan S. Katz wrote: > I took most of these changes. I don't think we should add the MD5 "will be > removed in a future release" in the beta announcement, as that could lead to > confusion (e.g. it's removed in beta 2!). I'm fine with adding that in the

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-05-07 Thread Nathan Bossart
On Mon, May 05, 2025 at 02:23:25PM -0500, Nathan Bossart wrote: > That leaves pg_shdepend. For now, I've just instructed pg_upgrade to COPY > the relevant pg_shdepend rows as an independent step, but perhaps there's a > reasonably straightforward way to put that in pg_dump,

Re: regdatabase

2025-05-06 Thread Nathan Bossart
On Wed, May 07, 2025 at 08:45:15AM +0900, Michael Paquier wrote: > For regdatabase, there would be at least two simplications related to > the dump of subscriptions, where we could switch the queries to the > new grammar for backend versions able to support the new grammar, > meaning that we could

Re: regdatabase

2025-05-06 Thread Nathan Bossart
On Wed, May 07, 2025 at 09:18:28AM +0900, Ian Lawrence Barwick wrote: > Hah, I put together a patch to implement just that a while back, but > then concluded > for some reason that it would likely be rejected so saved myself the > humiliation of > submitting it... > > Attaching patch for reference

regdatabase

2025-05-06 Thread Nathan Bossart
Every once in a while, I find myself wanting to use regdatabase for something like current_database()::regdatabase, and I'm always surprised when I inevitably rediscover that it doesn't exist. I only found one reference to the idea in the archives [0]. So, I have two questions: * Is there an eas

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-05-06 Thread Nathan Bossart
On Wed, Apr 30, 2025 at 09:57:46AM +0900, Michael Paquier wrote: > On Tue, Apr 29, 2025 at 02:01:54PM -0400, Tom Lane wrote: >> I'm inclined to argue that it's a bug fix and therefore still in-scope >> for v18. The fact that we can't back-patch such a change is all the >> more reason to not let it

Re: add --sequence-data to pg_dumpall

2025-05-06 Thread Nathan Bossart
On Wed, Apr 30, 2025 at 02:52:27PM -0500, Nathan Bossart wrote: > Since there's precedent, I'll plan on committing this in the next few days > unless someone objects. I've added the rest of the RMT to this thread, > too, just in case. I brought this up in the RMT meeting

Re: PostgreSQL 18 Beta 1 release announcement draft

2025-05-06 Thread Nathan Bossart
On Mon, May 05, 2025 at 03:07:01PM -0400, Jonathan S. Katz wrote: > Before PostgreSQL 18, an important step after performing a major version > upgrade was to run the `ANALYZE` to generate statistics, which is a > critical component of helping PostgreSQL to select the most efficient > query plan. Ba

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-05-05 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:07:09PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I do think it's worth considering going back to copying >> pg_largobject_metadata's files for upgrades from v16 and newer. > > (If we do this) I don't see why we'd nee

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-05-02 Thread Nathan Bossart
On Fri, May 02, 2025 at 10:42:25AM -0700, Jacob Champion wrote: > On Fri, May 2, 2025 at 10:35 AM Tom Lane wrote: >> FWIW, on my Mac a meson build from HEAD goes through fine, with or >> without this patch. I'm getting openssl and libcurl from MacPorts >> not Homebrew, but I don't know why that w

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-05-02 Thread Nathan Bossart
On Fri, May 02, 2025 at 10:05:26AM -0700, Jacob Champion wrote: > Nathan, if you get a chance, does the attached patch work for you? Yup, thanks! -- nathan

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-05-02 Thread Nathan Bossart
After commit b0635bf, I'm seeing the following meson build failures on macOS: In file included from ../postgresql/src/interfaces/libpq-oauth/oauth-curl.c:51: ../postgresql/src/interfaces/libpq/libpq-int.h:70:10: fatal error: 'openssl/ssl.h' file not found 70 | #include

Re: Remove Instruction Synchronization Barrier in spin_delay() for ARM64 architecture

2025-05-01 Thread Nathan Bossart
On Thu, May 01, 2025 at 04:08:06PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> ... commit 3d0b4b1 recently added a non-locking >> initial test in AArch64's TAS_SPIN, so I wonder if the ISB is still >> appropriate. It'd be interesting to see the performanc

Re: Remove Instruction Synchronization Barrier in spin_delay() for ARM64 architecture

2025-05-01 Thread Nathan Bossart
On Thu, May 01, 2025 at 02:48:59PM -0400, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 30, 2025 at 4:53 AM Salvatore Dipietro >> wrote: >>> we would like to propose the removal of the Instruction >>> Synchronization Barrier (isb) for aarch64 architectures. Based on our >>> testing on Grav

Re: not not - pgupgrade.sgml

2025-05-01 Thread Nathan Bossart
On Thu, May 01, 2025 at 05:13:07PM +0900, Fujii Masao wrote: > On 2025/05/01 15:30, Erik Rijkers wrote: >>   It seems to me that, in doc/src/sgml/ref/pgupgrade.sgml, this phrase: >> >> "Because not all statistics are not transferred" >> >>   should be: >> >> "Because not all statistics are tra

Re: Should shared_preload_libraries be loaded during binary upgrade?

2025-05-01 Thread Nathan Bossart
On Thu, May 01, 2025 at 10:05:48AM -0400, Tom Lane wrote: > Bruce Momjian writes: >> On Thu, May 1, 2025 at 11:05:56AM +0530, Dilip Kumar wrote: >>> Does it make sense to load "shared_preload_libraries" during binary >>> upgrade mode? > >> Well, the library might be required to load the data. W

Re: add --sequence-data to pg_dumpall

2025-04-30 Thread Nathan Bossart
On Wed, Apr 30, 2025 at 09:29:59AM +0900, Michael Paquier wrote: > On Tue, Apr 29, 2025 at 03:55:08PM -0500, Nathan Bossart wrote: >> Assuming we want this patch, should we apply it to v18? It's arguably an >> oversight in the pg_dump --sequence-data commit, and pg_dumpall w

Re: vacuumdb --missing-stats-only and pg_upgrade from PG13

2025-04-30 Thread Nathan Bossart
On Thu, Apr 24, 2025 at 03:25:55PM +0200, Christoph Berg wrote: > Re: Nathan Bossart >> Here is what I have staged for commit. I'll aim to commit these patches >> sometime next week to give time for additional feedback. > > I confirm my PG13 test table gets a

  1   2   3   4   5   6   7   8   9   10   >