Re: let's disallow ALTER ROLE bootstrap_superuser NOSUPERUSER

2022-07-21 Thread Nathan Bossart
think it makes sense to disallow removing its superuser attribute (at least via ALTER ROLE NOSUPERUSER). It seems like there is much work to do before a no-superuser configuration could be formally supported. If/when such support materializes, it might be possible to remove the restriction that Rober

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-07-21 Thread Nathan Bossart
ated by pg_db_role_setting? Also, instead of using PGC_USERSET in all the set_config_option() call sites, shouldn't we remove the "context" argument altogether? I am likely misunderstanding your proposal, but while I think simplifying set_config_option() is worthwhile, I don't see why it would preclude storing the context in pg_db_role_setting. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fwd: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-21 Thread Nathan Bossart
*/ +StartTransactionCommand(); process_session_preload_libraries(); +CommitTransactionCommand(); /* * Send this backend's cancellation info to the frontend. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fwd: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-21 Thread Nathan Bossart
On Thu, Jul 21, 2022 at 07:30:20PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> +StartTransactionCommand(); >> process_session_preload_libraries(); >> +CommitTransactionCommand(); > > Yeah, that way would avoid any questions about changing the or

privileges for ALTER ROLE/DATABASE SET

2022-07-22 Thread Nathan Bossart
what would they look like? My first instinct is to add GRANT ALTER ROLE ON PARAMETER and GRANT ALTER DATABASE ON PARAMETER. Thoughts? [0] https://postgr.es/m/1732511.1658332210%40sss.pgh.pa.us [1] https://postgr.es/m/20220714225735.GB3173833%40nathanxps13 -- Nathan Bossart Amazon Web Services:

predefined role(s) for VACUUM and ANALYZE

2022-07-22 Thread Nathan Bossart
93f4.camel%40j-davis.com [1] https://postgr.es/m/20211104224636.5qg6cfyjkw52r...@alap3.anarazel.de -- Nathan Bossart Amazon Web Services: https://aws.amazon.com commit 0b1397397c0f490525d3a15f5e9d5eb8f6023aa9 Author: Nathan Bossart Date: Fri Jul 22 12:21:16 2022 -0700 introduce pg_vacuum_ana

Re: privileges for ALTER ROLE/DATABASE SET

2022-07-22 Thread Nathan Bossart
another role. And I agree that requiring extra permissions for this feels excessive. Maybe someone else has a use-case in mind, though. I figured it would be good to hash this out prior to 15.0, at which point changing the behavior would become substantially more difficult. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-22 Thread Nathan Bossart
!IsBackgroundWorker && > + !am_walsender) > + process_session_preload_libraries(); I worry that this will be easily missed when adding new types of non-interactive sessions, but I can't claim to have a better idea. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-22 Thread Nathan Bossart
re-proof anyway? I hesitated to suggest this exactly because of the WAL sender problem, but it does seem slightly more future-proof, so +1 for this approach. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-24 Thread Nathan Bossart
== DestRemote && !am_walsender) would be a reasonably future-proof way to avoid the need for a new InitPostgres() argument. Otherwise, the patch looks good to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-24 Thread Nathan Bossart
8 currently during discussion. > > So that, subxid’s hash table will never be used, right? This array will store up to TOTAL_MAX_CACHED_SUBXIDS transactions, which will typically be much greater than 64. When there isn't any overflow, subxip stores all of the subxids for all of the entries

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-24 Thread Nathan Bossart
On Sun, Jul 24, 2022 at 11:49:23PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I noticed that a couple of places check whether whereToSendOutput is set to >> DestRemote to determine if this is an interactive session. > > IIRC, that would end in not loading the p

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-24 Thread Nathan Bossart
On Fri, Jul 22, 2022 at 01:37:35PM -0700, Nathan Bossart wrote: > The attached patch adds a pg_vacuum_analyze role that allows VACUUM and > ANALYZE commands on all relations. I started by trying to introduce > separate pg_vacuum and pg_analyze roles, but that quickly became > complic

Re: allow specifying action when standby encounters incompatible parameter settings

2022-07-25 Thread Nathan Bossart
On Mon, Jul 18, 2022 at 03:17:10PM -0700, Nathan Bossart wrote: > Given this feedback, I intend to mark the associated commitfest entry as > Withdrawn at the conclusion of the current commitfest. Done. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Pre-allocating WAL files

2022-07-25 Thread Nathan Bossart
On Thu, Jul 14, 2022 at 11:34:07AM -0700, Nathan Bossart wrote: > It's now been over a year since I first posted a patch in this thread, and > I still sense very little interest for this feature. I intend to mark it > as Withdrawn at the end of this commitfest. Done. -- Nathan

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-25 Thread Nathan Bossart
elp users might be to create additional aggregate predefined roles (like pg_monitor) for common combinations. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-25 Thread Nathan Bossart
On Sat, Jul 16, 2022 at 08:59:57PM -0700, Nathan Bossart wrote: > On Fri, Jul 15, 2022 at 01:08:57PM -0700, Andres Freund wrote: >> I wonder if we additionally / alternatively could use a faster method of >> searching the array linearly, e.g. using SIMD. > > I looked into u

Re: remove more archiving overhead

2022-07-26 Thread Nathan Bossart
On Tue, Jul 26, 2022 at 04:26:23PM +0900, Fujii Masao wrote: > Anyway, since the patches look good to me, I pushed them. Thanks a lot! > If necessary, we can keep improving the docs later. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

out of date comment in commit_ts.c

2022-07-26 Thread Nathan Bossart
. IIUC the ability for callers to request WAL record generation is no longer possible as of 08aa89b [0]. Should the second sentence be removed? [0] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=08aa89b -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: out of date comment in commit_ts.c

2022-07-27 Thread Nathan Bossart
On Tue, Jul 26, 2022 at 10:33:43AM -0700, Nathan Bossart wrote: > IIUC the ability for callers to request WAL record generation is no longer > possible as of 08aa89b [0]. Should the second sentence be removed? Here's a patch. -- Nathan Bossart Amazon Web Services: https://aws.amaz

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-28 Thread Nathan Bossart
On Tue, Jul 26, 2022 at 11:19:06AM -0700, Andres Freund wrote: > On 2022-07-25 12:04:19 -0700, Nathan Bossart wrote: >> From the discussion thus far, it seems there is interest in optimizing >> [sub]xip lookups, so I'd like to spend some time moving it forward. I >&

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-28 Thread Nathan Bossart
han 32 privilege bits. However, if we actually do have 2 bits remaining, we wouldn't need to do that work now unless someone else uses them first. That being said, it's certainly worth thinking about the future of this stuff. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [Patch] Fix bounds check in trim_array()

2022-07-28 Thread Nathan Bossart
ust be 0 for this case. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Oversight in slab.c SlabContextCreate(), initial memory allocation size is not populated to context->mem_allocated

2022-07-29 Thread Nathan Bossart
thrash malloc() when a context is repeatedly reset after small allocations, * which is typical behavior for per-tuple contexts. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Oversight in slab.c SlabContextCreate(), initial memory allocation size is not populated to context->mem_allocated

2022-07-29 Thread Nathan Bossart
On Fri, Jul 29, 2022 at 01:55:10PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Fri, Jul 29, 2022 at 12:43:45PM -0400, Reid Thompson wrote: >>> slab.c >>> does not in SlabContextCreate(). Is this intentional, it seems to be an >>> oversight to me. >

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-29 Thread Nathan Bossart
On Sat, Jul 30, 2022 at 12:02:02PM +0700, John Naylor wrote: > On Fri, Jul 29, 2022 at 4:34 AM Nathan Bossart > wrote: >> * I borrowed USE_SSE2 from one of John Naylor's patches [0]. I'm not > sure >>whether this is committable, > > I'll be the f

Re: allow building trusted languages without the untrusted versions

2022-08-01 Thread Nathan Bossart
On Mon, Aug 01, 2022 at 02:41:21PM -0700, Jacob Champion wrote: > On 7/13/22 12:49, Tom Lane wrote: >> Nathan Bossart writes: >>> Given the discussion in this thread, I intend to mark the commitfest entry >>> as Withdrawn shortly. > > I'll mark this RwF

Re: remove more archiving overhead

2022-08-01 Thread Nathan Bossart
> reason to return false, this paragraph could give the reason. I don't know of > such a reason, though. Your suggestion seems reasonable to me. I've attached a small patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/doc/src/sgml/backup.sgml b

Re: support for SSE2 intrinsics

2022-08-02 Thread Nathan Bossart
akes use of SSE2 instructions. That being said, I imagine it'll be especially important to keep an eye on the buildfarm when this change is committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-02 Thread Nathan Bossart
On Fri, Jul 29, 2022 at 10:38:11PM -0700, Nathan Bossart wrote: > On Sat, Jul 30, 2022 at 12:02:02PM +0700, John Naylor wrote: >> I'll be the first to say it's not committable and needs some thought. Since >> there are several recently proposed patches that take advantage

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-02 Thread Nathan Bossart
On Tue, Aug 02, 2022 at 03:55:39PM -0700, Andres Freund wrote: > FWIW, I'd split the introduction of the helper and the use of it in snapmgr > into separate patches. Will do. > On 2022-08-02 15:13:01 -0700, Nathan Bossart wrote: >> diff --git a/src/include/c.h b/src/

Re: Adding Support for Copy callback functionality on COPY TO api

2022-08-02 Thread Nathan Bossart
On Wed, Sep 30, 2020 at 04:41:51PM +0900, Michael Paquier wrote: > This feedback has not been answered after two weeks, so I have marked > the patch as returned with feedback. I've rebased this patch and will register it in the next commitfest shortly. -- Nathan Bossart Amazon W

Re: support for SSE2 intrinsics

2022-08-03 Thread Nathan Bossart
On Wed, Aug 03, 2022 at 12:00:39PM +0700, John Naylor wrote: > Thanks for checking! Here's a concrete patch for testing. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-03 Thread Nathan Bossart
/CAFBsxsGktHL7%3DJXbgnKTi_uL0VRPcH4FSAqc6yK-3%2BJYfqPPjA%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From bd523948876f801b7f1b909f399b2cc41acf06cf Mon Sep 17 00:00:00 2001 From: John Naylor Date: Wed, 3 Aug 2022 11:07:40 +0700 Subject: [PATCH v5 1/3] Support S

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-03 Thread Nathan Bossart
On Wed, Aug 03, 2022 at 11:06:58AM -0700, Andres Freund wrote: > On 2022-08-02 16:43:57 -0700, Nathan Bossart wrote: >> >> +#ifdef USE_SSE2 >> >> +pg_attribute_no_sanitize_alignment() >> >> +#endif >> > >> > What's the deal with this a

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-04 Thread Nathan Bossart
't matter -- it's just a hint to the reader. Will do. > Out of curiosity do we know how much we get by loading four registers > rather than two? The small program I've been using for testing takes about 40% more time with the two register approach. The majority of this te

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-05 Thread Nathan Bossart
y small testing program to perform many searches on small arrays, and I wasn't able to identify any impact, so perhaps thіs is good enough. Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 205737c56c7e49e8de25e6b4afca6a96abbb4e60 Mon Sep 17 00:00:00 2001 Fr

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-06 Thread Nathan Bossart
it seems to support the idea that the _mm_packs_epi* approach might have a slight edge. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-06 Thread Nathan Bossart
On Sat, Aug 06, 2022 at 11:13:26AM -0700, Nathan Bossart wrote: > On Fri, Aug 05, 2022 at 03:04:34PM -0700, Andres Freund wrote: >> But mainly I'd expect to find a difference if the SIMD code were optimized a >> further on the basis of not needing to return the offset.

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread Nathan Bossart
84 768 450 536 1024 350 494 > And also add the const > markers we discussed upthread? Oops, sorry about that. This is done in v9. > Aside from that, I plan to commit this > week unless there is further bikeshedding. Great, thanks. -- Nathan Bossart

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread Nathan Bossart
s were worthwhile. While substituting linear searches with pg_lfind32() is probably an improvement in most cases, I think we ought to demonstrate the benefits for each one. [0] https://postgr.es/m/20220802221301.GA742739%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: out of date comment in commit_ts.c

2022-08-08 Thread Nathan Bossart
On Tue, Aug 09, 2022 at 01:02:15PM +1200, Thomas Munro wrote: > Pushed. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread Nathan Bossart
ind.h. Fixed in v10. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From ed80cfaf146f82b930ea09b8efc062cc4088d4b6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 3 Aug 2022 09:49:04 -0700 Subject: [PATCH v10 1/2] Introduce optimized routine for linear searches through an arra

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-08 Thread Nathan Bossart
;t > removed at that moment. Also, I originally proposed to change > elog-ERROR to elog-LOG in CheckPointLogicalRewriteHeap for unlink() > failures for the same reason. This was my initial instinct as well, but this thread has received contradictory feedback during the months since. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread Nathan Bossart
ged on their own merit, anyway. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-09 Thread Nathan Bossart
place. Presumably this isn't the sort of thing that Postgres should be expected to support. > In other words, if we're going to LOG and press on, maybe we should > just press on anyway. Would the error message be any less > informative? Would the risk of unlinking a character d

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-09 Thread Nathan Bossart
is test coverage for pg_lfind32(), but I don't know if that syscache code is the right choice. For non-USE_SSE2 builds, it might make these lookups more expensive. I'll look around to see if there are any other suitable candidates. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-09 Thread Nathan Bossart
On Tue, Aug 09, 2022 at 01:00:37PM -0700, Nathan Bossart wrote: > Your adjustments in 0002 seem reasonable to me. I think it makes sense to > ensure there is test coverage for pg_lfind32(), but I don't know if that > syscache code is the right choice. For non-USE_SSE2 builds,

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-10 Thread Nathan Bossart
On Wed, Aug 10, 2022 at 10:50:02AM +0700, John Naylor wrote: > LGTM, let's see what the buildfarm thinks of 0001. Thanks! I haven't noticed any related buildfarm failures yet. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: use SSE2 for is_valid_ascii

2022-08-10 Thread Nathan Bossart
error_cum = _mm_or_si128(error_cum, + _mm_cmpeq_epi8(chunk, _mm_setzero_si128())); I wonder if reusing a zero vector (instead of creating a new one every time) has any noticeable effect on performance. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-10 Thread Nathan Bossart
On Thu, Aug 11, 2022 at 09:50:54AM +0700, John Naylor wrote: > I was waiting for all the Windows animals to report in, and it looks > like they have, so I've pushed 0002. Thanks for picking this topic up > again! Thanks for reviewing and committing. -- Nathan Bossart Amazon Web S

Re: use SSE2 for is_valid_ascii

2022-08-10 Thread Nathan Bossart
; OR data into accumulator por %xmm1, %xmm0; OR zero check results into accumulator cmpq%rdi, %rsi So the call to _mm_setzero_si128() within the loop is fine. Apologies for the noise. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Use SetInstallXLogFileSegmentActive() for setting XLogCtl->InstallXLogFileSegmentActive

2022-08-11 Thread Nathan Bossart
On Thu, Aug 11, 2022 at 09:42:18PM +0530, Bharath Rupireddy wrote: > Here's a small patch replacing the explicit setting of > XLogCtl->InstallXLogFileSegmentActive with the existing function > SetInstallXLogFileSegmentActive(), removes duplicate code and saves 4 > LOC. LGTM

Re: O(n) tasks cause lengthy startups and checkpoints

2022-08-11 Thread Nathan Bossart
On Wed, Jul 06, 2022 at 09:51:10AM -0700, Nathan Bossart wrote: > Here's a new revision where I've attempted to address all the feedback I've > received thus far. Notably, the custodian now uses a queue for registering > tasks and determining which tasks to execute

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-05 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 07:28:42PM +0200, Matthias van de Meent wrote: > As for "on my laptop", that sounds very reasonable, but could you > check the performance on systems with larger shared buffer > configurations? I'd imagine (but haven't checked) that binary upgrade > target systems may alread

Re: Optimizing COPY with SIMD

2024-06-05 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 01:46:44PM -0400, Neil Conway wrote: > We could go further and use the same code to handle both the tail of the > string in the vectorized case and the entire string in the non-vectorized > case, but I didn't bother with that -- as written, it would require taking > an unnec

Re: Better error message when --single is not the first arg to postgres executable

2024-06-05 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 02:51:05PM -0400, Greg Sabino Mullane wrote: > Please find attached a quick patch to prevent this particularly bad error > message for running "postgres", when making the common mistake of > forgetting to put the "--single" option first because you added an earlier > arg (es

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-05 Thread Nathan Bossart
On Mon, Jun 03, 2024 at 02:04:19PM -0500, Nathan Bossart wrote: > Of course, as soon as I committed this, I noticed another missing reference > to max_wal_senders in the paragraph about POSIX semaphores. I plan to > commit/back-patch the attached patch within the next couple days.

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-06 Thread Nathan Bossart
Here is a rebased version of the patch for v18 that adds a runtime-computed GUC. As I noted earlier, there still isn't a consensus on this approach. -- nathan >From 74f638f7df9c51f5ab47b282bb7107c4ba6cb5b6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 21 May 2024 14:02:

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-06 Thread Nathan Bossart
On Thu, Jun 06, 2024 at 03:31:53PM -0400, Robert Haas wrote: > I don't really like making this a GUC, but what's the other option? > It's reasonable for people to want to ask the server how many > resources it will need to start, and -C is the only tool we have for > that right now. So I feel like

Re: problems with "Shared Memory and Semaphores" section of docs

2024-06-09 Thread Nathan Bossart
On Thu, Jun 06, 2024 at 02:51:42PM -0500, Nathan Bossart wrote: > On Thu, Jun 06, 2024 at 03:31:53PM -0400, Robert Haas wrote: >> I do think the name could use some more thought, though. >> semaphores_required would end up being the same kind of thing as >> shared_memory_si

Re: Remove dependence on integer wrapping

2024-06-09 Thread Nathan Bossart
On Sun, Jun 09, 2024 at 04:59:22PM -0400, Joseph Koshakow wrote: > I originally added the helper functions to int.h thinking I'd find > many more instances of overflow due to integer negation, however I > didn't find that many. So let me know if you think we'd be better > off without the functions.

Re: CheckMyDatabase some error messages in two lines.

2024-06-09 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 08:00:00AM +0800, jian he wrote: > https://www.postgresql.org/docs/devel/error-style-guide.html#ERROR-STYLE-GUIDE-FORMATTING > "Don't end a message with a newline." > > > accidentally, I found some error messages in the function > CheckMyDatabase spread into two lines. > s

Re: Remove dependence on integer wrapping

2024-06-09 Thread Nathan Bossart
On Sun, Jun 09, 2024 at 09:57:54PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Sun, Jun 09, 2024 at 04:59:22PM -0400, Joseph Koshakow wrote: >>> The following comment was in the code for parsing timestamps: >>> /* check for just-barely overflow (ok

Re: Non-text mode for pg_dumpall

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 08:58:49AM -0400, Andrew Dunstan wrote: > Tom and Nathan opined recently that providing for non-text mode for > pg_dumpall would be a Good Thing (TM). Not having it has been a > long-standing complaint, so I've decided to give it a go. Thank you! > I think we would need to

Re: Non-text mode for pg_dumpall

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 10:51:42AM -0400, Andrew Dunstan wrote: > On 2024-06-10 Mo 10:14, Nathan Bossart wrote: >> I'm curious why we couldn't also support the "custom" format. > > We could, but the housekeeping would be a bit harder. We'd need to k

Re: Non-text mode for pg_dumpall

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 04:52:06PM +0200, Magnus Hagander wrote: > On Mon, Jun 10, 2024 at 4:14 PM Nathan Bossart > wrote: >> I'm curious why we couldn't also support the "custom" format. > > Or maybe even a combo - a directory of custom format files? Plus

Re: Track the amount of time waiting due to cost_delay

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 06:05:13AM +, Bertrand Drouvot wrote: > During the last pgconf.dev I attended Robert´s presentation about autovacuum > and > it made me remember of an idea I had some time ago: $SUBJECT This sounds like useful information to me. I wonder if we should also surface the

Re: Non-text mode for pg_dumpall

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 05:45:19PM +0200, Magnus Hagander wrote: > On Mon, Jun 10, 2024 at 5:03 PM Nathan Bossart > wrote: >> Is there a particular advantage to that approach as opposed to just using >> "directory" mode for everything? I know pg_upgrade uses &quo

Re: Track the amount of time waiting due to cost_delay

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 05:48:22PM +, Bertrand Drouvot wrote: > On Mon, Jun 10, 2024 at 10:36:42AM -0500, Nathan Bossart wrote: >> I wonder if we should also >> surface the effective cost limit for each autovacuum worker. > > I'm not sure about it as I think that i

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-10 Thread Nathan Bossart
On Fri, Jun 07, 2024 at 11:10:25AM +0200, Matthias van de Meent wrote: > My primary concern isn't the IO, but the O(shared_buffers) that we > have to go through during a checkpoint. As I mentioned upthread, it is > reasonably possible the new cluster is already setup with a good > fraction of the o

Re: Allow logical failover slots to wait on synchronous replication

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 03:51:05PM -0700, John H wrote: > The existing 'standby_slot_names' isn't great for users who are running > clusters with quorum-based synchronous replicas. For instance, if > the user has synchronous_standby_names = 'ANY 3 (A,B,C,D,E)' it's a > bit tedious to have to recon

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-11 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 10:39:51AM +0200, Matthias van de Meent wrote: > On Tue, 11 Jun 2024 at 04:01, Nathan Bossart wrote: >> I did a handful of benchmarks on an r5.24xlarge that seem to prove your >> point. The following are the durations of the pg_restore step of >> pg

Re: Remove dependence on integer wrapping

2024-06-11 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 09:31:39AM -0400, Joseph Koshakow wrote: > tmp is an uint16 here, it seems like you might have read it as an > int16? We would need some helper method like > > static inline bool > pg_neg_u16_overflow(uint16 a, int16 *result); > > and then we could replace

Re: Track the amount of time waiting due to cost_delay

2024-06-11 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 07:25:11AM +, Bertrand Drouvot wrote: > So I think that in v2 we could: 1) measure the actual wait time instead, 2) > count the number of times the vacuum slept. We could also 3) reports the > effective cost limit (as proposed by Nathan up-thread) (I think that 3) could

Re: Track the amount of time waiting due to cost_delay

2024-06-11 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 06:19:23PM +, Imseih (AWS), Sami wrote: >> I'm not convinced that reporting the number of waits is useful. If we >> were going to report a possibly-inaccurate amount of actual waiting, >> then also reporting the number of waits might make it easier to figure >> out when

Re: small pg_dump code cleanup

2024-06-11 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 01:58:54PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Wed, Jun 05, 2024 at 12:22:03PM -0400, Neil Conway wrote: >>> (2) These functions malloc() a single ntups * sizeof(struct) allocation and >>> then index into it to fill-in each str

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-12 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 10:39:51AM +0200, Matthias van de Meent wrote: > On Tue, 11 Jun 2024 at 04:01, Nathan Bossart wrote: >> It looks like pg_dump always uses template0, so AFAICT we don't even need >> the suggested manual checkpoint after restoring template1. > > T

Re: Remove dependence on integer wrapping

2024-06-12 Thread Nathan Bossart
On Tue, Jun 11, 2024 at 09:10:44PM -0400, Joseph Koshakow wrote: > The attached patch has updated this file too. For some reason I was > under the impression that I should leave the ecpg/ files alone, though > I can't remember why. Thanks. This looks pretty good to me after a skim, so I'll see ab

Re: Remove dependence on integer wrapping

2024-06-12 Thread Nathan Bossart
On Wed, Jun 12, 2024 at 11:45:20AM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Thanks. This looks pretty good to me after a skim, so I'll see about >> committing/back-patching it in the near future. IIUC there is likely more >> to come in this area, but I se

Re: Allow non-superuser to cancel superuser tasks.

2024-06-12 Thread Nathan Bossart
I adjusted 0001 based on my upthread feedback. -- nathan >From 118f95346fcf8099ab28d2f9186185171e3b88af Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 12 Jun 2024 15:38:14 -0500 Subject: [PATCH v5 1/1] Introduce pg_signal_autovacuum_worker. Since commit 3a9b18b309, roles w

Re: Harmonizing pg_bsd_indent parameter names

2024-06-12 Thread Nathan Bossart
On Wed, Jun 12, 2024 at 05:14:44PM -0400, Peter Geoghegan wrote: > I have been putting this off because I wasn't sure that the policy > should be the same for pg_bsd_indent. Is there any reason to think > that this will create more work down the line? It seems like it might, > due to some kind of n

Re: Changing default -march landscape

2024-06-12 Thread Nathan Bossart
On Thu, Jun 13, 2024 at 11:11:56AM +1200, Thomas Munro wrote: > David R and I were discussing vectorisation and microarchitectures and > what you can expect the target microarchitecture to be these days, and > it seemed like some of our choices are not really very > forward-looking. > > Distros ta

Re: Harmonizing pg_bsd_indent parameter names

2024-06-12 Thread Nathan Bossart
On Wed, Jun 12, 2024 at 05:59:14PM -0400, Peter Geoghegan wrote: > There is also one oddball case, not quite in either category. This > involves zic.c's declaration of > link(), when it should actually just be using the #include > declaration. That's another weird upstream code thing -- this isn'

Re: Changing default -march landscape

2024-06-12 Thread Nathan Bossart
On Thu, Jun 13, 2024 at 01:20:17PM +1200, Thomas Munro wrote: > The way I think about it, it's not our place to require anything (I > mean, we can't literally put -march=XXX into our build files, or if we > do the Debian et al maintainers will have to remove them by local > patch because they are i

improve predefined roles documentation

2024-06-13 Thread Nathan Bossart
IMHO there are a couple of opportunities for improving the predefined roles documentation [0]: * Several of the roles in the table do not have corresponding descriptions in the paragraphs below the table (e.g., pg_read_all_data, pg_write_all_data, pg_checkpoint, pg_maintain, pg_use_reserved_

Re: improve predefined roles documentation

2024-06-13 Thread Nathan Bossart
On Thu, Jun 13, 2024 at 01:05:33PM -0700, David G. Johnston wrote: > One of the main attributes for the GUCs is their category. If we want to > improve organization we'd need to assign categories first. We already > implicitly do so in the description section where we do group them together > and

Re: Allow non-superuser to cancel superuser tasks.

2024-06-14 Thread Nathan Bossart
On Fri, Jun 14, 2024 at 12:06:36PM +0500, Andrey M. Borodin wrote: > This patch looks good to me. Thanks for looking. > Spellchecker is complaining about "signaling" instead of "signalling", > but ISTM it´s OK. I think this is an en-US versus en-GB thing. We've standardized on en-US for "cancel

Re: use CREATE DATABASE STRATEGY = FILE_COPY in pg_upgrade

2024-06-14 Thread Nathan Bossart
On Wed, Jun 12, 2024 at 09:41:01AM -0500, Nathan Bossart wrote: > Actually, I think you are right that we need a manual checkpoint, except I > think we need it to be after prepare_new_globals(). set_frozenxids() > connects to each database (including template0) and updates a bunch of &

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-06-17 Thread Nathan Bossart
On Mon, Jun 17, 2024 at 05:55:04PM +0530, Bharath Rupireddy wrote: > Here are my thoughts on when to do the XID age invalidation. In all > the patches sent so far, the XID age invalidation happens in two > places - one during the slot acquisition, and another during the > checkpoint. As the suggest

Re: Refactoring backend fork+exec code

2024-06-17 Thread Nathan Bossart
While looking into [0], I noticed that main() still only checks for the --fork prefix, but IIUC commit aafc05d removed all --fork* options except for --forkchild. I've attached a patch to strengthen the check in main(). This is definitely just a nitpick. [0] https://postgr.es/m/CAKAnmmJkZtZAiSry

Re: Better error message when --single is not the first arg to postgres executable

2024-06-17 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 11:38:48PM -0400, Greg Sabino Mullane wrote: > On Wed, Jun 5, 2024 at 3:18 PM Nathan Bossart > wrote: >> Could we remove the requirement that --single must be first? I'm not >> thrilled about adding a list of "must be first" options t

Re: New GUC autovacuum_max_threshold ?

2024-06-17 Thread Nathan Bossart
I didn't see a commitfest entry for this, so I created one to make sure we don't lose track of this: https://commitfest.postgresql.org/48/5046/ -- nathan

Re: improve predefined roles documentation

2024-06-18 Thread Nathan Bossart
On Mon, Jun 17, 2024 at 02:10:22PM -0400, Robert Haas wrote: > On Thu, Jun 13, 2024 at 3:48 PM Nathan Bossart > wrote: >> I think we could improve matters by abandoning the table and instead >> documenting these roles more like we document GUCs, i.e., each one has a >>

Re: cost delay brainstorming

2024-06-18 Thread Nathan Bossart
On Mon, Jun 17, 2024 at 03:39:27PM -0400, Robert Haas wrote: > I think we might able to get fairly far by observing that if the > number of running autovacuum workers is equal to the maximum allowable > number of running autovacuum workers, that may be a sign of trouble, > and the longer that situa

Re: allow changing autovacuum_max_workers without restarting

2024-06-18 Thread Nathan Bossart
On Mon, Jun 03, 2024 at 04:24:27PM -0700, Andres Freund wrote: > On 2024-06-03 14:28:13 -0500, Nathan Bossart wrote: >> On Mon, Jun 03, 2024 at 12:08:52PM -0700, Andres Freund wrote: >> > Why do we think that increasing the number of PGPROC slots, heavyweight >> >

fix pg_upgrade comment

2024-06-18 Thread Nathan Bossart
This comment was first added when pg_upgrade was introduced (commit c2e9b2f288), but I imagine it predates even that. I've attached a patch to fix this. Barring objections, I'll probably commit this soon. -- nathan >From 379793a07dd98f65466452806756bfa0f6312bdf Mon Sep 17 00:00:00

Re: allow changing autovacuum_max_workers without restarting

2024-06-18 Thread Nathan Bossart
On Tue, Jun 18, 2024 at 01:43:34PM -0700, Andres Freund wrote: > I just don't see much point in reserving 256 worker "possibilities", tbh. I > can't think of any practical system where it makes sense to use this much (nor > do I think it's going to be reasonable in the next 10 years) and it's just

Re: cost delay brainstorming

2024-06-18 Thread Nathan Bossart
On Tue, Jun 18, 2024 at 01:32:38PM -0700, Andres Freund wrote: > On 2024-06-18 13:50:46 -0500, Nathan Bossart wrote: >> Have we ruled out further adjustments to the cost parameters as a first >> step? > > I'm not against that, but I it doesn't address the issue th

Re: allow changing autovacuum_max_workers without restarting

2024-06-18 Thread Nathan Bossart
On Tue, Jun 18, 2024 at 02:33:31PM -0700, Andres Freund wrote: > Another one: > > Have a general cap of 64, but additionally limit it to something like > max(1, min(WORKER_CAP, max_connections / 4)) > > so that cases like tap tests don't end up allocating vastly more worker slots > than actu

<    10   11   12   13   14   15   16   17   18   19   >