Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-11 Thread Nathan Bossart
Here is a rebased patch set. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 506aa95dd77f16dc64d7fe9c52ca67dd3c10212e Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Jan 2022 19:24:22 + Subject: [PATCH v4 1/8] Introduce custodian. The custodian process i

Re: Fix overflow in justify_interval related functions

2022-02-14 Thread Nathan Bossart
hould we instead rework the logic to avoid overflow for this case? [0] https://www.postgresql.org/docs/devel/datatype-datetime.html -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fix overflow in justify_interval related functions

2022-02-14 Thread Nathan Bossart
On Mon, Feb 14, 2022 at 01:55:56PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> On Sun, Feb 13, 2022 at 01:28:38PM -0500, Joseph Koshakow wrote: >>> +SELECT justify_hours(interval '2147483647 days 24 hrs'); >>> +ERROR: interval out of range > >

Re: Fix overflow in justify_interval related functions

2022-02-14 Thread Nathan Bossart
On Mon, Feb 14, 2022 at 04:57:07PM -0500, Joseph Koshakow wrote: > On Mon, Feb 14, 2022 at 2:15 PM Nathan Bossart > wrote: >> Makes sense. So we could likely avoid it for justify_interval, but the >> others are at the mercy of the interval implementation. > > I'

Re: Fix overflow in justify_interval related functions

2022-02-14 Thread Nathan Bossart
dinarily an extra space before errmsg() so that it lines up with errcode(). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-02-15 Thread Nathan Bossart
>> Also, convert "could not parse filename" and "could not remove file" >> errors to LOG messages in CheckPointLogicalRewriteHeap. This will >> enable checkpoint not to waste the amount of work that it had done. > > I still doubt this is a go

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-02-15 Thread Nathan Bossart
gt; It'd probably be good to add a reference to get_dirent_type() to > ReadDir[Extended]()'s docs. Agreed. I might give this a try. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-02-15 Thread Nathan Bossart
On Tue, Feb 15, 2022 at 10:37:58AM -0800, Nathan Bossart wrote: > On Tue, Feb 15, 2022 at 10:10:34AM -0800, Andres Freund wrote: >> I generally think it'd be a good exercise to go through an use >> get_dirent_type() in nearly all ReadDir() style loops - it's a nice >&g

USE_BARRIER_SMGRRELEASE on Linux?

2022-02-15 Thread Nathan Bossart
9024/log/src/test/regress/regression.diffs [2] https://postgr.es/m/CA%2BhUKGJp-m8uAD_wS7%2BdkTgif013SNBSoJujWxvRUzZ1nkoUyA%40mail.gmail.com [3] https://postgr.es/m/CA%2BhUKGLdemy2gBm80kz20GTe6hNVwoErE8KwcJk6-U56oStjtg%40mail.gmail.com [4] https://git.postgresql.org/gitweb/?p=postgresql.git;a=co

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-15 Thread Nathan Bossart
'm assuming it's not the root cause. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-15 Thread Nathan Bossart
On Tue, Feb 15, 2022 at 09:06:45PM -0800, Andres Freund wrote: > On 2022-02-15 20:59:11 -0800, Nathan Bossart wrote: >> On Wed, Feb 16, 2022 at 03:34:08PM +1300, Thomas Munro wrote: >> > So it's not getting unlinked until the *next* checkpoint cycle. I >> > don&

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-15 Thread Nathan Bossart
On Tue, Feb 15, 2022 at 10:14:04PM -0800, Nathan Bossart wrote: > It looks like register_unlink_segment() is called prior to the checkpoint, > but the checkpointer is not calling RememberSyncRequest() until after > SyncPreCheckpoint(). This means that the requests are registered with t

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-16 Thread Nathan Bossart
On Tue, Feb 15, 2022 at 10:57:32PM -0800, Nathan Bossart wrote: > On Tue, Feb 15, 2022 at 10:14:04PM -0800, Nathan Bossart wrote: >> It looks like register_unlink_segment() is called prior to the checkpoint, >> but the checkpointer is not calling RememberSyncReques

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-16 Thread Nathan Bossart
On Wed, Feb 16, 2022 at 08:44:42AM -0800, Nathan Bossart wrote: > On Tue, Feb 15, 2022 at 10:57:32PM -0800, Nathan Bossart wrote: >> On Tue, Feb 15, 2022 at 10:14:04PM -0800, Nathan Bossart wrote: >>> It looks like register_unlink_segment() is called prior to the check

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-16 Thread Nathan Bossart
v1 [0] does. Then the extra absorb would be sufficient for single-user mode if the requests were not absorbed immediately. [0] https://www.postgresql.org/message-id/attachment/130994/v1-0001-call-AbsorbSyncRequests-before-advancing-checkpoi.patch -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-16 Thread Nathan Bossart
Here is another rebased patch set. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From c11a6893d2d509df1389a1c03081b6cc563d5683 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Jan 2022 19:24:22 + Subject: [PATCH v5 1/8] Introduce custodian. The custodian proc

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-16 Thread Nathan Bossart
Here is a rebased patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 8fc744a0b22a1fff9f3470bfdc0e08d9dd5da5be Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 1 Dec 2021 23:50:49 + Subject: [PATCH v2 1/1] remove exclusive backup mode --- doc/src/s

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-16 Thread Nathan Bossart
ies for removal, wake up the >> + * custodian process so that it deletes all the files in the staged >> + * directories as well as the directories themselves. >> + */ >> +if (stage && ProcGlobal->custodianLatch) >> +SetLatch(ProcG

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: > On 2022-02-16 20:14:04 -0800, Nathan Bossart wrote: >> >> - while ((spc_de = ReadDirExtended(spc_dir, "pg_tblspc", LOG)) != NULL) >> >> + while (!ShutdownRequestPending && >>

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
On Thu, Feb 17, 2022 at 11:27:09AM -0800, Andres Freund wrote: > On 2022-02-17 10:23:37 -0800, Nathan Bossart wrote: >> On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: >> > They're accessed by xid. The LSN is just for cleanup. Accessing files >>

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
good place to start. As I mentioned above, IME the temporary file cleanup is the most common problem, so I think even getting that one fixed would be a huge improvement. [0] https://postgr.es/m/CALj2ACXkkSL8EBpR7m%3DMt%3DyRGBhevcCs3x4fsp3Bc-D13yyHOg%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-18 Thread Nathan Bossart
wing connections to clean up temporary files. 3. Do small amounts of temporary file cleanup whenever there is an opportunity during recovery. 4. Wait until recovery completes before cleaning up temporary files. I'm not too thrilled about any of these options. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-02-18 Thread Nathan Bossart
On Wed, Feb 16, 2022 at 01:00:53PM -0800, Nathan Bossart wrote: > On Wed, Feb 16, 2022 at 11:27:31AM -0800, Andres Freund wrote: >> That doesn't strike me as great architecturally. E.g. in theory the same >> problem could exist in single user mode. I think it do

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-18 Thread Nathan Bossart
use the approach of renaming the directory out of the >>> > way as >>> > done in this patchset but perform the cleanup in the startup process after >>> > we're up. BTW I know you don't like the dedicated process approach, but one improvement to that appr

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-18 Thread Nathan Bossart
On Fri, Feb 18, 2022 at 10:48:10PM +0530, Ashutosh Sharma wrote: > Some review comments on the latest version: Thanks for the feedback! Before I start spending more time on this one, I should probably ask if this has any chance of making it into v15. -- Nathan Bossart Amazon Web Servi

Re: archive modules

2022-11-05 Thread Nathan Bossart
On Fri, Nov 04, 2022 at 12:05:26PM +0900, Ian Lawrence Barwick wrote: > cfbot reports the patch no longer applies [1]. As CommitFest 2022-11 is > currently underway, this would be an excellent time to update the patch. Indeed. Here is a new version of the patch. -- Nathan Bossart Amaz

Re: archive modules

2022-11-05 Thread Nathan Bossart
er to treat modules and commands as two separate interfaces to ease migration from older major versions (even though archive_command is now essentially a built-in archive module). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: thinko in basic_archive.c

2022-11-05 Thread Nathan Bossart
e cautious to only remove the temporary file if we know we created it. I still feel that trying to add this cleanup logic to basic_archive is probably more trouble than it's worth, but if there is a safe and effective way to do so, I won't object. [0] https://postgr.es/m/20221015211026.

Re: Suppressing useless wakeups in walreceiver

2022-11-05 Thread Nathan Bossart
TIMEOUT flag and set timeout to -1 whenever "nap" is calculated to be > INT_MAX, but I didn't think it was worth complicating the code in order to save an extra wakeup every ~25 days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-05 Thread Nathan Bossart
to demonstrate any meaningful difference between the two approaches. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-05 Thread Nathan Bossart
On Sat, Nov 05, 2022 at 03:00:55PM -0700, Nathan Bossart wrote: > On Mon, Oct 17, 2022 at 03:21:18PM +0900, Kyotaro Horiguchi wrote: >> Now that I see the fix for the implicit conversion: >> >> L527: + nap = Max(0, (nextWakeup - now + 999) / &g

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-06 Thread Nathan Bossart
On Fri, Sep 23, 2022 at 10:41:54AM -0700, Nathan Bossart wrote: > v11 adds support for building with meson. rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 367c5f3863457cfbd0fe8add0e8df3e630aaaea9 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Ja

Re: archive modules

2022-11-07 Thread Nathan Bossart
On Mon, Nov 07, 2022 at 03:20:31PM +0900, Michael Paquier wrote: > On Sat, Nov 05, 2022 at 02:08:58PM -0700, Nathan Bossart wrote: >> Perhaps we could eventually move the archive_command functionality to a >> contrib module (i.e., "shell_archive") so that users must alwa

Re: thinko in basic_archive.c

2022-11-07 Thread Nathan Bossart
ftover temporary file) if the ERROR or shutdown occurs after creating the file in copy_file() and before renaming the temporary file to its final destination. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-08 Thread Nathan Bossart
ide what the scope was for moving stuff into a struct (ie various >> pre-existing stuff representing the walreceiver's state), but I'm OK >> with considering that as a pure refactoring project for a separate >> thread. That array should be "static", though. >

Re: Suppressing useless wakeups in walreceiver

2022-11-08 Thread Nathan Bossart
h, I think so. At least for its idle/nap case (waiting for flush > is also a technically interesting case, but harder, and applies to > non-idle systems so the polling is a little less offensive). Bharath, are you planning to pick this up? If not, I can probably spend some time on it.

Re: Suppressing useless wakeups in walreceiver

2022-11-13 Thread Nathan Bossart
syncing_tables() isn't called as often. It wouldn't surprise me if there's also something in walreceiver.c that depends upon the frequent wakeups. I suspect this will require a revert. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-13 Thread Nathan Bossart
all of the code paths that write/flush are careful to send a reply shortly afterward, which should keep writePtr/flushPtr updated. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
the initial reply? ISTM the previous behavior essentially caused the first reply (and feedback message) to be sent at the first opportunity after streaming begins. My instinct is to do something like the attached. I wonder if we ought to do something similar in the ConfigReloadPending path in c

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
wouldn't rule out scheduling luck. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-14 Thread Nathan Bossart
but I do > think that this patch is ready and adds something concrete to the ongoing > discussion. Thanks for taking a look! Here is a rebased version of the patch set. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From eb6d4f8bb2bb055ec161767bbec741e375314dcb Mon Sep

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
Another option might be to just force initial reply/feedback messages when streaming starts. The attached patch improves src/test/recovery test runtime just like the previous one I posted. On Mon, Nov 14, 2022 at 11:01:27AM -0800, Nathan Bossart wrote: > I wonder if we > ought to do som

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-14 Thread Nathan Bossart
On Mon, Nov 14, 2022 at 03:40:04PM -0800, Nathan Bossart wrote: > Thanks for taking a look! Here is a rebased version of the patch set. Oops, apparently object_aclcheck() cannot be used for pg_class. Here is another version that uses pg_class_aclcheck() instead. I'm not sure how I mis

Re: archive modules

2022-11-15 Thread Nathan Bossart
#x27;m content to just leave it be in PG15. Sounds good to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-11-15 Thread Nathan Bossart
On Tue, Nov 15, 2022 at 06:14:25PM +0100, Alvaro Herrera wrote: > On 2022-Nov-15, Nathan Bossart wrote: > >> On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > >> > The surrounding code has changed a bit between PG15 and master, so if we >> > wa

Re: allowing for control over SET ROLE

2022-11-15 Thread Nathan Bossart
t the approach, so I won't balk if you proceed with this change. It might be worth starting a new thread to discuss whether to treat predefined roles as a special case, but IMO that needn't hold up this patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Suppressing useless wakeups in walreceiver

2022-11-15 Thread Nathan Bossart
On Wed, Nov 16, 2022 at 04:57:08PM +1300, Thomas Munro wrote: > On Tue, Nov 15, 2022 at 5:49 PM Nathan Bossart > wrote: >> Another option might be to just force initial reply/feedback messages when >> streaming starts. The attached patch improves src/test/recovery test >&g

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-16 Thread Nathan Bossart
y remaining concerns. > If so I'm intending to > commit at least the first two of these patches fairly soon. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-18 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 0643a4dbc9f36d9fd383ef0cfebef13875237718 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 Sep 2022 22:25:29 -0700 Subject: [PATCH v10 1/4] Change AclMode from a uint32 to a uint64. --- src/backend/no

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-19 Thread Nathan Bossart
On Fri, Nov 18, 2022 at 09:05:04AM -0800, Nathan Bossart wrote: > rebased another rebase -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6e2001411107790991037e91f8d2f9411e2f4fa6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 Sep 2022 22:25:29 -0700 S

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-20 Thread Nathan Bossart
On Sat, Nov 19, 2022 at 10:50:04AM -0800, Nathan Bossart wrote: > another rebase Another rebase for cfbot. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 5f716f33e93187491686381b2180894ab2b1b92c Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 Sep 2022 22

Re: Reducing power consumption on idle servers

2022-11-20 Thread Nathan Bossart
ear future. [0] https://postgr.es/m/20221113222644.GA1269110%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 80452ebbadfe80f237ebf5c7cdf8fa5eb2a61e35 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Nov 2022 16:01:01 -0800 Subject: [PATCH v1 1/1] wake up logical workers after ALTER SUBSCRIPTION --- src/backend/a

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
o a single variable might not be enough. Perhaps we can put a list in TopTransactionContext. I'll do some more investigation and report back. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
e bulk of the logic to logical/worker.c. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 57bbd9e8d9da72c50d9a41704f3f42b37aba33ab Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Nov 2022 16:01:01 -0800 Subject: [PATCH v2 1/1] wake up logical workers after ALTER SUB

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-23 Thread Nathan Bossart
subid >> will >> be recorded twice. >> I'm not sure whether it may be caused some issued, but list_member_oid() can >> be used to avoid that. > > +1, list_append_unique_oid might be better. Done in v3. -- Nathan Bossart Amazon Web Services: https://aws.amazon

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-23 Thread Nathan Bossart
On Tue, Nov 22, 2022 at 04:59:28PM +0530, Amit Kapila wrote: > On Tue, Nov 22, 2022 at 6:11 AM Nathan Bossart > wrote: >> While working on avoiding unnecessary wakeups in logical/worker.c (as was >> done for walreceiver.c in 05a7be9), I noticed that the tests began taking >

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-23 Thread Nathan Bossart
case I think this comment would be better English with "might" > instead of "may": > > /* user may have the ANALYZE privilege */ I've attempted to address all your feedback in v13. Please let me know if anything needs further reworking. --

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-23 Thread Nathan Bossart
On Sun, Nov 06, 2022 at 02:38:42PM -0800, Nathan Bossart wrote: > rebased another rebase for cfbot -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From b2c36a6d0d8ca5cde374b1c8b34aafaabbd7f6c2 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Jan 2022 19:24:22

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-27 Thread Nathan Bossart
the others in a later CF/separate thread? That seems like a good idea since those are all relatively self-contained. I removed 0002-0004 in v14. [0] https://postgr.es/m/20220217065938.x2esfdppzypegn5j%40alap3.anarazel.de -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 4

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-27 Thread Nathan Bossart
On Thu, Nov 24, 2022 at 05:26:27AM +, Hayato Kuroda (Fujitsu) wrote: > I have no comments for the v3 patch. Thanks for reviewing! Does anyone else have thoughts on the patch? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-28 Thread Nathan Bossart
On Mon, Nov 28, 2022 at 12:13:13PM -0500, Andrew Dunstan wrote: > pushed. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Nathan Bossart
Nov 28, 2022 at 1:31 PM Andres Freund wrote: >> On 2022-11-28 13:08:57 +, Simon Riggs wrote: >> > On Sun, 27 Nov 2022 at 23:34, Nathan Bossart >> > wrote: >> > > > Rather than explicitly use DEBUG1 everywhere I would have an >> > > > #defin

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
odule. Of course. I found some existing tests for the test_decoding plugin that appear to reliably generate the files we want the custodian to clean up, so I added them there. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From d8342d121d39d04e995986b4244abf369b833730 Mon Sep 1

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
se changes are worthwhile. I think there's a good chance it would become a game of whac-a-mole. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
On Wed, Nov 30, 2022 at 05:27:40PM +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 5:23 PM Thomas Munro wrote: >> On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart >> wrote: >> > I spent some more time on the prevent-unnecessary-wakeups patch for >> > logical

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 07:56:53PM -0800, Nathan Bossart wrote: > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: >> The last important point for me is tests, in src/test/modules >> probably. It might be possible to reuse the final state of other >> modules'

Re: O(n) tasks cause lengthy startups and checkpoints

2022-12-01 Thread Nathan Bossart
dback. I intend to pick that up in a new thread. I haven't thought too much about the others yet. > Another comment: > IIUC, there's no custodian_delay GUC as we want to avoid unnecessary > wakeups for power savings (being discussed in the other thread). > However, can it h

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-01 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 09:04:41PM -0800, Nathan Bossart wrote: > I don't mind fixing it! There are a couple more I'd like to track down > before posting another revision. Okay, here is a new version of the patch. This seems to clear up everything that I could find via the t

Re: O(n) tasks cause lengthy startups and checkpoints

2022-12-02 Thread Nathan Bossart
On Fri, Dec 02, 2022 at 12:11:35PM +0530, Bharath Rupireddy wrote: > On Fri, Dec 2, 2022 at 3:10 AM Nathan Bossart > wrote: >> The test appears to reliably create snapshot and mapping files, so if the >> directories are empty at some point after the checkpoint at the end, we ca

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-02 Thread Nathan Bossart
On Thu, Dec 01, 2022 at 04:21:30PM -0800, Nathan Bossart wrote: > Okay, here is a new version of the patch. This seems to clear up > everything that I could find via the tests. I cleaned up the patch a bit. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

suppressing useless wakeups in logical/worker.c

2022-12-02 Thread Nathan Bossart
ached patch applied, those periodic wakeups are gone, so we need to make sure we wake up the logical workers as needed. [0] https://postgr.es/m/CA%2BhUKGJGhX4r2LPUE3Oy9BX71Eum6PBcS8L3sJpScR9oKaTVaA%40mail.gmail.com [1] https://postgr.es/m/20221122004119.GA132961%40nathanxps13 -- Nathan Bossart A

Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

2022-12-02 Thread Nathan Bossart
oducing memory ordering bugs. Perhaps I am missing something obvious. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Generate pg_stat_get_* functions with Macros

2022-12-02 Thread Nathan Bossart
les_returned */ > +PGSTAT_DEFINE_REL_INT64_FIELD_ACCESSOR(pg_stat_get_, tuples_returned); > + > +/* pg_stat_get_tuples_fetched */ > +PGSTAT_DEFINE_REL_INT64_FIELD_ACCESSOR(pg_stat_get_, tuples_fetched); Can we hard-code the prefix in the macro? It looks like all of these use the same one. -- N

Re: Generate pg_stat_get_* functions with Macros

2022-12-03 Thread Nathan Bossart
On Sat, Dec 03, 2022 at 10:31:19AM +0100, Drouvot, Bertrand wrote: > On 12/3/22 1:51 AM, Nathan Bossart wrote: >> Can we hard-code the prefix in the macro? It looks like all of these use >> the same one. > > Good point! Done in V2 attached. Thanks. I editorialized a b

Re: Generate pg_stat_get_* functions with Macros

2022-12-04 Thread Nathan Bossart
On Sun, Dec 04, 2022 at 06:07:37AM +0100, Drouvot, Bertrand wrote: > On 12/3/22 9:16 PM, Nathan Bossart wrote: >> Thanks. I editorialized a bit in the attached v3. I'm not sure that my >> proposed names for the macros are actually an improvement. WDYT? > > Than

Re: suppressing useless wakeups in logical/worker.c

2022-12-05 Thread Nathan Bossart
anted to fix this, we'd probably need to recompute the wakeup times using the values currently set. I haven't looked into this too closely, but it doesn't sound tremendously difficult. Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-12-05 Thread Nathan Bossart
On Mon, Dec 05, 2022 at 11:21:08PM +0300, Pavel Luzanov wrote: > But perhaps this behavior should be reviewed or at least documented? I wonder why \dpS wasn't added. I wrote up a patch to add it and the corresponding documentation that other meta-commands already have. -- Nathan Bossar

Re: Generate pg_stat_get_* functions with Macros

2022-12-05 Thread Nathan Bossart
47 deletions(-) > > Like the previous one, the functions have the same names and the field > names are updated to fit in the picture. Thoughts? I might alphabetize the functions, but otherwise it looks good to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-06 Thread Nathan Bossart
ded a check that two_phase mode is pending. Even so, there can still be unnecessary wakeups, but this adjustment should limit them. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 4be842c8a857a13b0bec2ebcbd9f8addf8b8562a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mo

add \dpS to psql

2022-12-06 Thread Nathan Bossart
-e465-85b2-9d8e-f9ed1a5a66e9%40postgrespro.ru -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index d3dd638b14..406936dd1c 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++ b/doc/src/sgml/ref/psql-ref.sgml @

Re: predefined role(s) for VACUUM and ANALYZE

2022-12-06 Thread Nathan Bossart
On Tue, Dec 06, 2022 at 04:57:37PM +0300, Pavel Luzanov wrote: > On 06.12.2022 03:04, Nathan Bossart wrote: >> I wonder why \dpS wasn't added. I wrote up a patch to add it and the >> corresponding documentation that other meta-commands already have. > > Yes, \dpS comman

Re: predefined role(s) for VACUUM and ANALYZE

2022-12-06 Thread Nathan Bossart
w, so maybe it's still worth adding a fast path. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-06 Thread Nathan Bossart
On Tue, Dec 06, 2022 at 11:25:51AM -0800, Nathan Bossart wrote: > On Tue, Dec 06, 2022 at 07:44:46PM +0300, Melih Mutlu wrote: >> - When the state is SYNCDONE and the apply worker has to wake up to change >> the state to READY. >> >> I think we already call logicalrep

remove extra space from dumped ALTER DEFAULT PRIVILEGES commands

2022-12-06 Thread Nathan Bossart
Hi hackers, pg_dump adds an extra space in ALTER DEFAULT PRIVILEGES commands. AFAICT it's been this way since the command was first introduced (249724c). I've attached a patch to fix it. This is admittedly just a pet peeve, but maybe it is bothering others, too. -- Nathan Bossart

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-07 Thread Nathan Bossart
re ready for two_phase mode. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 2457eca47c92b29409d5972fc5a3e1c8c038fa6d Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Nov 2022 16:01:01 -0800 Subject: [PATCH v8 1/1] wake up logical workers as needed instead of rel

fix and document CLUSTER privileges

2022-12-07 Thread Nathan Bossart
ched patch adds such documentation. [0] https://postgr.es/m/20220411140609.GF26620%40telsasoft.com [1] https://postgr.es/m/661148f4-c7f1-dec1-2bc8-29f3bd58e242%40postgrespro.ru -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/re

Re: add \dpS to psql

2022-12-07 Thread Nathan Bossart
be the table's owner or a > superuser." I created a new thread for this: https://postgr.es/m/20221207223924.GA4182184%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: fix and document CLUSTER privileges

2022-12-07 Thread Nathan Bossart
but I still think it's worthwhile to make the checks consistent. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add \dpS to psql

2022-12-07 Thread Nathan Bossart
t; Disclaimer: I have not been following these threads. I haven't formed an opinion on whether VACUUM FULL should get its own bit, but FWIW І just finished writing the first draft of a patch set to add bits for CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX. I plan to post that tomorrow.

Re: add \dpS to psql

2022-12-07 Thread Nathan Bossart
t deserve them. Perhaps I should take inventory and offer a plan for all the remaining privileges. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add \dpS to psql

2022-12-07 Thread Nathan Bossart
eriously > discussed on this list, and still leave 3 unused bits for future expansion. If we added CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX as individual privilege bits, we'd still have 13 remaining for future use. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add \dpS to psql

2022-12-08 Thread Nathan Bossart
On Thu, Dec 08, 2022 at 12:15:23AM -0500, Tom Lane wrote: > Nathan Bossart writes: >> On Wed, Dec 07, 2022 at 11:48:20PM -0500, Isaac Morland wrote: >>> My previous analysis >>> shows that there is no vast hidden demand for new privilege bits. If we >>> imp

Re: fix and document CLUSTER privileges

2022-12-08 Thread Nathan Bossart
ake sense to change all three of the aforementioned commands to look more like VACUUM/ANALYZE. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-08 Thread Nathan Bossart
s://postgr.es/m/20221206193606.GB3078082%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From a16e59770da430e43bc85c244705a239ddd03c7b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 Dec 2022 11:20:01 -0800 Subject: [PATCH v1 1/2] add grantable privileges fo

Re: add \dpS to psql

2022-12-08 Thread Nathan Bossart
I've created a new thread for making CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX grantable: https://postgr.es/m/20221208183707.GA55474%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Documenting MERGE INTO ONLY ...

2022-12-08 Thread Nathan Bossart
TM. I didn't see any tests for this in merge.sql or inherit.sql. Do you think we should add some? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add \dpS to psql

2022-12-09 Thread Nathan Bossart
On Fri, Dec 09, 2022 at 01:36:03PM +0900, Michael Paquier wrote: > On Thu, Dec 08, 2022 at 09:15:03AM -0800, Nathan Bossart wrote: >> The main idea behind this work is breaking out privileges into more >> granular pieces. If I want to create a role that only runs VACUUM on some &

Re: add \dpS to psql

2022-12-09 Thread Nathan Bossart
On Fri, Dec 09, 2022 at 10:40:55AM -0800, Nathan Bossart wrote: > On Fri, Dec 09, 2022 at 01:36:03PM +0900, Michael Paquier wrote: >> On Thu, Dec 08, 2022 at 09:15:03AM -0800, Nathan Bossart wrote: >>> The main idea behind this work is breaking out privileges into more >>

Re: Generate pg_stat_get_* functions with Macros

2022-12-09 Thread Nathan Bossart
ng as said line is blank ... but I think it's a foot-gun > waiting to bite somebody, because visually you'd think the macro > ends with the brace. So I'd leave off that last backslash. Indeed. Patch attached. -- Nathan Bossart Amazon Web Services: https://aws.amazon.co

<    6   7   8   9   10   11   12   13   14   15   >