Re: core dumps in auto_prewarm, tests succeed

2024-01-22 Thread Nathan Bossart
On Mon, Jan 22, 2024 at 03:38:15PM -0600, Nathan Bossart wrote: > On Mon, Jan 22, 2024 at 01:24:54PM -0800, Andres Freund wrote: >> On 2024-01-22 15:19:36 -0600, Nathan Bossart wrote: >>> I think this is because the autoprewarm state was moved to a DSM segment, >>> an

Re: introduce dynamic shared memory registry

2024-01-22 Thread Nathan Bossart
On Mon, Jan 22, 2024 at 05:00:48PM +0530, Bharath Rupireddy wrote: > On Mon, Jan 22, 2024 at 3:43 AM Nathan Bossart > wrote: >> Oops. I've attached an attempt at fixing this. I took the opportunity to >> clean up the surrounding code a bit. > > The code looks

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Nathan Bossart
On Mon, Jan 22, 2024 at 04:27:43PM -0600, Nathan Bossart wrote: > Here is a patch. I'd like to fix these crashes sooner than later, so I will plan on committing this tonight (barring objections or feedback). If this needs to be revisited later for some reason, I'm happy to do so

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Nathan Bossart
On Tue, Jan 23, 2024 at 06:33:25PM +0100, Alvaro Herrera wrote: > On 2024-Jan-22, Nathan Bossart wrote: >> This might be a topic for another thread, but I do wonder whether we could >> put a generic pg_controldata check in node->stop that would at least make >> sure that t

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Nathan Bossart
On Tue, Jan 23, 2024 at 12:22:58PM -0600, Nathan Bossart wrote: > On Tue, Jan 23, 2024 at 06:33:25PM +0100, Alvaro Herrera wrote: >> Does this actually detect a problem if you take out the fix? I think >> what's going to happen is that postmaster is going to crash, then do t

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Nathan Bossart
On Tue, Jan 23, 2024 at 06:33:25PM +0100, Alvaro Herrera wrote: > On 2024-Jan-22, Nathan Bossart wrote: > >> Here is a patch. > > Looks reasonable. Committed. Thank you for the report and the reviews. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: cleanup patches for incremental backup

2024-01-24 Thread Nathan Bossart
7; # doesn't match '(?^m:FORK main: block 0$)' I haven't been able to reproduce the issue on my machine, and I haven't figured out precisely what is happening yet, but I wanted to make sure there is awareness. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: POC: GROUP BY optimization

2024-01-24 Thread Nathan Bossart
A recent buildfarm failure [0] seems to indicate a name collision with the "abc" index in the aggregates.sql test and the "abc" table in namespace.sql. [0] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=piculet&dt=2024-01-24%2014%3A05%3A14 -- Nathan Bossart Am

Re: cleanup patches for incremental backup

2024-01-24 Thread Nathan Bossart
f time we compute here. */ cutoff_time = time(NULL) - 60 * wal_summary_keep_time; Otherwise, I think we'll probably need to add some additional logging to figure out what is happening... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: cleanup patches for incremental backup

2024-01-24 Thread Nathan Bossart
On Wed, Jan 24, 2024 at 02:08:08PM -0500, Robert Haas wrote: > On Wed, Jan 24, 2024 at 1:05 PM Nathan Bossart > wrote: >> Otherwise, I think we'll probably need to add some additional logging to >> figure out what is happening... > > Where, though? I suppose we cou

Re: Remove redundant HandleWalWriterInterrupts()

2024-01-24 Thread Nathan Bossart
propose to >> remove HandleWalWriterInterrupts() and make walwriter use >> HandleMainLoopInterrupts() instead for improved code simplicity. Thought? >> >> Patch attached. > > Nice catch. Indeed they both are the same after commit 1bdd54e662. The > patch LGTM. LGTM, too. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: cleanup patches for incremental backup

2024-01-25 Thread Nathan Bossart
On Thu, Jan 25, 2024 at 10:06:41AM -0500, Robert Haas wrote: > On Wed, Jan 24, 2024 at 2:39 PM Nathan Bossart > wrote: >> That seems like a reasonable starting point. Even if it doesn't help >> determine the root cause, it should at least help rule out concurrent >&g

Re: cleanup patches for incremental backup

2024-01-26 Thread Nathan Bossart
On Fri, Jan 26, 2024 at 11:04:37AM -0500, Robert Haas wrote: > Here is v2 with that addition. Looks reasonable. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Hide exposed impl detail of wchar.c

2024-01-26 Thread Nathan Bossart
On Thu, Jan 04, 2024 at 04:43:29PM -0600, Nathan Bossart wrote: > On Mon, Nov 20, 2023 at 10:39:43PM -0600, Nathan Bossart wrote: >> Alright. The next minor release isn't until February, so I'll let this one >> sit a little while longer in case anyone objects to back-p

Re: cleanup patches for incremental backup

2024-01-27 Thread Nathan Bossart
; 0/155E750 > [15:21:58.925](0.001s) ok 1 - WAL summary file exists Ah, I think this query: SELECT tli, start_lsn, end_lsn from pg_available_wal_summaries() WHERE tli = $summarized_tli AND end_lsn > '$summarized_lsn' is returning more than one row in

Re: Should we remove -Wdeclaration-after-statement?

2024-01-29 Thread Nathan Bossart
aving the declarations in a predictable location. I actually find the alternative less readable, but that could just be because I spend so much time looking at Postgres code. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Hide exposed impl detail of wchar.c

2024-01-29 Thread Nathan Bossart
On Fri, Jan 26, 2024 at 01:24:19PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> I see that I was planning on back-patching this to v16, but since >> is_valid_ascii() was introduced in v15, I'm wondering if it'd be better to >> back-patch it there so that is_

Re: cleanup patches for incremental backup

2024-01-29 Thread Nathan Bossart
On Sat, Jan 27, 2024 at 10:31:09AM -0600, Nathan Bossart wrote: > On Sat, Jan 27, 2024 at 11:00:01AM +0300, Alexander Lakhin wrote: >> I'm discouraged by "\n1" in the file name and in the >> "examining summary..." message. >> regress_log_002_blocks

Re: Optmize bitmapword macros calc (src/backend/nodes/bitmapset.c)

2024-01-29 Thread Nathan Bossart
why we'd cast to bitmapword and not straight to uint32. I don't think the intent is that callers will provide a bitmapword to these macros. I also wonder if it's worth asserting that x is >= 0 before casting here. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Optmize bitmapword macros calc (src/backend/nodes/bitmapset.c)

2024-01-29 Thread Nathan Bossart
On Mon, Jan 29, 2024 at 04:43:32PM -0300, Ranier Vilela wrote: > Em seg., 29 de jan. de 2024 às 16:32, Nathan Bossart < > nathandboss...@gmail.com> escreveu: >> -#define WORDNUM(x) ((x) / BITS_PER_BITMAPWORD) >> -#define BITNUM(x) ((x) % BITS_PER_BITMAPWOR

Re: cleanup patches for incremental backup

2024-01-29 Thread Nathan Bossart
s of that happening in a test are probably pretty small. Could we get the LSN before and after making the change and then inspect all summaries that include that LSN range? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Optmize bitmapword macros calc (src/backend/nodes/bitmapset.c)

2024-01-29 Thread Nathan Bossart
On Tue, Jan 30, 2024 at 11:23:57AM +1300, David Rowley wrote: > On Tue, 30 Jan 2024 at 08:32, Nathan Bossart wrote: >> I'm currently +0.1 for this change. I don't see any huge problem with >> trimming a few instructions, but I'm dubious there's any measurable i

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-02-06 Thread Nathan Bossart
trying to get into v17. I'd be happy to take it forward if Daniel does not intend to work on it. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Set log_lock_waits=on by default

2024-02-06 Thread Nathan Bossart
n for enabling log_checkpoints by default [0], I'm hesitant to commit something like this without further community discussion. [0] https://postgr.es/m/CALj2ACX-rW_OeDcp4gqrFUAkf1f50Fnh138dmkd0JkvCNQRKGA%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 05:47:56PM +0100, Daniel Gustafsson wrote: >> On 6 Feb 2024, at 17:32, Nathan Bossart wrote: >> IMHO this patch is worth trying to get into v17. I'd be happy to take it >> forward if Daniel does not intend to work on it. > > I actually had t

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread Nathan Bossart
k3.anarazel.de#6eb7595873392621d60e6b5a723941bc > > I agree that its easier to not have to refer back to the macros only to > see that they're all invoking StartChildProcess(X). All invocations are > within postmaster.c. Seems reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
Session PID| 21624 > (8 rows) My first reaction is that this should instead return a single row with the same set of columns for any connection type (the not-applicable ones would just be set to NULL). That would match the other meta-commands like \l and \du, and you could still get an exp

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread Nathan Bossart
On Wed, Feb 07, 2024 at 12:48:00AM +0530, Bharath Rupireddy wrote: > +1. Patch LGTM. Unless there are objections, I'll plan on committing this in the next day or two. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: glibc qsort() vulnerability

2024-02-06 Thread Nathan Bossart
ssue doesn't apply to Postgres, I'm tempted to suggest that we make it project policy that comparison functions must be transitive. There might be no real issues today, but if we write all comparison functions the way Mats is suggesting, it should be easier to reason about overflow risks. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: glibc qsort() vulnerability

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 03:55:58PM -0500, Tom Lane wrote: > A comparison routine that is not is probably broken, agreed. > I didn't look through the details of the patch --- I was more > curious whether we had a version of the qsort bug, because > if we do, we should fix that too

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
"Port", pg_catalog.inet_client_addr() AS "Client Address", pg_catalog.inet_client_port() AS "Client Port", pg_catalog.pg_backend_pid() AS "Session PID"; -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 03:06:05PM -0600, Nathan Bossart wrote: > On Tue, Feb 06, 2024 at 08:52:09PM +, Maiquel Grassi wrote: >> I made the adjustment in the code and updated the patch. I believe this >> is the format suggested by you all. Would this be it? > > I was th

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
Now we're > on a better path than the initial one. We can still add the socket > directory and the host. Agreed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: common signal handler protection

2024-02-06 Thread Nathan Bossart
On Tue, Nov 21, 2023 at 03:20:08PM -0600, Nathan Bossart wrote: > * Overhead: The wrapper handler calls a function pointer and getpid(), > which AFAICT is a real system call on most platforms. That might not be > a tremendous amount of overhead, but it's not zero

Re: common signal handler protection

2024-02-07 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 06:48:53PM -0800, Andres Freund wrote: > On 2024-02-06 20:39:41 -0600, Nathan Bossart wrote: >> I finally spent some time trying to measure this overhead. Specifically, I >> sent many, many SIGUSR2 signals to postmaster, which just uses >> dummy_

Re: common signal handler protection

2024-02-07 Thread Nathan Bossart
Sorry for the noise. On Wed, Feb 07, 2024 at 11:06:50AM -0600, Nathan Bossart wrote: > I'd like to get this committed (to HEAD only) in the next few weeks. TBH > I'm not wild about the weird caveats (e.g., race conditions when pqsignal() > is called within a signal handler),

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-07 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 02:37:25PM -0600, Nathan Bossart wrote: > Unless there are objections, I'll plan on committing this in the next day > or two. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: common signal handler protection

2024-02-07 Thread Nathan Bossart
On Wed, Feb 07, 2024 at 10:40:50AM -0800, Andres Freund wrote: > On 2024-02-07 11:15:54 -0600, Nathan Bossart wrote: >> Perhaps we should add a file global bool that is only set during >> wrapper_handler(). Then we could Assert() or elog(ERROR, ...) if >> pqsignal() is called

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
> our qsort() doesn't overflow, no matter what the comparison function does. > > Looking at our ST_SORT(), it seems safe to me. Cool. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From f5850579e92f201218c3025327b91d820eabd18e Mon Sep 17 00:00:00 2001 From: Nathan

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
central helper for this somewhere? Maybe said helper could use __builtin_sub_overflow() and fall back to the slow "if" version only if absolutely necessary. The assembly for that looks encouraging, but I still need to actually test it... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
On Wed, Feb 07, 2024 at 04:42:07PM -0800, Andres Freund wrote: > On 2024-02-07 16:21:24 -0600, Nathan Bossart wrote: >> The assembly for that looks encouraging, but I still need to actually test >> it... > > Possible. For 16bit upcasting to 32bit is clearly the best way. For

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
On Wed, Feb 07, 2024 at 06:06:37PM -0800, Andres Freund wrote: > Another branchless variant is (a > b) - (a < b). It seems to get a similar > improvement as the overflow-checking version. Well, that's certainly more elegant. I updated the patch to that approach for now. -

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
rsi testrdi, rdi setgal shr rdi, 63 sub eax, edi ret while the approach Andres suggested compiles to xor eax, eax cmp edi, esi setldl setgal movzx edx, dl sub eax, edx

Re: glibc qsort() vulnerability

2024-02-07 Thread Nathan Bossart
Mats, I apologize for steamrolling a bit here. I'll take a step back into a reviewer role. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: glibc qsort() vulnerability

2024-02-08 Thread Nathan Bossart
- (int32) (rhs) but for int32, we need to do someting more like what's in the patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: glibc qsort() vulnerability

2024-02-08 Thread Nathan Bossart
27;d not be too surprised if that's defined in some > library. > > > I think it's worth following int.h's pattern of including [s]igned/[u]nsigned > in the name, an efficient implementation for signed might not be the same as > for unsigned. And if we use static

Re: Psql meta-command conninfo+

2024-02-08 Thread Nathan Bossart
re I would consider that to be connection information. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fix the description of GUC "max_locks_per_transaction" and "max_pred_locks_per_transaction" in guc_table.c

2023-02-22 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 12:40:07PM +, wangw.f...@fujitsu.com wrote: > On Wed, Feb 22, 2023 at 8:37 AM Nathan Bossart > wrote: >> So, even with your patch applied, I don't think the formulas are correct. >> I don't know if it's worth writing out the exact for

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-22 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 12:30:20PM +0900, Michael Paquier wrote: > Okay, applied and backpatched with a minimal test set, then. I have > kept the tweaks I did to the tests with extra comments. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Reducing connection overhead in pg_upgrade compat check phase

2023-02-22 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 10:37:35AM +0100, Daniel Gustafsson wrote: >> On 18 Feb 2023, at 06:46, Nathan Bossart wrote: >> The last 4 are for supported versions and, from a very >> quick glance, seem possible to consolidate. That would bring us to a total >> of 11 sepa

Re: Proposal: %T Prompt parameter for psql for current time (like Oracle has)

2023-02-22 Thread Nathan Bossart
on on adding a prompt option, but I've wanted this > (without realizing this was the format of it) many times. +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: verbose mode for pg_input_error_message?

2023-02-23 Thread Nathan Bossart
On Tue, Jan 10, 2023 at 03:41:12PM -0800, Nathan Bossart wrote: > My vote would be to redefine the existing pg_input_error_message() function > to return a record, but I recognize that this would inflate the patch quite > a bit due to all the existing uses in the tests. If this is

Re: verbose mode for pg_input_error_message?

2023-02-23 Thread Nathan Bossart
On Thu, Feb 23, 2023 at 10:40:48AM -0800, Nathan Bossart wrote: > On Tue, Jan 10, 2023 at 03:41:12PM -0800, Nathan Bossart wrote: >> My vote would be to redefine the existing pg_input_error_message() function >> to return a record, but I recognize that this would inflate the patch

Re: verbose mode for pg_input_error_message?

2023-02-23 Thread Nathan Bossart
On Thu, Feb 23, 2023 at 11:30:38AM -0800, Nathan Bossart wrote: > Will post a new version shortly. As promised... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From b284ba24f771b6ccbf599839bdc813af718b91a1 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 23 Feb 2

stopgap fix for signal handling during restore_command

2023-02-23 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 09:48:10PM +1300, Thomas Munro wrote: > On Tue, Feb 21, 2023 at 5:50 PM Nathan Bossart > wrote: >> I'm happy to create a new thread if needed, but I can't tell if there is >> any interest in this stopgap/back-branch fix. Perhaps we should

Re: Weird failure with latches in curculio on v15

2023-02-23 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 09:48:10PM +1300, Thomas Munro wrote: > On Tue, Feb 21, 2023 at 5:50 PM Nathan Bossart > wrote: >> On Tue, Feb 21, 2023 at 09:03:27AM +0900, Michael Paquier wrote: >> > Perhaps beginning a new thread with a patch and a summary would be >> >

Re: stopgap fix for signal handling during restore_command

2023-02-23 Thread Nathan Bossart
7;s easier to grok and to wrangle.) Thanks for providing the reproducer! I am seeing the behavior that you described on my Linux machine. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6bdafd9980854fc9d81f37d450b86e5f5d4a9d84 Mon Sep 17 00:00:00 2001 From: Nathan Bossart

Re: Inconsistency in ACL error message

2023-02-24 Thread Nathan Bossart
On Fri, Feb 24, 2023 at 12:23:27PM -0500, Joseph Koshakow wrote: > I noticed a very minor inconsistency in some ACL error messages. When > you are try and alter a role, it just says "permission denied": You might be interested in https://commitfest.postgresql.org/42/4

Re: stopgap fix for signal handling during restore_command

2023-02-25 Thread Nathan Bossart
On Sat, Feb 25, 2023 at 11:07:42AM -0800, Andres Freund wrote: > On 2023-02-23 20:33:23 -0800, Nathan Bossart wrote:> >> if (in_restore_command) >> -proc_exit(1); >> +{ >> +/* >> + * If we are in a child

Re: stopgap fix for signal handling during restore_command

2023-02-25 Thread Nathan Bossart
On Sat, Feb 25, 2023 at 11:28:25AM -0800, Nathan Bossart wrote: > On Sat, Feb 25, 2023 at 11:07:42AM -0800, Andres Freund wrote: >> I think the much more interesting assertion here would be to check that >> MyProc->pid equals the current pid. > > I don't mind changing

Re: stopgap fix for signal handling during restore_command

2023-02-25 Thread Nathan Bossart
place to put this, I noticed that there's a write_stderr() function in elog.c that we might be able to use. I used that in v9. WDYT? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 7dcf8fe947ab7b6e0b37ddd42a08bbbc560d4a37 Mon Sep 17 00:00:00 2001 From: Nathan Bossart

Re: verbose mode for pg_input_error_message?

2023-02-25 Thread Nathan Bossart
hatever folks think is best. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: verbose mode for pg_input_error_message?

2023-02-25 Thread Nathan Bossart
input_error_info() seems more descriptive to me. I changed the name to that in v4. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From e06bba8374ba486c8593138b10a256aeef42a8af Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 23 Feb 2023 10:31:24 -0800 Subject: [P

Re: stopgap fix for signal handling during restore_command

2023-02-26 Thread Nathan Bossart
On Sun, Feb 26, 2023 at 10:00:29AM -0800, Andres Freund wrote: > On 2023-02-25 14:06:29 -0800, Nathan Bossart wrote: >> On Sat, Feb 25, 2023 at 11:52:53AM -0800, Andres Freund wrote: >> > I think I opined on this before, but we really ought to have a function to >> >

Re: verbose mode for pg_input_error_message?

2023-02-27 Thread Nathan Bossart
On Tue, Feb 28, 2023 at 09:01:48AM +0900, Michael Paquier wrote: > On Mon, Feb 27, 2023 at 11:25:01AM -0800, Nathan Bossart wrote: >> I found a couple of more small changes required to make cfbot happy. >> Otherwise, it looks good to me. > > Thanks, I have confirmed

Re: Track Oldest Initialized WAL Buffer Page

2023-02-27 Thread Nathan Bossart
I think you can simplify the conditional to if (XLogRecPtrIsInvalid(XLogCtl->OldestInitializedPage) || XLogRecPtrToBufIdx(XLogCtl->OldestInitializedPage) == nextidx) It's confusing to me that OldestInitializedPage is set to NewPageBeginPtr. Doesn't that se

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-02-27 Thread Nathan Bossart
ptr += nread; > + nbytes -= nread; > + dst += nread; > + *read_bytes += nread; Why do we only read a page at a time in XLogReadFromBuffersGuts()? What is preventing us from copying all the data we need in one go? -- Nathan Bossart

Re: Beautify pg_walinspect docs a bit

2023-02-28 Thread Nathan Bossart
AULT false) returns setof record -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-02-28 Thread Nathan Bossart
On Tue, Feb 28, 2023 at 10:38:31AM +0530, Bharath Rupireddy wrote: > On Tue, Feb 28, 2023 at 6:14 AM Nathan Bossart > wrote: >> Why do we only read a page at a time in XLogReadFromBuffersGuts()? What is >> preventing us from copying all the data we need in one go? > &g

Re: Track Oldest Initialized WAL Buffer Page

2023-02-28 Thread Nathan Bossart
On Tue, Feb 28, 2023 at 11:12:29AM +0530, Bharath Rupireddy wrote: > On Tue, Feb 28, 2023 at 5:52 AM Nathan Bossart > wrote: >> It's confusing to me that OldestInitializedPage is set to NewPageBeginPtr. >> Doesn't that set it to the beginning of the newest initialize

Re: Weird failure with latches in curculio on v15

2023-02-28 Thread Nathan Bossart
k that makes the current state worthless. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: stopgap fix for signal handling during restore_command

2023-02-28 Thread Nathan Bossart
On Sun, Feb 26, 2023 at 12:12:27PM -0800, Andres Freund wrote: > On 2023-02-26 11:39:00 -0800, Nathan Bossart wrote: >> What precisely did you have in mind? AFAICT you are asking for a wrapper >> around write(). > > Partially I just want something that can easily be search

Re: add PROCESS_MAIN to VACUUM

2023-03-01 Thread Nathan Bossart
dding some tests in 100_vacuumdb.pl > that rely on vacuumdb --verbose and check the logs produced? We > should make sure that the toast or the main relation are processed, > by tracking, for example, logs like vacuuming "schema.table". When > FULL is involved, we may want to track the changes on relfilenodes > depending on what's wanted. That seems like a good idea. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-03-01 Thread Nathan Bossart
we may want to track the changes on relfilenodes >> depending on what's wanted. > > Maybe instead of reading the log, read values from pg_stat_all_tables. Here is an attempt at that. Thanks for the idea. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From

Re: Making empty Bitmapsets always be NULL

2023-03-01 Thread Nathan Bossart
d a separate > boolean as an easy way out, but maybe that code could do with a more > thorough revisit. The separate boolean certainly seems less fragile. That might even be worthwhile independent of the rest of the patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: stopgap fix for signal handling during restore_command

2023-03-01 Thread Nathan Bossart
On Tue, Feb 28, 2023 at 08:36:03PM -0800, Nathan Bossart wrote: > On Sun, Feb 26, 2023 at 12:12:27PM -0800, Andres Freund wrote: >> Partially I just want something that can easily be searched for, that can >> have >> comments attached to it documenting why what it is doing

Re: stopgap fix for signal handling during restore_command

2023-03-01 Thread Nathan Bossart
. Good to know. I couldn't tell whether that was a safe assumption from briefly reading around. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Making empty Bitmapsets always be NULL

2023-03-01 Thread Nathan Bossart
ah. I split out those executor fixes as 0002; 0003 is the changes > to bitmapsets proper, and then 0004 removes now-dead code. These all looked reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-03-01 Thread Nathan Bossart
_reindexdb.pl: > regexp_replace(b.indname::text, '(pg_toast.pg_toast_)\\d+(_index)', > '\\1\\2') > > If you make that part of the view definition, the result is the same, > so that's up to which solution one prefers. Here's a new version of the patch that

Re: add PROCESS_MAIN to VACUUM

2023-03-01 Thread Nathan Bossart
? done > An extra question: should we check the behavior of the commands when > applying a list of relations to VACUUM? I don't feel a strong need for that, especially now that we aren't modifying params anymore. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >

Re: Simplify standby state machine a bit in WaitForWALToBecomeAvailable()

2023-03-03 Thread Nathan Bossart
patch proposed here just reverts that commit [1] > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=4d894b41cd12179b710526eba9dc62c2b99abc4d. > > That commit fixes an issue - "If there is a WAL segment with same ID > but different TLI present in both the WAL archive and pg_xlog, prefe

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote: > That was mostly OK for me, so applied after tweaking a couple of > places in the tests (extra explanations, for one), the comments and > the code. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
n your comment, it might be better to just surround this entire section with an "if (params->options & VACOPT_PROCESS_MAIN)" check. WDYT? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote: > On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart > wrote: >> On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: >> > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is &

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-06 Thread Nathan Bossart
o there's no need for the "else if". if (count == read_bytes) return true; buf += read_bytes; startptr += read_bytes; count -= read_bytes; -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
SS_TOAST is set), we'll force PROCESS_MAIN to be set when we recurse to the TOAST table so that it gets processed here. How does that sound? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-07 Thread Nathan Bossart
On Tue, Mar 07, 2023 at 12:39:13PM +0530, Bharath Rupireddy wrote: > On Tue, Mar 7, 2023 at 3:30 AM Nathan Bossart > wrote: >> Is it possible to memcpy more than a page at a time? > > It would complicate things a lot there; the logic to figure out the > last page bytes th

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-03-07 Thread Nathan Bossart
h the proposed feature. IMO the current set of trade-offs (e.g., unavoidable bloat and WAL buildup) would make this feature virtually unusable for a lot of workloads, so it's probably worth exploring an alternative approach. In any case, we probably shouldn't rush this into v16 in its cu

Re: add PROCESS_MAIN to VACUUM

2023-03-07 Thread Nathan Bossart
On Tue, Mar 07, 2023 at 12:39:29PM -0500, Melanie Plageman wrote: > Yes, sounds clear to me also! Here is an updated patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 580f966499..0acc42a

optimize several list functions with SIMD intrinsics

2023-03-07 Thread Nathan Bossart
ly have any concrete examples of workloads where this might help. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 4e04f84766d98f9ba6bb6fdd03bcb431c8aad1d3 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Sat, 4 Mar 2023 23:16:07 -0800 Subject: [PATCH v1 1/1] speed up several l

Re: optimize several list functions with SIMD intrinsics

2023-03-07 Thread Nathan Bossart
he moment, the support for SIMD intrinsics in Postgres is limited to 64-bit (simd.h has the details). But yes, if we want to make this work for 32-bit builds, additional work is required. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: psql \watch 2nd argument: iteration count

2023-03-08 Thread Nathan Bossart
leep argument? I often use commands like "\watch .1" to run statements repeatedly with very little time in between, and I'd use "\watch 0" instead if it was available. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize several list functions with SIMD intrinsics

2023-03-08 Thread Nathan Bossart
cfbot's Windows build wasn't happy with a couple of casts. I applied a fix similar to c6a43c2 in v2. The patch is still very much a work in progress. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 055717233c47518ae48119938ebd203cc55f7f3c Mon Sep 17 00:00

Re: Add error functions: erf() and erfc()

2023-03-08 Thread Nathan Bossart
man pages for these seem to indicate that underflow can occur. Do we need to check for that? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Get rid of PgStat_BackendFunctionEntry

2023-03-08 Thread Nathan Bossart
ted > to get rid of PgStat_BackendFunctionEntry in a separate patch. Looks reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improve logging when using Huge Pages

2023-03-08 Thread Nathan Bossart
On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: >> On Mon, Feb 13, 2023 at 08:18:52PM -0800, Nathan Bossart wrote: >>> I'm curious why you chose to make this a string instead of an enum. Th

Re: Add error functions: erf() and erfc()

2023-03-08 Thread Nathan Bossart
On Wed, Mar 08, 2023 at 11:29:12PM +, Dean Rasheed wrote: > On Wed, 8 Mar 2023 at 20:11, Nathan Bossart wrote: >> The man pages for these seem to indicate that underflow can occur. Do we >> need to check for that? > > No, I don't think so. The docs indicate t

Re: Add error functions: erf() and erfc()

2023-03-08 Thread Nathan Bossart
On Thu, Mar 09, 2023 at 12:27:47AM +, Dean Rasheed wrote: > On Thu, 9 Mar 2023 at 00:13, Nathan Bossart wrote: >> I'm also wondering about whether we need the isinf() checks. IIUC that >> should never happen, but maybe you added that "just in case."

Re: improving user.c error messages

2023-03-09 Thread Nathan Bossart
On Thu, Mar 09, 2023 at 10:55:54AM +0100, Peter Eisentraut wrote: > On 20.02.23 23:58, Nathan Bossart wrote: >> For now, I've reworded these as "must inherit privileges of". > > I don't have a good mental model of all this role inheritance, personally, >

Re: psql \watch 2nd argument: iteration count

2023-03-09 Thread Nathan Bossart
ot;\\watch: invalid delay interval: %s", opt); + free(opt); + resetPQExpBuffer(query_buf); + return PSQL_CMD_ERROR; Is this missing psql_scan_reset(scan_state)? I haven't had a chance to look closely at 0

<    1   2   3   4   5   6   7   8   9   10   >