Re: [PATCH] Add tests for Bitmapset

2025-09-22 Thread Greg Burd
On Sep 22 2025, at 3:57 am, Michael Paquier wrote: > On Fri, Sep 19, 2025 at 02:48:43PM -0400, Greg Burd wrote: >> I think I've incorporated your feedback which did indeed make the patch >> more readable/crisp and maintainable, thank you. >> >> covera

Re: [PATCH] Add tests for Bitmapset

2025-09-19 Thread Greg Burd
On Sep 19 2025, at 2:36 am, Michael Paquier wrote: > On Thu, Sep 18, 2025 at 02:50:45PM -0400, Greg Burd wrote: >> Thanks for all the feedback and time spent reviewing this patch. I >> switched out the encode/decode functions to use the nodeToString() and >> stringToNode()

Re: [PATCH] Add tests for Bitmapset

2025-09-18 Thread Greg Burd
Hello hackers, :) Thanks for all the feedback and time spent reviewing this patch. I switched out the encode/decode functions to use the nodeToString() and stringToNode() functions and change all the SQL testing function signatures to TEXT from BYTEA. This exercises more code and that's good for

Re: [PATCH] Add tests for Bitmapset

2025-09-18 Thread Greg Burd
I've re-written the set of tests as suggested (I hope!). I've not re-run the coverage report (yet) to ensure we're at least as well covered as v3, but attached is v4 for your inspection (amusement?). I've tried to author the SQL tests such that failures clearly indicate what's at gone wrong. Thi

Re: [PATCH] Add tests for Bitmapset

2025-09-17 Thread Greg Burd
On Sep 16 2025, at 11:25 pm, Michael Paquier wrote: > On Tue, Sep 16, 2025 at 03:04:39PM -0400, Greg Burd wrote: >> I've re-written the set of tests as suggested (I hope!). I've not >> re-run the coverage report (yet) to ensure we're at least as well >> c

Re: [PATCH] Add tests for Bitmapset

2025-09-17 Thread Greg Burd
On Sep 17 2025, at 9:55 am, Robert Haas wrote: > On Wed, Sep 17, 2025 at 9:18 AM Greg Burd wrote: >> > +static void >> > +elog_bitmapset(int elevel, const char *label, const Bitmapset *bms) >> > >> >> I added a function bms_values() that prints out

Re: [PATCH] Add tests for Bitmapset

2025-09-17 Thread Greg Burd
On Sep 16 2025, at 8:35 pm, Michael Paquier wrote: > On Tue, Sep 16, 2025 at 03:00:40PM -0700, Masahiko Sawada wrote: >> Thank you for updating the patch. It seems cfbot caught a regression >> test error[1] in a 32-bit build. > > - bitmap_hash [1,3,5] | 49870778 > + bitmap_hash [1,3,5] | 150

Re: [PATCH] Add tests for Bitmapset

2025-09-16 Thread Greg Burd
On Sep 16 2025, at 8:02 am, Robert Haas wrote: > On Tue, Sep 16, 2025 at 2:04 AM Michael Paquier wrote: >> one SQL function mapping to each C function we are testing? > > Yes, I think we should do this, if possible. Michael, Robert, Thanks for your time reviewing the proposed code and for

Re: [PATCH] Add tests for Bitmapset

2025-09-15 Thread Greg Burd
On Sep 15 2025, at 3:03 pm, Greg Burd wrote: > On Sep 15 2025, at 2:54 pm, Robert Haas wrote: > >> On Mon, Sep 15, 2025 at 2:00 PM Greg Burd wrote: >>> For reference radixtree has: >>> >>> coverage: HEAD >>> lines..: 98.3

Re: [PATCH] Add tests for Bitmapset

2025-09-15 Thread Greg Burd
On Sep 15 2025, at 2:54 pm, Robert Haas wrote: > On Mon, Sep 15, 2025 at 2:00 PM Greg Burd wrote: >> For reference radixtree has: >> >> coverage: HEAD >> lines..: 98.3 >> functions..: 97.2 >> branches...: 89.4 > > +

Re: [PATCH] Add tests for Bitmapset

2025-09-15 Thread Greg Burd
On Sep 13 2025, at 10:23 am, Greg Burd wrote: > > Sawada-san, Michael, > > Thank you both for the push to measure.  Before the patch as it stands > now the > coverage for src/backend/nodes/bitmapset.c is 63.5% and after it is > 66.5%.  Not > an amazing difference, b

Re: [PATCH] Add tests for Bitmapset

2025-09-13 Thread Greg Burd
> On Sep 11, 2025, at 9:36 PM, Michael Paquier wrote: > > On Thu, Sep 11, 2025 at 06:56:07AM -0400, Greg Burd wrote: >> Just for reference I started this not to increase coverage, which is a good >> goal just not the one I had. I was reviewing the API and considering som

Re: [PATCH] Add tests for Bitmapset

2025-09-11 Thread Greg Burd
On Thu, Sep 11, 2025, at 2:48 AM, Masahiko Sawada wrote: > On Mon, Sep 8, 2025 at 11:21 AM Burd, Greg wrote: >> >> >> > On Sep 5, 2025, at 2:43 PM, Nathan Bossart >> > wrote: >> > >> > On Fri, Sep 05, 2025 at 10:48:21AM -0400, Burd, Greg wrote: >> >> I looked at both radix tree and binary heap

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-09-05 Thread Greg Burd
On Fri, Sep 5, 2025, at 12:27 PM, Andres Freund wrote: > Hi, > > On 2025-09-04 13:24:00 -0400, Greg Burd wrote: >> On Thu, Sep 4, 2025, at 12:59 PM, Andres Freund wrote: >> > Hi, >> > >> > On 2025-08-27 15:42:48 -0400, Greg Burd wrote: >> >>

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-09-04 Thread Greg Burd
On Thu, Sep 4, 2025, at 12:59 PM, Andres Freund wrote: > Hi, > > On 2025-08-27 15:42:48 -0400, Greg Burd wrote: >> Regardless, I feel the first two patches on this set address the >> intention of this thread. > > I'm planning to commit the first two patches after

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-08-27 Thread Greg Burd
On Aug 17 2025, at 12:57 am, Thomas Munro wrote: > On Sun, Aug 17, 2025 at 4:34 PM Thomas Munro wrote: >> Or if you don't like those odds, maybe it'd be OK to keep % but use it >> rarely and without the CAS that can fail. > > ... or if we wanted to try harder to avoid %, could we relegate it

Re: `pg_ctl init` crashes when run concurrently; semget(2) suspected

2025-08-16 Thread Greg Burd
> On Aug 15, 2025, at 10:25 PM, Thomas Munro wrote: > > On Sat, Aug 16, 2025 at 5:58 AM Burd, Greg wrote: >>> 1. They use CAS in sem_post() because they want to report EOVERFLOW if >>> you exceed SEM_VALUE_MAX, but POSIX doesn't seem to require that, so I >>> just used fetch-and-add. Is that

[PATCH] Add tests for Bitmapset

2025-08-15 Thread Greg Burd
Hello, I noticed that there are no tests for Bitmapset in src/test/modules as is the case for other similar things like radixtree, rbtree, etc. so I created one. I realize that Bitmapset is already "tested" by all the other code that uses it, but I was able to find one minor oversight[1] in that

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
> On Aug 14, 2025, at 3:45 PM, Tom Lane wrote: > > Greg Burd writes: >> Spoiler alert, I have larger plans[2] for Bitmapset which is why I >> started down this road to begin with as I wanted to capture the existing >> API/contract for it before I proposed chang

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
Tom, David, I really appreciate the time spent and the thoughtful replies on this thread today. Honestly, I thought this would be a nice and easy single line fix ahead of a proposal for the addition of tests for Bitmapset[1] (which identified the issue) I think, with v5, we're fixing this small i

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
On Aug 14 2025, at 11:49 am, David Rowley wrote: > On Fri, 15 Aug 2025 at 03:43, Greg Burd wrote: >> Well, that was rushed. Apologies. > > Would you be ok with adding the Assert after the "a == NULL" check?, i.e: > > if (a == NULL || prevbit == 0) >r

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
On Aug 14 2025, at 11:37 am, Greg Burd wrote: > > On Aug 14 2025, at 11:14 am, Tom Lane wrote: > >> David Rowley writes: >>> It is valid to pass prevbit as a->nwords * BITS_PER_BITMAPWORD as the >>> code does "prevbit--;". Maybe it wou

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
On Aug 14 2025, at 11:14 am, Tom Lane wrote: > David Rowley writes: >> It is valid to pass prevbit as a->nwords * BITS_PER_BITMAPWORD as the >> code does "prevbit--;". Maybe it would be less confusing if it were >> written as: >> * "prevbit" must be less than or equal to "a->nwords * BITS_PER_

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
On Aug 14 2025, at 10:06 am, Tom Lane wrote: > David Rowley writes: >> There's a comment saying: >> * "prevbit" must NOT be more than one above the highest possible bit >> that can >> * be set at the Bitmapset at its current size. >> So looks like it's the fault of the calling code and not a

Re: [PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
On Aug 14 2025, at 9:46 am, David Rowley wrote: > On Fri, 15 Aug 2025 at 01:21, Greg Burd wrote: >> I've been working on Bitmapset and while creating a test suite for it I >> found that there is a missing bounds check in bms_prev_member(). The >> function take

[PATCH] bms_prev_member() can read beyond the end of the array of allocated words

2025-08-14 Thread Greg Burd
Hello, I've been working on Bitmapset and while creating a test suite for it I found that there is a missing bounds check in bms_prev_member(). The function takes the prevbit argument and converts it to an index into the words array using WORDNUM() without checking to ensure that prevbit is within

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-08-12 Thread Greg Burd
etermined inside those function, otherwise every caller would have to > repeat that. I think this is unnecessary/inconvenient. Okay, I understand that. At one point I had implementations for ClockPro and SIEVE but you're right. > regards > > > [1] > https://www.post

Re: Convert varatt.h macros to static inline functions

2025-07-31 Thread Greg Burd
> On Jul 31, 2025, at 10:06 AM, Tom Lane wrote: > > Peter Eisentraut writes: >> I had this lying around as a draft patch, as part of my ongoing campaign  >> to convert many complicated macros to static inline functions.  Since  >> the topic was mentioned in another thread [0], I cleaned up the

Re: Enable data checksums by default

2025-07-31 Thread Greg Burd
> On Jul 30, 2025, at 8:09 AM, Daniel Gustafsson wrote: > >> On 30 Jul 2025, at 11:58, Laurenz Albe wrote: >> >> On Tue, 2025-07-29 at 20:24 +0200, Tomas Vondra wrote: >>> So, what should we do with the PG18 open item? We (the RMT team) would >>> like to know if we shall keep the checksums e

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-27 Thread Greg Burd
On 7/25/25 15:02, Greg Burd wrote: > Patch set is now: > > 1) remove freelist > > 2) remove buffer_strategy_lock > > 3) abstract clock-sweep to type and API > > > > -greg Somehow including the test.c file as an attachment on my last email confused the CI and

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-25 Thread Greg Burd
On 7/22/25 14:43, Greg Burd wrote: > On 7/21/25 14:35, Andres Freund wrote: >> Hi, >> >> On 2025-07-21 13:37:04 -0400, Greg Burd wrote: >>> On 7/18/25 13:03, Andres Freund wrote: >>> Hello.  Thanks again for taking the time to review the email and patch, >

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-22 Thread Greg Burd
On 7/21/25 14:35, Andres Freund wrote: > Hi, > > On 2025-07-21 13:37:04 -0400, Greg Burd wrote: >> On 7/18/25 13:03, Andres Freund wrote: >> Hello.  Thanks again for taking the time to review the email and patch, >> I think we're onto something good here. >>

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-21 Thread Greg Burd
in autoprewarm, it's a rather trivial > patch. And I really couldn't measure regressions above the noise level, even > if absurdly extreme use cases. Hmmm... was "argue for keeping the clock sweep" supposed to read "argue for keeping the freelist"? > On 2025-0

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-17 Thread Greg Burd
On Fri, Jul 11, 2025, at 2:52 PM, Andres Freund wrote: Hi, On 2025-07-11 13:26:53 -0400, Greg Burd wrote: In conversations [1] recently about considering how best to adapt the code to become NUMA-aware Andres commented, "FWIW, I've started to wonder if we shouldn't just

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-11 Thread Greg Burd
On Fri, Jul 11, 2025, at 2:50 PM, Nathan Bossart wrote: > On Fri, Jul 11, 2025 at 01:26:53PM -0400, Greg Burd wrote: >> This change does remove the have_free_buffer() function used by the >> contrib/pg_prewarm module. On the surface this doesn't seem to cause any >>

[PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-11 Thread Greg Burd
Hello, In conversations [1] recently about considering how best to adapt the code to become NUMA-aware Andres commented, "FWIW, I've started to wonder if we shouldn't just get rid of the freelist entirely" and because I'm a glutton for punishment (and I think this idea has some merit) I took hi

Re: Adding basic NUMA awareness

2025-07-09 Thread Greg Burd
On Jul 9 2025, at 12:35 pm, Andres Freund wrote: > FWIW, I've started to wonder if we shouldn't just get rid of the freelist > entirely. While clocksweep is perhaps minutely slower in a single > thread than > the freelist, clock sweep scales *considerably* better [1]. As it's rather > rare to