[PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread David Christensen
depending on how the blocks are specified. This currently affects only the main fork, but we could presumably add the option to filter by fork as well, if that is considered useful. Best, David >From 9194b2cb07172e636030b9b4e979b7f2caf7cbc0 Mon Sep 17 00:00:00 2001 From: David Christensen D

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread David Christensen
Added to commitfest as: https://commitfest.postgresql.org/37/3565/

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
> Cool. I think we can report an error instead of reading wal files, > if the tablespace, database, or relation is invalid. Does there any > WAL record that has invalid tablespace, database, or relation OID? The only sort of validity check we could do here is range checking for the underlying d

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
On Fri, Feb 25, 2022 at 7:33 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Thanks for the patch. This is not adding something that users can't do > right now, but definitely improves the usability of the pg_waldump as > it avoids external filterings. Also, it can give the

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
On Fri, Feb 25, 2022 at 7:08 AM Japin Li wrote: > > On Fri, 25 Feb 2022 at 20:48, David Christensen < > david.christen...@crunchydata.com> wrote: > >> Cool. I think we can report an error instead of reading wal files, > >> if the tablespace, database, or rel

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
Bharath Rupireddy writes: > On Fri, Feb 25, 2022 at 12:36 AM David Christensen > wrote: >> >> Greetings, >> >> This patch adds the ability to specify a RelFileNode and optional BlockNum >> to limit output of >> pg_waldump records to only those which

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-21 Thread David Christensen
On Sun, Mar 20, 2022 at 11:56 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 4:36 PM Thomas Munro > wrote: > > On Sat, Feb 26, 2022 at 7:58 AM David Christensen > > wrote: > > > Attached is V2 with additional feedback from this email, as well as > the specifi

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-21 Thread David Christensen
Updated to include the V3 fixes as well as the unsigned int/enum fix. > v4-0001-Add-additional-filtering-options-to-pg_waldump.patch Description: Binary data

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-21 Thread David Christensen
On Mon, Mar 21, 2022 at 4:39 PM Thomas Munro wrote: [snip] I guess you did this because init fork references aren't really > expected in the WAL, but I think it's more consistent to allow up to > MAX_FORKNUM, not least because your documentation mentions 3 as a > valid value. So I adjust this t

Re: [PATCH] pgbench: add multiconnect option

2022-03-22 Thread David Christensen
On Sat, Mar 19, 2022 at 11:43 AM Fabien COELHO wrote: > > Hi Sami, > > > Pgbench is a simple benchmark tool by design, and I wonder if adding > > a multiconnect feature will cause pgbench to be used incorrectly. > > Maybe, but I do not see how it would be worse that what pgbench already > allows.

Re: [PATCH] Proof of concept for GUC improvements

2022-03-22 Thread David Christensen
> On Mar 21, 2022, at 7:53 PM, Tom Lane wrote: > > Andres Freund writes: >> My impression is that there's not a lot of enthusiasm for the concept? If >> that's true we maybe ought to mark the CF entry as rejected? > > Yeah, I'm kind of leaning that way too. I don't see how we can > incorpor

Re: psql \df choose functions by their arguments

2020-11-01 Thread David Christensen
> * Removed the tab-complete bit, it was too fragile and unhelpful I can’t speak for the specific patch, but tab completion of proc args for \df, \ef and friends has long been a desired feature of mine, particularly when you are dealing with functions with huge numbers of arguments and the sam

Re: [PATCH] Add `truncate` option to subscription commands

2020-11-25 Thread David Christensen
the implications of locking these tables on the > main apply process. > > > Thank you, > > > On Mon, Oct 12, 2020 at 11:31 PM David Christensen wrote: >> >> > On Oct 11, 2020, at 10:00 PM, Amit Kapila wrote: >> > >> > On Mon, Oct 12, 2020 at 3:4

Re: [PATCH] Proof of concept for GUC improvements

2022-01-12 Thread David Christensen
> Hi, > > According to the cfbot, the patch doesn't apply anymore and needs a > rebase: http://cfbot.cputube.org/patch_36_3290.log V4 rebased attached. special-guc-values-v4.patch Description: Binary data

CREATE ROLE IF NOT EXISTS

2021-10-19 Thread David Christensen
Greetings -hackers, Enclosed is a patch that implements CREATE ROLE IF NOT EXISTS (along with the same support for USER/GROUP). This is a fairly straightforward approach in that we do no validation of anything other than existence, with the user needing to ensure that permissions/grants are set u

Re: CREATE ROLE IF NOT EXISTS

2021-10-21 Thread David Christensen
On Tue, Oct 19, 2021 at 4:29 PM Isaac Morland wrote: > On Tue, 19 Oct 2021 at 16:12, David Christensen < > david.christen...@crunchydata.com> wrote: > >> Greetings -hackers, >> >> Enclosed is a patch that implements CREATE ROLE IF NOT EXISTS (along with >>

Re: [PATCH] Proof of concept for GUC improvements

2021-11-03 Thread David Christensen
> On Nov 3, 2021, at 5:35 AM, Daniel Gustafsson wrote: > >  >> >>> On 15 Oct 2021, at 23:54, Cary Huang wrote: >> >> I scanned through the GUC list and found that the following parameters can >> potentially be categorized in the "special_disabled0" group, just for your >> reference. > > >

Re: CREATE ROLE IF NOT EXISTS

2021-11-03 Thread David Christensen
> > > This fails the roleattributes test in "make check", with what seems to be a > trivial change in the output. Can you please submit a rebased version > fixing > the test? > Updated version attached. David CREATE-ROLE-IF-NOT-EXISTS-v2.patch Description: Binary data

Re: CREATE ROLE IF NOT EXISTS

2021-11-09 Thread David Christensen
On Mon, Nov 8, 2021 at 1:22 PM Mark Dilger wrote: > > On Nov 8, 2021, at 10:38 AM, Stephen Frost wrote: > > > I don't quite follow this. The entire point of Alice writing a script > > that uses IF NOT EXISTS is to have that command not fail if, indeed, > > that role already exists, but for the

Re: CREATE ROLE IF NOT EXISTS

2021-11-09 Thread David Christensen
On Tue, Nov 9, 2021 at 9:55 AM Mark Dilger wrote: > > On Nov 9, 2021, at 7:36 AM, David Christensen < > david.christen...@crunchydata.com> wrote: > > > > If CINE semantics are at issue, what about the CREATE OR REPLACE > semantics with some sort of merge into th

Re: CREATE ROLE IF NOT EXISTS

2021-11-09 Thread David Christensen
On Tue, Nov 9, 2021 at 10:22 AM Stephen Frost wrote: > Greetings, > > * David Christensen (david.christen...@crunchydata.com) wrote: > > Well, the CREATE OR REPLACE via just setting the role's attributes > > explicitly based on what you passed it could work (not stri

Re: CREATE ROLE IF NOT EXISTS

2021-11-10 Thread David Christensen
Modulo other issues/discussions, here is a version of this patch that implements CREATE OR REPLACE ROLE just by handing off to AlterRole if it's determined that the role already exists; presumably any/all additional considerations would need to be added in both places were there a separate code pat

Re: CREATE ROLE IF NOT EXISTS

2021-11-22 Thread David Christensen
On Mon, Nov 22, 2021 at 6:49 AM Daniel Gustafsson wrote: > > On 10 Nov 2021, at 18:14, David Christensen < > david.christen...@crunchydata.com> wrote: > > > Modulo other issues/discussions, here is a version of this patch.. > > This patch fails to compile since

Documentation patch for ALTER SUBSCRIPTION

2020-01-23 Thread David Christensen
10. Best, David 0001-Be-explicit-about-the-behavior-of-REFRESH-PUBLICATIO.patch Description: Binary data -- David Christensen Senior Software and Database Engineer End Point Corporation da...@endpoint.com 785-727-1171 signature.asc Description: Message signed with OpenPGP

Re: Adding comments to help understand psql hidden queries

2024-02-01 Thread David Christensen
On Thu, Feb 1, 2024 at 4:34 PM Greg Sabino Mullane wrote: > > The use of the --echo-hidden flag in psql is used to show people the way psql > performs its magic for its backslash commands. None of them has more magic > than "\d relation", but it suffers from needing a lot of separate queries to

Constant Splitting/Refactoring

2024-02-07 Thread David Christensen
As part of the reserved page space/page features[1] work, there is a need to convert some of the existing constants into variables, since the size of those will no longer be fixed at compile time. At FOSDEM, there was some interest expressed about seeing what a cleanup patch like that would look l

Re: Adding comments to help understand psql hidden queries

2024-03-21 Thread David Christensen
Hi Jim, Thanks for the feedback. Enclosed is a v2 of this series(?) rebased and with that warning fixed; @Greg Sabino Mullane I just created a commit on your behalf with the message to hackers. I'm also creating a commit-fest entry for this thread. Best, David v2-0002-Add-output-of-the-comma

Re: Adding comments to help understand psql hidden queries

2024-03-21 Thread David Christensen
Created the CF entry in commitfest 48 but didn't see it was already in 47; closing the CFEntry in 48. (Doesn't appear to be a different status than "Withdrawn"...)

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-21 Thread David Christensen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Did a quick review of this one; CFbot is now happy, local regress

Re: Adding comments to help understand psql hidden queries

2024-03-22 Thread David Christensen
On Fri, Mar 22, 2024 at 9:47 AM Greg Sabino Mullane wrote: > > On Thu, Mar 21, 2024 at 6:20 PM Peter Eisentraut wrote: >> >> lines are supposed to align vertically. With your patch, the first line >> would have variable length depending on the command. > > > Yes, that is a good point. Aligning t

Re: Adding comments to help understand psql hidden queries

2024-04-03 Thread David Christensen
I got Greg's blessing on squashing the commits down, and now including a v4 with additional improvements on the output formatting front. Main changes: - all generated comments are the same width - width has been bumped to 80 - removed _() functions for consumers of the new output functions This p

Re: Adding comments to help understand psql hidden queries

2024-04-04 Thread David Christensen
On Thu, Apr 4, 2024 at 9:32 AM Peter Eisentraut wrote: > > On 03.04.24 19:16, David Christensen wrote: > > I removed _() in the output of the query/stars since there'd be no > > sensible existing translations for the constructed string, which > > included the query

Re: Constant Splitting/Refactoring

2024-02-14 Thread David Christensen
This should target PG 18, but that status is not available in the CF app yet, so just making a note here.

Re: Constant Splitting/Refactoring

2024-03-13 Thread David Christensen
Here is a version 2 of this patch, rebased atop 97d85be365. As before, this is a cleanup/prerequisite patch series for the page features/reserved page size patches[1]. (Said patch series is going to be updated shortly.) This splits each of the 4 constants that care about page size into Cluster-s

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-14 Thread David Christensen
Hi Bharath, I can get one sent in tomorrow. Thanks, David

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-15 Thread David Christensen
On Thu, Dec 15, 2022 at 12:36 AM Michael Paquier wrote: > > On Wed, Dec 14, 2022 at 04:44:34PM -0600, David Christensen wrote: > > I can get one sent in tomorrow. This v10 should incorporate your feedback as well as Bharath's. > -XLogRecordHasFPW(XLogReaderState *record)

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-23 Thread David Christensen
On Wed, Dec 21, 2022 at 5:47 AM Bharath Rupireddy wrote: > > On Fri, Dec 16, 2022 at 4:47 AM David Christensen > wrote: > > > > On Thu, Dec 15, 2022 at 12:36 AM Michael Paquier > > wrote: > > > > > > On Wed, Dec 14, 2022 at 04:44:34PM -0600, David

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-23 Thread David Christensen
On Mon, Dec 19, 2022 at 12:23 AM Michael Paquier wrote: > > On Thu, Dec 15, 2022 at 05:17:46PM -0600, David Christensen wrote: > > On Thu, Dec 15, 2022 at 12:36 AM Michael Paquier > > wrote: > > This v10 should incorporate your feedback as well as Bharath's. >

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-23 Thread David Christensen
On Fri, Dec 23, 2022 at 12:57 PM David Christensen wrote: > > On Wed, Dec 21, 2022 at 5:47 AM Bharath Rupireddy > wrote: [snip] > > 2. +$node->init(extra => ['-k'], allows_streaming => 1); > > When enabled with allows_streaming, there are a bunch of

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-26 Thread David Christensen
> On Dec 26, 2022, at 1:29 AM, Michael Paquier wrote: > > On Sat, Dec 24, 2022 at 06:23:29PM +0530, Bharath Rupireddy wrote: >> Thanks for the patch. I've made the above change as well as renamed >> the test file name to be save_fpi.pl, everything else remains the same >> as v11. Here's the v12

Re: Improving btree performance through specializing by key shape, take 2

2023-01-12 Thread David Christensen
Hi Matthias, I'm going to look at this patch series if you're still interested. What was the status of your final performance testing for the 0008 patch alone vs the specialization series? Last I saw on the thread you were going to see if the specialization was required or not. Best, David

Re: Kerberos delegation support in libpq and postgres_fdw

2023-04-03 Thread David Christensen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Did a code review pass here; here is some feedback. + /* If p

Re: Kerberos delegation support in libpq and postgres_fdw

2023-04-05 Thread David Christensen
On Wed, Apr 5, 2023 at 3:30 PM Stephen Frost wrote: > Greetings, > > * David Christensen (david...@pgguru.net) wrote: > > Did a code review pass here; here is some feedback. > > Thanks! > > > + /* If password was used to connect, make sure it was

Re: Kerberos delegation support in libpq and postgres_fdw

2023-04-07 Thread David Christensen
Reviewed v8; largely looking good, though I notice this hunk, which may arguably be a bug fix, but doesn't appear to be relevant to this specific patch, so could probably be debated independently (and if a bug, should probably be backpatched): diff --git a/contrib/postgres_fdw/option.c b/contrib/p

Re: Kerberos delegation support in libpq and postgres_fdw

2023-04-07 Thread David Christensen
Ok, based on the interdiff there, I'm happy with that last change. Marking as Ready For Committer. Best, David

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread David Christensen
> On Mar 24, 2022, at 6:43 AM, Thomas Munro wrote: > > On Fri, Mar 25, 2022 at 12:26 AM Thomas Munro wrote: >>> On Fri, Mar 25, 2022 at 12:01 AM Peter Eisentraut >>> wrote: >>> Or even: Why are we exposing fork *numbers* in the user interface? >>> Even low-level tools such as pageinspect us

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread David Christensen
> On Mar 24, 2022, at 4:12 PM, Thomas Munro wrote: > > On Fri, Mar 25, 2022 at 1:43 AM David Christensen > wrote: >>>> On Mar 24, 2022, at 6:43 AM, Thomas Munro wrote: >>> On Fri, Mar 25, 2022 at 12:26 AM Thomas Munro >>> wrote: >>>&

[PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-22 Thread David Christensen
Hi -hackers, Enclosed is a patch to allow extraction/saving of FPI from the WAL stream via pg_waldump. Description from the commit: Extracts full-page images from the WAL stream into a target directory, which must be empty or not exist. These images are subject to the same filtering rules as no

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-23 Thread David Christensen
On Sat, Apr 23, 2022 at 9:49 AM Matthias van de Meent wrote: > Regardless of my (lack of) opinion on the inclusion of this patch in > PG (I did not significantly review this patch); I noticed that you do > not yet identify the 'fork' of the FPI in the file name. > > A lack of fork identifier in t

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-25 Thread David Christensen
On Mon, Apr 25, 2022 at 1:11 AM Michael Paquier wrote: > > On Sat, Apr 23, 2022 at 01:43:36PM -0500, David Christensen wrote: > > Hi Matthias, great point. Enclosed is a revised version of the patch > > that adds the fork identifier to the end if it's a non-main fork.

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-25 Thread David Christensen
On Mon, Apr 25, 2022 at 2:00 AM Drouvot, Bertrand wrote: > > Hi, > > On 4/25/22 8:11 AM, Michael Paquier wrote: > > On Sat, Apr 23, 2022 at 01:43:36PM -0500, David Christensen wrote: > >> Hi Matthias, great point. Enclosed is a revised version of the patch > >&

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-25 Thread David Christensen
On Mon, Apr 25, 2022 at 6:03 AM Bharath Rupireddy wrote: > Thanks for working on this. I'm just thinking if we can use these FPIs > to repair the corrupted pages? I would like to understand more > detailed usages of the FPIs other than inspecting with pageinspect. My main use case was for being a

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-26 Thread David Christensen
On Mon, Apr 25, 2022 at 9:54 PM Michael Paquier wrote: > On Mon, Apr 25, 2022 at 10:11:10AM -0500, David Christensen wrote: > > On Mon, Apr 25, 2022 at 1:11 AM Michael Paquier wrote: > >> I don't think that there is any need to rely on a new logic if there > >> is

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-26 Thread David Christensen
On Mon, Apr 25, 2022 at 9:42 PM Michael Paquier wrote: > On Mon, Apr 25, 2022 at 10:24:52AM -0500, David Christensen wrote: > > On Mon, Apr 25, 2022 at 6:03 AM Bharath Rupireddy > > wrote: > >> Thanks for working on this. I'm just thinking if we can use these FPIs

Re: Moving forward with TDE [PATCH v3]

2023-10-31 Thread David Christensen
On Tue, Oct 31, 2023 at 4:30 PM Bruce Momjian wrote: > On Tue, Oct 31, 2023 at 04:23:17PM -0500, David Christensen wrote: > > Greetings, > > > > I am including an updated version of this patch series; it has been > rebased > > onto 6ec62b7799 and reworked somewh

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread David Christensen
Hi, thanks for the detailed feedback here. I do think it's worth addressing the question Stephen raised as far as what we use for the IV[1]; whether LSN or something else entirely, and if so what. The choice of LSN here is fairly fundamental to the existing implementation, so if we decide to do s

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread David Christensen
On Fri, Nov 3, 2023 at 9:53 PM Andres Freund wrote: > On 2023-11-02 19:32:28 -0700, Andres Freund wrote: > > > From 327e86d52be1df8de9c3a324cb06b85ba5db9604 Mon Sep 17 00:00:00 2001 > > > From: David Christensen > > > Date: Fri, 29 Sep 2023 15:16:00 -0400 >

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 6:47 PM Andres Freund wrote: > Hi, > > On 2023-11-06 11:26:44 +0100, Matthias van de Meent wrote: > > On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > > > I'm quite surprised at the significant number of

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 5:49 PM Andres Freund wrote: > Hi, > > On 2023-11-06 09:56:37 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > I still am quite quite unconvinced that using the LSN as a nonce is a > good > > > design decision. > > > > This is a really impor

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread David Christensen
On Wed, Nov 8, 2023 at 8:04 AM Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2023-05-09 17:08:26 -0500, David Christensen wrote: > > > From 965309ea3517fa734c4bc89c144e2031cdf6c0c3 Mon Sep 17 00:00:00 2001 > > > From: D

Re: [PATCHES] Post-special page storage TDE support

2023-11-13 Thread David Christensen
On Tue, Nov 7, 2023 at 6:20 PM Andres Freund wrote: > Hi, > > On 2023-05-09 17:08:26 -0500, David Christensen wrote: > > From 965309ea3517fa734c4bc89c144e2031cdf6c0c3 Mon Sep 17 00:00:00 2001 > > From: David Christensen > > Date: Tue, 9 May 2023 16:56:15 -0500 >

Re: [PATCHES] Post-special page storage TDE support

2023-11-13 Thread David Christensen
On Mon, Nov 13, 2023 at 2:27 PM Andres Freund wrote: > Hi, > > On 2023-11-08 18:47:56 -0800, Peter Geoghegan wrote: > > On Wed, Nov 8, 2023 at 6:04 AM Stephen Frost wrote: > > > In conversations with folks (my memory specifically is a discussion > with > > > Peter G, added to CC, and my apologie

Re: [PATCHES] Post-special page storage TDE support

2023-11-13 Thread David Christensen
On Mon, Nov 13, 2023 at 2:52 PM Andres Freund wrote: > > > This scheme would open up space per page that would now be available for > > plenty of other things; the encoding in the header and the corresponding > > available space in the footer would seem to open up quite a few options > > now, no?

Re: [PATCHES] Post-special page storage TDE support

2023-11-29 Thread David Christensen
On Tue, Nov 7, 2023 at 6:20 PM Andres Freund wrote: > Hi, > > - IMO the patch touches many places it shouldn't need to touch, because of > essentially renaming a lot of existing macro names to *Limit, > necessitating modifying a lot of users. I think instead the few places > that > care abo

Re: [PATCHES] Post-special page storage TDE support

2023-12-22 Thread David Christensen
Hi again! Per some offline discussion with Stephen, I've continued to work on some modifications here; this particular patchset is intended to facilitate review by highlighting the mechanical nature of many of these changes. As such, I have taken the following approach to this rework: 0001 - Cre

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
Enclosed are TPC-H results for 1GB shared_buffers, 64MB work_mem on a 64GB laptop with SSD storage; everything else is default settings. TL;DR: unpatched version: 17.30 seconds, patched version: 17.15; there are some slight variations in runtime, but seems to be within the noise level at this poin

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
> + * pg_fastmod - calculates the modulus of a 32-bit number against a constant > + * divisor without using the division operator > + */ > +static inline uint32 pg_fastmod(uint32 n, uint32 divisor, uint64 fastinv) > +{ > +#ifdef HAVE_INT128 > + uint64_t lowbits = fastinv * n; > + return ((uint128)l

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
> I was definitely hand-waving additional implementation here for > non-native 128 bit support; the modulus algorithm as presented > requires 4 times the space as the divisor, so a uint16 implementation > should work for all 64-bit machines. Certainly open to other ideas or > implementations, this

Re: Initdb-time block size specification

2023-09-05 Thread David Christensen
On Tue, Sep 5, 2023 at 9:04 AM Robert Haas wrote: > > On Sat, Sep 2, 2023 at 3:09 PM Tomas Vondra > wrote: > > Except that no one is forcing you to actually go 130mph or 32mph, right? > > You make it seem like this patch forces people to use some other page > > size, but that's clearly not what i

Re: Initdb-time block size specification

2023-09-05 Thread David Christensen
On Tue, Sep 5, 2023 at 2:52 PM Hannu Krosing wrote: > > Something I also asked at this years Unconference - Do we currently > have Build Farm animals testing with different page sizes ? > > I'd say that testing all sizes from 4KB up (so 4, 8, 16, 32) should be > done at least before each release i

Re: DELETE CASCADE

2022-01-31 Thread David Christensen
On Sun, Jan 30, 2022 at 9:47 PM Julien Rouhaud wrote: > Hi, > > On Tue, Jan 25, 2022 at 10:26:53PM +0800, Julien Rouhaud wrote: > > > > It's been almost 4 months since your last email, and almost 2 weeks > since the > > notice that this patch doesn't apply anymore. Without update in the next > >

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-04 Thread David Christensen
On Fri, Nov 4, 2022 at 1:38 PM Justin Pryzby wrote: > > > As I recall, that's due to relying on "cp". And "rsync", which > > > shouldn't be assumed to exist by regression tests). Will poke around other TAP tests to see if there's a more consistent interface, what perl version we can assume and a

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-07 Thread David Christensen
Hi Justin et al, Enclosed is v5 of this patch which now passes the CirrusCI checks for all supported OSes. I went ahead and reworked the test a bit so it's a little more amenable to the OS-agnostic approach for testing. Best, David v5-0001-Teach-pg_waldump-to-extract-FPIs-from-the-WAL-str.patc

Re: [PATCHES] Post-special page storage TDE support

2022-11-08 Thread David Christensen
Looking into some CF bot failures which didn't show up locally. Will send a v3 when resolved.

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-08 Thread David Christensen
On Tue, Nov 8, 2022 at 4:45 PM Justin Pryzby wrote: > > On Mon, Nov 07, 2022 at 05:01:01PM -0600, David Christensen wrote: > > Hi Justin et al, > > > > Enclosed is v5 of this patch which now passes the CirrusCI checks for > > all supported OSes. I went ahead and rew

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-08 Thread David Christensen
Enclosed is v6, which squashes your refactor and adds the additional recent suggestions. Thanks! v6-0001-Teach-pg_waldump-to-extract-FPIs-from-the-WAL-str.patch Description: Binary data

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-09 Thread David Christensen
On Wed, Nov 9, 2022 at 6:30 AM Bharath Rupireddy wrote: > > On Wed, Nov 9, 2022 at 5:08 AM David Christensen > wrote: > > > > Enclosed is v6, which squashes your refactor and adds the additional > > recent suggestions. > > Thanks for working on this feature.

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-09 Thread David Christensen
> > 6. > > +if (dir_status == 0 && mkdir(config.save_fpw_path, 0700) < 0) > > Use pg_dir_create_mode instead of hard-coded 0007? > > I think I thought of that when I first looked at the patch ... but, I'm > not sure, since it says: > > src/include/common/file_perm.h-/* Modes for creating d

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-09 Thread David Christensen
On Wed, Nov 9, 2022 at 2:08 PM David Christensen wrote: > Justin sez: > > I was wondering if there's any reason to do "CREATE DATABASE". The vast > > majority of TAP tests don't. > > > > $ git grep -ho 'safe_psql[^ ]*' '*pl&#

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-10 Thread David Christensen
On Thu, Nov 10, 2022 at 2:33 AM Bharath Rupireddy wrote: > > On Thu, Nov 10, 2022 at 1:31 AM David Christensen > wrote: > > > > Thanks for providing the v7 patch, please see my comments and responses below. Hi Bharath, Thanks for the feedback. > > > 2. I'm un

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-11 Thread David Christensen
On Fri, Nov 11, 2022 at 8:15 AM Justin Pryzby wrote: > > On Wed, Nov 09, 2022 at 02:37:29PM -0600, David Christensen wrote: > > On Wed, Nov 9, 2022 at 2:08 PM David Christensen > > wrote: > > > Justin sez: > > > > I was wondering if there's a

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-14 Thread David Christensen
On Fri, Nov 11, 2022 at 4:57 AM Bharath Rupireddy wrote: > > Well if it's not the same output then I guess you're right and there's > > not a use for the `--fixup` mode. By the same token, I'd say > > calculating/setting the checksum also wouldn't need to be done, we > > should just include the p

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-14 Thread David Christensen
Enclosed is v8, which uses the replication slot method to retain WAL as well as fsync'ing the output directory when everything is done. v8-0001-Teach-pg_waldump-to-extract-FPIs-from-the-WAL-str.patch Description: Binary data

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-15 Thread David Christensen
On Tue, Nov 15, 2022 at 4:41 AM Bharath Rupireddy wrote: > > On Tue, Nov 15, 2022 at 1:29 AM David Christensen > wrote: > > > > Enclosed is v8, which uses the replication slot method to retain WAL > > as well as fsync'ing the output directory when everything is d

Re: Moving forward with TDE

2022-11-15 Thread David Christensen
> On Nov 15, 2022, at 1:08 PM, Jacob Champion wrote: > > On Mon, Oct 24, 2022 at 9:29 AM David Christensen > wrote: >> I would love to open a discussion about how to move forward and get >> some of these features built out. The historical threads here are >>

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-15 Thread David Christensen
Enclosed is v9. - code style consistency (FPI instead of FPW) internally. - cleanup of no-longer needed checksum-related pieces from code and tests. - test cleanup/simplification. - other comment cleanup. Passes all CI checks. Best, David v9-0001-Teach-pg_waldump-to-extract-FPIs-from-the-WAL-

ScanSourceDatabasePgClass

2022-11-16 Thread David Christensen
Hi Robert, In 9c08aea6a you introduce the block-by-block strategy for creating a copy of the database. In the main loop, this utilizes this call: buf = ReadBufferWithoutRelcache(rlocator, MAIN_FORKNUM, blkno, RBM_NORMAL, bstrategy, false); Here, the last parameter is "false" for the permanence

Re: Moving forward with TDE

2022-11-17 Thread David Christensen
Hi Jacob, Thanks, I've added this patch in my tree [1]. (For now, just adding fixes and the like atop the original separate patches, but will eventually get things winnowed down into probably the same 12 parts the originals were reviewed in. Best, David [1] https://github.com/pgguru/postgres/tr

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-17 Thread David Christensen
On Wed, Nov 16, 2022 at 3:30 AM Bharath Rupireddy wrote: > 1. > -if (config.filter_by_fpw && !XLogRecordHasFPW(xlogreader_state)) > +if (config.filter_by_fpw && !XLogRecordHasFPI(xlogreader_state)) > These changes are not related to this feature, hence renaming those > variables/fu

Re: Moving forward with TDE

2022-11-17 Thread David Christensen
Hi Dilip, Thanks for the feedback here. I will review the docs changes and add to my tree. Best, David

[PATCH] expand the units that pg_size_pretty supports on output

2021-04-14 Thread David Christensen
Hi folks, Enclosed is a patch that expands the unit output for pg_size_pretty(numeric) going up to Yottabytes; I reworked the existing numeric output code to account for the larger number of units we're using rather than just adding nesting levels. There are also a few other places that could ben

Issue in recent pg_stat_statements?

2021-04-26 Thread David Christensen
-hackers, So in doing some recent work on pg_stat_statements, I notice that while the regression test still passes on HEAD, it appears that 4f0b096 (per git bisect) changed/broke how this works compared to historical versions. Essentially, when doing a fresh install of pg_stat_statements on a new

Re: Issue in recent pg_stat_statements?

2021-04-26 Thread David Christensen
> > > Is this an expected change, or is this in fact broken? In previous > revisions, this was showing the INSERT and SELECT at the very least. I'm > unclear as to why the regression test is still passing, so want to verify > that I'm not doing something wrong in the testing. > > Yes, you want to

Re: Issue in recent pg_stat_statements?

2021-04-26 Thread David Christensen
On Mon, Apr 26, 2021 at 12:18 PM Julien Rouhaud wrote: > On Mon, Apr 26, 2021 at 11:40 PM David Christensen > wrote: > >> > >> > Is this an expected change, or is this in fact broken? In previous > revisions, this was showing the INSERT and SELECT at the very l

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-04-28 Thread David Christensen
A second patch to teach the same units to pg_size_bytes(). Best, David On Wed, Apr 14, 2021 at 11:13 AM David Christensen < david.christen...@crunchydata.com> wrote: > Hi folks, > > Enclosed is a patch that expands the unit output for > pg_size_pretty(numeric) going u

[PATCH] comment fixes for delayChkptFlags

2022-10-14 Thread David Christensen
Enclosed is a trivial fix for a typo and misnamed field I noted when doing some code review. Best, David 0001-fix-comment-typos-for-delayChkptFlags.patch Description: Binary data

Re: [PATCHES] Post-special page storage TDE support

2022-10-25 Thread David Christensen
> > Explicitly > > locking (assuming you stay in your lane) should only need to guard > > against access from other > > backends of this type if using shared buffers, so will be use-case > > dependent. > > I'm not sure what you mean here? I'm mainly pointing out that the specific code that manage

Re: [PATCHES] Post-special page storage TDE support

2022-10-28 Thread David Christensen
Hi Matthias, > Did you read the related thread with related discussion from last June, "Re: > better page-level checksums" [0]? In that I argued that space at the end of a > page is already allocated for the AM, and that reserving variable space at > the end of the page for non-AM usage is wast

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-04 Thread David Christensen
On Nov 4, 2022, at 9:02 AM, Justin Pryzby wrote: > > On Fri, Nov 04, 2022 at 11:52:59AM +0900, Ian Lawrence Barwick wrote: >> 2022年5月3日(火) 8:45 David Christensen : >>> >>> ...and pushing a couple fixups pointed out by cfbot, so here's v4. >> >&g

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-06-03 Thread David Christensen
New versions attached to address the initial CF feedback and rebase on HEAD as of now. 0001-Expand-the-units-that-pg_size_pretty-numeric-knows-a.patch - expands the units that pg_size_pretty() can handle up to YB. 0002-Expand-the-supported-units-in-pg_size_bytes-to-cover.patch - expands the uni

  1   2   >