Re: Reducing output size of nodeToString

2024-03-22 Thread Peter Eisentraut
On 20.03.24 13:03, Matthias van de Meent wrote: On Wed, 20 Mar 2024 at 12:49, Peter Eisentraut wrote: On 19.03.24 17:13, Peter Eisentraut wrote: On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe m

Re: Reducing output size of nodeToString

2024-03-20 Thread Matthias van de Meent
On Wed, 20 Mar 2024 at 12:49, Peter Eisentraut wrote: > > On 19.03.24 17:13, Peter Eisentraut wrote: > > On 11.03.24 21:52, Matthias van de Meent wrote: > >>> * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch > >>> > >>> This looks sensible, but maybe making Location a global type

Re: Reducing output size of nodeToString

2024-03-20 Thread Peter Eisentraut
On 19.03.24 17:13, Peter Eisentraut wrote: On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe making Location a global type is a bit much?  Maybe something more specific like ParseLocation, or ParseLoc

Re: Reducing output size of nodeToString

2024-03-19 Thread Matthias van de Meent
On Tue, 19 Mar 2024 at 17:13, Peter Eisentraut wrote: > > On 11.03.24 21:52, Matthias van de Meent wrote: > >> * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch > >> > >> This looks sensible, but maybe making Location a global type is a bit > >> much? Maybe something more specific

Re: Reducing output size of nodeToString

2024-03-19 Thread Peter Eisentraut
On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe making Location a global type is a bit much? Maybe something more specific like ParseLocation, or ParseLoc, to keep it under 12 characters. I've gone

Re: Reducing output size of nodeToString

2024-03-11 Thread Matthias van de Meent
On Mon, 11 Mar 2024 at 14:19, Peter Eisentraut wrote: > > On 22.02.24 16:07, Matthias van de Meent wrote: > > On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent > > wrote: > >> > >> On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent > >> wrote: > >>> Attached the updated version of the patch o

Re: Reducing output size of nodeToString

2024-03-11 Thread Peter Eisentraut
On 22.02.24 16:07, Matthias van de Meent wrote: On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent wrote: On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent wrote: Attached the updated version of the patch on top of 5497daf3, which incorporates this last round of feedback. Now attached r

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent wrote: > > On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent > wrote: > > Attached the updated version of the patch on top of 5497daf3, which > > incorporates this last round of feedback. > > Now attached rebased on top of 93db6cbd to fix conf

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent wrote: > Attached the updated version of the patch on top of 5497daf3, which > incorporates this last round of feedback. Now attached rebased on top of 93db6cbd to fix conflicts with fbc93b8b and an issue in the previous patchset: I attached one

Re: Reducing output size of nodeToString

2024-02-15 Thread Matthias van de Meent
On Thu, 15 Feb 2024 at 13:59, Peter Eisentraut wrote: > > Thanks, this patch set is a good way to incrementally work through these > changes. > > I have looked at > v4-0001-pg_node_tree-Omit-serialization-of-fields-with-de.patch today. > Here are my thoughts: > > I believe we had discussed offline

Re: Reducing output size of nodeToString

2024-02-15 Thread Peter Eisentraut
Thanks, this patch set is a good way to incrementally work through these changes. I have looked at v4-0001-pg_node_tree-Omit-serialization-of-fields-with-de.patch today. Here are my thoughts: I believe we had discussed offline to not omit enum fields with value 0 (WRITE_ENUM_FIELD). This i

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Mon, 12 Feb 2024 at 20:32, Matthias van de Meent wrote: > > On Mon, 12 Feb 2024 at 19:03, Matthias van de Meent > wrote: > > Attached is patchset v2, which contains the improvements from these patches: > > Attached v3, which fixes an out-of-bounds read in pg_strtoken_next, > detected by asan,

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Mon, 12 Feb 2024 at 19:03, Matthias van de Meent wrote: > Attached is patchset v2, which contains the improvements from these patches: Attached v3, which fixes an out-of-bounds read in pg_strtoken_next, detected by asan, that was a likely cause of the problems in CFBot's FreeBSD regression tes

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Wed, 31 Jan 2024 at 18:47, Robert Haas wrote: > > On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent > wrote: > > I was also thinking about smaller per-attribute expression storage, for > > index attribute expressions, table default expressions, and functions. > > Other than that, less m

Re: Reducing output size of nodeToString

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent wrote: > I was also thinking about smaller per-attribute expression storage, for index > attribute expressions, table default expressions, and functions. Other than > that, less memory overhead for the serialized form of these constructs als

Re: Reducing output size of nodeToString

2024-01-31 Thread Matthias van de Meent
On Wed, 31 Jan 2024, 09:16 Peter Eisentraut, wrote: > On 30.01.24 12:26, Matthias van de Meent wrote: > >> Most of the other defaults I'm doubtful about. First, we are colliding > >> here between the goals of minimizing the storage size and making the > >> debug output more readable. > > I've ne

Re: Reducing output size of nodeToString

2024-01-31 Thread Peter Eisentraut
On 30.01.24 12:26, Matthias van de Meent wrote: Most of the other defaults I'm doubtful about. First, we are colliding here between the goals of minimizing the storage size and making the debug output more readable. I've never really wanted to make the output "more readable". The current one is

Re: Reducing output size of nodeToString

2024-01-30 Thread Matthias van de Meent
On Tue, 9 Jan 2024, 09:23 Peter Eisentraut, wrote: > > On 04.01.24 00:23, Matthias van de Meent wrote: > > Something like the attached? It splits out into the following > > 0001: basic 'omit default values' > > /* Write an integer field (anything written as ":fldname %d") */ > -#define WRITE_INT

Re: Reducing output size of nodeToString

2024-01-09 Thread Peter Eisentraut
On 04.01.24 00:23, Matthias van de Meent wrote: Something like the attached? It splits out into the following 0001: basic 'omit default values' /* Write an integer field (anything written as ":fldname %d") */ -#define WRITE_INT_FIELD(fldname) \ +#define WRITE_INT_FIELD_DIRECT(fldname) \

Re: Reducing output size of nodeToString

2024-01-03 Thread Matthias van de Meent
On Tue, 2 Jan 2024 at 11:30, Peter Eisentraut wrote: > > On 06.12.23 22:08, Matthias van de Meent wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to 472KiB. Th

Re: Reducing output size of nodeToString

2024-01-03 Thread Matthias van de Meent
On Wed, 3 Jan 2024 at 03:02, David Rowley wrote: > > On Thu, 14 Dec 2023 at 19:21, Matthias van de Meent > wrote: > > > > On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > > We could also easily serialize plans to binary format for copying to > > > parallel workers rather than converting them

Re: Reducing output size of nodeToString

2024-01-02 Thread David Rowley
On Thu, 14 Dec 2023 at 19:21, Matthias van de Meent wrote: > > On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > We could also easily serialize plans to binary format for copying to > > parallel workers rather than converting them to a text-based > > serialized format. It would also allow us t

Re: Reducing output size of nodeToString

2024-01-02 Thread Peter Eisentraut
On 06.12.23 22:08, Matthias van de Meent wrote: PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its

Re: Reducing output size of nodeToString

2023-12-13 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > On Thu, 7 Dec 2023 at 10:09, Matthias van de Meent > wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to 4

Re: Reducing output size of nodeToString

2023-12-07 Thread David Rowley
On Thu, 7 Dec 2023 at 10:09, Matthias van de Meent wrote: > PFA a patch that reduces the output size of nodeToString by 50%+ in > most cases (measured on pg_rewrite), which on my system reduces the > total size of pg_rewrite by 33% to 472KiB. This does keep the textual > pg_node_tree format alive,

Re: Reducing output size of nodeToString

2023-12-07 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 11:26, Peter Eisentraut wrote: > > On 06.12.23 22:08, Matthias van de Meent wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to 472KiB. Th

Re: Reducing output size of nodeToString

2023-12-07 Thread Peter Eisentraut
On 06.12.23 22:08, Matthias van de Meent wrote: PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its

Reducing output size of nodeToString

2023-12-06 Thread Matthias van de Meent
Hi, PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its size signficantly. The basic techniques used