Re: Memory consumed by paths during partitionwise join planning

2025-07-18 Thread Ashutosh Bapat
On Mon, Jul 7, 2025 at 8:43 PM Andrei Lepikhov wrote: > > On 27/6/2025 12:01, Andrei Lepikhov wrote: > > On 6/2/2024 13:51, Ashutosh Bapat wrote: > >> On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > >> wrote: > >> First patch is no longer required. Here's rebased set > >> > >> The patches are ra

Re: Memory consumed by paths during partitionwise join planning

2025-07-07 Thread Andrei Lepikhov
On 27/6/2025 12:01, Andrei Lepikhov wrote: On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: First patch is no longer required. Here's rebased set The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this

Re: Memory consumed by paths during partitionwise join planning

2025-06-27 Thread Andrei Lepikhov
On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: First patch is no longer required. Here's rebased set The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this is useful and whether the design is on the

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Andrei Lepikhov
On 19/9/2024 13:12, Ashutosh Bapat wrote: On Thu, Sep 19, 2024 at 4:18 PM Andrei Lepikhov wrote: At the same time, this technique (while highly useful in general) adds fragility and increases complexity: a developer needs to remember to link the path using the pointer in different places of the

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Ashutosh Bapat
On Thu, Sep 19, 2024 at 4:18 PM Andrei Lepikhov wrote: > > On 6/2/2024 13:51, Ashutosh Bapat wrote: > > On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > > wrote: > > > >>> > >>> That looks pretty small considering the benefits. What do you think? > >>> > >>> [1] > >>> https://www.postgresql.org/

Re: Memory consumed by paths during partitionwise join planning

2024-09-19 Thread Andrei Lepikhov
On 6/2/2024 13:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: That looks pretty small considering the benefits. What do you think? [1] https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com If you want to e

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Tue, Feb 20, 2024 at 8:19 AM Andrei Lepikhov wrote: > > On 19/2/2024 19:25, Ashutosh Bapat wrote: > > On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov > > wrote: > >> Live example: right now, I am working on the code like MSSQL has - a > >> combination of NestLoop and HashJoin paths and switchi

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list at th

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: > Live example: right now, I am working on the code like MSSQL has - a > combination of NestLoop and HashJoin paths and switching between them in > real-time. It requires both paths in the path list at the moment when > extensions are coming.

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 19:06, Ashutosh Bapat wrote: On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov But I'm not sure about freeing unreferenced paths. I would have to see alternatives in the pathlist. I didn't understand this. Can you please elaborate? A path in any pathlist is referenced. An unreferenc

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Ashutosh Bapat
On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov wrote: > > On 6/2/2024 19:51, Ashutosh Bapat wrote: > > On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > > The patches are raw. make check has some crashes that I need to fix. I > > am waiting to hear whether this is useful and whether the design is

Re: Memory consumed by paths during partitionwise join planning

2024-02-14 Thread Andrei Lepikhov
On 6/2/2024 19:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this is useful and whether the design is on the right track. Let me write words of opinion on that feature. I

Re: Memory consumed by paths during partitionwise join planning

2024-02-06 Thread Ashutosh Bapat
On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: > > > > That looks pretty small considering the benefits. What do you think? > > > > [1] > > https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com > > If you want to experiment, please use at

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
Forgot to mention, On Thu, Dec 14, 2023 at 5:34 PM Ashutosh Bapat wrote: > > On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > > > Maybe we can try to move forward with your refcount idea and see how > > the performance looks. If that's intolerable then that might help us > > decide on the

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > Maybe we can try to move forward with your refcount idea and see how > the performance looks. If that's intolerable then that might help us > decide on the next best alternative solution. > Here are performance numbers setup create table t

Re: Memory consumed by paths during partitionwise join planning

2023-12-08 Thread Ashutosh Bapat
On Fri, Dec 8, 2023 at 1:02 PM David Rowley wrote: > > On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat > wrote: > > given path. E.g. we have three path chains as follows > > 1. joinpath_1->joinpath_2->seqpath_1, > > 2. joinpath_3->joinpath_4->seqpath_1, > > 3. joinpath_5->joinpath_2->seqpath_1 > > >

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat wrote: > given path. E.g. we have three path chains as follows > 1. joinpath_1->joinpath_2->seqpath_1, > 2. joinpath_3->joinpath_4->seqpath_1, > 3. joinpath_5->joinpath_2->seqpath_1 > > Please note that this is not full path tree/forest. It is difficult

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat > wrote: > > I am fine to work on this further if the community thinks it is > > acceptable. It seems from your point of view the worth of this work is > > as follows > > a. memory savings - not wor

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat wrote: > I am fine to work on this further if the community thinks it is > acceptable. It seems from your point of view the worth of this work is > as follows > a. memory savings - not worth it > b. getting rid of !IsA(old_path, IndexPath) - worth it > c

Re: Memory consumed by paths during partitionwise join planning

2023-11-30 Thread Ashutosh Bapat
On Wed, Nov 29, 2023 at 1:10 AM David Rowley wrote: > > On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat > wrote: > > Table 1: Join between unpartitioned tables > > Number of tables | without patch | with patch | % reduction | > > being joined ||| | > > --

Re: Memory consumed by paths during partitionwise join planning

2023-11-28 Thread David Rowley
On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat wrote: > Table 1: Join between unpartitioned tables > Number of tables | without patch | with patch | % reduction | > being joined ||| | > -- >

Memory consumed by paths during partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
Hi All, If add_path() and add_partial_path() do not find a new path to be superior to any existing paths, they free the new path. They free an existing path if it is found to be inferior to the new path. But not all paths surviving in a RelOptInfo are used to create paths for relations which use i