Re: Showing applied extended statistics in explain Part 2

2025-04-24 Thread Andrei Lepikhov
On 2/10/25 16:56, Tomas Vondra wrote: On 2/10/25 10:09, Andrei Lepikhov wrote: On 8/2/2025 20:50, Tomas Vondra wrote: The main profit here - you see all the stats involved in estimations (and their state), even if final plan doesn't contain estimated stuff at all. OK, that seems very underwh

Re: Showing applied extended statistics in explain Part 2

2025-02-12 Thread Tomas Vondra
On 2/12/25 06:02, Tatsuro Yamada wrote: > Hi Tomas and ALL, > >>I wonder what Yamada-san thinks about these suggestions ... He's the one >>actually developing the patch, so I'd like to know his opinions. > > I will state my thoughts on the two points of discussion. > > I often use SQL Server

Re: Showing applied extended statistics in explain Part 2

2025-02-11 Thread Tatsuro Yamada
Hi Tomas and ALL, >I wonder what Yamada-san thinks about these suggestions ... He's the one >actually developing the patch, so I'd like to know his opinions. I will state my thoughts on the two points of discussion. I often use SQL Server to compare execution plans generated by Postgre

Re: Showing applied extended statistics in explain Part 2

2025-02-10 Thread Tomas Vondra
On 2/10/25 10:09, Andrei Lepikhov wrote: > On 8/2/2025 20:50, Tomas Vondra wrote: >> >> >> On 1/24/25 11:17, Andrei Lepikhov wrote: >>> On 11/1/24 12:22, Tatsuro Yamada wrote: >>> I often use SQL Server to compare execution plans generated by >>> PostgreSQL, and I appreciate how they display the

Re: Showing applied extended statistics in explain Part 2

2025-02-10 Thread Andrei Lepikhov
On 8/2/2025 20:50, Tomas Vondra wrote: On 1/24/25 11:17, Andrei Lepikhov wrote: On 11/1/24 12:22, Tatsuro Yamada wrote: I often use SQL Server to compare execution plans generated by PostgreSQL, and I appreciate how they display the usage of extended statistics. They clearly identify which sta

Re: Showing applied extended statistics in explain Part 2

2025-02-09 Thread Tatsuro Yamada
Hi All, I've organized the discussion so far and improved the patch. The issues and their status are below. * Issues and status (or comment): I've numbered them for ease of management. T6. Changed option to show extended statistics (from VERBOSE to STATS) -> Done already on the previous

Re: Showing applied extended statistics in explain Part 2

2025-02-08 Thread Tomas Vondra
On 1/24/25 11:17, Andrei Lepikhov wrote: > On 11/1/24 12:22, Tatsuro Yamada wrote: >> Hi All, >> >> I apologize for not being able to continue development due to various >> circumstances. >> The attached file is the rebased patch. >> I will now catch up on the discussion and try to revise the pa

Re: Showing applied extended statistics in explain Part 2

2025-02-05 Thread Ilia Evdokimov
On 05.02.2025 09:28, Tatsuro Yamada wrote: Hi All, Thank you everyone for your cooperation with comments on the patch and solution ideas. I am sorting through your review comments now. And after rebasing the patch, I plan to send a patch that addresses the comments as much as possible to -h

Re: Showing applied extended statistics in explain Part 2

2025-02-04 Thread Tatsuro Yamada
Hi All, Thank you everyone for your cooperation with comments on the patch and solution ideas. I am sorting through your review comments now. And after rebasing the patch, I plan to send a patch that addresses the comments as much as possible to -hackers by Feb 21 at the latest. Therefore, the st

Re: Showing applied extended statistics in explain Part 2

2025-01-24 Thread Andrei Lepikhov
On 11/1/24 12:22, Tatsuro Yamada wrote: Hi All, I apologize for not being able to continue development due to various circumstances. The attached file is the rebased patch. I will now catch up on the discussion and try to revise the patch. I wonder why it’s so important to know exactly where a

Re: Showing applied extended statistics in explain Part 2

2024-12-02 Thread Tom Lane
Ilia Evdokimov writes: > On 19.11.2024 00:38, Tomas Vondra wrote: >> On 11/18/24 22:15, Tomas Vondra wrote: >>> So I think the correct solution is to not pass any expressions with >>> RestrictInfo to deparse_expression(). Either by stripping the nodes, or >>> by not adding them at all. >>> >>> Th

Re: Showing applied extended statistics in explain Part 2

2024-12-02 Thread Ilia Evdokimov
On 19.11.2024 00:38, Tomas Vondra wrote: On 11/18/24 22:15, Tomas Vondra wrote: ... So I think the correct solution is to not pass any expressions with RestrictInfo to deparse_expression(). Either by stripping the nodes, or by not adding them at all. The patch tries to do the stripping by ma

Re: Showing applied extended statistics in explain Part 2

2024-11-18 Thread Tomas Vondra
On 11/18/24 22:15, Tomas Vondra wrote: > ... > > So I think the correct solution is to not pass any expressions with > RestrictInfo to deparse_expression(). Either by stripping the nodes, or > by not adding them at all. > > The patch tries to do the stripping by maybe_extract_actual_clauses(), > b

Re: Showing applied extended statistics in explain Part 2

2024-11-18 Thread Tomas Vondra
On 11/18/24 13:52, Ilia Evdokimov wrote: > Hi everyone! > > Thank you for your work. > > 1) While exploring extended statistics, I encountered a bug that occurs > when using EXPLAIN (STATS) with queries containing OR conditions: > > CREATE TABLE t (a int, b int, c int, d int); > INSERT INTO t SE

Re: Showing applied extended statistics in explain Part 2

2024-11-18 Thread Ilia Evdokimov
Hi everyone! Thank you for your work. 1) While exploring extended statistics, I encountered a bug that occurs when using EXPLAIN (STATS) with queries containing OR conditions: CREATE TABLE t (a int, b int, c int, d int); INSERT INTO t SELECT x/10+1, x, x + 10, x * 2 FROM generate_series(1,10

Re: Showing applied extended statistics in explain Part 2

2024-10-31 Thread Tatsuro Yamada
Hi All, I apologize for not being able to continue development due to various circumstances. The attached file is the rebased patch. I will now catch up on the discussion and try to revise the patch. Regards, Tatsuro Yamada On Fri, Jul 19, 2024 at 7:17 PM wrote: > > On 7/18/24 12:37, masahiro

RE: Showing applied extended statistics in explain Part 2

2024-07-19 Thread Masahiro.Ikeda
> On 7/18/24 12:37, masahiro.ik...@nttdata.com wrote: > >> Let me share my opinion on those questions ... > > ...> > >> For ndistinct, I think we don't show this because it doesn't go > >> through clauselist_selectivity, which is the only thing I modified in the > >> PoC. > >> But I guess we might

Re: Showing applied extended statistics in explain Part 2

2024-07-18 Thread Tomas Vondra
On 7/18/24 12:37, masahiro.ik...@nttdata.com wrote: >> Let me share my opinion on those questions ... > ...> >> For ndistinct, I think we don't show this because it doesn't go through >> clauselist_selectivity, which is the only thing I modified in the PoC. >> But I guess we might improve estimate_

RE: Showing applied extended statistics in explain Part 2

2024-07-18 Thread Masahiro.Ikeda
> Let me share my opinion on those questions ... Thanks! I could understand the patch well thanks to your comments. > On 7/12/24 12:09, masahiro.ik...@nttdata.com wrote: > > Is it better to make the order of output consistent? For example, even > > though there are three clauses shown in the be

Re: Showing applied extended statistics in explain Part 2

2024-07-15 Thread Tomas Vondra
Hi, Let me share my opinion on those questions ... On 7/12/24 12:09, masahiro.ik...@nttdata.com wrote: > Hi, > > Thanks for working the feature. As a user, I find it useful, and I'd like to > use > it in v18! Although I've just started start looking into it, I have a few > questions. > > >

Re: Showing applied extended statistics in explain Part 2

2024-07-15 Thread Tomas Vondra
On 6/28/24 13:16, Tatsuro Yamada wrote: > Hi Tomas and All, > > Attached file is a new patch including: > 6) Add stats option to explain command > 7) The patch really needs some docs (partly) > > >4) Add new node (resolve errors in cfbot and prepared statement) > > I tried adding a new

Re: Showing applied extended statistics in explain Part 2

2024-07-15 Thread Tomas Vondra
On 6/26/24 11:06, Tatsuro Yamada wrote: > Hi Tomas! > > Thanks for the comments! > > 1) The patch is not added to the CF app, which I think is a mistake. Can >> you please add it to the 2024-07 commitfest? Otherwise people may not be >> aware of it, won't do reviews etc. It'll require posting

RE: Showing applied extended statistics in explain Part 2

2024-07-12 Thread Masahiro.Ikeda
Hi, Thanks for working the feature. As a user, I find it useful, and I'd like to use it in v18! Although I've just started start looking into it, I have a few questions. (1) Is it better to make the order of output consistent? For example, even though there are three clauses shown in the below

Re: Showing applied extended statistics in explain Part 2

2024-06-28 Thread Tatsuro Yamada
Hi Tomas and All, Attached file is a new patch including: 6) Add stats option to explain command 7) The patch really needs some docs (partly) >4) Add new node (resolve errors in cfbot and prepared statement) I tried adding a new node in pathnode.h, but it doesn't work well. So, it needs

Re: Showing applied extended statistics in explain Part 2

2024-06-27 Thread Tatsuro Yamada
pplied-extended-statistics-in-explain-r3.patch Description: Binary data

Re: Showing applied extended statistics in explain Part 2

2024-06-26 Thread Tatsuro Yamada
Hi Tomas! Thanks for the comments! 1) The patch is not added to the CF app, which I think is a mistake. Can > you please add it to the 2024-07 commitfest? Otherwise people may not be > aware of it, won't do reviews etc. It'll require posting a rebased > patch, but should not be a big deal. > I a

Re: Showing applied extended statistics in explain Part 2

2024-06-12 Thread Tomas Vondra
Hello Yamada-san, I finally got to look at this patch again - apologies for taking so long, I'm well aware it's rather frustrating to wait for feedback. I'll try to pay more attention to this patch, and don't hesitate to ping me off-list if immediate input is needed. I looked at the patch from Ma

Re: Showing applied extended statistics in explain Part 2

2024-03-01 Thread Tomas Vondra
On 3/1/24 01:19, Tatsuro Yamada wrote: > Hi, > > This original patch made by Tomas improves the usability of extended > statistics, > so I rebased it on 362de947, and I'd like to re-start developing it. > > The previous thread [1] suggested something to solve. I'll try to solve it as > best I

Showing applied extended statistics in explain Part 2

2024-02-29 Thread Tatsuro Yamada
Hi, This original patch made by Tomas improves the usability of extended statistics, so I rebased it on 362de947, and I'd like to re-start developing it. The previous thread [1] suggested something to solve. I'll try to solve it as best I can, but I think this feature is worth it with some li

Re: Showing applied extended statistics in explain

2022-08-01 Thread Jacob Champion
As discussed in [1], we're taking this opportunity to return some patchsets that don't appear to be getting enough reviewer interest. This is not a rejection, since we don't necessarily think there's anything unacceptable about the entry, but it differs from a standard "Returned with Feedback" in

Re: Showing applied extended statistics in explain

2022-01-18 Thread Tatsuro Yamada
Hi Tomas! >> What exactly use case do you have in mind? >Well, my goal was to help users investigating the plan/estimates, >because once you create multiple "candidate" statistics objects it may >get quite tricky to determine which of them were applied, in what order, >etc. It's not all that strai

Re: Showing applied extended statistics in explain

2021-07-29 Thread Dmitry Dolgov
> On Tue, Jul 27, 2021 at 10:20:34PM +0200, Tomas Vondra wrote: > > >> 1) The information is stashed in multiple lists added to a Plan. Maybe > >> there's a better place, and maybe we need to invent a better way to > >> track the info (a new node stashed in a single List). > >> > >> ... > >> > >> 3

Re: Showing applied extended statistics in explain

2021-07-28 Thread Robert Haas
On Tue, Jul 27, 2021 at 4:50 PM Tom Lane wrote: > TBH I do not agree that this is a great idea. I think it's inevitably > exposing a lot of unstable internal planner details. Well, that is a risk, but I think I like the alternative even less. Imagine if we had a CREATE INDEX command but no way -

Re: Showing applied extended statistics in explain

2021-07-27 Thread Tomas Vondra
On 7/27/21 10:50 PM, Tom Lane wrote: > Tomas Vondra writes: >> On 7/27/21 12:21 PM, Dmitry Dolgov wrote: So it seems useful to include this into in the explain plan - show which statistics were applied, in which order. Attached is an early PoC patch doing that in VERBOSE mode. I'l

Re: Showing applied extended statistics in explain

2021-07-27 Thread Tom Lane
Tomas Vondra writes: > On 7/27/21 12:21 PM, Dmitry Dolgov wrote: >>> So it seems useful to include this into in the explain plan - show which >>> statistics were applied, in which order. Attached is an early PoC patch >>> doing that in VERBOSE mode. I'll add it to the next CF. > Yes. I think prov

Re: Showing applied extended statistics in explain

2021-07-27 Thread Tomas Vondra
Hi, On 7/27/21 12:21 PM, Dmitry Dolgov wrote: >> On Sat, Mar 27, 2021 at 01:50:54AM +0100, Tomas Vondra wrote: >> Hi, >> >> With extended statistics it may not be immediately obvious if they were >> applied and to which clauses. If you have multiple extended statistics, >> we may also apply them i

Re: Showing applied extended statistics in explain

2021-07-27 Thread Dmitry Dolgov
> On Sat, Mar 27, 2021 at 01:50:54AM +0100, Tomas Vondra wrote: > Hi, > > With extended statistics it may not be immediately obvious if they were > applied and to which clauses. If you have multiple extended statistics, > we may also apply them in different order, etc. And with expressions, > there

Re: Showing applied extended statistics in explain

2021-07-23 Thread Ronan Dunklau
Le samedi 27 mars 2021, 01:50:54 CEST Tomas Vondra a écrit : > The current implementation is a bit ugly PoC, with a couple annoying > issues that need to be solved: > Hello Thomas, I haven't looked at the implementation at all but I think it's an interesting idea. > 1) The information is stash

Showing applied extended statistics in explain

2021-03-26 Thread Tomas Vondra
Hi, With extended statistics it may not be immediately obvious if they were applied and to which clauses. If you have multiple extended statistics, we may also apply them in different order, etc. And with expressions, there's also the question of matching expressions to the statistics. So it seem