Re: Custom explain options

2024-07-22 Thread Pavel Stehule
Hi po 22. 7. 2024 v 17:08 odesílatel Konstantin Knizhnik napsal: > > On 16/01/2024 5:38 pm, Tomas Vondra wrote: > > By "broken" you mean that you prefetch items only from a single leaf > > page, so immediately after reading the next one nothing is prefetched. > Correct? > > > Yes, exactly. It me

Re: Custom explain options

2024-01-23 Thread David G. Johnston
Came across this while looking for patches to review. IMO this thread has been hijacked to the point of being not useful for the subject. I suggest this discussion regarding prefetch move to its own thread and this thread and commitfest entry be ended/returned with feedback. Also IMO, the commit

Re: Custom explain options

2024-01-16 Thread Konstantin Knizhnik
On 16/01/2024 5:38 pm, Tomas Vondra wrote: By "broken" you mean that you prefetch items only from a single leaf page, so immediately after reading the next one nothing is prefetched. Correct? Yes, exactly. It means that reading first heap page from next leaf page will be done without prefetc

Re: Custom explain options

2024-01-16 Thread Tomas Vondra
On 1/15/24 21:42, Konstantin Knizhnik wrote: > > On 15/01/2024 5:08 pm, Tomas Vondra wrote: >> >> My patch does not care about prefetching internal index pages. Yes, it's >> a limitation, but my assumption is the internal pages are maybe 0.1% of >> the index, and typically very hot / cached. Ye

Re: Custom explain options

2024-01-15 Thread Konstantin Knizhnik
On 15/01/2024 5:08 pm, Tomas Vondra wrote: My patch does not care about prefetching internal index pages. Yes, it's a limitation, but my assumption is the internal pages are maybe 0.1% of the index, and typically very hot / cached. Yes, if the index is not used very often, this may be untrue. B

Re: Custom explain options

2024-01-15 Thread Tomas Vondra
On 1/15/24 15:22, Konstantin Knizhnik wrote: > > On 14/01/2024 11:47 pm, Tomas Vondra wrote: >> The thing that was not clear to me is who decides what to prefetch, >> which code issues the prefetch requests etc. In the github links you >> shared I see it happens in the index AM code (in nbtsear

Re: Custom explain options

2024-01-15 Thread Konstantin Knizhnik
On 14/01/2024 11:47 pm, Tomas Vondra wrote: The thing that was not clear to me is who decides what to prefetch, which code issues the prefetch requests etc. In the github links you shared I see it happens in the index AM code (in nbtsearch.c). It is up to the particular plan node (seqscan, in

Re: Custom explain options

2024-01-14 Thread Tomas Vondra
On 1/13/24 17:13, Konstantin Knizhnik wrote: > > On 13/01/2024 4:51 pm, Tomas Vondra wrote: >> >> On 1/12/24 20:30, Konstantin Knizhnik wrote: >>> On 12/01/2024 7:03 pm, Tomas Vondra wrote: On 10/21/23 14:16, Konstantin Knizhnik wrote: > Hi hackers, > > EXPLAIN statement has a lis

Re: Custom explain options

2024-01-13 Thread Konstantin Knizhnik
On 13/01/2024 4:51 pm, Tomas Vondra wrote: On 1/12/24 20:30, Konstantin Knizhnik wrote: On 12/01/2024 7:03 pm, Tomas Vondra wrote: On 10/21/23 14:16, Konstantin Knizhnik wrote: Hi hackers, EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful

Re: Custom explain options

2024-01-13 Thread Tomas Vondra
On 1/12/24 20:30, Konstantin Knizhnik wrote: > > On 12/01/2024 7:03 pm, Tomas Vondra wrote: >> On 10/21/23 14:16, Konstantin Knizhnik wrote: >>> Hi hackers, >>> >>> EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, >>> COST,...) which help to provide useful details of query execut

Re: Custom explain options

2024-01-12 Thread Konstantin Knizhnik
On 12/01/2024 7:03 pm, Tomas Vondra wrote: On 10/21/23 14:16, Konstantin Knizhnik wrote: Hi hackers, EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added PREFETCH option which shows information

Re: Custom explain options

2024-01-12 Thread Tomas Vondra
On 10/21/23 14:16, Konstantin Knizhnik wrote: > Hi hackers, > > EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, > COST,...) which help to provide useful details of query execution. > In Neon we have added PREFETCH option which shows information about page > prefetching during query

Re: Custom explain options

2024-01-10 Thread Andrei Lepikhov
On 10/1/2024 20:27, Konstantin Knizhnik wrote: On 10/01/2024 8:46 am, Michael Paquier wrote: On Wed, Jan 10, 2024 at 01:29:30PM +0700, Andrei Lepikhov wrote: What do you think about this really useful feature? Do you wish to develop it further? I am biased here.  This seems like a lot of cod

Re: Custom explain options

2024-01-10 Thread Konstantin Knizhnik
On 09/01/2024 10:33 am, vignesh C wrote: On Sat, 21 Oct 2023 at 18:34, Konstantin Knizhnik wrote: Hi hackers, EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added PREFETCH option which shows i

Re: Custom explain options

2024-01-10 Thread Konstantin Knizhnik
On 10/01/2024 8:29 am, Andrei Lepikhov wrote: On 30/11/2023 22:40, Konstantin Knizhnik wrote: In all this cases we are using array of `Instrumentation` and if it contains varying part, then it is not clear where to place it. Yes, there is also code which serialize and sends instrumentations b

Re: Custom explain options

2024-01-10 Thread Konstantin Knizhnik
On 10/01/2024 8:46 am, Michael Paquier wrote: On Wed, Jan 10, 2024 at 01:29:30PM +0700, Andrei Lepikhov wrote: What do you think about this really useful feature? Do you wish to develop it further? I am biased here. This seems like a lot of code for something we've been delegating to the exp

Re: Custom explain options

2024-01-09 Thread Michael Paquier
On Wed, Jan 10, 2024 at 01:29:30PM +0700, Andrei Lepikhov wrote: > What do you think about this really useful feature? Do you wish to develop > it further? I am biased here. This seems like a lot of code for something we've been delegating to the explain hook for ages. Even if I can see the appe

Re: Custom explain options

2024-01-09 Thread Andrei Lepikhov
On 30/11/2023 22:40, Konstantin Knizhnik wrote: In all this cases we are using array of `Instrumentation` and if it contains varying part, then it is not clear where to place it. Yes, there is also code which serialize and sends instrumentations between worker processes  and I have updated this

Re: Custom explain options

2024-01-09 Thread vignesh C
On Sat, 21 Oct 2023 at 18:34, Konstantin Knizhnik wrote: > > Hi hackers, > > EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) > which help to provide useful details of query execution. > In Neon we have added PREFETCH option which shows information about page > prefetchi

Re: Custom explain options

2023-11-30 Thread Andrei Lepikhov
On 30/11/2023 22:40, Konstantin Knizhnik wrote: On 30/11/2023 5:59 am, Andrei Lepikhov wrote: On 21/10/2023 19:16, Konstantin Knizhnik wrote: EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added

Re: Custom explain options

2023-11-30 Thread Konstantin Knizhnik
On 30/11/2023 5:59 am, Andrei Lepikhov wrote: On 21/10/2023 19:16, Konstantin Knizhnik wrote: EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added PREFETCH option which shows information about

Re: Custom explain options

2023-11-29 Thread Andrei Lepikhov
On 21/10/2023 19:16, Konstantin Knizhnik wrote: EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added PREFETCH option which shows information about page prefetching during query execution (prefetch

Re: Custom explain options

2023-11-29 Thread Pavel Stehule
usage, char* cust_usag foreach (lc, pgCustInstr) { - CustomInstrumentation *ci = (CustomInstrumentation*)lfirst(lc); + CustomInstrumentation *ci = (CustomInstrumentation*) lfirst(lc); + ci->accum(cust_usage, ci->usage, cust_save); cust_usage += ci->size; cust_save += ci->size; @@ -316,13

Custom explain options

2023-10-21 Thread Konstantin Knizhnik
Hi hackers, EXPLAIN statement has a list of options (i.e. ANALYZE, BUFFERS, COST,...) which help to provide useful details of query execution. In Neon we have added PREFETCH option which shows information about page prefetching during query execution (prefetching is more critical for Neon archi