Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-08 Thread David Rowley
On Tue, 9 Jul 2024 at 11:51, David Rowley wrote: > I think we should consider aligning Parallel Hash > with the other Parallel node behaviour. I looked at that and quickly realised that it makes sense that Parallel Hash does something different here. All the workers are contributing to building

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-08 Thread David Rowley
On Mon, 8 Jul 2024 at 15:43, David Rowley wrote: > > On Sun, 18 Feb 2024 at 11:31, Tomas Vondra > wrote: > > 2) Leader vs. worker counters > > > > It seems to me this does nothing to add the per-worker values from "Heap > > Blocks" into the leader, which means we get stuff like this: > > > >

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-07 Thread David Rowley
On Sun, 18 Feb 2024 at 11:31, Tomas Vondra wrote: > 2) Leader vs. worker counters > > It seems to me this does nothing to add the per-worker values from "Heap > Blocks" into the leader, which means we get stuff like this: > > Heap Blocks: exact=102 lossy=10995 > Worker 0: actual time=50.5

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-07 Thread David Rowley
On Mon, 8 Jul 2024 at 12:19, David Rowley wrote: > Notice the "Heap Blocks: exact=225000" is missing on Windows. > This is because it wrapped around to a negative value and > show_tidbitmap_info() only shows > 0 values. > > I feel this is a good enough justification to increase the width of >

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-07 Thread David Rowley
On Fri, 5 Jul 2024 at 12:52, David Rowley wrote: > I propose we change these to uint64 while causing churn in this area, > probably as a follow-on patch. I think a uint32 isn't wide enough as > you could exceed the limit with rescans. I wondered how large a query it would take to cause this prob

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-04 Thread David Rowley
On Fri, 5 Jul 2024 at 01:59, David Rowley wrote: > I also made a pass over the patch, and I also changed: > > 1. Fixed up a few outdated comments in execnodes.h. > 2. Added a comment in ExecEndBitmapHeapScan() to explain why we += the > stats rather than memcpy the BitmapHeapScanInstrumentation. >

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-07-04 Thread David Rowley
On Wed, 26 Jun 2024 at 22:22, wrote: > 1) Unify the print format of leader and worker > > In show_tidbitmap_info(), the number of exact/loosy blocks of the leader and > workers > are printed. I think the printed format should be same. Currently, the leader > does not > print the blocks of exact/

RE: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-06-26 Thread Masahiro.Ikeda
Hi, Thanks for working it! I'm interested in this feature, so I'd like to participate in the patch review. Though I've just started looking at the patch, I have two comments about the v6 patch. (And I want to confirm the thread active.) 1) Unify the print format of leader and worker In show_ti

RE: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-06-26 Thread Masahiro.Ikeda
Hi, Thanks for working it! I'm interested in this feature, so I'd like to participate in the patch review. Though I've just started looking at the patch, I have two comments about the v6 patch. (And I want to confirm the thread active.) 1) Unify the print format of leader and worker In show_ti

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-04-08 Thread Alena Rybakina
Hi! Thank you for your work on this issue! Your patch required a little revision. I did this and attached the patch. Also, I think you should add some clarification to the comments about printing 'exact' and 'loosy' pages in show_hashagg_info function, which you get from planstate->stats, wher

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-27 Thread Donghang Lin
On Mon, Mar 25, 2024 at 2:11 PM Melanie Plageman wrote: > As for whether or not per-worker stats should be displayed by default > or only with VERBOSE, it sounds like there are two different > precedents. I don't have a strong feeling one way or the other. > Whichever is most consistent. > Donghan

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-25 Thread Melanie Plageman
On Mon, Mar 25, 2024 at 2:29 AM Donghang Lin wrote: > > > > On Sat, Feb 17, 2024 at 2:31 PM Tomas Vondra > > wrote: > > 2) Leader vs. worker counters > > > > It seems to me this does nothing to add the per-worker values from "Heap > > Blocks" into the leader, which means we get stuff like this:

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-24 Thread Donghang Lin
Hi Tomas, > > On Sat, Feb 17, 2024 at 2:31 PM Tomas Vondra < tomas.von...@enterprisedb.com> wrote: > Hi David, > > Do you plan to work continue working on this patch? I did take a look, > and on the whole it looks reasonable - it modifies the right places etc. > > I think there are two things that

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-14 Thread Heikki Linnakangas
On 14/03/2024 22:00, Melanie Plageman wrote: On Thu, Mar 14, 2024 at 05:30:30PM +0200, Heikki Linnakangas wrote: typedef struct SharedBitmapHeapInstrumentation { int num_workers; BitmapHeapScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER]; } SharedBitmapH

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-14 Thread Melanie Plageman
On Thu, Mar 14, 2024 at 05:30:30PM +0200, Heikki Linnakangas wrote: > On 18/02/2024 00:31, Tomas Vondra wrote: > > Do you plan to work continue working on this patch? I did take a look, > > and on the whole it looks reasonable - it modifies the right places etc. > > +1 > > > I think there are two

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-14 Thread Heikki Linnakangas
On 18/02/2024 00:31, Tomas Vondra wrote: Do you plan to work continue working on this patch? I did take a look, and on the whole it looks reasonable - it modifies the right places etc. +1 I think there are two things that may need an improvement: 1) Storing variable-length data in ParallelBi

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-01 Thread Melanie Plageman
On Sat, Feb 17, 2024 at 5:31 PM Tomas Vondra wrote: > > Hi David, > > Do you plan to work continue working on this patch? I did take a look, > and on the whole it looks reasonable - it modifies the right places etc. I haven't started reviewing this patch yet, but I just ran into the behavior that

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-02-17 Thread Tomas Vondra
Hi David, Do you plan to work continue working on this patch? I did take a look, and on the whole it looks reasonable - it modifies the right places etc. I think there are two things that may need an improvement: 1) Storing variable-length data in ParallelBitmapHeapState I agree with Robert the

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-11-04 Thread Dilip Kumar
On Fri, Jan 20, 2023 at 2:04 PM David Geier wrote: > > Hi hackers, > > EXPLAIN ANALYZE for parallel Bitmap Heap Scans currently only reports > the number of heap blocks processed by the leader. It's missing the > per-worker stats. The attached patch adds that functionality in the > spirit of e.g.

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-10-17 Thread Robert Haas
On Mon, Oct 16, 2023 at 12:31 PM Michael Christofides wrote: > According to the docs[1]: "In a parallel bitmap heap scan, one process is > chosen as the leader. That process performs a scan of one or more indexes and > builds a bitmap indicating which table blocks need to be visited. These > bl

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-10-16 Thread Michael Christofides
> EXPLAIN ANALYZE for parallel Bitmap Heap Scans currently only reports > the number of heap blocks processed by the leader. It's missing the > per-worker stats. Hi David, According to the docs[1]: "In a parallel bitmap heap scan, one process is chosen as the leader. That process performs a scan

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-10-06 Thread Dmitry Dolgov
> On Wed, Sep 20, 2023 at 03:42:43PM +0200, David Geier wrote: > Another, not yet discussed, option I can see work is: > > 4. Allocate a fixed amount of memory for the instrumentation stats based on > MAX_PARALLEL_WORKER_LIMIT: MAX_PARALLEL_WORKER_LIMIT is 1024 and used as the > limit of the max_pa

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-09-20 Thread David Geier
Hi Dmitry, Thanks for looking at the patch and sorry for the long line. On 3/17/23 21:14, Dmitry Dolgov wrote: The idea sounds reasonable to me, but I have to admit snapshot_and_stats implementation looks awkward. Maybe it would be better to have a separate structure field for both stats and sn

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-03-17 Thread Dmitry Dolgov
> On Tue, Feb 21, 2023 at 01:02:35PM +0100, David Geier wrote: > Hi, > > On 1/20/23 09:34, David Geier wrote: > > EXPLAIN ANALYZE for parallel Bitmap Heap Scans currently only reports > > the number of heap blocks processed by the leader. It's missing the > > per-worker stats. The attached patch ad

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2023-02-21 Thread David Geier
Hi, On 1/20/23 09:34, David Geier wrote: EXPLAIN ANALYZE for parallel Bitmap Heap Scans currently only reports the number of heap blocks processed by the leader. It's missing the per-worker stats. The attached patch adds that functionality in the spirit of e.g. Sort or Memoize. Here is a simpl