Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Michael Paquier
On Wed, Mar 01, 2023 at 12:43:40AM +, Imseih (AWS), Sami wrote: > Yes, that makes sense. Okay, thanks. Done that now on HEAD. -- Michael signature.asc Description: PGP signature

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Imseih (AWS), Sami
> + > + Queries on which normalization can be applied may be observed with constant > + values in pg_stat_statements, especially when there > + is a high rate of entry deallocations. To reduce the likelihood of this > + happening, consider increasing pg_stat_statements.max. > + The pg_stat_stateme

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Michael Paquier
On Tue, Feb 28, 2023 at 11:11:30PM +, Imseih (AWS), Sami wrote: > I agree. We introduce the concept of a plannable statement in a > previous section and we can then make this distinction in the new > paragraph. > > I also added a link to pg_stat_statements_info since that is introduced > lat

Re: Doc update for pg_stat_statements normalization

2023-02-28 Thread Imseih (AWS), Sami
> I am OK with an addition to the documentation to warn that one may > have to increase the maximum number of entries that can be stored if > seeing a non-normalized entry that should have been normalized. I agree. We introduce the concept of a plannable statement in a previous section and we can

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Michael Paquier
On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > I think the only thing to do here is to call this out in docs with a > suggestion to increase > pg_stat_statements.max to reduce the likelihood. I also attached the suggested > doc enhancement as well. + + A query text may

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Michael Paquier
On Mon, Feb 27, 2023 at 10:53:26PM +, Imseih (AWS), Sami wrote: > Wouldn't be less in terms of memory usage to just store the required > JumbleState fields in Query[Desc]? > > clocations_count, > highest_extern_param_id, > clocations_locations, > clocations_length Yes, these would be enough t

Re: Doc update for pg_stat_statements normalization

2023-02-27 Thread Imseih (AWS), Sami
>On Sat, Feb 25, 2023 at 01:59:04PM +, Imseih (AWS), Sami wrote:> >> The overhead of storing this additional private data for the life of the > query >> execution may not be desirable. >Okay, but why? Additional memory to maintain the JumbleState data in other structs, and t

Re: Doc update for pg_stat_statements normalization

2023-02-26 Thread Michael Paquier
On Sat, Feb 25, 2023 at 01:59:04PM +, Imseih (AWS), Sami wrote: > The overhead of storing this additional private data for the life of the query > execution may not be desirable. Okay, but why? > I think we also will need to copy the > private data to QueryDesc as well to make it available t

Re: Doc update for pg_stat_statements normalization

2023-02-25 Thread Imseih (AWS), Sami
>> Could things be done in a more stable way? For example, imagine that >> we have an extra Query field called void *private_data that extensions >> can use to store custom data associated to a query ID, then we could >> do something like that: >> - In the post-analyze hook, ch

Re: Doc update for pg_stat_statements normalization

2023-02-25 Thread Julien Rouhaud
On Sat, Feb 25, 2023 at 02:58:36PM +0900, Michael Paquier wrote: > On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > > I think the only thing to do here is to call this out in docs with a > > suggestion to increase pg_stat_statements.max to reduce the > > likelihood. I also atta

Re: Doc update for pg_stat_statements normalization

2023-02-24 Thread Michael Paquier
On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > I think the only thing to do here is to call this out in docs with a > suggestion to increase pg_stat_statements.max to reduce the > likelihood. I also attached the suggested doc enhancement as well. Improving the docs about th

Doc update for pg_stat_statements normalization

2023-02-24 Thread Imseih (AWS), Sami
Replacing constants in pg_stat_statements is on a best effort basis. It is not unlikely that on a busy workload with heavy entry deallocation, the user may observe the query with the constants in pg_stat_statements. From what I can see, this is because the only time an entry is normalized is durin