Hi, I'd propose the following approach: 1) Enable history by default. Becuase otherwise users will have to restart the node to enable it, or we will have to implement dynamic history enable, which is complex thing. Default value should be relatively small yet allowing to accommodate typical workloads. E.g. 1000 entries. This should not put any serious pressure to GC. 2) Split queries by: schema, query, local flag 3) Track only growing values: execution count, error count, minimum duration, maximum duration 4) Implement ability to clear history - JMX, SQL command, whatever (may be this is different ticket) 5) History cleanup might be implemented similarly to current approach: store everything in CHM. Periodically check it's size. If it is too big - evict oldest entries. But this should be done with care - under some workloads new queries will be generated very quickly. In this case we should either fallback to synchronous evicts, or do not log history at all.
Thoughts? Vladimir. - On Fri, Dec 21, 2018 at 11:22 AM Юрий <jury.gerzhedow...@gmail.com> wrote: > Alexey, > > Yes, such property to configuration history size will be added. I think > default value should be 0 and history by default shouldn't be gather at > all, and can be switched on by property in case when it required. > > Currently I planned use the same way to evicting old data as for > queryMetrics - scheduled task will evict will old data by oldest start time > of query. > > Will be gathered statistics for only initial clients queries, so internal > queries will not including. For the same queries we will have one record in > history with merged statistics. > > All above points just my proposal. Please revert back in case you think > anything should be implemented by another way. > > > > > > чт, 20 дек. 2018 г. в 18:23, Alexey Kuznetsov <akuznet...@apache.org>: > > > Yuriy, > > > > I have several questions: > > > > Are we going to add some properties to cluster configuration for history > > size? > > > > And what will be default history size? > > > > Will the same queries count as same item of historical data? > > > > How we will evict old data that not fit into history? > > > > Will we somehow count "reduce" queries? Or only final "map" ones? > > > > -- > > Alexey Kuznetsov > > > > > -- > Живи с улыбкой! :D >
