Nikita, Thanks for preparing the IEP and breaking the initiative down into pieces. I studied the IEP and it feels like the goal is to built a tool similar to Oracle AWR. If it's so then shouldn't we add this as an extra function to either visorcmd or control.sh. Hopefully, we can start moving in the direction of the tooling unification.
Speaking of the probes/statistics collection approach, is it supposed to reuse tracing capabilities that are to be added as part of IEP-35? [1]. According to the draft of our roadmap [2], @Ivan Rakov <ira...@gridgain.com> is planning to contribute to the feature this year. [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392 [2] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap - Denis On Fri, Apr 24, 2020 at 8:48 AM Nikita Amelchev <nsamelc...@gmail.com> wrote: > Slava, > > The web-console is used to interactive configuration wizard, > management and monitoring tool [1]. > > I want to implement a profiling tool. It's one of the monitoring parts > and helps to answer the question: What will be running slow? See > IEP-35 [2]. > > [1] https://apacheignite-tools.readme.io/docs/ignite-web-console > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392 > > пт, 24 апр. 2020 г. в 18:46, Nikita Amelchev <nsamelc...@gmail.com>: > > > > Maxim, > > > > > Can you provide more information about the binary logging format? > > I started to investigate. It should be with minimal effect on > > performance. I haven't any details for now. > > > > > What the additional log category are you talking about? > > PoC is based on logging performance statistics to the "profiling" > > category to split with other logging and be able to log to a separate > > file. See IgniteLogger#getLogger. > > > > пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev <ilya.kasnach...@gmail.com > >: > > > > > > Hello! > > > > > > You need to throw in control.sh also, which does some kind of > statistics > > > too, such as idle_verify. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин < > slava.kopti...@gmail.com>: > > > > > > > Hello Nikita, > > > > > > > > Perhaps, I am missing something... > > > > Apache Ignite already has a web-console tool. Do we want to improve > the > > > > existing tool instead of creating a new one? > > > > It seems to me, this can be confusing for users. > > > > - visor (which is deprecated) > > > > - web-console (to be honest, I don't quite understand the status of > this > > > > tool) > > > > - new ignite-profiling (which is a monitoring tool as well, judging > by the > > > > provided link [1] ) > > > > > > > > I am not against the new tool, I just want to understand the > motivation to > > > > not improve the existing sub-projects. > > > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool > > > > > > > > Thanks, > > > > S. > > > > > > > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev <nsamelc...@gmail.com>: > > > > > > > > > Hi, Igniters. > > > > > > > > > > I'm working on cluster profiling and the tool for creating a > > > > > performance report. [1] I have prepared PoC based on performance > > > > > logging to a separate category of Ignite log. The report contains: > > > > > > > > > > - Cache operations and its distribution by types [2] > > > > > - Transactions and histogram of durations [3] > > > > > - SQL and Scan query statistics, top of slowest queries, logical > and > > > > > physical reads by query [4] > > > > > - Compute statistics, top of slowest tasks and their jobs [5] > > > > > Soon I will add: > > > > > - Topology and Ignite versions info > > > > > - Client ID in case of operations from clients > > > > > > > > > > For now, I'm developing a binary logging format to reduce the > effect > > > > > on performance. I'll try to reuse Ignite mechanisms. > > > > > > > > > > I would like to hear suggestions for the profiling and the > performance > > > > > report. > > > > > > > > > > [1] > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool > > > > > [2] > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png > > > > > [3] > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png > > > > > [4] > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png > > > > > [5] > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png > > > > > > > > > > -- > > > > > Best wishes, > > > > > Amelchev Nikita > > > > > > > > > > > > > > > > > -- > > Best wishes, > > Amelchev Nikita > > > > -- > Best wishes, > Amelchev Nikita >
