gerlowskija commented on code in PR #406: URL: https://github.com/apache/solr/pull/406#discussion_r1884046331
########## solr/benchmark/docs/jmh-profilers-setup.md: ########## @@ -0,0 +1,406 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +# JMH Profiler Setup (Async-Profiler and Perfasm) + +JMH ships with a number of built-in profiler options that have grown in number over time. The profiler system is also pluggable, +allowing for "after-market" profiler implementations to be added on the fly. + +Many of these profilers, most often the ones that stay in the realm of Java, will work across platforms and architectures and do +so right out of the box. Others may be targeted at a specific OS, though there is a good chance a similar profiler for other OS's +may exist where possible. A couple of very valuable profilers also require additional setup and environment to either work fully +or at all. + +[TODO: link to page that only lists commands with simple section] + +- [JMH Profiler Setup (Async-Profiler and Perfasm)](#jmh-profiler-setup-async-profiler-and-perfasm) + - [Async-Profiler](#async-profiler) + - [Install async-profiler](#install-async-profiler) + - [Install Java Debug Symbols](#install-java-debug-symbols) + - [Ubuntu](#ubuntu) + - [Arch](#arch) + - [Perfasm](#perfasm) + - [Arch](#arch-1) + - [Ubuntu](#ubuntu-1) + +<br/> +This guide will cover setting up both the async-profiler and the Perfasm profiler. Currently, we roughly cover two Linux family trees, +but much of the information can be extrapolated or help point in the right direction for other systems. + +<br/> <br/> + +|<b>Path 1: Arch, Manjaro, etc</b>|<b>Path 2: Debian, Ubuntu, etc</b>| +| :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| <image src="https://user-images.githubusercontent.com/448788/137563725-0195a732-da40-4c8b-a5e8-fd904a43bb79.png"/><image src="https://user-images.githubusercontent.com/448788/137563722-665de88f-46a4-4939-88b0-3f96e56989ea.png"/> | <image src="https://user-images.githubusercontent.com/448788/137563909-6c2d2729-2747-47a0-b2bd-f448a958b5be.png"/><image src="https://user-images.githubusercontent.com/448788/137563908-738a7431-88db-47b0-96a4-baaed7e5024b.png"/> | + +<br/> + +If you run `jmh.sh` with the `-lprof` argument, it will make an attempt to only list the profilers that it detects will work in your particular environment. + +You should do this first to see where you stand. + + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +./jmh.sh -lprof` +``` + +</div> + + +<br/> + +In our case, we will start with very **minimal** Arch and Ubuntu clean installations, and so we already know there is _**no chance**_ that async-profiler or Perfasm +are going to run. + +In fact, first we have to install a few project build requirements before thinking too much about JMH profiler support. + +We will run on **Arch/Manjaro**, but there should not be any difference than on **Debian/Ubuntu** for this stage. + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 5px 10px 10px;padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +sudo pacman -S wget jdk-openjdk11 +``` + +</div> + +<br/> + +Here we give **async-profiler** a try on **Arch** anyway and observe the failure indicating that we need to obtain the async-profiler library and +put it in the correct location at a minimum. + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +./jmh.sh BenchMark -prof async +``` + +<pre> + <image src="https://user-images.githubusercontent.com/448788/137534191-01c2bc7a-5c1f-42a2-8d66-a5d1a5280db4.png"/> Profilers failed to initialize, exiting. + + Unable to load async-profiler. Ensure asyncProfiler library is on LD_LIBRARY_PATH (Linux) + DYLD_LIBRARY_PATH (Mac OS), or -Djava.library.path. + + Alternatively, point to explicit library location with: '-prof async:libPath={path}' + + no asyncProfiler in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib] + </pre> + +</div> + +### Async-Profiler + +#### Install async-profiler + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +wget -c https://github.com/jvm-profiling-tools/async-profiler/releases/download/v2.5/async-profiler-2.5-linux-x64.tar.gz -O - | tar -xz +sudo mkdir -p /usr/java/packages/lib +sudo cp async-profiler-2.5-linux-x64/build/* /usr/java/packages/lib +``` + +</div> + +<br/> + +That should work out better, but there is still an issue that will prevent a successful profiling run. async-profiler relies on Linux's perf, +and in any recent Linux kernel, perf is restricted from doing its job without some configuration loosening. + +Manjaro should have perf available, but you may need to install it in the other cases. + +<br/> + + + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +apt-get install linux-tools-common linux-tools-generic linux-tools-`uname -r` +``` + +</div> + +<br/> + + + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +pacman -S perf +``` + +</div> + + +<br/> + +And now the permissions issue. The following changes will persist across restarts, and that is likely how you should leave things. + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + +```zsh +sudo sysctl -w kernel.kptr_restrict=0 +sudo sysctl -w kernel.perf_event_paranoid=1 +``` + +</div> + +<br/> + +Now we **should** see some success: + +<div style="z-index: 8; background-color: #364850; border-style: solid; border-width: 1px; border-color: #3b4d56;border-radius: 0px; margin: 0px 5px 3px 10px; padding-bottom: 1px;padding-top: 5px;" data-code-wrap="true"> + + + +```Shell +./jmh.sh FuzzyQuery -prof async:output=flamegraph +``` + +</div> + +<br/> + + + +<br/> + +But you will also find an important _warning_ if you look closely at the logs. + +<br/> + + +<span style="color: yellow; margin-left: 5px;">[WARN]</span> `Install JVM debug symbols to improve profile accuracy` + +<br/> + +We do not want **debug symbols** stripped from Java for the best experience. Review Comment: Made some wording tweaks here to be more specific 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
![https://user-images.githubusercontent.com/448788/137563725-0195a732-da40-4c8b-a5e8-fd904a43bb79.png"/](https://user-images.githubusercontent.com/448788/137563725-0195a732-da40-4c8b-a5e8-fd904a43bb79.png"/){kind=link}
![https://user-images.githubusercontent.com/448788/137563722-665de88f-46a4-4939-88b0-3f96e56989ea.png"/](https://user-images.githubusercontent.com/448788/137563722-665de88f-46a4-4939-88b0-3f96e56989ea.png"/){kind=link}
![https://user-images.githubusercontent.com/448788/137563909-6c2d2729-2747-47a0-b2bd-f448a958b5be.png"/](https://user-images.githubusercontent.com/448788/137563909-6c2d2729-2747-47a0-b2bd-f448a958b5be.png"/){kind=link}
![https://user-images.githubusercontent.com/448788/137563908-738a7431-88db-47b0-96a4-baaed7e5024b.png"/](https://user-images.githubusercontent.com/448788/137563908-738a7431-88db-47b0-96a4-baaed7e5024b.png"/){kind=link}
![https://user-images.githubusercontent.com/448788/137534191-01c2bc7a-5c1f-42a2-8d66-a5d1a5280db4.png"/](https://user-images.githubusercontent.com/448788/137534191-01c2bc7a-5c1f-42a2-8d66-a5d1a5280db4.png"/){kind=link}
