lhotari opened a new pull request, #24688: URL: https://github.com/apache/pulsar/pull/24688
### Motivation Async Profiler is a powerful profiler, but it requires using Linux to get more accurate profiling results. When the main development workflow is on Mac for many like me, I started to wonder if it's possible to set up Async Profiler with the more accurate profiling mode using Docker. I use [OrbStack](https://orbstack.dev/) (instead of Docker Desktop) on Mac to develop Apache Pulsar. I wanted to be able to create simple scenarios that take a few minutes to run and analyze the results with various tools. For example, converting flamegraphs to .html format with various views (such as per-thread CPU flamegraphs, allocation flamegraphs, and lock flamegraphs) and using [Eclipse Mission Control](https://adoptium.net/jmc) to analyze GC events and other details that Async Profiler is able to collect together with Java Flight Recorder in its "jfrsync" mode. This work extends the Async Profiler support that was added in #24623 for directly profiling unit tests in the same process. The downside of that approach is that there will be too many aspects included, which skew the results and make it not useful for validating the performance of actual production code. It also had the problem that Async Profiler on Mac doesn't support the more accurate CPU profiling modes that Async Profiler supports on Linux. This is explained in https://github.com/async-profiler/async-profiler/blob/master/docs/CpuSamplingEngines.md. On Mac, it's not possible to profile native methods either, such as profiling the sources of exceptions. With Async Profiler, it's possible to specify `event=Java_java_lang_Throwable_fillInStackTrace,tree,reverse` to profile exception sources. (btw. JFR has also `jdk.JavaExceptionThrow` and `jdk.JavaErrorThrow` events which could be used, but the exception event isn't enabled by default). ### Modifications - Add changes to inttest Dockerfiles to install async-profiler - Change build to support running and passing necessary parameters to integration tests for profiling - Make changes to the PulsarContainer used in integration tests to integrate with Async Profiler and configure the Docker container with necessary privileges and security configuration - Add a sample test class which profiles a broker while pulsar-perf consume and produce are running ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [ ] `doc` <!-- Your PR contains doc changes. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [x] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
