Re: Profiling the internal of Hive

2015-10-21 Thread Sergey Shelukhin
For Tez containers, you can specify JVM args via hive.tez.java.opts config setting, there’s probably a similar setting for MR. You can add the profiler agent to this setting, there e.g. for Yourkit something like: "-agentpath:/opt/yourkit/bin/linux-x86-64/libyjpagent.so=disablej2ee,disabl etracing,

Profiling the internal of Hive

2015-10-20 Thread Justin Wong
Hi there, I'm recently doing a research program about hive, and I need to profile the internal performance of Hive. I've managed to run hive over hadoop localJobRunner and attached JProfiler to it, while the CPU hostspot stopped at level "runJar". I also tried hadoop's built in profiling, but