Re: Profile Spark Executors

2019-01-18 Thread Oleg Mazurov
statsd-jvm-profiler is a very basic profiler written in Java. It's little better than running jstack in a loop. From that perspective, it can't do real CPU profiling. What it presents as such is a collection of stack traces of Java threads in RUNNABLE state. RUNNABLE doesn't mean actually running o

Profile Spark Executors

2019-01-18 Thread Jack Kolokasis
Hi all,     I try to profile my spark executors performance when use on Heap persistent level in compare to use off-Heap persistent level. I use statsd-jvm-profiler to profile each executor. From the results i see that application spends 71,92% of its threads running the method sun.nio.ch.EP