Hi, I am using YourKit tool to profile Spark jobs that is run in my Single Node Spark Cluster. When I see the YourKit UI Performance Charts, the thread count always remains at All threads: 34 Daemon threads: 32
Here are my questions: 1. My system can run only 4 threads simultaneously, and obviously my system does not have 34 threads. What could 34 threads mean? 2. I tried running the same job with four different datasets, two small and two relatively big. But in the UI the thread count increases by two, irrespective of data size. Does this mean that the number of threads allocated to each job depending on data size is not taken care by the framework? Thank You
