Re: Debugging Spark itself in standalone cluster mode

2016-07-01 Thread cbruegg
e Spark Developers List] > <[hidden > email] <http:///user/SendEmail.jtp?type=node&node=18145&i=0>> wrote: > >> Yes, scheduling is centralized in the driver. >> >> For debugging, I think you'd want to set the executor JVM, not the worker >> JVM

Debugging Spark itself in standalone cluster mode

2016-06-30 Thread cbruegg
Hello everyone, I'm a student assistant in research at the University of Paderborn, working on integrating Spark (v1.6.2) with a new network resource management system. I have already taken a deep dive into the source code of spark-core w.r.t. its scheduling systems. We are running a cluster in s