Re: Getting the IP address of Spark Driver in yarn-cluster mode

2016-10-25 Thread Masood Krohy
;Could not connect to server on %s' % nodes[amHost] -- Masood Krohy, Ph.D. Data Scientist, Intact Lab-R&D Intact Financial Corporation De : Steve Loughran A : Masood Krohy Cc :"user@spark.apache.org" Date : 2016-10-24 17:09 Obj

Re: Getting the IP address of Spark Driver in yarn-cluster mode

2016-10-24 Thread Steve Loughran
On 24 Oct 2016, at 19:34, Masood Krohy mailto:masood.kr...@intact.net>> wrote: Hi everyone, Is there a way to set the IP address/hostname that the Spark Driver is going to be running on when launching a program through spark-submit in yarn-cluster mode (PySpark 1.6.0)? I do not see an option

Getting the IP address of Spark Driver in yarn-cluster mode

2016-10-24 Thread Masood Krohy
Hi everyone, Is there a way to set the IP address/hostname that the Spark Driver is going to be running on when launching a program through spark-submit in yarn-cluster mode (PySpark 1.6.0)? I do not see an option for this. If not, is there a way to get this IP address after the Spark app has st