Re: Securing Spark's Network

2014-04-25 Thread Jacob Eisinger
Technologies jeis...@us.ibm.com - (512) 286-6075 From: Akhil Das To: user@spark.apache.org Date: 04/25/2014 12:51 PM Subject:Re: Securing Spark's Network Sent by:ak...@mobipulse.in Hi Jacob, This post might give you a brief idea about the ports being used

Re: Securing Spark's Network

2014-04-25 Thread Akhil Das
Hi Jacob, This post might give you a brief idea about the ports being used https://groups.google.com/forum/#!topic/spark-users/PN0WoJiB0TA On Fri, Apr 25, 2014 at 8:53 PM, Jacob Eisinger wrote: > Howdy, > > We tried running Spark 0.9.1 stand-alone inside docker containers > distributed ove

Securing Spark's Network

2014-04-25 Thread Jacob Eisinger
Howdy, We tried running Spark 0.9.1 stand-alone inside docker containers distributed over multiple hosts. This is complicated due to Spark opening up ephemeral / dynamic ports for the workers and the CLI.  To ensure our docker solution doesn't break Spark in unexpected ways and maintains a sec