subject:"Re\: Question about RDD pipe"

Re: Question about RDD pipe

2019-01-18 Thread Mkal

Thanks a lot for the answer! It solved my problem. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Question about RDD pipe

2019-01-17 Thread Arun Mahadevan

Yes, the script should be present on all the executor nodes. You can pass your script via spark-submit (e.g. --files script.sh) and then you should be able to refer that (e.g. "./script.sh") in rdd.pipe. - Arun On Thu, 17 Jan 2019 at 14:18, Mkal wrote: > Hi, im trying to run an external script