Since you're using YARN, you may also need to set SPARK_YARN_USER_ENV to
"PYSPARK_PYTHON=/your/desired/python/on/slave/nodes".
2014-09-04 9:59 GMT-07:00 Davies Liu :
> Hey Oleg,
>
> In pyspark, you MUST have the same version of Python in all the
> machines of the cluster,
> which means when you
Hey Oleg,
In pyspark, you MUST have the same version of Python in all the
machines of the cluster,
which means when you run `python` on these machines, all of them
should be the same
version ( 2.6 or 2.7).
With PYSPARK_PYTHON, you can run pyspark with a specified version of
Python. Also,
you shou
