As long as SPARK_HOME is concerned, this might work. However, other variables, such as PYTHONPATH and SPARK_YARN_USER_ENV should be defined in zeppelin-env.sh as well. Without those variables, zeppelin cannot access, for instance, py4j or pyspark. And guess where py4j sits? in SPARK_HOME/python/lib/ So it's a chicken-and-egg problem: if I define the SPARK_HOME in zeppelin-env.sh, I cannot run both spark version. And if I don't, then other paths are not defined, and zeppelin doesn't know where to look for py4j and pyspark.
2018-01-23 12:45 GMT+01:00 Jeff Zhang <zjf...@gmail.com>: > > Just remove SPARK_HOME in spark-env.sh, and instead define them in spark’s > interpreter setting. You can create 2 spark interpreter, one for spark 1.6, > another for spark 2. Only difference between them is the SPARK_HOME you > defined in interpreter setting. > > > > Michaël Van de Borne <michael.vandebo...@gmail.com>于2018年1月23日周二 下午6:42写道： > >> Hi list, >> >> I'd like my notebooks to support both spark 1.6 & spark 2. >> I managed to get both versions working fine, with the SPARK_HOME variable >> in zeppelin-env.sh. But just one at a time. >> So I need to change the variable and restart zeppelin when I want to swap >> spark versions. >> Is it possible to somehow map a spark distribution to an interpreter >> group so that I can get both versions supported at the same time? >> >> >> Thank you, >> >> cheers, >> >> m. >> >
