Re: zeppelin's py4j conflicts with spark's

2021-04-01 Thread Rui Lu
Hi Jeff, I added one line into zeppelin-env.sh export PYTHONPATH=/usr/lib/spark/python:/usr/lib/spark/python/lib/py4j-src.zip:${PYTHONPATH} # note spark's py4j is newer than zeppelin's This modification is picked up however as far as I can tell some scripts for python interpreter initialisation

Re: zeppelin's py4j conflicts with spark's

2021-04-01 Thread Jeff Zhang
Could you share how do you include pyspark's py4j to python interpreter ? Rui Lu 于2021年3月25日周四 下午10:49写道: > Hi all, > > I’m trying to switch from pyspark interpreter to python interpreter and > ran into weird errors of py4j like “key error ‘x’” or “invalid command” or > so when creating spark se

zeppelin's py4j conflicts with spark's

2021-03-25 Thread Rui Lu
Hi all, I’m trying to switch from pyspark interpreter to python interpreter and ran into weird errors of py4j like “key error ‘x’” or “invalid command” or so when creating spark session. A little digging reveals that zeppelin has its own py4j stuffed into PYTHONPATH of python interpreter, the val