Anyone knows how to make the built in pyspark work?
I built the source code with command:
mvn clean package -Pspark-1.4 -DskipTests
And all the Spark tags worked except %pyspark. No matter what python code I
run, it will report
pyspark is not responding
I checked the logs, only two lines was recorded if I run a HelloWorld python
code based on %pyspark:
INFO [2015-07-30 19:26:21,220] ({pool-2-thread-5}
SchedulerFactory.java[jobStarted]:132) - Job remoteInterpretJob_1438309581220
started by scheduler interpreter_2047324099
INFO [2015-07-30 19:26:31,233] ({pool-2-thread-5}
SchedulerFactory.java[jobFinished]:138) - Job remoteInterpretJob_1438309581220
finished by scheduler interpreter_2047324099
Thank you so much.
