Jon, How are you loading the Python module?
The only way I could get it to load was to modify spark/src/main/resources/python/zeppelin_pyspark.py and add it to the SparkContext. On Tue, May 19, 2015 at 10:43 AM Jonathan Haddad <[email protected]> wrote: > Yep, I believe that was the one. > > On Tue, May 19, 2015 at 10:41 AM Marty B <[email protected]> wrote: > >> Jon, >> >> Is this the error message you see? >> >> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error occurred >> while calling o77.loadClass.\n', JavaObject id=o78), <traceback object at >> 0x7f8949d124d0>) >> >> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <[email protected]> >> wrote: >> >>> I believe was able to make this part work by pointing PYTHONPATH to the >>> egg. The part I couldn't get working was the driver class path, etc. I'm >>> not really a Java guy so it's a lot of stumbling around for me. >>> >>> On Tue, May 19, 2015 at 9:46 AM Marty B <[email protected]> wrote: >>> >>>> The problem as I see it is that there appears to be no way to request >>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra >>>> interface). >>>> >>>> I opened ZEPPELIN-71 for this, but have seen no progress other than >>>> acknowledgement. >>>> >>>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin >>>> team with prioritization. :-) >>>> >>>> I hope to find time over the upcoming holiday weekend (US) to look >>>> further into fixing it - my first attempt failed. >>>> >>>> >>>> >>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <[email protected]> >>>> wrote: >>>> >>>>> What I meant was I couldn't figure out how to get the Cassandra >>>>> connector to work in pyspark. I had asked previously (including the flags >>>>> I pass to pyspark) and gotten no response. >>>>> >>>>> If you've got a recommendation as to how to make the equivilent of >>>>> this work in Zeppelin I'd love to give it a try: >>>>> >>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target >>>>> export PYSPARK_DRIVER_PYTHON=ipython >>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook" >>>>> >>>>> pyspark \ >>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \ >>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \ >>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \ >>>>> --conf spark.cassandra.connection.host=127.0.0.1 \ >>>>> --master spark://127.0.0.1:7077 \ >>>>> >>>>> Jon >>>>> >>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <[email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> First, thanks for trying Zeppelin. >>>>>> >>>>>> You guys mean pyspark is not working? >>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it. >>>>>> >>>>>> If you can try latest code and post some log files, it'll be helpful >>>>>> to investigate the problems. >>>>>> >>>>>> Thanks, >>>>>> moon >>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> How did you get it to work? I tried and failed miserably. >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from >>>>>>> today, but I'm getting some weirdness. Before digging into it further I >>>>>>> also tried Spark 1.3.1 but got other strange things (output of notes not >>>>>>> showing anything). >>>>>>> >>>>>>> >>>>>>> So curious, what versions are recommended or preferred? I'm >>>>>>> trying on OSX so far. >>>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> >>>>>>> Tyler >>>>>>> >>>>>>>
