export PYTHONPATH=/path/to/module.egg I think is what I was doing. On Tue, May 19, 2015 at 11:27 AM Marty B <[email protected]> wrote:
> Jon, > > How are you loading the Python module? > > The only way I could get it to load was to modify > spark/src/main/resources/python/zeppelin_pyspark.py and add it to the > SparkContext. > > > On Tue, May 19, 2015 at 10:43 AM Jonathan Haddad <[email protected]> > wrote: > >> Yep, I believe that was the one. >> >> On Tue, May 19, 2015 at 10:41 AM Marty B <[email protected]> wrote: >> >>> Jon, >>> >>> Is this the error message you see? >>> >>> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error >>> occurred while calling o77.loadClass.\n', JavaObject id=o78), <traceback >>> object at 0x7f8949d124d0>) >>> >>> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <[email protected]> >>> wrote: >>> >>>> I believe was able to make this part work by pointing PYTHONPATH to the >>>> egg. The part I couldn't get working was the driver class path, etc. I'm >>>> not really a Java guy so it's a lot of stumbling around for me. >>>> >>>> On Tue, May 19, 2015 at 9:46 AM Marty B <[email protected]> wrote: >>>> >>>>> The problem as I see it is that there appears to be no way to request >>>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra >>>>> interface). >>>>> >>>>> I opened ZEPPELIN-71 for this, but have seen no progress other than >>>>> acknowledgement. >>>>> >>>>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin >>>>> team with prioritization. :-) >>>>> >>>>> I hope to find time over the upcoming holiday weekend (US) to look >>>>> further into fixing it - my first attempt failed. >>>>> >>>>> >>>>> >>>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <[email protected]> >>>>> wrote: >>>>> >>>>>> What I meant was I couldn't figure out how to get the Cassandra >>>>>> connector to work in pyspark. I had asked previously (including the >>>>>> flags >>>>>> I pass to pyspark) and gotten no response. >>>>>> >>>>>> If you've got a recommendation as to how to make the equivilent of >>>>>> this work in Zeppelin I'd love to give it a try: >>>>>> >>>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target >>>>>> export PYSPARK_DRIVER_PYTHON=ipython >>>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook" >>>>>> >>>>>> pyspark \ >>>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \ >>>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \ >>>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \ >>>>>> --conf spark.cassandra.connection.host=127.0.0.1 \ >>>>>> --master spark://127.0.0.1:7077 \ >>>>>> >>>>>> Jon >>>>>> >>>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> First, thanks for trying Zeppelin. >>>>>>> >>>>>>> You guys mean pyspark is not working? >>>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it. >>>>>>> >>>>>>> If you can try latest code and post some log files, it'll be helpful >>>>>>> to investigate the problems. >>>>>>> >>>>>>> Thanks, >>>>>>> moon >>>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> How did you get it to work? I tried and failed miserably. >>>>>>>> >>>>>>>> Jon >>>>>>>> >>>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin >>>>>>>> from today, but I'm getting some weirdness. Before digging into it >>>>>>>> further >>>>>>>> I also tried Spark 1.3.1 but got other strange things (output of notes >>>>>>>> not >>>>>>>> showing anything). >>>>>>>> >>>>>>>> >>>>>>>> So curious, what versions are recommended or preferred? I'm >>>>>>>> trying on OSX so far. >>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> >>>>>>>> Tyler >>>>>>>> >>>>>>>>
