[GitHub] zeppelin pull request #1831: [ZEPPELIN-1883] Can't import spark submitted pa...

[1ambda]

GitHub user 1ambda opened a pull request:

    https://github.com/apache/zeppelin/pull/1831

    [ZEPPELIN-1883] Can't import spark submitted packages in PySpark

    ### What is this PR for?
    
    Fixed importing packages in pyspack requested by `SPARK_SUBMIT_OPTION`
    
    ### What type of PR is it?
    [Bug Fix]
    
    ### Todos
    
    Nothing
    
    ### What is the Jira issue?
    
    [ZEPPELIN-1883](https://issues.apache.org/jira/browse/ZEPPELIN-1883)
    
    ### How should this be tested?
    
    1. Set `SPARK_HOME` and `SPARK_SUBMIT_OPTION` in `conf/zeppelin-env.sh` like
    
    ```sh
    export SPARK_HOME="~/github/apache-spark/1.6.2-bin-hadoop2.6"
    export SPARK_SUBMIT_OPTIONS="--packages 
com.datastax.spark:spark-cassandra-connector_2.10:1.6.2,TargetHolding:pyspark-cassandra:0.3.5
 --exclude-packages org.slf4j:slf4j-api"
    ```
    
    2. Test whether submitted packages can be import or not
    
    ```
    %pyspark
    
    import pyspark_cassandra
    ```
    
    ### Screenshots (if appropriate)
    
    ```
    import pyspark_cassandra
    
    
    Traceback (most recent call last):
      File 
"/var/folders/lr/8g9y625n5j39rz6qhkg8s6640000gn/T/zeppelin_pyspark-5266742863961917074.py",
 line 267, in <module>
        raise Exception(traceback.format_exc())
    Exception: Traceback (most recent call last):
      File 
"/var/folders/lr/8g9y625n5j39rz6qhkg8s6640000gn/T/zeppelin_pyspark-5266742863961917074.py",
 line 265, in <module>
        exec(code)
      File "<stdin>", line 1, in <module>
    ImportError: No module named pyspark_cassandra
    ```
    
    ### Questions:
    * Does the licenses files need update? - NO
    * Is there breaking changes for older versions? - NO
    * Does this needs documentation? - NO


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/1ambda/zeppelin 
ZEPPELIN-1883/cant-import-submitted-packages-in-pyspark

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/1831.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1831
    
----
commit c735bd54b1ce712641ae9d2c4b780d954c0e985c
Author: 1ambda <1am...@gmail.com>
Date:   2017-01-02T04:52:40Z

    fix: Import spark submit packages in pyspark

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---