[GitHub] incubator-zeppelin pull request: Add a (local mode) Scalding Inter...

Mon, 21 Dec 2015 16:27:12 -0800 

GitHub user sriramkrishnan opened a pull request:

    https://github.com/apache/incubator-zeppelin/pull/561

    Add a (local mode) Scalding Interpreter to Zeppelin

    ### What is this PR for?
    Scalding (https://github.com/twitter/scalding) is a Scala library for 
writing MapReduce jobs.
    This issue tracks the addition of a Scalding interpreter for Zeppelin. To 
keep this work incremental, this PR will focus on just a local mode 
implementation. The Hadoop mode can be a subsequent addition.
    
    ### What type of PR is it?
    Feature
    
    ### Todos
    * Addition of Hadoop mode for Scalding
    
    ### Is there a relevant Jira issue?
    https://issues.apache.org/jira/browse/ZEPPELIN-526
    
    ### How should this be tested?
    Run the tests in: 
scalding/src/test/java/org/apache/zeppelin/scalding/ScaldingInterpreterTest.java
    
    ### Screenshots
    <img width="1167" alt="scalding-example" 
src="https://cloud.githubusercontent.com/assets/1509691/11944979/8788d5c2-a7ff-11e5-9863-2a1216c51896.png";>
    <img width="1151" alt="scalding-screenshot" 
src="https://cloud.githubusercontent.com/assets/1509691/11944978/8787e3ec-a7ff-11e5-8383-456adb16b977.png";>
    
    
    ### Questions:
    * This could use documentation, which could just be the example in the 
screenshot. Where can I contribute that?

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sriramkrishnan/incubator-zeppelin scalding

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-zeppelin/pull/561.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #561
    
----
commit 721dcb7bcfbc56d923c101e5b701cd9c2ad23372
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-17T03:34:17Z

    Getting a basic interpreter going. Next step is to hook in the Scalding 
REPL.

commit 35fc032d7e6cc679cf3604f44cf6ba4a3545a53b
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-17T21:22:52Z

    Initial version of a ScaldingInterpreter running in local mode. Need to get 
console output next.
    And add tests, and make it work for HDFS.

commit e13576fcfea4fb86f8fafd46df716d2d50ef0b3c
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-17T22:35:53Z

    Now seem to be getting the console out, but only for last line. Will need 
some debugging.

commit b19fda41b586486e315f9dc6eaa3a71e4ecf7632
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-18T00:20:02Z

    More cleanup - flushing output stream. Still can't seem to get the Scala 
console output. Need to figure that out.

commit 1ffbb3b2d70d57ff65d0d43ef7f02defe263ac41
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-18T05:00:13Z

    Fixing output of stdout from console

commit d3916b751627b7fd02f9b583214d578236178864
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-18T05:01:02Z

    Adding modified version of ScaldingILoop for grabbing Console output - will 
want to move this to Scalding itself

commit 36a2dace020bea6bf9787778bfa6bf2ca5900132
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-18T19:58:30Z

    Added a link to the scalding code where the ILoop was lifted from.

commit 8944b0c3aebd2c4c66a12518613cb5f1724c0dee
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-21T22:08:59Z

    Merge remote-tracking branch 'upstream/master' into scalding

commit c27ec48ff73093ebfd9a71a3f0d31b9626edc91c
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-21T22:25:34Z

    Formatting, license

commit 368dc042ef71c08d3e0b5ccc879bef0b1bce2da1
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-21T23:00:25Z

    Cleaning up imports, comments, etc

commit 7ec294138078bc3a196e985a3c3f211c51fe2566
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-21T23:22:08Z

    More code cleanup

commit 7a9ceeb5b801017fb7354ef5b3d0e2b3256d7ae9
Author: Sriram Krishnan <[email protected]>
Date:   2015-12-22T00:04:21Z

    Adding some tests for the Scalding interpreter

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---

Reply via email to