Hi forks,
There were an offline meeting yesterday at PaloAlto with contributors and
users. We've shared idea about current state of project and future project
roadmap and wishlists (meeting note [1]). It was really inspiring and
exciting time. Let me try summarize, move this discussion to online.
There were many ideas related to Interpreter. Especially, there were
consensus that Spark support is one of biggest strength of Zeppelin and
need to make further improvement to keep the strengths.
- Spark
- Immediate support of new spark release
- Ramp up support of current Spark feature (e.g. Display job progress
correctly)
- Spark streaming support
- Handling Livy timeout
- Other interpreters
- Better Hive support (e.g. configuration)
- Latest version PrestoDB support (pass property correctly)
- Run interpreter in containerized environment
- Let individual user upload custom library from user's machine directly
- Interpreter documentation is not detail enough
And people in the meeting excited about ConInterpreter ZEPPELIN-3085 [2] in
upcoming release, regarding dynamic/inline configuration of interpreter.
And there were ideas on other areas, too. like
- Separate Admin role and user role
- Sidebar with plugin widget
- Better integration with emerging framework like Tensorflow/MXNet/Ray
- Sharing data
- Schedule notebook from external scheduler
Regarding scheduling notebook, Luciano shared his project NotebookTools[3]
and it made people really excited.
Also, there were inspiring discussions about the community/project. Current
status and how can we make community/project more healthy. And here's some
ideas around the topic
- Need more frequent release
- More attention to code review to speed up
- Publishing roadmap beforehand to help contribution
- 'Newbie', 'low hanging fruit' tag helps contribution
- Enterprise friendly is another biggest strength of Zeppelin (in
addition to Spark support) need to keep improve.
I probably missed many idea shared yesterday. Please feel free to
add/correct the summary. Hope more people in the mailinglist join and
develop the idea together. And I think this discussion can leads community
shape 0.9 and future version of Zeppelin, and update and publish future
roadmap[4].
Best,
moon
Special thanks to ZEPL <https://www.zepl.com> for the swag and dinner.
[1]
https://docs.google.com/document/d/18Wc3pEFx3qm9XoME_V_B9k_LlAd1PLyKQQEveR1on1o/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/ZEPPELIN-3085
[3]
https://github.com/SparkTC/notebook-exporter/tree/master/notebook-exporter
[4] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
