If I'm reading https://issues.apache.org/jira/browse/ZEPPELIN-635 correctly- this integrates the spark, markdown, and shell interpreters.
Back to my original post, I essentially want to add Flink to that list. To your point about keeping a small and managable code-base: Under the hood it seems like Zeppelin is a front end for Spark and oh btw, here are some hacks to make other stuff work too. For instance there is a lot of code reusage in any scala based interpreter. Wouldn't it make more sense to have a generic Scala interpreter and extend it for special quirks of each interpreter as needed, e.g. for the variable bindings of the particular interpreter, and loading configurations. Consider the companion object bug, essentially the same code had to be copy and pasted across 4 interpreters, and the Ignite interpreter (as I recall) never even got the fix because of a quirk in the way the tests are written for that interpreter. I was looking through your resourcePools. I am under the impression I can use those to pass a variable from one paragraph to another, in an akward sort of fasion (but I may be going about it all wrong). Supposing that can be done (or possibly is already done, but I haven't read the PRs you listed carefully), it would solve what I want to do for the time being. Also consider the Python Flink I want to add to this, there will once again be a lot of duplication of code from the Spark Python interpreter. A generic Python interpreter also seems like a more reasonable approach here. So basically I've broken this conversation into two parts- 1) I'm trying to pass variables/object back and forth between Spark/Flink/Angular/etc. Please help. Seems possible but I'm having a slow time figuring it out 2) If we want to keep the code base compact and clean, would it be wiser to refactor in a less Spark-centric way? Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org *"Fortunate is he, who is able to know the causes of things." -Virgil* On Fri, Apr 22, 2016 at 3:41 PM, DuyHai Doan <doanduy...@gmail.com> wrote: > In this case, it is already implemented. > > Look at those merged PR: > > - https://github.com/apache/incubator-zeppelin/pull/739 > - https://github.com/apache/incubator-zeppelin/pull/740 > - https://github.com/apache/incubator-zeppelin/pull/741 > - https://github.com/apache/incubator-zeppelin/pull/742 > - https://github.com/apache/incubator-zeppelin/pull/744 > - https://github.com/apache/incubator-zeppelin/pull/745 > - https://github.com/apache/incubator-zeppelin/pull/832 > > There is one last JIRA pending for documentation, I'll do a PR for this > next week: https://issues.apache.org/jira/browse/ZEPPELIN-742 > > On Fri, Apr 22, 2016 at 9:52 PM, Trevor Grant <trevor.d.gr...@gmail.com> > wrote: > > > I want to be able to put/get/watch variables. Specifically so I can > > interface with AngularJS for visualizations. > > > > I've been groking the codebase trying to find a less invasive way to do > > this. > > > > I get wanting to keep the code base clean but sharing variables is a > really > > nice feature set and shouldn't be that hard to implement? > > > > Thoughts? > > > > Trevor Grant > > Data Scientist > > https://github.com/rawkintrevo > > http://stackexchange.com/users/3002022/rawkintrevo > > http://trevorgrant.org > > > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > > > > On Fri, Apr 22, 2016 at 1:06 PM, DuyHai Doan <doanduy...@gmail.com> > wrote: > > > > > I think we should rather let ZeppelinContext un-modified. > > > > > > If we update ZeppelinContext for every kind of interpreter, it would > > become > > > quickly a behemoth and un-manageable. > > > > > > The reason ZeppelinContext has some support for Spark is because it's > > > historical. Now that the project is going to gain wider audience, we > > should > > > focus on keeping the code as cleanest and as modular as possible. > > > > > > Can you explain which feature you want to add to ZeppelinContext that > > will > > > be useful for Flink ? > > > > > > > > > > > > On Fri, Apr 22, 2016 at 7:12 PM, Trevor Grant < > trevor.d.gr...@gmail.com> > > > wrote: > > > > > > > If one were to extend the Zeppelin context for Flink, I was thinking > it > > > > would make the most sense to update > > > > > > > > ../spark/src/main/java/org/apache/zeppelin/spark/ZeppelinContext.java > > > > > > > > Any thoughts from those who are more familiar with that end of the > code > > > > base than I? > > > > > > > > Ideally we'd have a solution that extend the Zeppelin Context to all > > > > interpreters. I know y'all love Spark but there ARE others out > > there... > > > > > > > > Anyone have any branches / previous attempts I could check out? > > > > > > > > tg > > > > > > > > > > > > Trevor Grant > > > > Data Scientist > > > > https://github.com/rawkintrevo > > > > http://stackexchange.com/users/3002022/rawkintrevo > > > > http://trevorgrant.org > > > > > > > > *"Fortunate is he, who is able to know the causes of things." > -Virgil* > > > > > > > > > >
