Appreciate the quick responses! The patch (#pr836) does sound like a good solution, as it will generically address this issue (and makes use of the existing ResourcePool framework). The example of this working with the shell is also nice. Obviously my vote is to include it in 0.6.0. :)
Are there any good examples on how to use the resource pool? Or a pointer to some documentation / guide / notes of what data lives where (within interpreter, Angular, ResourcePool, etc.)? Cameron On Thu, Jun 16, 2016 at 12:30 PM, moon soo Lee <m...@apache.org> wrote: > ResourcePool [1] is a data store that all the different types of > interpreter can access. > It allows exchange data between paragraphs/notebooks/interpreters. > > However, not all interpreter uses ResourcePool, at the moment. > For example Spark interpreter family provides API to use it (spark, > pyspark, sparkr), which is z.put(), z.get(), but all other interpreter > implementation does not leverage ResourcePool at the moment. > > We can simply modify PostgreSQL interpreter to push result into > ResourcePool, so other interpreters, for example SparkInterpreter, can get > result from ResourcePool and process. I think this implementation will be > as simple as few lines of code. > > Alternative approach is #pr836[2] (hope we can merge this, soon). This > patch includes the code that automatically pushes all the results into > ResourcePool, without modification of each interpreter. > > Thanks, > moon > > [1] > https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/resource/ResourcePool.java > [2] https://github.com/apache/zeppelin/pull/836 > > > On Thu, Jun 16, 2016 at 9:13 AM Mohit Jaggi <mohitja...@gmail.com> wrote: > >> I believe z.context() is the only way to share data between interpreters. >> Within an interpreter data is usually available across paragraphs…perhaps >> even across notebooks as I guess zeppelin will create a single interpreter >> in the backend unless you somehow make it use separate ones. >> >> > On Jun 16, 2016, at 9:24 PM, Cameron McBride <cameron.mcbr...@gmail.com> >> wrote: >> > >> > Greetings, >> > >> > I'm brand new to zeppelin, and this notebook technology looks great. >> I'm evaluating using it for our data science team, but got it up and >> running quickly using some PostgreSQL data and some spark tests. The >> distributed nature of each paragraph, and naturally varying interpreters >> within a notebook is fantastic. Some of the current built in technology is >> so refreshing compared with trying to beat in such functionality to other >> notebook / sharing solutions. Anyhow, I'm stoked and wanted to share that >> first as part of my first post. :) >> > >> > In using the PostgreSQL interpreter, is there a way to share some >> output data with other paragraphs? I'm trying to run a SQL query, then >> perhaps do some analysis or derived logic in a different interpreter later >> in the notebook. Sorry if I missed this somewhere, but my digging couldn't >> find a workable answer. >> > >> > TIA, >> > >> > Cameron >> > >> > >> >>
