Moon: Are z. Context and resource pool different or the same? Sent from my iPhone
> On Jun 16, 2016, at 10:23 PM, Cameron McBride <cameron.mcbr...@gmail.com> > wrote: > > Appreciate the quick responses! > > The patch (#pr836) does sound like a good solution, as it will generically > address this issue (and makes use of the existing ResourcePool framework). > The example of this working with the shell is also nice. Obviously my vote is > to include it in 0.6.0. :) > > Are there any good examples on how to use the resource pool? Or a pointer to > some documentation / guide / notes of what data lives where (within > interpreter, Angular, ResourcePool, etc.)? > > Cameron > > > >> On Thu, Jun 16, 2016 at 12:30 PM, moon soo Lee <m...@apache.org> wrote: >> ResourcePool [1] is a data store that all the different types of interpreter >> can access. >> It allows exchange data between paragraphs/notebooks/interpreters. >> >> However, not all interpreter uses ResourcePool, at the moment. >> For example Spark interpreter family provides API to use it (spark, pyspark, >> sparkr), which is z.put(), z.get(), but all other interpreter implementation >> does not leverage ResourcePool at the moment. >> >> We can simply modify PostgreSQL interpreter to push result into >> ResourcePool, so other interpreters, for example SparkInterpreter, can get >> result from ResourcePool and process. I think this implementation will be as >> simple as few lines of code. >> >> Alternative approach is #pr836[2] (hope we can merge this, soon). This patch >> includes the code that automatically pushes all the results into >> ResourcePool, without modification of each interpreter. >> >> Thanks, >> moon >> >> [1] >> https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/resource/ResourcePool.java >> [2] https://github.com/apache/zeppelin/pull/836 >> >> >>> On Thu, Jun 16, 2016 at 9:13 AM Mohit Jaggi <mohitja...@gmail.com> wrote: >>> I believe z.context() is the only way to share data between interpreters. >>> Within an interpreter data is usually available across paragraphs…perhaps >>> even across notebooks as I guess zeppelin will create a single interpreter >>> in the backend unless you somehow make it use separate ones. >>> >>> > On Jun 16, 2016, at 9:24 PM, Cameron McBride <cameron.mcbr...@gmail.com> >>> > wrote: >>> > >>> > Greetings, >>> > >>> > I'm brand new to zeppelin, and this notebook technology looks great. I'm >>> > evaluating using it for our data science team, but got it up and running >>> > quickly using some PostgreSQL data and some spark tests. The distributed >>> > nature of each paragraph, and naturally varying interpreters within a >>> > notebook is fantastic. Some of the current built in technology is so >>> > refreshing compared with trying to beat in such functionality to other >>> > notebook / sharing solutions. Anyhow, I'm stoked and wanted to share that >>> > first as part of my first post. :) >>> > >>> > In using the PostgreSQL interpreter, is there a way to share some output >>> > data with other paragraphs? I'm trying to run a SQL query, then perhaps >>> > do some analysis or derived logic in a different interpreter later in the >>> > notebook. Sorry if I missed this somewhere, but my digging couldn't find >>> > a workable answer. >>> > >>> > TIA, >>> > >>> > Cameron >>> > >>> > >
