it in
>> future although personally I prefer to define DAG for note.
>>
>>
>> Best Regard,
>> Jeff Zhang
>>
>>
>> From: Michael Segel
>> Reply-To: "users@zeppelin.apache.org"
>> Date: Friday, October 6, 2017 at 10:08 PM
>> To: &quo
> Date: Friday, October 6, 2017 at 10:08 PM
> To: "users@zeppelin.apache.org"
> Subject: Re: Implementing run all paragraphs sequentially
>
> Guys…
>
> 1) You’re posting this to the user list… Isn’t this a dev question?
>
> 2) +1 on the run serial… but doesn’t that alr
ph4
>
>
>
> Dave
>
>
>
> *From: *Jianfeng (Jeff) Zhang
> *Sent: *Saturday, 7 October 2017 11:57 AM
>
>
> *To: *users@zeppelin.apache.org
> *Subject: *Re: Implementing run all paragraphs sequentially
>
> Since almost everyone agree on to run serial by de
m>
Sent: Saturday, 7 October 2017 11:57 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Re: Implementing run all paragraphs sequentially
Since almost everyone agree on to run serial by default. We could implement it
first. Regarding the parallel mode, we
ply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>"
mailto:users@zeppelin.apache.org>>
Date: Friday, October 6, 2017 at 10:08 PM
To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>"
mailto:users@zeppelin.apache.org>>
Subject
https://issues.apache.org/jira/browse/ZEPPELIN-2368
Valeriy Polyakov
From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: Friday, October 06, 2017 10:14 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Re: Implementing run all paragraphs sequentially
+1 for serial ru
Polyakov
From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: Friday, October 06, 2017 10:14 AM
To: users@zeppelin.apache.org
Subject: Re: Implementing run all paragraphs sequentially
+1 for serial run by default. Let's leave others in future.
Mohit Jaggi mailto:mohitja...@gmail.com>>于2017
se that simplicity doesn’t work.
>>>>
>>>> You will want to run some things serial and some things in parallel.
>>>>
>>>> Which is why you will need a dependency graph.
>>>>
>>>> On Oct 2, 2017, at 10:40 AM, Herval Freire wrote:
>>>>
>>>
Mon, Oct 2, 2017 at 8:48 AM, Michael Segel
>>>>>> wrote:
>>>>>> Because that simplicity doesn’t work.
>>>>>>
>>>>>> You will want to run some things serial and some things in parallel.
>>>>>>
>>>>>> W
support running
>>> everything sequentially or everything in parallel?
>>>
>>> 3) add a “run mode” to the note. If it’s “sequential”, run the
>>> paragraphs one at a time, in the order they’re defined. If parallel, run
>>> using current scheme (as many
the same time as the threadpool permits)
Simpler and covers all cases, imo
From: Polyakov Valeriy mailto:v.polja...@tinkoff.ru>>
Sent: Monday, October 2, 2017 8:24:35 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: RE: Implementing run all paragraphs seque
same time as the threadpool permits)
>>
>> Simpler and covers all cases, imo
>>
>> --
>> *From:* Polyakov Valeriy
>> *Sent:* Monday, October 2, 2017 8:24:35 AM
>> *To:* users@zeppelin.apache.org
>> *Subject:* RE: Implementing run
pool permits)
Simpler and covers all cases, imo
From: Polyakov Valeriy mailto:v.polja...@tinkoff.ru>>
Sent: Monday, October 2, 2017 8:24:35 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: RE: Implementing run all paragraphs sequentially
Let me try to summarize t
+1 on DAG.
+1 on adopting an existing mature DAG framework.
+1 on ability to stitch together different Interpreters in this framework (e.g.
Python, ElasticSearch).
Patrick Maroney
Principal Engineer - Data Science & Analytics
Wapack Labs LLC
Public Key: http://pgp.mit.edu/pks/lookup?op=get&sear
he order they’re defined. If parallel, run using current
> scheme (as many at the same time as the threadpool permits)
>
> Simpler and covers all cases, imo
>
> --
> *From:* Polyakov Valeriy
> *Sent:* Monday, October 2, 2017 8:24:35 AM
> *To:* users
rs all cases, imo
From: Polyakov Valeriy mailto:v.polja...@tinkoff.ru>>
Sent: Monday, October 2, 2017 8:24:35 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: RE: Implementing run all paragraphs sequentially
Let me try to summar
02, 2017 6:41 PM
To: users@zeppelin.apache.org; users@zeppelin.apache.org
Subject: Re: Implementing run all paragraphs sequentially
Why do you need rules and graphs and any of that to support running everything
sequentially or everything in parallel?
3) add a "run mode" to the not
same time as the threadpool permits)
Simpler and covers all cases, imo
From: Polyakov Valeriy
Sent: Monday, October 2, 2017 8:24:35 AM
To: users@zeppelin.apache.org
Subject: RE: Implementing run all paragraphs sequentially
Let me try to summarize the discussion
]
Sent: Saturday, September 30, 2017 4:22 PM
To: users@zeppelin.apache.org
Subject: Re: Implementing run all paragraphs sequentially
Sorry to jump in…
If you want to run paragraphs in parallel, you are going to want to have some
sort of dependency graph. Think of a common set up where you need to
entially are
expected to *always* run sequentially - thus it makes more sense to be a note
option than a per-run mode
H
_
From: moon soo Lee mailto:m...@apache.org>>
Sent: Thursday, September 28, 2017 9:03 PM
Subject: Re: Implementing run all paragraphs seque
>> H
>>
>> _
>> From: moon soo Lee
>> Sent: Thursday, September 28, 2017 9:03 PM
>> Subject: Re: Implementing run all paragraphs sequentially
>> To:
>>
>>
>> This is going to be really useful!
>>
>&
ode
>
> H
>
> _
> From: moon soo Lee
> Sent: Thursday, September 28, 2017 9:03 PM
> Subject: Re: Implementing run all paragraphs sequentially
> To:
>
>
> This is going to be really useful!
>
> Curios why do you prefer 'note option' instea
7 9:03 PM
Subject: Re: Implementing run all paragraphs sequentially
To: mailto:users@zeppelin.apache.org>>
This is going to be really useful!
Curios why do you prefer 'note option' instead of 'run option'?
Could you compare their pros and cons?
Thanks,
moon
On Thu
even
frustrating, especially when ‘20170929-143857_1744629322’ names are involved.
Sergey Sotnichenko
From: Polyakov Valeriy [mailto:v.polja...@tinkoff.ru]
Sent: Friday, September 29, 2017 3:11 PM
To: users@zeppelin.apache.org
Subject: RE: Implementing run all paragraphs sequentially
This can
if the latter then the paragraph ids wouldn’t be very easy for
someone to read and follow the dependency relationships…
From: Jeff Zhang [mailto:zjf...@gmail.com<mailto:zjf...@gmail.com>]
Sent: 29 September 2017 11:58
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org&
’s imagine we have 20 paragraphs with such
>> names.
>>
>>
>>
>>
>>
>>
>> *Sergey Sotnichenko *
>>
>>
>>
>>
>>
>> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
>> *Sent:* Friday, September 29, 2017 2:35 PM
>> *To:*
t; *Sent:* Friday, September 29, 2017 2:35 PM
> *To:* users@zeppelin.apache.org
> *Subject:* Re: Implementing run all paragraphs sequentially
>
>
>
>
>
> 'p1', 'p2' is paragraphId. Regarding the readability, we could allow user
> to set paragraph name
and to use.
Valeriy Polyakov
From: Sotnichenko Sergey [mailto:s.sotniche...@tinkoff.ru]
Sent: Friday, September 29, 2017 2:45 PM
To: users@zeppelin.apache.org
Subject: RE: Implementing run all paragraphs sequentially
It would be very complicated to be honest to build a DAG with names like
Subject: Re: Implementing run all paragraphs sequentially
'p1', 'p2' is paragraphId. Regarding the readability, we could allow user to
set paragraph name, but this is another story, could be an improvement later.
Partridge, Lucas (GE Aviation)
mailto:lucas.partri...@ge.com&g
ationships…
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* 29 September 2017 11:58
> *To:* users@zeppelin.apache.org
> *Subject:* EXT: Re: Implementing run all paragraphs sequentially
>
>
>
>
>
> I don't think 2 note setting (parallel/sequen
to read and follow the dependency relationships…
From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: 29 September 2017 11:58
To: users@zeppelin.apache.org
Subject: EXT: Re: Implementing run all paragraphs sequentially
I don't think 2 note setting (parallel/sequential) is sufficient for para
I don't think 2 note setting (parallel/sequential) is sufficient for
paragraph scheduling (take the spark tutorial note as an example, we should
run the loading bank data paragraph first and then could run all the sql
paragraph parallelly). So the key is how we define the dependency
relationship b
+1
I think this is one of the most important features. don't know why this
requirement has been skipped.
/afancy
On Thu, Sep 28, 2017 at 5:28 PM, Belousov Maksim Eduardovich <
m.belou...@tinkoff.ru> wrote:
> Hello, users!
>
> At the moment our analysts often use mixes of interpreters in their n
8 AM
To: users@zeppelin.apache.org
Subject: Implementing run all paragraphs sequentially
Hello, users!
At the moment our analysts often use mixes of interpreters in their notes.
For example, they prepare data using %jdbc and then use it in %pyspark.
Besides, they often use scheduling to ma
also often request this
>
> --
> *From:* Belousov Maksim Eduardovich
> *Sent:* Thursday, September 28, 2017 8:28:58 AM
> *To:* users@zeppelin.apache.org
> *Subject:* Implementing run all paragraphs sequentially
>
>
> Hello, users!
>
> At the moment our analysts
+1, our internal users at Twitter also often request this
From: Belousov Maksim Eduardovich
Sent: Thursday, September 28, 2017 8:28:58 AM
To: users@zeppelin.apache.org
Subject: Implementing run all paragraphs sequentially
Hello, users!
At the moment our analysts
Hello, users!
At the moment our analysts often use mixes of interpreters in their notes.
For example, they prepare data using %jdbc and then use it in %pyspark.
Besides, they often use scheduling to make some regular reporting. And they
should do something like `time.sleep()` to wait for the data
37 matches
Mail list logo
