Run all paragraph above/below the highlighted one

2017-10-02 Thread Luca Falsina
Hello everyone, I looked through the UI and across previous JIRAs but I could not find any mention for this feature so I'll ask here: Does anyone know if there is an efficient way / shortcut in Zeppelin to run all paragraphs above or below a cell? I only know of the 'run all paragraphs' options,

Is any limitation of maximum interpreter processes?

2017-10-02 Thread Belousov Maksim Eduardovich
Hello, users! Our analysts run notes with such interpreters: markdown, one or two jdbc and pyspark. The interpreters are instantiated Per User in isolated process and Per Note in isolated process. And the analysts complain that sometimes paragraphs aren't processed and stay in status 'Pending'

How to update paragraph text

2017-10-02 Thread Plamen Paskov
Hello everyone, I need a functionality to update the paragraph text from spark interpreter. I was searching over the internet and looked at the source code but i didn't find anything. Is it possible to achieve that? If not how i can achieve the following: - connect to kinesis stream - create

RE: Implementing run all paragraphs sequentially

2017-10-02 Thread Polyakov Valeriy
Let me try to summarize the discussion. Evidently, current behavior of running notes does not meet actual requirements. The most important thing that we need is the ability of sequential running. However, at the same time we want to keep functionality of parallel running. We discussed that the m

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Herval Freire
Why do you need rules and graphs and any of that to support running everything sequentially or everything in parallel? 3) add a “run mode” to the note. If it’s “sequential”, run the paragraphs one at a time, in the order they’re defined. If parallel, run using current scheme (as many at the sam

RE: Implementing run all paragraphs sequentially

2017-10-02 Thread Polyakov Valeriy
Option (3) does not solve the problem of manageable running order which is really useful in some cases (sequential running of different interpreters and parallel running of same interpreters in one note). Valeriy Polyakov From: Herval Freire [mailto:hfre...@twitter.com] Sent: Monday, October

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Michael Segel
Because that simplicity doesn’t work. You will want to run some things serial and some things in parallel. Which is why you will need a dependency graph. On Oct 2, 2017, at 10:40 AM, Herval Freire mailto:hfre...@twitter.com>> wrote: Why do you need rules and graphs and any of that to support r

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Herval Freire
Did anyone request such a case ("running some in parallel and some in sequence")? I haven't seen any requests for this in the wild (nor on this thread), other than theoretical "what if" - which is totally fine, when it doesn't introduce a lot of unecessary complexity for little to no gain (which se

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Patrick Maroney
+1 on DAG. +1 on adopting an existing mature DAG framework. +1 on ability to stitch together different Interpreters in this framework (e.g. Python, ElasticSearch). Patrick Maroney Principal Engineer - Data Science & Analytics Wapack Labs LLC Public Key: http://pgp.mit.edu/pks/lookup?op=get&sear

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Michael Segel
Yes… You have bunch of unit tests you can run in parallel where you only need one constructor and one cleanup. I would strongly suggest that you really, really think about this long and hard before you start to pound code. Its going to be harder to back out and fix than if you take the time to

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Herval Freire
"nice to have" isn't a very strong requirement. I strongly uggest you really, really think about this before you start pounding an overengineered solution to a non-issue :-) h On Mon, Oct 2, 2017 at 9:12 AM, Michael Segel wrote: > Yes… > You have bunch of unit tests you can run in parallel wh

Re: How to update paragraph text

2017-10-02 Thread Chi-Lang Ngo
Not sure if it solves your use-case but updating paragraph text via Zeppelin REST API will soon be available (Pending PR https://github.com/apache/zeppelin/pull/2604) On 2 October 2017 at 15:32, Plamen Paskov wrote: > Hello everyone, > > I need a functionality to update the paragraph text from s

Re: Implementing run all paragraphs sequentially

2017-10-02 Thread Michael Segel
Whoa! Seems I walked in to something. Herval, What do you suggest? A simple switch that runs everything in serial, or everything in parallel? That would be a very bad idea. I gave you an example of a class of solutions where you don’t want that behavior. E.g Unit testing where you have one se

Re: Is any limitation of maximum interpreter processes?

2017-10-02 Thread Jianfeng (Jeff) Zhang
Which interpreter is pending ? It is possible that spark interpreter pending due to yarn resource capacity if you run it in yarn client mode If it is pending, you can check the log first. Best Regard, Jeff Zhang From: Belousov Maksim Eduardovich mailto:m.belou...@tinkoff.ru>> Reply-To: "us

How to execute spark-submit on Note

2017-10-02 Thread 小野圭二
Hi all, I searched this topic on the archive of ml, but still could not find out the solution clearly. So i have tried to post this again(maybe). I am using ver 0.8.0, and have installed spark 2.2 on the other path, just for checking my test program. Then i wrote a quite simple sample python code