Hi Moon,
Some of my requirements.
1. Can we achieve better memory management for notebooks ? I'm also
facing some similar OOM issue, like Dafeng mentioned in other
discussion.I'm using the iframe view of a paragraph, can we load that
code+results to memory only when requested ? I think this is one area to be
focused on.
2. In table/graph view can we include below features along with
pagination ?
a) Search , similar to
https://docs.angularjs.org/api/ng/filter/filter
b) Sorting of columns. Also custom sorting algorithms ?
Also any idea on GA for these suggested improvements ?
Regards
Shabeel
On Mon, Feb 29, 2016 at 1:51 PM, Vinayak Agrawal <[email protected]
> wrote:
> Moon,
> The new roadmap looks very promising. I am very happy to see security in
> the list.
> I have some suggestions regarding Enterprise Ready features:
>
> 1. Job Scheduler - Can this be improved?
> Currently the scheduler can be used with Cron expression or a pre-set
> time. But in an enterprise solution, a notebook might be one piece of the
> workflow. Can we look towards the functionality of scheduling notebook's
> based on other notebooks finishing their job successfully?
> This requirement would arise in any ETL workflow, where all the downstream
> users wait for the ETL notebook to finish successfully. Only after that,
> other business oriented notebooks can be executed.
>
> 2. Importing a notebook - Is there a current requirement or future plan to
> implement a feature that allows import-notebook-from-github? This would
> allow users to share notebooks seamlessly.
>
> Thanks
> Vinayak
>
> On Sun, Feb 28, 2016 at 11:22 PM, moon soo Lee <[email protected]> wrote:
>
>> Zhong Wang,
>> Right, Folder support would be quite useful. Thanks for the opinion.
>> Hope i can finish the work pr-190
>> <https://github.com/apache/incubator-zeppelin/pull/190>.
>>
>> Sourav,
>> Regarding concurrent running, Zeppelin doesn't have limitation of run
>> paragraph/query concurrently. Interpreter can implement it's own scheduling
>> policy. For example, SparkSQL interpreter and ShellInterpreter can already
>> run paragraph/query concurrently.
>>
>> SparkInterpreter is implemented with FIFO scheduler considering nature of
>> scala compiler. That's why user can not run multiple paragraph concurrently
>> when they work with SparkInterpreter.
>> But as Zhong Wang mentioned, pr-703 enables each notebook will have
>> separate scala compiler so paragraphs run concurrently, while they're in
>> different notebooks.
>> Thanks for the feedback!
>>
>> Best,
>> moon
>>
>> On Sat, Feb 27, 2016 at 8:59 PM Zhong Wang <[email protected]>
>> wrote:
>>
>>> Sourav: I think this newly merged PR can help you
>>> https://github.com/apache/incubator-zeppelin/pull/703#issuecomment-185582537
>>>
>>> On Sat, Feb 27, 2016 at 1:46 PM, Sourav Mazumder <
>>> [email protected]> wrote:
>>>
>>>> Hi Moon,
>>>>
>>>> This looks great.
>>>>
>>>> My only suggestion would be to include a PR/feature - Support for
>>>> Running Concurrent paragraphs/queries in Zeppelin.
>>>>
>>>> Right now if more than one user tries to run paragraphs in multiple
>>>> notebooks concurrently through a single Zeppelin instance (and single
>>>> interpreter instance) the performance is very slow. It is obvious that the
>>>> queue gets built up within the zeppelin process and interpreter process in
>>>> that scenario as the time taken to move the status from start to pending
>>>> and pending to running is very high compared to the actual running time of
>>>> a paragraph.
>>>>
>>>> Without this the multi tenancy support would be meaningless as no one
>>>> can practically use it in a situation where multiple users are trying to
>>>> connect to the same instance of Zeppelin (and the related interpreter). A
>>>> possible solution would be to spawn separate instance of the same
>>>> interpreter at every notebook/user level.
>>>>
>>>> Regards,
>>>> Sourav
>>>>
>>>> On Sat, Feb 27, 2016 at 12:48 PM, moon soo Lee <[email protected]> wrote:
>>>>
>>>>> Hi Zeppelin users and developers,
>>>>>
>>>>> The roadmap we have published at
>>>>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
>>>>> is almost 9 month old, and it doesn't reflect where the community goes
>>>>> anymore. It's time to update.
>>>>>
>>>>> Based on mailing list, jira issues, pullrequests, feedbacks from
>>>>> users, conferences and meetings, I could summarize the major interest of
>>>>> users and developers in 7 categories. Enterprise ready, Usability
>>>>> improvement, Pluggability, Documentation, Backend integration, Notebook
>>>>> storage, and Visualization.
>>>>>
>>>>> And i could list related subjects under each categories.
>>>>>
>>>>> - Enterprise ready
>>>>> - Authentication
>>>>> - Shiro authentication ZEPPELIN-548
>>>>> <https://issues.apache.org/jira/browse/ZEPPELIN-548>
>>>>> - Authorization
>>>>> - Notebook authorization PR-681
>>>>> <https://github.com/apache/incubator-zeppelin/pull/681>
>>>>> - Security
>>>>> - Multi-tenancy
>>>>> - Stability
>>>>> - Usability Improvement
>>>>> - UX improvement
>>>>> - Better Table data support
>>>>> - Download data as csv, etc PR-725
>>>>> <https://github.com/apache/incubator-zeppelin/pull/725>,
>>>>> PR-714
>>>>> <https://github.com/apache/incubator-zeppelin/pull/714>, PR-6
>>>>> <https://github.com/apache/incubator-zeppelin/pull/6>, PR-89
>>>>> <https://github.com/apache/incubator-zeppelin/pull/89>
>>>>> - Featureful table data display (pagenation, etc)
>>>>> - Pluggability ZEPPELIN-533
>>>>> <https://issues.apache.org/jira/browse/ZEPPELIN-533>
>>>>> - Pluggable visualization
>>>>> - Dynamic Interpreter, notebook, visualization loading
>>>>> - Repository and registry for pluggable components
>>>>> - Improve documentation
>>>>> - Improve contents and readability
>>>>> - more tutorials, examples
>>>>> - Interpreter
>>>>> - Generic JDBC Interpreter
>>>>> - (spark)R Interpreter
>>>>> - Cluster manager for interpreter (Proposal
>>>>>
>>>>> <https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Proposal>
>>>>> )
>>>>> - more interpreters
>>>>> - Notebook storage
>>>>> - Versioning ZEPPELIN-540
>>>>> <http://issues.apache.org/jira/browse/ZEPPELIN-540>
>>>>> - more notebook storages
>>>>> - Visualization
>>>>> - More visualizations PR-152
>>>>> <https://github.com/apache/incubator-zeppelin/pull/152>, PR-728
>>>>> <https://github.com/apache/incubator-zeppelin/pull/728>, PR-336
>>>>> <https://github.com/apache/incubator-zeppelin/pull/336>, PR-321
>>>>> <https://github.com/apache/incubator-zeppelin/pull/321>
>>>>> - Customize graph (show/hide label, color, etc)
>>>>>
>>>>>
>>>>> It will help anyone quickly get overall interest of project and the
>>>>> direction. And based on this roadmap, we can discuss and re-define the
>>>>> next
>>>>> release 0.6.0 scope and it's schedule.
>>>>>
>>>>> What do you think? Any feedback would be appreciated.
>>>>>
>>>>> Thanks,
>>>>> moon
>>>>>
>>>>>
>>>>
>
>
> --
> Vinayak Agrawal
>
>
> "To Strive, To Seek, To Find and Not to Yield!"
> ~Lord Alfred Tennyson
>
