Re: New Flink team member - Kate Eri.

Hi Kate, that's great news. This would help to boost ML on Flink a lot :) Best regards, Felix 2017-02-13 14:09 GMT+01:00 Katherin Eri : > Hello guys, > > > > It seems that issue FLINK-1730 > significantly impacts > integration of Flink with Sys

Re: New Flink team member - Kate Eri.

Hello guys, It seems that issue FLINK-1730 significantly impacts integration of Flink with SystemML. They have checked several integrations, and Flink’s integration is slowest

Re: New Flink team member - Kate Eri.

I have created the ticket to discuss GPU related questions futher https://issues.apache.org/jira/browse/FLINK-5782 пт, 10 февр. 2017 г. в 18:16, Katherin Eri : > Thank you, Trevor! > > You have shared very valuable points; I will consider them. > > So I think, I should create finally ticket at Fl

Re: New Flink team member - Kate Eri.

Thank you, Trevor! You have shared very valuable points; I will consider them. So I think, I should create finally ticket at Flink’s JIRA, at least for Flink's GPU support and move the related discussion there? I will contact to Suneel regarding DL4J, thanks! пт, 10 февр. 2017 г. в 17:44, Trev

Re: New Flink team member - Kate Eri.

Also RE: DL4J integration. Suneel had done some work on this a while back, and ran into issues. You might want to chat with him about the pitfalls and 'gotchyas' there. Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgr

Re: New Flink team member - Kate Eri.

Sorry for chiming in late. GPUs on Flink. Till raised a good point- you need to be able to fall back to non-GPU resources if they aren't available. Fun fact: this has already been developed for Flink vis-a-vis the Apache Mahout project. In short- Mahout exposes a number of tensor functions (vec

Re: New Flink team member - Kate Eri.

Thank you Felix, for provided information. Currently I analyze the provided integration of Flink with SystemML. And also gather the information for the ticket FLINK-1730 , maybe we will take it to work, to unlock SystemML/Flink integration. чт

Re: New Flink team member - Kate Eri.

Hi Kate, 1) - Broadcast: https://cwiki.apache.org/confluence/display/FLINK/FLIP-5%3A+Only+send+data+to+each+taskmanager+once+for+broadcasts - Caching: https://issues.apache.org/jira/browse/FLINK-1730 2) I have no idea about the GPU implementation. The SystemML mailing list will probably help you

Re: New Flink team member - Kate Eri.

Thank you Felix, for your point, it is quite interesting. I will take a look at the code, of the provided Flink integration. 1)You have these problems with Flink: >>we realized that the lack of a caching operator and a broadcast issue highly effects the performance, have you already asked abo

Re: New Flink team member - Kate Eri.

Hi Katherin, we are also working in a similar direction. We implemented a prototype to integrate with SystemML: https://github.com/apache/incubator-systemml/pull/119 SystemML provides many different matrix formats, operations, GPU support and a couple of DL algorithms. Unfortunately, we realized t

Re: New Flink team member - Kate Eri.

Thank you, Till. 1) Regarding ND4J, I didn’t know about such a pity and critical restriction of it -> lack of sparsity optimizations, and you are right: this issue is still actual for them. I saw that Flink uses Breeze, but I thought its usage caused by some historical reasons. 2) Regar

Re: New Flink team member - Kate Eri.

Thanks for initiating this discussion Katherin. I think you're right that in general it does not make sense to reinvent the wheel over and over again. Especially if you only have limited resources at hand. So if we could integrate Flink with some existing library that would be great. In the past,

Re: New Flink team member - Kate Eri.

Thank you Theodore, for your reply. 1)Regarding GPU, your point is clear and I agree with it, ND4J looks appropriate. But, my current understanding is that, we also need to cover some resource management questions -> when we need to provide GPU support we also need to manage it like resource.

Re: New Flink team member - Kate Eri.

Hell all, This is point that has come up in the past: Given the multitude of ML libraries out there, should we have native implementations in FlinkML or try to integrate other libraries instead? We haven't managed to reach a consensus on this before. My opinion is that there is definitely value i

Re: New Flink team member - Kate Eri.

Hello, guys. Theodore, last week I started the review of the PR: https://github.com/apache/flink/pull/2735 related to *word2Vec for Flink*. During this review I have asked myself: why do we need to implement such a very popular algorithm like *word2vec one more time*, when there is already avai

Re: New Flink team member - Kate Eri.

Sorry, guys I need to finish this letter first. Full version of it will come shortly. пн, 6 февр. 2017 г. в 12:49, Katherin Eri : > Hello, guys. > Theodore, last week I started the review of the PR: > https://github.com/apache/flink/pull/2735 related to *word2Vec for Flink*. > > During this rev

Re: New Flink team member - Kate Eri.

Hello, guys. Theodore, last week I started the review of the PR: https://github.com/apache/flink/pull/2735 related to *word2Vec for Flink*. During this review I have asked myself: why do we need to implement such a very popular algorithm like *word2vec one more time*, when there is already availab

Re: New Flink team member - Kate Eri.

Hi Katherin, welcome to the Flink community. Always great to see new people joining the community :-) Cheers, Till On Tue, Jan 17, 2017 at 1:02 PM, Katherin Sotenko wrote: > ok, I've got it. > I will take a look at https://github.com/apache/flink/pull/2735. > > вт, 17 янв. 2017 г. в 14:36, Th

Re: New Flink team member - Kate Eri.

ok, I've got it. I will take a look at https://github.com/apache/flink/pull/2735. вт, 17 янв. 2017 г. в 14:36, Theodore Vasiloudis < theodoros.vasilou...@gmail.com>: > Hello Katherin, > > Welcome to the Flink community! > > The ML component definitely needs a lot of work you are correct, we are

Re: New Flink team member - Kate Eri.

Hello Katherin, Welcome to the Flink community! The ML component definitely needs a lot of work you are correct, we are facing similar problems to CEP, which we'll hopefully resolve with the restructuring Stephan has mentioned in that thread. If you'd like to help out with PRs we have many open,

Re: New Flink team member - Kate Eri.

Hi Katherin, welcome to the Flink community! Help with reviewing PRs is always very welcome and a great way to contribute. Best, Fabian 2017-01-17 11:17 GMT+01:00 Katherin Sotenko : > Thank you, Timo. > I have started the analysis of the topic. > And if it necessary, I will try to perform the

Re: New Flink team member - Kate Eri.

Thank you, Timo. I have started the analysis of the topic. And if it necessary, I will try to perform the review of other pulls) вт, 17 янв. 2017 г. в 13:09, Timo Walther : > Hi Katherin, > > great to hear that you would like to contribute! Welcome! > > I gave you contributor permissions. You ca

Re: New Flink team member - Kate Eri.

Hi Katherin, great to hear that you would like to contribute! Welcome! I gave you contributor permissions. You can now assign issues to yourself. I assigned FLINK-1750 to you. Right now there are many open ML pull requests, you are very welcome to review the code of others, too. Timo Am 17

New Flink team member - Kate Eri.

Hello, All! I'm Kate Eri, I'm java developer with 6-year enterprise experience, also I have some expertise with scala (half of the year). Last 2 years I have participated in several BigData projects that were related to Machine Learning (Time series analysis, Recommender systems, Social network