Asking as someone that never did NN on Flink, would you implement it using JCuda? And would you implement it with model parallelization? Is there any theoretical limit to implement "model and data parallelism" in Flink? If you don't use GPUs and you don't parallelize models and data at the same time, what is your motivation to do such a thing on Flink instead of a local enviroment that would probably be more performant on a certain degree?
2016-02-12 14:58 GMT+01:00 Trevor Grant <trevor.d.gr...@gmail.com>: > Agreed. Our reasoning for for contributing straight to Flink was we plan on > doing a lot of wierd monkey-ing around with these things, and were going to > have to get our hands dirty with some code eventually anyway. The LSTM > isn't *that* difficult to implement, and it seems easier to write our own > than to understand someone else's insanity. > > The plan is to get a 'basic' version going, then start tweaking the special > cases. We have a use case for bi-directional, but it's not our primary > motivation. I have no problem exposing new flavors as we make them. > > tg > > > Trevor Grant > Data Scientist > https://github.com/rawkintrevo > http://stackexchange.com/users/3002022/rawkintrevo > http://trevorgrant.org > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > On Fri, Feb 12, 2016 at 7:51 AM, Suneel Marthi <suneel.mar...@gmail.com> > wrote: > > > On Fri, Feb 12, 2016 at 8:45 AM, Trevor Grant <trevor.d.gr...@gmail.com> > > wrote: > > > > > Hey all, > > > > > > I had a post a while ago about needing neural networks. We > specifically > > > need a very special type that are good for time series/sensors called > > > LSTM. We had a talk about pros/cons of using deeplearning4j for this > use > > > case and eventually decided it made more sense to implement in native > > Flink > > > for our use case. > > > > > > So, this is somewhat relevant to what Theodore just said, but different > > > enough that I wanted a separate thread. > > > > > > "Focusing on Flink does well and implement algorithms built around > > inherent > > > advantages..." > > > > > > One thing that jumps to mind is doing online learning. The batch > nature > > of > > > all of the other 'big boys' means that they are by definition going to > > > always be offline modes. > > > > > > Also, even though LTSMs are somewhat of a corner case in the NN world, > > the > > > streaming nature of Flink (a sequence of data) makes fairly relevant to > > > people who would be using Flink in the first place (? IMHO) > > > > > > Finally, there should be some positive externalities that come from > this > > > such as a back propegation algorithm, which should then be reusable for > > > things like HMMs. > > > > > > So at any rate, the research Spike for me started earlier this week- I > > hope > > > to start cutting some scala code over the weekend or beginning of next > > > week. Also I'm asking to check out FLINK-2259 because I need some sort > of > > > functionality like that before I get started, and I could use the git > > > practice. > > > > > > Idk if there is any interest in adding this or if you want to make a > JIRA > > > for LTSM neural nets (or if I should write one, with appropriate papers > > > cited, as seems to be the fashion), or maybe wait and see what I end up > > > with? > > > > > > It would be good if we also supported Bidirectional LSTMs. > > > > http://www.cs.toronto.edu/~graves/asru_2013.pdf > > > > http://www.cs.toronto.edu/~graves/phd.pdf > > > > > > > > > > > Also- I'll probably be blowing you up with questions. > > > > > > Best, > > > > > > tg > > > > > > > > > > > > Trevor Grant > > > Data Scientist > > > https://github.com/rawkintrevo > > > http://stackexchange.com/users/3002022/rawkintrevo > > > http://trevorgrant.org > > > > > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > > > > >
