Hey all, I had a post a while ago about needing neural networks. We specifically need a very special type that are good for time series/sensors called LSTM. We had a talk about pros/cons of using deeplearning4j for this use case and eventually decided it made more sense to implement in native Flink for our use case.
So, this is somewhat relevant to what Theodore just said, but different enough that I wanted a separate thread. "Focusing on Flink does well and implement algorithms built around inherent advantages..." One thing that jumps to mind is doing online learning. The batch nature of all of the other 'big boys' means that they are by definition going to always be offline modes. Also, even though LTSMs are somewhat of a corner case in the NN world, the streaming nature of Flink (a sequence of data) makes fairly relevant to people who would be using Flink in the first place (? IMHO) Finally, there should be some positive externalities that come from this such as a back propegation algorithm, which should then be reusable for things like HMMs. So at any rate, the research Spike for me started earlier this week- I hope to start cutting some scala code over the weekend or beginning of next week. Also I'm asking to check out FLINK-2259 because I need some sort of functionality like that before I get started, and I could use the git practice. Idk if there is any interest in adding this or if you want to make a JIRA for LTSM neural nets (or if I should write one, with appropriate papers cited, as seems to be the fashion), or maybe wait and see what I end up with? Also- I'll probably be blowing you up with questions. Best, tg Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org *"Fortunate is he, who is able to know the causes of things." -Virgil*
