Re: Current alternatives for async I/O

2016-10-13 Thread Fabian Hueske
Hi Ken, FYI: we just received a pull request for FLIP-12 [1]. Best, Fabian [1] https://github.com/apache/flink/pull/2629 2016-10-11 9:35 GMT+02:00 Fabian Hueske : > Hi Ken, > > I think your solution should work. > You need to make sure though, that you properly manage the state of your > funct

Re: Current alternatives for async I/O

2016-10-11 Thread Fabian Hueske
Hi Ken, I think your solution should work. You need to make sure though, that you properly manage the state of your function, i.e., memorize all records which have been received but haven't be emitted yet. Otherwise records might get lost in case of a failure. Alternatively, you can implement thi

Current alternatives for async I/O

2016-10-08 Thread Ken Krugler
Hi all, I’ve been watching the FLIP-12 design discussion, and it looks like a promising solution for the issues we’ve got with needing to make asynchronous multi-threaded requests in a Flink operator. What’s the best