An interesting apache project: Reef

2016-01-20 Thread kovas boguta
Some people here (especially Flink contributors) might be interested to know about this project: https://reef.apache.org/index.html It is lower-level than Flink (and less mature), but with similar architectural sensibilities and emphasis on interfaces. It would be pretty interesting to compare the

Re: Questions re: ExecutionGraph & ResultPartitions for interactive use a la Spark

2016-01-14 Thread kovas boguta
On Thu, Jan 14, 2016 at 4:00 PM, kovas boguta wrote: > > For a "real" solution, the REPL needs seem related to the WebUI, which I > haven't studied yet. One would want a fairly detailed view into the running > execution graph, possibly but not necessarily as an HTTP api

Re: Questions re: ExecutionGraph & ResultPartitions for interactive use a la Spark

2016-01-14 Thread kovas boguta
On Thu, Jan 14, 2016 at 5:52 AM, Ufuk Celebi wrote: > Hey Kovas > > sorry for the long delay. > It was worth the wait! Thanks for the detailed response. > Ideally, I could force certain ResultPartitions to only be manually > releasable, so I can consume them over and over. > > How would you lik

Re: Questions re: ExecutionGraph & ResultPartitions for interactive use a la Spark

2016-01-09 Thread kovas boguta
oper description of the current > situation and some pointers. > > Cheers, > Aljoscha > > On 04 Jan 2016, at 22:47, kovas boguta wrote: > > > > I'm impressed with the Flink API, it seems simpler and more composable > than what I've seen elsewhere. > >

Questions re: ExecutionGraph & ResultPartitions for interactive use a la Spark

2016-01-04 Thread kovas boguta
I'm impressed with the Flink API, it seems simpler and more composable than what I've seen elsewhere. I'm trying to see how to achieve a more interactive, REPL-driven experience, similar to Spark. I'm consuming Flink from Clojure. For now I'm only interested in smaller clusters & interactive usag