Re: design question

2016-04-23 Thread John Sherwood
This sounds like you have some per-key state to keep track of, so the 'correct' way to do it would be to keyBy the guid. I believe that if you run your environment using the Rocks DB state backend you will not OOM regardless of the number of GUIDs that are eventually tracked. Whether flink/stream p

Flink first() operator

2016-04-23 Thread Biplob Biswas
Hi, It might be a naive question but I was concerned as I am trying to read from a file. My question is if I have a file with n lines and i want m lines out of that where n << m, would the first operator process only the first m lines or would it go through the entire file? If it does go through

Re: Count windows missing last elements?

2016-04-23 Thread Konstantin Kulagin
I finally was able to do that. Kinda ugly, but works: https://gist.github.com/krolen/ed1344e4d7be5b2116061685268651f5 On Fri, Apr 22, 2016 at 6:14 PM, Konstantin Kulagin wrote: > I was trying to implement this (force flink to handle all values from > input) but had no success... > Probably I

[ANNOUNCE] Flink 1.0.2 Released

2016-04-23 Thread Ufuk Celebi
The Flink PMC is pleased to announce the availability of Flink 1.0.2. The official release announcement: http://flink.apache.org/news/2016/04/22/release-1.0.2.html Release binaries: http://apache.openmirror.de/flink/flink-1.0.2/ Please update your Maven dependencies to the new 1.0.2 version and

design question

2016-04-23 Thread Chen Bekor
hi all, I have a stream of incoming object versions (objects change over time) and a requirement to fetch from a datastore the last known object version in order to link it with the id of the new version, so that I end up with a linked list of object versions. all object versions contain the sam

Re: Flink program without a line of code

2016-04-23 Thread Flavio Pompermaier
Hi Alexander, since I was looking for something similar some days ago here is what I know about this argument: during the Stratosphere project there was Meteor and Supremo allowing that [1] but then it was dismissed in favour of Pig integration that I don't wheter it was ever completed. Yiu could g