Re: Conceptual difference Windows and DataSet

2016-08-12 Thread Stephan Ewen
Hi Kevin! The windows in Flink's DataStream API are organized by key. The reason is that the windows are very flexible, and each key can form different windows than the other (think sessions per user - each session starts and stops differently). There has been discussion about introducing somethi

Re: Conceptual difference Windows and DataSet

2016-08-06 Thread Theodore Vasiloudis
Hello Kevin, I'm not very familiar with the stream API, but I think you can achieve what you want by mapping over your elements to turn the strings into one-item lists, so that you get a key-value that is (K: String, V: (List[String], Int)) and then apply the window reduce function, which produce

Conceptual difference Windows and DataSet

2016-08-04 Thread Kevin Jacobs
Hi, I have the following use case: 1. Group by a specific field. 2. Get a list of all messages belonging to the group. 3. Count the number of records in the group. With the use of DataSets, it is fairly easy to do this (see http://stackoverflow.com/questions/38745446/apache-flink