RE: Session windows - Evaluation of addition of element + window expires/gets discarded after some set time of inactivity

2016-10-21 Thread bart.kastermans
Here is a session trigger that I wrote (not quite the same rules around what a session is, but should hopefully be a good start to work from). I'd love to get any feedback on how it could be improved. - bart import org.apache.flink.api.common.state.{ReducingState, ReducingStateDescriptor, Valu

Testing a streaming data flow

2016-10-17 Thread bart.kastermans
What are the standard approaches for testing a streaming algorithm? I have been able to come up with the below where I 1) create a data source that emits events in bunches with set times so that I know the events will be in the same window, 2) end the stream with a mapWithState where the state ch