Hey, our intern, Pablo pointed out that there is some problem with mixed windowing policies. When you write ... .window(Count ...) .every(Time ...) .mapWindow(...) ...
The result makes no sense, as the window is not of the specified length. Maybe, there is some conflict between Time and Count logic. Is this a related problem to yours? Peter 2015-04-29 21:15 GMT+02:00 Aljoscha Krettek <aljos...@apache.org>: > I am on it, yes. :D > On Apr 29, 2015 8:31 PM, "Gyula Fóra" <gyf...@apache.org> wrote: > > > Hey, > > You are right, so there seems to be some error in the logic that checks > for > > the end of the slide. > > I checked the count pre-reducer and the error there is clear, but if I > fix > > it it breaks some tests so I think some other stuff must not work > properly > > either. > > > > I cannot work on this until I get back from holiday, it would be good if > > you could look into it. > > > > Cheers, > > Gyula > > > > On Wed, Apr 29, 2015 at 7:09 PM, Aljoscha Krettek <aljos...@apache.org> > > wrote: > > > > > Oh, I saw them, but in updateCurrent() (both grouped and non-grouped) > > > the else path is never taken when I run the windowed WordCount > > > example. > > > > > > I have a branch where I just throw a RuntimeException in the else > > > branches and the whole things still works: mvn verify doesn't fail, > > > the examples still run. When looking at the behaviour of the > > > functions, it seems clear to me that the else path can never be taken. > > > > > > On Wed, Apr 29, 2015 at 6:15 PM, Gyula Fóra <gyf...@apache.org> wrote: > > > > Hey, > > > > > > > > They actually work :P Although I have to admit I need to do some > > > > refactoring of the method names and parameters. > > > > > > > > I made some quick refactoring and added some comments for the key > > > methods: > > > > > > > > > > > > > > https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L96 > > > > > > > > > > > > > > https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L105 > > > > > > > > > > > > > > https://github.com/mbalassi/flink/blob/dfa028ce5102b1f0d098eaec6c4b4d3744bee379/flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/SlidingPreReducer.java#L132 > > > > > > > > Those three parts contain the key logic, and now the method names > > should > > > > make more sense as well :) > > > > > > > > Cheers, > > > > Gyula > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Apr 29, 2015 at 4:44 PM, Aljoscha Krettek < > aljos...@apache.org > > > > > > > wrote: > > > > > > > >> Hi, > > > >> one of my notorious (by now) investigative questions: How do these > > > >> work? To meet it seems that they do not perform any actual pre > > > >> aggregation but just keep all the elements in a LinkedList > > > >> (non-grouped case) or HashMap (grouped-case) and aggregate the > > > >> elements when the window is to be emitted. > > > >> > > > >> Am I missing something again? :D > > > >> > > > >> Cheers, > > > >> Aljoscha > > > >> > > > > > >
