Stephan Ewen created FLINK-2674:
-----------------------------------
Summary: Rework windowing logic
Key: FLINK-2674
URL: https://issues.apache.org/jira/browse/FLINK-2674
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 0.10
Reporter: Stephan Ewen
Priority: Critical
Fix For: 0.10
The windowing logic needs a major overhaul. This follows the design documents:
- https://cwiki.apache.org/confluence/display/FLINK/Time+and+Order+in+Streams
- https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=60624830
Specifically, the following shortcomings need to be addressed:
- Global parallel windows should be dropped
-> for time, local windows are aligned and serve the same purpose
-> there is currently no known robust and efficient parallel
implementation of custom strategies
- Event time and out of order arrival needs to be supported
- Eviction of not accessed keys does not work. Non-accessed keys linger
infinitely
- Performance is currently bad for time windows, due to a overly general
implementation
- Resources are leaking, threads are not shut down
- Elements are stored multiple times (discretizers, window buffers)
- Finally, many implementations are buggy, produce wrong results
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
