Aljoscha Krettek created FLINK-2317:
---------------------------------------
Summary: Stream Jobs with Windows Leave Lingering State
Key: FLINK-2317
URL: https://issues.apache.org/jira/browse/FLINK-2317
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 0.9, 0.9.1
Reporter: Aljoscha Krettek
Priority: Blocker
The problem is that some components in the windowing system create Threads that
are not properly cleaned up after a Job finished (is canceled). Specifically
these are the {{CentralCheck}} thread of the {{GroupedActiveDiscretizer}} and
the {{TimeCheck}} thread in {{TimeTriggerPolicy}}. These threads have
references to other parts of the windowing system, such as the window buffers.
This then means that the complete state of the buffers at the time a job is
canceled stays around indefinitely. Over time this will fill up the memory and
make successive windowing jobs incredibly slow or lead to GC overheat
exceptions.
Also, to prevent stuff like this from happening in the future we should put all
streaming Tests on a common {{TestBase}} that verifies that we have no
lingering threads after job execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
