This also happens for cluster setups. On Wed, Apr 8, 2015 at 11:29 AM, Maximilian Michels (JIRA) <j...@apache.org> wrote:
> Maximilian Michels created FLINK-1843: > ----------------------------------------- > > Summary: Job History gets cleared too fast > Key: FLINK-1843 > URL: https://issues.apache.org/jira/browse/FLINK-1843 > Project: Flink > Issue Type: Bug > Components: JobManager > Affects Versions: 0.9 > Reporter: Maximilian Michels > Fix For: 0.9 > > > As per FLINK-1442, the JobManager stores the archived ExecutionGraph > behind a SoftReference. At least for local setups, this mechanism doesn't > seem to work properly. There are two issues: > > - The history gets cleared too fast > - The history gets cleared in a non-sequential fashion, i.e. arbitrary old > ExecutionGraph are discarded > > To solve these problems we might > > - Store the least recent ExecutionGraph behind a SoftReference > - Store the most recent ExecutionGraphs without a SoftReference > > That way, we can save memory but have the latest history available to the > user. We might introduce a configuration variable where the user can > specify the number of ExecutionGraphs that should be held in memory. The > remaining can be stored behind a SoftReference. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) >
