[
https://issues.apache.org/jira/browse/FLINK-32806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18119857#comment-18119857
]
william commented on FLINK-32806:
---------------------------------
Hi all,
I sent a [DISCUSS] thread to the dev list a couple of days ago about picking up
this issue.
After sending it I went back through the history here and realised I had missed
two things,
so let me correct that:
1. This issue is already assigned to hk__lrzy.
2. mapohl already stated a preference for keeping the default TTL at
infinity/max (to align
with the current behaviour) while adding a config option to make it tunable.
Sorry for not reading through the discussion before posting. I agree with the
direction
mapohl described.
While reading PR #23531 I also noticed one problem with it: it builds the cache
with
`new Configuration()`, i.e. an empty configuration, so the added option would
always resolve
to its default value and any user setting would be silently ignored. Making the
option
actually work means passing the real Configuration into EmbeddedJobResultStore,
which today
is created with a no-arg constructor in AbstractNonHaServices (plus around 12
test call
sites).
hk__lrzy: are you still working on this? If not, I'd be happy to take it over
and implement
it following the direction above (default TTL = infinity, a new config option
to tune it,
and the Configuration actually wired through).
One practical note: my account (williamliu) does not seem to have the "Assign
Issues"
permission yet, so I cannot assign this issue to myself. If hk__lrzy is no
longer working
on it, could someone with that permission assign it to me?
I have a local implementation ready that follows the direction above (no
eviction unless
job-result-store.clean-job-result.ttl is configured, with the Configuration
actually wired
through AbstractNonHaServices and StandaloneHaServices), and I would open a PR
for it.
Thanks,
William
> EmbeddedJobResultStore keeps the non-dirty job entries forever
> --------------------------------------------------------------
>
> Key: FLINK-32806
> URL: https://issues.apache.org/jira/browse/FLINK-32806
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.18.0, 1.17.1, 1.19.0
> Reporter: Matthias Pohl
> Assignee: hk__lrzy
> Priority: Major
> Labels: pull-request-available, stale-assigned, starter
>
> The {{EmbeddedJobResultStore}} keeps the entries of cleaned-up jobs in-memory
> forever. We might want to add a TTL to have those entries be removed after a
> certain amount of time to allow maintaining the memory footprint of the
> {{EmbeddedJobResultStore}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
