CodeTrainerMan opened a new pull request, #29304:
URL: https://github.com/apache/flink/pull/29304
## What is the purpose of the change
`EmbeddedJobResultStore` keeps the clean job result entries forever, so the
in-memory store
grows without bound in long-running sessions. See FLINK-32806.
This PR stores the clean job result entries in a Guava cache and adds the
`job-result-store.clean-job-result.ttl` option. **No expiration is applied
if the option is
not configured**, i.e. the current behaviour is kept unchanged. This follows
the direction
Matthias Pohl described in the Jira issue: keep the default TTL at
infinity/max to align with
the current behaviour, and add a configuration parameter to make it tunable.
The real `Configuration` is now passed into `EmbeddedJobResultStore` through
`AbstractNonHaServices` and `StandaloneHaServices`, so the option actually
takes effect.
(#23531 built the cache from an empty `Configuration`, which made the added
option always
resolve to its default value.)
## Brief change log
- Add `JobResultStoreOptions#CLEAN_JOB_RESULT_TTL`
(`job-result-store.clean-job-result.ttl`)
- `EmbeddedJobResultStore`: keep the clean entries in a Guava cache that
expires them after
the configured TTL (no expiration if the option is not configured)
- `AbstractNonHaServices` / `StandaloneHaServices`: add constructors
accepting a
`Configuration`; the existing signatures are kept, so no existing test
needed to be
touched
- `HighAvailabilityServicesUtils`: pass the real configuration on the
production path
- Add `EmbeddedJobResultStoreTtlTest` and update the generated configuration
docs
## Verifying this change
`EmbeddedJobResultStoreTtlTest` covers:
- clean entries are retained indefinitely without a configuration
- clean entries are retained indefinitely if the TTL is not configured
- the entry is still present within the configured TTL
- the entry is evicted after the configured TTL
`EmbeddedJobResultStoreContractTest` and `StandaloneHaServicesTest` pass
unchanged
(17 tests in total).
## Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: it adds a
new configuration option only
- The serializers: no
- The runtime per-record code paths: no
- Anything that affects deployment or recovery: no
## Documentation
- Does this pull request introduce a new feature? yes (a new configuration
option)
- The generated configuration docs are updated accordingly.
Jira: https://issues.apache.org/jira/browse/FLINK-32806
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
