Hi , Flink users
we are using Flink as the runtime of our beam jobs which works great,
recently we want to enable restart strategy
<https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/restart_strategies.html>
in our flink cluster, from the document I see restart strategy will only
work when checkpointing is enabled. I'm trying to find out if it's
possible to enable checkpointing from flink-conf.yaml which is
equivalent to the call "
flinkStreamEnv.enableCheckpointing(checkpointInterval);" in
StreamExecutionEnvironment.
the reason we want to config it through flink-conf.yaml is that we use
helm to create flink cluster ondemand for dedicated job which works
great on kubernetes env. with beam if we want to enable checkpointing ,
we have to create FlinkPipelineOptions which is cross platform. (like
you use Hibernate Session in JPA code). so we are trying to find a way
to enable it from flink-conf.yaml .
sample flink-conf.yaml as below.
flink-conf.yaml: |
blob.server.port: 6124
jobmanager.rpc.address: address-cache-flink-jobmanager
jobmanager.rpc.port: 6123
jobmanager.heap.mb: 256
taskmanager.heap.mb: 756
taskmanager.numberOfTaskSlots: 4
parallelism.default: 16
metrics.reporters: prom
metrics.reporter.prom.class:
org.apache.flink.metrics.prometheus.PrometheusReporter
metrics.reporter.prom.port: 9100-9101
metrics.reporter.prom.prefix: flink_jm_
restart-strategy: fixed-delay
restart-strategy.fixed-delay.attempts: 3
restart-strategy.fixed-delay.delay: 300 s
state.backend: filesystem
state.backend.fs.checkpointdir:
file:///var/nfs/ephem_store/flink/checkpoints
