Hi , Flink users

we are using Flink as the runtime of our beam jobs which works great, recently we want to enable restart strategy <https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/restart_strategies.html> in our flink cluster, from the document I see restart strategy will only work when checkpointing is enabled. I'm trying to find out if it's possible to enable checkpointing from flink-conf.yaml which is equivalent to the call " flinkStreamEnv.enableCheckpointing(checkpointInterval);" in StreamExecutionEnvironment.

the reason we want to config it through flink-conf.yaml is that we use helm to create flink cluster ondemand for dedicated job which works great on kubernetes env. with beam if we want to enable checkpointing , we have to create FlinkPipelineOptions which is cross platform. (like you use Hibernate Session in JPA code). so we are trying to find a way to enable it from flink-conf.yaml .

sample flink-conf.yaml as below.

  flink-conf.yaml: |
    blob.server.port: 6124
    jobmanager.rpc.address: address-cache-flink-jobmanager
    jobmanager.rpc.port: 6123
    jobmanager.heap.mb: 256
    taskmanager.heap.mb: 756
    taskmanager.numberOfTaskSlots: 4
    parallelism.default: 16
    metrics.reporters: prom
metrics.reporter.prom.class: org.apache.flink.metrics.prometheus.PrometheusReporter
    metrics.reporter.prom.port: 9100-9101
    metrics.reporter.prom.prefix: flink_jm_
    restart-strategy: fixed-delay
    restart-strategy.fixed-delay.attempts: 3
    restart-strategy.fixed-delay.delay: 300 s
    state.backend: filesystem
state.backend.fs.checkpointdir: file:///var/nfs/ephem_store/flink/checkpoints

Reply via email to