Cancelling job with savepoint fails sometimes

I notice that sometimes when I try to cancel a Flink job with savepoint, the cancel fails with the following error: org.apache.flink.util.FlinkException: Could not cancel job 3be3d380dca9bb6a5cf0d559d54d7ff8. at org.apache.flink.client.cli.CliFrontend.lambda$cancel$4(CliFrontend.java:581)

Re: Checkpointing in Flink 1.5.0

checkpointing directory using glusterfs >> volume mount (thus file access protocol file:///) was working fine till >> 1.4.2 for us. So we like to understand where the breakage happened in >> 1.5.0. >> >> Can you please mention me the relevant source code files related to

Re: Checkpointing in Flink 1.5.0

, Jul 3, 2018 at 5:11 PM, Chesnay Schepler wrote: > Doesn't sound like intended behavior, can you give us the stacktrace? > > > On 03.07.2018 13:17, Data Engineer wrote: > >> The Flink documentation says that we need to specify the filesystem type >> (file://, hdfs://)

Checkpointing in Flink 1.5.0

The Flink documentation says that we need to specify the filesystem type (file://, hdfs://) when configuring the rocksdb backend dir. https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/state_backends.html#the-rocksdbstatebackend But when I do this, I get an error on job submis

Checkpointing on cluster shutdown

Hi, Suppose I have a working Flink cluster with 1 taskmanager and 1 jobmanager and I have enabled checkpointing with say an interval of 1 minute. Now if I shut down the Flink cluster in between checkpoints (say for some upgrade), will the JobManager automatically trigger a checkpoint before going

Reg. the checkpointing mechanism

I have a Flink application (v 1.4.2) where I have enabled checkpointing with an interval of 200 ms. This is what I have configured in my driver program: env.enableCheckpointing(checkpointInterval, CheckpointingMode.EXACTLY_ONCE); CheckpointConfig config = env.getCheckpointC

Re: How does setMaxParallelism work

The actual parallelism can be set per job > in your program but also in the flink client: > flink run -p > > > Nico > > > > [1] > https://ci.apache.org/projects/flink/flink-docs- > master/ops/production_ready.html#set-maximum-parallelism- > for-operators-expli

Re: How does setMaxParallelism work

. But if I use setMaxParallelism(3), only 1 subtask is created. On Wed, Mar 28, 2018 at 12:29 PM, Jörn Franke wrote: > What was the input format, the size and the program that you tried to > execute > > On 28. Mar 2018, at 08:18, Data Engineer wrote: > > I went through

How does setMaxParallelism work

I went through the explanation on MaxParallelism in the official docs here: https://ci.apache.org/projects/flink/flink-docs-master/ops/production_ready.html#set-maximum-parallelism-for-operators-explicitly However, I am not able to figure out how Flink decides the parallelism value. For instance,

Testing flink class loading

I am trying to run the ClassLoaderTestProgram on flink. 1. I have started Flink in local mode with the following command: bin/jobmanager.sh start local 2. I ran the ClassLoaderTestProgram jar: bin/flink run ClassLoaderTestProgram.jar --resolve-first child --output out.txt I get a

How to get automatic fail over working in Flink

This question has been asked on StackOverflow: https://stackoverflow.com/questions/48262080/how-to-get-automatic-fail-over-working-in-flink I am using Apache Flink 1.4 on a cluster of 3 machines, out of which one is the JobManager and the other 2 host TaskManagers. I start flink in cluster mode a