Hi Ben,
You can try using StreamExecutionEnvironment
streamExecutionEnvironment =
StreamExecutionEnvironment.getExecutionEnvironment();
instead of directly creating a new one. This will allow to pick up the
configuration parameters you pass through the command line.
I hope this helps,
Kostas
On
Hello,
Is there some proxy to seeing the relative time it takes for records to
make it through an entire job plan? Maybe checkpoint alignment time would
be a proxy for this? Is there metrics for that or something else that would
provide signal here?
Thanks!
--
Rex Fenley | Software Engineer
I am afraid only the state handle is stored in the ConfigMap. The real
state data is stored in
the distributed storage configured via "high-availability.storageDir". I
believe you could find
more information in this class KubernetesStateHandleStore[1].
How could you find that the checkpointing inf
I understand this.
State storage Is defined defined by state.checkpointing.dir, for example
state.checkpoints.dir: file:///mnt/flink/storage/checkpoints
I am talking about reference defined in 2 places
> On Dec 20, 2020, at 8:05 PM, Yang Wang wrote:
>
> I am afraid only the state handle is sto
That worked. Thankyou, Kostas.
From: Kostas Kloudas
Date: Sunday, December 20, 2020 at 7:21 AM
To: Ben Beasley
Cc: user@flink.apache.org
Subject: Re: No execution.target specified in your configuration file
Hi Ben,
You can try using StreamExecutionEnvironment
streamExecutionEnvironment =
Strea
IIUC, "state.checkpoints.dir" is specifying an external checkpoint path,
which will not be cleaned up unless
the users configured it explicitly[1].
However, for "high-availability.storageDir", it will be cleaned up
automatically when all the jobs in the application
reaches to the terminal state. M
Hi Rex,
I think Latency Marker is what you need [1].
Best,
Yun
[1]
https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#latency-tracking
--
Sender:Rex Fenley
Date:2020/12/21 04:57:59
Recipient:user
Cc:Brad
Hello -
In
https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesConfigOptions.java
the various supported options are declared as constants.
I see that there is no support for options like Volumes and VolumeMounts.
Also I se
