Re: Changing application configuration when restoring from checkpoint/savepoint

2020-12-28 Thread Aljoscha Krettek
On Wed Dec 16, 2020 at 6:41 PM CET, vishalovercome wrote: > 1. Is there any way to restore from a checkpoint as well as recreate > client > using newer configuration? I think that would only work if you somehow read the configuration from an external system > 2. If we take a savepoint (drain and

Re: Changing application configuration when restoring from checkpoint/savepoint

2020-12-16 Thread vishalovercome
Will this work - In main method, serialize config into a string and store it using ParameterTool with key as taskName and value as config (serialized as string). Then in the open method, lookup the relevant configuration using getTaskName(). A follow up to this would be configuring custom windowi

Changing application configuration when restoring from checkpoint/savepoint

2020-12-16 Thread vishalovercome
My flink job loads several configuration files that contain job, operator and business configuration. One of the operators is an AsyncOperator with function like so: class AsyncFun(config: T) extends RichAsyncFunction[X, Y] { @transient private lazy val client = f(config, metricGroup, etc.)