f the resp. kafka source,
>>>>- Configure start-by-timestamp for the resp. source with a
>>>>timestamp that lies within the rolling time range (of above)
>>>>- Configure the job to ignore recovery for state that does not
>>>>have a cor
ery for state that does not have
>>>a corresponding operator in the job (the previous kafka source uid()s)
>>>- Start the job on new cloud provider, wait for it to pick
>>> up/back-fill
>>>- Take a savepoint
>>>- Remove deduplicatio
he job on new cloud provider, wait for it to pick
>>up/back-fill
>>- Take a savepoint
>>- Remove deduplication operator if that causes too much
>>load/latency/whatever
>>
>>
>>
>> This scheme sounds more complicated than
load/latency/whatever
>
>
>
> This scheme sounds more complicated than it really is … and has saved my
> sanity quite a number of times 😊
>
>
>
> Good luck and ready to answer more details
>
>
>
> Thias
>
>
>
> *From:* Hemanga Borah
> *Sent:* Tu
complicated than it really is … and has saved my sanity
quite a number of times 😊
Good luck and ready to answer more details
Thias
From: Hemanga Borah
Sent: Tuesday, May 3, 2022 3:12 AM
To: user@flink.apache.org
Subject: Migrating Flink apps across cloud with state
Hello,
We are attempting to
Hey Hemanga,
That's quite annoying of MirrorMaker to change the offsets on you. One
solution would be to use the State Processor API[1] to read the savepoint
and update the offsets to the new ones — does MirrorMaker give you these
ahead of time? There might also be more specific tricks people coul
Any ideas, guys?
On Mon, May 2, 2022 at 6:11 PM Hemanga Borah
wrote:
> Hello,
> We are attempting to port our Flink applications from one cloud provider
> to another.
>
> These Flink applications consume data from Kafka topics and output to
> various destinations (Kafka or databases). The appl
Hello,
We are attempting to port our Flink applications from one cloud provider
to another.
These Flink applications consume data from Kafka topics and output to
various destinations (Kafka or databases). The applications have states
stored in them. Some of these stored states are aggregations,
