Thanks for info.
Attached the POC code for reference. Class
ServerCustomSocketStreamFunction.java is custom server socket source and class
FlinkClientApp.java is main program.
From: liu ron
Sent: 05 August 2023 02:28 PM
To: Kamal Mittal
Cc: Paul Lam ; user@flink.apache.org
Subject: Re: Flink
Hi, we are also paying attention to this issue and have completed the
validation of the minibatch join optimization including the intermediate
message folding you mentioned. We plan to officially release it in Flink 1.19.
This optimization could significantly improves the performance of join
op
Hi, David.
Yes, all records with the same key will be shuffled to a single downstream
subtask. Otherwise, the computed results will be wrong.
Best,
Ron
xiangyu feng 于2023年8月4日周五 09:45写道：
> Hi David,
>
> keyBy() is implemented with hash partitioning. If you use the keyBy
> function, the records
Hi, Kamal
According to your description, I think this is related to Flink's fault
tolerance mechanism， you can see [1] for more detail.
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/task_failure_recovery/
Best,
Ron
Kamal Mittal via user 于2023年8月4日周五 15:06写道：
> Hello,
Hi, Fil
If we don't specify the ExternalizedCheckpointCleanup, the default
checkpoint retention strategy is never retaining after the job terminates,
which causes your issue. So I think your configure the
ExternalizedCheckpointCleanup
to `RETAIN_ON_CANCELLING`.
Best,
Ron
Filip Karnicki 于2023年8月
Hi, Kamal
Based on your context, it is okay, but the server socket function must be
singleton. Otherwise, there will be port conflict problems.
One more question, would you be so kind as to provide the PoC code? Maybe
it helps to us share more views.
Best,
Ron
Kamal Mittal via user 于2023年8月5
