I'm going to follow the guidelines you mentioned. On the other hand, do you
know if there is any real performance comparison between Flink SQL and
Flink? By using 100% Flink SQL, I think we gain a lot of ease of use, but
we might be losing performance. Additionally, there are features available
in
All in all Flink generates and stores uid hashes in state which is a
cryptic value.
Savepoint metadata can be read the following way:
var metadata = SavepointLoader.loadSavepointMetadata(path);
for (var operatorState: metadata.getOperatorStates()) {
operatorState.getOperatorID();
}
The task
Hi,
I think now I realize the importance of setting uid. Perhaps in Flink docs
they should lay emphasis on always setting up uid to your operator, just
like we set name.
Unfortunately for my current pipeline which is in production, there is no
uid set.
Is there a way I can find these uids?
Thanks
The UID must match in the Flink app `events.uid("my-uid")` and in the
reader app `forUid("my-uid")`.
In general it's a good practice to set uid in the Flink app for each and
every operator, otherwise Flink generates an almost random number for it.
When you don't know the generated uid then no worr
Hi Sachin,
You're on the right track in general and this should work :)
I know that version upgrade can be complex but it worth to do so because
we've just added a ~95% performance boost for the state processor api in
1.20 [1].
As you mentioned the SQL connector is on 2.x which is not yet feasible
Hi Sachin.
When you define your pipeline, there is an option to assign a UID to each task
in the job graph. If you do not, Flink will auto-generate UID for you – each
time you run the pipeline.
It is highly recommended to define your own UIDs, since any re-run would assign
new UIDs and effecti
