So, Flink is going to act as an OAuth2 Client. You would need to have fetch access token and cache it, either in Flink or externally. I you wish to cache in Flink, look into Broadcast State pattern. These are URLs.
https://flink.apache.org/2019/06/26/a-practical-guide-to-broadcast-state-in-apache-flink/ https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/fault-tolerance/broadcast_state/ Trouble with this approach is you’d have to put access token into a data stream. Not sure how that would work. We are utilizing an external cache in Valkey key-value store. Nix. From: Kamal Mittal via user <user@flink.apache.org> Date: Tuesday, February 11, 2025 at 12:17 PM To: user@flink.apache.org <user@flink.apache.org> Subject: Flink kafka source with OAuth authentication token refresh Hello, I am having a scenario where in flink kafka source (org.apache.flink.connector.kafka.source.KafkaSource) after fetching kafka record, there is need for OAuth authentication for one of the 3rd party REST API. This API needs authentication token and has some expiry time associated with it and token refresh needs to be triggered after that much expiry time to avoid REST call per kafka record. So, is there any way to solve it without opening a separate thread for expiry time to count? Rgds, Kamal
