Hi folks, I have a somewhat complex Flink job that has a few async stages, and a few stateful stages. It currently loads its configuration on startup, and doesn't attempt to refresh it.
Now I'm working on dynamic reconfiguration. I've written a polling source which sends a configuration snapshot whenever anything has changed, I've set up a broadcast of that source, and I'm updating the operators in the data (i.e. not config) stream to be BroadcastProcessFunctions. But now I've reached the first async operator, and I recall that async functions aren't allowed to be stateful. I've tried to find a best practice for this situation, without much luck. My best idea so far is to insert a new stage before the async one, which would tuple up each record with its corresponding config snapshot from the most recent broadcast state. This would increase the amount of data that needs to be serialized, and some of the configs are quite large, but would allow me to continue using async IO. Any suggestions? Thanks! -0xe1a
