Hi experts,
Here is my use case, it's a flink stateless streaming job for message
validation.
1. read from a kafka topic
2. perform validation of message, which requires query external system
2a. the metadata from the external system will be cached in memory
for 15minutes
2b. there i
Hi, Eleanore
1. AFAIK I think only the job could "pause" itself. For example the
"query" external system could pause when the external system is down.
2. Maybe you could try the "iterate" and send the failed message back to
retry if you use the DataStream api.
Best,
Guowei
On Mon, Nov 30, 2020
Hi Eleanore,
if the external system is down, you could simply fail the job after a given
timeout (for example, using asyncIO). Then the job would restart using the
restarting policies.
If your state is rather small (and thus recovery time okay), you would
pretty much get your desired behavior. Th