Hi community, I have a Hive table that stores tens of millions rows of data. In my Flink job, I want to process the data in batch manner:
- Split the data into batches, each batch has (maybe) 10,000 rows. - For each batch, call a batchPut() API on my redis client to dump in Redis. Doing so in a streaming manner is not expected, as that will cause too many round trips between Flink workers and Redis. Is there a way to do that? I find little clue in Flink docs, since almost all APIs feel better suited for streaming processing by default. Thank you! Best, Yik San
