One possible option is to look into the hybrid source released in Flink 1.14 to support your use-case:
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/hybridsource/ On Fri, Feb 25, 2022, 09:19 Jin Yi <j...@promoted.ai> wrote: > so we have a streaming job where the main work to be done is processing > infinite kafka sources. recently, i added a fromCollection (finite) source > to simply write some state once upon startup. this all seems to work > fine. the finite source operators all finish, while all the infinite > source operators continue running with watermarks. > > however, the weird thing is that savepointing throws exceptions, and there > have been no automatic checkpoints at all while the job has been running > for 90+minutes (checkpoint config is unaligned, exactly once every 5m w/ a > 1h timeout). > > is this mixed finity not a supported setup? >
