Hi Sriram G, Both the DataStream and Table API support filesystem as a source in unbounded (streaming mode) with exactly once guarantees. This is documented at https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/filesystem/ and https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/
Best regards, Martijn Op di 24 jan. 2023 om 10:26 schreef Sriram Ganesh <srigns...@gmail.com>: > Hi Everyone, > > I am thinking of switching my input source from Kafka to S3. First, I > couldn't find any streaming source connector for S3. I have some basic > questions about > > 1. How S3 will work as a streaming source with proper checkpointing.? > 2. How Flink will manage the last offset processed from a file? > 3. Is exactly_once possible while using S3 as a streaming source? > 4. What could be the pros and cons of using an S3 kind of storage as a > streaming source? > > Any help would be appreciated. Thanks in advance. > > Thanks, > -- > *Sriram G* > *Tech* > >
