Re: Reading FileSource Files in a particular order

Hi! Are you running a batch job or a streaming job? For batch jobs just use ORDER BY keyword in SQL to sort the records. For streaming jobs I'm afraid it is hard to do so. A custom FileEnumerator might help, however if the parallelism of your file system source is more than one then it is possible

Reading FileSource Files in a particular order

Hi all, We're interested in being able to use a FileSource read from a Google Cloud Storage (GCS) archive of messages from a Kafka topic, roughly in order. Our GCS archive is