Hi,

I am building a pretty straightforward processing pipeline as described
below, using *DataStream* *APIs* and *FileSystem connector*.

*filesystem-source -> transforms -> database-sink*

Everything worked well till the filesystem (source) had just a single type
(JSON) of files. Recently the filesystem got modified to store other types
of files too (like parquet, text, etc.).

It appeared just a matter of specifying some regex to the FileSource or
SplitEnumerator and creating different sources for different types of
files.

However, the two public methods *FileSource::forRecordStreamFormat *and
*FileSource::forBulkFileFormat* do not allow configuring the regex based
file paths.

What could be the recommended way to configure a custom regex for a
FileSource?

I am using version 1.19.1.

Regards,
Amogh.

Reply via email to