Re: DataStream API in Batch Execution mode

2021-06-09 Thread Marco Villalobos
o Villalobos > wrote: > >> How do I use a hierarchical directory structure as a file source in S3 >> when using the DataStream API in Batch Execution mode? >> >> I have been trying to find out if the API supports that, >> because currently our data is organized by

Re: DataStream API in Batch Execution mode

2021-06-07 Thread Guowei Ma
-connector-files/src/test/java/org/apache/flink/connector/file/src/FileSourceTextLinesITCase.java Best, Guowei On Tue, Jun 8, 2021 at 5:59 AM Marco Villalobos wrote: > How do I use a hierarchical directory structure as a file source in S3 > when using the DataStream API in Batch Executio

DataStream API in Batch Execution mode

2021-06-07 Thread Marco Villalobos
How do I use a hierarchical directory structure as a file source in S3 when using the DataStream API in Batch Execution mode? I have been trying to find out if the API supports that, because currently our data is organized by years, halves, quarters, months, and but before I launch the job, I