Hi, You can point a file-based input format to a directory and the input format should read all files in that directory. That works as well for TableSources that are internally use file-based input formats. Is that what you are looking for?
Best, Fabian Am Mo., 28. Jan. 2019 um 17:22 Uhr schrieb françois lacombe < francois.laco...@dcbrain.com>: > Hi all, > > I'm wondering if it's possible and what's the best way to achieve the > loading of multiple files with a Json source to a JDBC sink ? > I'm running Flink 1.7.0 > > Let's say I have about 1500 files with the same structure (same format, > schema, everything) and I want to load them with a *batch* job > Can Flink handle the loading of one and each file in a single source and > send data to my JDBC sink? > I wish I can provide the URL of the directory containing my thousand files > to the batch source to make it load all of them sequentially. > My sources and sinks are currently available for BatchTableSource, I guess > the cost to make them available for streaming would be quite expensive for > me for the moment. > > Have someone ever done this? > Am I wrong to expect doing so with a batch job? > > All the best > > François Lacombe > > > <http://www.dcbrain.com/> <https://twitter.com/dcbrain_feed?lang=fr> > <https://www.linkedin.com/company/dcbrain> > <https://www.youtube.com/channel/UCSJrWPBLQ58fHPN8lP_SEGw> > > [image: Arbre vert.jpg] Pensez à la planète, imprimer ce papier que si > nécessaire >
