Hi Kai, I took a look at the implementation of the filesystem connector. It will decide which files to read at startup and won't change during running. If you want to need this function, you may need to customize a new connector.
Best, Xingbo eef hhj <zzfu...@gmail.com> 于2020年11月21日周六 下午2:38写道： > Hi, > > I'm facing a situation where I want the Flink App to dynamically detect > the change of the Filesystem batch data source. As I tried in the following > example in sql-client.sh, it can query all the records under the folder for > the select. > > While I'm adding a new file to the folder, the query does not refresh and > it seems it cannot detect the new file. It can only reflect records in the > new file unless I cancel the current query and do select again. Is it > possible to make the App detect such file changes automatically as the one > in the stream source? > > CREATE TABLE fs_table ( > user_id STRING, > order_amount DOUBLE, > dt STRING, > hh STRING, > ) WITH ( > 'connector'='filesystem', > 'path'='file:///path/folder/', > 'format'='csv' > ); > > select * from fs_table; > > -- Best wishes > Kai >
