Hi. That's my bad when not reading the documentation carefully, so I didn't know that the success file and data files are completely different.
Thank you so much for supporting me. Best, Long -------------------------------------------------------------------------------------------------------------- Nguyen Dich Long, School of Information and Communication Technology (SoICT - https://www.soict.hust.edu.vn) Hanoi University of Science and Technology (https://www.hust.edu.vn) 601, B1 Building - No 1, Dai Co Viet Street, Hai Ba Trung District, Ha Noi, Vietnam Tel: +84 (0)3.54.41.76.76 Email: long.nd162...@sis.hust.edu.vn; longnguyen25111...@gmail.com On Mon, Nov 8, 2021, 09:41 Jingsong Li <jingsongl...@gmail.com> wrote: > Hi, > > yidan is correct. > > The success-file is not the data-file. [1] > > At present, there is no configuration with data file name. You can > create a JIRA for this. > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/#partition-commit-policy > > Best, > Jingsong > > On Mon, Nov 8, 2021 at 9:59 AM yidan zhao <hinobl...@gmail.com> wrote: > > > > Actually, the success file is another file which is written done to the > dir when the partition is done. It's content have nothing to do with your > bussiness. > > > > Long Nguyễn <longnguyen25111...@gmail.com> 于2021年11月5日周五 下午5:07写道： > >> > >> Thank you, Paul. > >> > >> The answer is so clear and helpful. But I'm still wondering what is the > purpose of the sink.partition-commit.success-file.name option if the sink > files must be named in that specific way. > >> > >> On Fri, Nov 5, 2021 at 3:23 PM Fabian Paul <fabianp...@ververica.com> > wrote: > >>> > >>> Hi, > >>> > >>> Currently this is expected because the FileSink is built to support > running with higher parallelism. Therefore it needs to periodically write > files. The > >>> respective file names always have a descriptor that the File Sink > knows which files have already been written. You can read more about the > FileSink > >>> here [1]. > >>> > >>> Best, > >>> Fabian > >>> > >>> [1] > https://ci.apache.org/projects/flink/flink-docs-master/docs/connectors/datastream/file_sink/ > >> > >> > >> > >> -- > >> > -------------------------------------------------------------------------------------------------------------- > >> Nguyen Dich Long, > >> School of Information and Communication Technology (SoICT - > https://www.soict.hust.edu.vn) > >> Hanoi University of Science and Technology (https://www.hust.edu.vn) > >> 601, B1 Building - No 1, Dai Co Viet Street, Hai Ba Trung District, Ha > Noi, Vietnam > >> Tel: +84 (0)3.54.41.76.76 > >> Email: long.nd162...@sis.hust.edu.vn; longnguyen25111...@gmail.com > > > > -- > Best, Jingsong Lee >
