Re: Streaming File Sink cannot generate _SUCCESS tag files

2020-10-19 Thread highfei2011
Hi, Jingsong Lee Thanks for taking the time to respond to the email, I will try following your suggestion. Best, Yang 在 2020年10月19日 11:56,Jingsong Li 写道: Hi, Yang, "SUCCESSFUL_JOB_OUTPUT_DIR_MARKER" does not work in StreamingFileSink. You can take a look to partition commit fea

Re: Streaming File Sink cannot generate _SUCCESS tag files

2020-10-18 Thread Jingsong Li
Hi, Yang, "SUCCESSFUL_JOB_OUTPUT_DIR_MARKER" does not work in StreamingFileSink. You can take a look to partition commit feature [1], [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit Best, Jingsong Lee On Thu, Oct 15, 2020 a

Streaming File Sink cannot generate _SUCCESS tag files

2020-10-15 Thread highfei2011
Hi, everyone! Currently experiencing a problem with the bucketing policy sink to hdfs using BucketAssigner of Streaming File Sink after consuming Kafka data with FLink -1.11.2, the _SUCCESS tag file is not generated by default. I have added the following to the configuration val ha