Re: How to write unit test for stateful operators in Pyflink apps

Hi Dian, I got it. A few days ago, I also found some test cases implemented with Python here in Flink's official repository. I took a look at them and it seems like many internal functions are used and since those are

Re: "sink.partition-commit.success-file.name" option in the FileSystem connector does not work

Hi. That's my bad when not reading the documentation carefully, so I didn't know that the success file and data files are completely different. Thank you so much for supporting me. Best, Long ---

Re: "sink.partition-commit.success-file.name" option in the FileSystem connector does not work

Hi, yidan is correct. The success-file is not the data-file. [1] At present, there is no configuration with data file name. You can create a JIRA for this. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/#partition-commit-policy Best, Jingsong On Mon

Re: "sink.partition-commit.success-file.name" option in the FileSystem connector does not work

Actually, the success file is another file which is written done to the dir when the partition is done. It's content have nothing to do with your bussiness. Long Nguyễn 于2021年11月5日周五 下午5:07写道： > Thank you, Paul. > > The answer is so clear and helpful. But I'm still wondering what is the > purpos

Re: How to write unit test for stateful operators in Pyflink apps

Hi Long, I agree with Fabian that currently you have to test it with a e2e job. There are still no such test harnesses for PyFlink jobs. Regards, Dian On Fri, Nov 5, 2021 at 5:22 PM Long Nguyễn wrote: > Thanks, Fabian. I'll check it out. > > Hope that Dian can also give me some advice. > > Bes

how to expose the current in-flight async i/o requests as metrics?

Hi community, While using Flink's async i/o for interacting with an external system, I got the following exception: 2021-11-06 10:38:35,270 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator[] - Triggering checkpoint 54 (type=CHECKPOINT) @ 1636162715262 for job f168a44ea33198cd717