Re: Verifying watermarks in integration test

Hi, I had sorted out how to run the topology in embedded mode. What wasn't clear to me is how I can verify the watermark, but as per following thread that can be done by inserting a process function: https://lists.apache.org/thread.html/9a47cb2284032527c1b63d35beb9bd2d4bdc36197849b84f5f69b768@%3C

Re: Verifying watermarks in integration test

Hi Thomas, generally speaking, if you want to test a whole job, just run the pipeline in your test case with a collection-based source and a result collecting sink. If your single operator tests passes while the integration test fails, maybe you should first check the timestamp / watermark assi

Re: Verifying watermarks in integration test

Hi Thomas, generally speaking, if you want to test a whole job, just run the pipeline in your test case with a collection-based source and a result collecting sink. If your single operator tests passes while the integration test fails, maybe you should first check the timestamp / watermark assi

Re: Verifying watermarks in integration test

Hi Xingcan, thanks, this is a good way of testing an individual operator. I had written my own mock code to intercept source context and collect the results, this is a much better approach for operator testing. I wonder how I can verify with an embedded Flink cluster though. Even though my single

Re: Verifying watermarks in integration test

Hi Thomas, some test cases in JoinHarnessTest show how to verify the emitted watermarks. Hope this helps. Best, Xingcan > On 21 Feb 2018,

Verifying watermarks in integration test

Hi, I have a streaming integration test with two operators. A source that emits records and watermarks, and a sink that collects the records. The topology runs in embedded mode and the results are collected in memory. Now, in addition to the records, I also want to verify that watermarks have been