Re: allowNonRestoredState: metadata file in checkpoint dir missing

2020-08-01 Thread Congxian Qiu
Hi Omkar `--allowNonRestoredState` would not affect the behavior of checkpoint, it only affects the restore logic. As the problem of not generate _metadata file, could you please check 1) if the job enabled checkpoint; 2) if there is any checkpoint complete successfully. Best, Congxian Des

Re: sporadic "Insufficient no of network buffers" issue

2020-08-01 Thread Rahul Patwari
>From the metrics in Prometheus, we observed that the minimum AvailableMemorySegments out of all the task managers is 4.5k when the exception was thrown. So there were enough network buffers. correction to the configs provided above: each TM CPU has 8 cores. Apart from having fewer network buffers

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-01 Thread Niels Basjes
No, I only have Java. On Fri, 31 Jul 2020, 21:57 Vijayendra Yadav, wrote: > Thank You Niels. Would you have something for the scala object class. Say > for example if I want to implement a unit test ( not integration test) for > below code or similar : > > > https://github.com/apache/flink/blo

FlinkML status

2020-08-01 Thread Mohamed Haseeb
Hi, What's the current status of FlinkML? is it still part of Flink? the last Flink release that has documentation about it is 1.8. Thanks, M. Haseeb