Re: latency critical job

2018-05-28 Thread makeyang
Rong Rong: my flink version is 1.4.2 since we are using the docker env which is sharing disk-io, based on our observation, disk-io spike cased by other process in the same physical machine can lead to long time operator processing. -- Sent from: http://apache-flink-user-mailing-list-arc

Re: latency critical job

2018-05-28 Thread makeyang
Timo: thanks for u suggestion -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: latency critical job

2018-05-25 Thread Rong Rong
Hi Makeyang, +1 on Timo's point. We have been dealing with these kind of problems before and in general Flink can handle latency if implemented correctly and assigned correct amount of computation resource (depending on what kind of resource isolation/containerization you are doing) to handle addi

Re: latency critical job

2018-05-25 Thread Timo Walther
Hi, usually Flink should have constant latency if the job is implemented correctly. But if you want to implement something like an external monitoring process., you can use the REST API [1] and metrics [2] to model such an behavior by restarting your application. In theory, you could also imp