Re: Flink task node shut it self off.

2019-12-20 Thread jingjing bai
hi john in our experience , the checkpoint interval we set interval 1-10 minute and timeout usurally 5*interval . mostly we set 2 or 5 minute and 10 or 20timeout. it depend on u data bulk per second and which window used. John Smith 于2019年12月21日周六 上午5:26写道: > Hi, using Flink 1.8.0 > > 1st off

Taskmanagers in Docker Fail to Resolve Own Hostnames and Won't Accept Tasks

2019-12-20 Thread Martin, Nick J [US] (IS)
I'm running Flink 1.7.2 in a Docker swarm. Intermittently, new task managers will fail to resolve their own host names when starting up. In the log I see "no hostname could be resolved" messages coming from TaskManagerLocation. The webUI on the jobmanager shows the taskmanagers as are associated

Flink task node shut it self off.

2019-12-20 Thread John Smith
Hi, using Flink 1.8.0 1st off I must say Flink resiliency is very impressive, we lost a node and never lost one message by using checkpoints and Kafka. Thanks! The cluster is a self hosted cluster and we use our own zookeeper cluster. We have... 3 zookeepers: 4 cpu, 8GB (each) 3 job nodes: 4 cpu,

Re: Deprecated SplitStream class - what should be use instead.

2019-12-20 Thread KristoffSC
Hi Kostas, Thank you for the answer and clarification. If Side-outputs are treated in the same way and there is no significant performance penalty then it seems that they are ok for my use case. I can accept the name mismatch ;) Regards, Krzysztof -- Sent from: http://apache-flink-user-maili

Re: [DISCUSS] Drop vendor specific repositories from pom.xml

2019-12-20 Thread Robert Metzger
Okay, I understand. I'm okay with removing the profile. On Thu, Dec 19, 2019 at 11:34 AM Till Rohrmann wrote: > The profiles make bumping ZooKeeper's version a bit more cumbersome. I > would be interested for this reason to get rid of them, too. > > Cheers, > Till > > On Wed, Dec 18, 2019 at 5:3

Re: Deprecated SplitStream class - what should be use instead.

2019-12-20 Thread Kostas Kloudas
Hi Krzysztof, If I get it correctly, your main reason behind not using side-outputs is that it seems that "side-output", by the name, seems to be a "second class citizen" compared to the main output. I see your point but in terms of functionality, there is no difference between the different outp