RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-08 Thread Ufuk Celebi
> Thanks, > Arnaud > > -Message d'origine- > De : Ufuk Celebi [mailto:u...@apache.org] > Envoyé : jeudi 8 décembre 2016 14:43 > À : LINZ, Arnaud ; user@flink.apache.org > Cc : rmetz...@apache.org > Objet : RE: Collect() freeze on yarn cluster on strange > recover/des

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-08 Thread LINZ, Arnaud
re 2016 14:43 À : LINZ, Arnaud ; user@flink.apache.org Cc : rmetz...@apache.org Objet : RE: Collect() freeze on yarn cluster on strange recover/deserialization error Good point with the collect() docs. Would you mind opening a JIRA issue for that? I'm not sure whether you can specify it via th

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-08 Thread Ufuk Celebi
e launching the batch > > (with /usr/bin/zookeeper-client -server $FLINK_HA_ZOOKEEPER_SERVERS > > rmr $FLINK_HA_ZOOKEEPER_PATH_BATCH), and the "recovery" log line appears > > only before > the collect() phase, not at the beginning. > > > > Full log is avail

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-08 Thread LINZ, Arnaud
-Message d'origine- De : Ufuk Celebi [mailto:u...@apache.org] Envoyé : jeudi 8 décembre 2016 10:49 À : LINZ, Arnaud ; user@flink.apache.org Objet : RE: Collect() freeze on yarn cluster on strange recover/deserialization error I also don't get why the job is recovering, but the oversized

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-08 Thread Ufuk Celebi
Environment(). > > Do I have to change the way I'm doing things ? How ? > > Thanks, > Arnaud > > -Message d'origine- > De : LINZ, Arnaud > Envoyé : mercredi 30 novembre 2016 08:59 > À : user@flink.apache.org > Objet : RE: Collect() freeze on yarn clus

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-12-07 Thread LINZ, Arnaud
ment() but ExecutionEnvironment.getExecutionEnvironment(). Do I have to change the way I'm doing things ? How ? Thanks, Arnaud -Message d'origine- De : LINZ, Arnaud Envoyé : mercredi 30 novembre 2016 08:59 À : user@flink.apache.org Objet : RE: Collect() freeze on yarn cluste

RE: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-11-30 Thread LINZ, Arnaud
g. Full log is availlable here : https://ftpext.bouyguestelecom.fr/?u=JDhCUdcAImsANZQdys86yID6UNq8H2r Thanks, Arnaud -Message d'origine- De : Ufuk Celebi [mailto:u...@apache.org] Envoyé : mardi 29 novembre 2016 18:43 À : LINZ, Arnaud ; user@flink.apache.org Objet : Re: Collect() freez

Re: Collect() freeze on yarn cluster on strange recover/deserialization error

2016-11-29 Thread Ufuk Celebi
Hey Arnaud, could this be a left over job that is recovered from ZooKeeper? Recovery only happens if the configured ZK root contains data. A job is removed from ZooKeeper only if it terminates (e.g. finishes, fails terminally w/o restarting, cancelled). If you just shut down the cluster this i

Collect() freeze on yarn cluster on strange recover/deserialization error

2016-11-29 Thread LINZ, Arnaud
Hello, I have a Flink 1.1.3 batch application that makes a simple aggregation but freezes when collect() is called when the app is deployed on a ha-enabled yarn cluster (it works on a local cluster). Just before it hangs, I have the following deserialization error in the logs : (...) 2016-11-29