Re: Re: flink Broadcast

2017-03-27 Thread Till Rohrmann
stance(x._2, y._2) > (toId,score) > }.toList.sortWith((x,y)=>x._2>y._2).take(20) > (fromId,docSims) > } > res.writeAsText(..) > > - 原始邮件 - > 发件人:Stephan Ewen > 收件人:user@flink.apache.org > 抄送人:亘谷 > 主题:Re: flink Broadcast &

回复:Re: flink Broadcast

2017-03-24 Thread rimin515
= y._1 val score = 1-cosDistince.distance(x._2, y._2)(toId,score) }.toList.sortWith((x,y)=>x._2>y._2).take(20) (fromId,docSims) }res.writeAsText(..) - 原始邮件 - 发件人:Stephan Ewen 收件人:user@flink.apache.org 抄送人:亘谷 主题:Re: flink Broadcast 日期:20

Re: flink Broadcast

2017-03-24 Thread Stephan Ewen
The program consists of two executions - one that only collects() back to the client, one that executes the map function. Are you running this as a "YARN single job" execution? IN that case, there may be an issue that this incorrectly tries to submit to a stopping YARN cluster. On Fri, Mar 24,

Re: flink Broadcast

2017-03-24 Thread Robert Metzger
Hi, Can you provide more logs to help us understand whats going on? One note regarding your application: You are calling .collect() and send the collection with the map() call to the cluster again. This is pretty inefficient and can potentially break your application (in particular the RPC system