Re: The driver hangs at DataFrame.rdd in Spark 2.1.0

2017-02-26 Thread Liang-Chi Hsieh
; > > -- Original -- > From: "Cheng Lian [via Apache Spark Developers > List]";<ml-node+s1001551n21069...@n3.nabble.com>; > Send time: Friday, Feb 24, 2017 2:28 AM > To: "Stan Zhai"<m...@zhaishidan.cn>; > > Subject: Re: The d

Re: The driver hangs at DataFrame.rdd in Spark 2.1.0

2017-02-26 Thread Liang-Chi Hsieh
Hi Stan, Looks like it is the same issue we are working to solve. Related PRs are: https://github.com/apache/spark/pull/16998 https://github.com/apache/spark/pull/16785 You can take a look of those PRs and help review too. Thanks. StanZhai wrote > Hi all, > > > The driver hangs at DataFrame

Re: The driver hangs at DataFrame.rdd in Spark 2.1.0

2017-02-23 Thread StanZhai
vided. -- Original -- From: "Cheng Lian [via Apache Spark Developers List]";; Send time: Friday, Feb 24, 2017 2:28 AM To: "Stan Zhai"; Subject: Re: The driver hangs at DataFrame.rdd in Spark 2.1.0 This one seems to be relevant, but it

Re: The driver hangs at DataFrame.rdd in Spark 2.1.0

2017-02-23 Thread Cheng Lian
17733 ? -- Original -- *From: * "Cheng Lian-3 [via Apache Spark Developers List]";<[hidden email] >; *Send time:* Thursday, Feb 23, 2017 9:43 AM *To:* "Stan Zhai"<[hidden email] >; *Subject: * Re: The driver hangs at DataFrame.rdd in Spark 2.1.0 Just from t

Re: The driver hangs at DataFrame.rdd in Spark 2.1.0

2017-02-22 Thread StanZhai
an't get it in Spark 2.1.0): ... -- Original ---------- Subject: Re: The driver hangs at DataFrame.rdd in Spark 2.1.0 What is the query plan? We had once observed query plans that grow exponentially