Re: Spark SQL joins taking too long

2016-01-27 Thread Raghu Ganti
“counties”); > … > And then doing the join. > > > From: Raghu Ganti [mailto:raghuki...@gmail.com] > Sent: Thursday, January 28, 2016 3:06 AM > To: Ted Yu; Дмитро Попович > Cc: user > Subject: Re: Spark SQL joins taking too long > > The problem is with the way S

RE: Spark SQL joins taking too long

2016-01-27 Thread Cheng, Hao
then doing the join. From: Raghu Ganti [mailto:raghuki...@gmail.com] Sent: Thursday, January 28, 2016 3:06 AM To: Ted Yu; Дмитро Попович Cc: user Subject: Re: Spark SQL joins taking too long The problem is with the way Spark query plan is being created, IMO, what was happening before is that the

Re: Spark SQL joins taking too long

2016-01-27 Thread Raghu Ganti
oinDf.count(); //gives me the correct answer of 8 >> >> Any suggestions on what I can do to optimize and debug this piece of code? >> >> Regards, >> Raghu >> >> >> >> -- >> View this message in context: >> http://apache-spark-

Re: Spark SQL joins taking too long

2016-01-26 Thread Raghu Ganti
ring', h.location)"); >> long count = joinDf.count(); //gives me the correct answer of 8 >> >> Any suggestions on what I can do to optimize and debug this piece of code? >> >> Regards, >> Raghu >> >> >> >> -- >> View this message

Re: Spark SQL joins taking too long

2016-01-26 Thread Ted Yu
e correct answer of 8 > > Any suggestions on what I can do to optimize and debug this piece of code? > > Regards, > Raghu > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-joins-taking-too-long-tp26078.html > Sent from

Spark SQL joins taking too long

2016-01-26 Thread raghukiran
f.count(); //gives me the correct answer of 8 Any suggestions on what I can do to optimize and debug this piece of code? Regards, Raghu -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-joins-taking-too-long-tp26078.html Sent from the Apache