Re: Spark SQL Optimization

2016-03-23 Thread Takeshi Yamamuro
Filter (user_name#13659 = user) >> Scan >> >> ParquetRelation[snackfs://shared:9042/aladdin_data_beta/table3.parquet][anc_user_group#13658,user_name#13659] >>ConvertToUnsafe >> Project [perm_group#13667,user_group#13665] >> Filter (perm_t

Re: Spark SQL Optimization

2016-03-21 Thread Rishi Mishra
e > Project [perm_group#13667,user_group#13665] > Filter (perm_type#13666 = TEST) > Scan > > ParquetRelation[snackfs://shared:9042/aladdin_data_beta/table2.parquet][perm_group#13667,user_group#13665,perm_type#13666] > ConvertToUnsafe > Scan > >

Re: Spark SQL Optimization

2016-03-21 Thread gtinside
nsafe Scan ParquetRelation[snackfs://shared:9042/aladdin_data_beta/table1.parquet][portfolio_code#14119,anc_port_group#14117] -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Optimization-tp26548p26553.html Sent from the Apache Spark User

Re: Spark SQL Optimization

2016-03-21 Thread Xiao Li
code >> from table1 uge, table2 p, table3 pge >> where uge.user_name = 'user' and p.perm_type = 'TEST' >> and p.perm_group = pge.anc_port_group >> and p.user_group=uge.anc_user_group >> >> >> Execution plan is more optimized for the optimized query an

Re: Spark SQL Optimization

2016-03-21 Thread Michael Armbrust
ery > executes faster. All the tables are being sourced from parquet files > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Optimization-tp26548.html > Sent from the Apache Spark User List mailing list archive at

Spark SQL Optimization

2016-03-21 Thread gtinside
executes faster. All the tables are being sourced from parquet files -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Optimization-tp26548.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --