Spark submit running spark-sql-perf and additional jar

2016-10-23 Thread Mr rty ff
Hi I run the following script home/spark-2.0.1-bin-hadoop2.7/bin/spark-submit --conf "someconf" "--jars /home/user/workspace/auxdriver/target/auxdriver.jar,/media/sf_VboxShared/tpc-ds/spark-sql-perf-v.0.2.4/spark-sql-perf-assembly-0.2.4.jar --benchmark DatabasePerformance --iterations 1 --spark

Using explain plan to optimize sql query

2016-02-14 Thread Mr rty ff
HiI have some queries that take a long time to execute so I used an df.explain(true) to print physical and logical plans to see where the bottlenecks.As the query is very complicated I got a very unreadable   result.How can I parse it to some thing more readable and  analyze it?And another quest