EXCEPT is not the same as EXCEPT ALL Had they implemented EXCEPT ALL in SparkSQL one could have easily obtained EXCEPT by adding a disctint() to the results
________________________________ From: hareesh makam <makamhare...@gmail.com> Sent: Thursday, July 6, 2017 12:48:18 PM To: jeff saremi Cc: user@spark.apache.org Subject: Re: Is there "EXCEPT ALL" in Spark SQL? There is Except in DataFrame API. df1.except(df2) Same can be used in SQL as well. public DataFrame<https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/DataFrame.html> except(DataFrame<https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/DataFrame.html> other) Returns a new DataFrame<https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/DataFrame.html> containing rows in this frame but not in another frame. This is equivalent to EXCEPT in SQL. -Hareesh On 6 July 2017 at 12:22, jeff saremi <jeffsar...@hotmail.com<mailto:jeffsar...@hotmail.com>> wrote: I tried this query in 1.6 and it failed: SELECT * FROM Table1 EXCEPT ALL SELECT * FROM Table2 Exception in thread "main" java.lang.RuntimeException: [1.32] failure: ``('' expected but `all' found thanks Jeff
