Spark SQL doesn’t support “NOT IN”, but I think HiveQL does, so give using the HiveContext a try rather than SQLContext. Here’s the spark 1.2 docs on it, but it’s basically identical to running the SQLContext
https://spark.apache.org/docs/1.2.0/sql-programming-guide.html#tab_scala_6 https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/sql/hive/HiveContext.html Thanks, Ewan From: Akhil Das [mailto:[email protected]] Sent: 04 September 2015 13:12 To: Pietro Gentile <[email protected]> Cc: [email protected] Subject: Re: NOT IN in Spark SQL I think spark doesn't support NOT IN clauses, but you can do the same with a LEFT OUTER JOIN, Something like: SELECT A.id FROM A LEFT OUTER JOIN B ON (B.id = A.id) WHERE B.id IS null Thanks Best Regards On Thu, Sep 3, 2015 at 8:46 PM, Pietro Gentile <[email protected]<mailto:[email protected]>> wrote: Hi all, How can I do to use the "NOT IN" clause in Spark SQL 1.2 ?? He continues to give me syntax errors. But the question is correct in SQL. Thanks in advance, Best regards, Pietro.
