n or use
the ParquetRelation2 for workaround.
(https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/parquet/newParquet.scala)
Cheng Hao
-Original Message-
From: Jerry Raj [mailto:jerry@gmail.com
<mailto:jerry@gmail.com&g
Thanks, that helped. And I needed SchemaRDD.as() to provide an alias for
the RDD.
-Jerry
On 17/12/14 12:12 pm, Tobias Pfeiffer wrote:
Jerry,
On Wed, Dec 17, 2014 at 3:35 PM, Jerry Raj mailto:jerry@gmail.com>> wrote:
Another problem with the DSL:
t1.where('term == &q
Another problem with the DSL:
t1.where('term == "dmin").count() returns zero. But
sqlCtx.sql("select * from t1 where term = 'dmin').count() returns 700,
which I know is correct from the data. Is there something wrong with how
I'm using the DSL?
Thanks
On
Hi,
I'm using the Scala DSL for Spark SQL, but I'm not able to do joins. I
have two tables (backed by Parquet files) and I need to do a join across
them using a common field (user_id). This works fine using standard SQL
but not using the language-integrated DSL neither
t1.join(t2, on = 't1.us
Hi,
If I create a SchemaRDD from a file that I know is sorted on a certain
field, is it possible to somehow pass that information on to Spark SQL
so that SQL queries referencing that field are optimized?
Thanks
-Jerry
-
To un
Hi,
Can a UDF return a list of values that can be used in a WHERE clause?
Something like:
sqlCtx.registerFunction("myudf", {
Array(1, 2, 3)
})
val sql = "select doc_id, doc_value from doc_table where doc_id in
myudf()"
This does not work:
Exception in thread "main" java.l
