Re: Spark SQL : how to find element where a field is in a given set

2014-11-02 Thread Rishi Yadav
did you create SQLContext? On Sat, Nov 1, 2014 at 7:51 PM, abhinav chowdary wrote: > I have same requirement of passing list of values to in clause, when i am > trying to do > > i am getting below error > > scala> val longList = Seq[Expression]("a", "b") > :11: error: type mismatch; > found :

Re: Spark SQL : how to find element where a field is in a given set

2014-11-01 Thread abhinav chowdary
I have same requirement of passing list of values to in clause, when i am trying to do i am getting below error scala> val longList = Seq[Expression]("a", "b") :11: error: type mismatch; found : String("a") required: org.apache.spark.sql.catalyst.expressions.Expression val longList = S

Re: Spark SQL : how to find element where a field is in a given set

2014-08-29 Thread Michael Armbrust
This feature was not part of that version. It will be in 1.1. On Fri, Aug 29, 2014 at 12:33 PM, Jaonary Rabarisoa wrote: > > 1.0.2 > > > On Friday, August 29, 2014, Michael Armbrust > wrote: > >> What version are you using? >> >> >> >> On Fri, Aug 29, 2014 at 2:22 AM, Jaonary Rabarisoa >> wr

Re: Spark SQL : how to find element where a field is in a given set

2014-08-29 Thread Jaonary Rabarisoa
1.0.2 On Friday, August 29, 2014, Michael Armbrust wrote: > What version are you using? > > > > On Fri, Aug 29, 2014 at 2:22 AM, Jaonary Rabarisoa > wrote: > >> Still not working for me. I got a compilation error : *value in is not a >> member of Symbol.* Any ideas ? >> >> >> On Fri, Aug 29, 2

Re: Spark SQL : how to find element where a field is in a given set

2014-08-29 Thread Michael Armbrust
What version are you using? On Fri, Aug 29, 2014 at 2:22 AM, Jaonary Rabarisoa wrote: > Still not working for me. I got a compilation error : *value in is not a > member of Symbol.* Any ideas ? > > > On Fri, Aug 29, 2014 at 9:46 AM, Michael Armbrust > wrote: > >> To pass a list to a variadic f

Re: Spark SQL : how to find element where a field is in a given set

2014-08-29 Thread Jaonary Rabarisoa
Still not working for me. I got a compilation error : *value in is not a member of Symbol.* Any ideas ? On Fri, Aug 29, 2014 at 9:46 AM, Michael Armbrust wrote: > To pass a list to a variadic function you can use the type ascription :_* > > For example: > > val longList = Seq[Expression]("a", "

Re: Spark SQL : how to find element where a field is in a given set

2014-08-29 Thread Michael Armbrust
To pass a list to a variadic function you can use the type ascription :_* For example: val longList = Seq[Expression]("a", "b", ...) table("src").where('key in (longList: _*)) Also, note that I had to explicitly specify Expression as the type parameter of Seq to ensure that the compiler converts

Re: Spark SQL : how to find element where a field is in a given set

2014-08-28 Thread Jaonary Rabarisoa
ok, but what if I have a long list do I need to hard code like this every element of my list of is there a function that translate a list into a tuple ? On Fri, Aug 29, 2014 at 3:24 AM, Michael Armbrust wrote: > You don't need the Seq, as in is a variadic function. > > personTable.where('name i

Re: Spark SQL : how to find element where a field is in a given set

2014-08-28 Thread Michael Armbrust
You don't need the Seq, as in is a variadic function. personTable.where('name in ("foo", "bar")) On Thu, Aug 28, 2014 at 3:09 AM, Jaonary Rabarisoa wrote: > Hi all, > > What is the expression that I should use with spark sql DSL if I need to > retreive > data with a field in a given set. > Fo