Re: Unable to register UDF with StructType

2015-11-06 Thread Michael Armbrust
This isn't supported today. On Fri, Nov 6, 2015 at 1:28 AM, Rishabh Bhardwaj wrote: > Thanks for your response. > Actually I want to return a Row or (return struct). > But, attempting to register UDF returning Row throws the following error, > > scala> def test(r:Row):Row = r > test: (r: org.apa

Re: Unable to register UDF with StructType

2015-11-06 Thread Rishabh Bhardwaj
Thanks for your response. Actually I want to return a Row or (return struct). But, attempting to register UDF returning Row throws the following error, scala> def test(r:Row):Row = r test: (r: org.apache.spark.sql.Row)org.apache.spark.sql.Row scala> sqlContext.udf.register("test",test _) java.lan

Re: Unable to register UDF with StructType

2015-11-05 Thread Michael Armbrust
You are returning the type StructType not an instance of a struct (i.e. StringType instead of "string"). If you'd like to return a struct you should return a case class. case class StringInfo(numChars: Int, firstLetter: String) udf((s: String) => StringInfo(s.size, s.head)) If you'd like to take

Unable to register UDF with StructType

2015-11-05 Thread Rishabh Bhardwaj
Hi all, I am unable to register a UDF with return type as StructType: scala> def test(r:StructType):StructType = { r } > > test: (r: >> org.apache.spark.sql.types.StructType)org.apache.spark.sql.types.StructType > > >> scala> sqlContext.udf.register("test",test _ ) > > scala.MatchError: org.apach