Hi, It's a syntax error in Spark-1.3. The next release of spark supports the kind of UDF calls in DataFrame. See a link below.
https://issues.apache.org/jira/browse/SPARK-6379 On Sat, Apr 11, 2015 at 3:30 AM, Yana Kadiyska <yana.kadiy...@gmail.com> wrote: > Hi, I'm running into some trouble trying to register a UDF: > > scala> sqlContext.udf.register("strLen", (s: String) => s.length()) > res22: org.apache.spark.sql.UserDefinedFunction = > UserDefinedFunction(<function1>,IntegerType) > > scala> cleanProcessDF.withColumn("dii",strLen(col("di"))) > <console>:33: error: not found: value strLen > cleanProcessDF.withColumn("dii",strLen(col("di"))) > > > > Where cleanProcessDF is a dataframe > Is my syntax wrong? Or am I missing an import of some sort? > -- --- Takeshi Yamamuro
