Re: NullPointerException with functions.rand()

2015-06-12 Thread Ted Yu
t >> org.apache.spark.sql.catalyst.expressions.RDG.rng$lzycompute(random.scala:39) >> at org.apache.spark.sql.catalyst.expressions.RDG.rng(random.scala:39) >> .. >> >> >> Does any one know why? >> >> Thanks. >> >> Justin >> >> -

Re: NullPointerException with functions.rand()

2015-06-10 Thread Ted Yu
; at > org.apache.spark.sql.catalyst.expressions.RDG.rng$lzycompute(random.scala:39) > at org.apache.spark.sql.catalyst.expressions.RDG.rng(random.scala:39) > .. > > > Does any one know why? > > Thanks. > > Justin > > -- >

NullPointerException with functions.rand()

2015-06-10 Thread Justin Yip
Hello, I am using 1.4.0 and found the following weird behavior. This case works fine: scala> sc.parallelize(Seq((1,2), (3, 100))).toDF.withColumn("index", rand(30)).show() +--+---+---+ |_1| _2| index| +--+---+---+ | 1| 2| 0.6662967911724369| | 3|100|