yeah i was surprised with the Option. it works in 1.6.0-SNAPSHOT, and its a
pretty neat way to indicate nullability i guess.
i will file jira. i saw similar behavior with other types than Option. this
was just the easiest to show.
On Mon, Feb 15, 2016 at 3:52 PM, Reynold Xin wrote:
> Looks like
Looks like a bug. I'm also not sure whether we support Option yet. (If not,
we should definitely support that in 2.0.)
Can you file a JIRA ticket?
On Mon, Feb 15, 2016 at 7:12 AM, Koert Kuipers wrote:
> i noticed some things stopped working on datasets in spark 2.0.0-SNAPSHOT,
> and with a con
i noticed some things stopped working on datasets in spark 2.0.0-SNAPSHOT,
and with a confusing error message (cannot resolved some column with input
columns []).
for example in 1.6.0-SNAPSHOT:
scala> val ds = sc.parallelize(1 to 10).toDS
ds: org.apache.spark.sql.Dataset[Int] = [value: int]
scala
