Hi,
I am trying SparkSQL based on the example on doc ...
....
val people =
sc.textFile("/data/spark/examples/src/main/resources/people.txt").map(_.split(",")).map(p
=> Person(p(0), p(1).trim.toInt))
val olderThanTeans = people.where('age > 19)
val youngerThanTeans = people.where('age < 13)
val nonTeans = youngerThanTeans.union(olderThanTeans)
I can do a orderBy('age) on first two (which are SchemaRDD) but not on
third. The nonTeans is a UnionRDD that does not supports orderBy. This
seems different than the SQL behavior where results of 2 SQL unions is a
SQL itself with same functionality ...
Not clear why union of 2 SchemaRDDs does not produces a SchemaRDD ....
Thanks,
