That should be possible, although I'm not super familiar with thrift. You'll probably need access to the generated metadata <http://people.apache.org/~thejas/thrift-0.9/javadoc/org/apache/thrift/meta_data/package-frame.html> .
<Shameless plug> If you find yourself reading a lot of thrift data you might consider writing a library that goes against the new SQL Data Source API <https://github.com/apache/spark/pull/2475>, which is about to be merged in. Its essentially applySchema on steroids. This code for avro is possibly a useful reference: https://github.com/marmbrus/sql-avro On Thu, Oct 30, 2014 at 2:13 PM, ankits <ankitso...@gmail.com> wrote: > I have one job with spark that creates some RDDs of type X and persists > them > in memory. The type X is an auto generated Thrift java class (not a case > class though). Now in another job, I want to convert the RDD to a SchemaRDD > using sqlContext.applySchema(). Can I derive a schema from the thrift > definitions to convert RDD[X] to SchemaRDD[X]? > > > > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Creating-a-SchemaRDD-from-RDD-of-thrift-classes-tp17766.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
