Did you checkout the latest streaming programming guide? http://spark.apache.org/docs/latest/streaming-programming-guide.html#dataframe-and-sql-operations
You also need to be aware of that to convert json RDDs to dataframe, sqlContext has to make a pass on the data to learn the schema. This will fail if a batch has no data. You have to safeguard against that. On Wed, Apr 22, 2015 at 6:19 AM, ayan guha <guha.a...@gmail.com> wrote: > What about sqlcontext.createDataframe(rdd)? > On 22 Apr 2015 23:04, "Sergio Jiménez Barrio" <drarse.a...@gmail.com> > wrote: > >> Hi, >> >> I am using Kafka with Apache Stream to send JSON to Apache Spark: >> >> val messages = KafkaUtils.createDirectStream[String, String, StringDecoder, >> StringDecoder](ssc, kafkaParams, topicsSet) >> >> Now, I want parse the DStream created to DataFrame, but I don't know if >> Spark 1.3 have some easy way for this. ¿Any suggestion? I can get the >> message with: >> >> val lines = messages.map(_._2) >> >> Thank u for all. Sergio J. >> >> >>
