Re: Kafka->HDFS to store as Parquet format

2014-10-07 Thread Buntu Dev
- Original Message ----- > From: "bdev" < buntu...@gmail.com > > To: u...@spark.incubator.apache.org > Sent: Tuesday, October 7, 2014 9:51:40 AM > Subject: Re: Kafka->HDFS to store as Parquet format > > After a bit of looking around, I found saveAsNewAPIHa

Re: Kafka->HDFS to store as Parquet format

2014-10-07 Thread Soumitra Kumar
to new schema. - Original Message - From: "Buntu Dev" To: "Soumitra Kumar" Cc: u...@spark.incubator.apache.org Sent: Tuesday, October 7, 2014 10:18:16 AM Subject: Re: Kafka->HDFS to store as Parquet format Thanks for the info Soumitra.. its a good start for me.

Re: Kafka->HDFS to store as Parquet format

2014-10-07 Thread Buntu Dev
leName (outputDir, em, time), > classOf[Void], classOf[Group], classOf[ExampleOutputFormat], conf) > > - Original Message - > From: "bdev" > To: u...@spark.incubator.apache.org > Sent: Tuesday, October 7, 2014 9:51:40 AM > Subject: Re: Kafka->HDFS to store

Re: Kafka->HDFS to store as Parquet format

2014-10-07 Thread Soumitra Kumar
(rddToFileName (outputDir, em, time), classOf[Void], classOf[Group], classOf[ExampleOutputFormat], conf) - Original Message - From: "bdev" To: u...@spark.incubator.apache.org Sent: Tuesday, October 7, 2014 9:51:40 AM Subject: Re: Kafka->HDFS to store as Parquet format After a b

Re: Kafka->HDFS to store as Parquet format

2014-10-07 Thread bdev
-HDFS-to-store-as-Parquet-format-tp15768p15852.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h

Kafka->HDFS to store as Parquet format

2014-10-05 Thread bdev
with this or it is dependent on the specified time window? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kafka-HDFS-to-store-as-Parquet-format-tp15768.html Sent from the Apache Spark User List mailing list archive at Nabble.com