Also Kafka has a Hadoop consumer API for doing such things, please refer to http://kafka.apache.org/081/documentation.html#kafkahadoopconsumerapi
2015-05-06 12:22 GMT+08:00 MrAsanjar . <afsan...@gmail.com>: > why not try https://github.com/linkedin/camus - camus is kafka to HDFS > pipeline > > On Tue, May 5, 2015 at 11:13 PM, Rendy Bambang Junior < > rendy.b.jun...@gmail.com> wrote: > >> Hi all, >> >> I am planning to load data from Kafka to HDFS. Is it normal to use spark >> streaming to load data from Kafka to HDFS? What are concerns on doing this? >> >> There are no processing to be done by Spark, only to store data to HDFS >> from Kafka for storage and for further Spark processing >> >> Rendy >> > >
