Re: kafka-connect-hdfs failure due to corrupt WAL file

2016-08-02 Thread Ewen Cheslack-Postava
If you're not worried about duplicates, then yes, you can delete the WAL to recover. By the way, we're aware of some issues in that code that we're addressing here: https://github.com/confluentinc/kafka-connect-hdfs/pull/96 -Ewen On Tue, Aug 2, 2016 at 3:12 PM, Prabhu V wrote: > Hi, > > I am u

kafka-connect-hdfs failure due to corrupt WAL file

2016-08-02 Thread Prabhu V
Hi, I am using kafka-connect-hdfs in a 2 nodes and one of the nodes had to be rebooted when the process was running. Upon restart the process fails with 16/08/02 21:43:30 ERROR hdfs.TopicPartitionWriter: Recovery failed at state RECOVERY_PARTITION_PAUSED org.apache.kafka.connect.errors.ConnectEx