Can you explain more about why you're writing a file with the data? Presumably, this is for another application to consume; could it not take the data from Kafka directly, whether with a native client or over the REST proxy? Oftentimes local files are unnecessary 'duck tape' for integration that can be done in a better way.
-- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Mon, 13 May 2019 at 01:35, Vinay Jain <vina...@gmail.com> wrote: > Hi > > I would like to consume a Topic and save the AVRO messages in local > directory in the AVRO file formats. As per Kafka Connect File Stream > documentation, it is not for production use. > > Other Option I am thinking to use Kafka Connect - HDFS Sink but I am not > sure if it can also write to the Local directory if we pass in the variable > hdfs.url the URL for local file system instead of HDFS path. > > Will this work or are there any other ready made options which can be used > for the same. > > Regards > > Vinay >
