Thanks in advance. Jeyhun
On Thu, Aug 11, 2016 at 7:09 AM Gwen Shapira <g...@confluent.io> wrote: > Maybe you need a different KafkaConnect source? I think this one may > fit your needs better: > https://github.com/jcustenborder/kafka-connect-spooldir > > It was built to copy data from files in directory into Kafka... > > > > On Wed, Aug 10, 2016 at 8:56 AM, Jeyhun Karimov <je.kari...@gmail.com> > wrote: > > Hi community, > > > > I am using Kafka-Connect to import text files to kafka. Because I have to > > give exact file name (like ./data.txt) and not the parent directory (./*) > > as an input to Kafka-Connect, how can I import files in particular > > directory as they are created in runtime? The solutions that come to my > > mind are below but I don't know about their efficiency. > > - I may create a new file in particular directory (after the > kafka-connect > > started already) and start new connect instance to point that file > > -or I may write to only one file (just one ./data.txt) all the time (and > no > > other data files created) and kafka-connect will transfer the data only > > from one file and it will from where it left (as data writes are made) > > -or any other way to efficiently handle this use case. > > > > My second question is, is it safe to override the data transfer methods > of > > kafka-connect? For example I want to put thread.sleeps in kafka-streams > > side while transferring data and see the behaviour in kafka side or in > > application side. You can think of as simulation of load. > > > > Cheers > > Jeyhun > > > > > > -- > > -Cheers > > > > Jeyhun > > > > -- > Gwen Shapira > Product Manager | Confluent > 650.450.2760 | @gwenshap > Follow us: Twitter | blog > -- -Cheers Jeyhun
