RE: Hello, Help!

2017-07-07 Thread Tauzell, Dave
As others mentioned this is not a forum to discuss the works of Franz Kafka. Here are some places to get you started: 1. The works of Franz Kafka: https://www.vanderbilt.edu/olli/class-materials/Franz_Kafka.pdf 2. Literature stack exchange: https://literature.stackexchange.com/ -Dave

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
I’m not really familiar with Netty so I won’t be of much help. Maybe try posting on a Netty forum to see what they think? -Dave From: SenthilKumar K [mailto:senthilec...@gmail.com] Sent: Wednesday, June 21, 2017 10:28 AM To: Tauzell, Dave Cc: us...@kafka.apache.org; senthilec...@apache.org

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
seems possible with the right sort of kafka producer tuning. -Dave From: SenthilKumar K [mailto:senthilec...@gmail.com] Sent: Wednesday, June 21, 2017 8:55 AM To: Tauzell, Dave Cc: us...@kafka.apache.org; senthilec...@apache.org; dev@kafka.apache.org; Senthil kumar Subject: Re: Handling 2 to 3

RE: Handling 2 to 3 Million Events before Kafka

2017-06-21 Thread Tauzell, Dave
What are your configurations? - production - brokers - consumers Is the problem that web servers cannot send to Kafka fast enough or your consumers cannot process messages off of kafka fast enough? What is the average size of these messages? -Dave -Original Message- From: SenthilKumar

RE: Kafka duplicate offset at Consumer

2016-09-20 Thread Tauzell, Dave
Are you using the new java consumer? What method are you using to commit offsets? -Dave -Original Message- From: Ghosh, Achintya (Contractor) [mailto:achintya_gh...@comcast.com] Sent: Tuesday, September 20, 2016 8:56 AM To: us...@kafka.apache.org Cc: dev@kafka.apache.org Subject: Kafka

Re: Brokers are crash due to __consumer_offsets folder are deleted

2016-07-02 Thread Tauzell, Dave
/var/log would be a better default. Dave > On Jul 2, 2016, at 07:09, Ismael Juma wrote: > > Hi Peter, > > It's a good question why `log.dir` defaults to `/tmp`. I assume it's to > make it easier for people to get started with Kafka, but unsafe defaults > should be avoided as much as possible in

RE: Kafka HDFS Connector

2016-06-22 Thread Tauzell, Dave
I don't see any built-in support for this but I think that you can write a class that implements io.confluent.connect.hdfs.Format public interface Format { RecordWriterProvider getRecordWriterProvider(); SchemaFileReader getSchemaFileReader(AvroData avroData); HiveUtil getHiveUtil(HdfsSinkC