Offset management in multi-threaded high-level consumer

2015-01-06 Thread Rafi Shamim
Hello, I would like to write a multi-threaded consumer for the high-level consumer in Kafka 0.8.1. I have found two ways that seem feasible while keeping the guarantee that messages in a partition are processed in order. I would appreciate any feedback this list has. Option 1 - Create mu

Re: New producer hangs inifitely when it looses connection to Kafka cluster

2015-08-17 Thread Rafi Shamim
Is there a known workaround for this? I believe I'm also encountering this issue. We are being affected by a separate issue that causes the network to drop periodically. When this happens, my producer hangs for up to 10 minutes minutes. The retries work immediately. Also, this seems to only affect