Re: Questions about failure recovery, load balancing and producer ack/async mode

2013-10-23 Thread Jun Rao
1,3 Take a look at request.required.acks in http://kafka.apache.org/documentation.html#producerconfigs 2. The producer does random distribution by default. However, you can provide a partitioning key and a partitioning function. For details on how consumer load balancing works, see http://research

Re: How to deal with error code in FetchResponse

2013-10-23 Thread xingcan
I see. Thanks Neha. On Thursday, October 24, 2013, Neha Narkhede wrote: > That was one of the examples I gave that doesn't cover all cases. We have > admin tools that can move leader away selectively. In that case, the > partitions would have different error codes. > > Thanks, > Neha > > > On Wed

IRC channel has moved to #apache-kafka

2013-10-23 Thread David Arthur
We have created and registered a new IRC channel for the project. Freenode has setup a forward from #kafka to #apache-kafka for us, but people already in this channel will need to /join #apache-kafka Sorry for the inconvenience. -David

issue for embedding KafkaServer in java container

2013-10-23 Thread Jason Rosenberg
https://issues.apache.org/jira/browse/KAFKA-1101

Questions about failure recovery, load balancing and producer ack/async mode

2013-10-23 Thread Shafaq
Hi, Following need more elaboration after reading kafka docs: 1- In a scenario during leader fails over, what happens to messages that are not committed to other followers and to the messages that producer keep in sending (in async mode) till new leader is elected. Can the producer buffer thes

Re: How to deal with error code in FetchResponse

2013-10-23 Thread Neha Narkhede
That was one of the examples I gave that doesn't cover all cases. We have admin tools that can move leader away selectively. In that case, the partitions would have different error codes. Thanks, Neha On Wed, Oct 23, 2013 at 1:36 AM, xingcan wrote: > Neha, > > Thanks for your prompt reply. And

Re: How to deal with error code in FetchResponse

2013-10-23 Thread xingcan
Neha, Thanks for your prompt reply. And I got another two questions. As I wrote before, my mechanism is to add all partitions belonging to the same leader broker to a single request. And then send these requests one by one for each broker. Is this necessary? And if that, all topic and partitions i