offset fetch request - response issue and wire protocol kafka 0.8

2013-06-17 Thread Milind Parikh
Given the https://cwiki.apache.org/KAFKA/a-guide-to-the-kafka-protocol.html#AGuideToTheKafkaProtocol-OffsetFetchRequestas basis OffsetFetchRequest => ConsumerGroup [TopicName [Partition]] ConsumerGroup => string TopicName => string Partition => int32 and OffsetFetchResponse => [TopicName [

Re: metadata request/response question

2013-06-12 Thread Milind Parikh
Since you send it a Topic that does not exist, it sends you a blank list of brokers. So extending it, I suppose, would mean that it would send only those brokers that contain data. Regards Milind On Wed, Jun 12, 2013 at 2:24 PM, Dave Peterson wrote: > Suppose I send a metadata request for a si

Re: Offset commit api

2012-12-20 Thread Milind Parikh
+1 on limiting the size. But could you do 2k instead of 1k? Using Interval Time Clocks gets you a lot on distributed autonomous processing; but most large scale ITCs go upto 1.5K. http://code.google.com/p/itclocks/refer to the link on conference paper. Regards Milind On Thu, Dec 20, 2012

Re: Offset commit api

2012-12-17 Thread Milind Parikh
Perhaps I don't understand the motivation well enough and perhaps I am misreading the intent. But I thought that the design principle behind kafka is for state (from a consumer standpoint) was to be managed by consumer and not broker. I understand that "These APIs are optional, clients can store