Re: Producer/Consumer questions 0.7

2013-08-29 Thread Philip O'Toole
On Thu, Aug 29, 2013 at 1:28 PM, Mark wrote: > I'm assuming for this auto-detection of partitions is only with the > ZookeeperConsumerConnector and not the ConsumerConnector? > Plain ol' ConsumerConnector does it for us, in our Java-based high-level consumers. > > On Aug 29, 2013, at 1:04 PM,

Re: Producer/Consumer questions 0.7

2013-08-29 Thread Mark
I'm assuming for this auto-detection of partitions is only with the ZookeeperConsumerConnector and not the ConsumerConnector? On Aug 29, 2013, at 1:04 PM, Philip O'Toole wrote: > On Thu, Aug 29, 2013 at 11:09 AM, Mark wrote: > >> 1) Should a producer be aware of which broker to write to or is

Re: Producer/Consumer questions 0.7

2013-08-29 Thread Philip O'Toole
On Thu, Aug 29, 2013 at 11:11 AM, Mark wrote: > Also, are the consumer offsets store in Kafka or Zookeeper? > Zookeeper. > > On Aug 29, 2013, at 11:09 AM, Mark wrote: > > > 1) Should a producer be aware of which broker to write to or is this > somehow managed by Kafka itself. For example, If

Re: Producer/Consumer questions 0.7

2013-08-29 Thread Philip O'Toole
On Thu, Aug 29, 2013 at 11:09 AM, Mark wrote: > 1) Should a producer be aware of which broker to write to or is this > somehow managed by Kafka itself. For example, If I have 2 brokers with a > configured partition size of 1 will my messages be written in a round-robin > type of fashion to each b

Re: Producer/Consumer questions 0.7

2013-08-29 Thread Mark
Also, are the consumer offsets store in Kafka or Zookeeper? On Aug 29, 2013, at 11:09 AM, Mark wrote: > 1) Should a producer be aware of which broker to write to or is this somehow > managed by Kafka itself. For example, If I have 2 brokers with a configured > partition size of 1 will my messa

Producer/Consumer questions 0.7

2013-08-29 Thread Mark
1) Should a producer be aware of which broker to write to or is this somehow managed by Kafka itself. For example, If I have 2 brokers with a configured partition size of 1 will my messages be written in a round-robin type of fashion to each broker or is this completed dependent on which broker