Hello David,
In short, problem is not with your topic, it is with consumer offsets topic
initialization.
You could modify your code to just retry fetching offsets (until successful
where successful is also return of -1 offset, or timeout), or alternatively
you could trigger consumer offsets topic init (by fetching consumer offsets
topic metadata from broker) before issuing offset fetch request.
For the init option you have (at least) two alternatives: do it after or
before request to create your topic.
If you chose to do consumer offsets topic init before request to create
your (first) topic, make sure to configure broker with replication factor
of 1 for consumer offsets topic (offsets.topic.replication.factor config
property) otherwise consumer offsets topic init will fail. Do this config
change only for integration test with single broker, but not for production
cluster which should have 3 or more brokers anyway.
Kind regards,
Stevo Slavic.
On Sun, Nov 1, 2015 at 9:38 PM, David Corbin <dcor...@lancope.com> wrote:
> Yes. I know all of that. I guess my original message was not clear.
>
> The topic metadata indicates that there is a leader.
>
> Please see my comments interspersed below.
> Thanks
> David Corbin
>
> On 10/29/15, 17:53, "Mayuresh Gharat" <gharatmayures...@gmail.com> wrote:
>
> > NotCoordinatorForConsumerException is for consumer side.
>
> I am attempting to consume messages, so this means I¹m getting an
> exception that might be reasonable for what I¹m doing.
>
> >I think if you
> >are using simpleConsumer, then you have to do your own offset management.
>
> I am attempting to manage my offsets myself; that¹s why I called
> SimpleConsumer#fetchOffsets() which is throwing the exception. I assume
> you are not suggesting that my offset management should be done entirely
> ³outside the scope² of Kafka. If that is true, why do 4 of 3 methods on
> SimpleConsumer refer to offsets?
>
>
> >
> >TopicMetadata tells you whether there is a leader for the topic
> >partitions,
> >replicas, ISR.
>
>
> Yes. After I create the topic, I ³poll² the metadata until the one
> topic/partition has a leader, before moving on. If the metadata says
> there is a leader, I don¹t understand whyI would get
> NotCoordinatorForConsumerException
>
> >
> >Thanks,
> >
> >Mayuresh
> >
> >On Wed, Oct 28, 2015 at 8:23 AM, David Corbin <dcor...@lancope.com>
> wrote:
> >
> >> I'm working on a project that I hope to use the SimpleConsumer on. I'm
> >> trying to write some test code around our code that is wrapping the
> >> SimpleConsumer.
> >>
> >> The environment is 1 kafka broker, and 1 zookeeper
> >> The test goes like this:
> >>
> >> Create a topic ("foo", 1 partition, 1 replica)
> >> Create a Producer, and send a message
> >> Create a SimpleConsumer, and try to read the offset
> >>
> >> Failure with: NotCoordinatorForConsumerException
> >>
> >> If I continue to require for an extended period, I continue to get that
> >> exception.
> >>
> >> As near as I can tell, the topic metadata says there is a leader, but
> >>the
> >> the broker thinks it's being rebalanced.
> >> I've done the above test immediately after stop, clean out old data, and
> >> restart both zookeeper and kafka.
> >>
> >> Suggestions welcome.
> >>
> >>
> >
> >
> >--
> >-Regards,
> >Mayuresh R. Gharat
> >(862) 250-7125
>
>
