Steven, you may be hitting on KAFKA-1642
<https://issues.apache.org/jira/browse/KAFKA-1642>.
As Jay said, a bunch of such issues are fixed in the new release. Please
let us know if you still see the issue with it.
Guozhang
On Tue, Feb 3, 2015 at 8:52 PM, Steven Wu <stevenz...@gmail.com> wrote:
> sure. will try my unit test again with 0.8.2.0 release tomorrow and report
> back my findings.
>
> On Tue, Feb 3, 2015 at 8:42 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
>
> > Hey Steven,
> >
> > That sounds like a bug. I think we fixed a few producer high cpu issues
> > since the beta, I wonder if you could repeat the same test with the
> 0.8.2.
> > final release?
> >
> > -Jay
> >
> > On Tue, Feb 3, 2015 at 8:37 PM, Steven Wu <stevenz...@gmail.com> wrote:
> >
> > > actually, my local test can reproduce the issue although not
> immediately.
> > > seems to happen after a few mins. I enabled TRACE level logging. here
> > seems
> > > to be the tight loop. you can see that there are two metadata requests
> in
> > > one milli-seconds.
> > >
> > > kafka-producer-network-thread | foo 20:34:32,626 TRACE
> NetworkClient:301
> > -
> > > Ignoring empty metadata response with correlation id 360185.
> > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG
> NetworkClient:369
> > -
> > > Trying to send metadata request to node -2
> > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG
> NetworkClient:374
> > -
> > > Sending metadata request ClientRequest(expectResponse=true,
> payload=null,
> > >
> > >
> >
> request=RequestSend(header={api_key=3,api_version=0,correlation_id=360186,client_id=foo},
> > > body={topics=[]})) to node -2
> > > kafka-producer-network-thread | foo 20:34:32,626 TRACE
> NetworkClient:301
> > -
> > > Ignoring empty metadata response with correlation id 360186.
> > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG
> NetworkClient:369
> > -
> > > Trying to send metadata request to node -2
> > >
> > >
> > > On Tue, Feb 3, 2015 at 8:10 PM, Steven Wu <stevenz...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > We have observed high cpu and high network traffic problem when
> > > > 1) cluster (0.8.1.1) has no topic
> > > > 2) KafkaProducer (0.8.2-beta) object is created without sending any
> > > traffic
> > > >
> > > > We have observed such problem twice. In both cases, problem went away
> > > > immediately after one/any topic is created.
> > > >
> > > > Is this a known issue? Just want to check with the community first
> > before
> > > > I spend much time to reproduce it.
> > > >
> > > > I couldn't reproduce the issue with similar setup with unit test code
> > in
> > > > IDE. start two brokers with no topic locally on my laptop. create a
> > > > KafkaProducer object without sending any msgs. but I only tested with
> > > > 0.8.2-beta for both broker and producer.
> > > >
> > > > Thanks,
> > > > Steven
> > > >
> > >
> >
>
--
-- Guozhang
