I have re-run my unit test with 0.8.2.0. same tight-loop problem happened after a few mins.
On Tue, Feb 3, 2015 at 10:00 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Steven, you may be hitting on KAFKA-1642 > <https://issues.apache.org/jira/browse/KAFKA-1642>. > > As Jay said, a bunch of such issues are fixed in the new release. Please > let us know if you still see the issue with it. > > Guozhang > > On Tue, Feb 3, 2015 at 8:52 PM, Steven Wu <stevenz...@gmail.com> wrote: > > > sure. will try my unit test again with 0.8.2.0 release tomorrow and > report > > back my findings. > > > > On Tue, Feb 3, 2015 at 8:42 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > > > Hey Steven, > > > > > > That sounds like a bug. I think we fixed a few producer high cpu issues > > > since the beta, I wonder if you could repeat the same test with the > > 0.8.2. > > > final release? > > > > > > -Jay > > > > > > On Tue, Feb 3, 2015 at 8:37 PM, Steven Wu <stevenz...@gmail.com> > wrote: > > > > > > > actually, my local test can reproduce the issue although not > > immediately. > > > > seems to happen after a few mins. I enabled TRACE level logging. here > > > seems > > > > to be the tight loop. you can see that there are two metadata > requests > > in > > > > one milli-seconds. > > > > > > > > kafka-producer-network-thread | foo 20:34:32,626 TRACE > > NetworkClient:301 > > > - > > > > Ignoring empty metadata response with correlation id 360185. > > > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG > > NetworkClient:369 > > > - > > > > Trying to send metadata request to node -2 > > > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG > > NetworkClient:374 > > > - > > > > Sending metadata request ClientRequest(expectResponse=true, > > payload=null, > > > > > > > > > > > > > > request=RequestSend(header={api_key=3,api_version=0,correlation_id=360186,client_id=foo}, > > > > body={topics=[]})) to node -2 > > > > kafka-producer-network-thread | foo 20:34:32,626 TRACE > > NetworkClient:301 > > > - > > > > Ignoring empty metadata response with correlation id 360186. > > > > kafka-producer-network-thread | foo 20:34:32,626 DEBUG > > NetworkClient:369 > > > - > > > > Trying to send metadata request to node -2 > > > > > > > > > > > > On Tue, Feb 3, 2015 at 8:10 PM, Steven Wu <stevenz...@gmail.com> > > wrote: > > > > > > > > > Hi, > > > > > > > > > > We have observed high cpu and high network traffic problem when > > > > > 1) cluster (0.8.1.1) has no topic > > > > > 2) KafkaProducer (0.8.2-beta) object is created without sending any > > > > traffic > > > > > > > > > > We have observed such problem twice. In both cases, problem went > away > > > > > immediately after one/any topic is created. > > > > > > > > > > Is this a known issue? Just want to check with the community first > > > before > > > > > I spend much time to reproduce it. > > > > > > > > > > I couldn't reproduce the issue with similar setup with unit test > code > > > in > > > > > IDE. start two brokers with no topic locally on my laptop. create a > > > > > KafkaProducer object without sending any msgs. but I only tested > with > > > > > 0.8.2-beta for both broker and producer. > > > > > > > > > > Thanks, > > > > > Steven > > > > > > > > > > > > > > > > > > -- > -- Guozhang >
