No this is not normal. Checking twice a second (using 500ms default) for new data shouldn't cause high network traffic (that should be like < 1KB of overhead). I don't think that explains things. Is it possible that setting has been overridden?
-Jay On Tue, Feb 4, 2014 at 9:25 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hi Carl, > > For each partition the follower will also fetch data from the leader > replica, even if there is no new data in the leader replicas. > > One thing you can try to increase replica.fetch.wait.max.ms (default value > 500ms) so that the followers's fetching request frequency to the leader can > be reduced, and see if that has some effect on the traffic. > > Guozhang > > > On Tue, Feb 4, 2014 at 8:46 PM, Carl Lerche <m...@carllerche.com> wrote: > > > Hello, > > > > I'm running a 0.8.0 Kafka cluster of 3 servers. The service that it is > > for is not in full production yet, so the data written to cluster is > > minimal (seems to average between 100kb/s -> 300kb/s per server). I > > have configured Kafka to have a 3 replicas. I am noticing that each > > Kafka server is talking to all the others at a data rate of 40MB/s for > > each server (so, a total of 80MB/s for each server). This > > communication is constant. > > > > Is this normal? This seems like very strange behavior and I'm not > > exactly sure how to debug. > > > > Thanks, > > Carl > > > > > > -- > -- Guozhang >
