Hi, I am currently trying to setup Kafka 0.8 and ran into a problem with the producer. Every time the server triggers a garbage collection, because permgen space is full, the server descends into a very unusual memory state where a lot of garbage collection is suddenly happening (~5-10% gc cpu time, when normally it is 0-0.5%). Additionally a lot of objects suddenly get promoted from eden space to survivor space, old gen and perm gen. This obviously slows down the whole server and turns up the system load by a lot. This behaviour only appears when the producer is active during a perm gen gc and can be remedied by reloading the producer(s). Since reloading the producer(s) every 5 seconds really is not how it is supposed to work, I am wondering if any one has any pointers as to where the problem may be?
My system has the following specs: - oracle jdk 1.6.0_22-b04 - sending about 12000 messages per second, about 60 bytes each - producer counts tried: 1, 10, 25, 50 - async producer - ack makes no difference - batch sizes tried: 100 - 5000 I made screenshot of the behaviour: http://i.imgur.com/xtlVNaK.png at ~10:24:30 the full gc happens, surivor space fills up and old gen becomes a sawtooth. at ~10:34 I reloaded the producers and everything was fine again. Cheers, Flo
