My KStreams app seems to be having some memory issues.

1. I start it `java -Xmx8G -jar <app>.jar`

2. Wait 5-10 minutes - see lots of 'org.apache.zookeeper.ClientCnxn - Got
ping response for sessionid: 0xc58abee3e000013 after 0ms'  messages

3. When it _finally_ starts reading values it typically goes for a minute
or so, reads a few thousand values and then the OS kills it with 'Out of
memory' error.

The topology is (essentially):

stream->aggregate->filter->foreach

It's reading values and creating a rolling average.

During phase 2 (above) I see lots of IO wait and the 'scratch' buffer
(usually "/tmp/kafka-streams/appname") fills with 10s of Gb of .. ? (I had
to create a special scratch partition with 100Gb of space as kafka would
fill the / partition and make the system v v unhappy)

Have I misconfigured something?

Reply via email to