Hi, I am running a simple kafka streams app (0.11.0.1) that counts messages per hour per partition. The app runs in a docker container with a memory limit set, which is always reached by the app within few minutes and then container is killed. After running it with various number of instances, different memory limits and in-memory store instead - it looks like it is off-heap memory being taken by rocks db. After playing with different memory limits it looks like rocksdb assumes it can grab all the physical memory of the machine, so if the container limit is less than it gets killed on the way.
Also I have not changed any rocksdb config settings, but the defaults mentioned here: https://docs.confluent.io/current/streams/developer-guide.html#streams-developer-guide-rocksdb-config looks nowhere close to the consumption observed. Few details about the app: I use windowed store defined as follows: StateStoreSupplier windowCounts = Stores.create(WINDOW_COUNT_STORE) .withIntegerKeys() .withLongValues() .persistent() .enableCaching() .windowed(MINUTES.toMillis(1), HOURS.toMillis(5), 10, false) .build(); There is a processor that updates a count for a partition for a timestamp that is rounded to an hour boundary: store.put( context.partition(), current(floor(value.getIngressTime())) + 1, floor(value.getIngressTime()) ); Any hints on what might cause this or any config settings? Thank you, Stanislav.
