Re: Accessing Kafka stream's KTable underlying RocksDB memory usage

2019-02-17 Thread Patrik Kleindl
Hi How many partitions do your topics have? As far as I understand there is a RocksDB for every partition of every KTable and this can add up quickly. Depending on how many instances you are using one of them might have to handle the complete load temporarily which will use more memory. Also, Roc

Re: Accessing Kafka stream's KTable underlying RocksDB memory usage

2019-02-15 Thread Matthias J. Sax
Cross posted on SO: https://stackoverflow.com/questions/54701449/accessing-kafka-streams-ktable-underlying-rocksdb-memory-usage On 2/14/19 9:24 PM, P. won wrote: > Hi, > > I have a kafka stream app that currently takes 3 topics and aggregates > them into a KTable. This app resides inside a micros

Accessing Kafka stream's KTable underlying RocksDB memory usage

2019-02-14 Thread P. won
Hi, I have a kafka stream app that currently takes 3 topics and aggregates them into a KTable. This app resides inside a microservice which has been allocated 512 MB memory to work with. After implementing this, I've noticed that the docker container running the microservice eventually runs out of