Hello dear Kafka community, We are planning to build a Kafka Streams application that will build a pretty big state (~100Gb) in real time from various sources.
We later on need to query this state as fast as possible, and the main idea is to use the built-in in-memory RockDB and get the data via, some sort of REST API we will build on top. The question is how do we identify where the data we need is, since the state will obviously not fit in the memory of a single instance and we need to scale somehow in case this state keeps growing, and it will… Is there a way to identify where the data we care about resides, on what kafka streams app instance ? I tried to find the answer in the documentation, but was not able to figure it our unfortunately. Thank you in advance ! — Andrian Jardan Infrastructure and DevOps expert cell: +49 174 2815994 Skype: macrosdnb
