AW: Question regarding Kafka Streams Global State Store

2020-05-19 Thread Schmidt-Dumont Georg (BCI/ESW17)
Hi Bruno, I just had a discussion with a colleague of mine regarding this and I wanted to give you a quick contextual update. With regards to the global state, I realize that having this state consistent in a distributed system is very difficult. My expectation was that since it is a global sta

AW: Question regarding Kafka Streams Global State Store

2020-05-19 Thread Schmidt-Dumont Georg (BCI/ESW17)
Hi Bruno, The data I am processing comes from machines. The key is the identifier for the machine which produced a specific message. Currently we only have a couple of these machines producing data. This number will increase quite a lot over the coming years. Is the limit on the number of parti

AW: Question regarding Kafka Streams Global State Store

2020-05-19 Thread Schmidt-Dumont Georg (BCI/ESW17)
Hi Bruno, Thanks for your quick reply! I decided to use a global state store for two reasons. If the application crashes, the store is populated properly once the reason for the crash has been fixed and the app starts again, i.e. I feel that it gives me a certain resiliency. Second we will be