Re: Same key found on 2 different partitions of compacted topic (kafka-streams)

2019-05-19 Thread Nitay Kufert
In case someone will be in the same situation I was, here is some extra info regarding this situation: So, even after resting the streams application again - the same problem continued. I still had the same key which was supposed to be "compacted" on 2 different partitions. After reading more car

Re: Same key found on 2 different partitions of compacted topic (kafka-streams)

2019-05-17 Thread Svante Karlsson
Yes that sound likely, if you changed the number of partitions then the hashing of the key's will change destination. You need to either clear the data (ie change retention to very small and roll the logs) or recreate the topic. /svante Den fre 17 maj 2019 kl 12:32 skrev Nitay Kufert : > I would

Re: Same key found on 2 different partitions of compacted topic (kafka-streams)

2019-05-17 Thread Nitay Kufert
I would like to add that during this time I played around with my kafka-streams application, including several resets of the application. I think maybe I caused it by not deleting all the messages from the input topic of the streams application before increasing the number of partitions and resetti

Same key found on 2 different partitions of compacted topic (kafka-streams)

2019-05-17 Thread Nitay Kufert
Hey all, I am trying to understand a situation I came across and can't find an explanation... In my compacted topic I have 32 partitions, and when searching the files I have found that a message is found on 2 different partitions (4 and 16). It seems that the "right" partition is 16 because it ke