Hey all, I am trying to understand a situation I came across and can't find an explanation...
In my compacted topic I have 32 partitions, and when searching the files I have found that a message is found on 2 different partitions (4 and 16). It seems that the "right" partition is 16 because it keeps updating the value, while the message on partition 4 is never updated and contains an out-of-date value. This topic is an output of my kafka-stream application. The problem I get is when I consume the whole topic, I get the "bad" message last and this will cause the value I am seeing to be wrong (out-of-date). I can solve it programmatically but I am not sure this situation should even happen (maybe I am wrong making this assumption?) Here is a snippet of the values I get when consuming and filtering for the desired message (in *bold* you can see the "bad" message): CreateTime:1557847043142 unique_key_123 14351.7399916489 CreateTime:1557848585761 unique_key_123 14352.6399916489 CreateTime:1557849467135 unique_key_123 14353.0899916489 CreateTime:1557849565972 unique_key_123 14356.8399916489 CreateTime:1557850075281 unique_key_123 14357.5199916489 CreateTime:1557850272513 unique_key_123 14358.1999916489 CreateTime:1557850323990 unique_key_123 14358.6499916489 CreateTime:1557850919067 unique_key_123 14360.8999916489 CreateTime:1557851600162 unique_key_123 14361.5799916489 CreateTime:1557852735188 unique_key_123 14367.4299916489 CreateTime:1557852870033 unique_key_123 14369.6799916489 CreateTime:1557852913943 unique_key_123 14372.8299916489 CreateTime:1557853111931 unique_key_123 14375.9799916489 CreateTime:1557853257402 unique_key_123 14376.4299916489 CreateTime:1557853654326 unique_key_123 14378.6799916489 CreateTime:1557854126273 unique_key_123 14382.2799916489 CreateTime:1557854882530 unique_key_123 14387.6799916489 CreateTime:1557855665133 unique_key_123 14392.8599916489 CreateTime:1557856458505 unique_key_123 14394.6099916489 CreateTime:1557856814818 unique_key_123 14398.2099916489 CreateTime:1557857560956 unique_key_123 14400.4599916489 CreateTime:1557858721649 unique_key_123 14404.0599916489 CreateTime:1557859251577 unique_key_123 14405.8099916489 CreateTime:1557859736903 unique_key_123 14409.4099916489 *CreateTime:1556637853098 unique_key_123 9954.93* CreateTime:1557860847333 unique_key_123 14411.4099916489 CreateTime:1557861171424 unique_key_123 14413.2099916489 CreateTime:1557861411332 unique_key_123 14413.8899916489 CreateTime:1557861459234 unique_key_123 14414.5699916489 CreateTime:1557861550401 unique_key_123 14422.6699916489 CreateTime:1557861709925 unique_key_123 14425.3699916489 CreateTime:1557862790701 unique_key_123 14427.6199916489 CreateTime:1557863450236 unique_key_123 14428.0699916489 CreateTime:1557865167738 unique_key_123 14433.0699916489 CreateTime:1557865378520 unique_key_123 14435.3199916489 CreateTime:1557865441654 unique_key_123 14437.0699916489 CreateTime:1557865503081 unique_key_123 14437.5199916489 CreateTime:1557865907130 unique_key_123 14438.1999916489 CreateTime:1557867190954 unique_key_123 14440.6799916489 CreateTime:1557867416951 unique_key_123 14441.1299916489 CreateTime:1557869616387 unique_key_123 14443.6099916489 CreateTime:1557870569098 unique_key_123 14445.3599916489 CreateTime:1557871319283 unique_key_123 14450.3599916489 CreateTime:1557872731646 unique_key_123 14451.2599916489 I would really appreciate an explanation or reassurance that this is not expected behavior. Let me know if I can supply more information Thanks! -- Nitay Kufert Backend Developer [image: ironSource] <http://www.ironsrc.com> email nita...@ironsrc.com mobile +972-54-5480021 fax +972-77-5448273 skype nitay.kufert.ssa 9 Ehad Ha'am st. Tel- Aviv ironsrc.com <http://www.ironsrc.com> [image: linkedin] <https://www.linkedin.com/company/ironsource> [image: twitter] <https://twitter.com/ironsource> [image: facebook] <https://www.facebook.com/ironSource> [image: googleplus] <https://plus.google.com/+ironsrc> This email (including any attachments) is for the sole use of the intended recipient and may contain confidential information which may be protected by legal privilege. If you are not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any use, dissemination, distribution or copying of this communication and/or its content is strictly prohibited. If you are not the intended recipient, please immediately notify us by reply email or by telephone, delete this email and destroy any copies. Thank you.
