shanthoosh commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer. URL: https://github.com/apache/samza/pull/918#discussion_r261468553
########## File path: samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala ########## @@ -208,7 +223,11 @@ class SystemConsumers ( if (startpoint != null) { consumer.register(systemStreamPartition, startpoint) } else { - consumer.register(systemStreamPartition, offset) + val existingOffset = sspToRegisteredOffsets.get(systemStreamPartition) + val systemAdmin = systemAdmins.getSystemAdmin(systemStreamPartition.getSystem) + if (existingOffset == null || systemAdmin.offsetComparator(existingOffset, offset) > 0) { + sspToRegisteredOffsets.put(systemStreamPartition, offset) Review comment: 1. This would happen in the broadcast stream scenario. This older-offset-comparator check is done by some of the SystemConsumer implementation and not in the others. I'm moving it to a common layer like `SystemConsumers` above to ensure functional correctness. More context [here](https://github.com/apache/samza/pull/918#discussion_r257523873). 2. Currently this behavior is what all the system consumer implementations that support broadcast streams do. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services