I recently realised that our consumers, which don’t use auto commit, will suffer from the issue outlined here (https://issues.apache.org/jira/browse/KAFKA-3806 <https://issues.apache.org/jira/browse/KAFKA-3806>). Looking through the kafka streams code it will also suffer from this issue. I know there is a work around to set the retention time to some large value, but given that stream aggregation is likely to be something that can happy over a long period of time, and that there might per times when a given partition is inactive, would the streams guys consider putting in some protection similar to what Jun Rao suggested in the issue ticket? Something which will check the last actual commit offset time and override commitOffsetNeeded if it hasn’t ben saved within that time.
Thanks.
