That is not supported out-of-box. Configuration "auto.offset.reset" only triggers, if there are not committed offsets and there is KS config to change this behavior.
A possible workaround might be (but not sure if I want to recommend this), to increase KafkaStreams commit interval via StreamsConfig (you could set it to MAX_VALUE, effectively disable committing). Thus, auto.offset.reset should trigger on restart. You might want to try it out and see if it works for your... Note, if we never commit, we also never flush KTable caches, thus you might need to disable caching, too (by setting cache size to zero). As an alternative, you could manipulate offsets manually before startup using bin/kafka-consumer-groups.sh --- the application.id is the group.id and and you could "seek to end" before you restart the application. Hope this helps. -Matthias On 1/19/18 9:22 AM, Saloni Vithalani wrote: > Our requirement is such that if a kafka-stream app is consuming a > partition, it should start it's consumption from latest offset of that > partition. > > This seems like do-able using > > streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest") > > Now, let's say using above configuration, the kafka-stream app started > consuming data from latest offset for a partition. And after some time, the > app crashes. When the app comes back live, we want it to consume data from > the latest offset of that partition, instead of the where it left last > reading. > > But I can't find anything that can help achieve it using kafka-streams api. > > P.S. We are using kafka-1.0.0. > > Saloni Vithalani > Developer > Email salo...@thoughtworks.com > Telephone +91 8552889571 <8552889571> > [image: ThoughtWorks] > <http://www.thoughtworks.com/?utm_campaign=saloni-vithalani-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator> >
signature.asc
Description: OpenPGP digital signature
