You would need to delete committed offsets for the application.id and set `auto.offset.reset="latest" to get the behavior you want.
-Matthias On 8/12/19 1:20 AM, Tim Ward wrote: > I believe not, because that only causes the application to start reading from > latest when there is no recorded offset at application start, no? > > What I need is to be able to specify, by topic, that when the application > starts it doesn't want to see anything other than new data, regardless of > what offset it committed last time it ran. > > Tim Ward > > -----Original Message----- > From: Boyang Chen <reluctanthero...@gmail.com> > Sent: 09 August 2019 17:23 > To: users@kafka.apache.org > Subject: Re: How to start a stream from only new records? > > Hey Tim, > > if you are talking about avoid re-processing data and start consumption > from latest, you could set your `offset.reset.policy` to latest. > > Let me know if this answers your question. > > On Fri, Aug 9, 2019 at 7:09 AM Tim Ward <tim.w...@origamienergy.com> wrote: > >> With a real time application, nobody is interested in old data, and in >> particular they're not interested in paying to spend time processing it >> only to throw it away, thereby delaying up to date data. >> >> How do I tell StreamsBuilder.stream() to read only new data from its >> topic, and not process any backlog? >> >> Tim Ward >> >> This email is from Origami Energy Limited. The contents of this email and >> any attachment are confidential to the intended recipient(s). If you are >> not an intended recipient: (i) do not use, disclose, distribute, copy or >> publish this email or its contents; (ii) please contact Origami Energy >> Limited immediately; and then (iii) delete this email. For more >> information, our privacy policy is available here: >> https://origamienergy.com/privacy-policy/. Origami Energy Limited >> (company number 8619644) is a company registered in England with its >> registered office at Ashcombe Court, Woolsack Way, Godalming, GU7 1LQ. >> > This email is from Origami Energy Limited. The contents of this email and any > attachment are confidential to the intended recipient(s). If you are not an > intended recipient: (i) do not use, disclose, distribute, copy or publish > this email or its contents; (ii) please contact Origami Energy Limited > immediately; and then (iii) delete this email. For more information, our > privacy policy is available here: https://origamienergy.com/privacy-policy/. > Origami Energy Limited (company number 8619644) is a company registered in > England with its registered office at Ashcombe Court, Woolsack Way, > Godalming, GU7 1LQ. >
signature.asc
Description: OpenPGP digital signature
