Re: [Kafka Streams], Processor API for KTable and KGroupedStream

You cannot add a `Processor`. You can only use `aggregate() / reduce() / count()` (which of course will add a pre-defined processor). `groupByKey()` is really just a "meta operation" that checks if the key was changes upstream, and to insert a repartition/shuffle step if necessary. Thus, if y

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

Thanks a lot for explanation but could you provide a bit more details about KGroupedStream? It is just interface and not extends KStream so how I can add processor in the case below? /   KStream someStream = / /  someStream / /     .groupByKey() / */how to add processor for resulted grouped

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

`KGroupedStream` is just an "intermediate representation" to get a better flow in the DSL. It's not a "top level" abstraction like KStream/KTable. For `KTable` there is `transformValue()` -- there is no `transform()` because keying must be preserved -- if you want to change the keying you ne

Re: Kafka Streams Processor API state stores not restored via changelog topics

Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 2:10 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Ka

Re: Kafka Streams Processor API state stores not restored via changelog topics

From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 1:00 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Kafka Streams Processor API state stores not restored via > changelog topics > > Hello Upesh, > > These are super helpful logs, and I think I'm very

Re: Kafka Streams Processor API state stores not restored via changelog topics

; > > Best, > > Upesh > > > Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 1:0

Re: Kafka Streams Processor API state stores not restored via changelog topics

Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Tuesday, March 30, 2021 at 1:00 PM To: Users Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, These are super helpful logs

Re: Kafka Streams Processor API state stores not restored via changelog topics

or give us more > insight into where this issue could originate from? > > > > Thanks, > Upesh > > > Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> >

Re: Kafka Streams Processor API state stores not restored via changelog topics

Date: Thursday, March 25, 2021 at 6:46 PM To: users@kafka.apache.org Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics We have not tried running 2.6 brokers and 2.7 client, I will try and get back to you. We are not enabling EOS on the streams

Re: Kafka Streams Processor API state stores not restored via changelog topics

instantly. Best, Upesh Upesh Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Thursday, March 25, 2021 at 6:31 PM To: Users Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics BTW, yes that indicates the

Re: Kafka Streams Processor API state stores not restored via changelog topics

ppen when we run everything on Kafka version 2.6. >> >> >> >> Thanks, >> >> Upesh >> >> >> Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* >> >> *www.itrsgroup.com* <https://www.itrsgroup.com/> >> <https:

Re: Kafka Streams Processor API state stores not restored via changelog topics

ww.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Thursday, March 25, 2021 at 4:01 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Kafka Streams Processor API state stores not restored via > changelog topics > >

Re: Kafka Streams Processor API state stores not restored via changelog topics

Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, Could you confirm a few more things for me: 1. After you stopped the application, and wiped out the state dir; check if the corresponding changelog topic has one record indeed at offset 0 --- this

Re: Kafka Streams Processor API state stores not restored via changelog topics

gt; Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Wednesday, March 24, 2021 at 1:37 PM > *To: *Users > *Cc: *Bart Lilje > *S

Re: Kafka Streams Processor API state stores not restored via changelog topics

...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Wednesday, March 24, 2021 at 1:37 PM To: Users Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, Thanks for the detailed report. I looked through the code and tried to

Re: Kafka Streams Processor API state stores not restored via changelog topics

Hello Upesh, Thanks for the detailed report. I looked through the code and tried to reproduce the issue, but so far have not succeeded. I think I may need some further information from you to help my further investigation. 1) The `bufferedLimitIndex == 0` itself does not necessarily mean there's

Re: Kafka Streams & Processor API

Hey Navneeth, thank you for your interest in trying out Kafka Streams! Normally we will redirect new folks to the stream FAQ first in case you haven't checked it out. For details to your question: 1. Joining 2 topics using processor API (we cal