Re: Kafka streams - message not materialized in intermediate topics

2021-10-28 Thread Tomer Cohen
gt; > Thanks, > Bill > > On Thu, Oct 28, 2021 at 12:07 PM Tomer Cohen wrote: > > > Hi Bill/Matthias, > > > > Thanks for the replies. > > > > The issue is I never see a result, I have a log that shows the message > > coming in, but the adder/subtra

Re: Kafka streams - message not materialized in intermediate topics

2021-10-28 Thread Tomer Cohen
ote: > > > For this case, you can call `aggregate(...).suppress()`. > > > > -Matthias > > > > On 10/27/21 12:42 PM, Tomer Cohen wrote: > > > Hi Bill, > > > > > > Thanks for the prompt reply. > > > > > > Setting to 0 forces a no collection

Re: Kafka streams - message not materialized in intermediate topics

2021-10-27 Thread Tomer Cohen
ors. > > If you want every Kafka Streams to forward every record you'll need to set > the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0. > > If I haven't understood your experience accurately can you provide a few > more details? > > Thanks, > Bill > &

Kafka streams - message not materialized in intermediate topics

2021-10-27 Thread Tomer Cohen
Hello Kafka team, I am seeing an odd behavior when using kafka streams. During periods of heavier volumes, there are messages coming in. However, they do look like they are processed by the adder/subtractor and are not materialized in the intermediate topics which causes them not to be outputted

Re: Reading all messages from a Kafka topic for a state

2020-12-08 Thread Tomer Cohen
loose your local state, Kafka Streams will first restore the > state before resuming processing. > > > -Matthias > > On 11/30/20 6:20 AM, Tomer Cohen wrote: > > Thanks Matthias for the detailed and helpful explanation > > > > Is there a way that I can ensure that t

Re: Reading all messages from a Kafka topic for a state

2020-11-30 Thread Tomer Cohen
Thanks Matthias for the detailed and helpful explanation Is there a way that I can ensure that the KTable will be read from the beginning of the topic on every restart of the application without having to generate a new "application.id"? Thanks, Tomer

Reading all messages from a Kafka topic for a state

2020-11-20 Thread Tomer Cohen
Hello everyone I have the following use case: I have two Kafka topics, one is meant to be used as a stream of incoming messages to be processed, the other is meant as a store of records that is meant to be used as a bootstrap to the initial state of the application. Is there a way to do the foll