Re: Kafka streams DSL going into infinite loop

2020-01-30 Thread Matthias J. Sax
>> I really don't know what TOPOLOGY_OPTIMIZATION is for. If you enable optimization, Kafka Streams tries to generate a more efficient Topology when translating the DSL program. We are working on some more documentation of this feature atm: https://cwiki.apache.org/confluence/display/KAFKA/DSL+Opt

Re: Kafka streams DSL going into infinite loop

2020-01-30 Thread Sachin Mittal
Only streams specific props I am using are: props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 0); props.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 1000); props.put(StreamsConfig.TOPOLOGY_OPTIMIZATION, StreamsConfig.OPTIMIZE); Yes there are three sub topologies and key does change betwee

Re: Kafka streams DSL going into infinite loop

2020-01-30 Thread Matthias J. Sax
Your program does not seem to contain a loop. Hence, it's unclear to me atm what could be the issue. Does the application commit offset on the input topic, what would be an indicator that it actually does make progress. Do you change the key between joins, ie, is there multiple sub-topologies tha