Re: getting intermittent TimeoutException at producer side in streams application

2017-06-09 Thread Sachin Mittal
Hi, Please not that REPLICATION_FACTOR_CONFIG is already set as three. What is observed is that no mater what the producer request timeout is increased to for one or two partitions it still gets timed out after that time. Streams side log simply has message like this org.apache.kafka.common.errors

Re: getting intermittent TimeoutException at producer side in streams application

2017-06-09 Thread Eno Thereska
Hi Sachin, As Damian mentioned it'd be useful to see some logs from both broker and streams. One thing that comes to mind is whether your topics are replicated at all. You could try setting the replication factor of streams topics (e.g., changelogs and repartition topics) to 2 or 3 using Strea

Re: getting intermittent TimeoutException at producer side in streams application

2017-06-09 Thread Sachin Mittal
Hi All, We still intermittently get this error. We had added the config props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE); and timeout as mentioned above is set as: props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, 180); So we increased from default 30 sec to 3 min to 30 minutes.

Re: getting intermittent TimeoutException at producer side in streams application

2016-12-19 Thread Damian Guy
Hi, You could use this: https://github.com/apache/kafka/blob/trunk/streams/src/test/resources/log4j.properties as a starting point. You will probably want to change the level from WARN to INFO. Thanks On Mon, 19 Dec 2016 at 12:49 Sachin Mittal wrote: > Hi, > In order to enable streams logging

Re: getting intermittent TimeoutException at producer side in streams application

2016-12-19 Thread Sachin Mittal
Hi, In order to enable streams logging we would need to add log4j.properties as part of our classpath right? Is there any sample log properties we can use. I think this issue mostly happens when we run streams app in a different DC than the brokers. However would like to see in logs if there is

Re: getting intermittent TimeoutException at producer side in streams application

2016-12-19 Thread Damian Guy
Hi Sachin, This would usually indicate that may indicate that there is a connectivity issue with the brokers. You would need to correlate the logs etc on the brokers with the streams logs to try and understand what is happening. Thanks, Damian On Sun, 18 Dec 2016 at 07:26 Sachin Mittal wrote: