Hi there, If you're only shutting down a single broker of many and you have a replication factor more than 1, those consumer and producer configs should handle it. However, if you have only a single broker, I'd recommend getting some replication in before doing any maintenance work - Kafka is really designed to operate with replication.
Thanks Tom Crayford Heroku Kafka On Tue, Sep 6, 2016 at 5:30 AM, Harald Kirsch <harald.kir...@raytion.com> wrote: > Hi all, > > there are so many timeouts to tweak mentioned in the documentation that I > wonder what the correct configuration for producer and consumer is to > survive a, say, 1 hour, broker shutdown. > > With "survive" I mean that the processes are idle or blocked and keep > trying to send their data, and just pick up work shortly after the broker > appears again. > > I have the following suspects to tweak: > > PRODUCER: > connections.max.idle.ms: (9 minutes) Does a lost connection count as idle? > > max.block.ms: (default 1 minute) Seems a definite candidate to raise to 1 > hour. > > request.timeout.ms: (default 1/2 minute) > metadata.fetch.timeout.ms: (default 1 minute) > retry.backoff.ms: (default 100ms) > > CONSUMER: > connections.max.idle.ms: (same as above I guess) > request.timeout.ms > > What would be a good combination of settings? > > Harald. >
