Hi, This is a type of problem where you operating more than network capacity. This can be handled at two places (you decide whichever is useful/practical for use case) :-
1. In case the bottleneck is because of the broker slowness, increase the number of partitions of your topic, balance them out to newer nodes (if required) and retry. Make sure your producer can write to the new partitions as well 2. In case this is because your producer is creating too many messages, there are 2 ways you can commit them (select the most appropriate according to your use case. I'm considering that you dont want to lose your messages):- ------> a. increase the number of producer machines (i.e. balance your producer load among more machines) -------> b. produce at a slower rate (in case your producers are not scalable, you are in a problem. you can solve this problem by sacrificing latency). Write to your queue in batches and wait for it to get cleared before you commit the next batch to your local queue. Regards, Prabcs On Mon, Oct 26, 2015 at 10:41 PM, Eduardo Costa Alfaia < e.costaalf...@unibs.it> wrote: > Hi Magnus > I think this answer > c) producing messages at a higher rate than the network or broker can > handle > How could I manager this? > > > > On 26 Oct 2015, at 17:45, Magnus Edenhill <mag...@edenhill.se> wrote: > > > > c) producing messages at a higher rate than the network or broker can > > handle > > > -- > Informativa sulla Privacy: http://www.unibs.it/node/8155 > -- --------------------------------------------------------- "There are only 10 types of people in the world: Those who understand binary, and those who don't"
