So, basically, you want the clients to gracefully handle timeouts and short circuit if it is unavailable broker exception and also clients should be able automatically recover when it is up? What about the messages that come in when broker is not available?
On Sun, Oct 30, 2016 at 12:25 PM, Andrey Dyachkov <andrey.dyach...@gmail.com > wrote: > R Krishna, > > We have default timeout for producer and consumer it is 30 secs. > If Kafka is not available we will wait 30 secs on each request. > Decreasing timeout won't help much because we have a lot requests > but circuiting them will help us tremendous. We could implement such > logic before actually calling Kafka client but there is problem what the > trigger is to open the circuit. > I thought about circuiting by unavailable broker > but it is quite an expansive operation to figure out in the client's code > on the flip side Kaka client already does all the job to make out the > broker. > > For us very important to stay live even if Kafka is unavailable (it > happens) that's why I think avoiding cascading issues will be very useful. > > On Sat, 29 Oct 2016 at 19:22 R Krishna <krishna...@gmail.com> wrote: > > > What are the issues and problems you want to solve though? > > On Oct 29, 2016 3:59 AM, "Andrey Dyachkov" <andrey.dyach...@gmail.com> > > wrote: > > > > > Hi, > > > > > > We use Kafka for around half a year as a main backend system for > storing > > > events from big variety of internal microservices. In general It works > > > quite well and we experience issues only in some rare 'corner cases' > > where > > > we either run against a bug or do not understand why Kafka behaves in > > > particular manner. The fact Kafka is the main system is used under our > > > software If it fails our system will fail too. That's why I am writing > to > > > ask about the solution which can help us tolerate different issues on > > Kafka > > > side. First of all could you tell me why doesn't Kafka client have the > > > solution for this kind of problems? Have you ever had thoughts about > it? > > > Thank you in advance! > > > -- > > > > > > With great enthusiasm, > > > Andrey > > > > > > -- > > With great enthusiasm, > Andrey > -- Radha Krishna, Proddaturi 253-234-5657
