Re: Zookeeper on same server as Kafka

2017-06-03 Thread Michal Borowiecki
I'm not an expert but I prefer keeping zookeepers on the same hosts as kafka brokers and mimic each-others topology. The reason is to minimize the chance of e.g. kafka brokers being able to talk to one another but zookeepers not, or vice-versa. So, I'd say I /do/ want my kafka broker and the co

Re: Losing messages in Kafka Streams after upgrading

2017-06-03 Thread Frank Lyaruu
Hi Matthias, Ok, that clarifies quite a bit. I never really went into the timestamp aspects, as time does not really play a role in my application (aside from the repartition topics, I have no KStreams or Windowed operation, just different kind of KTable join). I do think that the fail case I see

Re: Kafka Over TLS Error - Failed to send SSL Close message - Broken Pipe

2017-06-03 Thread Waleed Fateem
Unless you're experiencing any connection issues then I would say this is a benign warning message. Essentially what it's saying here is that the connection was closed (broken pipe) before it (wherever you're getting the log from) was able to send the last close_notify message. When one party deci

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-06-03 Thread Jan Filipiak
Could not agree more! But then I think the easiest is still: print exception and die. From there on its the easiest way forward: fix, redeploy, start => done All the other ways to recover a pipeline that was processing partially all the time and suddenly went over a "I cant take it anymore" thr