Re: Kafka Streams upgrade.from config while upgrading to 2.1.0

2019-03-26 Thread Anirudh Vyas
Hey, We run 3 instances. Anirudh On Tue, Mar 26, 2019 at 9:28 PM Matthias J. Sax wrote: > Not sure. How many instances to do you run? If it's only one, you don't > need the config. > > -Matthias > > On 3/26/19 5:17 AM, Anirudh Vyas wrote: > > Hi, > > I am in the process of upgrading my Kafka st

Re: Proxying the Kafka protocol

2019-03-26 Thread Martin Gainty
mg>that depends on the underlying protocol you are attempting to proxy (see below) From: James Grant Sent: Monday, March 25, 2019 1:21 PM To: users@kafka.apache.org Subject: Re: Proxying the Kafka protocol Thank you all. We have in the past exposed message stre

Re: Tracking progress for messages generated by a batch process

2019-03-26 Thread Tim Gent
Thanks all for the feedback and suggestions. @Henn I did consider something like your suggestion, though it brings a couple of challenges: 1) Not all messages emitted from batch job will make it to streaming app 3, some will be filtered out (granted you could change all applications to not filter

Re: [ANNOUNCE] Apache Kafka 2.2.0

2019-03-26 Thread Stephane Maarek
Congratulations on this amazing release! Lots of cool new features :) I've also released a YouTube video that will hopefully help the community get up to speed: https://www.youtube.com/watch?v=kaWbp1Cnfo4&t=5s Happy watching! On Tue, Mar 26, 2019 at 7:02 PM Matthias J. Sax wrote: > The Apache

[ANNOUNCE] Apache Kafka 2.2.0

2019-03-26 Thread Matthias J. Sax
The Apache Kafka community is pleased to announce the release for Apache Kafka 2.2.0 - Added SSL support for custom principal name - Allow SASL connections to periodically re-authenticate - Command line tool bin/kafka-topics.sh adds AdminClient support - Improved consumer group management -

Re: Kafka Streams upgrade.from config while upgrading to 2.1.0

2019-03-26 Thread Matthias J. Sax
Not sure. How many instances to do you run? If it's only one, you don't need the config. -Matthias On 3/26/19 5:17 AM, Anirudh Vyas wrote: > Hi, > I am in the process of upgrading my Kafka streams services from 1.1 to > 2.1.0. I am following the upgrade guide: > https://kafka.apache.org/20/docume

Re: KafkaStreams backoff for non-existing topic

2019-03-26 Thread Murilo Tavares
Thank you guys for your response. That was very helpful. With that in mind, and with some discussions with the team, we decided to let the application die, so we can monitor and relaunch it externally. I think we had something similar to what is described in KAFKA-7970, where our streams applicatio

Kafka Streams upgrade.from config while upgrading to 2.1.0

2019-03-26 Thread Anirudh Vyas
Hi, I am in the process of upgrading my Kafka streams services from 1.1 to 2.1.0. I am following the upgrade guide: https://kafka.apache.org/20/documentation/streams/upgrade-guide . My service is running on kafka version 2.0 and using kafka streams 1.1.1. I updated my kafka-streams to 2.1.0 but DI