Reg: Dual Kafka Clients - Issues with jar version conflicts

[Gowtham S]

Hello,

As per our team requirements, we are supposed to use and communicate
between two different Kafka Clients, provided and managed by two separate
teams). Mentioned below are the client versions

Version 1 - kafka-clients-2.4.0.jar
Version 2 - kafka-clients-0.10.0.1.jar

Please suggest if possible to use them without compatibility issues. Is it
necessary to move the topics to a single Kafka?
If necessary to upgrade the clients, suggest the best practices for the
same.


With regards,
Gowtham S, MCA

Re: High write operations rate on disk

[Soumyajit Sahu]

Our typical IOPS stays at ~10K write ops/min, but it goes to 37K write
ops/min (which is where AWS throttles).
The spike in write ops isn't accompanied by any spike in write throughput
or produce requests (except for the first few minutes of catch up). The
write ops spike stays up (persistently for an hour or two) until we stop
the broker ec2 instance for about 30 mins and then start it back.

@Liam, no, we are not using log compaction except for a few consumer offset
topics and config topic (for Kafka Connect), and schema registry store.

@Suman, are you using m5 or r5 instances. Recently, we migrated from r5 to
m5, and I wonder if that has a hand in this.

We have about 1000 partitions residing on each disk, but I don't think that
matters as most of the time the brokers run flawlessly (even during peak
traffic hours).

Thanks!

On Mon, Apr 6, 2020 at 11:39 PM Suman B N  wrote:

> We too have a similar setup but we never observed any such spikes.
>
> Are you sure your disk IOPS is good enough? Check if that is throttling.
>
> After a broker restarts, there might be more traffic as well because of
> followers trying to catch up with the leader.
>
> -Suman
>
> On Tue, Apr 7, 2020 at 11:59 AM Soumyajit Sahu 
> wrote:
>
> > We are running Kafka on AWS EC2 instances (m5.2xlarge) with mounted EBS
> st1
> > volume (one on each machine).
> > Occasionally, we have noticed that the write ops/second goes through the
> > roof and we get throttled by AWS while the data throughput wouldn't have
> > changed much. As far as our observation goes, it happens usually after a
> > broker restart.
> >
> > Has anyone else come across this behavior?
> >
> > Thanks!
> >
>
>
> --
> *Suman*
> *OlaCabs*
>

Re: High write operations rate on disk

[Seva Feldman]

ST1 EBS fit only for sequential rights and reads. Once you have many
partitions on EBS it will be mostly random.
Interesting to monitor random vs sequential...

We tested kafka on ST1 with 1xx partitions on each EBS and it was
constantly lagging.

BR

On Tue, Apr 7, 2020 at 10:06 AM Soumyajit Sahu 
wrote:

> Our typical IOPS stays at ~10K write ops/min, but it goes to 37K write
> ops/min (which is where AWS throttles).
> The spike in write ops isn't accompanied by any spike in write throughput
> or produce requests (except for the first few minutes of catch up). The
> write ops spike stays up (persistently for an hour or two) until we stop
> the broker ec2 instance for about 30 mins and then start it back.
>
> @Liam, no, we are not using log compaction except for a few consumer offset
> topics and config topic (for Kafka Connect), and schema registry store.
>
> @Suman, are you using m5 or r5 instances. Recently, we migrated from r5 to
> m5, and I wonder if that has a hand in this.
>
> We have about 1000 partitions residing on each disk, but I don't think that
> matters as most of the time the brokers run flawlessly (even during peak
> traffic hours).
>
> Thanks!
>
> On Mon, Apr 6, 2020 at 11:39 PM Suman B N  wrote:
>
> > We too have a similar setup but we never observed any such spikes.
> >
> > Are you sure your disk IOPS is good enough? Check if that is throttling.
> >
> > After a broker restarts, there might be more traffic as well because of
> > followers trying to catch up with the leader.
> >
> > -Suman
> >
> > On Tue, Apr 7, 2020 at 11:59 AM Soumyajit Sahu  >
> > wrote:
> >
> > > We are running Kafka on AWS EC2 instances (m5.2xlarge) with mounted EBS
> > st1
> > > volume (one on each machine).
> > > Occasionally, we have noticed that the write ops/second goes through
> the
> > > roof and we get throttled by AWS while the data throughput wouldn't
> have
> > > changed much. As far as our observation goes, it happens usually after
> a
> > > broker restart.
> > >
> > > Has anyone else come across this behavior?
> > >
> > > Thanks!
> > >
> >
> >
> > --
> > *Suman*
> > *OlaCabs*
> >
>


-- 
Seva Feldman
VP R&D Mobile Delivery
[image: ironSource] 

email sev...@ironsrc.com
mobile +972544346089

ironSource HQ - 121 Derech Menachem Begin st. Tel Aviv

Re: High write operations rate on disk

[Suman B N]

We have used st1 volumes and we never saw any issue.
Yes, we are using m-series. Even t-series worked for us :D

During those spikes, do you observe any background operations going on?
Check server logs, controller logs.

On Tue, Apr 7, 2020 at 12:49 PM Seva Feldman  wrote:

> ST1 EBS fit only for sequential rights and reads. Once you have many
> partitions on EBS it will be mostly random.
> Interesting to monitor random vs sequential...
>
> We tested kafka on ST1 with 1xx partitions on each EBS and it was
> constantly lagging.
>
> BR
>
> On Tue, Apr 7, 2020 at 10:06 AM Soumyajit Sahu 
> wrote:
>
> > Our typical IOPS stays at ~10K write ops/min, but it goes to 37K write
> > ops/min (which is where AWS throttles).
> > The spike in write ops isn't accompanied by any spike in write throughput
> > or produce requests (except for the first few minutes of catch up). The
> > write ops spike stays up (persistently for an hour or two) until we stop
> > the broker ec2 instance for about 30 mins and then start it back.
> >
> > @Liam, no, we are not using log compaction except for a few consumer
> offset
> > topics and config topic (for Kafka Connect), and schema registry store.
> >
> > @Suman, are you using m5 or r5 instances. Recently, we migrated from r5
> to
> > m5, and I wonder if that has a hand in this.
> >
> > We have about 1000 partitions residing on each disk, but I don't think
> that
> > matters as most of the time the brokers run flawlessly (even during peak
> > traffic hours).
> >
> > Thanks!
> >
> > On Mon, Apr 6, 2020 at 11:39 PM Suman B N  wrote:
> >
> > > We too have a similar setup but we never observed any such spikes.
> > >
> > > Are you sure your disk IOPS is good enough? Check if that is
> throttling.
> > >
> > > After a broker restarts, there might be more traffic as well because of
> > > followers trying to catch up with the leader.
> > >
> > > -Suman
> > >
> > > On Tue, Apr 7, 2020 at 11:59 AM Soumyajit Sahu <
> soumyajit.s...@gmail.com
> > >
> > > wrote:
> > >
> > > > We are running Kafka on AWS EC2 instances (m5.2xlarge) with mounted
> EBS
> > > st1
> > > > volume (one on each machine).
> > > > Occasionally, we have noticed that the write ops/second goes through
> > the
> > > > roof and we get throttled by AWS while the data throughput wouldn't
> > have
> > > > changed much. As far as our observation goes, it happens usually
> after
> > a
> > > > broker restart.
> > > >
> > > > Has anyone else come across this behavior?
> > > >
> > > > Thanks!
> > > >
> > >
> > >
> > > --
> > > *Suman*
> > > *OlaCabs*
> > >
> >
>
>
> --
> Seva Feldman
> VP R&D Mobile Delivery
> [image: ironSource] 
>
> email sev...@ironsrc.com
> mobile +972544346089
>
> ironSource HQ - 121 Derech Menachem Begin st. Tel Aviv
>


-- 
*Suman*
*OlaCabs*

Re: High write operations rate on disk

[Soumyajit Sahu]

@Suman, thanks for confirming. I will dig more then. The instances are
dedicated to running Kafka, and so is the mounted volume.

@Seva, thanks for the insight. I guess if nothing works, then we will move
from st1 to gp2 volumes.

On Tue, Apr 7, 2020 at 12:28 AM Suman B N  wrote:

> We have used st1 volumes and we never saw any issue.
> Yes, we are using m-series. Even t-series worked for us :D
>
> During those spikes, do you observe any background operations going on?
> Check server logs, controller logs.
>
> On Tue, Apr 7, 2020 at 12:49 PM Seva Feldman  wrote:
>
> > ST1 EBS fit only for sequential rights and reads. Once you have many
> > partitions on EBS it will be mostly random.
> > Interesting to monitor random vs sequential...
> >
> > We tested kafka on ST1 with 1xx partitions on each EBS and it was
> > constantly lagging.
> >
> > BR
> >
> > On Tue, Apr 7, 2020 at 10:06 AM Soumyajit Sahu  >
> > wrote:
> >
> > > Our typical IOPS stays at ~10K write ops/min, but it goes to 37K write
> > > ops/min (which is where AWS throttles).
> > > The spike in write ops isn't accompanied by any spike in write
> throughput
> > > or produce requests (except for the first few minutes of catch up). The
> > > write ops spike stays up (persistently for an hour or two) until we
> stop
> > > the broker ec2 instance for about 30 mins and then start it back.
> > >
> > > @Liam, no, we are not using log compaction except for a few consumer
> > offset
> > > topics and config topic (for Kafka Connect), and schema registry store.
> > >
> > > @Suman, are you using m5 or r5 instances. Recently, we migrated from r5
> > to
> > > m5, and I wonder if that has a hand in this.
> > >
> > > We have about 1000 partitions residing on each disk, but I don't think
> > that
> > > matters as most of the time the brokers run flawlessly (even during
> peak
> > > traffic hours).
> > >
> > > Thanks!
> > >
> > > On Mon, Apr 6, 2020 at 11:39 PM Suman B N 
> wrote:
> > >
> > > > We too have a similar setup but we never observed any such spikes.
> > > >
> > > > Are you sure your disk IOPS is good enough? Check if that is
> > throttling.
> > > >
> > > > After a broker restarts, there might be more traffic as well because
> of
> > > > followers trying to catch up with the leader.
> > > >
> > > > -Suman
> > > >
> > > > On Tue, Apr 7, 2020 at 11:59 AM Soumyajit Sahu <
> > soumyajit.s...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > We are running Kafka on AWS EC2 instances (m5.2xlarge) with mounted
> > EBS
> > > > st1
> > > > > volume (one on each machine).
> > > > > Occasionally, we have noticed that the write ops/second goes
> through
> > > the
> > > > > roof and we get throttled by AWS while the data throughput wouldn't
> > > have
> > > > > changed much. As far as our observation goes, it happens usually
> > after
> > > a
> > > > > broker restart.
> > > > >
> > > > > Has anyone else come across this behavior?
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > *Suman*
> > > > *OlaCabs*
> > > >
> > >
> >
> >
> > --
> > Seva Feldman
> > VP R&D Mobile Delivery
> > [image: ironSource] 
> >
> > email sev...@ironsrc.com
> > mobile +972544346089
> >
> > ironSource HQ - 121 Derech Menachem Begin st. Tel Aviv
> >
>
>
> --
> *Suman*
> *OlaCabs*
>

Re: High write operations rate on disk

[Seva Feldman]

We are using mainly ephemeral instances like i3en as our pattern is more
fit for it.

On Tue, Apr 7, 2020 at 10:40 AM Soumyajit Sahu 
wrote:

> @Suman, thanks for confirming. I will dig more then. The instances are
> dedicated to running Kafka, and so is the mounted volume.
>
> @Seva, thanks for the insight. I guess if nothing works, then we will move
> from st1 to gp2 volumes.
>
> On Tue, Apr 7, 2020 at 12:28 AM Suman B N  wrote:
>
> > We have used st1 volumes and we never saw any issue.
> > Yes, we are using m-series. Even t-series worked for us :D
> >
> > During those spikes, do you observe any background operations going on?
> > Check server logs, controller logs.
> >
> > On Tue, Apr 7, 2020 at 12:49 PM Seva Feldman  wrote:
> >
> > > ST1 EBS fit only for sequential rights and reads. Once you have many
> > > partitions on EBS it will be mostly random.
> > > Interesting to monitor random vs sequential...
> > >
> > > We tested kafka on ST1 with 1xx partitions on each EBS and it was
> > > constantly lagging.
> > >
> > > BR
> > >
> > > On Tue, Apr 7, 2020 at 10:06 AM Soumyajit Sahu <
> soumyajit.s...@gmail.com
> > >
> > > wrote:
> > >
> > > > Our typical IOPS stays at ~10K write ops/min, but it goes to 37K
> write
> > > > ops/min (which is where AWS throttles).
> > > > The spike in write ops isn't accompanied by any spike in write
> > throughput
> > > > or produce requests (except for the first few minutes of catch up).
> The
> > > > write ops spike stays up (persistently for an hour or two) until we
> > stop
> > > > the broker ec2 instance for about 30 mins and then start it back.
> > > >
> > > > @Liam, no, we are not using log compaction except for a few consumer
> > > offset
> > > > topics and config topic (for Kafka Connect), and schema registry
> store.
> > > >
> > > > @Suman, are you using m5 or r5 instances. Recently, we migrated from
> r5
> > > to
> > > > m5, and I wonder if that has a hand in this.
> > > >
> > > > We have about 1000 partitions residing on each disk, but I don't
> think
> > > that
> > > > matters as most of the time the brokers run flawlessly (even during
> > peak
> > > > traffic hours).
> > > >
> > > > Thanks!
> > > >
> > > > On Mon, Apr 6, 2020 at 11:39 PM Suman B N 
> > wrote:
> > > >
> > > > > We too have a similar setup but we never observed any such spikes.
> > > > >
> > > > > Are you sure your disk IOPS is good enough? Check if that is
> > > throttling.
> > > > >
> > > > > After a broker restarts, there might be more traffic as well
> because
> > of
> > > > > followers trying to catch up with the leader.
> > > > >
> > > > > -Suman
> > > > >
> > > > > On Tue, Apr 7, 2020 at 11:59 AM Soumyajit Sahu <
> > > soumyajit.s...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > We are running Kafka on AWS EC2 instances (m5.2xlarge) with
> mounted
> > > EBS
> > > > > st1
> > > > > > volume (one on each machine).
> > > > > > Occasionally, we have noticed that the write ops/second goes
> > through
> > > > the
> > > > > > roof and we get throttled by AWS while the data throughput
> wouldn't
> > > > have
> > > > > > changed much. As far as our observation goes, it happens usually
> > > after
> > > > a
> > > > > > broker restart.
> > > > > >
> > > > > > Has anyone else come across this behavior?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Suman*
> > > > > *OlaCabs*
> > > > >
> > > >
> > >
> > >
> > > --
> > > Seva Feldman
> > > VP R&D Mobile Delivery
> > > [image: ironSource] 
> > >
> > > email sev...@ironsrc.com
> > > mobile +972544346089
> > >
> > > ironSource HQ - 121 Derech Menachem Begin st. Tel Aviv
> > >
> >
> >
> > --
> > *Suman*
> > *OlaCabs*
> >
>


-- 
Seva Feldman
VP R&D Mobile Delivery
[image: ironSource] 

email sev...@ironsrc.com
mobile +972544346089

ironSource HQ - 121 Derech Menachem Begin st. Tel Aviv

How to set custom properties and message expiration for Kafka topic message.

[Naveen Kumar M]

Hello Friends,

Hope you are all doing good!

I am bit new to Kafka messaging system.

We are using Kafka as messaging broker.

Could someone let me know how to set custom properties and message
expiration while sending message to Kafka topic and how to read customer
properties in consumer end?

Thanks and regards,
Naveen

EOL

[Gaudet, Timothy]

Hello,

I am trying to find the End of Life (EOL) of the current version of Kafka.  I 
found links that showed tables and probably answered the question, but I found 
them to be a bit confusing.  Can someone just give me a version number and the 
EOL for that version?  If there is a set time period for the EOL that would be 
very helpful too.

Timothy Gaudet
Linux Engineer
General Dynamics Information Technology _Intelligence Solututions Div.  (ISD)
US BICES-X and Intelligence Programs
Tampa, FL.
813-826-0110
001282659 BICES
timothy.gau...@gdit.com
timothy.d.gaudet@socom.mil
timothy.gaudet.ctr...@usa.bices.org

RE: Brokers session terminated and removed

[Pradeep Badiger]

Hi All,

Can somebody help me on this issue?

Thanks,
Pradeep V.B.

-Original Message-
From: Pradeep Badiger 
Sent: Monday, March 30, 2020 12:58 PM
To: users@kafka.apache.org
Subject: Brokers session terminated and removed

Hi,

I am running into an issue where the Kafka brokers (0.10.2.1) are getting 
removed from the Zookeeper (3.4.14). Here is the setup.

We have 3 Zookeeper nodes and 3 Kafka nodes in AWS. We are making use of 
auto-scaling group to get the replacement nodes on failures.

When the Zookeeper and Kafka clusters are running, I can see the brokers 
registered in Zookeeper under /brokers/ids path.

I then terminate the leader Zookeeper node and wait for AWS auto-scaling group 
to provide a replacement Zookeeper node. I then check /brokers/ids path to 
confirm if the brokers are still connected.

I then terminate the second Zookeeper node and check for the path when a new 
Zookeeper node comes up. I don't have an issue till here.

When I terminate the third Zookeeper node in the original list of Zookeeper 
nodes and I see that all the Kafka brokers' sessions are terminated and the 
brokers are removed from Zookeeper. The ids under /brokers/ids is empty.

I can see the below logs in one of the Zookeeper nodes when the final Zookeeper 
node is replaced.

2020-03-26 20:29:20,303 [myid:3] - INFO  [SessionTracker:ZooKeeperServer@355] - 
Expiring session 0x10003b973b50016, timeout of 6000ms exceeded
2020-03-26 20:29:20,303 [myid:3] - INFO  [SessionTracker:ZooKeeperServer@355] - 
Expiring session 0x10003b973b5000e, timeout of 6000ms exceeded
2020-03-26 20:29:20,303 [myid:3] - INFO  [SessionTracker:ZooKeeperServer@355] - 
Expiring session 0x30003a126690002, timeout of 6000ms exceeded
2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] - 
Deleting ephemeral node /brokers/ids/1002 for session 0x10003b973b50016
2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] - 
Deleting ephemeral node /brokers/ids/1003 for session 0x10003b973b5000e
2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] - 
Deleting ephemeral node /controller for session 0x30003a126690002
2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] - 
Deleting ephemeral node /brokers/ids/1001 for session 0x30003a126690002

I found a ticket 
KAFKA-5473 within Kafka JIRA that talks about an issue with Zookeeper session 
expiration handling. I am not 100% sure that it is related to the current 
issue. But I do see the same behaviour where the DMC broker knows to renew the 
session with old Zookeeper nodes but not with the new replacement one.

Can anyone help me with this?

Thanks,
Pradeep V.B.

This email and any files transmitted with it are confidential, proprietary and 
intended solely for the individual or entity to whom they are addressed. If you 
have received this email in error please delete it immediately.
This email and any files transmitted with it are confidential, proprietary and 
intended solely for the individual or entity to whom they are addressed. If you 
have received this email in error please delete it immediately.

Adding new connector Jars dynamically in Kafka Connect

[nitin agarwal]

Hi,

I have a use case where new connectors will keep on adding to existing
running Kafka Connect cluster. Is there any way in Kafka Connect to submit
the new connector jar dynamically without restarting the Connect process?

Thanks,
Nitin

Re: EOL

[Guozhang Wang]

Hello Tim,

The following link should be the source of truth for EOL policy:
https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy?


Guozhang

On Tue, Apr 7, 2020 at 5:54 AM Gaudet, Timothy 
wrote:

> Hello,
>
> I am trying to find the End of Life (EOL) of the current version of
> Kafka.  I found links that showed tables and probably answered the
> question, but I found them to be a bit confusing.  Can someone just give me
> a version number and the EOL for that version?  If there is a set time
> period for the EOL that would be very helpful too.
>
> Timothy Gaudet
> Linux Engineer
> General Dynamics Information Technology _Intelligence Solututions Div.
> (ISD)
> US BICES-X and Intelligence Programs
> Tampa, FL.
> 813-826-0110
> 001282659 BICES
> timothy.gau...@gdit.com
> timothy.d.gaudet@socom.mil
> timothy.gaudet.ctr...@usa.bices.org timothy.gaudet.ctr...@usa.bices.org>
>
>
>

-- 
-- Guozhang

RE: EOL

[Gaudet, Timothy]

I saw that and to me it sounded like the EOL was one year.

Timothy Gaudet
Linux Engineer
General Dynamics Information Technology _Intelligence Solututions Div.  (ISD)
US BICES-X and Intelligence Programs
Tampa, FL.
813-826-0110
001282659 BICES
timothy.gau...@gdit.com
timothy.d.gaudet@socom.mil
timothy.gaudet.ctr...@usa.bices.org



-Original Message-
From: Guozhang Wang  
Sent: Tuesday, April 7, 2020 1:46 PM
To: users@kafka.apache.org
Subject: Re: EOL

Hello Tim,

The following link should be the source of truth for EOL policy:
https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy?


Guozhang

On Tue, Apr 7, 2020 at 5:54 AM Gaudet, Timothy 
wrote:

> Hello,
>
> I am trying to find the End of Life (EOL) of the current version of 
> Kafka.  I found links that showed tables and probably answered the 
> question, but I found them to be a bit confusing.  Can someone just 
> give me a version number and the EOL for that version?  If there is a 
> set time period for the EOL that would be very helpful too.
>
> Timothy Gaudet
> Linux Engineer
> General Dynamics Information Technology _Intelligence Solututions Div.
> (ISD)
> US BICES-X and Intelligence Programs
> Tampa, FL.
> 813-826-0110
> 001282659 BICES
> timothy.gau...@gdit.com
> timothy.d.gaudet@socom.mil
> timothy.gaudet.ctr...@usa.bices.org timothy.gaudet.ctr...@usa.bices.org>
>
>
>

--
-- Guozhang

Re: EOL

[Israel Ekpo]

After reviewing the documentation, it appears the document focuses more on
release schedule and bug fixes for the 3 most recent major release
versions. It is not really clear when the support for specific versions end
and when they become deprecated.

We need an official deprecation and EOL policy that is explicit in terms of
how many versions back we are going to support.

We can take a look at these documents from other projects for inspiration
to come up with a clearer guidance to the community

https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions

https://access.redhat.com/articles/1299013#OpenJDK_Life_Cycle

We also have to be careful not to alarm existing users of the 1.x series

Those are my thoughts


On Tue, Apr 7, 2020 at 2:33 PM Gaudet, Timothy 
wrote:

> I saw that and to me it sounded like the EOL was one year.
>
> Timothy Gaudet
> Linux Engineer
> General Dynamics Information Technology _Intelligence Solututions Div.
> (ISD)
> US BICES-X and Intelligence Programs
> Tampa, FL.
> 813-826-0110
> 001282659 BICES
> timothy.gau...@gdit.com
> timothy.d.gaudet@socom.mil
> timothy.gaudet.ctr...@usa.bices.org
>
>
>
> -Original Message-
> From: Guozhang Wang 
> Sent: Tuesday, April 7, 2020 1:46 PM
> To: users@kafka.apache.org
> Subject: Re: EOL
>
> Hello Tim,
>
> The following link should be the source of truth for EOL policy:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan#TimeBasedReleasePlan-WhatIsOurEOLPolicy
> ?
>
>
> Guozhang
>
> On Tue, Apr 7, 2020 at 5:54 AM Gaudet, Timothy 
> wrote:
>
> > Hello,
> >
> > I am trying to find the End of Life (EOL) of the current version of
> > Kafka.  I found links that showed tables and probably answered the
> > question, but I found them to be a bit confusing.  Can someone just
> > give me a version number and the EOL for that version?  If there is a
> > set time period for the EOL that would be very helpful too.
> >
> > Timothy Gaudet
> > Linux Engineer
> > General Dynamics Information Technology _Intelligence Solututions Div.
> > (ISD)
> > US BICES-X and Intelligence Programs
> > Tampa, FL.
> > 813-826-0110
> > 001282659 BICES
> > timothy.gau...@gdit.com
> > timothy.d.gaudet@socom.mil
> > timothy.gaudet.ctr...@usa.bices.org > timothy.gaudet.ctr...@usa.bices.org>
> >
> >
> >
>
> --
> -- Guozhang
>

[VOTE] 2.5.0 RC3

[David Arthur]

Hello Kafka users, developers and client-developers,

This is the forth candidate for release of Apache Kafka 2.5.0.

* TLS 1.3 support (1.2 is now the default)
* Co-groups for Kafka Streams
* Incremental rebalance for Kafka Consumer
* New metrics for better operational insight
* Upgrade Zookeeper to 3.5.7
* Deprecate support for Scala 2.11

Release notes for the 2.5.0 release:
https://home.apache.org/~davidarthur/kafka-2.5.0-rc3/RELEASE_NOTES.html

*** Please download, test and vote by Friday April 10th 5pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~davidarthur/kafka-2.5.0-rc3/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~davidarthur/kafka-2.5.0-rc3/javadoc/

* Tag to be voted upon (off 2.5 branch) is the 2.5.0 tag:
https://github.com/apache/kafka/releases/tag/2.5.0-rc3

* Documentation:
https://kafka.apache.org/25/documentation.html

* Protocol:
https://kafka.apache.org/25/protocol.html

Successful Jenkins builds to follow

Thanks!
David

Oauthbearer - Azure

[Antony Alphonse]

Hi,

I'm looking to implement authentication using Oauthbearer mechanism in my
Kafka cluster. My Oauth server will be Azure AD. If anyone have implemented
similar setup on Azure and can share some tips on how to proceed?

Thanks
AA