Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1279

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-14284) "current-vote" metric is always leaderId under non-election state

[Luke Chen (Jira)]

Luke Chen created KAFKA-14284:
-

 Summary: "current-vote" metric is always leaderId under 
non-election state
 Key: KAFKA-14284
 URL: https://issues.apache.org/jira/browse/KAFKA-14284
 Project: Kafka
  Issue Type: Bug
  Components: kraft
Affects Versions: 3.3.1
Reporter: Luke Chen
Assignee: Luke Chen


The current-vote metric in:
{code:java}
kafka.server:type=raft-metrics,name=current-vote{code}
It is always showing the leader id, even it's not in election state. Compared 
to the "quorum-state" file, the votedId will show -1 if it's not in "candidate" 
or "voted" state, which makes more sense.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re:[DISCUSS] KIP-874: TopicRoundRobinAssignor

[Hector Geraldino (BLOOMBERG/ 919 3RD A)]

Hi Mathieu. I took a look at your KIP and have a couple questions. 

If the goal is to do the partition assignments at a topic level, wouldn't 
having single-partition topics solve this problem? 

You also mentioned that your goal is to minimize the potential of a poison pill 
message breaking all members of a group (by keeping track of which topics have 
'failed'), but it is not clear how this can be achieved with this assignor. If 
we imagine an scenario where:

* A group has 3 members (A, B, C)
* Members are subscribed to 3 topics (T1, T2, T3)
* Each member is assigned one topic (A[T1], B[T2], C[T3])
* One member fails to consume from a topic/partition (B[T2]), and goes into 
failed state 

How will the group leader know that T2 should not be re-assigned on the next 
rebalance? Can you elaborate a bit more on the mechanisms used to communicate 
this state to the other group members?

Thanks

From: dev@kafka.apache.org At: 10/05/22 03:47:33 UTC-4:00To:  
dev@kafka.apache.org
Subject: [DISCUSS] KIP-874: TopicRoundRobinAssignor

Hi Kafka Developers,

My proposal is to add a new partition assignment strategy at the topic
level to :
 - have a better data consistency by consumed topic in case of exception
 - have a solution much thread safe for the consumer
In case there are multiple consumers and multiple topics.

Here is the link to the KIP with all the explanations :
https://cwiki.apache.org/confluence/x/XozGDQ

Thank you in advance for your feedbacks,
Mathieu

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1280

[Apache Jenkins Server]

See 

[jira] [Resolved] (KAFKA-14283) Fix connector creation authorization tests not doing anything

[Matthew de Detrich (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthew de Detrich resolved KAFKA-14283.

Resolution: Fixed

> Fix connector creation authorization tests not doing anything
> -
>
> Key: KAFKA-14283
> URL: https://issues.apache.org/jira/browse/KAFKA-14283
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Reporter: Matthew de Detrich
>Assignee: Matthew de Detrich
>Priority: Major
>
> Currently the testCreateConnectorWithoutHeaderAuthorization and 
> testCreateConnectorWithHeaderAuthorization tests within 
> ConnectorsResourceTest aren't actually anything. This is because in reality 
> the requests should be forwarded to a leader and tests aren't actually 
> testing that a leader RestClient request is made



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[DISCUSS] KIP-875: First-class offsets support in Kafka Connect

[Chris Egerton]

Hi all,

I'd like to begin discussion on a KIP to add offsets support to the Kafka
Connect REST API:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect

Cheers,

Chris

Re: Permission Request - contribute to Kafka

[Chris Egerton]

Hi Paresh,

You should be good to go now.

Cheers,

Chris

On Thu, Oct 6, 2022 at 10:45 PM Paresh Thatte  wrote:

> Contributor Permission request for Kafka.
>
>- *Wiki-ID*: pat70
>- *Jira-ID*: pat70
>

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1281

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-14285) Delete quota node in zookeeper when configs are empty

[Junyang Liu (Jira)]

Junyang Liu created KAFKA-14285:
---

 Summary: Delete quota node in zookeeper when configs are empty
 Key: KAFKA-14285
 URL: https://issues.apache.org/jira/browse/KAFKA-14285
 Project: Kafka
  Issue Type: Improvement
  Components: core
Affects Versions: 3.3.1, 2.7.2
Reporter: Junyang Liu


When configuring flow control quota for users and clients, Kafka will add a 
child node to /config/users or /config/clients in zookeeper. When deleting a 
flow control quota config, the existing node in zookeeper will not be deleted 
even if the configs in the node are empty. So if a client repeatedly configure 
and delete flow control quota for different users and clients, the node in 
zookeeper will increase continuously, which may leads to severe problems such 
as memory leak.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re:Re: Delete quota node in zookeeper when configs are empty

[Junyang Liu]

Hi,
Thank you for your advice.
I have opened a JIRA for this improvement: 
https://issues.apache.org/jira/browse/KAFKA-14285
And I have submitted a PR for it: 
https://github.com/apache/kafka/pull/12727


Thank you.
-- Junyang Liu

At 2022-09-21 11:24:31, "Luke Chen"  wrote:
>Hi Junyang,
>
>Could you open a JIRA for this improvement?
>Also, if you're interested, welcome to submit a PR for it.
>
>Thank you.
>Luke
>
>On Wed, Sep 21, 2022 at 2:08 AM Junyang Liu  wrote:
>
>> Hi,
>>
>>
>> When configuring flow control quota for users and clients, Kafka will add
>> a child node to /config/users or /config/clients in zookeeper. When
>> deleting a flow control quota config, the existing node in zookeeper will
>> not be deleted even if the configs in the node are empty. So if a client
>> repeatedly configure and delete flow control quota for different users and
>> clients, the node in zookeeper will increase continuously, which may leads
>> to severe problems such as memory leak.
>>
>>
>> I suggest that we can delete the node in zookeeper (child nodes under
>> /config/users and /config/clients) if the configs in the node are empty.
>>
>>
>> Thanks,
>> -- Junyang Liu

Re: [DISCUSS] Apache Kafka 3.4.0 release

[Ismael Juma]

Thanks for volunteering Sophie!

Ismael

On Wed, Oct 5, 2022 at 3:01 PM Sophie Blee-Goldman
 wrote:

> Hey all,
>
> I'd like to volunteer as release manager for the next feature release,
> which will be Apache
> Kafka 3.4.0. If that sounds good to everyone I'll update this thread with
> the release plan in the coming week.
>
> Cheers,
> A. Sophie Blee-Goldman
>

Re: [DISCUSS] Apache Kafka 3.4.0 release

[Israel Ekpo]

Thanks for volunteering Sophie



On Fri, Oct 7, 2022 at 4:17 PM Ismael Juma  wrote:

> Thanks for volunteering Sophie!
>
> Ismael
>
> On Wed, Oct 5, 2022 at 3:01 PM Sophie Blee-Goldman
>  wrote:
>
> > Hey all,
> >
> > I'd like to volunteer as release manager for the next feature release,
> > which will be Apache
> > Kafka 3.4.0. If that sounds good to everyone I'll update this thread with
> > the release plan in the coming week.
> >
> > Cheers,
> > A. Sophie Blee-Goldman
> >
>
-- 
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/

[jira] [Created] (KAFKA-14286) Time based cluster metadata snapshots

[Jira]

José Armando García Sancio created KAFKA-14286:
--

 Summary: Time based cluster metadata snapshots
 Key: KAFKA-14286
 URL: https://issues.apache.org/jira/browse/KAFKA-14286
 Project: Kafka
  Issue Type: New Feature
Reporter: José Armando García Sancio
Assignee: José Armando García Sancio






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1282

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 499152 lines...]
[2022-10-07T21:07:36.910Z] 
[2022-10-07T21:07:36.910Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > AdjustStreamThreadCountTest > shouldRemoveStreamThread() 
PASSED
[2022-10-07T21:07:36.910Z] 
[2022-10-07T21:07:36.910Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > AdjustStreamThreadCountTest > 
shouldResizeCacheAfterThreadRemovalTimesOut() STARTED
[2022-10-07T21:07:37.582Z] 
[2022-10-07T21:07:37.582Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) STARTED
[2022-10-07T21:07:38.058Z] 
[2022-10-07T21:07:38.058Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > AdjustStreamThreadCountTest > 
shouldResizeCacheAfterThreadRemovalTimesOut() PASSED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldOnlyReadRecordsWhereEarliestSpecifiedWithNoCommittedOffsetsWithGlobalAutoOffsetResetLatest()
 STARTED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldOnlyReadRecordsWhereEarliestSpecifiedWithNoCommittedOffsetsWithGlobalAutoOffsetResetLatest()
 PASSED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldThrowExceptionOverlappingPattern() STARTED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldThrowExceptionOverlappingPattern() PASSED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldThrowExceptionOverlappingTopic() STARTED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldThrowExceptionOverlappingTopic() PASSED
[2022-10-07T21:07:43.477Z] 
[2022-10-07T21:07:43.477Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 166 > FineGrainedAutoResetIntegrationTest > 
shouldOnlyReadRecordsWhereEarliestSpecifiedWithInvalidCommittedOffsets() STARTED
[2022-10-07T21:07:43.988Z] 
[2022-10-07T21:07:43.988Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldAggregateSlidingWindows(TestInfo) PASSED
[2022-10-07T21:07:43.988Z] 
[2022-10-07T21:07:43.988Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldReduceSessionWindows() STARTED
[2022-10-07T21:07:45.020Z] 
[2022-10-07T21:07:45.020Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldReduceSessionWindows() PASSED
[2022-10-07T21:07:45.020Z] 
[2022-10-07T21:07:45.020Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldReduceSlidingWindows(TestInfo) STARTED
[2022-10-07T21:07:48.269Z] 
[2022-10-07T21:07:48.269Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldReduceSlidingWindows(TestInfo) PASSED
[2022-10-07T21:07:48.269Z] 
[2022-10-07T21:07:48.269Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > shouldReduce(TestInfo) 
STARTED
[2022-10-07T21:07:52.655Z] 
[2022-10-07T21:07:52.656Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > shouldReduce(TestInfo) 
PASSED
[2022-10-07T21:07:52.656Z] 
[2022-10-07T21:07:52.656Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldAggregate(TestInfo) STARTED
[2022-10-07T21:07:56.898Z] 
[2022-10-07T21:07:56.898Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldAggregate(TestInfo) PASSED
[2022-10-07T21:07:56.898Z] 
[2022-10-07T21:07:56.898Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > shouldCount(TestInfo) 
STARTED
[2022-10-07T21:08:00.951Z] 
[2022-10-07T21:08:00.951Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > shouldCount(TestInfo) 
PASSED
[2022-10-07T21:08:00.951Z] 
[2022-10-07T21:08:00.951Z] Gradle Test Run :streams:integrationTest > Gradle 
Test Executor 163 > KStreamAggregationIntegrationTest > 
shouldGro

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1283

[Apache Jenkins Server]

See