Thanks for your all your hard work Colin!
--
Igor
Congratulations Jeff!
--
Igor
Congratulations Josep!
--
Igor
Igor Soarez created KAFKA-17467:
---
Summary: Flaky test shouldStayDeadAfterTwoCloses
org.apache.kafka.streams.processor.internals.GlobalStreamThreadTest
Key: KAFKA-17467
URL: https://issues.apache.org/jira/browse
Congratulations Lianet!
--
Igor
Hi Omnia,
Thanks for this KIP, this is a valuable improvement, and
apologies for my late feedback on this.
IS1: Without an indication of how many pages or items are left,
it is impossible for the client to calculate progress.
Having the cursor in the response just indicates whether there
are more
My understanding was that the reason for the shorter cycle
to the 3.9 release was based on the assumption that KIP-1012
would be ready soon, so we could get to 4.0 quicker.
If we can't move to 4.0 sooner, what's to gain with an early 3.9?
--
Igor
Hi Josep,
That makes sense to me, thanks for clarifying.
+1 non-binding from me then.
--
Igor
Hi Josep,
Thank you once again for running this release.
I noticed that site-docs/upgrade.html in kafka_2.13-3.8.0-site-docs.tgz
does not include the corrections from
https://github.com/apache/kafka-site/pull/614
namely the replacement of version 3.6 with 3.7 in the upgrade notes header,
and the
[
https://issues.apache.org/jira/browse/KAFKA-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-15368.
-
Resolution: Fixed
This was done as part of https://github.com/apache/kafka/pull/15394
> T
[
https://issues.apache.org/jira/browse/KAFKA-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-15359.
-
Resolution: Fixed
> Support log.dir.failure.timeout.ms configuration for J
Hi Josep,
Thanks for running this release.
+1 non-binding.
I performed the following checks:
- Verified signatures and checksums
- Built from source
- Basic workload against a KRaft cluster using the artifacts for Scala 2.13
- Browsed Javadoc
- Checked LICENSE
- Ran and cecked Docker image con
Hi Josep,
Thanks for running this release.
+1 non-binding. I performed the following checks:
- Verified signatures and checksums
- Built from source
- Basic workload against a KRaft cluster using the artifacts for both Scala
versions
- Browsed Javadoc
- Checked LICENSE
Best,
--
Igor
Hi all,
I had left a comment on the JIRA earlier, and thanks to Mickael I was made
aware of this thread, so I'll share it here too:
I agree it makes sense to make this information available through RPCs,
but the Quorum is probably not the right place to represent registered (but
inactive
Hi Manikumar,
Thanks for pointing this out. The apache/kafka:latest has now been updated from
apache/kafka:3.7.1.
Best,
--
Igor
A correction to a funny mistake in my previous message:
That's *61 contributors* to this release, not 1086!
--
Igor Soarez
On Mon, Jul 1, 2024, at 11:25 AM, Igor Soarez wrote:
> The Apache Kafka community is pleased to announce the release for Apache
> Kafka 3.7.1
>
>
Zhang, Chia-Ping Tsai, Chris
Egerton, Colin Patrick McCabe, David Arthur, David Jacot, Divij Vaidya, Dmitry
Werner, Edoardo Comar, flashmouse, Florin Akermann, Gantigmaa Selenge, Gaurav
Narula, Greg Harris, Igor Soarez, ilyazr, Ismael Juma, Jason Gustafson, Jeff
Kim, jiangyuan, Joel Hamill, John Yu
Hi Luke, I was just waiting for "please vote by" date in the first email.
Hi all,
Thank you all for having a look at this RC and voting.
I'm now closing the vote.
The vote passes with:
- 3 +1 binding votes from Mickael, Luke and Justine
- 1 +1 non-binding votes from Jakub
- 0 -1 votes
I'll mov
**
Thanks,
--
Igor Soarez
[
https://issues.apache.org/jira/browse/KAFKA-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16961.
-
Resolution: Fixed
Verified test no longer failing with
{code:java}
TC_PATHS="tests/kafk
Hi Edoardo,
It is late, but not too late. I have cherry-picked your change
to the 3.7 branch and I'll build a second release candidate.
If you could have a look at the first RC, please let me know if
you spot any issues with it that can be avoided in the next RC.
Thanks,
--
Igor
Igor Soarez created KAFKA-16941:
---
Summary: Flaky test - testDynamicBrokerConfigUpdateUsingKraft [1]
Type=Raft-Combined, MetadataVersion=4.0-IV0,Security=PLAINTEXT –
kafka.admin.ConfigCommandIntegrationTest
Key: KAFKA-16941
Igor Soarez created KAFKA-16940:
---
Summary: Flaky test
"testNoConsumeWithDescribeAclViaSubscribe(String).quorum=kraft" –
kafka.api.DelegationTokenEndToEndAuthorizationWithOwnerTest
Key: KAFKA-16940
Igor Soarez created KAFKA-16934:
---
Summary: Clean up and refactor release.py
Key: KAFKA-16934
URL: https://issues.apache.org/jira/browse/KAFKA-16934
Project: Kafka
Issue Type: Improvement
Now attaching the container build reports, which I seem to have forgotten to
include before.
--
Igor
kafka/test:test (alpine 3.19.1)
===
Total: 0 (HIGH: 0, CRITICAL: 0)
3.7 branch:
Docker Build Test Pipeline:
https://github.com/apache/kafka/actions/runs/9455339546
/**
Thanks,
--
Igor Soarez
Igor Soarez created KAFKA-16920:
---
Summary: Flaky test testControlledShutdown() -
kafka.server.BrokerLifecycleManagerTest
Key: KAFKA-16920
URL: https://issues.apache.org/jira/browse/KAFKA-16920
Project
Igor Soarez created KAFKA-16919:
---
Summary: Flaky test testNoCheckpointsIfNoRecordsAreMirrored() –
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest
Key: KAFKA-16919
URL: https
[
https://issues.apache.org/jira/browse/KAFKA-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16583.
-
Resolution: Fixed
> Update from 3.4.0 to 3.7.0 image write failed in Kraft m
[
https://issues.apache.org/jira/browse/KAFKA-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-16886:
-
> KRaft partition reassignment failed after upgrade to 3.
[
https://issues.apache.org/jira/browse/KAFKA-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16886.
-
Resolution: Fixed
> KRaft partition reassignment failed after upgrade to 3.
Hi Justine,
I'm sorry this release is delayed. A few new blockers have come up and we're
working through them.
Here's the release plan:
https://cwiki.apache.org/confluence/display/KAFKA/Release+plan+3.7.1
Best,
--
Igor
[
https://issues.apache.org/jira/browse/KAFKA-16757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16757.
-
Resolution: Fixed
> Fix broker re-registration issues around MV 3.7-
Hi Claude,
MappedByteBuffer is the fastest, but allows for this global pause.
RandomAccessFile or FileChannel (without .map) will be slower, but involve a
syscall for IO. Because threads get marked safe before entering a syscall, any
delays in the operation affect just that thread, not the whol
Hi Claude,
Thanks for writing this KIP. This issue seems particularly
thorny, and I appreciate everyone's effort to address this.
I want to share my concern with the KIP's proposal of the
use of memory mapped files – mmap is Java's achilles heel,
Kafka should make less use of it, not more.
The J
[
https://issues.apache.org/jira/browse/KAFKA-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16645.
-
Resolution: Resolved
> CVEs in 3.7.0 docker im
[
https://issues.apache.org/jira/browse/KAFKA-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-16645:
-
Need to re-open to change the resolution, release_notes.py doesn't like the one
I picked
&
[
https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-16692:
-
Re-opening as 3.6 backport is still missing
> InvalidRequestException: ADD_PARTITIONS_TO_TXN w
[
https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16692.
-
Resolution: Fixed
> InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is
[
https://issues.apache.org/jira/browse/KAFKA-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16645.
-
Assignee: Igor Soarez
Resolution: Won't Fix
The vulnerability has already been addr
[
https://issues.apache.org/jira/browse/KAFKA-16688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16688.
-
Resolution: Fixed
> SystemTimer leaks resources on cl
[
https://issues.apache.org/jira/browse/KAFKA-16624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16624.
-
Resolution: Fixed
> Don't generate useless PartitionChangeRecord on
Hi Omnia, Hi Claude,
Thanks for putting this KIP together.
This is an important unresolved issue in Kafka,
which I have witnessed several times in production.
Please see my questions below:
10 Given the goal is to prevent OOMs, do we also need to
limit the number of KafkaPrincipals in use?
11.
Igor Soarez created KAFKA-16636:
---
Summary: Flaky test - testStickyTaskAssignorLargePartitionCount –
org.apache.kafka.streams.processor.internals.StreamsAssignmentScaleTest
Key: KAFKA-16636
URL: https
Igor Soarez created KAFKA-16635:
---
Summary: Flaky test
"shouldThrottleOldSegments(String).quorum=kraft" –
kafka.server.ReplicationQuotasTest
Key: KAFKA-16635
URL: https://issues.apache.org/jira/browse/K
Igor Soarez created KAFKA-16634:
---
Summary: Flaky test - testFenceMultipleBrokers() –
org.apache.kafka.controller.QuorumControllerTest
Key: KAFKA-16634
URL: https://issues.apache.org/jira/browse/KAFKA-16634
Igor Soarez created KAFKA-16633:
---
Summary: Flaky test -
testDescribeExistingGroupWithNoMembers(String,
String).quorum=kraft+kip848.groupProtocol=consumer –
org.apache.kafka.tools.consumer.group.DescribeConsumerGroupTest
Key
Igor Soarez created KAFKA-16632:
---
Summary: Flaky test
testDeleteOffsetsOfStableConsumerGroupWithTopicPartition [1]
Type=Raft-Isolated, MetadataVersion=3.8-IV0, Security=PLAINTEXT
Igor Soarez created KAFKA-16631:
---
Summary: Flaky test -
testDeleteOffsetsOfStableConsumerGroupWithTopicOnly [1] Type=Raft-Isolated,
MetadataVersion=3.8-IV0, Security=PLAINTEXT
Igor Soarez created KAFKA-16630:
---
Summary: Flaky test
"testPollReturnsRecords(GroupProtocol).groupProtocol=CLASSIC" –
org.apache.kafka.clients.consumer.KafkaConsumerTest
Key: KAFKA-16630
Thanks everyone, I'm very honoured to join!
--
Igor
Hi everyone,
I'd like to volunteer to be the release manager for a 3.7.1 release.
Please keep in mind, this would be my first release, so I might have some
questions,
and it might also take me a bit longer to work through the release process.
So I'm thinking a good target would be toward the end
[
https://issues.apache.org/jira/browse/KAFKA-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16610.
-
Resolution: Resolved
> Replace "Map#entrySet#forEach" by
[
https://issues.apache.org/jira/browse/KAFKA-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-16606:
-
Assignee: Igor Soarez
> JBOD support in KRaft does not seem to be gated by the metad
Hi Omnia,
Thanks for your answers, and I see you've updated the KIP so thanks for the
changes too.
+1 (binding), thanks for the KIP
--
Igor
Igor Soarez created KAFKA-16602:
---
Summary: Flaky test –
org.apache.kafka.controller.QuorumControllerTest.testBootstrapZkMigrationRecord()
Key: KAFKA-16602
URL: https://issues.apache.org/jira/browse/KAFKA-16602
Igor Soarez created KAFKA-16601:
---
Summary: Flaky test –
org.apache.kafka.controller.QuorumControllerMetricsIntegrationTest.testClosingQuorumControllerClosesMetrics()
Key: KAFKA-16601
URL: https://issues.apache.org
Igor Soarez created KAFKA-16597:
---
Summary: Flaky test -
org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQuerySpecificStalePartitionStoresMultiStreamThreads()
Key: KAFKA-16597
URL: https
Igor Soarez created KAFKA-16596:
---
Summary: Flaky test –
org.apache.kafka.clients.ClientUtilsTest.testParseAndValidateAddressesWithReverseLookup()
Key: KAFKA-16596
URL: https://issues.apache.org/jira/browse/KAFKA
Hi Omnia,
Thanks for this KIP.
11. These seem to me to be small misspellings, please double-check:
s/MM2 main features/MM2's main features
s/syncing consumer group offset/syncing consumer group offsets
s/relays/relies
s/recored's offset/recorded offsets
s/clusters without need for/clusters withou
[
https://issues.apache.org/jira/browse/KAFKA-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-15793:
-
This has come up again:
{code:java}
[2024-04-09T21:06:17.307Z] Gradle Test Run :core:test
Igor Soarez created KAFKA-16504:
---
Summary: Flaky test
org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations
Key: KAFKA-16504
URL: https://issues.apache.org/jira/browse/KAFKA-16504
[
https://issues.apache.org/jira/browse/KAFKA-16403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16403.
-
Resolution: Not A Bug
> Flaky t
[
https://issues.apache.org/jira/browse/KAFKA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez resolved KAFKA-16404.
-
Resolution: Not A Bug
Same as KAFKA-16403, this only failed once. It was likely the result of a
Igor Soarez created KAFKA-16422:
---
Summary: Flaky test
org.apache.kafka.controller.QuorumControllerMetricsIntegrationTest."testFailingOverIncrementsNewActiveControllerCount(boolean).true"
Key: KAFKA-16422
Igor Soarez created KAFKA-16404:
---
Summary: Flaky test
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testGetStreamsConfig
Key: KAFKA-16404
URL: https://issues.apache.org/jira/browse/KAFKA-16404
Igor Soarez created KAFKA-16403:
---
Summary: Flaky test
org.apache.kafka.streams.examples.wordcount.WordCountDemoTest.testCountListOfWords
Key: KAFKA-16403
URL: https://issues.apache.org/jira/browse/KAFKA-16403
Igor Soarez created KAFKA-16402:
---
Summary: Flaky test
org.apache.kafka.controller.QuorumControllerTest.testSnapshotSaveAndLoad
Key: KAFKA-16402
URL: https://issues.apache.org/jira/browse/KAFKA-16402
Igor Soarez created KAFKA-16365:
---
Summary: AssignmentsManager mismanages completion notifications
Key: KAFKA-16365
URL: https://issues.apache.org/jira/browse/KAFKA-16365
Project: Kafka
Issue
Igor Soarez created KAFKA-16363:
---
Summary: Storage crashes if dir is unavailable
Key: KAFKA-16363
URL: https://issues.apache.org/jira/browse/KAFKA-16363
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-16297:
---
Summary: Race condition while promoting future replica can lead to
partition unavailability.
Key: KAFKA-16297
URL: https://issues.apache.org/jira/browse/KAFKA-16297
t; API. Would it make sense to authorize these requests as other inter-broker
> protocol calls are usually authorized, that is ClusterAction on Cluster
> resource?
>
> Thanks,
> Viktor
>
> On Tue, Nov 28, 2023 at 4:18 PM Igor Soarez wrote:
>
> > Hi everyone,
> &g
Igor Soarez created KAFKA-15955:
---
Summary: Migrating ZK brokers send dir assignments
Key: KAFKA-15955
URL: https://issues.apache.org/jira/browse/KAFKA-15955
Project: Kafka
Issue Type: Sub-task
y is
offline is now more of an open question. It's unclear if this will
actually be necessary.
Please share if you have any thoughts.
Best,
--
Igor
On Tue, Oct 10, 2023, at 5:28 AM, Igor Soarez wrote:
> Hi Colin,
>
> Thanks for the renaming suggestions. UNASSIGNED is better then
Igor Soarez created KAFKA-15893:
---
Summary: Bump MetadataVersion for directory assignments
Key: KAFKA-15893
URL: https://issues.apache.org/jira/browse/KAFKA-15893
Project: Kafka
Issue Type: Sub
Igor Soarez created KAFKA-15886:
---
Summary: Always specify directories for new partition registrations
Key: KAFKA-15886
URL: https://issues.apache.org/jira/browse/KAFKA-15886
Project: Kafka
Igor Soarez created KAFKA-15858:
---
Summary: Broker stays fenced until all assignments are correct
Key: KAFKA-15858
URL: https://issues.apache.org/jira/browse/KAFKA-15858
Project: Kafka
Issue
Hi all,
I think at least one of those is my fault, apologies.
I'll try to make sure all my tests are passing from now on.
It doesn't help that GitHub always shows that the tests have failed,
even when they have not. I suspect this is because Jenkins always
marks the builds as unstable, even when
Igor Soarez created KAFKA-15650:
---
Summary: Data-loss on leader shutdown right after partition
creation?
Key: KAFKA-15650
URL: https://issues.apache.org/jira/browse/KAFKA-15650
Project: Kafka
Igor Soarez created KAFKA-15649:
---
Summary: Handle directory failure timeout
Key: KAFKA-15649
URL: https://issues.apache.org/jira/browse/KAFKA-15649
Project: Kafka
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/KAFKA-15355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Soarez reopened KAFKA-15355:
-
closed by mistake
> Message schema changes
> --
>
>
Hi Colin,
> I would call #2 LOST. It was assigned in the past, but we don't know where.
> I see that you called this OFFLINE). This is not really normal...
> it should happen only when we're migrating from ZK mode to KRaft mode,
> or going from an older KRaft release with multiple directories to a
Hi David,
Thanks for shedding light on migration goals, makes sense.
Your preference for option a) makes it even more attractive.
We'll keep that as the preferred approach, thanks for the advice.
> One question with this approach is how the KRaft controller learns about
> the multiple log directo
Hi everyone,
Earlier today Colin, Ron, Proven and I had a chat about this work.
We discussed several aspects which I’d like to share here.
## A new reserved UUID
We'll reserve a third UUID to indicate an unspecified dir,
but one that is known to be selected. As opposed to the
default UNKNOWN_DIR
Hi everyone,
After a conversation with Colin McCabe and Proven Provenzano yesterday,
we decided that the benefits outweigh the concerns with the overhead
of associating a directory UUID to every replica in the metadata
partition records.
i.e. We prefer to always associate the log dir UUID even whe
Igor Soarez created KAFKA-15514:
---
Summary: Controller-side replica management changes
Key: KAFKA-15514
URL: https://issues.apache.org/jira/browse/KAFKA-15514
Project: Kafka
Issue Type: Sub
Hi Ron,
I think we can generalize the deconfigured directory scenario
in your last question to address this situation too.
When handling a broker registration request, the controller
can check if OfflineLogDirs=false and any UUIDs are missing
in OnlineLogDirs, compared with the previous registrat
Hi everyone,
I believe we can close this voting thread now, as there
were three +1 binding votes from Ziming, Mickael and Ron.
With that, this vote passes.
Thanks to everyone who participated in reviewing,
and/or taking the time to vote on this KIP!
Best,
--
Igor
023 at 10:04 AM Ron Dagostino wrote:
> >
> > Ok, great, that makes sense, Igor. Thanks. +1 (binding) on the KIP from
> > me.
> >
> > Ron
> >
> > > On Sep 13, 2023, at 11:58 AM, Igor Soarez
> > > wrote:
> > >
> > > Hi R
Hi Ron,
Thanks for drilling down on this. I think the KIP isn't really clear here,
and the metadata caching section you quoted needs clarification.
The "hosting broker's latest registration" refers to the previous,
not the current registration. The registrations are only compared by
the controlle
Hi Ron,
Thank you for having a look a this KIP.
Indeed, the log directory UUID should always be generated
and loaded. I've have corrected the wording in the KIP to clarify.
It is a bit of a pain to replace the field, but I agree that is
the best approach for the same reason you pointed out.
I h
Hi Ziming,
Thank you for having a look and taking the time to vote.
I have already opened some PRs, see:
https://issues.apache.org/jira/browse/KAFKA-14127
Best,
--
Igor
Igor Soarez created KAFKA-15451:
---
Summary: Include offline dirs in BrokerHeartbeatRequest
Key: KAFKA-15451
URL: https://issues.apache.org/jira/browse/KAFKA-15451
Project: Kafka
Issue Type: Sub
Igor Soarez created KAFKA-15426:
---
Summary: Process and persist directory assignments
Key: KAFKA-15426
URL: https://issues.apache.org/jira/browse/KAFKA-15426
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-15368:
---
Summary: Test ZK JBOD to KRaft migration
Key: KAFKA-15368
URL: https://issues.apache.org/jira/browse/KAFKA-15368
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-15367:
---
Summary: Test KRaft JBOD enabling migration
Key: KAFKA-15367
URL: https://issues.apache.org/jira/browse/KAFKA-15367
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-15366:
---
Summary: Log directory failure integration test
Key: KAFKA-15366
URL: https://issues.apache.org/jira/browse/KAFKA-15366
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-15365:
---
Summary: Replica management changes
Key: KAFKA-15365
URL: https://issues.apache.org/jira/browse/KAFKA-15365
Project: Kafka
Issue Type: Sub-task
Igor Soarez created KAFKA-15364:
---
Summary: Handle log directory failure in the Controller
Key: KAFKA-15364
URL: https://issues.apache.org/jira/browse/KAFKA-15364
Project: Kafka
Issue Type: Sub
Igor Soarez created KAFKA-15363:
---
Summary: Broker log directory failure changes
Key: KAFKA-15363
URL: https://issues.apache.org/jira/browse/KAFKA-15363
Project: Kafka
Issue Type: Sub-task
1 - 100 of 183 matches
Mail list logo
