[jira] [Resolved] (KAFKA-15502) Handle large keystores in SslEngineValidator

[Manikumar (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar resolved KAFKA-15502.
---
Fix Version/s: 3.4.2
   3.5.2
   3.7.0
   3.6.1
   Resolution: Fixed

> Handle large keystores in SslEngineValidator
> 
>
> Key: KAFKA-15502
> URL: https://issues.apache.org/jira/browse/KAFKA-15502
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.6.0
>Reporter: Manikumar
>Assignee: Manikumar
>Priority: Major
> Fix For: 3.4.2, 3.5.2, 3.7.0, 3.6.1
>
>
> We have observed an issue where inter broker SSL listener is not coming up 
> for large keystores (size >16K)
> 1. Currently validator code doesn't work well with large stores. Right now, 
> WRAP returns if there is already data in the buffer. But if we need more data 
> to be wrapped for UNWRAP to succeed, we end up looping forever.
> 2. Observed large TLSv3 post handshake messages are not getting read and 
> causing validator code loop forever. This is observed with JDK17+
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

[Yash Mayya]

Hi Chris,

Thanks for the KIP!
+1 (binding)

Yash

On Fri, Oct 6, 2023 at 9:54 PM Greg Harris 
wrote:

> Hey Chris,
>
> Thanks for the KIP!
> I think that preserving the ephemeral nature of the logging change is
> the right choice here, and using the config topic for intra-cluster
> broadcast is better than REST forwarding.
>
> +1 (binding)
>
> Thanks,
> Greg
>
> On Fri, Oct 6, 2023 at 9:05 AM Chris Egerton 
> wrote:
> >
> > Hi all,
> >
> > I'd like to call for a vote on KIP-976, which augments the existing
> dynamic
> > logger adjustment REST API for Kafka Connect to apply changes
> cluster-wide
> > instead on a per-worker basis.
> >
> > The KIP:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-976:+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect
> >
> > The discussion thread:
> > https://lists.apache.org/thread/w3x3f3jmyd1vfjxho06y8xgt6mhhzpl5
> >
> > Cheers,
> >
> > Chris
>

Re: [PR] MINOR Update default docs point to 3.6.0 release docs [kafka-site]

[via GitHub]

divijvaidya commented on code in PR #556:
URL: https://github.com/apache/kafka-site/pull/556#discussion_r1349660455


##
downloads.html:
##
@@ -6,12 +6,41 @@

 Download
 
-3.5.1 is the latest release. The current stable version is 3.5.1
+3.6.0 is the latest release. The current stable version is 3.6.0
 
 
 You can verify your download by following these https://www.apache.org/info/verification.html";>procedures and using 
these https://downloads.apache.org/kafka/KEYS";>KEYS.
 
 
+
+3.6.0
+
+
+Released Oct 7, 2023
+
+
+https://archive.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html";>Release 
Notes

Review Comment:
   makes sense



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.6 #87

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 307414 lines...]
> Task :core:classes
> Task :core:compileTestJava NO-SOURCE

Gradle Test Run :streams:test > Gradle Test Executor 92 > 
EosV2UpgradeIntegrationTest > [false] > shouldUpgradeFromEosAlphaToEosV2[false] 
PASSED

Gradle Test Run :streams:test > Gradle Test Executor 92 > 
EosV2UpgradeIntegrationTest > [true] > shouldUpgradeFromEosAlphaToEosV2[true] 
STARTED
> Task :core:compileTestScala

Gradle Test Run :streams:test > Gradle Test Executor 92 > 
EosV2UpgradeIntegrationTest > [true] > shouldUpgradeFromEosAlphaToEosV2[true] 
PASSED

7127 tests completed, 3 failed, 1 skipped
There were failing tests. See the report at: 
file:///home/jenkins/workspace/Kafka_kafka_3.6/streams/build/reports/tests/test/index.html
> Task :core:testClasses

Deprecated Gradle features were used in this build, making it incompatible with 
Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings 
and determine if they come from your own scripts or plugins.

For more on this, please refer to 
https://docs.gradle.org/8.2.1/userguide/command_line_interface.html#sec:command_line_warnings
 in the Gradle documentation.

BUILD SUCCESSFUL in 2h 40m 49s
296 actionable tasks: 109 executed, 187 up-to-date

Publishing build scan...
https://ge.apache.org/s/vj7e2pp4cr3wm


See the profiling report at: 
file:///home/jenkins/workspace/Kafka_kafka_3.6/build/reports/profile/profile-2023-10-08-07-07-50.html
A fine-grained performance profile is available: use the --scan option.
[Pipeline] junit
Recording test results
[Checks API] No suitable checks publisher found.
[Pipeline] echo
Skipping Kafka Streams archetype test for Java 20
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
> Task :streams:compileTestJava
> Task :streams:testClasses
> Task :streams:testJar
> Task :streams:testSrcJar
> Task :streams:publishMavenJavaPublicationToMavenLocal
> Task :streams:publishToMavenLocal

Deprecated Gradle features were used in this build, making it incompatible with 
Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings 
and determine if they come from your own scripts or plugins.

For more on this, please refer to 
https://docs.gradle.org/8.2.1/userguide/command_line_interface.html#sec:command_line_warnings
 in the Gradle documentation.

BUILD SUCCESSFUL in 5m 12s
94 actionable tasks: 41 executed, 53 up-to-date

Publishing build scan...
https://ge.apache.org/s/ss7s4iqwrdokg

[Pipeline] sh
+ grep ^version= gradle.properties
+ cut -d= -f 2
[Pipeline] dir
Running in 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_3.6/streams/quickstart
[Pipeline] {
[Pipeline] sh
+ mvn clean install -Dgpg.skip
[INFO] Scanning for projects...
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Kafka Streams :: Quickstart[pom]
[INFO] streams-quickstart-java[maven-archetype]
[INFO] 
[INFO] < org.apache.kafka:streams-quickstart >-
[INFO] Building Kafka Streams :: Quickstart 3.6.1-SNAPSHOT[1/2]
[INFO]   from pom.xml
[INFO] [ pom ]-
[INFO] 
[INFO] --- clean:3.0.0:clean (default-clean) @ streams-quickstart ---
[INFO] 
[INFO] --- remote-resources:1.5:process (process-resource-bundles) @ 
streams-quickstart ---
[INFO] 
[INFO] --- site:3.5.1:attach-descriptor (attach-descriptor) @ 
streams-quickstart ---
[INFO] 
[INFO] --- gpg:1.6:sign (sign-artifacts) @ streams-quickstart ---
[INFO] 
[INFO] --- install:2.5.2:install (default-install) @ streams-quickstart ---
[INFO] Installing 
/home/jenkins/jenkins-agent/workspace/Kafka_kafka_3.6/streams/quickstart/pom.xml
 to 
/home/jenkins/.m2/repository/org/apache/kafka/streams-quickstart/3.6.1-SNAPSHOT/streams-quickstart-3.6.1-SNAPSHOT.pom
[INFO] 
[INFO] --< org.apache.kafka:streams-quickstart-java >--
[INFO] Building streams-quickstart-java 3.6.1-SNAPSHOT[2/2]
[INFO]   from java/pom.xml
[INFO] --[ maven-archetype ]---
[INFO] 
[INFO] --- clean:3.0.0:clean (default-clean) @ streams-quickstart-java ---
[INFO] 
[INFO] --- remote-resources:1.5:process (process-resource-bundles) @ 
streams-quickstart-java ---
[INFO] 
[INFO] --- resources:2.7:resources (default-resources) @ 
streams-quickstart-java ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 6 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- resources:2.7:testResources (default-testResources) @ 

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2265

[Apache Jenkins Server]

See 

Re: Apache Kafka 3.6.0 release

[Satish Duggana]

+1 to add it as part of upgrade page. Users refer to the upgrade page
to access any information related to upgrades for a release as it is
expected to contain all the relevant text and references regarding the
upgrade in that particular release.

David,

On Thu, 5 Oct 2023 at 17:22, Ismael Juma  wrote:
>
> The upgrade page is the right place to add it, in my opinion.
>
> Ismael
>
> On Thu, Oct 5, 2023 at 5:02 PM David Arthur
>  wrote:
>
> > Hey folks, we found a significant ZK migration bug today
> > https://issues.apache.org/jira/browse/KAFKA-15552. Since the release is
> > already voted in, the fix will wait for 3.6.1. In the meantime, do we have
> > a way to indicate known significant issues in the release notes? Maybe just
> > a note in the blog?
> >
> > Thanks!
> > David
> >
> > On Fri, Sep 29, 2023 at 1:42 PM Ismael Juma  wrote:
> >
> > > Thanks Satish!
> > >
> > > Ismael
> > >
> > > On Thu, Sep 28, 2023 at 6:39 AM Satish Duggana  > >
> > > wrote:
> > >
> > > > We do not have any pending release blockers for now. RC1 release
> > > > blocker KAFKA-15498[1] is merged to 3.6.
> > > >  I will create RC2 by 29th Sep 12:00 pm PT and start a new RC thread.
> > > >
> > > > 1. https://github.com/apache/kafka/pull/14434
> > > >
> > > > Thanks,
> > > > Satish.
> > > >
> > > >
> > > >
> > > > On Wed, 27 Sept 2023 at 13:36, Divij Vaidya 
> > > > wrote:
> > > > >
> > > > > Ismael,
> > > > > Thank you for checking.
> > > > > Multiple other folks have validated after I left the comment here
> > that
> > > > > it doesn't impact log truncation and hence won't lead to data loss. I
> > > > > agree that it's not a blocker.
> > > > >
> > > > > (ref: https://github.com/apache/kafka/pull/14457)
> > > > >
> > > > > --
> > > > > Divij Vaidya
> > > > >
> > > > > On Wed, Sep 27, 2023 at 8:50 PM Ismael Juma 
> > wrote:
> > > > > >
> > > > > > Doesn't look like a blocker to me.
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Wed, Sep 27, 2023 at 2:36 AM Divij Vaidya <
> > > divijvaidy...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey team
> > > > > > >
> > > > > > > I need help in determining whether
> > > > > > > https://github.com/apache/kafka/pull/14457 is a release blocker
> > > bug
> > > > or
> > > > > > > not. If someone is familiar with replication protocol (on the log
> > > > > > > diverange and reconciliation process), please add your comments
> > on
> > > > the
> > > > > > > PR.
> > > > > > >
> > > > > > > --
> > > > > > > Divij Vaidya
> > > > > > >
> > > > > > > On Wed, Sep 27, 2023 at 10:43 AM Divij Vaidya <
> > > > divijvaidy...@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > A community member reported another bug in TS feature in 3.6 -
> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15511
> > > > > > > >
> > > > > > > > I don't consider it as a blocker for release because the bug
> > > > occurs in
> > > > > > > > rare situations when the index on disk or in a remote store is
> > > > > > > > corrupted and fails a sanity check.
> > > > > > > > Sharing it here as an FYI.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Divij Vaidya
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2023 at 11:16 AM Divij Vaidya <
> > > > divijvaidy...@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Found a bug while testing TS feature in 3.6 -
> > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15481
> > > > > > > > >
> > > > > > > > > I don't consider it as a blocker for release since it's a
> > > > concurrency
> > > > > > > > > bug that should occur rarely for a feature which is early
> > > access.
> > > > > > > > > Sharing it here as FYI in case someone else thinks
> > differently.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Divij Vaidya
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2023 at 1:26 AM Satish Duggana <
> > > > > > > satish.dugg...@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Thanks Divij for raising a PR for doc formatting issue.
> > > > > > > > > >
> > > > > > > > > > On Thu, 21 Sep, 2023, 2:22 PM Divij Vaidya, <
> > > > divijvaidy...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hey Satish
> > > > > > > > > > >
> > > > > > > > > > > I filed a PR to fix the website formatting bug in 3.6
> > > > > > > documentation -
> > > > > > > > > > > https://github.com/apache/kafka/pull/14419
> > > > > > > > > > > Please take a look when you get a chance.
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Divij Vaidya
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Sep 19, 2023 at 5:36 PM Chris Egerton
> > > > > > > 
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Satish,
> > > > > > > > > > > >
> > > > > > > > > > > > I think this qualifies as a blocker. This API has been
> > > > around
> > > > > > > for years
> > > > > > > > > > > now
> > > > > > > > > > > > and, while we don't document it as not exposing
> > > > duplicates*, it
> > > > >

[jira] [Created] (KAFKA-15565) KafkaAdminClient does not honor request timeout ms

[Sankalp Bhatia (Jira)]

Sankalp Bhatia created KAFKA-15565:
--

 Summary: KafkaAdminClient does not honor request timeout ms 
 Key: KAFKA-15565
 URL: https://issues.apache.org/jira/browse/KAFKA-15565
 Project: Kafka
  Issue Type: Bug
Reporter: Sankalp Bhatia
Assignee: Sankalp Bhatia


It seems to me there is a bug in this line in the KafkaAdminClient. For the 
constructor arg defaultRequestTimeoutMs of NetworkClient [1], it uses a 
hardcoded value of 1 hour.  Ideally, this should be derived from the client 
config     /** request.timeout.ms */ from the 
AdminClientConfig[2]. 


References

[1]https://github.com/apache/kafka/blob/1c3eb4395a15cf4f45b6dc0d39effb3dc087f5a4/clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java#L521

[2]https://github.com/apache/kafka/blob/1c3eb4395a15cf4f45b6dc0d39effb3dc087f5a4/clients/src/main/java/org/apache/kafka/clients/admin/AdminClientConfig.java#L98



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] KIP-980: Allow creating connectors in a stopped state

[Yash Mayya]

Hi all,

I'd like to start a vote on KIP-980 which proposes allowing the creation of
connectors in a stopped (or paused) state.

KIP -
https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state

Discussion Thread -
https://lists.apache.org/thread/om803vl191ysf711qm7czv94285rtt5d

Thanks,
Yash

CodeStyle Configuration

[Arpit Goyal]

Can anyone share  the configuration they are using in intellij editor which
automatically formats the code based on defined rules and standards and
adheres to the  guidelines provided by the kafka community.

Thanks and Regards
Arpit Goyal
8861094754