Re: [DISCUSS] Kafka 3.0

That's my view here. I think there are two ways this could work: 1) 2.x is the bridge release, and 3.0 is the one that completes the ZK removal. 2) 3.0 is the bridge release, and 4.0 is the one that completes the ZK removal. On 06/05/2020, 05:15, "Jeff Widman" wrote: IMO a bridge release,

Build failed in Jenkins: kafka-trunk-jdk14 #59

See Changes: [github] MINOR: Pass `-release 8` to scalac and upgrade to Gradle 6.4 (#8538) -- [...truncated 3.08 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-585: Conditional SMT

Hi Tom, I think that either proposal is sufficiently readable and they both have elements of cryptic syntax. I remember talking to an engineering director once who said he didn't hire "curly braces people" any more. This is an interface for curly braces people whichever way we go here. I slight

Re: Avro DeSerializeation Issue in Kafka Streams

Thanks for the info Nagendra. Thanks C Suresh On Wednesday, May 6, 2020, Nagendra Korrapati wrote: > When specific.avro.reader is set to true Deserializer tries to create the > instance of the Class. The class name is formed by reading the schema > (writer schema) from schema registry and conc

Build failed in Jenkins: kafka-trunk-jdk8 #4505

See Changes: [github] MINOR: Pass `-release 8` to scalac and upgrade to Gradle 6.4 (#8538) -- [...truncated 3.06 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi Boyang, While re-reading the KIP, I've got few small questions/comments: 1. When auto topic creation is enabled, brokers will send a CreateTopicRequest to the controller instead of writing to ZK directly. It means that creation of these topics are subject to be rejected with an error if a Crea

Build failed in Jenkins: kafka-2.5-jdk8 #109

See Changes: [cmccabe] KAFKA-9718; Don't log passwords for AlterConfigs in request logs (#8294) [cmccabe] KAFKA-9625: Fix altering and describing dynamic broker configurations ---

RE: [DISCUSS] Kafka 3.0

Ryanne Dolan wrote on 05/05/2020 20:36:49: > Exactly. Why would 3.1 be the breaking release? No one would expect > everything to break going from 3.0 to 3.1 Agree completely > > Ryanne > > On Tue, May 5, 2020 at 2:34 PM Gwen Shapira wrote: > > > It sounds like the decision to make the next

default auto.offset.reset value is reset to none

Hello, I'm facing an issue in one of our Kafka Streams applications using GlobalKTable. The idea is to have a GlobalKTable over compacted topic and be able to re-read it on startup. We had a consumer group and topic sometime ago, recently I've recreated a topic, requiring consumer offsets to be re

Build failed in Jenkins: kafka-trunk-jdk11 #1428

See Changes: [github] MINOR: Pass `-release 8` to scalac and upgrade to Gradle 6.4 (#8538) -- [...truncated 1.76 MB...] kafka.coordinator.transaction.TransactionStateMa

Warning free builds and Scala 2.13 as the default build

Hi all, I would like to share a few recent changes that should make the development experience a little better: 1. Warning free builds for Scala. We have had this when it comes to Java code for some time and with Scala 2.13.2's support for warning suppression, we can finally have it for Scala too

Re: [VOTE] KIP-577: Allow HTTP Response Headers Configured for Kafka Connect

Thanks for putting this together. +1 (binding) On Fri, Apr 17, 2020 at 2:02 PM Aneel Nazareth wrote: > Thanks Jeff, this seems like it addresses a user need. > > +1 (non-binding) > > On Fri, Apr 17, 2020 at 1:28 PM Zhiguo Huang > wrote: > > > > Thanks to everyone for their input. I've incorpor

Please give me the permission to create KIPs

My id: eddiejamsession Eddie

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Hi Jeff, Thanks for the KIP. It looks useful since it allows authorizers to use the broker's metrics instance. We could perhaps use this in AclAuthorizer to generate authorizer metrics? Regards, Rajini On Tue, May 5, 2020 at 9:04 PM Zhiguo Huang wrote: > > https://cwiki.apache.org/confluence/

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Thanks for the KIP. A couple of questions: 1. Is it intended for this method to return null or the broker metrics instance? 2. Is the Metrics class returned in any public APIs today or this the first time we are doing it? Ismael On Wed, May 6, 2020 at 8:10 AM Rajini Sivaram wrote: > Hi Jeff, >

[DISCUSS] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

Hi all, I'd like to discuss KIP-607 that aims to add RocksDB memory usage metrics to Kafka Streams. https://cwiki.apache.org/confluence/display/KAFKA/KIP-607%3A+Add+Metrics+to+Record+the+Memory+Used+by+RocksDB+to+Kafka+Streams Best, Bruno

[jira] [Created] (KAFKA-9962) Admin client throws UnsupportedVersion exception when talking to old broker

Jose Armando Garcia Sancio created KAFKA-9962: - Summary: Admin client throws UnsupportedVersion exception when talking to old broker Key: KAFKA-9962 URL: https://issues.apache.org/jira/browse/KAFKA-996

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

1. It is intended to return broker metrics. 2. I did searching, there is no any public API return broker Metrics class so far. So this will be first time. I did find connect also expose the Metrics class, but it is not in public API. org.apache.kafka.connect.runtime.ConnectMetrics /** * Get the

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Will update KIP. thanks! On 2020/05/06 15:09:53, Rajini Sivaram wrote: > Hi Jeff, > > Thanks for the KIP. It looks useful since it allows authorizers to use the > broker's metrics instance. We could perhaps use this in AclAuthorizer to > generate authorizer metrics? > > Regards, > > Rajini

Re: [DISCUSS] KIP-605 Expand Connect Worker Internal Topic Settings

Thanks, folks. The KIP appears to be satisfactory, so I plan to start a vote thread tomorrow unless there are objections. Best regards, Randall On Mon, May 4, 2020 at 2:56 PM Christopher Egerton wrote: > Hi Randall, > > Looks great! Definitely in favor of a WARN level message instead of faili

[jira] [Resolved] (KAFKA-4969) State-store workload-aware StreamsPartitionAssignor

[ https://issues.apache.org/jira/browse/KAFKA-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-4969. Fix Version/s: (was: 1.1.0) 2.6.0 Resolution: Fixed T

[jira] [Resolved] (KAFKA-8207) StickyPartitionAssignor for KStream

[ https://issues.apache.org/jira/browse/KAFKA-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-8207. Resolution: Not A Problem Closing this as "Not a Problem" as the partition assignor

[jira] [Resolved] (KAFKA-8858) Kafka Streams - Failed to Rebalance Error and stream consumer stuck for some reason

[ https://issues.apache.org/jira/browse/KAFKA-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-8858. Assignee: Guozhang Wang Resolution: Duplicate > Kafka Streams - Failed to Reb

[jira] [Resolved] (KAFKA-6063) StreamsException is thrown after the changing `partitions`

[ https://issues.apache.org/jira/browse/KAFKA-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-6063. Resolution: Not A Problem > StreamsException is thrown after the changing `partitio

[jira] [Resolved] (KAFKA-9112) Combine streams `onAssignment` with `partitionsAssigned` task creation

[ https://issues.apache.org/jira/browse/KAFKA-9112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-9112. Fix Version/s: 2.6.0 Assignee: Guozhang Wang Resolution: Fixed Thin

Re: [DISCUSS] KIP-587 Suppress detailed responses for handled exceptions in security-sensitive environments

Hi Chris, Apologies for the name confusion! I've been working with the my customer sponsor over the last few weeks, and we finally have an answer regarding "only exceptions or all responses." This organization is really interested in removing stack traces from all responses, which will expand t

Re: [Discuss] KIP-582 Add a "continue" option for Kafka Connect error handling

Hi Chris, Thanks a lot for the reply. To make error messages easy to find, I think we can add messages to the header of a broken record, just like the mechanism in DLQ. This way even if the broken record is stored into external systems, error messages and broken records are paired. Currently

Re: [VOTE] KIP-586: Deprecate commit records without record metadata

Thanks for putting this KIP together, Mario. +1 (binding) Randall On Mon, Apr 27, 2020 at 2:05 PM Mario Molina wrote: > Hi all, > > I'd like to start a vote for KIP-586. You can find the link for this KIP > here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-586%3A+Deprecate+commit

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

Thanks for starting the vote, Yu. +1 (binding) Randall On Sat, Dec 21, 2019 at 1:22 AM Yu Watanabe wrote: > Thank for the KIP. > I really want this for my project. > > +1 (non-binding) >

Build failed in Jenkins: kafka-2.3-jdk8 #200

See Changes: [konstantine] KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses [konstantine] KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd (#8554) [konstantine] KAFKA-9633: Ensure ConfigP

Build failed in Jenkins: kafka-trunk-jdk14 #60

See Changes: [github] KAFKA-6342; Remove unused workaround for JSON parsing of non-escaped -- [...truncated 3.08 MB...] org.apache.kafka.streams.TestTopicsTest > testEmpty

[jira] [Created] (KAFKA-9963) High CPU

Evan Williams created KAFKA-9963: Summary: High CPU Key: KAFKA-9963 URL: https://issues.apache.org/jira/browse/KAFKA-9963 Project: Kafka Issue Type: Bug Components: core Affects

Build failed in Jenkins: kafka-trunk-jdk8 #4506

See Changes: [github] KAFKA-6342; Remove unused workaround for JSON parsing of non-escaped -- [...truncated 3.10 MB...] org.apache.kafka.streams.test.OutputVerifierTest

[jira] [Resolved] (KAFKA-9947) TransactionsBounceTest may leave threads running

[ https://issues.apache.org/jira/browse/KAFKA-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9947. Resolution: Fixed > TransactionsBounceTest may leave threads running >

[jira] [Created] (KAFKA-9964) Better description of RoundRobinPartitioner behavior for AK 2.4+

Michael Bingham created KAFKA-9964: -- Summary: Better description of RoundRobinPartitioner behavior for AK 2.4+ Key: KAFKA-9964 URL: https://issues.apache.org/jira/browse/KAFKA-9964 Project: Kafka

[jira] [Resolved] (KAFKA-9419) Integer Overflow Possible with CircularIterator

[ https://issues.apache.org/jira/browse/KAFKA-9419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-9419. --- Resolution: Fixed > Integer Overflow Possible with CircularIterator > ---

[jira] [Resolved] (KAFKA-9964) Better description of RoundRobinPartitioner behavior for AK 2.4+

[ https://issues.apache.org/jira/browse/KAFKA-9964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Bingham resolved KAFKA-9964. Resolution: Invalid > Better description of RoundRobinPartitioner behavior for AK 2.4+ > --

[jira] [Created] (KAFKA-9965) Uneven distribution with RoundRobinPartitioner in AK 2.4+

Michael Bingham created KAFKA-9965: -- Summary: Uneven distribution with RoundRobinPartitioner in AK 2.4+ Key: KAFKA-9965 URL: https://issues.apache.org/jira/browse/KAFKA-9965 Project: Kafka I

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi David and Jun, I wanted to add to the discussion about using requests/sec vs. time on server threads (similar to request quota) for expressing quota for topic ops. I think request quota does not protect the brokers from overload by itself -- it still requires tuning and sometimes re-tuning, be

Jenkins build is back to normal : kafka-trunk-jdk11 #1429

See

[jira] [Created] (KAFKA-9966) Flaky Test EosBetaUpgradeIntegrationTest#shouldUpgradeFromEosAlphaToEosBeta

Matthias J. Sax created KAFKA-9966: -- Summary: Flaky Test EosBetaUpgradeIntegrationTest#shouldUpgradeFromEosAlphaToEosBeta Key: KAFKA-9966 URL: https://issues.apache.org/jira/browse/KAFKA-9966 Project

Build failed in Jenkins: kafka-2.4-jdk8 #196

See Changes: [konstantine] KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses [konstantine] KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd (#8554) [konstantine] KAFKA-9633: Ensure ConfigP

[jira] [Resolved] (KAFKA-9768) rest.advertised.listener configuration is not handled properly by the worker

[ https://issues.apache.org/jira/browse/KAFKA-9768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-9768. --- Resolution: Fixed > rest.advertised.listener configuration is not handled pro

Jenkins build is back to normal : kafka-trunk-jdk14 #61

See

Jenkins build is back to normal : kafka-trunk-jdk8 #4507

See

Re: [DISCUSS] Kafka 3.0

On Mon, May 4, 2020, at 17:12, Ryanne Dolan wrote: > Hey Colin, I think we should wait until after KIP-500's "bridge > release" so there is a clean break from Zookeeper after 3.0. The > bridge release by definition is an attempt to not break anything, so > it theoretically doesn't warrant a major r

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hey David, thanks for the feedbacks! On Wed, May 6, 2020 at 2:06 AM David Jacot wrote: > Hi Boyang, > > While re-reading the KIP, I've got few small questions/comments: > > 1. When auto topic creation is enabled, brokers will send a > CreateTopicRequest > to the controller instead of writing to

Re: [DISCUSS] Kafka 3.0

On Mon, May 4, 2020, at 17:33, Gwen Shapira wrote: > +1 for removing MM 1.0 when we cut a breaking release. It is sad to see > that we are still investing in it (I just saw a KIP toward improving its > reset policy). > > My understanding was that KIP-590 is not breaking compatibility, I think > Gu

[jira] [Created] (KAFKA-9967) SASL PLAIN authentication with custom callback handler

indira created KAFKA-9967: - Summary: SASL PLAIN authentication with custom callback handler Key: KAFKA-9967 URL: https://issues.apache.org/jira/browse/KAFKA-9967 Project: Kafka Issue Type: Bug

Re: [DISCUSS] Kafka 3.0

On Tue, May 5, 2020, at 12:36, Ryanne Dolan wrote: > > In 3.0 it sounds like nothing is breaking and our big change won't be > > complete... so, what's the motivation for the major release? > > Exactly. Why would 3.1 be the breaking release? No one would expect > everything to break going from 3.0

Build failed in Jenkins: kafka-trunk-jdk14 #62

See Changes: [github] KAFKA-9419: Fix possible integer overflow in CircularIterator (#7950) [github] KAFKA-9768: Fix handling of rest.advertised.listener config (#8360) -

Re: [DISCUSS] Kafka 3.0

> In fact, we know that the bridge release will involve at least one > incompatible change. We will need to drop support for the --zookeeper > flags in the command-line tools. If the bridge release(s) and the subsequent post-ZK release are _both_ breaking changes, I think we only have one option:

Re: [DISCUSS] Kafka 3.0

> This will allow us to get an "alpha" version of the KIP-500 mode out early for people to experiment with I think this is a non-sequitur. It's not a requirement that KIP-500 be merged to master and released in order for people to experiment with it. Nor does it sound great to call for a major rel

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

On Tue, May 5, 2020, at 08:12, Tom Bentley wrote: > Hi Colin, > > SCRAM is better than SASL PLAIN because it doesn't send the password over > the wire in the clear. Presumably this property is important for some users > who have chosen to use SCRAM. This proposal does send the password in the > cl

Build failed in Jenkins: kafka-trunk-jdk8 #4508

See Changes: [github] KAFKA-9768: Fix handling of rest.advertised.listener config (#8360) -- [...truncated 3.06 MB...] org.apache.kafka.streams.TopologyTestDriverTest > s

Jenkins build is back to normal : kafka-2.3-jdk8 #201

See

Build failed in Jenkins: kafka-trunk-jdk11 #1431

See Changes: [github] KAFKA-9768: Fix handling of rest.advertised.listener config (#8360) -- [...truncated 3.08 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Build failed in Jenkins: kafka-trunk-jdk8 #4509

See Changes: [github] MINOR - Increase the number of Trogdor Histogram buckets to 1 -- [...truncated 3.06 MB...] org.apache.kafka.streams.TopologyTestDriverTest > sh

Build failed in Jenkins: kafka-2.5-jdk8 #110

See Changes: [konstantine] KAFKA-9419: Fix possible integer overflow in CircularIterator (#7950) -- [...truncated 5.90 MB...] org.apache.kafka.streams.MockProcessorContext