Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hi Divij, Thanks for the KIP! My only question: KT1. In the case where we change the constraints so that a user's previously valid configuration is now invalid, do we do anything other than throw a ConfigException? Thanks, Kirk On Mon, Nov 18, 2024, at 2:13 AM, Divij Vaidya wrote: > Hey folk

Re: Configuration format for Log4j Core 2.x

Hello Chia-Ping I have just included the configuration migration section in the KIP-653 . Please take a look. Best, TengYao Jörn Franke 於 2024年11月19日 週二 上午12:36寫道： > Yaml allows to add comments compared to JSON which is not only for > examples good

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #122

See

Re: [DISCUSS] KIP-1112: allow custom processor wrapping

One more small change -- originally I proposed to only add this config to the TopologyConfig, and not to the StreamsConfig. However while implementing a POC I noticed that TopologyConfig does not have a constructor that accepts a plain properties or config map, and only one that takes in a StreamsC

Re: [DISCUSS] KIP-1109: Unifying Kafka Consumer Topic Metrics

Hi, Apoorv, We did the dot replacement in Yammer metric scope to help with the naming schema used in Graphite and the java consumer code just copied it. Since this is an issue very specific to Graphite, it's better not to change the name directly in Kafka metric. If needed, a Graphite specific rep

Re: [DISCUSS] KIP-1112: allow custom processor wrapping

Thanks Almog! That makes sense to me, I've updated the KIP so that the ProcessorWrapper will extend Configurable but gave it a default no-op implementation so that it's optional. On Mon, Nov 18, 2024 at 1:57 PM Almog Gavra wrote: > Thanks Sophie! This KIP will certainly make it easier to impleme

Re: [VOTE] KIP-1112: allow custom processor wrapping

Thanks for the KIP, Sophie, seems like a useful addition. +1(binding) -Bill On Mon, Nov 18, 2024 at 3:23 PM Sophie Blee-Goldman wrote: > Since there have been no concerns about this proposal I'm going to go ahead > and call for a vote on KIP-1112: allow custom processor wrapping. > > > https:/

Re: [DISCUSS] KIP-1112: allow custom processor wrapping

Thanks Sophie! This KIP will certainly make it easier to implement any kind of custom functionality across all processors in the DSL, I can imagine quite a few use cases for this. One suggestion, we should consider including a configure() method that takes in Map configs, so that it can be configu

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

Hi Chia-Ping, Thanks for your suggestions and feedback. Q1: I have updated this according your suggestions. Q2: This is necessary change since there is a assumption about *RecourdAccumulator *that all records have same acks(e.g. ProducerConfig.acks) so we need to a method to distinguish which ack

Re: [DISCUSS] KIP-1111: Enforcing Explicit Naming for Kafka Streams Internal Topics

Hi Sebastien, Thanks for the KIP! In general, I'm a fan of giving users the tools they need to protect their organization so I'm supportive of this proposal. A few nits and comments: A1. [nit] consider 'topics.internal.require.explicit.naming' so that (a) we can group anything else we introduce f

[PR] MINOR: Fix 3.7.1 is a fixed version for CVE-2024-31141 [kafka-site]

gharris1727 opened a new pull request, #646: URL: https://github.com/apache/kafka-site/pull/646 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Resolved] (KAFKA-17887) Handle response log result error in DelayedShareFetch

[ https://issues.apache.org/jira/browse/KAFKA-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apoorv Mittal resolved KAFKA-17887. --- Resolution: Fixed Not an issue anymore. > Handle response log result error in DelayedShareF

[VOTE] KIP-1112: allow custom processor wrapping

Since there have been no concerns about this proposal I'm going to go ahead and call for a vote on KIP-1112: allow custom processor wrapping. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1112%3A+allow+custom+processor+wrapping Please direct any new feedback (if any) to the [DISCUSS] thre

Re: [PR] Add CVE-2024-31141 to CVE list [kafka-site]

gharris1727 merged PR #645: URL: https://github.com/apache/kafka-site/pull/645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apa

[PR] Add CVE-2024-31141 to CVE list [kafka-site]

gharris1727 opened a new pull request, #645: URL: https://github.com/apache/kafka-site/pull/645 As announced here: https://lists.apache.org/thread/9whdzfr0zwdhr364604w5ssnzmg4v2lv -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[jira] [Created] (KAFKA-18031) Flaky PlaintextConsumerTest testCloseLeavesGroupOnInterrupt

Lianet Magrans created KAFKA-18031: -- Summary: Flaky PlaintextConsumerTest testCloseLeavesGroupOnInterrupt Key: KAFKA-18031 URL: https://issues.apache.org/jira/browse/KAFKA-18031 Project: Kafka

[jira] [Resolved] (KAFKA-18008) Flaky testMultiConsumerSessionTimeoutOnStopPolling

[ https://issues.apache.org/jira/browse/KAFKA-18008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-18008. Resolution: Fixed Seems to pass consistently after the related fix so closing it.  [https

[jira] [Created] (KAFKA-18036) TransactionsWithTieredStoreTest testReadCommittedConsumerShouldNotSeeUndecidedData is flaky

David Arthur created KAFKA-18036: Summary: TransactionsWithTieredStoreTest testReadCommittedConsumerShouldNotSeeUndecidedData is flaky Key: KAFKA-18036 URL: https://issues.apache.org/jira/browse/KAFKA-18036

[jira] [Created] (KAFKA-18035) TransactionsTest testBumpTransactionalEpochWithTV2Disabled failed on trunk

David Arthur created KAFKA-18035: Summary: TransactionsTest testBumpTransactionalEpochWithTV2Disabled failed on trunk Key: KAFKA-18035 URL: https://issues.apache.org/jira/browse/KAFKA-18035 Project: K

[jira] [Created] (KAFKA-18034) CommitRequestManager should fail pending requests on fatal coordinator errors

Lianet Magrans created KAFKA-18034: -- Summary: CommitRequestManager should fail pending requests on fatal coordinator errors Key: KAFKA-18034 URL: https://issues.apache.org/jira/browse/KAFKA-18034 Pro

[jira] [Created] (KAFKA-18033) ShareConsumerTest testAcknowledgementCommitCallbackInvalidRecordStateException is flaky

David Arthur created KAFKA-18033: Summary: ShareConsumerTest testAcknowledgementCommitCallbackInvalidRecordStateException is flaky Key: KAFKA-18033 URL: https://issues.apache.org/jira/browse/KAFKA-18033

[jira] [Resolved] (KAFKA-18016) Fix bugs in handling piggyback acknowledgements in ShareFetch

[ https://issues.apache.org/jira/browse/KAFKA-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivsundar R resolved KAFKA-18016. -- Resolution: Fixed > Fix bugs in handling piggyback acknowledgements in ShareFetch > --

Re: Configuration format for Log4j Core 2.x

Yaml allows to add comments compared to JSON which is not only for examples good, but also to document certain implications when deploying production configuration. > Am 18.11.2024 um 15:02 schrieb TengYao Chi : > > Hi everyone, > > From my experience, it has been quite challenging to underst

[jira] [Created] (KAFKA-18032) ProducerFencedExceptions should wipe out the state store only for the tasks that are related to the failed transaction

Eduwer Camacaro created KAFKA-18032: --- Summary: ProducerFencedExceptions should wipe out the state store only for the tasks that are related to the failed transaction Key: KAFKA-18032 URL: https://issues.apache.o

[jira] [Resolved] (KAFKA-17561) Operator Metrics for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-17561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-17561. - Resolution: Fixed > Operator Metrics for Kafka Streams > -- > >

RE: Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

On 2024/11/14 01:45:57 Chia-Ping Tsai wrote: > hi All > > This KIP is based on our use case where an edge application with many sensors wants to use a single producer to deliver ‘few but varied’ records with different acks settings. The reason for using a single producer is to minimize resource usa

Re: Configuration format for Log4j Core 2.x

hi Mickael, Thanks for the clarification. I'm still +1 on using YAML, as this PR ultimately changes the 'content' of the log4j properties file. So why not use a more readable structure with this change? Additionally, users can still use the deprecated properties when migrating to 4.0. hi TengYao

[VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi All, I would like to start vote on KIP-1099: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group Discussion thread: https://lists.apache.org/thread/lo64123wm3kjmjsx72xg67wvjx9nykk8 Thanks, PoAn

[jira] [Resolved] (KAFKA-17910) Create integration tests for Admin.listGroups and Admin.describeClassicGroups

[ https://issues.apache.org/jira/browse/KAFKA-17910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17910. Fix Version/s: 4.0.0 Resolution: Fixed > Create integration tests for Admin.listGro

Re: [VOTE] KIP-1073: Return fenced brokers in DescribeCluster response

Hi, +1 (binding), thanks for the KIP Mickael On Wed, Nov 6, 2024 at 9:05 AM Gantigmaa Selenge wrote: > > Hi everyone, > > A quick reminder that this KIP is still awaiting more votes. You can find > the implementation of this KIP in this draft PR: > https://github.com/apache/kafka/pull/17524. >

Re: [DISCUSS] KIP-1071: Streams Rebalance Protocol

Actually, scratch that. On reflection I think I prefer Bruno's original idea to specify it in the configuration. Cheers, Nick On Sat, 16 Nov 2024 at 17:59, Nick Telford wrote: > Hey everyone, > > With respect to Bruno's proposal, could instances cache their topology > epoch on disk, and then up

Re: Configuration format for Log4j Core 2.x

Hi everyone, >From my experience, it has been quite challenging to understand and transform the .properties file from log4j 1 to log4j 2, as the properties format is fairly flat and difficult to understand. In comparison, YAML is more structured, easier to read, and widely used. I would like to +1

Re: Configuration format for Log4j Core 2.x

Hi Chia-Ping, Yes in 4.0.0 we will only provide log4j2 configuration files. We will still support log4j.properties files for compatibility but that will print a warning when these are used. However my question is about the format of our log4j2 files. Currently in the PR we are using the "properti

[jira] [Resolved] (KAFKA-18029) Fix quorum_reconfiguration_test.py

[ https://issues.apache.org/jira/browse/KAFKA-18029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18029. Resolution: Fixed 3.9: [https://github.com/apache/kafka/commit/cbb8cfe520e0fbd33fe013e43b

Re: Configuration format for Log4j Core 2.x

hi, Mickael I'm +1 for using the log4j2 format. Otherwise, it's odd for users to see deprecation warnings about log4j.properties when using our example files. Best, Chia-Ping Mickael Maison 於 2024年11月18日 週一 下午7:33寫道： > Hi, > > The log4j2 migration PR is pretty much ready to be merged and the

Re: [VOTE] KIP-1074: Allow the replication of user internal topics

Thanks everyone for the votes and your time reviewing this KIP. The vote passed with 3 binding votes and 1 non-binding vote. Best, Patrik On Thu, Nov 14, 2024 at 4:35 PM Mickael Maison wrote: > Hi, > > +1 (binding) > Thanks for the KIP! > > Mickael > > On Thu, Nov 14, 2024 at 3:42 PM Patrik Ma

Re: Configuration format for Log4j Core 2.x

Hi, The log4j2 migration PR is pretty much ready to be merged and the first deadlines for 4.0.0 is approaching fast. I think we should decide which format to use in our example log4j2 files to avoid having to update the format shortly after 4.0.0. This point is not really covered by KIP-653: Upgr

[jira] [Resolved] (KAFKA-14934) KafkaClusterTestKit makes FaultHandler accessible

[ https://issues.apache.org/jira/browse/KAFKA-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-14934. Fix Version/s: 4.0.0 Resolution: Fixed > KafkaClusterTestKit makes FaultHandler acc

[DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hey folks With 4.0, we have an opportunity to reset the default values and add constraints in the configurations based on our learnings since 3.0. Here's a KIP which modifies defaults for some properties and modifies the constraints for a few others. https://cwiki.apache.org/confluence/display/K

[jira] [Resolved] (KAFKA-18003) add test to make sure `Admin#deleteRecords` can handle the corrupted records

[ https://issues.apache.org/jira/browse/KAFKA-18003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-18003. -- Resolution: Fixed > add test to make sure `Admin#deleteRecords` can handle the corrupted recor

Re: [VOTE] KIP-1106: Add duration based offset reset option for consumer clients

Thanks everyone for the votes. The vote passed with 6 binding votes (Kamal, David Arthur, Lianet, David Jacot, Jun, Matthias) and 2 non-binding votes (Andrew, Apoorv). Regards, Manikumar On Mon, Nov 18, 2024 at 2:41 PM Manikumar wrote: > Thanks Jun, > > Yes, Updated the KIP > > On Sat, Nov 1

Re: [VOTE] KIP-1106: Add duration based offset reset option for consumer clients

Thanks Jun, Yes, Updated the KIP On Sat, Nov 16, 2024 at 1:21 AM Jun Rao wrote: > Hi, ManiKumar, > > Thanks for the KIP. +1 > > Should we document that duration can't be negative in the following too? > > AutoOffsetReset duration(Duration duration) > > Jun > > On Thu, Nov 14, 2024 at 5:31 AM Ma