[jira] [Created] (KAFKA-18906) Upgrade to Junit 5.13 and apply ParameterizedClass
Chia-Ping Tsai created KAFKA-18906: -- Summary: Upgrade to Junit 5.13 and apply ParameterizedClass Key: KAFKA-18906 URL: https://issues.apache.org/jira/browse/KAFKA-18906 Project: Kafka Issue Type: Test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Junit 5.13 will include the feature: {{ParameterizedClass}} which allows use to define parameters on the class. That can remove many duplicate annotations from tests. ref: https://github.com/junit-team/junit5/issues/878 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-18908) the size of appended value can't be larger than Short.MAX_VALUE
Chia-Ping Tsai created KAFKA-18908: -- Summary: the size of appended value can't be larger than Short.MAX_VALUE Key: KAFKA-18908 URL: https://issues.apache.org/jira/browse/KAFKA-18908 Project: Kafka Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai related to https://issues.apache.org/jira/browse/KAFKA-18907 we should document it for 4.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-17981) add Integration test for ConfigCommand to add config `key=[val1,val2]`
[ https://issues.apache.org/jira/browse/KAFKA-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17981. Fix Version/s: 4.1.0 Resolution: Fixed > add Integration test for ConfigCommand to add config `key=[val1,val2]` > -- > > Key: KAFKA-17981 > URL: https://issues.apache.org/jira/browse/KAFKA-17981 > Project: Kafka > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Nick Guo >Priority: Minor > Fix For: 4.1.0 > > > follow-up of KAFKA-17583 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path
hi > If this Connect case is the only one that deviates from that rule, I would recommended reverting it. But if there are many more cases, we may need to look into them in more detail. I briefly checked the commit history, and this is the only one that needs to be highlighted. Fortunately, it is easy to revert. > why was it removed? Oversight (ie, accidentally too early) or deliberately? deliberately. I have left a common on the PR ( https://github.com/apache/kafka/pull/17412#issuecomment-2692600494) Best, Chia-Ping Matthias J. Sax 於 2025年3月2日 週日 上午6:32寫道: > Don't have a strong opinion about the Connect case. > > I guess the question is: why was it removed? Oversight (ie, accidentally > too early) or deliberately? Also, what is the impact if we keep the API > and revert the removal? > > IMHO, the 3-releases-rule can have exceptions, if there are good reason > for it. I just don't have enough context about the change to judge. > > > > >> One more thing, 3.7 was released in Feb 2024 and hence it will be more > than > >> 12 months by the time 4.0 is released. Technically, it would be ok to > >> remove APIs deprecated in 3.7. > > Not sure if this is a strong argument. We cannot time releases totally > accurately, so the 3-prior-releases rule should be the "source of truth" > IMHO, that just practically translated to "about 1 year". But as I said > above, I also don't think we need to be dogmatic about it. > > > > -Matthias > > On 3/1/25 8:37 AM, Ismael Juma wrote: > > One more thing, 3.7 was released in Feb 2024 and hence it will be more > than > > 12 months by the time 4.0 is released. Technically, it would be ok to > > remove APIs deprecated in 3.7. > > > > Ismael > > > > On Sat, Mar 1, 2025, 8:34 AM Ismael Juma wrote: > > > >> Hi Chia-Ping, > >> > >> Is this Connect removal the only one that did not meet the stated > >> requirement? > >> > >> We do indeed aim not to remove APIs deprecated within the last 12 months > >> before the next major release, but I'm not sure how strictly this is > >> followed (since it's not enforced). > >> > >> If this Connect case is the only one that deviates from that rule, I > would > >> recommended reverting it. But if there are many more cases, we may need > to > >> look into them in more detail. > >> > >> Ismael > >> > >> On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai > wrote: > >> > >>> hi Matthias > >>> > >>> thanks for your updates. it looks great and readable. > >>> > >>> Regarding API compatibility, we state "we only provide API backward > >>> compatibility for version 3.7, 3.8, and 3.9" However, within Connect > APIs, > >>> we have removed a public API that was deprecated by 3.7 [0][1]. > >>> > >>> Since this KIP will likely include Connect APIs, it's important to > >>> discuss this now. We could: 1) add extra descriptions to highlight the > >>> difference for Connect APIs, or 2) revert the commit that removed the > >>> public API for consistent documentation. > >>> > >>> WDYT? > >>> > >>> [0] > >>> > https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e > >>> [1] > >>> > https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 > >>> > >>> Best, > >>> Chia-Ping > >>> > >>> On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: > Thanks for pointing me to test Ismael. Glad to see I just missed it, > >>> and > that we indeed have a test for it. > > > >> Feel free to edit the KIP directly – I'm totally fine with it. > >>> Thanks a lot! > > Thanks Kuan-Po! > > I did update the wiki page and added more details. I did follow > Bruno's > suggestion, to be a little bit more elaborate, as the KIP page is > >>> mainly > for Kafka maintainers. -- We might want to do it differently in user > facing docs, but detailed doc updates seems to be out-of-scope for the > KIP discussion and we can take it on the corresponding PR, to simplify > it further. > > > I did keep the 2.8.x bridge release for KS for now.; we can still > >>> change > the recommendation and skip 2.8.x as bridge release of course, if we > think it would be better for users. > > > Curious the get your feedback. > > > -Matthias > > > On 2/28/25 8:15 AM, Kuan Po Tseng wrote: > > Hi Matthias, > > > > Feel free to edit the KIP directly – I'm totally fine with it. Thanks > >>> a lot! > > > > Best, > > Kuan-Po Tseng > > > > On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: > >> Thanks all. I am happy that we are making progress to close this > out. > >> > >> As mentioned in my (long) email last Friday, I also think that the > >>> KIP > >> makes a lot of implicit assumptions, and it might benefit from begin > >> more explicit, even explaining things that are "set in stone" > >>> already, > >> and the KIP does not change them (like API deprecation timelines and > >> guarantees). Bu
Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path
Don't have a strong opinion about the Connect case. I guess the question is: why was it removed? Oversight (ie, accidentally too early) or deliberately? Also, what is the impact if we keep the API and revert the removal? IMHO, the 3-releases-rule can have exceptions, if there are good reason for it. I just don't have enough context about the change to judge. One more thing, 3.7 was released in Feb 2024 and hence it will be more than 12 months by the time 4.0 is released. Technically, it would be ok to remove APIs deprecated in 3.7. Not sure if this is a strong argument. We cannot time releases totally accurately, so the 3-prior-releases rule should be the "source of truth" IMHO, that just practically translated to "about 1 year". But as I said above, I also don't think we need to be dogmatic about it. -Matthias On 3/1/25 8:37 AM, Ismael Juma wrote: One more thing, 3.7 was released in Feb 2024 and hence it will be more than 12 months by the time 4.0 is released. Technically, it would be ok to remove APIs deprecated in 3.7. Ismael On Sat, Mar 1, 2025, 8:34 AM Ismael Juma wrote: Hi Chia-Ping, Is this Connect removal the only one that did not meet the stated requirement? We do indeed aim not to remove APIs deprecated within the last 12 months before the next major release, but I'm not sure how strictly this is followed (since it's not enforced). If this Connect case is the only one that deviates from that rule, I would recommended reverting it. But if there are many more cases, we may need to look into them in more detail. Ismael On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai wrote: hi Matthias thanks for your updates. it looks great and readable. Regarding API compatibility, we state "we only provide API backward compatibility for version 3.7, 3.8, and 3.9" However, within Connect APIs, we have removed a public API that was deprecated by 3.7 [0][1]. Since this KIP will likely include Connect APIs, it's important to discuss this now. We could: 1) add extra descriptions to highlight the difference for Connect APIs, or 2) revert the commit that removed the public API for consistent documentation. WDYT? [0] https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e [1] https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 Best, Chia-Ping On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: Thanks for pointing me to test Ismael. Glad to see I just missed it, and that we indeed have a test for it. Feel free to edit the KIP directly – I'm totally fine with it. Thanks a lot! Thanks Kuan-Po! I did update the wiki page and added more details. I did follow Bruno's suggestion, to be a little bit more elaborate, as the KIP page is mainly for Kafka maintainers. -- We might want to do it differently in user facing docs, but detailed doc updates seems to be out-of-scope for the KIP discussion and we can take it on the corresponding PR, to simplify it further. I did keep the 2.8.x bridge release for KS for now.; we can still change the recommendation and skip 2.8.x as bridge release of course, if we think it would be better for users. Curious the get your feedback. -Matthias On 2/28/25 8:15 AM, Kuan Po Tseng wrote: Hi Matthias, Feel free to edit the KIP directly – I'm totally fine with it. Thanks a lot! Best, Kuan-Po Tseng On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: Thanks all. I am happy that we are making progress to close this out. As mentioned in my (long) email last Friday, I also think that the KIP makes a lot of implicit assumptions, and it might benefit from begin more explicit, even explaining things that are "set in stone" already, and the KIP does not change them (like API deprecation timelines and guarantees). But as Bruno said, it would help a lot to make it easier to understand the KIP. Only a very small number of people seems to fully understand all the nuances, and it took quite a while to collect and curate all the cases, to shape this KIP... And we know that a lot of people actually read KIPs, and the title for KIP-1124 is or sure click-bait... I am also happy to edit the wiki page directly if that's ok with Kuan, to add background information that I believe is missing and important. My overall understanding of the state of the KIP, ie, its actual content seems to be, that there is a high level agreement, and the content of the KIP is ok. The only open question seems to be, if we want to recommend `2.8.x` as a bridge release for Kafka Streams or not. As said in my email on Friday, I am personally ok either way, with a preference to add `2.8.x` as bridge release, as it simplifies updating the code by avoiding compilation errors. With a 2.8.x bridge release, KS application code would _always_ compile, and the JavaDocs help a lot to guide users how to migrate off deprecated APIs. -- With a direct upgrade from older versions to 3.9.x, code does not compile, and there is n
[jira] [Created] (KAFKA-18905) Avoid out of order sequence errors from multiple in-flight batches
Sean Quah created KAFKA-18905: - Summary: Avoid out of order sequence errors from multiple in-flight batches Key: KAFKA-18905 URL: https://issues.apache.org/jira/browse/KAFKA-18905 Project: Kafka Issue Type: Bug Components: producer Reporter: Sean Quah Consider a similar setup to KAFKA-9199: The broker attempts to cache the state of the last 5 batches in order to enable duplicate detection. It is possible in some cases for this to result in a sequence such as the following: # Send batch n # Batch n successfully written, but response is delayed # Send Batch n+1, receive successful response for n+1 # Send Batch n+2, receive successful response for n+2 # Send Batch n+3, receive successful response for n+3 # Send Batch n+4, receive successful response for n+4 # Send Batch n+5, receive successful response for n+5 # Batch n times out, or we receive a {{NOT_LEADER_OR_FOLLOWER}} response # Retry batch n, the response is {{OUT_OF_ORDER_SEQUENCE}}, retry again and again This situation could happen with a {{max.in.flight.requests.per.connection}} as low as 2. To avoid this situation, we can avoid putting batch n+5 in flight while batch n is in flight. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1133: AK Documentation and Website in Markdown
Hi, I am totally in favor to make this happen, and I was disappointed each time in the past, when it was proposed but we did not do it. Given that it is a huge effort, I am wondering if it would be sufficient to just to this change for 4.0 release though, and not translate any older versions at all? For including the docs in the release, I am actually wondering what the value is? Does anybody actually use these artifact? Of course, we need to publish JavaDocs, but the content of the web page seems to have very low value to the community? -- I did also check ASF guidelines about it and the webpage [1] say: This material may include documentation concerning the release [...] So it is optional, but not required to publish docs artifacts from my understanding. I have no objections to keep publishing the web page docs, just saying, if the effort is too large and the value to low, we could consider just dropping it. I am also in favor of having easy access to `trunk / SNAPSHOT` version of the docs if possible. Some other ASF project do this already, for example Flink [2]. It's much easier to work with, and allows to spot issues during development. While it does not help on a PR directly, it is still a good way to verify a PR after it was merged, so any errors can be detected and corrected more easily. Also agree to the other raised points about generated content and permalinks. For permalinks, if effort is reasonable, I would rather try to redirect as many as possible. -Matthias [1] https://www.apache.org/legal/release-policy.html#distribute-other-artifacts [2] https://nightlies.apache.org/flink/flink-docs-master/ On 2/25/25 8:43 AM, David Arthur wrote: MM1: I suspect Hugo has some way of doing url redirection, but at the very least we can add redirects in our htaccess file https://github.com/apache/kafka-site/blob/asf-site/.htaccess Excluding the "front matter" (intro, quickstart, powered by, etc) I think the main links we need to worry about are the configuration docs pages. In the demo, it looks like we are using the same anchors for specific config sections. E.g., https://kafka-site-md-711970345036.us-central1.run.app/39/configuration/configuration/#topicconfigs_compression.gzip.level is the same content as https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level So maybe it shouldn't be too hard? Personally, I think the only links we should consider to be "permalinks" are the front matter and the documentation pages with the version in the URL. The unversioned documentation pages (like https://kafka.apache.org/documentation/#topicconfigs_compression.zstd.level) can change from release to release, so they are not expected to be permanent. Do we have any data on inbound links to the site? Maybe from Google Analytics or similar? -David A On Mon, Feb 24, 2025 at 10:17 AM Mickael Maison wrote: Hi Harish, This is definitively something we want to do! MM1. One aspect to keep in mind in terms of compatibility is whether existing links will still work. Thousands of online resources link to the Apache Kafka website. Is it possible to keep all links working? MM2. A bunch of sections are generated. To do so we have a number of classes in the public API (for example 0, 1) with toHtml() or main() methods that output HTML. Should we update these to output Markdown directly instead? Converting the website has been discussed several times, I'd suggest checking at least https://issues.apache.org/jira/browse/KAFKA-2967 to see what was previously said. 0: https://kafka.apache.org/39/javadoc/org/apache/kafka/common/config/ConfigDef.html#toHtml() 1: https://kafka.apache.org/39/javadoc/org/apache/kafka/clients/consumer/ConsumerConfig.html#main(java.lang.String%5B%5D) Thanks, Mickael On Mon, Feb 24, 2025 at 3:33 PM Bruno Cadonna wrote: Hi Harish, Thanks for the KIP! Great initiative! Do you plan to have a development version of the website online in parallel to the old HTML version? I understand that locally with the markdown files and an markdown editor, we see the rough structure of the docs (which is already an improvement), but we will not see the end product, because for that we again need a web server. I assume that after you migrated all docs to markdown, we want to see the complete website, review the content, and make some refinements. Best, Bruno On 23.02.25 07:52, Harish Vishwanath wrote: Thank you David. Appreciate your feedback. My comments below: DA1) I think this is a very good suggestion. While building the prototype, I did spend additional effort for older versions of the doc since there are fairly significant differences in the structure and format of the documents as compared to 3.x versions. I have written the automation which can do this, but I do think hosting the older versions (<3.x) in its current format and moving the rest to markdown will reduce the overall testing/validation effort. That said, if we
[jira] [Resolved] (KAFKA-18880) Remove kafka.cluster.Broker and BrokerEndPointNotAvailableException
[ https://issues.apache.org/jira/browse/KAFKA-18880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18880. Fix Version/s: 4.1.0 Resolution: Fixed > Remove kafka.cluster.Broker and BrokerEndPointNotAvailableException > --- > > Key: KAFKA-18880 > URL: https://issues.apache.org/jira/browse/KAFKA-18880 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: xuanzhang gong >Priority: Minor > Fix For: 4.1.0 > > > they are not used anymore after removing zk code -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-18868) add the "default value" explanation to the docs of num.replica.alter.log.dirs.threads
[ https://issues.apache.org/jira/browse/KAFKA-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18868. Resolution: Fixed trunk: [https://github.com/apache/kafka/commit/2ccc73783e30bde27aebb2a26a1f32e1327f6530] 4.0: https://github.com/apache/kafka/commit/ae798f12be4fb2f770b92d87b0986832e8fd4d82 > add the "default value" explanation to the docs of > num.replica.alter.log.dirs.threads > - > > Key: KAFKA-18868 > URL: https://issues.apache.org/jira/browse/KAFKA-18868 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Logan Zhu >Priority: Trivial > Fix For: 4.0.0 > > > The default value of {{num.replica.alter.log.dirs.threads}} is equal to the > number of log directories, but the documentation doesn't mention this. Users > only see a "null" default from the doc -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path
Hi Chia-Ping, Is this Connect removal the only one that did not meet the stated requirement? We do indeed aim not to remove APIs deprecated within the last 12 months before the next major release, but I'm not sure how strictly this is followed (since it's not enforced). If this Connect case is the only one that deviates from that rule, I would recommended reverting it. But if there are many more cases, we may need to look into them in more detail. Ismael On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai wrote: > hi Matthias > > thanks for your updates. it looks great and readable. > > Regarding API compatibility, we state "we only provide API backward > compatibility for version 3.7, 3.8, and 3.9" However, within Connect APIs, > we have removed a public API that was deprecated by 3.7 [0][1]. > > Since this KIP will likely include Connect APIs, it's important to discuss > this now. We could: 1) add extra descriptions to highlight the difference > for Connect APIs, or 2) revert the commit that removed the public API for > consistent documentation. > > WDYT? > > [0] > https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e > [1] > https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 > > Best, > Chia-Ping > > On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: > > Thanks for pointing me to test Ismael. Glad to see I just missed it, and > > that we indeed have a test for it. > > > > > > >> Feel free to edit the KIP directly – I'm totally fine with it. Thanks > a lot! > > > > Thanks Kuan-Po! > > > > I did update the wiki page and added more details. I did follow Bruno's > > suggestion, to be a little bit more elaborate, as the KIP page is mainly > > for Kafka maintainers. -- We might want to do it differently in user > > facing docs, but detailed doc updates seems to be out-of-scope for the > > KIP discussion and we can take it on the corresponding PR, to simplify > > it further. > > > > > > I did keep the 2.8.x bridge release for KS for now.; we can still change > > the recommendation and skip 2.8.x as bridge release of course, if we > > think it would be better for users. > > > > > > Curious the get your feedback. > > > > > > -Matthias > > > > > > On 2/28/25 8:15 AM, Kuan Po Tseng wrote: > > > Hi Matthias, > > > > > > Feel free to edit the KIP directly – I'm totally fine with it. Thanks > a lot! > > > > > > Best, > > > Kuan-Po Tseng > > > > > > On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: > > >> Thanks all. I am happy that we are making progress to close this out. > > >> > > >> As mentioned in my (long) email last Friday, I also think that the KIP > > >> makes a lot of implicit assumptions, and it might benefit from begin > > >> more explicit, even explaining things that are "set in stone" already, > > >> and the KIP does not change them (like API deprecation timelines and > > >> guarantees). But as Bruno said, it would help a lot to make it easier > to > > >> understand the KIP. > > >> > > >> Only a very small number of people seems to fully understand all the > > >> nuances, and it took quite a while to collect and curate all the > cases, > > >> to shape this KIP... And we know that a lot of people actually read > > >> KIPs, and the title for KIP-1124 is or sure click-bait... > > >> > > >> I am also happy to edit the wiki page directly if that's ok with Kuan, > > >> to add background information that I believe is missing and important. > > >> > > >> > > >> > > >> My overall understanding of the state of the KIP, ie, its actual > content > > >> seems to be, that there is a high level agreement, and the content of > > >> the KIP is ok. > > >> > > >> The only open question seems to be, if we want to recommend `2.8.x` > as a > > >> bridge release for Kafka Streams or not. As said in my email on > Friday, > > >> I am personally ok either way, with a preference to add `2.8.x` as > > >> bridge release, as it simplifies updating the code by avoiding > > >> compilation errors. With a 2.8.x bridge release, KS application code > > >> would _always_ compile, and the JavaDocs help a lot to guide users how > > >> to migrate off deprecated APIs. -- With a direct upgrade from older > > >> versions to 3.9.x, code does not compile, and there is no guidance as > > >> the old methods (and corresponding JavaDocs) are already removed, > making > > >> it significant harder for user to update their code and make it > compile > > >> again. > > >> > > >> In the end, we would only _recommend_ 2.8.x as a bridge release to > > >> users, and there won't be any claim that an direct upgrade to 3.9.x is > > >> not supported. If one wants to do this, it's tested, and ok. Knock > > >> yourself out -- there is for sure apps out there, that are just a > simple > > >> `builder.stream(...).filter(...).to(...)` the there won't be an API > > >> incompatibility issues. > > >> > > >> But I don't think it would be wise to _recommend_ a direct upgrade to > > >> 3.9.x for the reasons stated, as many ap
[jira] [Resolved] (KAFKA-17039) KIP-919 supports for `unregisterBroker`
[ https://issues.apache.org/jira/browse/KAFKA-17039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17039. Fix Version/s: 4.1.0 Resolution: Fixed > KIP-919 supports for `unregisterBroker` > --- > > Key: KAFKA-17039 > URL: https://issues.apache.org/jira/browse/KAFKA-17039 > Project: Kafka > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: TengYao Chi >Priority: Minor > Fix For: 4.1.0 > > > as title -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path
One more thing, 3.7 was released in Feb 2024 and hence it will be more than 12 months by the time 4.0 is released. Technically, it would be ok to remove APIs deprecated in 3.7. Ismael On Sat, Mar 1, 2025, 8:34 AM Ismael Juma wrote: > Hi Chia-Ping, > > Is this Connect removal the only one that did not meet the stated > requirement? > > We do indeed aim not to remove APIs deprecated within the last 12 months > before the next major release, but I'm not sure how strictly this is > followed (since it's not enforced). > > If this Connect case is the only one that deviates from that rule, I would > recommended reverting it. But if there are many more cases, we may need to > look into them in more detail. > > Ismael > > On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai wrote: > >> hi Matthias >> >> thanks for your updates. it looks great and readable. >> >> Regarding API compatibility, we state "we only provide API backward >> compatibility for version 3.7, 3.8, and 3.9" However, within Connect APIs, >> we have removed a public API that was deprecated by 3.7 [0][1]. >> >> Since this KIP will likely include Connect APIs, it's important to >> discuss this now. We could: 1) add extra descriptions to highlight the >> difference for Connect APIs, or 2) revert the commit that removed the >> public API for consistent documentation. >> >> WDYT? >> >> [0] >> https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e >> [1] >> https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 >> >> Best, >> Chia-Ping >> >> On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: >> > Thanks for pointing me to test Ismael. Glad to see I just missed it, >> and >> > that we indeed have a test for it. >> > >> > >> > >> Feel free to edit the KIP directly – I'm totally fine with it. >> Thanks a lot! >> > >> > Thanks Kuan-Po! >> > >> > I did update the wiki page and added more details. I did follow Bruno's >> > suggestion, to be a little bit more elaborate, as the KIP page is >> mainly >> > for Kafka maintainers. -- We might want to do it differently in user >> > facing docs, but detailed doc updates seems to be out-of-scope for the >> > KIP discussion and we can take it on the corresponding PR, to simplify >> > it further. >> > >> > >> > I did keep the 2.8.x bridge release for KS for now.; we can still >> change >> > the recommendation and skip 2.8.x as bridge release of course, if we >> > think it would be better for users. >> > >> > >> > Curious the get your feedback. >> > >> > >> > -Matthias >> > >> > >> > On 2/28/25 8:15 AM, Kuan Po Tseng wrote: >> > > Hi Matthias, >> > > >> > > Feel free to edit the KIP directly – I'm totally fine with it. Thanks >> a lot! >> > > >> > > Best, >> > > Kuan-Po Tseng >> > > >> > > On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: >> > >> Thanks all. I am happy that we are making progress to close this out. >> > >> >> > >> As mentioned in my (long) email last Friday, I also think that the >> KIP >> > >> makes a lot of implicit assumptions, and it might benefit from begin >> > >> more explicit, even explaining things that are "set in stone" >> already, >> > >> and the KIP does not change them (like API deprecation timelines and >> > >> guarantees). But as Bruno said, it would help a lot to make it >> easier to >> > >> understand the KIP. >> > >> >> > >> Only a very small number of people seems to fully understand all the >> > >> nuances, and it took quite a while to collect and curate all the >> cases, >> > >> to shape this KIP... And we know that a lot of people actually read >> > >> KIPs, and the title for KIP-1124 is or sure click-bait... >> > >> >> > >> I am also happy to edit the wiki page directly if that's ok with >> Kuan, >> > >> to add background information that I believe is missing and >> important. >> > >> >> > >> >> > >> >> > >> My overall understanding of the state of the KIP, ie, its actual >> content >> > >> seems to be, that there is a high level agreement, and the content of >> > >> the KIP is ok. >> > >> >> > >> The only open question seems to be, if we want to recommend `2.8.x` >> as a >> > >> bridge release for Kafka Streams or not. As said in my email on >> Friday, >> > >> I am personally ok either way, with a preference to add `2.8.x` as >> > >> bridge release, as it simplifies updating the code by avoiding >> > >> compilation errors. With a 2.8.x bridge release, KS application code >> > >> would _always_ compile, and the JavaDocs help a lot to guide users >> how >> > >> to migrate off deprecated APIs. -- With a direct upgrade from older >> > >> versions to 3.9.x, code does not compile, and there is no guidance as >> > >> the old methods (and corresponding JavaDocs) are already removed, >> making >> > >> it significant harder for user to update their code and make it >> compile >> > >> again. >> > >> >> > >> In the end, we would only _recommend_ 2.8.x as a bridge release to >> > >> users, and there won't be any claim that an direct upgrade to 3.9.x >>
[jira] [Created] (KAFKA-18907) add suitable error message when the appended value is too larger
Chia-Ping Tsai created KAFKA-18907: -- Summary: add suitable error message when the appended value is too larger Key: KAFKA-18907 URL: https://issues.apache.org/jira/browse/KAFKA-18907 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai In ZooKeeper mode, large config values can be created by appending. In contrast, KRaft mode does not allow this. The root cause is that {{ConfigRecord}} assumes a maximum value size of {{{}Short.MAX_VALUE{}}}. This causes a runtime error, which is then converted to "UnknownServerException" for users, making the issue unclear. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] KIP-1136: Make ConsumerGroupMetadata an interface
Hi Kirk Many thanks for very good questions! KT1. I don't think that internal implementation should totally prevent of passing alternative implementation, but for sure we need to log warning when producer receives different than default implementaion. I think that checking internally is the class implementation exactly DefaultConsumerGroupMetadata is against some basic principles like liskov substition and if we like to design this mechanism that way, we shouldn't expose an interface. KT2. I am also not 100% satisfied with that name, I don't think that there will be need for future different implementations as it is simple data struct. KT3. It will be place in producer/internals package and default java scope for constructor and whole class. @Matthias J. Sax having those KT1 and KT2 in mind, should we move forward with an interface or maybe ConsumerGroupMetadata has to be a class, but we have to limit possibility to create an instance manualy by making the class final and changing constructor visibility to private or package scope? czw., 27 lut 2025 o 01:51 Kirk True napisał(a): > Hi Paweł, > > Thanks for the KIP! > > My questions: > > KT1. What will prevent developers from implementing their own > ConsumerGroupMetadata and passing that to sendOffsetsToTransaction()? I > assume the code will check the incoming object is of type > DefaultConsumerGroupMetadata? > > KT2. To me, the use of the adjective "default" in > DefaultConsumerGroupMetadata implies that there could be other > implementations. Is the intention that there could be other implementations > in the future? > > KT3. DefaultConsumerGroupMetadata should be defined in an "internals" > package of some sort, right? Will users ever reference the implementation > class name in their code? I'm assuming not. > > Thanks! > Kirk > > On Tue, Feb 25, 2025, at 8:24 AM, Paweł Szymczyk wrote: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1136%3A+Make+ConsumerGroupMetadata+an+interface > > > > -- > > Pozdrawiam > > Paweł Szymczyk > > > -- Pozdrawiam Paweł Szymczyk