Hi all, We found a blocker in 3.8.0-rc1 https://issues.apache.org/jira/browse/KAFKA-17150 that is now merged to trunk and backported to 3.8. Additionally I've merged and backported the lower-severity https://issues.apache.org/jira/browse/KAFKA-17148 to trunk, 3.8, and 3.7 because Dmitry Werner provided a very prompt fix. Thanks to Chia-Ping, Chris, and Josep for prompt reviews. At this time I am not aware of any more blockers for 3.8.0 and we can proceed with rc2.
Thanks! Greg On Wed, Jul 10, 2024 at 10:49 PM Josep Prat <josep.p...@aiven.io.invalid> wrote: > Hi Greg, > > I'll make sure the PR with the fix ( > https://github.com/apache/kafka/pull/16565) gets merged today. Thanks for > bringing this up! > > Best, > > ------------------ > Josep Prat > Open Source Engineering Director, Aiven > josep.p...@aiven.io | +491715557497 | aiven.io > Aiven Deutschland GmbH > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > Amtsgericht Charlottenburg, HRB 209739 B > > On Wed, Jul 10, 2024, 22:55 Greg Harris <greg.har...@aiven.io.invalid> > wrote: > > > Hi Josep, > > > > A contributor just raised a regression [1] that I think should be > addressed > > in 3.8.0 prior to the release. > > > > Summary: This change [2] causes multiple ERROR logs to appear during > worker > > startup when operators install other, unrelated plugins that package the > > Jackson library. > > Severity: Workers will continue to operate normally and the errors will > be > > cosmetic. Operators and automatic systems that watch logs may roll-back > > upgrades due to the perception of a severe problem. > > Impact: I found 12 out of 250 third-party plugins that package the > Jackson > > library and trigger this error on upgrade. This will almost certainly > > affect several users upon upgrades, and does not require obscure setups. > > Risk: The contributor has opened a simple fix PR [3], and I have verified > > that it addresses the problem, and can be merged tomorrow. As an > > alternative, we can revert the performance change completely [4] but I > want > > to avoid this. > > > > With the above, what would you like to do for this release? Merge the > fix, > > revert, or leave as-is? > > > > Thanks, > > Greg > > > > [1] https://issues.apache.org/jira/browse/KAFKA-17111 > > [2] https://issues.apache.org/jira/browse/KAFKA-15996 > > [3] https://github.com/apache/kafka/pull/16565 > > [4] https://github.com/apache/kafka/pull/16568 > > > > On Wed, Jul 10, 2024 at 7:37 AM Mickael Maison <mickael.mai...@gmail.com > > > > wrote: > > > > > Hi Dongjin, > > > > > > It's great to see you back! > > > I hope what we did with KIP-390 matches your expectations. Looking > > > forward to seeing the reboot of KIP-780. > > > > > > Thanks, > > > Mickael > > > > > > On Wed, Jul 10, 2024 at 4:21 PM Dongjin Lee <dong...@apache.org> > wrote: > > > > > > > > Hi Josep, > > > > > > > > OMG, what happened while I could not be involved with the Kafka > > > community? > > > > Thanks for digging down the whole situation. > > > > > > > > @Mickael I greatly appreciate your effort in finalizing the feature. > It > > > > seems like I only have to re-boot the KIP-780. > > > > > > > > Thanks, > > > > Dongjin > > > > > > > > On Tue, Jul 9, 2024 at 12:52 AM Josep Prat > <josep.p...@aiven.io.invalid > > > > > > > wrote: > > > > > > > > > Hi Dongjin, > > > > > > > > > > KIP-390 is part of the 3.8 release because the JIRA associated with > > it: > > > > > https://issues.apache.org/jira/browse/KAFKA-7632 is closed as > > > resolved, > > > > > hence the KIP is declared done and ready. I did some digging, and I > > saw > > > > > that Mickael was the one doing the PR that closed the JIRA ticket: > > > > > https://github.com/apache/kafka/pull/15516 > > > > > This means that the KIP work is merged and unfortunately it is now > > > quite > > > > > late to perform a rollback for this feature. > > > > > > > > > > @Mickael Maison <mickael.mai...@gmail.com> let me know if > anything I > > > > > mentioned is not accurate (as you were the one bringing the KIP to > > > > > completion). > > > > > > > > > > Best, > > > > > > > > > > On Mon, Jul 8, 2024 at 5:38 PM Dongjin Lee <dong...@apache.org> > > wrote: > > > > > > > > > > > Hi Josep, > > > > > > > > > > > > Thanks for managing the 3.8 release. I have a request: could you > > > please > > > > > > move the KIP-390 into the 3.9 release? > > > > > > > > > > > > Here is the background: KIP-390 was adopted first but hasn't been > > > > > released > > > > > > for a long time. After some time, I proposed KIP-780 with further > > > > > > improvements and also corrected an obvious design error > > > > > > (`compression.level` → `compression.(gzip|lz4|zstd). level`), but > > it > > > > > hasn't > > > > > > been adopted due to the community's lack of response, my changing > > > job, > > > > > > focusing the in-house fork, etc. And last weekend, I found that > > > KIP-380 > > > > > has > > > > > > been included in the 3.8 release plan. > > > > > > > > > > > > - KIP-390: > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level > > > > > > - KIP-780: > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-780%3A+Support+fine-grained+compression+options > > > > > > > > > > > > However, shipping those two features at once has the following > > > benefits: > > > > > > > > > > > > 1. Full functionality without design error. > > > > > > > > > > > > We can provide full functionality, particularly useful with > tiered > > > > > storage > > > > > > feature at once. I found that several users of tiered storage use > > > > > > server-side recompression and want to improve the compression > > > efficiency. > > > > > > Of course, it does not include any design errors :) > > > > > > > > > > > > 2. More chance of testing. > > > > > > > > > > > > Currently, I am managing an in-house fork of Apache Kafka and > > Cruise > > > > > > Control[^1], running on thousands of clusters on k8s. With our > > > ongoing > > > > > work > > > > > > on the tiered storage plugin, we can test both KIPs at once. > Since > > > we are > > > > > > planning to move the terabytes of logs from thousands of > > > microservices > > > > > into > > > > > > the object storage, some of them can be ideal testbeds. > > > > > > > > > > > > If you are okay, I will re-initiate the discussion of KIP-780 and > > > rework > > > > > > KIP-380 on the latest trunk. > > > > > > > > > > > > Thanks, > > > > > > Dongjin > > > > > > > > > > > > [^1]: For example: > > > https://github.com/linkedin/cruise-control/pull/2145 > > > > > > > > > > > > On Mon, Feb 26, 2024 at 8:38 PM Josep Prat > > > <josep.p...@aiven.io.invalid> > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'd like to volunteer as release manager for the Apache Kafka > > 3.8.0 > > > > > > > release. > > > > > > > If there are no objections, I'll start building a release plan > > (or > > > > > > adapting > > > > > > > the one Colin made some weeks ago) in the wiki in the next > days. > > > > > > > > > > > > > > Thank you. > > > > > > > > > > > > > > -- > > > > > > > [image: Aiven] <https://www.aiven.io> > > > > > > > > > > > > > > *Josep Prat* > > > > > > > Open Source Engineering Director, *Aiven* > > > > > > > josep.p...@aiven.io | +491715557497 > > > > > > > aiven.io <https://www.aiven.io> | < > > > > > > https://www.facebook.com/aivencloud > > > > > > > > > > > > > > > <https://www.linkedin.com/company/aiven/> < > > > > > > > https://twitter.com/aiven_io> > > > > > > > *Aiven Deutschland GmbH* > > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > *Dongjin Lee* > > > > > > > > > > > > *A hitchhiker in the mathematical world.* > > > > > > > > > > > > > > > > > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > > > > > <https://github.com/dongjinleekr>keybase: > > > > > https://keybase.io/dongjinleekr > > > > > > <https://keybase.io/dongjinleekr>linkedin: > > > > > kr.linkedin.com/in/dongjinleekr > > > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > > > > > > speakerdeck.com/dongjin > > > > > > <https://speakerdeck.com/dongjin>* > > > > > > > > > > > > > > > > > > > > > -- > > > > > [image: Aiven] <https://www.aiven.io> > > > > > > > > > > *Josep Prat* > > > > > Open Source Engineering Director, *Aiven* > > > > > josep.p...@aiven.io | +491715557497 > > > > > aiven.io <https://www.aiven.io> | < > > > https://www.facebook.com/aivencloud > > > > > > > > > > > <https://www.linkedin.com/company/aiven/> < > > > > > https://twitter.com/aiven_io> > > > > > *Aiven Deutschland GmbH* > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > -- > > > > *Dongjin Lee* > > > > > > > > *A hitchhiker in the mathematical world.* > > > > > > > > > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > > > <https://github.com/dongjinleekr>keybase: > > > https://keybase.io/dongjinleekr > > > > <https://keybase.io/dongjinleekr>linkedin: > > > kr.linkedin.com/in/dongjinleekr > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > > > speakerdeck.com/dongjin > > > > <https://speakerdeck.com/dongjin>* > > > > > >
