Here's an update on the Pulsar 4.0 release timeline. We are currently slightly behind the original schedule; however, we still have a good chance of keeping the release schedule on track without major slippage.
There are currently a few release blockers for 4.0 (tagged issues: https://github.com/apache/pulsar/labels/release%2Fblocker). The most important one is about handling the Protobuf CVE-2024-7254: https://github.com/apache/pulsar/issues/23341. That CVE is categorized as high (8.7/10). It's a potential denial-of-service issue that doesn't pose a practical risk for Pulsar users. Since it's in the high category, we must address it before the release. Addressing CVE-2024-7254 will require a Bookkeeper release since protobuf-java can only be upgraded in Pulsar when protobuf-java is first upgraded in Bookkeeper together with a compatible grpc-java version. The protobuf-java cross-version runtime guarantee is documented at https://protobuf.dev/support/cross-version-runtime-guarantee/. There is no real guarantee across versions for protoc-generated stub classes. A similar policy applies to grpc stubs generated with protoc-gen-grpc-java. A specific grpc-java version is compatible with certain protobuf-java versions. There's more information about this coupling in the Bookkeeper dev mailing list thread: https://lists.apache.org/thread/odg7p617zwqjngq6fk6qf8xfzbfwgfgq. Unfortunately, I haven't had the chance to put in the effort to break the dependency cycles. One additional complexity with Pulsar is the Pulsar release where all Pulsar and Bookkeeper dependencies are placed in a single "lib" directory. All dependencies must be compatible since at runtime both Pulsar and Bookkeeper will contain the same jar files on the classpath. In addition to just plain classes, Pulsar adds the Metadata Store implementations which Bookkeeper also uses. That's how Bookkeeper can also use the same Metadata Store implementations as Pulsar. There's a plan to decouple the Pulsar client from the protobuf-java version so that client applications could choose to use a protobuf-java version that is compatible with their own protoc-generated stub classes. The issue to track this is https://github.com/apache/pulsar/issues/22263. There are currently issues with the Pulsar shaded client shading configurations which include protobuf-java in the shaded jar as classes. This prevents client applications from using other protobuf-java versions. This should be fixed before 4.0. Besides the release blockers, I'd like to have an implementation ready for "PIP-379: Key_Shared Draining Hashes for Improved Message Ordering" as part of Pulsar 4.0, given that the proposal is ready to be voted upon and we have sufficient confidence in including it as part of Pulsar 4.0. There's also another PIP that I'll propose to finish the ManagedLedger cleanup work that was started as part of the email thread: https://lists.apache.org/thread/b2f4vkql693x3frdxm75tndk686crh9k. The last part is https://github.com/apache/pulsar/pull/23313, which is not yet complete. I'll document the details about the ManagedLedger cleanup in a new PIP. Please reply to this email thread if there are other important PIPs that aren't yet completed but community members see as valuable additions to Pulsar 4.0 LTS. I can see that "PIP-381: Handle large PositionInfo state" is a good candidate to be included too. I hope that the small detail of having the feature flag to control the feature could be considered during the implementation. That will allow opting out from using the feature due to risk management concerns. For example, that might be useful in ensuring that possible rollbacks are smooth and a user might want to enable the feature only after migrating to Pulsar 4.0 and having their environment running for some time. Now finally to the Pulsar 4.0 release. I'm going to rename the first two releases as "preview releases" since we don't intend to release them or vote on them. It's currently a very confusing part of our release process documented at https://pulsar.apache.org/contribute/release-policy/#release-cycles. I'll update the process documentation too. I was confused when Pulsar 3.2.0 preview releases were running: https://lists.apache.org/thread/mlj710kwgn00zx9mmc4c8bshnvc5bfo4. I'm volunteering to handle the release manager role for Pulsar 4.0. Here's the updated schedule. One of the unresolved parts is how to handle the Bookkeeper release so that we can address Protobuf CVE-2024-7254 before Pulsar 4.0 is released. The target publishing date of the 4.0 release candidate 1 is the same date as it was in the previous timeline, Oct 10th. The clarification here is that this is the publishing date of release candidate 1. We'll set the target date for announcing 4.0.0 on Oct 17th. 2024-09-26 - Target publishing date for 4.0 preview 1 2024-10-03 - Target publishing date for 4.0 preview 2 2024-10-07 - Code freeze for 4.0 by branching branch-4.0 from the master branch 2024-10-10 - Target publishing date for 4.0 release candidate 1 2024-10-15 - Reserved for 4.0 release candidate 2 if it's needed 2024-10-17 - Announcement date for 4.0.0 I'll start a discussion on the Bookkeeper dev mailing list about releases that include Protobuf 3.25.5 to address CVE-2024-7254 together with a compatible grpc-java version. Since we are currently using Bookkeeper 4.17.1 in the master branch, it might be the safest option to continue using this release branch for Pulsar 4.0.0. Please reply to this thread if you have any feedback/questions or would like to volunteer in addressing the remaining release blockers of 4.0. We have busy weeks ahead in the Apache Pulsar project, lets roll up our sleeves and get to work! -Lari On Wed, 11 Sept 2024, 7.43 Lari Hotari, <lhot...@apache.org> wrote: > > Hi all, > > I'm volunteering to handle the release manager role for Pulsar 4.0. > For the timeline, I'll suggest a slight update. Instead of having 3 release > candidates, we would have 2 planned release candidates. > The target release date remains Oct 10th. > > 2024-09-23 - Code freeze for 4.0 by branching branch-4.0 from the master > branch > 2024-09-26 - Target release date for 4.0 release candidate 1 > 2024-10-03 - Target release date for 4.0 release candidate 2 > 2024-10-10 - Target release date for 4.0 > > Regards, > > -Lari > > On 2024/08/09 08:36:05 Lari Hotari wrote: > > Hi all, > > > > We have the next LTS release, Pulsar 4.0, scheduled for October. > > For LTS, releases, our release policy states a 3 week recycle [1]. > > > > Suggested timeline: > > 2024-08-09 - Create & merge PR to bump master branch's version to > > 4.0.0-SNAPSHOT > > 2024-09-15 - code freeze for 4.0 by branching branch-4.0 from master branch > > 2024-09-19 - target release date of 4.0 release candidate 1 > > 2024-09-26 - target release date of 4.0 release candidate 2 > > 2024-10-03 - target release date of 4.0 release candidate 3 > > 2024-10-10 - target release date of 4.0 > > > > Unless there's another proposal for the timeline, let's do a lazy consensus > > decision on this timeline in the next 72 hours. We can always adjust this > > if there's a need to do so. > > > > -Lari > > > > 1 - https://pulsar.apache.org/contribute/release-policy/#release-cycles > >
