Hi all, I would like to start a discussion on accelerating the Apache Spark release cadence. Over the past four months, we have been running preview releases, and the process has been smooth and effective. As mentioned in the preview release discussion thread, I’d now like to extend this approach to official releases.
During this period, I also looked into how other large projects, such as Kubernetes and Python, manage their release timelines. Based on that research and our own recent experience, I’ve drafted a proposal for an updated Apache Spark release plan. TL;DR: - Introduce a predictable release schedule: annual major releases and quarterly minor releases, so users can benefit from new features earlier. - With a faster cadence for minor releases, we should take a more conservative approach toward behavior changes in minor versions, while still allowing new features and improvements. I’d love to hear your thoughts and feedback. More details can be found in SPIP: Accelerating Apache Spark Release Cadence <https://docs.google.com/document/d/1gBoZ4KH5zQUWpgK3M7zAN7p6Glz4S_e9bO3PvQA9sQs/edit?usp=sharing> Thanks!
