Thanks for starting this discussion! I did a data analysis a while ago but didn't have time to act on it. The analysis shows:
*58* maven dep upgrades in the last 3 months. *46%* (27/58) within 7 days of release ≤7d : 27 / 58 (47%) 8d–30d : 12 / 58 (21%) >30d : 19 / 58 (32%) You can find the raw data in the attached file. This does look a bit aggressive. I build Spark locally everyday, and I believe I'm not the only one. Having a couple of weeks as the buffer time is a good idea to protect developers like me from potential supply chain attacks. On Tue, Apr 21, 2026 at 6:24 AM Hyukjin Kwon <[email protected]> wrote: > SGTM I think it's good practice to give a couple of weeks before the > upgrade > > On Tue, 21 Apr 2026 at 07:13, Tian Gao via dev <[email protected]> > wrote: > >> Hi, I want to start a discussion about our dependency upgrade policy for >> active development. >> >> Our current dependency upgrade (mostly for Java, but Python should be >> included too) is a bit spontaneous. People find that a dependency has a new >> version available and we just do the upgrade. >> >> This raises concerns about potential supply chain attacks. We already >> established a few sets of rules (including pinning the github action >> versions) to avoid the supply chain attack, but manually upgrading the >> dependency version too eagerly could also be risky. >> >> It normally takes time for a bad release to be recognized, so I think we >> should set a buffer time before upgrading to the latest version. For >> example, we can wait a week or two after the latest release before we set >> our development dependency to it. This could reduce the possibility of >> being impacted by malicious releases, or just give them enough time to fix >> their own severe bugs. >> >> The cost for this policy is very low - it barely impacts us if we can’t >> use the “latest” version of dependencies. >> >> Of course, there should be exceptions when dependency upgrades include >> security fixes for known vulnerabilities; we should upgrade as fast as >> possible. >> >> Tian >> >
dependency,new_version,dep_released,spark_merged_utc,lag_days,spark_jira,pr_url vertx-core,4.5.26,2026-03-26,2026-03-25,-1,SPARK-56209,https://github.com/apache/spark/pull/55013 netty-all,4.2.12.Final,2026-03-26,2026-03-26,0,SPARK-56214,https://github.com/apache/spark/pull/55016 lz4-java,1.10.4,2026-03-02,2026-03-03,1,SPARK-55803,https://github.com/apache/spark/pull/54585 aircompressor,2.0.3,2026-02-24,2026-02-25,1,SPARK-55688,https://github.com/apache/spark/pull/54486 xbean-asm9-shaded,4.30,2026-01-28,2026-01-29,1,SPARK-55233,https://github.com/apache/spark/pull/53997 kubernetes-client,7.5.1,2026-01-15,2026-01-16,1,SPARK-55068,https://github.com/apache/spark/pull/53833 commons-pool2,2.13.1,2026-01-06,2026-01-07,1,SPARK-54912,https://github.com/apache/spark/pull/53689 RoaringBitmap,1.6.10,2026-02-23,2026-02-24,1,SPARK-55616,https://github.com/apache/spark/pull/54393 objenesis,3.5,2026-01-26,2026-01-28,2,SPARK-55255,https://github.com/apache/spark/pull/54032 jackson-dataformat-yaml,2.21.2,2026-03-20,2026-03-23,3,SPARK-56156,https://github.com/apache/spark/pull/54956 snowflake-jdbc,4.0.2,2026-03-13,2026-03-16,3,SPARK-56010,https://github.com/apache/spark/pull/54831 jaxb-runtime,4.0.7,2026-03-14,2026-03-17,3,SPARK-56013,https://github.com/apache/spark/pull/54890 jetty-util-ajax,12.1.7,2026-03-04,2026-03-07,3,SPARK-55478,https://github.com/apache/spark/pull/54261 kubernetes-client,7.6.0,2026-03-02,2026-03-05,3,SPARK-55850,https://github.com/apache/spark/pull/54634 netty-all,4.2.10.Final,2026-02-05,2026-02-08,3,SPARK-55420,https://github.com/apache/spark/pull/54203 kubernetes-client,7.5.2,2026-01-23,2026-01-26,3,SPARK-55166,https://github.com/apache/spark/pull/53949 jackson-dataformat-yaml,2.21.0,2026-01-19,2026-01-22,3,SPARK-55116,https://github.com/apache/spark/pull/53886 orc-core,2.2.2,2026-01-09,2026-01-12,3,SPARK-54979,https://github.com/apache/spark/pull/53743 arrow-vector,19.0.0,2026-03-12,2026-03-17,5,SPARK-56000,https://github.com/apache/spark/pull/54820 kubernetes-client,7.6.1,2026-03-05,2026-03-10,5,SPARK-55936,https://github.com/apache/spark/pull/54733 lz4-java,1.10.3,2026-01-21,2026-01-26,5,SPARK-55189,https://github.com/apache/spark/pull/53971 log4j-slf4j2-impl,2.25.4,2026-03-25,2026-03-31,6,SPARK-56307,https://github.com/apache/spark/pull/55114 orc-core,2.3.0,2026-02-25,2026-03-03,6,SPARK-55685,https://github.com/apache/spark/pull/54481 protobuf-java,4.33.5,2026-01-29,2026-02-04,6,SPARK-55309,https://github.com/apache/spark/pull/54090 blas,3.1.1,2026-02-13,2026-02-19,6,SPARK-55605,https://github.com/apache/spark/pull/54380 zstd-jni,1.5.7-7,2026-02-04,2026-02-10,6,SPARK-55456,https://github.com/apache/spark/pull/54233 postgresql,42.7.10,2026-02-11,2026-02-18,7,SPARK-55573,https://github.com/apache/spark/pull/54347 scala-maven-plugin,4.9.9,2026-01-23,2026-01-30,7,SPARK-55276,https://github.com/apache/spark/pull/54057 mockito-core,5.23.0,2026-03-12,2026-03-20,8,SPARK-56098,https://github.com/apache/spark/pull/54915 jackson-dataformat-yaml,2.21.1,2026-02-23,2026-03-04,9,SPARK-55841,https://github.com/apache/spark/pull/54633 byte-buddy,1.18.4,2026-01-16,2026-01-27,11,SPARK-55232,https://github.com/apache/spark/pull/53996 byte-buddy-agent,1.18.4,2026-01-16,2026-01-27,11,SPARK-55232,https://github.com/apache/spark/pull/53996 hadoop-client-api,3.4.3,2026-02-12,2026-02-25,13,SPARK-54276,https://github.com/apache/spark/pull/54029 xz,1.12,2026-03-01,2026-03-17,16,SPARK-56012,https://github.com/apache/spark/pull/54833 mysql-connector-j,9.6.0,2026-01-29,2026-02-18,20,SPARK-55575,https://github.com/apache/spark/pull/54349 commons-codec,1.21.0,2026-01-23,2026-02-13,21,SPARK-55513,https://github.com/apache/spark/pull/54304 parquet-column,1.17.0,2025-12-22,2026-01-14,23,SPARK-54822,https://github.com/apache/spark/pull/53582 zookeeper,3.9.5,2026-02-11,2026-03-09,26,SPARK-55894,https://github.com/apache/spark/pull/54698 icu4j,78.2,2026-01-08,2026-02-03,26,SPARK-55308,https://github.com/apache/spark/pull/54089 gcs-connector,hadoop3-2.2.31,2025-12-10,2026-01-06,27,SPARK-54913,https://github.com/apache/spark/pull/53690 kafka-clients,3.9.2,2026-02-07,2026-03-10,31,SPARK-55940,https://github.com/apache/spark/pull/54735 junit-jupiter,6.0.3,2026-02-15,2026-03-20,33,SPARK-56104,https://github.com/apache/spark/pull/54920 jetty-util-ajax,12.1.5,2025-12-05,2026-01-09,35,SPARK-47086,https://github.com/apache/spark/pull/53116 log4j-slf4j2-impl,2.25.3,2025-12-15,2026-01-22,38,SPARK-55130,https://github.com/apache/spark/pull/53912 netty-tcnative-boringssl-static,2.0.75.Final,2026-02-04,2026-03-17,41,SPARK-56009,https://github.com/apache/spark/pull/54830 bcprov-jdk18on,1.83,2025-11-26,2026-01-06,41,SPARK-54911,https://github.com/apache/spark/pull/53688 compress-lzf,1.2.0,2026-01-02,2026-02-13,42,SPARK-55508,https://github.com/apache/spark/pull/54292 ojdbc17,23.26.1.0.0,2026-01-30,2026-03-25,54,SPARK-56183,https://github.com/apache/spark/pull/54984 asm,9.9.1,2025-12-06,2026-01-29,54,SPARK-55233,https://github.com/apache/spark/pull/53997 mariadb-java-client,3.5.7,2025-12-16,2026-02-18,64,SPARK-55574,https://github.com/apache/spark/pull/54348 analyticsaccelerator-s3,1.3.1,2025-11-11,2026-01-28,78,SPARK-55254,https://github.com/apache/spark/pull/54031 tink,1.20.0,2025-12-09,2026-03-17,98,SPARK-56008,https://github.com/apache/spark/pull/54829 commons-cli,1.11.0,2025-11-08,2026-02-25,109,SPARK-55677,https://github.com/apache/spark/pull/54471 jersey-server,3.1.11,2025-08-08,2026-01-09,154,SPARK-47086,https://github.com/apache/spark/pull/53116 guava,33.5.0-jre,2025-09-17,2026-02-25,161,SPARK-55656,https://github.com/apache/spark/pull/54447 jjwt-api,0.13.0,2025-08-20,2026-02-13,177,SPARK-55515,https://github.com/apache/spark/pull/54306 jakarta.servlet-api,6.0.0,2022-05-12,2026-01-09,1338,SPARK-47086,https://github.com/apache/spark/pull/53116
--------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
