7 days is long enough to catch most (all?) malicious attacks. Regarding developers, there's a strong case to be made for only doing builds and especially tests in isolated containers, even though artifacts will leak across shared containers through a shared maven repo. It still limits the damage malicious binaries can do.
On Tue, 21 Apr 2026 at 23:58, Jungtaek Lim <[email protected]> wrote: > +1 > > We tend to consider that merging to master branch gives some time to bake > before releasing. But we (Spark devs) are people who build Spark and > run some tests against the master branch almost day to day. For us, there > is literally no time for these library upgrades to be baked - we are > exposed to any kind of potential CVE from these library upgrades. > > It's arguable whether we should stay up to date with the recent release > version for dependencies, but that'd probably be uneasy to make consensus; > there is a clear trade-off. The current proposal sounds to me as a good > compromise - IMHO delaying by 2 weeks (14 days) seems reasonable, but > strict 1 week (7 days) is better than nothing if anyone is concerned 2 > weeks is too long. > > On Tue, Apr 21, 2026 at 9:45 PM Szehon Ho <[email protected]> wrote: > >> +1 make sense to me as well. We should of course be fast for security >> upgrades, but make sense to avoid such eager upgrades for the rest of >> the hundreds of Spark dependencies, due to the increased supply chain >> attack risks in the ecosystem. >> >> Thanks >> Szehon >> >> On Tue, Apr 21, 2026 at 3:32 AM Wenchen Fan <[email protected]> wrote: >> >>> Thanks for starting this discussion! I did a data analysis a while ago >>> but didn't have time to act on it. The analysis shows: >>> >>> *58* maven dep upgrades in the last 3 months. >>> *46%* (27/58) within 7 days of release >>> ≤7d : 27 / 58 (47%) >>> 8d–30d : 12 / 58 (21%) >>> >30d : 19 / 58 (32%) >>> >>> You can find the raw data in the attached file. This does look a bit >>> aggressive. I build Spark locally everyday, and I believe I'm not the only >>> one. Having a couple of weeks as the buffer time is a good idea to protect >>> developers like me from potential supply chain attacks. >>> >>> On Tue, Apr 21, 2026 at 6:24 AM Hyukjin Kwon <[email protected]> >>> wrote: >>> >>>> SGTM I think it's good practice to give a couple of weeks before the >>>> upgrade >>>> >>>> On Tue, 21 Apr 2026 at 07:13, Tian Gao via dev <[email protected]> >>>> wrote: >>>> >>>>> Hi, I want to start a discussion about our dependency upgrade policy >>>>> for active development. >>>>> >>>>> Our current dependency upgrade (mostly for Java, but Python should be >>>>> included too) is a bit spontaneous. People find that a dependency has a >>>>> new >>>>> version available and we just do the upgrade. >>>>> >>>>> This raises concerns about potential supply chain attacks. We already >>>>> established a few sets of rules (including pinning the github action >>>>> versions) to avoid the supply chain attack, but manually upgrading the >>>>> dependency version too eagerly could also be risky. >>>>> >>>>> It normally takes time for a bad release to be recognized, so I think >>>>> we should set a buffer time before upgrading to the latest version. For >>>>> example, we can wait a week or two after the latest release before we set >>>>> our development dependency to it. This could reduce the possibility of >>>>> being impacted by malicious releases, or just give them enough time to fix >>>>> their own severe bugs. >>>>> >>>>> The cost for this policy is very low - it barely impacts us if we >>>>> can’t use the “latest” version of dependencies. >>>>> >>>>> Of course, there should be exceptions when dependency upgrades include >>>>> security fixes for known vulnerabilities; we should upgrade as fast as >>>>> possible. >>>>> >>>>> Tian >>>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >> >>
