Hi all,
This picks up the thread Hongyu started in November 2023, "How about
enable dependabot?":
https://lists.apache.org/thread/1p8346rm8bpmkhgzjgvd78o7bg86cc9l
That thread agreed on the goal and stalled on the mechanism. Julian
said we should stay on recent versions, since a mature library's
latest release is
more likely to fix a security problem than to introduce a bug, but
objected to Dependabot's noise and to restructuring the build files,
and asked:
"Is there a straightforward way to script those upgrades?" Stamatis
reported Hive's experience with the bot: 52 PRs, 34 with a failing
build, 3 merged,
and named the condition that decides it, that someone has to follow up
on them. Guillaume suggested a monthly schedule to cut the noise.
Francis suggested refreshVersions. Hongyu and Jiajun offered to do the work.
Since then, 32 of the 111 dependency and plugin versions in
gradle.properties have moved and 79 have not. 33 still hold the value
CALCITE-2905
gave them when the build moved to Gradle in November 2019. Immutables
is on 2.8.8 from September 2021 (2.12.2 is current),
Checker Framework on 3.10.0 (4.2.2), Error Prone on 2.5.1 (2.50.0),
Checkstyle on 8.28 (14.1.0).
I would like to answer Julian's question directly: yes, and the script
is Renovate, configured to run on our release cadence rather than on a
calendar.
1. Renovate once per release cycle, not continuously
The noise objection is a configuration question, and Renovate is the
tool that lets us configure it away. Its schedule takes a cron
expression,
so a run can be pinned to the weeks before a release rather than to
every Monday. minimumReleaseAge holds a version back until it has been
public
for a set number of days, which keeps a bad release out of the queue.
And a group rule puts every non-major update into one PR,
leaving separate PRs only for the majors that need individual judgement.
On our cadence of two releases a year, that is one batched "bump
dependencies" PR per cycle plus a handful of majors.
That is Hongyu's PR 3504, on a timer, with CI attached. It automates
the process Julian proposed in that thread rather than replacing it.
refreshVersions reports which newer versions exist, which is useful,
but it does not open a PR that CI can fail.
The expensive question is not what is available, it is what still
passes, and only a PR answers that.
If people would rather have Dependabot, everything below still applies.
Renovate is my preference for the scheduling, batching and configurability.
2. The version catalog is the blocker Hongyu already found
Hongyu noted in 2023 that Dependabot cannot read gradle.properties, so
the build files would have to change. That is still true, and it holds
for Renovate too,
for a specific reason: a coordinate and its version live in different
files and are joined at configuration time.
bom/build.gradle.kts writes apiv("com.google.guava:guava") and the
helper appends guava.version. The version never appears next to the
coordinate, so no tool has a pair to match on.
"Restructuring the build files" here means adopting one standard
Gradle feature, gradle/libs.versions.toml, which is the format both
bots parse natively.
It lets us delete the .v() and apiv() helpers we maintain ourselves.
It also stands on its own: it makes a manual sweep before a release
easier whether or not a bot ever runs.
3. Somebody has to follow up, which is Stamatis's point and the real risk
Hive's numbers are the thing to answer, not to argue with. As a
comparison on a Gradle project of similar size, apache/solr has run
Renovate since February 2023:
802 bot PRs, 398 merged, and none open today. The unmerged half is
largely Renovate closing its own PRs once a newer version supersedes
them, not 400 rejections.
Their volume, 265 PRs in the last year, is higher than I would want
here, since they run it on two branches and batch less aggressively
than I am proposing.
That only works because someone tends it, so here is the concrete
offer. I will triage the bot's output for its first two runs, one per
release cycle,
which on the current cadence is about a year. Triage means reading
every PR it opens, separating a real regression from pre-existing debt
and from a flaky test,
fixing or closing accordingly, and reporting back to this list at the
end with what it cost and what it caught. If it looks like Hive by
then, we turn the bot off, and the catalog stays either way.
4. What I am asking
I have been away from the project for a while, so please read this as
a proposal rather than a plan.
- Is the version catalog welcome on its own merits, independent of any
bot? If so, I will file a JIRA and send a PR covering one module
first, so the shape is visible before anything wide happens.
- If a bot is worth another try, what cadence would people accept:
once per release cycle, quarterly, or monthly?
I will not push any part of this over an open objection.
Vladimir
