Hi Giann, I thought this could be done fully automated at first before the agent created several very big PRs that contained changes of hundreds. So I changed the way how we migrate these test cases:
1. Use agent to Identify dependencies of changes and split changes into multiple PRs so that one PR does not contain lots of file changes and PRs don't conflict with another This took some human effort, I have to review the dependencies, plans proposed by AI, guide agents to revise plans. The issue #13948 <https://github.com/apache/druid/issues/13948> records the dependencies and how we can migrate test cases in parallel. 2. Use agent to do migration and create changes without my intervention and all CI problems are fixed by agent automatically without intervention. This is the most work that is automated, less human effort is involved. 3. Review the changes manually and ask agent to do optimize This is where most human effort was in. The human effort here is not about correctness(because we already have AI reviews to ensure it), but to ensure agents always use the same coding rules across all test cases, for example, using the extracted shared API. I can't remember how many hours I spent, but it should not be too big. 4. Use agent to identity master branch CI failures including flaky tests, JUnit 5 performance regression introduced by ParameterizedClass Human effort was also involved, because I have to review the findings/causes reported by the AI, and reviewed the changes proposed by AI Because the total number of PRs are not big, the effort is acceptable. 5. Use agent to Make sure shared code are extracted and ensure all test cases to use these shared APIs (like TemporaryExtension) Although the whole work spanned over more than 15 days, it didn't mean that I spent effort everyday on this. If I have to say the hours I spent on it, maybe 1-2 hours a day on average. On Mon, Sep 7, 2026 at 10:30 AM Gian Merlino <[email protected]> wrote: > Thank you for writing this up and for driving this work. I wonder, how > much effort was it for you? Do you do much in terms of watching over the > work, guiding it, etc, or is it pretty much automated? > > Gian > > On 2026/08/24 14:56:11 Frank Chen wrote: > > Hi everyone, > > > > The JUnit 5 migration is now complete; the final PR was merged this > > afternoon. I would like to share a brief summary of the journey, the main > > changes, and one known caveat. > > A long-running effort > > > > This work began with issue #13948 > > <https://github.com/apache/druid/issues/13948>, proposed by > @UngureanuSerg > > on March 20, 2023. > > > > The first migration PR, #16106 < > https://github.com/apache/druid/pull/16106>, > > opened by @kgyrtkirk was merged on March 19, 2024. > > > > In April 2026, nearly two years later, @clintropolis resumed the effort > by > > migrating druid-processing tests across 10 PRs. > > > > I had not initially planned to take on the remaining work. However, a > > reminder that new test cases should use JUnit 5 prompted me to revisit > the > > migration last month. Since I had sufficient model budget available, I > > decided to complete the remaining work. > > Main changes > > > > - Removed JUnit 4, Vintage, JUnitParams, migration-support, and > obsolete > > test-rule dependencies. > > - Retained Hamcrest and opentest4j only where active tests still > require > > them. > > - Consolidated shared testing utilities under > org.apache.druid.testing. > > - Updated AGENTS.md to guide contributors and coding agents toward > JUnit > > 5 for new tests. > > > > Known caveat: @ParameterizedClass > > > > One known caveat is JUnit 5’s @ParameterizedClass. Depending on how it is > > used, class-level parameterization may introduce performance overhead. > > > > Where practical, @ParameterizedTest is preferred. For more technical > > details, see: > > > > - #19859 <https://github.com/apache/druid/pull/19859> > > - #20088 <https://github.com/apache/druid/pull/20088> > > - #20108 <https://github.com/apache/druid/pull/20108> > > > > I did not convert every @ParameterizedClass usage to > > @ParameterizedTest. Although > > AI can automate much of this conversion, doing so everywhere would have > > increased review effort and broadened the scope of the migration. > > Review and execution > > > > During the migration, @GWphua <https://github.com/GWphua> reviewed and > > approved all changes, which significantly accelerated the work so that > the > > whole migration can be done in 3 weeks. Many thanks for the thorough > > reviews and support. > > > > For transparency, the implementation and validation work was performed > with > > GPT-5.6 Luna models at maximum reasoning effort. Approximately at least > 50 > > billion model tokens were used. > > Migration PRs > > > > The following list contains the migration-related PRs tracked by #13948. > > PRs #14488 <https://github.com/apache/druid/pull/14488> and #19911 > > <https://github.com/apache/druid/pull/19911> were closed without > merging; > > their work was superseded by later PRs. > > > > - #14488 <https://github.com/apache/druid/pull/14488> — Druid server > > cache package migration to JUnit 5 (initial commit) @UngureanuSerg > > - #16106 <https://github.com/apache/druid/pull/16106> — Update > > Calcite*Test to use JUnit 5 @kgyrtkirk > > - #19289 <https://github.com/apache/druid/pull/19289> — Convert > > druid-processing tests to JUnit 5, pt. 1 @clintropolis > > - #19299 <https://github.com/apache/druid/pull/19299> — Convert > > druid-processing tests to JUnit 5, pt. 2 @clintropolis > > - #19302 <https://github.com/apache/druid/pull/19302> — Convert > > druid-processing tests to JUnit 5, pt. 3 @clintropolis > > - #19313 <https://github.com/apache/druid/pull/19313> — Convert > > druid-processing tests to JUnit 5, pt. 4 @clintropolis > > - #19327 <https://github.com/apache/druid/pull/19327> — Convert > > druid-processing tests to JUnit 5, pt. 5 @clintropolis > > - #19346 <https://github.com/apache/druid/pull/19346> — Convert > > druid-processing tests to JUnit 5, pt. 6 @clintropolis > > - #19360 <https://github.com/apache/druid/pull/19360> — Convert > > druid-processing tests to JUnit 5, pt. 7 @clintropolis > > - #19387 <https://github.com/apache/druid/pull/19387> — Convert > > druid-processing tests to JUnit 5, pt. 8 @clintropolis > > - #19601 <https://github.com/apache/druid/pull/19601> — Convert > > druid-processing tests to JUnit 5, pt. 9 @clintropolis > > - #19875 <https://github.com/apache/druid/pull/19875> — Migrate > > extensions-core and processing tests to JUnit 5 > > - #19876 <https://github.com/apache/druid/pull/19876> — Migrate > > extension-core format and lookup tests to JUnit 5 > > - #19877 <https://github.com/apache/druid/pull/19877> — Migrate > > extension-core catalog stats and security tests to JUnit 5 > > - #19878 <https://github.com/apache/druid/pull/19878> — Migrate > > extension-core Google, Kafka, and lookup tests to JUnit 5 > > - #19879 <https://github.com/apache/druid/pull/19879> — Migrate > > histogram extension tests to JUnit 5 > > - #19880 <https://github.com/apache/druid/pull/19880> — Migrate S3 > > extension tests to JUnit 5 > > - #19881 <https://github.com/apache/druid/pull/19881> — Migrate > > DataSketches extension tests to JUnit 5 > > - #19882 <https://github.com/apache/druid/pull/19882> — Convert > > druid-processing tests to JUnit 5, pt. 10 @clintropolis > > - #19908 <https://github.com/apache/druid/pull/19908> — Migrate nine > > extensions-contrib modules to JUnit 5 > > - #19909 <https://github.com/apache/druid/pull/19909> — Migrate SQL > leaf > > and Quidem tests to JUnit 5 > > - #19910 <https://github.com/apache/druid/pull/19910> — Migrate task > > actions and compaction tests to JUnit 5 > > - #19911 <https://github.com/apache/druid/pull/19911> — Migrate > server > > tests to JUnit 5 > > - #19915 <https://github.com/apache/druid/pull/19915> — Migrate the > > second extensions-contrib batch to JUnit 5 > > - #19916 <https://github.com/apache/druid/pull/19916> — Migrate the > > third extensions-contrib batch to JUnit 5 > > - #19917 <https://github.com/apache/druid/pull/19917> — Migrate the > > final extensions-contrib batch to JUnit 5 > > - #19920 <https://github.com/apache/druid/pull/19920> — Migrate > common > > support tests to JUnit 5 > > - #19922 <https://github.com/apache/druid/pull/19922> — Migrate > Overlord > > tests to JUnit 5 > > - #19923 <https://github.com/apache/druid/pull/19923> — Migrate > > seekable-stream tests to JUnit 5 > > - #19925 <https://github.com/apache/druid/pull/19925> — Migrate > worker > > tests to JUnit 5 > > - #19926 <https://github.com/apache/druid/pull/19926> — Migrate > > ingestion and parallel tests to JUnit 5 > > - #19939 <https://github.com/apache/druid/pull/19939> — Migrate SQL > core > > and schema tests to JUnit 5 > > - #19940 <https://github.com/apache/druid/pull/19940> — Migrate > > processing batch 1 to JUnit 5 > > - #19980 <https://github.com/apache/druid/pull/19980> — Migrate > > processing query tests to JUnit 5 > > - #19981 <https://github.com/apache/druid/pull/19981> — Migrate > > processing segment tests to JUnit 5 > > - #19982 <https://github.com/apache/druid/pull/19982> — Migrate > > processing batch 2 to JUnit 5 > > - #20031 <https://github.com/apache/druid/pull/20031> — Migrate the > > first server test batch to JUnit 5 > > - #20057 <https://github.com/apache/druid/pull/20057> — Migrate > scoped > > server tests to JUnit 5 > > - #20058 <https://github.com/apache/druid/pull/20058> — Migrate > > coordination and service tests to JUnit 5 > > - #20059 <https://github.com/apache/druid/pull/20059> — Migrate > > coordinator tests to JUnit 5 > > - #20061 <https://github.com/apache/druid/pull/20061> — Migrate PR3 > > server tests to JUnit 5 > > - #20062 <https://github.com/apache/druid/pull/20062> — Migrate > scoped > > server tests to JUnit 5 > > - #20064 <https://github.com/apache/druid/pull/20064> — Migrate MSQ > > tests to JUnit 5 > > - #20125 <https://github.com/apache/druid/pull/20125> — Finish the > > remaining JUnit 5 migration cleanup > > > > Thanks to everyone who proposed, implemented, reviewed, and supported > this > > long-running effort. > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
