Re: Github build queue

2024-02-09 Thread David Arthur
I tried to enable the merge queue on my public fork, but the option is not available. I did a little searching and it looks like ASF does not allow this feature to be used. I've filed an INFRA ticket to ask again https://issues.apache.org/jira/browse/INFRA-25485 -David On Fri, Feb 9, 2024 at 7:18

Re: Github build queue

2024-02-09 Thread Ismael Juma
Also, on the mockito stubbings point, we did upgrade to Mockito 5.8 for the Java 11 and newer builds: https://github.com/apache/kafka/blob/trunk/gradle/dependencies.gradle#L64 So, we should be good when it comes to that too. Ismael On Fri, Feb 9, 2024 at 4:15 PM Ismael Juma wrote: > Nice! > >

Re: Github build queue

2024-02-09 Thread Ismael Juma
Nice! Ismael On Fri, Feb 9, 2024 at 3:43 PM Greg Harris wrote: > Hey all, > > I implemented a fairly aggressive PR [1] to demote flaky tests to > integration tests, and the end result is a much faster (10m locally, > 1h on Jenkins) build which is also very reliable. > > I believe this would mak

Re: Github build queue

2024-02-09 Thread Greg Harris
Hey all, I implemented a fairly aggressive PR [1] to demote flaky tests to integration tests, and the end result is a much faster (10m locally, 1h on Jenkins) build which is also very reliable. I believe this would make unitTest suitable for use in the merge queue, with the caveat that it doesn't

Re: Github build queue

2024-02-09 Thread Ismael Juma
Please check https://github.com/apache/kafka/pull/14186 before making the `unitTest` and `integrationTest` split. Ismael On Fri, Feb 9, 2024 at 9:16 AM Josep Prat wrote: > Regarding "Split our CI "test" job into unit and integration so we can > start collecting data on those suites", can we run

Re: Github build queue

2024-02-09 Thread Greg Harris
Hey Ismael, I took David's statement here: > Split our CI "test" job into unit and integration so we can start collecting > data on those suites to include moving all the flaky tests to the integrationTest target by adding the annotation. We can do that while the merge queue is just running the

Re: Github build queue

2024-02-09 Thread David Arthur
> Regarding "Split our CI "test" job into unit and integration I believe all of the "steps" inside the "stage" directive are run on the same node sequentially. I think we could do something like steps { doValidation() doUnitTest() doIntegrationTest() tryStreamsArchetype() } and it should

Re: Github build queue

2024-02-09 Thread Josep Prat
Regarding "Split our CI "test" job into unit and integration so we can start collecting data on those suites", can we run these 2 tasks in the same machine? So they won't need to compile classes twice for the same exact code? On Fri, Feb 9, 2024 at 6:05 PM Ismael Juma wrote: > Why can't we add @

Re: Github build queue

2024-02-09 Thread Ismael Juma
Why can't we add @Tag("integration") for all of those tests? Seems like that would not be too hard. Ismael On Fri, Feb 9, 2024 at 9:03 AM Greg Harris wrote: > Hi David, > > +1 on that strategy. > > I see several flaky tests that aren't marked with @Tag("integration") > or @IntegrationTest, and

Re: Github build queue

2024-02-09 Thread Greg Harris
Hi David, +1 on that strategy. I see several flaky tests that aren't marked with @Tag("integration") or @IntegrationTest, and I think those would make using the unitTest target ineffective here. We could also start a new tag @Tag("flaky") and exclude that. Thanks, Greg On Fri, Feb 9, 2024 at 8:

Re: Github build queue

2024-02-09 Thread David Arthur
I do think we can add a PR to the merge queue while bypassing branch potections (like we do for the Merge button today), but I'm not 100% sure. I like the idea of running unit tests, though I don't think we have data on how long just the unit tests run on Jenkins (since we run the "test" target whi

Re: Github build queue

2024-02-09 Thread Josep Prat
Hi David, I like the idea, it will solve the problem we've seen a couple of times in the last 2 weeks where compilation for some Scala version failed, it was probably overlooked during the PR build because of the flakiness of tests and the compilation failure was buried among the amount of failed t

Re: Github build queue

2024-02-09 Thread Ismael Juma
Hi David, I think this is a helpful thing (and something I hoped we would use when I learned about it), but it does require the validation checks to be reliable (or else the PR won't be merged). Sounds like you are suggesting to skip the tests for the merge queue validation. Could we perhaps inclu

Re: Github build queue

2024-02-09 Thread Chris Egerton
+1, would love this. On Fri, Feb 9, 2024, 10:04 David Arthur wrote: > Hey folks, > > I recently learned about Github's Merge Queue feature, and I think it could > help us out. > > Essentially, when you hit the Merge button on a PR, it will add the PR to a > queue and let you run a CI job before

Github build queue

2024-02-09 Thread David Arthur
Hey folks, I recently learned about Github's Merge Queue feature, and I think it could help us out. Essentially, when you hit the Merge button on a PR, it will add the PR to a queue and let you run a CI job before merging. Just something simple like compile + static analysis would probably save u