Finally all passing and merged! Thanks for the reviews. The docs for running the integration tests with Mac M1/M2/M3 is at https://github.com/apache/bookkeeper/tree/master/tests#workaround-for-running-the-tests-with-mac-apple-silicon . The plan is to replace Arquillian Cube with Testcontainers and Junit 5 for the integration tests. Until that is in place, a workaround is needed for Mac OS on Apple Silicon.
-Lari On 2024/02/08 07:56:47 Lari Hotari wrote: > After 91 commits, I'm very close in getting integration tests running and > fixed in CI. > > Integration tests have been ignored in apache/bookkeeper CI for a very long > time. Here's some evidence in the comment I added to #3675: > https://github.com/apache/bookkeeper/pull/3675#issuecomment-1933485215 > To get integration tests to pass, I had to fix a bug in "bkctl ledger > simpletest" and it appears that it was already merged as invalid in November > 2022. 😱 > > The good news is that integration tests are now passing in the PR builds for > https://github.com/apache/bookkeeper/pull/4197. 😮💨 > > -Lari > > On 2024/02/06 23:10:35 Lari Hotari wrote: > > After some night hacking (58 commits), I'm finally making progress in > > fixing Bookkeeper CI. It's a long story to explain all details, so it's > > better to just check the PR: > > https://github.com/apache/bookkeeper/pull/4197 > > > > Btw. I also found the reason why tests didn't run for the integration tests > > at all. Maven surefire had picked TestNG provider for some reason and > > didn't find any tests. Perhaps some dependency pulled in TestNG and that > > caused the trouble. > > > > Besides fixing a lot of issues across a lot of different areas related to > > the integration & backwards compatibility tests, I also documented how the > > old Arquillian Cube Docker tests > > can be run on Mac Apple Silicon (M1/M2/M3). > > https://github.com/lhotari/bookkeeper/blob/3e13e0247d1c570c7cae67dadf200d6f3b095d7a/tests/README.md > > This helped resolve the remaining issues since I could simple debug things > > in my local development environment on Mac M3. > > > > The plan is to replace Arquillian Cube with Testcontainers & Junit 5 after > > this PR has been completed with reviews and merged. > > Any other volunteers that would be interested in participating in this work? > > > > -Lari > > > > On 2024/02/02 14:21:30 Lari Hotari wrote: > > > It has been a rabbit hole to fix the CI issues. > > > I have put a lot of commits into > > > https://github.com/apache/bookkeeper/pull/4197/commits to handle many > > > problems. The integration tests run now, but there are failures. I didn't > > > have time to continue investigating and fixing problems. I'll continue > > > later. > > > > > > I noticed that the circe-checksum native library has been broken too. > > > Fixing it required adding this to nar-maven-plugin's configuration's cpp > > > part: > > > <includes> > > > <include>**/*.cpp</include> > > > </includes> > > > > > > I wonder if others have noticed these problems? > > > > > > -Lari > > > > > > On 2024/02/01 19:44:02 Lari Hotari wrote: > > > > Thanks Nicolo. > > > > > > > > > I'd suggest to just disable them at this point > > > > > > > > Why would you recommend that? > > > > > > > > I wonder what has been causing tests to be skipped in the Maven build? > > > > In the Junit 5 PR, I have upgraded maven-surefire-plugin and tests get > > > > run with Junit 5's vintage engine. Is there a known bug in older maven > > > > plugin versions that tests get skipped without notice? > > > > > > > > -Lari > > > > > > > > On 2024/02/01 19:38:09 Nicolò Boschi wrote: > > > > > Hi Lari, > > > > > They haven't run for years now. > > > > > > > > > > When we migrated to Gradle, I made them to work again. See > > > > > https://lists.apache.org/thread/p9twnlh75kp7t74d8ghmtqbw1f5v4kgk > > > > > > > > > > > > > > > But after switching back to Maven, nobody took care of them and no > > > > > attempts > > > > > have been made to make them runnable again > > > > > > > > > > I'd suggest to just disable them at this point > > > > > > > > > > Nicolò Boschi > > > > > > > > > > > > > > > Il giorno gio 1 feb 2024 alle ore 20:23 Lari Hotari > > > > > <lhot...@apache.org> ha > > > > > scritto: > > > > > > > > > > > Hi, > > > > > > > > > > > > I'm currently working on a PR to enable writing tests with Junit 5 > > > > > > and > > > > > > having Junit 5 tests in addition to Junit 4 tests. This is in PR > > > > > > https://github.com/apache/bookkeeper/pull/4197 > > > > > > > > > > > > Suddenly there are test failures! I first thought that the upgrade > > > > > > had > > > > > > broken something, but it turns out that my changes fix problems and > > > > > > start > > > > > > running tests that were previously ignored! > > > > > > > > > > > > Previously, integration tests ran in 29 seconds! > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > [INFO] BUILD SUCCESS > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > [INFO] Total time: 29.054 s > > > > > > [INFO] Finished at: 2024-01-31T02:32:17Z > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > Test current server with old clients: > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > [INFO] BUILD SUCCESS > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > [INFO] Total time: 5.464 s > > > > > > [INFO] Finished at: 2024-01-31T02:28:31Z > > > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > The docker container tests using arquillian-cube-docker have been > > > > > > skipped > > > > > > completely. I managed to get arquillian-cube-docker to run, but it > > > > > > turns > > > > > > out that the build isn't properly setup for this. I guess I put > > > > > > myself in a > > > > > > rabbit hole... I need some help. > > > > > > > > > > > > Could someone share more context about the state of Bookkeeper CI? > > > > > > Any ongoing effort to improve it? > > > > > > > > > > > > Thanks > > > > > > > > > > > > -Lari > > > > > > > > > > > > > > > > > > > > >
