Hi Lari, > The "CI - Integration - Cli / cli" build job passes cleanly for 2.7.4 . This > means that it's very likely that there's a regression in 2.7.5 . I'll > investigate the issue.
Any progress on this? Thanks, Haiting On 2022/02/16 10:54:14 Lari Hotari wrote: > I'll vote again with -1. > > The "CI - Integration - Cli / cli" build job passes cleanly for 2.7.4 . This > means that it's very likely that there's a regression in 2.7.5 . I'll > investigate the issue. > > -Lari > > On 2022/02/16 10:21:23 Lari Hotari wrote: > > The branch-2.7 build didn't complete cleanly for the release commit > > https://github.com/apache/pulsar/commit/ed40840a65c413202c0aab6b9b1a29ed370f6fb7 > > . > > (click on the red cross icon before the text "Release 2.7.5" to see the > > build results popup) > > > > There are 3 failing build jobs: > > - 2 for "CI - Build - Multiple - OS / build" > > - "CI - Integration - Cli / cli" > > > > It's safe to ignore the failures for "CI - Build - Multiple - OS / build" > > since that doesn't run tests and the build was also failing in the 2.7.4 > > release commit: > > https://github.com/apache/pulsar/commit/ab451b855d873a9bad2005f939a23118a583baa9 > > (click on the red cross icon to see build results) > > > > I am running the failing "CI - Integration - Cli / cli" build job locally > > with the command "mvn -f tests/pom.xml test > > -DintegrationTestSuiteFile=pulsar-cli.xml -DintegrationTests > > -DredirectTestOutputToFile=false" (after building the 2.7.5 docker images > > and docker test image locally). > > > > The test doesn't complete cleanly. This is the error message in integration > > tests: > > [integration] [INFO] [stdout] > > org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty > > bookies available > > [integration] [INFO] [stdout] 08:20:16.613 > > [bookkeeper-ml-workers-OrderedExecutor-3-0] WARN > > org.apache.pulsar.broker.service.ServerCnx - > > [/172.19.0.8:54068][persistent://compaction-test-rest-wmxw/ns1/topic1][sub1] > > Failed to create consumer: consumerId=0, > > org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty > > bookies available > > > > I can see this error message in the bookies (docker exec -it > > CLITest-gmwag-pulsar-bookie-1 cat /var/log/pulsar/bookie.log) > > 08:10:20.616 [bookie-io-1-9] ERROR org.apache.bookkeeper.proto.BookieServer > > - Unable to allocate memory, exiting bookie > > io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 > > byte(s) of direct memory (used: 536870912, max: 536870912) > > at > > io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:802) > > ~[io.netty-netty-common-4.1.68.Final.jar:4.1.68.Final] > > > > This is the flaky test issue https://github.com/apache/pulsar/issues/9622 > > fixed by PR https://github.com/apache/pulsar/pull/9623 which was missing > > from branch-2.7 . > > After cherry-picking PR 9623, I can run CLI integration tests locally. > > There were 3 test failures: > > - > > org.apache.pulsar.tests.integration.cli.ClusterMetadataTearDownTest#testDeleteCluster > > java.lang.AssertionError: expected [0] but found [18] > > at > > org.apache.pulsar.tests.integration.cli.ClusterMetadataTearDownTest.testDeleteCluster(ClusterMetadataTearDownTest.java:122) > > - > > org.apache.pulsar.tests.integration.cli.CLITest#testCreateSubscriptionCommand > > org.apache.pulsar.tests.integration.docker.ContainerExecException: > > /pulsar/bin/pulsar-admin topics create-subscription > > persistent://public/default/testCreateSubscriptionCommmand --subscription > > subscription-0 failed on > > e03e5466c18da298a834ddbeeac9e6c80e583c27b28ecbac0bd472f148696a1b with error > > code 1 > > - > > org.apache.pulsar.tests.integration.compaction.TestCompaction#testPublishCompactAndConsumeCLI > > org.testng.internal.thread.ThreadTimeoutException: Method > > org.apache.pulsar.tests.integration.compaction.TestCompaction.testPublishCompactAndConsumeCLI() > > didn't finish within the time-out 300000 > > > > I'll run the integration tests locally for 2.7.4 release to see if there's > > a regression or whether it's just the tests that are flaky. > > > > > > -Lari > > > > > > > > > > > > On 2022/02/15 13:54:18 PengHui Li wrote: > > > Hi Lari > > > > > > The PR you shared https://github.com/apache/pulsar/pull/14240 does not run > > > any tests. > > > All the tests were completed in 1 min, please check. > > > > > > Thanks, > > > Penghui > > > > > > > > > > > > On Tue, Feb 15, 2022 at 7:23 PM Lari Hotari <lhot...@apache.org> wrote: > > > > > > > The tests passed for the a PR based on revision a27e0853bda in > > > > branch-2.7 > > > > in this PR that I made to run the tests: > > > > https://github.com/apache/pulsar/pull/14240 . rev a27e0853bda is the > > > > last > > > > commit before the 2.7.5 release commit in branch-2.7 . > > > > > > > > I'll count your vote as +1 since all tests have passed. > > > > > > > > -Lari > > > > > > > > On 2022/02/15 00:36:28 PengHui Li wrote: > > > > > - Verified the signature > > > > > - Start the standalone and checked message publish and consumption > > > > > - Validate Cassandra connector > > > > > - Validate stateful function > > > > > > > > > > Hi Lari, I have checked the CI status for branch-2.7 > > > > > https://github.com/apache/pulsar/commits/branch-2.7 > > > > > There are 4 failed tests, could you please help check the failed test > > > > > and > > > > > make sure > > > > > the tests get passed for 2.7.5? Since we don't run any checks before > > > > > or > > > > > after cherry-picking, > > > > > I usually check the CI after cherry-picking, if there are failed > > > > > tests, > > > > > need to confirm if it related to breaking changes > > > > > or a flaky test to retry. > > > > > > > > > > If all tests pass, please treat my validation as +1 > > > > > > > > > > Thanks, > > > > > Penghui > > > > > > > > > > On Mon, Feb 14, 2022 at 9:12 PM Haiting Jiang > > > > > <jianghait...@apache.org> > > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > Checks: > > > > > > > > > > > > - Checksum and signatures > > > > > > > > > > > > - Compile from source w JDK11 > > > > > > > > > > > > - Validate Pub/Sub and Java Functions [1] > > > > > > > > > > > > - Validate Connectors [1] > > > > > > > > > > > > - Validate Stateful Functions [1] > > > > > > > > > > > > [1] > > > > > > https://github.com/apache/pulsar/wiki/Release-Candidate-Validation > > > > > > > > > > > > Thanks, > > > > > > Haiting > > > > > > > > > > > > On 2022/02/11 13:47:50 Lari Hotari wrote: > > > > > > > This is the first release candidate for Apache Pulsar, version > > > > > > > 2.7.5. > > > > > > > > > > > > > > It contains the following commits after the previous release: > > > > > > > https://github.com/apache/pulsar/compare/v2.7.4...v2.7.5-candidate-1 > > > > > > > > > > > > > > *** Please download, test and vote on this release. This vote will > > > > stay > > > > > > open > > > > > > > for at least 72 hours *** > > > > > > > > > > > > > > Note that we are voting upon the source (tag), binaries are > > > > > > > provided > > > > for > > > > > > > convenience. > > > > > > > > > > > > > > Source and binary files: > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.5-candidate-1/ > > > > > > > > > > > > > > SHA-512 checksums: > > > > > > > > > > > > > > > > > > > > > > > > b0d4e8c05870e0fe8cb50a80e15811b59972fc7f1d3fa93fa6cf2b61797176e80b434334aa67a5619de2a46a6ae9e51286dad524cac7af76239cda790d3574cc > > > > > > apache-pulsar-2.7.5-bin.tar.gz > > > > > > > > > > > > > > > > > e9a88d6847828fdb051fe2a4663e5b66beb0a066e9e170682df4e0330fe5d59d27c88908f493e2100f35a2469a8f77a97c94da752aac320bf4413aaed57570d6 > > > > > > apache-pulsar-2.7.5-src.tar.gz > > > > > > > > > > > > > > Maven staging repo: > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachepulsar-1138/ > > > > > > > > > > > > > > The tag to be voted upon: > > > > > > > v2.7.5-candidate-1 (db8761ebb370db1ae731a807afb583ac346378fe) > > > > > > > https://github.com/apache/pulsar/releases/tag/v2.7.5-candidate-1 > > > > > > > > > > > > > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > > > > > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > > > > > > > > > > > Please download the source package, and follow the README to build > > > > > > > and run the Pulsar standalone service. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
