-1 There was a thread leak in MLTransactionMetadataStore which also impacts production code. It is fixed by https://github.com/apache/pulsar/pull/14524 which is now merged to master branch. The thread leak resulted in a memory leak in tests and that's how I noticed it.
The thread leak was also visible in test output. In Pulsar tests we have a detector for thread leaks, it is https://github.com/apache/pulsar/blob/master/buildtools/src/main/java/org/apache/pulsar/tests/ThreadLeakDetectorListener.java . It will log threads that are active when the test ends. For example, when running mvn -DredirectTestOutputToFile=false -DtestRetryCount=0 test -Pcore-modules -pl pulsar-broker -DexcludedGroups= -Dtest=ReplicatorTest#testDoNotReplicateSystemTopic The leak detector logs warnings to the console: [INFO] [stdout] 2022-03-02T13:37:21,120+0200 [main] WARN org.apache.pulsar.tests.ThreadLeakDetectorListener - Tests in class ReplicatorTest created thread id 1407 with name 'Thread[transaction_coordinator_TransactionCoordinatorID(id=0)thread_factory-468-1,5,main]' [INFO] [stdout] 2022-03-02T13:37:21,120+0200 [main] WARN org.apache.pulsar.tests.ThreadLeakDetectorListener - Tests in class ReplicatorTest created thread id 1406 with name 'Thread[transaction_coordinator_TransactionCoordinatorID(id=1)thread_factory-467-1,5,main]' I hope this helps catch similar issues in the future earlier. -Lari On 2022/03/01 12:35:04 PengHui Li wrote: > This is the second release candidate for Apache Pulsar 2.10.0 > > It fixes the following issues: > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.10.0 > +is%3Amerged+-label%3Arelease%2F2.9.1+-label%3Arelease%2F2.9.2 > > *** Please download, test and vote on this release. This vote will stay open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.10.0-candidate-2/ > > SHA-512 checksums: > > 6fde1a547bb0f6b640afaaf5480f45b47b89000f9b6cd85cd00984f8078909d185e52ee753de49cf631ab0097d8785f5c52bcd953b3c0762e3df074aa8b0280f > apache-pulsar- > 2.10.0-bin.tar.gz > > a879be88f9605350f2851b815bff5013124c948e9b614f5e0e91c909ba7cba0c67920d705f97e3ae8b278e5354971e2ddda6a7d77dd788b0a448f8ade35510e8 > apache-pulsar- > 2.10.0-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1144/ > > The tag to be voted upon: > v2.10.0-candidate-2 (43a9b6fe9f5bca055acfefe144cd78dd29c614ba) > https://github.com/apache/pulsar/releases/tag/v2.10.0-candidate-2 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Docker images: > > https://hub.docker.com/layers/194902199/lph890127/pulsar/2.10.0-rc2/images/sha256-75af729b88c2a63193a4414ca10fc97f7092bf83abe30a37c3d57bef9287e9b2?context=repo > > https://hub.docker.com/layers/194901702/lph890127/pulsar-all/2.10.0-rc2/images/sha256-22155d193897029d74faa7a95098fa15198984809b9aa3decd5b961be66437ff?context=repo > > Please download the source package, and follow the Release Candidate > Validation[1] > to validate the release > > [1] https://github.com/apache/pulsar/wiki/Release-Candidate-Validation > > Thanks, > Penghui >
