Thanks @congbo for helping confirm the issue is not a regression in 2.10.0 https://github.com/apache/pulsar/issues/14436#issuecomment-1064758710
I will start a new RC soon. Penghui On Fri, Mar 4, 2022 at 11:38 AM PengHui Li <peng...@apache.org> wrote: > Hi all, > > We found an issue https://github.com/apache/pulsar/issues/14436, > This may be a new problem introduced in 2.10.0, need to confirm > the issue and rollout the new RC. > > Thanks, > Penghui > > On Wed, Mar 2, 2022 at 9:20 PM PengHui Li <peng...@apache.org> wrote: > >> Thanks Lari, >> >> I have cherry-picked https://github.com/apache/pulsar/pull/14524 into >> branch-2.10 and will >> send out the new RC tomorrow. >> >> Penghui >> >> On Wed, Mar 2, 2022 at 8:30 PM Lari Hotari <lhot...@apache.org> wrote: >> >>> -1 >>> >>> There was a thread leak in MLTransactionMetadataStore which also impacts >>> production code. >>> It is fixed by https://github.com/apache/pulsar/pull/14524 which is now >>> merged to master branch. The thread leak resulted in a memory leak in tests >>> and that's how I noticed it. >>> >>> The thread leak was also visible in test output. In Pulsar tests we have >>> a detector for thread leaks, it is >>> https://github.com/apache/pulsar/blob/master/buildtools/src/main/java/org/apache/pulsar/tests/ThreadLeakDetectorListener.java >>> . It will log threads that are active when the test ends. For example, when >>> running >>> >>> mvn -DredirectTestOutputToFile=false -DtestRetryCount=0 test >>> -Pcore-modules -pl pulsar-broker -DexcludedGroups= >>> -Dtest=ReplicatorTest#testDoNotReplicateSystemTopic >>> >>> The leak detector logs warnings to the console: >>> >>> [INFO] [stdout] 2022-03-02T13:37:21,120+0200 [main] WARN >>> org.apache.pulsar.tests.ThreadLeakDetectorListener - Tests in class >>> ReplicatorTest created thread id 1407 with name >>> 'Thread[transaction_coordinator_TransactionCoordinatorID(id=0)thread_factory-468-1,5,main]' >>> [INFO] [stdout] 2022-03-02T13:37:21,120+0200 [main] WARN >>> org.apache.pulsar.tests.ThreadLeakDetectorListener - Tests in class >>> ReplicatorTest created thread id 1406 with name >>> 'Thread[transaction_coordinator_TransactionCoordinatorID(id=1)thread_factory-467-1,5,main]' >>> >>> I hope this helps catch similar issues in the future earlier. >>> >>> -Lari >>> >>> On 2022/03/01 12:35:04 PengHui Li wrote: >>> > This is the second release candidate for Apache Pulsar 2.10.0 >>> > >>> > It fixes the following issues: >>> > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.10.0 >>> > +is%3Amerged+-label%3Arelease%2F2.9.1+-label%3Arelease%2F2.9.2 >>> > >>> > *** Please download, test and vote on this release. This vote will >>> stay open >>> > for at least 72 hours *** >>> > >>> > Note that we are voting upon the source (tag), binaries are provided >>> for >>> > convenience. >>> > >>> > Source and binary files: >>> > >>> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.10.0-candidate-2/ >>> > >>> > SHA-512 checksums: >>> > >>> > >>> 6fde1a547bb0f6b640afaaf5480f45b47b89000f9b6cd85cd00984f8078909d185e52ee753de49cf631ab0097d8785f5c52bcd953b3c0762e3df074aa8b0280f >>> > apache-pulsar- >>> > 2.10.0-bin.tar.gz >>> > >>> > >>> a879be88f9605350f2851b815bff5013124c948e9b614f5e0e91c909ba7cba0c67920d705f97e3ae8b278e5354971e2ddda6a7d77dd788b0a448f8ade35510e8 >>> > apache-pulsar- >>> > 2.10.0-src.tar.gz >>> > >>> > Maven staging repo: >>> > >>> https://repository.apache.org/content/repositories/orgapachepulsar-1144/ >>> > >>> > The tag to be voted upon: >>> > v2.10.0-candidate-2 (43a9b6fe9f5bca055acfefe144cd78dd29c614ba) >>> > https://github.com/apache/pulsar/releases/tag/v2.10.0-candidate-2 >>> > >>> > Pulsar's KEYS file containing PGP keys we use to sign the release: >>> > https://dist.apache.org/repos/dist/dev/pulsar/KEYS >>> > >>> > Docker images: >>> > >>> > >>> https://hub.docker.com/layers/194902199/lph890127/pulsar/2.10.0-rc2/images/sha256-75af729b88c2a63193a4414ca10fc97f7092bf83abe30a37c3d57bef9287e9b2?context=repo >>> > >>> > >>> https://hub.docker.com/layers/194901702/lph890127/pulsar-all/2.10.0-rc2/images/sha256-22155d193897029d74faa7a95098fa15198984809b9aa3decd5b961be66437ff?context=repo >>> > >>> > Please download the source package, and follow the Release Candidate >>> > Validation[1] >>> > to validate the release >>> > >>> > [1] https://github.com/apache/pulsar/wiki/Release-Candidate-Validation >>> > >>> > Thanks, >>> > Penghui >>> > >>> >>
