[GitHub] [pulsar-adapters] aditiwari01 commented on issue #29: Pulsar - Spark adapter for scala 2.11
aditiwari01 commented on issue #29: URL: https://github.com/apache/pulsar-adapters/issues/29#issuecomment-997713940 The current available spark pulsar adapter seems to be unreliable. I've written a reliable connector with addition consideration for back-pressure and rate limit. Currently testing around it. Can raise a PR if that's fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: Status of Pulsar 2.9.0 and starting 2.9.1
Hi Enrico, Thanks for your contribution. 1) A soft reminder that is not on the current release process [a]: Since 2.9.0 was delayed, some doc changes on the master are applied to 2.10.0 only, so generating the 2.9.0 doc set should be based on the "2.9.0 release time point" rather than the current master, or else the 2.9.0 doc set might contain 2.10 feature docs. 2) 2.9.0 is not announced and the 2.9.0 release note is available on the website. For the previous releases, we have tech blogs for them to (announce and) explain highlights [b], will you create a tech blog for 2.9.0? [a] https://github.com/apache/pulsar/wiki/Release-process#16-update-the-site [b] https://pulsar.apache.org/blog/2021/09/23/Apache-Pulsar-2-8-1/ On Fri, Dec 17, 2021 at 7:36 PM Enrico Olivelli wrote: > Il giorno ven 17 dic 2021 alle ore 09:45 Yu ha scritto: > > > Hi Enrico, > > > > Thanks for your great effort on the 2.9.0 release. > > > > Circling back to see if there is any progress of the 2.9.0 website > updates > > [1]. > > > > Currently, the 2.9.0 doc is not available on the website and the 2.9.0 > doc > > set has not been generated yet, any updates? Thanks > > > > > I will do it together with 2.9.1 > because 2.9.0 is broken, so it is better to not do much buzz around it > > Enrico > > > > > > [1] > > https://github.com/apache/pulsar/wiki/Release-process#16-update-the-site > > > > > > On Mon, Dec 13, 2021 at 3:32 PM Enrico Olivelli > > wrote: > > > > > Dave, > > > You are correct. > > > Pulsar 2.9.0 has already been released and also some people already > > started > > > to report issues. > > > The docker images have been deployed and we cannot change them. > > > > > > I am finishing the release process for 2.9.0 with the website updates. > > > > > > I am preparing 2.9.1. > > > > > > I propose to just skip the announcement for 2.9.0. > > > > > > If we are quick during the VOTE we can close this story within the end > of > > > the week > > > > > > Enrico > > > > > > Il Lun 13 Dic 2021, 06:34 Dave Fisher ha > > scritto: > > > > > > > (1) we have published 2.9.0 at > > > > https://downloads.apache.org/pulsar/pulsar-2.9.0/ > > > > > > > > (2) we have published 2.9.0 artifacts through maven central. They > don’t > > > > let anyone republish versions. > > > > > > > > There are no do overs on versions. We simply cannot redo 2.9.0 at > this > > > > moment. > > > > > > > > All the best, > > > > Dave > > > > > > > > Sent from my iPhone > > > > > > > > > On Dec 12, 2021, at 8:49 PM, Sijie Guo wrote: > > > > > > > > > > My take is - if we haven't announced 2.9, I would suggest just > > redoing > > > > the > > > > > 2.9.0 release. > > > > > > > > > > - Sijie > > > > > > > > > >> On Sun, Dec 12, 2021 at 6:35 PM Hang Chen > > > wrote: > > > > >> > > > > >> I am a little confused about why we should skip 2.9.0 and not > > continue > > > > >> to release 2.9.0 by including the critical bug fixes. In fact, the > > > > >> Pulsar 2.9.0 release is not yet completed. > > > > >> > > > > >> For users, they will worry about whether the Pulsar release > process > > is > > > > >> standardized if we skip 2.9.0. They will also worry about the > > release > > > > >> quality of Apache Pulsar if we have found the critical bugs before > > it > > > > >> is released but not included it into the release version. For > Pulsar > > > > >> 2.9.0, it couldn't be deployed into the production environment due > > to > > > > >> the critical bug https://github.com/apache/pulsar/pull/12993 > > > > >> > > > > >> Regards, > > > > >> Hang > > > > >> > > > > >> Dave Fisher 于2021年12月13日周一 09:40写道: > > > > >>> > > > > >>> It can be the case that releases are not announced. For example > > with > > > > >> Tomcat a version which fails to pass the vote is skipped. > > > > >>> > > > > >>> Let’s not announce 2.9.0 and go on to 2.9.1. > > > > >>> > > > > >>> Maybe there’s some website fixes to hide 2.9.0. > > > > >>> > > > > >>> > > > > >>> Sent from my iPhone > > > > >>> > > > > On Dec 12, 2021, at 5:28 PM, PengHui Li > > wrote: > > > > > > > > Another point is we have not announced the 2.9.0 release yet. > > > > > > > > This will make users feel confused that a new release from the > > > Pulsar > > > > community with the > > > > serious problem(log4j bug) but after the log4j has fixed the > issue > > > and > > > > provided the new release. > > > > > > > > I think we'd better contain the fix in 2.9.0 and 2.9.0 also has > a > > > > >> critical > > > > bug https://github.com/apache/pulsar/pull/12993 > > > > which will lead the topic stop to provide services for more than > > > 5min. > > > > >> It > > > > looks like, hey, we have a new release here but > > > > it has critical security issues and known serious bugs which > will > > > > >> seriously > > > > affect the core features. > > > > > > > > From the perspective of release, yes, the release vote has > closed. > > > But > > > > >> I > >
Re: Status of Pulsar 2.9.0 and starting 2.9.1
Yu, Il giorno lun 20 dic 2021 alle ore 10:01 Yu ha scritto: > Hi Enrico, > > Thanks for your contribution. > > 1) A soft reminder that is not on the current release process [a]: > Since 2.9.0 was delayed, some doc changes on the master are applied to > 2.10.0 only, so generating the 2.9.0 doc set should be based on the "2.9.0 > release time point" rather than the current master, or else the 2.9.0 doc > set might contain 2.10 feature docs. > This is actually a good point. I am not sure about how I can fix this. Maybe I can try to create the versioned docs from the v2.9.0 tag > > 2) 2.9.0 is not announced and the 2.9.0 release note is available on the > website. > For the previous releases, we have tech blogs for them to (announce and) > explain highlights [b], will you create a tech blog for 2.9.0? > I am sorry but I won't have time to do this before the beginning of January, it would be useful if someone could do it Enrico > > [a] > https://github.com/apache/pulsar/wiki/Release-process#16-update-the-site > [b] https://pulsar.apache.org/blog/2021/09/23/Apache-Pulsar-2-8-1/ > > > On Fri, Dec 17, 2021 at 7:36 PM Enrico Olivelli > wrote: > > > Il giorno ven 17 dic 2021 alle ore 09:45 Yu ha > scritto: > > > > > Hi Enrico, > > > > > > Thanks for your great effort on the 2.9.0 release. > > > > > > Circling back to see if there is any progress of the 2.9.0 website > > updates > > > [1]. > > > > > > Currently, the 2.9.0 doc is not available on the website and the 2.9.0 > > doc > > > set has not been generated yet, any updates? Thanks > > > > > > > > > I will do it together with 2.9.1 > > because 2.9.0 is broken, so it is better to not do much buzz around it > > > > Enrico > > > > > > > > > > [1] > > > > https://github.com/apache/pulsar/wiki/Release-process#16-update-the-site > > > > > > > > > On Mon, Dec 13, 2021 at 3:32 PM Enrico Olivelli > > > wrote: > > > > > > > Dave, > > > > You are correct. > > > > Pulsar 2.9.0 has already been released and also some people already > > > started > > > > to report issues. > > > > The docker images have been deployed and we cannot change them. > > > > > > > > I am finishing the release process for 2.9.0 with the website > updates. > > > > > > > > I am preparing 2.9.1. > > > > > > > > I propose to just skip the announcement for 2.9.0. > > > > > > > > If we are quick during the VOTE we can close this story within the > end > > of > > > > the week > > > > > > > > Enrico > > > > > > > > Il Lun 13 Dic 2021, 06:34 Dave Fisher ha > > > scritto: > > > > > > > > > (1) we have published 2.9.0 at > > > > > https://downloads.apache.org/pulsar/pulsar-2.9.0/ > > > > > > > > > > (2) we have published 2.9.0 artifacts through maven central. They > > don’t > > > > > let anyone republish versions. > > > > > > > > > > There are no do overs on versions. We simply cannot redo 2.9.0 at > > this > > > > > moment. > > > > > > > > > > All the best, > > > > > Dave > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > On Dec 12, 2021, at 8:49 PM, Sijie Guo > wrote: > > > > > > > > > > > > My take is - if we haven't announced 2.9, I would suggest just > > > redoing > > > > > the > > > > > > 2.9.0 release. > > > > > > > > > > > > - Sijie > > > > > > > > > > > >> On Sun, Dec 12, 2021 at 6:35 PM Hang Chen > > > > wrote: > > > > > >> > > > > > >> I am a little confused about why we should skip 2.9.0 and not > > > continue > > > > > >> to release 2.9.0 by including the critical bug fixes. In fact, > the > > > > > >> Pulsar 2.9.0 release is not yet completed. > > > > > >> > > > > > >> For users, they will worry about whether the Pulsar release > > process > > > is > > > > > >> standardized if we skip 2.9.0. They will also worry about the > > > release > > > > > >> quality of Apache Pulsar if we have found the critical bugs > before > > > it > > > > > >> is released but not included it into the release version. For > > Pulsar > > > > > >> 2.9.0, it couldn't be deployed into the production environment > due > > > to > > > > > >> the critical bug https://github.com/apache/pulsar/pull/12993 > > > > > >> > > > > > >> Regards, > > > > > >> Hang > > > > > >> > > > > > >> Dave Fisher 于2021年12月13日周一 09:40写道: > > > > > >>> > > > > > >>> It can be the case that releases are not announced. For example > > > with > > > > > >> Tomcat a version which fails to pass the vote is skipped. > > > > > >>> > > > > > >>> Let’s not announce 2.9.0 and go on to 2.9.1. > > > > > >>> > > > > > >>> Maybe there’s some website fixes to hide 2.9.0. > > > > > >>> > > > > > >>> > > > > > >>> Sent from my iPhone > > > > > >>> > > > > > On Dec 12, 2021, at 5:28 PM, PengHui Li > > > wrote: > > > > > > > > > > Another point is we have not announced the 2.9.0 release yet. > > > > > > > > > > This will make users feel confused that a new release from the > > > > Pulsar > > > > > community with the > > > > > serious problem(log4j bug) but after the log4j has fixed the > > issue > > > > an
[DISCUSSION] PIP-124: Create init subscription before sending message to DLQ
https://github.com/apache/pulsar/issues/13408 Pasted below for quoting convenience. —— ## Motivation If we enable the DLQ when consuming messages. For some messages that can't be processed successfully, the messages will be moved to the DLQ, but if we do not specify the data retention for the namespace or create a subscription for the DLQ to retain the data, the data of the DLQ topic will be removed automatically. Therefore, we need to create the initial subscription before sending messages to the DLQ. ## Goal Users can set the initial subscription name in the DeadLetterPolicy. The consumer will create the initial subscription before sending messages to the DLQ. At this point, subsequent messages produced to the DLQ are not automatically deleted unexpectedly. This PIP needs to be compatible with the previous behavior. The initial subscription name in the DeadLetterPolicy is optional. Default value is the subscription name of the consumer. ## API Changes Add `initSubscriptionName` to the `DeadLetterPolicy` ```java /** * Name of the initial subscription name of the dead letter topic. * The default value is the subscription name of the consumer. */ private String initSubscriptionName; ``` ## Implementation Before the deadLetterProducer is initialized, the consumer first tries to create a deadLetterConsumer using the initial subscription name in the DeadLetterPolicy. When this subscription already exists, the ConsumerBusy exception will occur. In this case, we can ignore that exception and create the deadLetterProducer. Prototype implementation PR: https://github.com/apache/pulsar/pull/13355 Thanks, Zike Yang
Re: [VOTE] Pulsar Release 2.7.4 Candidate 1
+1 (binding) - verified checksums and signatures - build from source - verified produce/consume and functions - verified cassandra connector - verified stateful function Thanks, Nozomi 2021年12月18日(土) 19:43 Enrico Olivelli : > Jiwei Guo, > Thanks for driving the release > today is Saturday, so I am not sure how many people will have time to test > the release candidate during the 72 hours period (for instance I can do it > only on Monday, hopefully). > Please take this into consideration when you are going to close the VOTE, > maybe waiting until Tuesday may be a good idea. > > Enrico > > > > Il giorno sab 18 dic 2021 alle ore 09:54 guo jiwei > ha scritto: > > > This is the first release candidate for Apache Pulsar, version 2.7.4. > > > > It fixes the following issues: > > > > > https://github.com/apache/pulsar/pulls?q=is%3Apr+is%3Aopen+label%3Arelease%2F2.7.4+is%3Apr+ > > > > Release note: > > https://github.com/apache/pulsar/pull/13391 > > > > *** Please download, test and vote on this release. This vote will > > stay openfor at least 72 hours *** > > > > Note that we are voting upon the source (tag), binaries are provided for > > convenience. > > > > Source and binary files: > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.4-candidate-1/ > > > > SHA-512 checksums: > > > > > > > 1a34a408085328500c8c4c72e5ca9b458058a088674b1bfd550fe01ad365182b8820516f99b5c010a03037e1886ce5539cb57f5ef4a2297f1b7ed8fb5844bcd0 > > apache-pulsar-2.7.4-bin.tar.gz > > > > > 37e75698946f3390a254fa2bd7d974fca27d1aed632e86d64a08dc0bf8f2e0ca830cbeed2820782e59ac6bac7368ae69b54f1f5cd3d6c2cad292166e3538e285 > > apache-pulsar-2.7.4-src.tar.gz > > > > > > Maven staging repo: > > https://repository.apache.org/content/repositories/orgapachepulsar-1113 > > > > The tag to be voted upon: > > v2.7.4-candidate-1 (ab451b855d873a9bad2005f939a23118a583baa9) > > https://github.com/apache/pulsar/releases/tag/v2.7.4-candidate-1 > > > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > Please download the source package, and follow the README to build > > and run the Pulsar standalone service. > > > > > > > > Regards > > Jiwei Guo (Tboy) > > >
Re: [apache/pulsar] [website] website upgrade (#11766)
Hi Dave, Thanks for your instructions. +1 for not migrating the incubating versioned docs since they are too old and almost no one uses them. Besides, as you suggested, we can regenerate those docs if needed. On Mon, Dec 20, 2021 at 11:18 AM Dave Fisher wrote: > This is a question for the dev@pulsar list and not a blocker for > infrastructure. > > I don’t think there is an overriding requirement to preserve docs from > incubating releases. If there were then there are other ways to find and > regenerate these docs. > > Regards, > Dave > > Sent from my iPhone > > > On Dec 19, 2021, at 6:58 PM, Anonymitaet > wrote: > > > > > > Hi @dave2wave Thanks for your answer. However, > https://archive.apache.org/dist/incubator/pulsar/ hosts code files rather > than doc files. We are looking for doc files (source files) of these > versions, do you know where hosts them? Thanks again > > > > 2.0.1-incubating > > 2.0.0-rc1-incubating > > 1.22.0-incubating > > 1.21.0-incubating > > 1.20.0-incubating > > 1.19.0-incubating > > — > > Reply to this email directly, view it on GitHub, or unsubscribe. > > Triage notifications on the go with GitHub Mobile for iOS or Android. > > You are receiving this because you were mentioned. >
[VOTE] Apache Pulsar 2.8.2 candidate 2
This is the second release candidate for Apache Pulsar, version 2.8.2 It fixes the following issues: https://github.com/apache/pulsar/issues?q=label%3Acherry-picked%2Fbranch-2.8+label%3Arelease%2F2.8.2+is%3Aclosed *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Note that we are voting upon the source (tag), binaries are provided for convenience. Source and binary files: https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.2-candidate-2/ SHA-512 checksums: 59aa0a14188a766ce802ba30cbaa2584a1904d26d8f41f164d3f97ea1970aa1391e11755d8077c96aeb136d2b9d73bf8b768978de7fa7f71d69cb57e2c1fce8c apache-pulsar-2.8.2-bin.tar.gz 82a1423fda4004297ca2867279077ed261c7149be96deca2c127ba5b91af08fec80dc4a8b15ee2ba8704e209efa577a0c7b4cfb78341d3a43a38bf476c894c5c apache-pulsar-2.8.2-src.tar.gz Maven staging repo: https://repository.apache.org/content/repositories/orgapachepulsar-1129/ The tag to be voted upon: v2.8.2-candidate-2 (4b9cadcd57e41bc8eb95cc9b9917f938365b1cca) https://github.com/apache/pulsar/releases/tag/v2.8.2-candidate-2 Pulsar's KEYS file containing PGP keys we use to sign the release: https://dist.apache.org/repos/dist/dev/pulsar/KEYS Release notes draft: https://github.com/apache/pulsar/pull/13400 Please download the source package, and follow the README to build and run the Pulsar standalone service.
Re: [RESULT] [VOTE] Apache Pulsar 2.9.1 candidate 2
Hi Enrico, Have you checked the CI status after completing the 2.9.1 PRs cherry-picking? Looks this test failed when I run the test with tag 2.9.1. ``` [INFO] Running org.apache.pulsar.broker.transaction.TransactionTest [ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 12, Time elapsed: 26.782 s <<< FAILURE! - in org.apache.pulsar.broker.transaction.TransactionTest [ERROR] testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) Time elapsed: 0.107 s <<< FAILURE! java.lang.AssertionError: expected [4] but found [3] at org.testng.Assert.fail(Assert.java:99) at org.testng.Assert.failNotEquals(Assert.java:1037) at org.testng.Assert.assertEqualsImpl(Assert.java:140) at org.testng.Assert.assertEquals(Assert.java:122) at org.testng.Assert.assertEquals(Assert.java:907) at org.testng.Assert.assertEquals(Assert.java:917) at org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(TransactionTest.java:145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45) at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73) at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) [INFO] Run 1: PASS [ERROR] Run 2: TransactionTest.testCreateTransactionSystemTopic:145 expected [4] but found [3] [INFO] [INFO] [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 7 ``` Regards, Penghui On Mon, Dec 20, 2021 at 3:06 PM Enrico Olivelli wrote: > Hello everyone, > The VOTE passed with 5 VOTEs, 3 of them were binding. > > - Matteo Merli (binding) > - Enrico Olivelli (binding) > - Peng Hui (binding) > - Nicolò Boschi > - Massimiliano Mirelli > > I will move forward with the next steps > https://github.com/apache/pulsar/wiki/Release-process > > Enrico > > > Il giorno sab 18 dic 2021 alle ore 11:41 PengHui Li > ha > scritto: > > > +1 binding > > > > Penghui > > > > Enrico Olivelli 于2021年12月18日 周六18:39写道: > > > > > +1 (binding) > > > > > > - Run release validation procedure > > > - CI is passing on those sources > > > > > > Enrico > > > > > > Il giorno sab 18 dic 2021 alle ore 02:51 PengHui Li < > peng...@apache.org> > > > ha > > > scritto: > > > > > > > > Will this issue be fixed in the future releases? > > > > > > > > Yes, 2.8.2 and 2.9.2 will fix the problem. > > > > > > > > Penghui > > > > > > > > On Sat, Dec 18, 2021 at 3:28 AM Massimiliano Mirelli < > > > > massimilianomirelli...@gmail.com> wrote: > > > > > > > > > Thank you for the rc! > > > > > > > > > > +1 (non-binding) > > > > > > > > > > * verify sha512 checksums > > > > > * verify gpg signatures > > > > > * build pulsar-all docker image > > > > > * execute Fallout distributed system test (produce / receive 10k > > > > messages) > > > > > > > > > > Building the docker image as indicated in the README: > > > > > > > > > > mvn clean install -DskipTests > > > > > mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests > > > > > > > > > > I still get the error described in this PR#11951 ( > > > > > https://github.com/apache/pulsar/pull/11951) which I suppose has > > been > > > > > cherry picked in 2.9.1. > > > > > > > > > > Giving enough permissions to /docker/pulsar/scripts/ in the src > > package > > > > and > > > > > then building the docker image again solved the issue. > > > > > > > > > > I also tested the problem with the image Enrico provided, this way: > > > > > > > > > > docker run -it --entrypoint bash eolivelli/pulsar-all:2.9.1rc2 > > > > > ls -al /pulsar/bin > > > > > > > > > > and that one does have the right permissions. > > > > > > > > > > Enrico, did you build the image using the same mvn commands (^^^), > or > > > is > > > > > there some other way to build it? > > > > > > > > > > Thank you, > > > > > Max > > > > > > > > > > On Fri, 17 Dec 2021 at 16:18, 陳智弘 wrote: > > > > > > > > > > > Hi PengHu, > > > > > > > > > > > > Will this issue be fixed in the future releases? > > > > > > > > > > > > PengHui Li 於 2021年12月17日 週五 21:53 寫道: > > > > > > > > > > > > > Hi Enrico, > > > > > > > > > > > > > > I'm ok, it only happens when the message is without a schema > > > version. > > > > > > >
Re: [RESULT] [VOTE] Apache Pulsar 2.9.1 candidate 2
Il giorno lun 20 dic 2021 alle ore 13:21 PengHui Li ha scritto: > Hi Enrico, > > Have you checked the CI status after completing the 2.9.1 PRs > cherry-picking? > When I created the tag I am pretty sure that CI on GH actions passed. I hope that no-one committed something to branch-2.9 If this happens, then we must enforce some rules about the fact that only the RM can commit to a branch under release. In ZooKeeper we create a "release branch" in order to prevent such problems Enrico > > Looks this test failed when I run the test with tag 2.9.1. > > ``` > [INFO] Running org.apache.pulsar.broker.transaction.TransactionTest > [ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 12, Time elapsed: > 26.782 s <<< FAILURE! - in > org.apache.pulsar.broker.transaction.TransactionTest > [ERROR] > > testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) > Time elapsed: 0.107 s <<< FAILURE! > java.lang.AssertionError: expected [4] but found [3] > at org.testng.Assert.fail(Assert.java:99) > at org.testng.Assert.failNotEquals(Assert.java:1037) > at org.testng.Assert.assertEqualsImpl(Assert.java:140) > at org.testng.Assert.assertEquals(Assert.java:122) > at org.testng.Assert.assertEquals(Assert.java:907) > at org.testng.Assert.assertEquals(Assert.java:917) > at > > org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(TransactionTest.java:145) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > > org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) > at > > org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45) > at > org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73) > at > org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] > > org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) > [INFO] Run 1: PASS > [ERROR] Run 2: TransactionTest.testCreateTransactionSystemTopic:145 > expected [4] but found [3] > [INFO] > [INFO] > [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 7 > ``` > > Regards, > Penghui > > On Mon, Dec 20, 2021 at 3:06 PM Enrico Olivelli > wrote: > > > Hello everyone, > > The VOTE passed with 5 VOTEs, 3 of them were binding. > > > > - Matteo Merli (binding) > > - Enrico Olivelli (binding) > > - Peng Hui (binding) > > - Nicolò Boschi > > - Massimiliano Mirelli > > > > I will move forward with the next steps > > https://github.com/apache/pulsar/wiki/Release-process > > > > Enrico > > > > > > Il giorno sab 18 dic 2021 alle ore 11:41 PengHui Li > > ha > > scritto: > > > > > +1 binding > > > > > > Penghui > > > > > > Enrico Olivelli 于2021年12月18日 周六18:39写道: > > > > > > > +1 (binding) > > > > > > > > - Run release validation procedure > > > > - CI is passing on those sources > > > > > > > > Enrico > > > > > > > > Il giorno sab 18 dic 2021 alle ore 02:51 PengHui Li < > > peng...@apache.org> > > > > ha > > > > scritto: > > > > > > > > > > Will this issue be fixed in the future releases? > > > > > > > > > > Yes, 2.8.2 and 2.9.2 will fix the problem. > > > > > > > > > > Penghui > > > > > > > > > > On Sat, Dec 18, 2021 at 3:28 AM Massimiliano Mirelli < > > > > > massimilianomirelli...@gmail.com> wrote: > > > > > > > > > > > Thank you for the rc! > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > * verify sha512 checksums > > > > > > * verify gpg signatures > > > > > > * build pulsar-all docker image > > > > > > * execute Fallout distributed system test (produce / receive 10k > > > > > messages) > > > > > > > > > > > > Building the docker image as indicated in the README: > > > > > > > > > > > > mvn clean install -DskipTests > > > > > > mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests > > > > > > > > > > > > I still get the error described in this PR#11951 ( > > > > > > https://github.com/apache/pulsar/pull/11951) which I suppose has > > > been > > > > > > cherry picked in 2.9.1. > > > > > > > > > > > > Giving enough permissions to /docker/pulsar/scripts/ in the src > > > package > > > > > and > > > > > > then building the docker image again solved the issue. > > > > > > > > > > > > I also tested the problem with the image Enrico provided, this > way: > > > > > > > > > > > > docker run -it --entrypoint bash eolivelli
Re: [RESULT] [VOTE] Apache Pulsar 2.9.1 candidate 2
I use the 2.9.1 tag to run the test, not branch-2.9. I think this one(2db23b8dd3eb859cc16e30686578be275026c347) is the last one that you cherry-picked? But there are failed tests: [image: image.png] Penghui On Mon, Dec 20, 2021 at 8:24 PM Enrico Olivelli wrote: > Il giorno lun 20 dic 2021 alle ore 13:21 PengHui Li > ha > scritto: > > > Hi Enrico, > > > > Have you checked the CI status after completing the 2.9.1 PRs > > cherry-picking? > > > > When I created the tag I am pretty sure that CI on GH actions passed. > > I hope that no-one committed something to branch-2.9 > > If this happens, then we must enforce some rules about the fact that only > the RM can commit to a branch under release. > > In ZooKeeper we create a "release branch" in order to prevent such problems > > Enrico > > > > > > > Looks this test failed when I run the test with tag 2.9.1. > > > > ``` > > [INFO] Running org.apache.pulsar.broker.transaction.TransactionTest > > [ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 12, Time elapsed: > > 26.782 s <<< FAILURE! - in > > org.apache.pulsar.broker.transaction.TransactionTest > > [ERROR] > > > > > testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) > > Time elapsed: 0.107 s <<< FAILURE! > > java.lang.AssertionError: expected [4] but found [3] > > at org.testng.Assert.fail(Assert.java:99) > > at org.testng.Assert.failNotEquals(Assert.java:1037) > > at org.testng.Assert.assertEqualsImpl(Assert.java:140) > > at org.testng.Assert.assertEquals(Assert.java:122) > > at org.testng.Assert.assertEquals(Assert.java:907) > > at org.testng.Assert.assertEquals(Assert.java:917) > > at > > > > > org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(TransactionTest.java:145) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > > > > org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) > > at > > > > > org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45) > > at > > > org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73) > > at > > > org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > > > [INFO] > > [INFO] Results: > > [INFO] > > [ERROR] Failures: > > [ERROR] > > > > > org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) > > [INFO] Run 1: PASS > > [ERROR] Run 2: TransactionTest.testCreateTransactionSystemTopic:145 > > expected [4] but found [3] > > [INFO] > > [INFO] > > [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 7 > > ``` > > > > Regards, > > Penghui > > > > On Mon, Dec 20, 2021 at 3:06 PM Enrico Olivelli > > wrote: > > > > > Hello everyone, > > > The VOTE passed with 5 VOTEs, 3 of them were binding. > > > > > > - Matteo Merli (binding) > > > - Enrico Olivelli (binding) > > > - Peng Hui (binding) > > > - Nicolò Boschi > > > - Massimiliano Mirelli > > > > > > I will move forward with the next steps > > > https://github.com/apache/pulsar/wiki/Release-process > > > > > > Enrico > > > > > > > > > Il giorno sab 18 dic 2021 alle ore 11:41 PengHui Li < > peng...@apache.org> > > > ha > > > scritto: > > > > > > > +1 binding > > > > > > > > Penghui > > > > > > > > Enrico Olivelli 于2021年12月18日 周六18:39写道: > > > > > > > > > +1 (binding) > > > > > > > > > > - Run release validation procedure > > > > > - CI is passing on those sources > > > > > > > > > > Enrico > > > > > > > > > > Il giorno sab 18 dic 2021 alle ore 02:51 PengHui Li < > > > peng...@apache.org> > > > > > ha > > > > > scritto: > > > > > > > > > > > > Will this issue be fixed in the future releases? > > > > > > > > > > > > Yes, 2.8.2 and 2.9.2 will fix the problem. > > > > > > > > > > > > Penghui > > > > > > > > > > > > On Sat, Dec 18, 2021 at 3:28 AM Massimiliano Mirelli < > > > > > > massimilianomirelli...@gmail.com> wrote: > > > > > > > > > > > > > Thank you for the rc! > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > * verify sha512 checksums > > > > > > > * verify gpg signatures > > > > > > > * build pulsar-all docker image > > > > > > > * execute Fallout distributed system test (produce / receive > 10k > > > > > > messages) > > > > > > > > > > > > > > Building the docker image as indicated in the README: > > > > > > > > > > > > > > mvn clean install -DskipTests > > > >
DISCUSS - [RESULT] [VOTE] Apache Pulsar 2.9.1 candidate 2
(changing the subject) Il giorno lun 20 dic 2021 alle ore 13:33 PengHui Li ha scritto: > I use the 2.9.1 tag to run the test, not branch-2.9. > > I think this one(2db23b8dd3eb859cc16e30686578be275026c347) is the last one > that you cherry-picked? > This is the latest commit: f52ac045f41acbb6c31da21a3463df3cfbe8f1b4 And you are correct, this is the latest commit I cherry-picked: 2db23b8dd3eb859cc16e30686578be275026c347 [Managed Ledger] Fix the incorrect total size when BrokerEntryMetadata is enabled (#12714) for reference: https://github.com/apache/pulsar/tree/v2.9.1 commit f52ac045f41acbb6c31da21a3463df3cfbe8f1b4 (HEAD -> branch-2.9, tag: v2.9.1-candidate-2, tag: v2.9.1, origin/branch-2.9) Author: Nicolò Boschi Date: Wed Dec 15 22:18:58 2021 +0100 [security] Upgrade Netty to 4.1.72 - CVE-2021-43797 (#13328) Enrico But there are failed tests: > > [image: image.png] > > Penghui > > On Mon, Dec 20, 2021 at 8:24 PM Enrico Olivelli > wrote: > >> Il giorno lun 20 dic 2021 alle ore 13:21 PengHui Li >> ha >> scritto: >> >> > Hi Enrico, >> > >> > Have you checked the CI status after completing the 2.9.1 PRs >> > cherry-picking? >> > >> >> When I created the tag I am pretty sure that CI on GH actions passed. >> >> I hope that no-one committed something to branch-2.9 >> >> If this happens, then we must enforce some rules about the fact that only >> the RM can commit to a branch under release. >> >> In ZooKeeper we create a "release branch" in order to prevent such >> problems >> >> Enrico >> >> >> >> > >> > Looks this test failed when I run the test with tag 2.9.1. >> > >> > ``` >> > [INFO] Running org.apache.pulsar.broker.transaction.TransactionTest >> > [ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 12, Time >> elapsed: >> > 26.782 s <<< FAILURE! - in >> > org.apache.pulsar.broker.transaction.TransactionTest >> > [ERROR] >> > >> > >> testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) >> > Time elapsed: 0.107 s <<< FAILURE! >> > java.lang.AssertionError: expected [4] but found [3] >> > at org.testng.Assert.fail(Assert.java:99) >> > at org.testng.Assert.failNotEquals(Assert.java:1037) >> > at org.testng.Assert.assertEqualsImpl(Assert.java:140) >> > at org.testng.Assert.assertEquals(Assert.java:122) >> > at org.testng.Assert.assertEquals(Assert.java:907) >> > at org.testng.Assert.assertEquals(Assert.java:917) >> > at >> > >> > >> org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(TransactionTest.java:145) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> > at >> > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > at >> > >> > >> org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) >> > at >> > >> > >> org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45) >> > at >> > >> org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73) >> > at >> > >> org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> > at >> > >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> > at >> > >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> > at java.lang.Thread.run(Thread.java:748) >> > >> > [INFO] >> > [INFO] Results: >> > [INFO] >> > [ERROR] Failures: >> > [ERROR] >> > >> > >> org.apache.pulsar.broker.transaction.TransactionTest.testCreateTransactionSystemTopic(org.apache.pulsar.broker.transaction.TransactionTest) >> > [INFO] Run 1: PASS >> > [ERROR] Run 2: TransactionTest.testCreateTransactionSystemTopic:145 >> > expected [4] but found [3] >> > [INFO] >> > [INFO] >> > [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 7 >> > ``` >> > >> > Regards, >> > Penghui >> > >> > On Mon, Dec 20, 2021 at 3:06 PM Enrico Olivelli >> > wrote: >> > >> > > Hello everyone, >> > > The VOTE passed with 5 VOTEs, 3 of them were binding. >> > > >> > > - Matteo Merli (binding) >> > > - Enrico Olivelli (binding) >> > > - Peng Hui (binding) >> > > - Nicolò Boschi >> > > - Massimiliano Mirelli >> > > >> > > I will move forward with the next steps >> > > https://github.com/apache/pulsar/wiki/Release-process >> > > >> > > Enrico >> > > >> > > >> > > Il giorno sab 18 dic 2021 alle ore 11:41 PengHui Li < >> peng...@apache.org> >> > > ha >> > > scritto: >> > > >> > > > +1 binding >> > > > >> > > > Penghui >> > > > >> > > > Enrico Olivelli 于2021年12月18日 周六18:39写道: >> > > > >> > > > > +1 (binding) >> > > > > >> > > > > - Run release validation procedure >> > > > > - CI is passing on those sources >> > > > > >> > > > > Enrico >> > > > > >> > > > > Il giorno sab 18 dic 2021 alle ore 02:51 PengHui Li
[GitHub] [pulsar-adapters] eolivelli commented on issue #29: Pulsar - Spark adapter for scala 2.11
eolivelli commented on issue #29: URL: https://github.com/apache/pulsar-adapters/issues/29#issuecomment-998040594 > Can raise a PR if that's fine Sure! thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [VOTE] Apache Pulsar 2.8.2 candidate 2
+1 (binding) Check signature, Run standalone, Verify Cassandra connector, Verify Function Verify PusarSQL, still have `java.nio.BufferUnderflowException`, but it only happens on the topic created by the Function, I have tried to publish new string messages to a new topic, and query data from the topic, it works. It's not a regression in 2.8.2 Penghui On Mon, Dec 20, 2021 at 7:19 PM linlin wrote: > This is the second release candidate for Apache Pulsar, version 2.8.2 > > It fixes the following issues: > > https://github.com/apache/pulsar/issues?q=label%3Acherry-picked%2Fbranch-2.8+label%3Arelease%2F2.8.2+is%3Aclosed > > *** Please download, test and vote on this release. This vote will stay > open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.2-candidate-2/ > > SHA-512 checksums: > > 59aa0a14188a766ce802ba30cbaa2584a1904d26d8f41f164d3f97ea1970aa1391e11755d8077c96aeb136d2b9d73bf8b768978de7fa7f71d69cb57e2c1fce8c > apache-pulsar-2.8.2-bin.tar.gz > > > 82a1423fda4004297ca2867279077ed261c7149be96deca2c127ba5b91af08fec80dc4a8b15ee2ba8704e209efa577a0c7b4cfb78341d3a43a38bf476c894c5c > apache-pulsar-2.8.2-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1129/ > > The tag to be voted upon: > v2.8.2-candidate-2 (4b9cadcd57e41bc8eb95cc9b9917f938365b1cca) > https://github.com/apache/pulsar/releases/tag/v2.8.2-candidate-2 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Release notes draft: > https://github.com/apache/pulsar/pull/13400 > > Please download the source package, and follow the README to build > and run the Pulsar standalone service. >
[DISCUSS] Remove Grafana and Prometheus Dockerfiles; Remove DCOS example
Hi Pulsar Community, I would like to clean up our main project's `docker/` directory. We have a couple of old Dockerfiles that either need to be updated or removed. My vote is to remove them. 1. Grafana: with every release, we build and upload a custom grafana docker image just to package dashboards. Grafana provides other mechanisms to load dashboards. I propose we remove the grafana image from our release process and then move our dashboard json files to a new top level directory named `grafana/`. [0] Our image is based on grafana/grafana:4.3.1. The current grafana release is grafana/grafana:8.3.3. 2. Prometheus: we have a prometheus Dockerfile that was initially merged as part of a DCOS example 4 years ago. We have never published this docker image as part of a release [1]. There is nothing pulsar specific in these files. [2] Our image is based on prom/prometheus:v1.8.2. The latest prometheus image prom/prometheus:v2.32.1. 3. Can we remove our DCOS examples in `deployment/dcos`? We haven't updated our DCOS example deployment spec in 3.5 years. The current spec relies on our custom grafana docker image and on a community member's custom build [3] of our prometheus-dcos docker image. I don't think our example deployment should rely on a community member's custom docker image. (I don't have a PR to remove this example yet.) Alternatively, we could update the dockerfiles and the DCOS example. However, the lack of activity in keeping these three project dependencies up to date indicates to me that we should remove them. Let me know what you think. Thanks, Michael [0] https://github.com/apache/pulsar/pull/13389 [1] https://hub.docker.com/u/apachepulsar/ [2] https://github.com/apache/pulsar/pull/13387 [3] https://github.com/apache/pulsar/blob/c62ca808fc8c5aed3bb96b99bf6ef0c8ed382e3f/deployment/dcos/PulsarGroups.json#L251
[GitHub] [pulsar-client-node] tschmidt64-SC commented on issue #134: Getting error while installing
tschmidt64-SC commented on issue #134: URL: https://github.com/apache/pulsar-client-node/issues/134#issuecomment-998219865 I'm still having this issue to this day. Tried building in a docker container, same issue there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] Release Pulsar 2.7.4
Thank you Penghui, Technoboy, and BewareMyPower for picking up where I left off to get the 2.7.4 release ready to go. You cherry-picked many commits, resolved many conflicts, and fixed a bunch of tests! Thanks, Michael On Thu, Dec 16, 2021 at 7:37 PM guo jiwei wrote: > > Hi, > After we have fixed some issue like ZookeeperCache NPE, listing namespace > exception, and skip some flaky tests (verified locally), now the CI have > passed. > Skipped flaky tests are tracked here: > https://github.com/apache/pulsar/issues/13299 > Now we decide to vote for releasing 2.7.4. > > Regards > Jiwei Guo (Tboy) > > > On Tue, Dec 14, 2021 at 11:58 AM PengHui Li wrote: > > > Thanks for the update, I will move it 2.7.5 > > > > Thanks, > > Penghui > > > > On Tue, Dec 14, 2021 at 9:47 AM Matteo Merli > > wrote: > > > > > Let's take https://github.com/apache/pulsar/pull/12484 out of the > > > picture since it's failing the tests. > > > > > > > > > -- > > > Matteo Merli > > > > > > > > > On Sun, Dec 12, 2021 at 11:06 PM PengHui Li wrote: > > > > > > > > Yes, > > > > > > > > https://github.com/apache/pulsar/pull/13215 has cherry-picked, so we > > can > > > > close it. > > > > https://github.com/apache/pulsar/pull/12484 blocked by the test. > > > > > > > > Penghui > > > > > > > > On Mon, Dec 13, 2021 at 2:35 PM Dave Fisher > > > wrote: > > > > > > > > > I see 2 PRs still open at > > > > > > > > > > https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.7.4 > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > On Dec 12, 2021, at 8:22 PM, guo jiwei > > wrote: > > > > > > > > > > > > I have pushed out some fixes in > > > > > https://github.com/apache/pulsar/pull/13243 > > > > > > After the tests get passed, I will send out the RC-1 VOTE for 2.7.4 > > > > > > > > > > > > Regards > > > > > > Jiwei Guo (Tboy) > > > > > > > > > > > > > > > > > >> On Sun, Dec 12, 2021 at 3:11 PM PengHui Li > > > wrote: > > > > > >> > > > > > >> Just put an update here. We have done the PR cherry-picking > > > > > >> > > > > > >> https://github.com/apache/pulsar/commits/branch-2.7 > > > > > >> > > > > > >> And most of the integration tests are fixed due to the docker > > image > > > > > issue > > > > > >> or the testcontainer issue, now some integration tests get passed, > > > but > > > > > some > > > > > >> are not. > > > > > >> And there are some failed tests, maybe a flaky test, we need to > > > ensure > > > > > it's > > > > > >> not a regression. > > > > > >> > > > > > >> We are continuing on the test part. > > > > > >> > > > > > >> Penghui > > > > > >> > > > > > >> > > > > > >> > > > > > >>> On Sat, Dec 11, 2021 at 5:36 PM PengHui Li > > > wrote: > > > > > >>> > > > > > >>> Hi Michael, > > > > > >>> > > > > > >>> +1, > > > > > >>> > > > > > >>> Thanks for the great work. > > > > > >>> We will continue on the PR cherry-picking and the release process > > > to > > > > > make > > > > > >>> sure the urgent release can be done ASAP. > > > > > >>> > > > > > >>> Penghui > > > > > >>> > > > > > >>> On Sat, Dec 11, 2021 at 3:42 PM Michael Marshall < > > > mmarsh...@apache.org > > > > > > > > > > > >>> wrote: > > > > > >>> > > > > > Given the log4j CVE, we should work to release 2.7.4. > > > > > > > > > > I started preparing the release today by cherry-picking merged > > PRs > > > > > that have the `release/2.7.4` label but have not yet been > > > > > cherry-picked to `branch-2.7` [0]. There are still 37 PRs that > > > have > > > > > not been cherry picked. I think it will take too long to cherry > > > pick > > > > > all of these commits, as many have conflicts, and we should > > > prioritize > > > > > releasing 2.7.4. The main commits that we should get > > cherry-picked > > > > > before creating the git tag are any labeled with > > > `component/security`. > > > > > There are only a few remaining commits to cherry pick. Please > > let > > > me > > > > > know if you think any other commits ought to be cherry-picked. > > > > > > > > > > The earliest I'll be able to build the release is Monday. If we > > > need > > > > > to start sooner, perhaps someone else will be available to > > manage > > > this > > > > > urgent release. > > > > > > > > > > Thanks, > > > > > Michael > > > > > > > > > > [0] - > > > > > > > > > > >> > > > > > > > > > > https://github.com/apache/pulsar/pulls?page=2&q=label%3Arelease%2F2.7.4+sort%3Acreated-asc+is%3Apr+-label%3Acherry-picked%2Fbranch-2.7 > > > > > [1] - > > > > > > > > > > >> > > > > > > > > > > https://github.com/apache/pulsar/pulls?q=label%3Arelease%2F2.7.4+sort%3Acreated-asc+is%3Apr+-label%3Acherry-picked%2Fbranch-2.7+label%3Acomponent%2Fsecurity > > > > > > > > > > > > > > > On Thu, Dec 9, 2021 at 4:03 PM Neng Lu > > wrote: > > > > > > > > > > > > +1 > > > > > > > > > > > > On 2021/12/09 15:29:55 Michael Marshall wrote: > > > > > >> H
Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ
Thanks for creating this PIP Zike Yang. This sounds like an important improvement. I have a couple thoughts. 1. Does the `retryLetterTopic` have the same problem with immediate message expiration? Based on a quick glance, I don't see anything creating a subscription for the retry topic. 2. Your prototype exposes a lacking in our existing client implementation: the non-admin client cannot directly create subscriptions. IIUC, this implementation does not work if `allowAutoSubscriptionCreation` is false. Further, it is inefficient to create and immediately close a consumer just to create a subscription. Perhaps we should just update the non-admin client so that it can create subscriptions? It could be private so that we don't create confusion with the non-admin and the admin clients. Thanks, Michael On Mon, Dec 20, 2021 at 4:18 AM Zike Yang wrote: > > https://github.com/apache/pulsar/issues/13408 > > Pasted below for quoting convenience. > > —— > > > ## Motivation > > If we enable the DLQ when consuming messages. For some messages that can't be > processed successfully, the messages will be moved to the DLQ, but if we do > not specify the data retention for the namespace or create a subscription for > the DLQ to retain the data, the data of the DLQ topic will be removed > automatically. Therefore, we need to create the initial subscription before > sending messages to the DLQ. > > ## Goal > > Users can set the initial subscription name in the DeadLetterPolicy. The > consumer will create the initial subscription before sending messages to the > DLQ. At this point, subsequent messages produced to the DLQ are not > automatically deleted unexpectedly. > > This PIP needs to be compatible with the previous behavior. The initial > subscription name in the DeadLetterPolicy is optional. Default value is the > subscription name of the consumer. > > ## API Changes > > Add `initSubscriptionName` to the `DeadLetterPolicy` > > ```java > /** > * Name of the initial subscription name of the dead letter topic. > * The default value is the subscription name of the consumer. > */ > private String initSubscriptionName; > ``` > > ## Implementation > > Before the deadLetterProducer is initialized, the consumer first tries to > create a deadLetterConsumer using the initial subscription name in the > DeadLetterPolicy. When this subscription already exists, the ConsumerBusy > exception will occur. In this case, we can ignore that exception and create > the deadLetterProducer. > > Prototype implementation PR: https://github.com/apache/pulsar/pull/13355 > > > > Thanks, > Zike Yang >
Re: [DISCUSSION] PIP-120: Enable client memory limit by default
> I think it's a good time for 2.10 to enable this setting by default and, > correspondingly, to disable by default the producer queue size limit. +1 > 64MB is picked because it's a small enough memory size that will guarantee > a very high producer throughput, irrespective of the individual messages size. Did you consider determining the default limit by inspecting the JVM's runtime? For example, in the broker, we set `maxMessagePublishBufferSizeInMB` to 1/2 of direct memory, by default. I concede that this config is more complicated than a broker config since we're talking about a client application. However, it seems like applications with 1 GB memory would probably want a different setting than one with 32 GB memory, and a fractional value could make the client work better "out of the box." We cannot make everyone happy with a default value, so if this dynamic default is too complicated to implement, I'm good with 64 MB. :) Thanks, Michael On Sun, Dec 19, 2021 at 11:17 PM Lan Liang wrote: > > +1 > > > On 12/17/2021 14:43,mattison chao wrote: > +1 > > On Fri, 17 Dec 2021 at 13:56, 陳智弘 wrote: > > +1 > > Sijie Guo 於 2021年12月17日 週五 12:38 寫道: > > +1 > > On Tue, Dec 14, 2021 at 11:20 AM Matteo Merli wrote: > > https://github.com/apache/pulsar/issues/13306 > > > Pasted below for quoting convenience. > > > > > ## Motivation > > In Pulsar 2.8, we have introduced a setting to control the amount of > memory > used by a client instance. > > ```java > interface ClientBuilder { > ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit); > } > ``` > > By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit > is > being > enforced. > > I think it's a good time for 2.10 to enable this setting by default > and, > correspondingly, to disable by default the producer queue size limit. > > This will simplify a lot the configuration that a producer application > will > have to come up with, when publishing with many topic/partitions or > when messages > are bigger than expected. > > ## Proposed changes > > In 2.10 release, for the `ClientBuilder`, change > * `memoryLimit`: 0 -> 64 MB > > For the `ProducerBuilder`, changes > * `maxPendingMessages`: 1000 -> 0 > > 64MB is picked because it's a small enough memory size that will > guarantee > a very high producer throughput, irrespective of the individual > messages > size. > > > > -- > Matteo Merli > > > >
Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires
+1 sounds like a great improvement. > With the re-acquirable resource locks, that problem was solved and thoroughly > tested to be robust. I haven't been able to follow the PIP 45 work, so I have some high level questions. Do we have documentation on the implementation of these re-acquirable resource locks? I'd like to understand how these work. Also, are you able to describe the thorough testing? Were these tests manual or are they already part of our GitHub CI or both? Thanks, Michael On Tue, Dec 14, 2021 at 12:03 PM Matteo Merli wrote: > > https://github.com/apache/pulsar/issues/13304 > > > Pasted below for quoting convenience. > > --- > > > ## Motivation > > After all the work done for PIP-45 that was already included in 2.8 and 2.9 > releases, it enabled the concept of re-acquirable resource locks and leader > election. > > Another important change was to avoid doing any deferrable metadata operation > when we know that we are not currently connected to the metadata service. > > Finally, that enabled stabilization in 2.9 the configuration setting that > allows > brokers to continue operating in a safe mode when the session with ZooKeeper > expires. > > The way it works is that, when we lose a ZooKeeper session, the data plane > will > continue to work undisturbed, relying on the BookKeeper fencing to avoid any > inconsistencies. > > New topics are not able to get started, but existing topics will see no > impact. > > The original intention for shutting down the brokers was to ensure that we > would automatically go back to a consistent state, with respect to which > resources are "owned" in ZooKeeper by a given broker. > > With the re-acquirable resource locks, that problem was solved and thoroughly > tested to be robust. > > ## Proposed changes > > In 2.10 release, for the setting: > > ```properties > # There are two policies to apply when a broker metadata session > expires: session expired happens, "shutdown" or "reconnect". > # With "shutdown", the broker will be restarted. > # With "reconnect", the broker will keep serving the topics, while > attempting to recreate a new session. > zookeeperSessionExpiredPolicy=shutdown > ``` > > Change its default value to `reconnect`. > > > -- > Matteo Merli >
Re: [DISCUSSION] PIP-119: Enable consistent hashing by default on KeyShared dispatcher
+1 On Thu, Dec 16, 2021 at 10:37 PM Sijie Guo wrote: > > +1 > > On Tue, Dec 14, 2021 at 10:15 AM Matteo Merli wrote: > > > Pasted below for quoting convenience. > > > > > > > > > > ## Motivation > > > > The consistent hashing implementation to uniformly assign keys to consumers > > in the context of a KeyShared subscription, was introduced in > > https://github.com/apache/pulsar/pull/6791, which was released in Pulsar > > 2.6.0. > > > > While consistent hashing can use slightly more memory in certain cases, it > > is > > more suitable as a general default implementation, as it leads to a fairer > > distribution of keys across consumers, and avoiding corner cases that > > depend > > on the sequence of addition/removal of consumers. > > > > ## Proposed changes > > > > In 2.10 release, for the setting: > > > > ```properties > > # On KeyShared subscriptions, with default AUTO_SPLIT mode, use > > splitting ranges or > > # consistent hashing to reassign keys to new consumers > > subscriptionKeySharedUseConsistentHashing=false > > ``` > > > > Change its default value to `true`. > > > > The `AUTO_SPLIT` mode will not be removed nor deprecated. Users will still > > be > > able to use the old implementation. > > > > > > > > -- > > Matteo Merli > > > >
Re: [DISCUSSION] PIP-119: Enable consistent hashing by default on KeyShared dispatcher
+1 On Tue, 21 Dec 2021 at 06:30, Michael Marshall wrote: > +1 > > On Thu, Dec 16, 2021 at 10:37 PM Sijie Guo wrote: > > > > +1 > > > > On Tue, Dec 14, 2021 at 10:15 AM Matteo Merli wrote: > > > > > Pasted below for quoting convenience. > > > > > > > > > > > > > > > ## Motivation > > > > > > The consistent hashing implementation to uniformly assign keys to > consumers > > > in the context of a KeyShared subscription, was introduced in > > > https://github.com/apache/pulsar/pull/6791, which was released in > Pulsar > > > 2.6.0. > > > > > > While consistent hashing can use slightly more memory in certain > cases, it > > > is > > > more suitable as a general default implementation, as it leads to a > fairer > > > distribution of keys across consumers, and avoiding corner cases that > > > depend > > > on the sequence of addition/removal of consumers. > > > > > > ## Proposed changes > > > > > > In 2.10 release, for the setting: > > > > > > ```properties > > > # On KeyShared subscriptions, with default AUTO_SPLIT mode, use > > > splitting ranges or > > > # consistent hashing to reassign keys to new consumers > > > subscriptionKeySharedUseConsistentHashing=false > > > ``` > > > > > > Change its default value to `true`. > > > > > > The `AUTO_SPLIT` mode will not be removed nor deprecated. Users will > still > > > be > > > able to use the old implementation. > > > > > > > > > > > > -- > > > Matteo Merli > > > > > > >
[GitHub] [pulsar-client-node] Matt-Esch commented on issue #99: Authenticating using a Token
Matt-Esch commented on issue #99: URL: https://github.com/apache/pulsar-client-node/issues/99#issuecomment-998386200 Can confirm this is an issue, occurs when upgrading to pulsar > 2.6.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [VOTE] Pulsar Release 2.7.4 Candidate 1
Hi, community. Due to the CVE-2021-25105 (more details at https://logging.apache.org/log4j/2.x/security.html), we have to upgrade log4j2 to 2.17, so suggest canceling this vote. I will send out the second-round vote later. Regards Jiwei Guo (Tboy) On Mon, Dec 20, 2021 at 6:39 PM Nozomi Kurihara wrote: > +1 (binding) > > - verified checksums and signatures > - build from source > - verified produce/consume and functions > - verified cassandra connector > - verified stateful function > > Thanks, > Nozomi > > 2021年12月18日(土) 19:43 Enrico Olivelli : > > > Jiwei Guo, > > Thanks for driving the release > > today is Saturday, so I am not sure how many people will have time to > test > > the release candidate during the 72 hours period (for instance I can do > it > > only on Monday, hopefully). > > Please take this into consideration when you are going to close the VOTE, > > maybe waiting until Tuesday may be a good idea. > > > > Enrico > > > > > > > > Il giorno sab 18 dic 2021 alle ore 09:54 guo jiwei > > > ha scritto: > > > > > This is the first release candidate for Apache Pulsar, version 2.7.4. > > > > > > It fixes the following issues: > > > > > > > > > https://github.com/apache/pulsar/pulls?q=is%3Apr+is%3Aopen+label%3Arelease%2F2.7.4+is%3Apr+ > > > > > > Release note: > > > https://github.com/apache/pulsar/pull/13391 > > > > > > *** Please download, test and vote on this release. This vote will > > > stay openfor at least 72 hours *** > > > > > > Note that we are voting upon the source (tag), binaries are provided > for > > > convenience. > > > > > > Source and binary files: > > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.4-candidate-1/ > > > > > > SHA-512 checksums: > > > > > > > > > > > > 1a34a408085328500c8c4c72e5ca9b458058a088674b1bfd550fe01ad365182b8820516f99b5c010a03037e1886ce5539cb57f5ef4a2297f1b7ed8fb5844bcd0 > > > apache-pulsar-2.7.4-bin.tar.gz > > > > > > > > > 37e75698946f3390a254fa2bd7d974fca27d1aed632e86d64a08dc0bf8f2e0ca830cbeed2820782e59ac6bac7368ae69b54f1f5cd3d6c2cad292166e3538e285 > > > apache-pulsar-2.7.4-src.tar.gz > > > > > > > > > Maven staging repo: > > > > https://repository.apache.org/content/repositories/orgapachepulsar-1113 > > > > > > The tag to be voted upon: > > > v2.7.4-candidate-1 (ab451b855d873a9bad2005f939a23118a583baa9) > > > https://github.com/apache/pulsar/releases/tag/v2.7.4-candidate-1 > > > > > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > > > Please download the source package, and follow the README to build > > > and run the Pulsar standalone service. > > > > > > > > > > > > Regards > > > Jiwei Guo (Tboy) > > > > > >
Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires
On 2021/12/14 18:03:20 Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13304 > > > Pasted below for quoting convenience. > > --- > > > ## Motivation > > After all the work done for PIP-45 that was already included in 2.8 and 2.9 > releases, it enabled the concept of re-acquirable resource locks and leader > election. > > Another important change was to avoid doing any deferrable metadata operation > when we know that we are not currently connected to the metadata service. > > Finally, that enabled stabilization in 2.9 the configuration setting that > allows > brokers to continue operating in a safe mode when the session with ZooKeeper > expires. > > The way it works is that, when we lose a ZooKeeper session, the data plane > will > continue to work undisturbed, relying on the BookKeeper fencing to avoid any > inconsistencies. > > New topics are not able to get started, but existing topics will see no > impact. > > The original intention for shutting down the brokers was to ensure that we > would automatically go back to a consistent state, with respect to which > resources are "owned" in ZooKeeper by a given broker. > > With the re-acquirable resource locks, that problem was solved and thoroughly > tested to be robust. > > ## Proposed changes > > In 2.10 release, for the setting: > > ```properties > # There are two policies to apply when a broker metadata session > expires: session expired happens, "shutdown" or "reconnect". > # With "shutdown", the broker will be restarted. > # With "reconnect", the broker will keep serving the topics, while > attempting to recreate a new session. > zookeeperSessionExpiredPolicy=shutdown > ``` > > Change its default value to `reconnect`. > > > -- > Matteo Merli > > +1 Please pay attention to the following scenarios: I tried to use `Reconnect` in the production environment, and there was a serious online failure: two Brokers kept opening and closing the Ledger of the same topic, and the Ledger of BK kept throwing fencing exceptions.
Re: [VOTE] Apache Pulsar 2.8.2 candidate 2
+1 (binding) - check signatures/checksums - Built sources - Validate Pub/Sub and Java Functions - Validate Connectors - Validate Stateful Functions Regards, Hiroyuki From: linlin Sent: Monday, December 20, 2021 20:18 To: dev@pulsar.apache.org Subject: [VOTE] Apache Pulsar 2.8.2 candidate 2 This is the second release candidate for Apache Pulsar, version 2.8.2 It fixes the following issues: https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fissues%3Fq%3Dlabel%253Acherry-picked%252Fbranch-2.8%2Blabel%253Arelease%252F2.8.2%2Bis%253Aclosed&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rEmdtatsDR3P9Vw2mIMkfagjE8nQyAvb4C2k7ByuTMs%3D&reserved=0 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Note that we are voting upon the source (tag), binaries are provided for convenience. Source and binary files: https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-2.8.2-candidate-2%2F&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PBXIUy6WYNWmSb%2F7cJM1C6mbFlfsRnnrnfhHvoMj9ag%3D&reserved=0 SHA-512 checksums: 59aa0a14188a766ce802ba30cbaa2584a1904d26d8f41f164d3f97ea1970aa1391e11755d8077c96aeb136d2b9d73bf8b768978de7fa7f71d69cb57e2c1fce8c apache-pulsar-2.8.2-bin.tar.gz 82a1423fda4004297ca2867279077ed261c7149be96deca2c127ba5b91af08fec80dc4a8b15ee2ba8704e209efa577a0c7b4cfb78341d3a43a38bf476c894c5c apache-pulsar-2.8.2-src.tar.gz Maven staging repo: https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachepulsar-1129%2F&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=98ZrSwMUZgKSrqA%2F9GGVCcuYEKIokc0bhaMPAgy9fDs%3D&reserved=0 The tag to be voted upon: v2.8.2-candidate-2 (4b9cadcd57e41bc8eb95cc9b9917f938365b1cca) https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Freleases%2Ftag%2Fv2.8.2-candidate-2&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Y2cOvtt%2BmSwoQM2Z7ZWX31MwjseFCl9Y93FxYgyN2Iw%3D&reserved=0 Pulsar's KEYS file containing PGP keys we use to sign the release: https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2FKEYS&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VyKYHq2dz1jUrJruX6jfgXh1378poKCBFDe2K24HvGs%3D&reserved=0 Release notes draft: https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F13400&data=04%7C01%7Chsakai%40yahoo-corp.jp%7C19af5d4d0d5740e89cfb08d9c3aa8d32%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755959550455483%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GGOM5mVzOaZfoVZawgwP%2BxRslSkJjTrLS2rSK6ma7qM%3D&reserved=0 Please download the source package, and follow the README to build and run the Pulsar standalone service.
Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires
Hi Lin Lin Which version you are working on the test? Do you have any steps to reproduce the issue? We'd better fix the problem in 2.10. Regards, Penghui On Tue, Dec 21, 2021 at 11:06 AM Lin Lin wrote: > > > On 2021/12/14 18:03:20 Matteo Merli wrote: > > https://github.com/apache/pulsar/issues/13304 > > > > > > Pasted below for quoting convenience. > > > > --- > > > > > > ## Motivation > > > > After all the work done for PIP-45 that was already included in 2.8 and > 2.9 > > releases, it enabled the concept of re-acquirable resource locks and > leader > > election. > > > > Another important change was to avoid doing any deferrable metadata > operation > > when we know that we are not currently connected to the metadata service. > > > > Finally, that enabled stabilization in 2.9 the configuration setting > that allows > > brokers to continue operating in a safe mode when the session with > ZooKeeper > > expires. > > > > The way it works is that, when we lose a ZooKeeper session, the data > plane will > > continue to work undisturbed, relying on the BookKeeper fencing to avoid > any > > inconsistencies. > > > > New topics are not able to get started, but existing topics will see no > > impact. > > > > The original intention for shutting down the brokers was to ensure that > we > > would automatically go back to a consistent state, with respect to which > > resources are "owned" in ZooKeeper by a given broker. > > > > With the re-acquirable resource locks, that problem was solved and > thoroughly > > tested to be robust. > > > > ## Proposed changes > > > > In 2.10 release, for the setting: > > > > ```properties > > # There are two policies to apply when a broker metadata session > > expires: session expired happens, "shutdown" or "reconnect". > > # With "shutdown", the broker will be restarted. > > # With "reconnect", the broker will keep serving the topics, while > > attempting to recreate a new session. > > zookeeperSessionExpiredPolicy=shutdown > > ``` > > > > Change its default value to `reconnect`. > > > > > > -- > > Matteo Merli > > > > > > +1 > > Please pay attention to the following scenarios: > I tried to use `Reconnect` in the production environment, and there was a > serious online failure: two Brokers kept opening and closing the Ledger of > the same topic, and the Ledger of BK kept throwing fencing exceptions. >