Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3
I'm closing this vote. 3 binding +1s: * Matteo * Penghui * Jiwei 1 non-binding +1: * Bo Thanks, Yunze On Fri, Dec 23, 2022 at 1:43 PM Dave Fisher wrote: > > I haven’t had a chance to review this RC yet, but I did notice the > reorganization of the directories and wanted to compliment this change as > very helpful! > > Also, adding a source release is responsive. > > Best, > Dave > > Sent from my iPhone > > > On Dec 22, 2022, at 9:32 PM, guo jiwei wrote: > > > > +1 (binding) > > > > - Checked the signature > > - Install the python .whl file (pip3 install > > pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > > 12.6(21G115) > > - Start the standalone (build from the master branch) > > - Start consumer (python3 ./examples/consumer.py) > > - Start producer (python3 ./examples/producer.py) > > > > > > Regards > > Jiwei Guo (Tboy) > > > > > >> On Wed, Dec 21, 2022 at 9:55 PM 丛搏 wrote: > >> > >> +1 (non-binding) > >> > >> python version: 3.7.6 > >> - Checked the signature > >> - Install the python .whl file (pip3 install > >> pulsar_client-3.0.0-cp37-cp37m-macosx_10_15_universal2.whl) on macOS > >> 12.3.1 > >> - Start the standalone (2.10.2) > >> - Start consumer (python3 ./examples/consumer.py) > >> - Start producer (python3 ./examples/producer.py ) > >> > >> Thanks, > >> Bo > >> > >> PengHui Li 于2022年12月21日周三 18:01写道： > >>> > >>> +1 > >>> > >>> - Checked the signature > >>> - Install the python .whl file (pip3 install > >>> pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > >> 13.0.1 > >>> (22A400) > >>> - Start the standalone (build from the master branch) > >>> - Start consumer (python3 ./examples/consumer.py) > >>> - Start producer (python3 ./examples/producer.py) > >>> > >>> -- > >>> Penghui > >>> > >>> On Sat, Dec 17, 2022 at 10:35 AM Matteo Merli > >>> wrote: > >>> > +1 > > * From candidate-2 I checked the source tarball for licenses, notice > and no binaries included. > -- > Matteo Merli > > > On Thu, Dec 15, 2022 at 11:30 PM Yunze Xu >>> > wrote: > > > > This is the third release candidate for Apache Pulsar Client Python, > > version 3.0.0. > > > > It fixes the following issues: > > https://github.com/apache/pulsar-client-python/milestone/1?closed=1 > > > > *** Please download, test and vote on this release. This vote will > > stay open for at least 72 hours *** > > > > Python wheels: > > > > >> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-python-3.0.0-candidate-3/ > > > > The supported python versions are 3.7, 3.8, 3.9 and 3.10. The > > supported platforms and architectures are: > > - Windows x86_64 (windows/) > > - glibc-based Linux x86_64 (linux-glibc-x86_64/) > > - glibc-based Linux arm64 (linux-glibc-arm64/) > > - musl-based Linux x86_64 (linux-musl-x86_64/) > > - musl-based Linux arm64 (linux-musl-arm64/) > > - macOS universal 2 (macos/) > > > > The tag to be voted upon: v3.0.0-candidate-3 > > (46acc487ad16fdc0aeea9dae64484030e62c1b96) > > > > >> https://github.com/apache/pulsar-client-python/releases/tag/v3.0.0-candidate-3 > > > > Pulsar's KEYS file containing PGP keys you use to sign the release: > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > Please download the Python wheels and follow the README to test. > > > > Thanks, > > Yunze > > >> >
Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3
Hi, Yunze, I found that there is still `-candidate-3` suffix in the source release files: https://dist.apache.org/repos/dist/release/pulsar/pulsar-client-python-3.0.0/ I think It should not contain the candidate suffix for both candidate files and release files. BR, Zike Yang On Mon, Dec 26, 2022 at 5:13 PM Yunze Xu wrote: > > I'm closing this vote. > > 3 binding +1s: > * Matteo > * Penghui > * Jiwei > > 1 non-binding +1: > * Bo > > Thanks, > Yunze > > On Fri, Dec 23, 2022 at 1:43 PM Dave Fisher wrote: > > > > I haven’t had a chance to review this RC yet, but I did notice the > > reorganization of the directories and wanted to compliment this change as > > very helpful! > > > > Also, adding a source release is responsive. > > > > Best, > > Dave > > > > Sent from my iPhone > > > > > On Dec 22, 2022, at 9:32 PM, guo jiwei wrote: > > > > > > +1 (binding) > > > > > > - Checked the signature > > > - Install the python .whl file (pip3 install > > > pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > > > 12.6(21G115) > > > - Start the standalone (build from the master branch) > > > - Start consumer (python3 ./examples/consumer.py) > > > - Start producer (python3 ./examples/producer.py) > > > > > > > > > Regards > > > Jiwei Guo (Tboy) > > > > > > > > >> On Wed, Dec 21, 2022 at 9:55 PM 丛搏 wrote: > > >> > > >> +1 (non-binding) > > >> > > >> python version: 3.7.6 > > >> - Checked the signature > > >> - Install the python .whl file (pip3 install > > >> pulsar_client-3.0.0-cp37-cp37m-macosx_10_15_universal2.whl) on macOS > > >> 12.3.1 > > >> - Start the standalone (2.10.2) > > >> - Start consumer (python3 ./examples/consumer.py) > > >> - Start producer (python3 ./examples/producer.py ) > > >> > > >> Thanks, > > >> Bo > > >> > > >> PengHui Li 于2022年12月21日周三 18:01写道： > > >>> > > >>> +1 > > >>> > > >>> - Checked the signature > > >>> - Install the python .whl file (pip3 install > > >>> pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > > >> 13.0.1 > > >>> (22A400) > > >>> - Start the standalone (build from the master branch) > > >>> - Start consumer (python3 ./examples/consumer.py) > > >>> - Start producer (python3 ./examples/producer.py) > > >>> > > >>> -- > > >>> Penghui > > >>> > > >>> On Sat, Dec 17, 2022 at 10:35 AM Matteo Merli > > >>> wrote: > > >>> > > +1 > > > > * From candidate-2 I checked the source tarball for licenses, notice > > and no binaries included. > > -- > > Matteo Merli > > > > > > On Thu, Dec 15, 2022 at 11:30 PM Yunze Xu > >>> > > wrote: > > > > > > This is the third release candidate for Apache Pulsar Client Python, > > > version 3.0.0. > > > > > > It fixes the following issues: > > > https://github.com/apache/pulsar-client-python/milestone/1?closed=1 > > > > > > *** Please download, test and vote on this release. This vote will > > > stay open for at least 72 hours *** > > > > > > Python wheels: > > > > > > > >> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-python-3.0.0-candidate-3/ > > > > > > The supported python versions are 3.7, 3.8, 3.9 and 3.10. The > > > supported platforms and architectures are: > > > - Windows x86_64 (windows/) > > > - glibc-based Linux x86_64 (linux-glibc-x86_64/) > > > - glibc-based Linux arm64 (linux-glibc-arm64/) > > > - musl-based Linux x86_64 (linux-musl-x86_64/) > > > - musl-based Linux arm64 (linux-musl-arm64/) > > > - macOS universal 2 (macos/) > > > > > > The tag to be voted upon: v3.0.0-candidate-3 > > > (46acc487ad16fdc0aeea9dae64484030e62c1b96) > > > > > > > >> https://github.com/apache/pulsar-client-python/releases/tag/v3.0.0-candidate-3 > > > > > > Pulsar's KEYS file containing PGP keys you use to sign the release: > > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > > > Please download the Python wheels and follow the README to test. > > > > > > Thanks, > > > Yunze > > > > >> > >
Re: [DISCUSS] Change the default IO threads and listener threads of Java Client
It seems this change will not affect the consumer, because the default executor is a single-thread executor service, increasing the listener threads number will allocate different single-thread executor services to different consumers. On 2022/12/21 09:01:39 Yunze Xu wrote: > I have a concern about the message ordering. If we have more than 1 > listener thread by default, could messages from the same topic be > passed to different listener threads? > > Thanks, > Yunze > > On Wed, Dec 21, 2022 at 11:12 AM 丛搏 wrote: > > > > +1 > > Our default configuration is best for most users. Multiple clients are > > a few cases. > > > > Thanks, > > Bo > > > > houxiaoyu 于2022年12月20日周二 16:02写道： > > > > > > +1 > > > > > > This change might bring thread number increment in case users create many > > > clients, but too many pulsar clients run in one machine is not a good use > > > case I think, so this change looks good to me. > > > > > > Thanks, > > > Xiaoyu Hou > > > > > > 于2022年12月20日周二 12:25写道： > > > > > > > +1 > > > > > > > > My concern is whether this change will affect some users who are > > > > creating > > > > many clients. I think we can wait for other users to confirm it. (If > > > > this > > > > will be affected, maybe we can give it a max_io_thread_num and then > > > > expand > > > > the size from 1 to max_io_thread_num when adding a new consumer or > > > > producer) > > > > > > > > > > > > Best, > > > > Mattison > > > > On Dec 20, 2022, 11:17 +0800, PengHui Li , wrote: > > > > > Hi all, > > > > > > > > > > I noticed the Java Client (I haven't checked other clients) uses 1 IO > > > > > thread and 1 listener > > > > > thread by default. It will require users to update the thread > > > > configuration > > > > > if they have > > > > > multiple cores and desired high throughput. > > > > > > > > > > Here is the example that we change to 16 IO threads in > > > > > openmessaging benchmark > > > > > > > > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > > > > > > > We can apply the configuration of the threads based on the CPU cores. > > > > > So > > > > > that for the > > > > > most common cases, users don't need to touch the thread configuration. > > > > > > > > > > ``` > > > > > private int numIoThreads = Runtime.getRuntime().availableProcessors(); > > > > > private int numListenerThreads = > > > > Runtime.getRuntime().availableProcessors(); > > > > > ``` > > > > > > > > > > WDYT? > > > > > > > > > > Thanks, > > > > > Penghui > > > > >
Re: [DISCUSS] Change the default IO threads and listener threads of Java Client
I am sorry, I am late on this train. Unfortunately if you have to connect using different authentication parameters you need to open many PulsarClients. I think that we can change this default but we should provide an official API to share the threadpools between the clients. There is no need to explicitly espose all the threadpools, that is difficult for users to deal with. We should have an API to allow sharing resources among multiple clients. Interface PulsarClientGroup PulsarClient client = newClient(). .withGroup(pulsarClientGroup) Enrico Il Lun 26 Dic 2022, 12:44 Ran Gao ha scritto: > It seems this change will not affect the consumer, because the default > executor is a single-thread executor service, increasing the listener > threads number will allocate different single-thread executor services to > different consumers. > > On 2022/12/21 09:01:39 Yunze Xu wrote: > > I have a concern about the message ordering. If we have more than 1 > > listener thread by default, could messages from the same topic be > > passed to different listener threads? > > > > Thanks, > > Yunze > > > > On Wed, Dec 21, 2022 at 11:12 AM 丛搏 wrote: > > > > > > +1 > > > Our default configuration is best for most users. Multiple clients are > > > a few cases. > > > > > > Thanks, > > > Bo > > > > > > houxiaoyu 于2022年12月20日周二 16:02写道： > > > > > > > > +1 > > > > > > > > This change might bring thread number increment in case users create > many > > > > clients, but too many pulsar clients run in one machine is not a > good use > > > > case I think, so this change looks good to me. > > > > > > > > Thanks, > > > > Xiaoyu Hou > > > > > > > > 于2022年12月20日周二 12:25写道： > > > > > > > > > +1 > > > > > > > > > > My concern is whether this change will affect some users who are > creating > > > > > many clients. I think we can wait for other users to confirm it. > (If this > > > > > will be affected, maybe we can give it a max_io_thread_num and > then expand > > > > > the size from 1 to max_io_thread_num when adding a new consumer or > producer) > > > > > > > > > > > > > > > Best, > > > > > Mattison > > > > > On Dec 20, 2022, 11:17 +0800, PengHui Li , > wrote: > > > > > > Hi all, > > > > > > > > > > > > I noticed the Java Client (I haven't checked other clients) uses > 1 IO > > > > > > thread and 1 listener > > > > > > thread by default. It will require users to update the thread > > > > > configuration > > > > > > if they have > > > > > > multiple cores and desired high throughput. > > > > > > > > > > > > Here is the example that we change to 16 IO threads in > > > > > > openmessaging benchmark > > > > > > > > > > > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > > > > > > > > > We can apply the configuration of the threads based on the CPU > cores. So > > > > > > that for the > > > > > > most common cases, users don't need to touch the thread > configuration. > > > > > > > > > > > > ``` > > > > > > private int numIoThreads = > Runtime.getRuntime().availableProcessors(); > > > > > > private int numListenerThreads = > > > > > Runtime.getRuntime().availableProcessors(); > > > > > > ``` > > > > > > > > > > > > WDYT? > > > > > > > > > > > > Thanks, > > > > > > Penghui > > > > > > > >
Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3
Hi zike, Thanks for your reminder. It's caused by the wrong steps of the release process. I think we can simply rename these files to remove the `-candidate-3` suffix, which requires the PMC's permission to do that. I will ask a PMC to do that. Thanks, Yunze On Mon, Dec 26, 2022 at 7:31 PM Zike Yang wrote: > > Hi, Yunze, > > I found that there is still `-candidate-3` suffix in the source > release files: > https://dist.apache.org/repos/dist/release/pulsar/pulsar-client-python-3.0.0/ > I think It should not contain the candidate suffix for both candidate > files and release files. > > BR, > Zike Yang > > On Mon, Dec 26, 2022 at 5:13 PM Yunze Xu wrote: > > > > I'm closing this vote. > > > > 3 binding +1s: > > * Matteo > > * Penghui > > * Jiwei > > > > 1 non-binding +1: > > * Bo > > > > Thanks, > > Yunze > > > > On Fri, Dec 23, 2022 at 1:43 PM Dave Fisher wrote: > > > > > > I haven’t had a chance to review this RC yet, but I did notice the > > > reorganization of the directories and wanted to compliment this change as > > > very helpful! > > > > > > Also, adding a source release is responsive. > > > > > > Best, > > > Dave > > > > > > Sent from my iPhone > > > > > > > On Dec 22, 2022, at 9:32 PM, guo jiwei wrote: > > > > > > > > +1 (binding) > > > > > > > > - Checked the signature > > > > - Install the python .whl file (pip3 install > > > > pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > > > > 12.6(21G115) > > > > - Start the standalone (build from the master branch) > > > > - Start consumer (python3 ./examples/consumer.py) > > > > - Start producer (python3 ./examples/producer.py) > > > > > > > > > > > > Regards > > > > Jiwei Guo (Tboy) > > > > > > > > > > > >> On Wed, Dec 21, 2022 at 9:55 PM 丛搏 wrote: > > > >> > > > >> +1 (non-binding) > > > >> > > > >> python version: 3.7.6 > > > >> - Checked the signature > > > >> - Install the python .whl file (pip3 install > > > >> pulsar_client-3.0.0-cp37-cp37m-macosx_10_15_universal2.whl) on macOS > > > >> 12.3.1 > > > >> - Start the standalone (2.10.2) > > > >> - Start consumer (python3 ./examples/consumer.py) > > > >> - Start producer (python3 ./examples/producer.py ) > > > >> > > > >> Thanks, > > > >> Bo > > > >> > > > >> PengHui Li 于2022年12月21日周三 18:01写道： > > > >>> > > > >>> +1 > > > >>> > > > >>> - Checked the signature > > > >>> - Install the python .whl file (pip3 install > > > >>> pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > > > >> 13.0.1 > > > >>> (22A400) > > > >>> - Start the standalone (build from the master branch) > > > >>> - Start consumer (python3 ./examples/consumer.py) > > > >>> - Start producer (python3 ./examples/producer.py) > > > >>> > > > >>> -- > > > >>> Penghui > > > >>> > > > >>> On Sat, Dec 17, 2022 at 10:35 AM Matteo Merli > > > >>> wrote: > > > >>> > > > +1 > > > > > > * From candidate-2 I checked the source tarball for licenses, notice > > > and no binaries included. > > > -- > > > Matteo Merli > > > > > > > > > On Thu, Dec 15, 2022 at 11:30 PM Yunze Xu > > > > > >>> > > > wrote: > > > > > > > > This is the third release candidate for Apache Pulsar Client Python, > > > > version 3.0.0. > > > > > > > > It fixes the following issues: > > > > https://github.com/apache/pulsar-client-python/milestone/1?closed=1 > > > > > > > > *** Please download, test and vote on this release. This vote will > > > > stay open for at least 72 hours *** > > > > > > > > Python wheels: > > > > > > > > > > >> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-python-3.0.0-candidate-3/ > > > > > > > > The supported python versions are 3.7, 3.8, 3.9 and 3.10. The > > > > supported platforms and architectures are: > > > > - Windows x86_64 (windows/) > > > > - glibc-based Linux x86_64 (linux-glibc-x86_64/) > > > > - glibc-based Linux arm64 (linux-glibc-arm64/) > > > > - musl-based Linux x86_64 (linux-musl-x86_64/) > > > > - musl-based Linux arm64 (linux-musl-arm64/) > > > > - macOS universal 2 (macos/) > > > > > > > > The tag to be voted upon: v3.0.0-candidate-3 > > > > (46acc487ad16fdc0aeea9dae64484030e62c1b96) > > > > > > > > > > >> https://github.com/apache/pulsar-client-python/releases/tag/v3.0.0-candidate-3 > > > > > > > > Pulsar's KEYS file containing PGP keys you use to sign the release: > > > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > > > > > Please download the Python wheels and follow the README to test. > > > > > > > > Thanks, > > > > Yunze > > > > > > >> > > >
Re: [DISCUSS] Change the default IO threads and listener threads of Java Client
> I think that we can change this default but we should provide an official API to share the threadpools between the clients. That sounds good to me. Since it will introduce a new API to the client side. I will create a new proposal for the API changes. Thanks, Penghui On Mon, Dec 26, 2022 at 8:21 PM Enrico Olivelli wrote: > I am sorry, I am late on this train. > > Unfortunately if you have to connect using different authentication > parameters you need to open many PulsarClients. > > I think that we can change this default but we should provide an official > API to share the threadpools between the clients. > > There is no need to explicitly espose all the threadpools, that is > difficult for users to deal with. > We should have an API to allow sharing resources among multiple clients. > > Interface PulsarClientGroup > > > PulsarClient client = newClient(). > .withGroup(pulsarClientGroup) > > > > > Enrico > > Il Lun 26 Dic 2022, 12:44 Ran Gao ha scritto: > > > It seems this change will not affect the consumer, because the default > > executor is a single-thread executor service, increasing the listener > > threads number will allocate different single-thread executor services to > > different consumers. > > > > On 2022/12/21 09:01:39 Yunze Xu wrote: > > > I have a concern about the message ordering. If we have more than 1 > > > listener thread by default, could messages from the same topic be > > > passed to different listener threads? > > > > > > Thanks, > > > Yunze > > > > > > On Wed, Dec 21, 2022 at 11:12 AM 丛搏 wrote: > > > > > > > > +1 > > > > Our default configuration is best for most users. Multiple clients > are > > > > a few cases. > > > > > > > > Thanks, > > > > Bo > > > > > > > > houxiaoyu 于2022年12月20日周二 16:02写道： > > > > > > > > > > +1 > > > > > > > > > > This change might bring thread number increment in case users > create > > many > > > > > clients, but too many pulsar clients run in one machine is not a > > good use > > > > > case I think, so this change looks good to me. > > > > > > > > > > Thanks, > > > > > Xiaoyu Hou > > > > > > > > > > 于2022年12月20日周二 12:25写道： > > > > > > > > > > > +1 > > > > > > > > > > > > My concern is whether this change will affect some users who are > > creating > > > > > > many clients. I think we can wait for other users to confirm it. > > (If this > > > > > > will be affected, maybe we can give it a max_io_thread_num and > > then expand > > > > > > the size from 1 to max_io_thread_num when adding a new consumer > or > > producer) > > > > > > > > > > > > > > > > > > Best, > > > > > > Mattison > > > > > > On Dec 20, 2022, 11:17 +0800, PengHui Li , > > wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > I noticed the Java Client (I haven't checked other clients) > uses > > 1 IO > > > > > > > thread and 1 listener > > > > > > > thread by default. It will require users to update the thread > > > > > > configuration > > > > > > > if they have > > > > > > > multiple cores and desired high throughput. > > > > > > > > > > > > > > Here is the example that we change to 16 IO threads in > > > > > > > openmessaging benchmark > > > > > > > > > > > > > > > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > > > > > > > > > > > We can apply the configuration of the threads based on the CPU > > cores. So > > > > > > > that for the > > > > > > > most common cases, users don't need to touch the thread > > configuration. > > > > > > > > > > > > > > ``` > > > > > > > private int numIoThreads = > > Runtime.getRuntime().availableProcessors(); > > > > > > > private int numListenerThreads = > > > > > > Runtime.getRuntime().availableProcessors(); > > > > > > > ``` > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > Thanks, > > > > > > > Penghui > > > > > > > > > > > >
Re: [PROPOSAL] Website precommit and move the source of docs to the site repo
FYI tracking issue has been created: https://github.com/apache/pulsar/issues/19064 I plan to finish it by the end of next month. Best, tison. tison 于2022年12月21日周三 11:33写道： > Thanks for your feedback! > > @Yu > > Thanks for sharing the previous thread. I looped in @michaeljmarshall > here. > > @Jun > > It's possible but causes a new shortcoming: Now you should tell the > contributor that the versioned docs are different from the NEXT version > docs, lol. > > If our developers don't complain about these separated sources. Like @Asaf > comment: > > > We can take, let's say, five features and see if they were actually done > in > > the same PR or separate PR. I guess that most documentation is actually > > updated separately. Thus, from that perspective, maybe it’s not a con. > > Then we can do this refactor thoroughgoing. > > Also, if we keep, somehow several sources in the main repo. We still have > shortcomings: > > 1. Duplicated CI workflows. > 2. Cumbersome preview scaffolding in the main repo. > > ... which is the original purpose I'd like to overcome. > > Best, > tison. > > > Jun Ma 于2022年12月21日周三 11:19写道： > >> Is it possible to come up with a compromised solution that has the pros >> of both sides but minimizes the side effect? I'm thinking maybe it's not >> necessary to sacrifice the current contribution process, as long as it can >> greatly reduce the load of back-end actions and source size. For example, >> if we only move out the versioned docs to the site repo but keep the source >> of the NEXT docs in the pulsar repo, does this help to win a large >> proportion of those pros when people can still contribute as usual? >> >> >> From: Jiaqi Shen >> Sent: Tuesday, December 20, 2022 17:15 >> To: dev@pulsar.apache.org >> Subject: Re: [PROPOSAL] Website precommit and move the source of docs to >> the site repo >> >> +1, it makes sense to me. >> >> Thanks, >> Jiaqi Shen >> >> >> Yu 于2022年12月19日周一 20:57写道： >> >> > Hi tison, >> > >> > Thanks for raising this up! >> > >> > Our community had a similar discussion previously and chose to "keep" >> the >> > doc repo stay in the Pulsar main repo at that time. >> > >> > [1] lists the pros and cons of "keep" and "not keep" solutions. >> > >> > I'm +0 on this proposal because I think the total scores of these two >> > solutions are almost equal after weighing the pros and cons. >> > >> > >> > >> > [1] https://lists.apache.org/thread/mf2xwntfgn84dq78ksqv22jk3drq6xb3 >> > >> > >> > On Mon, Dec 19, 2022 at 5:40 PM tison wrote: >> > >> > > Thanks for your feedback! >> > > >> > > @Asaf >> > > >> > > > pre-commit >> > > >> > > I mean CI checks before merging a patch. Currently, we don't run >> checks >> > for >> > > the content before merging them. This causes a series of syntax errors >> > and >> > > broken links issues. If we hold docs under site2 folder in the main >> repo >> > > and then copied to the site repo, we have two places to build such CI >> > > checks. What's worse, the checks for the main repo will be quite >> > > cumbersome (that you do some if-else logic in the whole Pulsar CI >> > > workflows, and do the sync sequentially in that workflow). >> > > >> > > If we hold the source of docs only in the site repo, we can extend the >> > > "precommit" workflow[1] I added recently to check for syntax errors >> and >> > > broken links also. >> > > >> > > > What does the apache/pulsar-site repo contain today? >> > > >> > > It should be covered by the documentation guide page[2]. It holds the >> > > source of the official website and the user docs are synced from the >> main >> > > repo. >> > > >> > > > What content do we have today in the pulsar repo related to the >> site? >> > > >> > > After issue-18014[3] is done, we host only user docs and some JSON >> > metadata >> > > in the main repo, which is synced by site_syncer.py[4]. >> > > >> > > > Can you explain that better? Are you saying pulsar source JARs >> contain >> > > the documentation? >> > > >> > > No. Source JARs contain only the Java files and necessary copyrights >> > info. >> > > The source release is, for example, >> > > >> > > >> > >> https://archive.apache.org/dist/pulsar/pulsar-2.10.2/apache-pulsar-2.10.2-src.tar.gz >> > > , >> > > which is extracted to 173M where 129M is occupied by the site2 folder. >> > > >> > > This also affects when developers do git clone to clone the repo. >> > > >> > > > I mean, if you wish to document a bug fix in 2.9.x, for example, >> would >> > > you do it in the 2.9.x branch under site2/docs or >> > > site2/website/versioned_docs/2.9.5? >> > > >> > > This is another question. Ideally, we should have hosted versioned >> docs >> > > associated with the specific version to that branch, like Apache Flink >> > does >> > > as I mentioned[5]. But we do not, and actually the situation is we >> update >> > > the versioned docs under the master branch and thus, the docs can be >> > synced >> > > properly. >> > > >>
[VOTE] Pulsar Release 2.11.0 Candidate-4
This is the fourth release candidate for Apache Pulsar, version 2.11.0. This release contains 1616 commits by 61 contributors. https://github.com/apache/pulsar/compare/v2.10.2...v2.11.0-candidate-4 CI for this release candidate https://github.com/Technoboy-/pulsar/pull/24 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Note that we are voting upon the source (tag), binaries are provided for convenience. Source and binary files: https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.11.0-candidate-4 SHA-512 checksums: 7d2a19bdf6faa2cc8d7b09f5ccf8b79c05b5f3b9d7ca206eeece77e148bcfc2945bd5df223c8edf52468822b756c65a6c00a1fc1e3097b4938209c822831e3a4 ./apache-pulsar-2.11.0-bin.tar.gz 19ee12544e719c1fd5a78d6863cf9373116dbdf8797e684ed749e06e2c30f4738818ca0b0e6db7807c2da04f31d3616f7e767339f3775c512918598fb7a61335 ./apache-pulsar-2.11.0-src.tar.gz Maven staging repo: https://repository.apache.org/content/repositories/orgapachepulsar-1202/ The tag to be voted upon: v2.11.0-candidate-4 (0247b634cb684504c38fed1096323ceea7ac899a) https://github.com/apache/pulsar/releases/tag/v2.11.0-candidate-4 Pulsar's KEYS file containing PGP keys we use to sign the release: https://dist.apache.org/repos/dist/dev/pulsar/KEYS Docker images: https://hub.docker.com/layers/mattison/pulsar-all/2.11.0-rc4/images/sha256-b905fc8966658beddc5eafd1ed43529db9493dd0bbfa5b770766a87cc573a04c https://hub.docker.com/layers/mattison/pulsar/2.11.0-rc4/images/sha256-f9527e475a992b210fcd0e3badaa8ac976938afedb1faf998f6475cef6b18eb7 Please download the source package, and follow the release-candidate-validation doc to build and run the Pulsar standalone service. https://pulsar.apache.org/contribute/validate-release-candidate Since the metadata store is changed from ZK to RocksDB, the verification of the `stateful functions` needs to set the parameter "export PULSAR_STANDALONE_USE_ZOOKEEPER=1" Regards Jiwei Guo (Tboy)
Re: [VOTE] Pulsar Release 2.11.0 Candidate-4
+1 (binding) - Checked the signature - Build from source - Checked license - Start standalone with zookeeper - Done some simple performance test( publish, consume, and drain backlog) - Checked function - Checked Cassandra connector - Checked stateful function Thanks, Penghui On Tue, Dec 27, 2022 at 8:32 AM guo jiwei wrote: > This is the fourth release candidate for Apache Pulsar, version 2.11.0. > > This release contains 1616 commits by 61 contributors. > https://github.com/apache/pulsar/compare/v2.10.2...v2.11.0-candidate-4 > > CI for this release candidate > https://github.com/Technoboy-/pulsar/pull/24 > > *** Please download, test and vote on this release. This vote will stay > open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.11.0-candidate-4 > > SHA-512 checksums: > > > 7d2a19bdf6faa2cc8d7b09f5ccf8b79c05b5f3b9d7ca206eeece77e148bcfc2945bd5df223c8edf52468822b756c65a6c00a1fc1e3097b4938209c822831e3a4 > > ./apache-pulsar-2.11.0-bin.tar.gz > > > 19ee12544e719c1fd5a78d6863cf9373116dbdf8797e684ed749e06e2c30f4738818ca0b0e6db7807c2da04f31d3616f7e767339f3775c512918598fb7a61335 > > ./apache-pulsar-2.11.0-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1202/ > > The tag to be voted upon: > v2.11.0-candidate-4 (0247b634cb684504c38fed1096323ceea7ac899a) > https://github.com/apache/pulsar/releases/tag/v2.11.0-candidate-4 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Docker images: > > https://hub.docker.com/layers/mattison/pulsar-all/2.11.0-rc4/images/sha256-b905fc8966658beddc5eafd1ed43529db9493dd0bbfa5b770766a87cc573a04c > > https://hub.docker.com/layers/mattison/pulsar/2.11.0-rc4/images/sha256-f9527e475a992b210fcd0e3badaa8ac976938afedb1faf998f6475cef6b18eb7 > > Please download the source package, and follow the > release-candidate-validation doc to build > and run the Pulsar standalone service. > https://pulsar.apache.org/contribute/validate-release-candidate > > Since the metadata store is changed from ZK to RocksDB, the verification of > the `stateful functions` needs to set the parameter "export > PULSAR_STANDALONE_USE_ZOOKEEPER=1" > > > > > Regards > Jiwei Guo (Tboy) >
[DISCUSS] Merge two conflicting configuration items about message dispacth throttling
Hi, pulsar community: We introduced dispatch throttling as part of PIP-3 which helps brokers to control fan-out and prevent broker's instability due to large message dispatch. There are two strategies for batch message sending: 1. Enable the conf `preciseDispatcherFlowControl`: Cache the avg number of messages per entry and estimate how many entries need to be read when the message is delivered (see #6719), which makes more precise message counting. 2. Enable the conf `dispatchThrottlingOnBatchMessageEnabled`: Support rate-limiting dispatching on the batch messages rather than individual messages within batch messages(see #12294). This improves throughput while keeping the BK load stable. But if both features are turned on simultaneously, neither will work well because they are mutually exclusive. I have two solutions to this problem: 1. Define priorities for the two configurations. If the configuration with a higher priority is enabled, another configuration is disabled automatically. 2. Merge two configurations into one `dispatchThrottlingModeOnBatchMessage`, value list is `none,` `precise,` `based on batch.`(prefer) Thanks Yubiao Feng
