Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[Zike Yang]

Hi, Yunze

> However, with the latest example, the output should be the following
> logs if you pressed the Ctrl+C:

Thanks for your explanation. It works fine for me now.

> I think it should not be a blocker, we can open an issue for that. The
> official example uses a try-except block to avoid this issue.

Will you highlight this in the release note? It seems it has changed
the default behavior. It's better to provide the user with good
practice for this.

Thanks,
Zike Yang

On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu  wrote:
>
> The reason is caused by the `client.close()` not being called. You
> should make sure `client.close()` is called when receiving a SIGINT
> signal.
>
> ```
> #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone .cold] ()
>from 
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
>from 
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> #10 0x7f195006a71b in
> pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> ()
>from 
> /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
>from 
> /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> #12 0x7f195003ac5f in pybind11_object_dealloc ()
>from 
> /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> Objects/dictobject.c:1905
> #14 0x7f1950acc284 in subtype_clear (self= 0x7f1950273cd0>) at Objects/typeobject.c:1123
> #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> at Modules/gcmodule.c:761
> #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> nofail=1) at Modules/gcmodule.c:913
> #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at Modules/gcmodule.c:1602
> #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> Modules/main.c:3127
> #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> argv=) at Modules/main.c:3160
> #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> , argc=2, argv=0x7fff65e5b668,
> --Type  for more, q to quit, c to continue without paging--
> t>, fini=, rtld_fini=,
> stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> #23 0x55e11cf1b08a in _start ()
> ```
>
> I think it should not be a blocker, we can open an issue for that. The
> official example uses a try-except block to avoid this issue.
>
> Thanks,
> Yunze
>
> On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> >
> > Hi Zike,
> >
> > Did you run the latest example? I still see the exceptional info:
> >
> > ```
> > line 1243, in receive
> > msg = self._consumer.receive()
> > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > ```
> >
> > However, with the latest example, the output should be the following
> > logs if you pressed the Ctrl+C:
> >
> > ```
> > ^CStop receiving messages
> > ```
> >
> > Because the exception from `consumer.receive()`  is caught.
> >
> > ```python3
> > try:
> > msg = consumer.receive()
> > # ...
> > except pulsar.Interrupted:
> > print("Stop receiving messages")
> > break
> > ```
> >
> > Thanks,
> > Yunze
> >
> > On Thu, Mar 9, 2023 at 3:02 PM Zike Yang  wrote:
> > >
> > > Hi Yunze
> > >
> > > The crash issue still exists in python 3.7. Here is the log
> > > ```
> > > ^CTraceback (most recent call last):
> > >   File 
> > > "/Users/aaronrobert/codebase/pulsar-client-python/examples/consumer.py",
> > > line 32, in 
> > > msg = consumer.receive()
> > >   File 
> > > "/Users/aaronrobert/.pyenv/versions/3.7.16/lib/python3.7/site-packages/pulsar/__init__.py",
> > > line 1243, in receive
> > > msg = self._consumer.receive()
> > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > 2023-03-09 12:18:14.326 WARN  [0x110900600] ConsumerImpl:126 |
> > > [persistent://public/default/my-topic, my-subscription, 0] Destroyed
> > > consumer which was not properly closed
> > > 2023-03-09 12:18:14.326 INFO  [0x110900600] ConsumerImpl:134 |
> > > [persistent://public/default/my-topic, my-subscription, 0] Closed
> > > consumer for race condition: 0
> > > libc++abi: terminating with uncaught exception of type
> > > std::__1::bad_weak_ptr: bad_weak_ptr
> > > [1]52874 abort  python37
> > > ~/codebase/pulsar-client-python/examples/consumer.py
> > > ```
> > >
> > > This issue also exists in python 3.10.8. But it worked fine when I
> > > upgraded it to the latest version of python 3.10: 3.10.10.
> > > However, python 3.7.16, which is the latest version of python 3.7
> > > still not working.
> > >
> > > Not sure if it's a python issue, but o

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[PengHui Li]

+1(binding)

- Install the wheel on macOS (python 3.9)
- Start a standalone
- Run the consumer and producer example

Thanks,
Penghui

On Fri, Mar 10, 2023 at 4:25 PM Zike Yang  wrote:

> Hi, Yunze
>
> > However, with the latest example, the output should be the following
> > logs if you pressed the Ctrl+C:
>
> Thanks for your explanation. It works fine for me now.
>
> > I think it should not be a blocker, we can open an issue for that. The
> > official example uses a try-except block to avoid this issue.
>
> Will you highlight this in the release note? It seems it has changed
> the default behavior. It's better to provide the user with good
> practice for this.
>
> Thanks,
> Zike Yang
>
> On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu 
> wrote:
> >
> > The reason is caused by the `client.close()` not being called. You
> > should make sure `client.close()` is called when receiving a SIGINT
> > signal.
> >
> > ```
> > #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone .cold]
> ()
> >from
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
> >from
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > #10 0x7f195006a71b in
> >
> pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> > ()
> >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
> >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > #12 0x7f195003ac5f in pybind11_object_dealloc ()
> >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> > Objects/dictobject.c:1905
> > #14 0x7f1950acc284 in subtype_clear (self= > 0x7f1950273cd0>) at Objects/typeobject.c:1123
> > #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> > <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> > at Modules/gcmodule.c:761
> > #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> > nofail=1) at Modules/gcmodule.c:913
> > #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at
> Modules/gcmodule.c:1602
> > #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> > #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> > #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> > Modules/main.c:3127
> > #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> > argv=) at Modules/main.c:3160
> > #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> > , argc=2, argv=0x7fff65e5b668,
> > --Type  for more, q to quit, c to continue without paging--
> > t>, fini=, rtld_fini=,
> > stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> > #23 0x55e11cf1b08a in _start ()
> > ```
> >
> > I think it should not be a blocker, we can open an issue for that. The
> > official example uses a try-except block to avoid this issue.
> >
> > Thanks,
> > Yunze
> >
> > On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> > >
> > > Hi Zike,
> > >
> > > Did you run the latest example? I still see the exceptional info:
> > >
> > > ```
> > > line 1243, in receive
> > > msg = self._consumer.receive()
> > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > ```
> > >
> > > However, with the latest example, the output should be the following
> > > logs if you pressed the Ctrl+C:
> > >
> > > ```
> > > ^CStop receiving messages
> > > ```
> > >
> > > Because the exception from `consumer.receive()`  is caught.
> > >
> > > ```python3
> > > try:
> > > msg = consumer.receive()
> > > # ...
> > > except pulsar.Interrupted:
> > > print("Stop receiving messages")
> > > break
> > > ```
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Thu, Mar 9, 2023 at 3:02 PM Zike Yang  wrote:
> > > >
> > > > Hi Yunze
> > > >
> > > > The crash issue still exists in python 3.7. Here is the log
> > > > ```
> > > > ^CTraceback (most recent call last):
> > > >   File
> "/Users/aaronrobert/codebase/pulsar-client-python/examples/consumer.py",
> > > > line 32, in 
> > > > msg = consumer.receive()
> > > >   File
> "/Users/aaronrobert/.pyenv/versions/3.7.16/lib/python3.7/site-packages/pulsar/__init__.py",
> > > > line 1243, in receive
> > > > msg = self._consumer.receive()
> > > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > > 2023-03-09 12:18:14.326 WARN  [0x110900600] ConsumerImpl:126 |
> > > > [persistent://public/default/my-topic, my-subscription, 0] Destroyed
> > > > consumer which was not properly closed
> > > > 2023-03-09 12:18:14.326 INFO  [0x110900600] ConsumerImpl:134 |
> > > > [persistent://public/default/my-topic, my-subscription, 0] Closed
> > > > consumer for race condition: 0
> > > > libc++abi: terminating with uncaught exception of type
> > > > std::__1::bad_weak_

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[Zike Yang]

+1 (non-binding)

My environment:
* macOS x86_64
* python 3.7, 3.9, and 3.10

- Check sha512 checksum
- Check gpg signature
- Install the wheel
- Run the producer and consumer example

Thanks,
Zike Yang

On Fri, Mar 10, 2023 at 4:27 PM PengHui Li  wrote:
>
> +1(binding)
>
> - Install the wheel on macOS (python 3.9)
> - Start a standalone
> - Run the consumer and producer example
>
> Thanks,
> Penghui
>
> On Fri, Mar 10, 2023 at 4:25 PM Zike Yang  wrote:
>
> > Hi, Yunze
> >
> > > However, with the latest example, the output should be the following
> > > logs if you pressed the Ctrl+C:
> >
> > Thanks for your explanation. It works fine for me now.
> >
> > > I think it should not be a blocker, we can open an issue for that. The
> > > official example uses a try-except block to avoid this issue.
> >
> > Will you highlight this in the release note? It seems it has changed
> > the default behavior. It's better to provide the user with good
> > practice for this.
> >
> > Thanks,
> > Zike Yang
> >
> > On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu 
> > wrote:
> > >
> > > The reason is caused by the `client.close()` not being called. You
> > > should make sure `client.close()` is called when receiving a SIGINT
> > > signal.
> > >
> > > ```
> > > #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone .cold]
> > ()
> > >from
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
> > >from
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > #10 0x7f195006a71b in
> > >
> > pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> > > ()
> > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
> > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #12 0x7f195003ac5f in pybind11_object_dealloc ()
> > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> > > Objects/dictobject.c:1905
> > > #14 0x7f1950acc284 in subtype_clear (self= > > 0x7f1950273cd0>) at Objects/typeobject.c:1123
> > > #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> > > <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> > > at Modules/gcmodule.c:761
> > > #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> > > nofail=1) at Modules/gcmodule.c:913
> > > #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at
> > Modules/gcmodule.c:1602
> > > #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> > > #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> > > #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> > > Modules/main.c:3127
> > > #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> > > argv=) at Modules/main.c:3160
> > > #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> > > , argc=2, argv=0x7fff65e5b668,
> > > --Type  for more, q to quit, c to continue without paging--
> > > t>, fini=, rtld_fini=,
> > > stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> > > #23 0x55e11cf1b08a in _start ()
> > > ```
> > >
> > > I think it should not be a blocker, we can open an issue for that. The
> > > official example uses a try-except block to avoid this issue.
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> > > >
> > > > Hi Zike,
> > > >
> > > > Did you run the latest example? I still see the exceptional info:
> > > >
> > > > ```
> > > > line 1243, in receive
> > > > msg = self._consumer.receive()
> > > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > > ```
> > > >
> > > > However, with the latest example, the output should be the following
> > > > logs if you pressed the Ctrl+C:
> > > >
> > > > ```
> > > > ^CStop receiving messages
> > > > ```
> > > >
> > > > Because the exception from `consumer.receive()`  is caught.
> > > >
> > > > ```python3
> > > > try:
> > > > msg = consumer.receive()
> > > > # ...
> > > > except pulsar.Interrupted:
> > > > print("Stop receiving messages")
> > > > break
> > > > ```
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > > On Thu, Mar 9, 2023 at 3:02 PM Zike Yang  wrote:
> > > > >
> > > > > Hi Yunze
> > > > >
> > > > > The crash issue still exists in python 3.7. Here is the log
> > > > > ```
> > > > > ^CTraceback (most recent call last):
> > > > >   File
> > "/Users/aaronrobert/codebase/pulsar-client-python/examples/consumer.py",
> > > > > line 32, in 
> > > > > msg = consumer.receive()
> > > > >   File
> > "/Users/aaronrobert/.pyenv/versions/3.7.16/lib/python3.7/site-packages/pulsar/__init__.py",
> > > > > line 1243, in receive
> > > > > msg = self._consumer.receive()
> > > > >

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[Yunze Xu]

Hi Zike,

It's not a regression. This error applies for Python client 3.0.0 as
well. Before 3.0.0, the Python client cannot be interrupted by a
signal. I'd rather treat it as the case that the implementation of
handling the UNIX signals is not so perfect. Here is the issue:
https://github.com/apache/pulsar-client-python/issues/103

Thanks,
Yunze

On Fri, Mar 10, 2023 at 4:25 PM Zike Yang  wrote:
>
> Hi, Yunze
>
> > However, with the latest example, the output should be the following
> > logs if you pressed the Ctrl+C:
>
> Thanks for your explanation. It works fine for me now.
>
> > I think it should not be a blocker, we can open an issue for that. The
> > official example uses a try-except block to avoid this issue.
>
> Will you highlight this in the release note? It seems it has changed
> the default behavior. It's better to provide the user with good
> practice for this.
>
> Thanks,
> Zike Yang
>
> On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu  wrote:
> >
> > The reason is caused by the `client.close()` not being called. You
> > should make sure `client.close()` is called when receiving a SIGINT
> > signal.
> >
> > ```
> > #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone .cold] ()
> >from 
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
> >from 
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > #10 0x7f195006a71b in
> > pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> > ()
> >from 
> > /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> > #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
> >from 
> > /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> > #12 0x7f195003ac5f in pybind11_object_dealloc ()
> >from 
> > /usr/local/lib/python3.7/site-packages/_pulsar.cpython-37m-x86_64-linux-gnu.so
> > #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> > Objects/dictobject.c:1905
> > #14 0x7f1950acc284 in subtype_clear (self= > 0x7f1950273cd0>) at Objects/typeobject.c:1123
> > #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> > <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> > at Modules/gcmodule.c:761
> > #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> > nofail=1) at Modules/gcmodule.c:913
> > #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at Modules/gcmodule.c:1602
> > #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> > #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> > #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> > Modules/main.c:3127
> > #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> > argv=) at Modules/main.c:3160
> > #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> > , argc=2, argv=0x7fff65e5b668,
> > --Type  for more, q to quit, c to continue without paging--
> > t>, fini=, rtld_fini=,
> > stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> > #23 0x55e11cf1b08a in _start ()
> > ```
> >
> > I think it should not be a blocker, we can open an issue for that. The
> > official example uses a try-except block to avoid this issue.
> >
> > Thanks,
> > Yunze
> >
> > On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> > >
> > > Hi Zike,
> > >
> > > Did you run the latest example? I still see the exceptional info:
> > >
> > > ```
> > > line 1243, in receive
> > > msg = self._consumer.receive()
> > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > ```
> > >
> > > However, with the latest example, the output should be the following
> > > logs if you pressed the Ctrl+C:
> > >
> > > ```
> > > ^CStop receiving messages
> > > ```
> > >
> > > Because the exception from `consumer.receive()`  is caught.
> > >
> > > ```python3
> > > try:
> > > msg = consumer.receive()
> > > # ...
> > > except pulsar.Interrupted:
> > > print("Stop receiving messages")
> > > break
> > > ```
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Thu, Mar 9, 2023 at 3:02 PM Zike Yang  wrote:
> > > >
> > > > Hi Yunze
> > > >
> > > > The crash issue still exists in python 3.7. Here is the log
> > > > ```
> > > > ^CTraceback (most recent call last):
> > > >   File 
> > > > "/Users/aaronrobert/codebase/pulsar-client-python/examples/consumer.py",
> > > > line 32, in 
> > > > msg = consumer.receive()
> > > >   File 
> > > > "/Users/aaronrobert/.pyenv/versions/3.7.16/lib/python3.7/site-packages/pulsar/__init__.py",
> > > > line 1243, in receive
> > > > msg = self._consumer.receive()
> > > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > > 2023-03-09 12:18:14.326 WARN  [0x110900600] ConsumerImpl:126 |
> > > > [persistent://public/default/my-topic, my-subscription, 0] Destroyed
> > > > consumer which was not properly closed
> > > > 2023-03-09 12:18:14.326 INFO  [0

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[Matteo Merli]

+1 (binding)
--
Matteo Merli



On Fri, Mar 10, 2023 at 7:44 AM Yunze Xu 
wrote:

> Hi Zike,
>
> It's not a regression. This error applies for Python client 3.0.0 as
> well. Before 3.0.0, the Python client cannot be interrupted by a
> signal. I'd rather treat it as the case that the implementation of
> handling the UNIX signals is not so perfect. Here is the issue:
> https://github.com/apache/pulsar-client-python/issues/103
>
> Thanks,
> Yunze
>
> On Fri, Mar 10, 2023 at 4:25 PM Zike Yang  wrote:
> >
> > Hi, Yunze
> >
> > > However, with the latest example, the output should be the following
> > > logs if you pressed the Ctrl+C:
> >
> > Thanks for your explanation. It works fine for me now.
> >
> > > I think it should not be a blocker, we can open an issue for that. The
> > > official example uses a try-except block to avoid this issue.
> >
> > Will you highlight this in the release note? It seems it has changed
> > the default behavior. It's better to provide the user with good
> > practice for this.
> >
> > Thanks,
> > Zike Yang
> >
> > On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu 
> wrote:
> > >
> > > The reason is caused by the `client.close()` not being called. You
> > > should make sure `client.close()` is called when receiving a SIGINT
> > > signal.
> > >
> > > ```
> > > #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone
> .cold] ()
> > >from
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
> > >from
> /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > #10 0x7f195006a71b in
> > >
> pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> > > ()
> > >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
> > >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #12 0x7f195003ac5f in pybind11_object_dealloc ()
> > >from /usr/local/lib/python3.7/site-packages/_
> pulsar.cpython-37m-x86_64-linux-gnu.so
> > > #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> > > Objects/dictobject.c:1905
> > > #14 0x7f1950acc284 in subtype_clear (self= > > 0x7f1950273cd0>) at Objects/typeobject.c:1123
> > > #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> > > <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> > > at Modules/gcmodule.c:761
> > > #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> > > nofail=1) at Modules/gcmodule.c:913
> > > #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at
> Modules/gcmodule.c:1602
> > > #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> > > #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> > > #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> > > Modules/main.c:3127
> > > #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> > > argv=) at Modules/main.c:3160
> > > #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> > > , argc=2, argv=0x7fff65e5b668,
> > > --Type  for more, q to quit, c to continue without paging--
> > > t>, fini=, rtld_fini=,
> > > stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> > > #23 0x55e11cf1b08a in _start ()
> > > ```
> > >
> > > I think it should not be a blocker, we can open an issue for that. The
> > > official example uses a try-except block to avoid this issue.
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> > > >
> > > > Hi Zike,
> > > >
> > > > Did you run the latest example? I still see the exceptional info:
> > > >
> > > > ```
> > > > line 1243, in receive
> > > > msg = self._consumer.receive()
> > > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > > ```
> > > >
> > > > However, with the latest example, the output should be the following
> > > > logs if you pressed the Ctrl+C:
> > > >
> > > > ```
> > > > ^CStop receiving messages
> > > > ```
> > > >
> > > > Because the exception from `consumer.receive()`  is caught.
> > > >
> > > > ```python3
> > > > try:
> > > > msg = consumer.receive()
> > > > # ...
> > > > except pulsar.Interrupted:
> > > > print("Stop receiving messages")
> > > > break
> > > > ```
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > > On Thu, Mar 9, 2023 at 3:02 PM Zike Yang  wrote:
> > > > >
> > > > > Hi Yunze
> > > > >
> > > > > The crash issue still exists in python 3.7. Here is the log
> > > > > ```
> > > > > ^CTraceback (most recent call last):
> > > > >   File
> "/Users/aaronrobert/codebase/pulsar-client-python/examples/consumer.py",
> > > > > line 32, in 
> > > > > msg = consumer.receive()
> > > > >   File
> "/Users/aaronrobert/.pyenv/versions/3.7.16/lib/python3.7/site-packages/pulsar/__init__.py",
> > > > > line 1243, in receive
> > > > > msg = self._cons

[VOTE] PIP-250: Add proxyVersion to CommandConnect

[Michael Marshall]

Hello Pulsar Community,

This thread is to start the vote for PIP 250.

Discussion: https://lists.apache.org/thread/ntzwkdz23d6s6ycyn2r198c515npp7vt
Issue: https://github.com/apache/pulsar/issues/19623
Work in Progress Implementation: https://github.com/apache/pulsar/pull/19618

I'll update the implementation to align with Enrico's proposal on the
discussion thread.

Voting typically stays open for at least 48h. Since I am starting this
on a Friday, I plan to keep it open for at least 96 hours (ending on
Tuesday).

Thanks,
Michael

[VOTE] PIP-253 Expose producer metrics for deadLetterProducer and retryLetterProducer

[Kai Levy]

Hello,

This thread is to start a vote for PIP-253

Discussion: https://lists.apache.org/thread/570x5yodg0bog4thgtstrgkgpthxmgfd
Issue: https://github.com/apache/pulsar/issues/19698

Voting will be open at least 48 hours.

Thanks!
Kai

Re: [VOTE] PIP-253 Expose producer metrics for deadLetterProducer and retryLetterProducer

[Michael Marshall]

+1 (binding)

Thanks for your PIP! Feel free to tag me to help review the PR when we
get to that step.

Thanks,
Michael

On Fri, Mar 10, 2023 at 2:27 PM Kai Levy  wrote:
>
> Hello,
>
> This thread is to start a vote for PIP-253
>
> Discussion: https://lists.apache.org/thread/570x5yodg0bog4thgtstrgkgpthxmgfd
> Issue: https://github.com/apache/pulsar/issues/19698
>
> Voting will be open at least 48 hours.
>
> Thanks!
> Kai

Re: [VOTE] PIP-253 Expose producer metrics for deadLetterProducer and retryLetterProducer

[Enrico Olivelli]

+1 (binding)

Enrico

Il Ven 10 Mar 2023, 21:36 Michael Marshall  ha
scritto:

> +1 (binding)
>
> Thanks for your PIP! Feel free to tag me to help review the PR when we
> get to that step.
>
> Thanks,
> Michael
>
> On Fri, Mar 10, 2023 at 2:27 PM Kai Levy  wrote:
> >
> > Hello,
> >
> > This thread is to start a vote for PIP-253
> >
> > Discussion:
> https://lists.apache.org/thread/570x5yodg0bog4thgtstrgkgpthxmgfd
> > Issue: https://github.com/apache/pulsar/issues/19698
> >
> > Voting will be open at least 48 hours.
> >
> > Thanks!
> > Kai
>

Re: [VOTE] PIP-250: Add proxyVersion to CommandConnect

[Enrico Olivelli]

+1 (binding)

Enrico

Il Ven 10 Mar 2023, 20:31 Michael Marshall  ha
scritto:

> Hello Pulsar Community,
>
> This thread is to start the vote for PIP 250.
>
> Discussion:
> https://lists.apache.org/thread/ntzwkdz23d6s6ycyn2r198c515npp7vt
> Issue: https://github.com/apache/pulsar/issues/19623
> Work in Progress Implementation:
> https://github.com/apache/pulsar/pull/19618
>
> I'll update the implementation to align with Enrico's proposal on the
> discussion thread.
>
> Voting typically stays open for at least 48h. Since I am starting this
> on a Friday, I plan to keep it open for at least 96 hours (ending on
> Tuesday).
>
> Thanks,
> Michael
>

Re: [VOTE] PIP-250: Add proxyVersion to CommandConnect

[ZhangJian He]

+1 (non-binding)

Thanks
ZhangJian He


On Sat, 11 Mar 2023 at 05:36, Enrico Olivelli  wrote:

> +1 (binding)
>
> Enrico
>
> Il Ven 10 Mar 2023, 20:31 Michael Marshall  ha
> scritto:
>
> > Hello Pulsar Community,
> >
> > This thread is to start the vote for PIP 250.
> >
> > Discussion:
> > https://lists.apache.org/thread/ntzwkdz23d6s6ycyn2r198c515npp7vt
> > Issue: https://github.com/apache/pulsar/issues/19623
> > Work in Progress Implementation:
> > https://github.com/apache/pulsar/pull/19618
> >
> > I'll update the implementation to align with Enrico's proposal on the
> > discussion thread.
> >
> > Voting typically stays open for at least 48h. Since I am starting this
> > on a Friday, I plan to keep it open for at least 96 hours (ending on
> > Tuesday).
> >
> > Thanks,
> > Michael
> >
>

Re: [DISCUSS] PIP-253: Expose producer metrics for deadLetterProducer and retryLetterProducer

[Kai Levy]

I made a start on the implementation with `setRetryLetterProducer`.
Unfortunately, the spotbugs plugin has issues with it, because now
ConsumerImpl is returning a mutable object from getStats. You can see my
progress here
.
The spotbugs errors are below:

[ERROR] Medium: org.apache.pulsar.client.impl.ConsumerImpl.getStats() may
expose internal representation by returning ConsumerImpl.stats
[org.apache.pulsar.client.impl.ConsumerImpl] At ConsumerImpl.java:[line
2518] EI_EXPOSE_REP
[ERROR] Medium:
org.apache.pulsar.client.impl.MultiTopicsConsumerImpl.getStats() may expose
internal representation by returning MultiTopicsConsumerImpl.stats
[org.apache.pulsar.client.impl.MultiTopicsConsumerImpl] At
MultiTopicsConsumerImpl.java:[line 851] EI_EXPOSE_REP


I would appreciate input on the best path forward.

Thanks,
Kai

On Tue, Mar 7, 2023 at 9:49 AM Kai Levy  wrote:

> Yes, that would work.
>
> Kai
>
> On Tue, Mar 7, 2023 at 12:41 AM Asaf Mesika  wrote:
>
>> On Mon, Mar 6, 2023 at 6:24 PM Kai Levy  wrote:
>>
>> > I agree, adding it to the ConsumerStats interface makes more logical
>> sense,
>> > but I believe the implementation will be harder that way, since the
>> > producers are lazily initialized. They won't be available when
>> > ConsumerStats is created, and there isn't currently a way to access them
>> > directly from the consumer.
>> >
>> >
>> In `ConsumerImp` you have
>>
>> private volatile Producer retryLetterProducer;
>>
>> You can just add setRetryLetterProducer on `ConsumerStatsRecorder`
>>
>>
>>
>> Kai
>> >
>> > On Sun, Mar 5, 2023 at 5:19 AM Asaf Mesika 
>> wrote:
>> >
>> > > I would rather see them as attributes of ConsumerStats .
>> > > Add
>> > >
>> > > ProducerStats deadLetterProducerStats;
>> > >
>> > > ProducerStats retryLetterProducerStats();
>> > >
>> > >
>> > > On Fri, Mar 3, 2023 at 2:54 AM Kai Levy  wrote:
>> > >
>> > > > Hello!
>> > > >
>> > > > I created a new PIP because I discovered there's no way for a user
>> to
>> > > > access the metrics for a consumer's deadLetterProducer /
>> > > > retryLetterProducer, since it is private to ConsumerImpl.java. I
>> would
>> > > like
>> > > > to propose an API change that would expose those statistics. More
>> > details
>> > > > on the github issue:
>> > > > https://github.com/apache/pulsar/issues/19698
>> > > >
>> > > > Thanks!
>> > > > Kai
>> > > >
>> > >
>> >
>>
>

Re: [VOTE] PIP-250: Add proxyVersion to CommandConnect

[Yubiao Feng]

+1 (non-binding)

Thanks
Yubiao Feng

On Sat, Mar 11, 2023 at 3:31 AM Michael Marshall 
wrote:

> Hello Pulsar Community,
>
> This thread is to start the vote for PIP 250.
>
> Discussion:
> https://lists.apache.org/thread/ntzwkdz23d6s6ycyn2r198c515npp7vt
> Issue: https://github.com/apache/pulsar/issues/19623
> Work in Progress Implementation:
> https://github.com/apache/pulsar/pull/19618
>
> I'll update the implementation to align with Enrico's proposal on the
> discussion thread.
>
> Voting typically stays open for at least 48h. Since I am starting this
> on a Friday, I plan to keep it open for at least 96 hours (ending on
> Tuesday).
>
> Thanks,
> Michael
>

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 4

[丛搏]

+1 (binding)

python version: 3.7.6
- Checked the signature
- Install the python .whl file
(pulsar_client-3.1.0-cp37-cp37m-macosx_10_15_universal2.whl) on macOS
12.3.1
- Start the standalone (2.11.0)
- Start consumer (python3 ./examples/consumer.py)
- Start producer (python3 ./examples/producer.py)
- ./tests/run-unit-tests.sh without interrupted_test.py
(apachepulsar/pulsar:2.11.0)

Thanks,
Bo

Matteo Merli  于2023年3月11日周六 00:15写道：
>
> +1 (binding)
> --
> Matteo Merli
> 
>
>
> On Fri, Mar 10, 2023 at 7:44 AM Yunze Xu 
> wrote:
>
> > Hi Zike,
> >
> > It's not a regression. This error applies for Python client 3.0.0 as
> > well. Before 3.0.0, the Python client cannot be interrupted by a
> > signal. I'd rather treat it as the case that the implementation of
> > handling the UNIX signals is not so perfect. Here is the issue:
> > https://github.com/apache/pulsar-client-python/issues/103
> >
> > Thanks,
> > Yunze
> >
> > On Fri, Mar 10, 2023 at 4:25 PM Zike Yang  wrote:
> > >
> > > Hi, Yunze
> > >
> > > > However, with the latest example, the output should be the following
> > > > logs if you pressed the Ctrl+C:
> > >
> > > Thanks for your explanation. It works fine for me now.
> > >
> > > > I think it should not be a blocker, we can open an issue for that. The
> > > > official example uses a try-except block to avoid this issue.
> > >
> > > Will you highlight this in the release note? It seems it has changed
> > > the default behavior. It's better to provide the user with good
> > > practice for this.
> > >
> > > Thanks,
> > > Zike Yang
> > >
> > > On Thu, Mar 9, 2023 at 10:10 PM Yunze Xu 
> > wrote:
> > > >
> > > > The reason is caused by the `client.close()` not being called. You
> > > > should make sure `client.close()` is called when receiving a SIGINT
> > > > signal.
> > > >
> > > > ```
> > > > #8  0x7f194f6aca99 in pulsar::ConsumerImpl::shutdown() [clone
> > .cold] ()
> > > >from
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > > #9  0x7f194f79dd74 in pulsar::ConsumerImpl::~ConsumerImpl() ()
> > > >from
> > /usr/local/lib/python3.7/site-packages/pulsar_client.libs/libpulsar-c7b16888.so
> > > > #10 0x7f195006a71b in
> > > >
> > pybind11::class_::dealloc(pybind11::detail::value_and_holder&)
> > > > ()
> > > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > > #11 0x7f1950039f8e in pybind11::detail::clear_instance(_object*) ()
> > > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > > #12 0x7f195003ac5f in pybind11_object_dealloc ()
> > > >from /usr/local/lib/python3.7/site-packages/_
> > pulsar.cpython-37m-x86_64-linux-gnu.so
> > > > #13 0x7f1950a47927 in dict_dealloc (mp=0x7f194f302320) at
> > > > Objects/dictobject.c:1905
> > > > #14 0x7f1950acc284 in subtype_clear (self= > > > 0x7f1950273cd0>) at Objects/typeobject.c:1123
> > > > #15 0x7f1950aa793d in delete_garbage (old=0x7f1950c4a000
> > > > <_PyRuntime+448>, collectable=0x7fff65e5b2a0)
> > > > at Modules/gcmodule.c:761
> > > > #16 collect (generation=2, n_collected=0x0, n_uncollectable=0x0,
> > > > nofail=1) at Modules/gcmodule.c:913
> > > > #17 0x7f1950af92a2 in _PyGC_CollectNoFail () at
> > Modules/gcmodule.c:1602
> > > > #18 0x7f1950aec340 in PyImport_Cleanup () at Python/import.c:526
> > > > #19 0x7f1950aee5a8 in Py_FinalizeEx () at Python/pylifecycle.c:1199
> > > > #20 0x7f1950af6b48 in pymain_main (pymain=0x7fff65e5b510) at
> > > > Modules/main.c:3127
> > > > #21 0x7f1950af6a1e in _Py_UnixMain (argc=,
> > > > argv=) at Modules/main.c:3160
> > > > #22 0x7f1950768d0a in __libc_start_main (main=0x55e11cf1b050
> > > > , argc=2, argv=0x7fff65e5b668,
> > > > --Type  for more, q to quit, c to continue without paging--
> > > > t>, fini=, rtld_fini=,
> > > > stack_end=0x7fff65e5b658) at ../csu/libc-start.c:308
> > > > #23 0x55e11cf1b08a in _start ()
> > > > ```
> > > >
> > > > I think it should not be a blocker, we can open an issue for that. The
> > > > official example uses a try-except block to avoid this issue.
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > > On Thu, Mar 9, 2023 at 9:48 PM Yunze Xu  wrote:
> > > > >
> > > > > Hi Zike,
> > > > >
> > > > > Did you run the latest example? I still see the exceptional info:
> > > > >
> > > > > ```
> > > > > line 1243, in receive
> > > > > msg = self._consumer.receive()
> > > > > _pulsar.Interrupted: Pulsar error: ResultInterrupted
> > > > > ```
> > > > >
> > > > > However, with the latest example, the output should be the following
> > > > > logs if you pressed the Ctrl+C:
> > > > >
> > > > > ```
> > > > > ^CStop receiving messages
> > > > > ```
> > > > >
> > > > > Because the exception from `consumer.receive()`  is caught.
> > > > >
> > > > > ```python3
> > > > > try:
> > > > > msg = consumer.receive()
> > > > > # ...
> > > > > except pulsar.Interrupted:
> >

Re: [DISCUSS] Retry topic should not create for a retry topic

[丛搏]

+1, I agree with you.

We should prohibit users from creating Retry Topic and DLQ Topic in a
loop. It will make uncontrollable behavior.
If we allow that, It may cause great trouble to users.

Thanks,
Bo


Asaf Mesika  于2023年2月14日周二 03:02写道：

>
> Sounds like a bug for sure.
> How did you plan on solving it?
>
>
> On Mon, Feb 13, 2023 at 12:46 AM Enrico Olivelli 
> wrote:
>
> > Il Dom 12 Feb 2023, 04:42 Yubiao Feng  > .invalid>
> > ha scritto:
> >
> > > Hi Enrico Olivelli
> > >
> > > > It is good to help users to not fall into bad situations but on the
> > other
> > > case we cannot deal with many silly configurations that you could set up,
> > > like creating a pipeline of functions that in the end create a cycle.
> > >
> > > Sorry, this test just helps to reproduce the problem quickly. The reality
> > > is that there is only one consumer, but every restart triggers this issue
> > > and ends up with a topic like this:
> > > "persistent://public/default/tp1-sub1-RETRY-sub1-RETRY-sub1-RETRY"
> > >
> > > > I wonder if we could simply document this fact instead of adding code
> > >
> > > ```java
> > > Consumer<> consumer = pulsarClient.newConsumer()
> > > .topicsPattern("my-property/my-ns/.*").subscriptionName("sub1")
> > > .enableRetry(true)
> > > ```
> > >
> > > With the client restarted, the code above will reproduce the problem.
> > >
> >
> >
> > I see the problem now.
> >
> > We must do something for this case. It must not happen. We have to fix it
> >
> > Thanks for your clarification
> >
> > Enrico
> >
> > >
> > > On Sun, Feb 12, 2023 at 3:31 AM Enrico Olivelli 
> > > wrote:
> > >
> > > > Yubiao,
> > > >
> > > > Il Sab 11 Feb 2023, 19:06 Yubiao Feng  > > > .invalid>
> > > > ha scritto:
> > > >
> > > > > Hi community
> > > > >
> > > > > I am starting a DISCUSS for "Retry topic should not create for a
> > retry
> > > > > topic."
> > > > >
> > > > > If we use regex-topic consumer and enable retry, it is possible to
> > > create
> > > > > such a topic
> > > > >
> > "persistent://public/default/tp1-sub1-RETRY-sub2-RETRY-sub3-RETRY".
> > > > You
> > > > > can reproduce this by using the test below.
> > > > >
> > > > > It probably doesn't make sense to create a RETRY/DLQ topic on
> > > RETRY/DLQ.
> > > > We
> > > > > should avoid this scenario if users use the default configuration
> > > (users
> > > > > can enable it if they need it).
> > > > >
> > > >
> > > > I agree that this is a bad case.
> > > > But should we really care?
> > > >
> > > > You must do it very intentionally.
> > > > It is good to help users to not fall into bad situations but on the
> > other
> > > > case we cannot deal with many silly configurations that you could set
> > up,
> > > > like creating a pipeline of functions that in the end create a cycle.
> > > >
> > > >
> > > > I wonder if we could simply document this fact instead of adding code
> > > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > >
> > > > > ```java
> > > > > @Test
> > > > > public void testRetryTopicWillNotCreatedForRetryTopic() throws
> > > > > Exception {
> > > > > final String topic = "persistent://my-property/my-ns/tp1";
> > > > > Producer producer =
> > > > pulsarClient.newProducer(Schema.BYTES)
> > > > > .topic(topic)
> > > > > .create();
> > > > > for (int i = 0; i < 100; i++) {
> > > > > producer.send(String.format("Hello Pulsar [%d]",
> > > > > i).getBytes());
> > > > > }
> > > > > producer.close();
> > > > >
> > > > > for (int i =0; i< 10; i++) {
> > > > > Consumer consumer =
> > > > > pulsarClient.newConsumer(Schema.BYTES)
> > > > > .topicsPattern("my-property/my-ns/.*")
> > > > > .subscriptionName("sub" + i)
> > > > > .enableRetry(true)
> > > > >
> > > > >
> > > >
> > >
> > .deadLetterPolicy(DeadLetterPolicy.builder().maxRedeliverCount(2).build())
> > > > >
> > > > > .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
> > > > > .subscribe();
> > > > > Message message = consumer.receive();
> > > > > log.info("consumer received message : {} {}",
> > > > > message.getMessageId(), new String(message.getData()));
> > > > > consumer.reconsumeLater(message, 1, TimeUnit.SECONDS);
> > > > > consumer.close();
> > > > > }
> > > > >
> > > > > Set tps =
> > > > >
> > > > >
> > > >
> > >
> > pulsar.getBrokerService().getTopics().keys().stream().collect(Collectors.toSet());
> > > > > try {
> > > > > for (String tp : tps) {
> > > > > assertTrue(howManyKeyWordRetryInTopicName(tp,
> > > > > RETRY_GROUP_TOPIC_SUFFIX) <= 1, tp);
> > > > > assertTrue(howManyKeyWordRetryInTopicName(tp,
> > > > > DLQ_GROUP_TOPIC_SUFFIX) <= 1, tp);
> > > > > }
> > > > > } finally {
> > > > > // cleanup.
> > > > > for (String