Re: Question about Pulsar gRPC client(s)

2023-10-31 Thread Kiryl Valkovich
* So I think we could not replace completely the Pulsar protocol by gRPC. Sure, I understand that it’s unlikely possible to implement a full replacement. I’m rather talking about it as a better, more type-safe alternative to WebSocket and REST APIs, with a similar amount of supported features

Re: Question about Pulsar gRPC client(s)

2023-10-31 Thread Dave Fisher
I think that Christophe suggested the protocol handler approach like KoP (also forked as DataStax Starlight for Kafka). Due to the several people on the PMC contributing to that the performance gains were huge and it nearly as performant as Pulsar. Best, Dave Sent from my iPhone > On Oct 31,

Re: Question about Pulsar gRPC client(s)

2023-10-31 Thread Yunze Xu
It's a protocol handler like https://github.com/streamnative/kop that supports the Kafka protocol. The protocol handler should not be maintained in the core repo because there are a few people maintaining the plugin. And just like Christophe and Zike said, compared with the native Pulsar protocol,

Re: Question about Pulsar gRPC client(s)

2023-10-31 Thread Zike Yang
Another point is that there are many features implemented on the client side, including batching, chunking, DLQ, etc. This makes it hard to replace the existing pulsar clients completely. Zike Yang On Wed, Nov 1, 2023 at 4:43 AM Christophe Bornet wrote: > > Hi Kiryl, > > Thanks for mentioning p

Re: Pulsar CI broken due to Eclipse Temurin repository broken

2023-10-31 Thread Lari Hotari
https://github.com/apache/pulsar/pull/21489 solved the problem "Error reading from server. Remote end closed connection" while downloading Eclipse Temurin apt packages. I have cherry-picked this to branch-3.0 and branch-3.1 as well. Pulsar CI is now unblocked. -Lari On Tue, 31 Oct 2023 at 20:42,

JVM's SIGSEGV crash bug JDK-8314024 possibly impacting Pulsar

2023-10-31 Thread Lari Hotari
Hi all, On CI, some pulsar-metadata tests fail frequently with JVM crashes. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7f10670d5113, pid=3902, tid=3941 # # JRE version: OpenJDK Runtime Environment Temurin-17.0.8.1+1 (17.0.8.1+1) (build 17.0

[DISCUSS] PIP-315: Configurable max delay limit for delayed delivery

2023-10-31 Thread Kevin Lu
Hi All, I just submitted a PIP to add an optional config to limit the max delay allowed for delayed delivery. This would be helpful for Pulsar administrators if they wanted to provide some limits on delayed delivery. Link - https://github.com/apache/pulsar/pull/21490 Thanks, Kevin

Re: Question about Pulsar gRPC client(s)

2023-10-31 Thread Christophe Bornet
Hi Kiryl, Thanks for mentioning pulsar-grpc. Indeed, using gRPC simplifies the implementation of the networking logic (keep-alive, reconnection, flow control,…). On the other hand, the Java gRPC implementation makes a lot of buffer copies to cleanly separate the network and app layers but that tak

Question about Pulsar gRPC client(s)

2023-10-31 Thread Kiryl Valkovich
Hi! Am I understanding it right, that if this project https://github.com/cbornet/pulsar-grpc is merged to the apache/pulsar repo, we could easily cover non-mainstream platforms that are supported by gRPC, but don't have ready-to-use Pulsar clients? https://github.com/apache/pulsar/wiki/PIP-59:-

Re: Pulsar CI broken due to Eclipse Temurin repository broken

2023-10-31 Thread Lari Hotari
Attempt to fix the problem: https://github.com/apache/pulsar/pull/21489 -Lari On Mon, 30 Oct 2023, 20.33 Lari Hotari, wrote: > There's another problem related to the Eclipse Temurin repository. > Sometimes the Pulsar docker image building fails with an error "Error > reading from server. Remote