Replies inline below: On Thu, Sep 2, 2021 at 3:37 AM Sijie Guo <guosi...@gmail.com> wrote:
> Lari, > > Thank you for bringing this up! > > In general, I would love to see this being accepted to the project. > Thanks! > > However, in the past, there were other language clients contributed to the > project. They were not accepted because the PMC doesn't have enough > bandwidth to staff it. > > https://github.com/apache/pulsar/wiki/PIP-77%3A-Contribute-Supernova-to-Apache-Pulsar > is one of the examples. > I don't think that Reactive Streams is comparable to this Supernova example. That is an example of a very marginal language. Reactive Streams support is a mainstream technology that originates from Netflix. It's not about a single programming language. The Reactive Streams is an interoperability specification and there are multiple implementations for the JVM. For example, this Reaction Pulsar Adapter library uses Project Reactor as the implementation, but the library can be used together with Akka Streams, RxJava 3, Vert.x, SmallRye Mutiny (RedHat/Quarkus) and others. This library is not about a single programming language, it's about all languages and platforms on the JVM that support Reactive Streams. It could be an extension directly to the Pulsar Java Client. I just thought that maintaining the project would be easier when it's a separate repository. If that is causing the staffing issue that you referred to being the showstopper issue, I could also restructure the project so that it could be added as modules to apache/pulsar repository. There's a short summary in https://www.reactive.foundation/post/why-reactive-principles-ensure-system-scalability-with-josh-long about the value proposition of Reactive Streams and why it matters. The recent Netflix Techblog explicitly mentions Reactive Streams in the blog post about Netflix's device management platform: https://netflixtechblog.com/towards-a-reliable-device-management-platform-4f86230ca623 "Back-Pressure Support - Because the processing workload varies significantly over time, the solution must first and foremost scale with the message load by providing back-pressure support as defined in the Reactive Streams specification — in other words, the solution should be able to switch between push and pull-based back-pressure models depending on the downstream being able to cope with the message production rate or not." > Also, the project is still relatively new and there is only one > contributor. > Everything has a beginning. :) I was assuming that this would be optimal that there is only one contributor and that other contributions could contaminate the IPR and make it harder to move to the Apache Pulsar project later. As mentioned above, this is more like an extension to the Pulsar Java Client and I could restructure the project so that it could be added as modules to the apache/pulsar repository if the separate repository causes the barrier. > Hence, my recommendation is to grow the contributors for this project so > that there are enough contributors to work on that project. Once there are > sufficient resources and contributors, the PMC can revisit this proposal > and see if the PMC is able to staff it. > Did the PMC already vote on this? :) I hope we could have the discussion before voting. Adding support for Reactive Streams will add a lot of value to Apache Pulsar. I'm talking about the value of Reactive Streams for Apache Pulsar application developers TODAY AT 11:05PT in my SpringOne talk "Reactive Applications with Apache Pulsar and Spring Boot" https://springone.io/2021/sessions/reactive-applications-with-apache-pulsar-and-spring-boot It is free to attend, https://springone.io to register. There's also an AMA session immediately after the session Q&A. Please join. Come and see! I'm looking forward to hearing more voices from the Apache Pulsar community in this email discussion. BR, Lari > > - Sijie > > > On Wed, Sep 1, 2021 at 4:31 AM Lari Hotari <lhot...@apache.org> wrote: > > > Dear Apache Pulsar community members, > > > > I have developed a Reactive Streams adapter for the Apache Pulsar Java > > Client. It uses Project Reactor as the Reactive Streams implementation. > > > > The repository is here: https://github.com/lhotari/reactive-pulsar > > and library is available in Maven Central: > > > https://search.maven.org/search?q=g:com.github.lhotari%20a:reactive-pulsar* > > > > I would like to contribute this library to the Apache Pulsar project. > > I would suggest that the library is committed to a separate repository, > for > > example https://github.com/apache/pulsar-reactive . > > > > # Motivation > > > > There's a need to "go reactive from end-to-end" when building modern > > reactive applications with platforms such as Spring Reactive. > > > > There are ways to adapt the Apache Pulsar Java client async API calls to > > Reactive Streams with a few lines of code. However, a lot will be missing > > and achieving the complete solution will require much more effort. > > > > The primary goal of this library is to provide a Reactive Streams adapter > > that wraps the Apache Pulsar Java client with Reactive APIs > > for sending, reading and consuming messages. > > The secondary goal is to provide a Spring Boot starter for easily > adopting > > the power of Apache Pulsar for Spring Boot applications. > > > > It is currently out of scope for this library to build a reactive client > > all the way from the binary protocol handling. > > This library builds upon the existing Apache Pulsar Java client and uses > > the async API for operations. > > > > # Features > > > > - Simple and intuitive reactive APIs for sending, reading and consuming > > messages. > > - Non-blocking backpressure support for the provided reactive APIs > > - Pulsar client resource lifecycle management that integrates with the > > Reactive programming model > > - Additional producer caching that caches producers. > > - Useful for API backends since the Pulsar client producer resource can > > be shared and cached across multiple REST calls. > > - Support for in-order parallel (concurrent) processing at the > application > > level > > - I/O intensive message processing can leverage Project Reactor's > > capabilities to efficiently > > handle thousands of simultaneous I/O calls to external REST APIs and > > resources that have Reactive clients/drivers. > > - In-order parallel processing support can retain key-order at the > level > > of a single consumer. > > > > I'm presenting the library and it's features tomorrow on September 2nd in > > the SpringOne virtual conference, which is free to attend. > > The session details and schedule are at > > > > > https://springone.io/2021/sessions/reactive-applications-with-apache-pulsar-and-spring-boot > > . > > > > # Licenses > > > > The Reactive Pulsar Adapter library is under the Apache Licence Version > 2.0 > > . The main dependencies are reactor-core > > (https://github.com/reactor/reactor-core) and jctools-core ( > > https://github.com/JCTools/JCTools) which are also > > ASL 2.0 licensed. > > There's also a dependency to Caffeine ( > > https://github.com/ben-manes/caffeine) > > and the Spring Boot Starter > > depends on Spring Boot and Spring Framework. These are all ASL 2.0 > licensed > > libraries. > > The additional Spring integration testing support library depends on > > Testcontainers, which is also ASL 2.0 licensed. > > > > > > I'm looking forward to the discussion about this. > > > > > > Best regards, > > > > Lari Hotari > > >
