Sent from my iPhone
> On Sep 19, 2022, at 7:29 PM, Baodi Shi <baodi....@icloud.com.invalid> wrote:
>
> Hi, @merlimat. Looks good to me.
>
>> Both C++ and Python clients will continue with their own individual
>> versioning.
>>
>> In order to not break anything or cause more confusion, we would need to use
>> a new version that is bigger than the current version (2.11.x).
>>
>> The suggestion is to start the new releases for both C++ and Python from
>> 3.0.0.
>
> In the future, Can we need to maintain a compatibility list? For the user,
> how should he choose the appropriate client version to match the broker?
I think that this a question for maintaining documentation for clients
including these Apache Pulsar provided clients.
- Java
- C++
- Python
- Node JS
- Go
- Spring Reactive
There could be a yaml schema defining the main Java client and each client
could cover the portion that they cover. A matrix could be built in the
pulsar-site build.
Maybe swagger files make more sense … again I see a parallel PIP.
Regards,
Dave
>
>>
>> #### Existing branches
>>
>> Existing branches of Pulsar, where the C++ client will still be in the same
>> main
>> the repository and will be receiving bug fixes in their current location.
>>
>> The different location of the new C++ code will make the cherry-picking
>> process
>> slightly more painful in the short term, though it will even out in long
>> term.
>
> The current existing issue, is need to move to the new repository?
>
> Thanks,
> Baodi Shi
>
>> On Sep 20, 2022, at 07:2520, Matteo Merli <mme...@apache.org> wrote:
>>
>> https://github.com/apache/pulsar/issues/17724
>>
>>
>>
>> ## Motivation
>>
>> Pulsar C++ code base is in the same main repository for the Pulsar project.
>>
>> While the decision was the right one at the time, there is a
>> considerable overhead
>> in keeping the C++ client in its current position.
>>
>> ### Issues with the current approach
>>
>> The Pulsar repository has grown a lot in size and number of active
>> developers.
>>
>> 1. The frequency of changes in various parts of the codebase has increased
>> to a
>> point where the amount of resources dedicated to CI is very significant.
>>
>> Every change in Java code will trigger the CI jobs for the C++
>> client and every
>> change in the C++ client will do the same.
>>
>> During a CI job we are building the C++ client multiple times:
>> 1. For C++ and Python client tests
>> 2. To build Python wheels to be included in the pulsar Docker
>> images (for supporting
>> Pulsar functions)
>>
>> 2. The release process for Pulsar has become very complex and
>> requires building a
>> large number of binaries for C++ and Python clients. This has
>> become too much
>> of a burden during the course of a Pulsar release.
>>
>>
>> ## Goal
>>
>> Decouple the development of C++ and Python client libraries from the
>> development
>> of the core components of Pulsar in Java.
>>
>>
>> ## Changes
>>
>> ### Repositories
>>
>> 1. Move the C++ client code to a new repository
>> `github.com/apache/pulsar-client-c++`
>> 2. Move the Python client code to a new repository
>> `github.com/apache/pulsar-client-python`
>>
>> The change will be done without losing any history, extracting a
>> sub-directory into
>> a new Git repository.
>>
>> ```
>> git filter-repo --subdirectory-filter pulsar-client-cpp
>> ```
>>
>> ### Release process
>>
>> The release process will be split in multiple parts:
>>
>> 1. the main Pulsar release will only contain the Java parts (server
>> distribution
>> and Java client library)
>> 2. The C++ client will have its own release schedule and versioning
>> 3. The Python client will have its own release schedule and versioning
>>
>> #### Versioning
>>
>> Both C++ and Python clients will continue with their own individual
>> versioning.
>>
>> In order to not break anything or cause more confusion, we would need to use
>> a new version that is bigger than the current version (2.11.x).
>>
>> The suggestion is to start the new releases for both C++ and Python from
>> 3.0.0.
>>
>>
>> #### Existing branches
>>
>> Existing branches of Pulsar, where the C++ client will still be in the same
>> main
>> the repository and will be receiving bug fixes in their current location.
>>
>> The different location of the new C++ code will make the cherry-picking
>> process
>> slightly more painful in the short term, though it will even out in long
>> term.
>>
>>
>> ### Projects dependencies
>>
>> #### C++/Python --> Pulsar
>>
>> Both C++ and Python unit/integration tests are designed to run against
>> a standalone
>> instance of Pulsar broker. In the current form, they're using the `master`
>> code
>> that is built to run the tests.
>>
>> After the split, the unit tests will use a Docker image of Pulsar. We
>> can use a few
>> different images to test for compatibility
>> 1. Latest stable (eg: 2.10.1)
>> 2. Nightly (Pulsar Docker image published every day from master branch)
>>
>> #### Pulsar --> Python
>>
>> To create a Pulsar image, we are now building the Python client wheel
>> file and then
>> installing it at build time.
>>
>> Instead, we are going to include a wheel file for a version of the Python
>> client
>> that has been already released.
>>
>> #### Python --> C++
>>
>> The Python client library is just a wrapper on top of the C++ client.
>> Today these
>> are built together, with Python wrapper code residing in a
>> sub-directory of C++ client
>> code, and compiled using the same CMake build script.
>>
>> By separating the Python client into a different repository, we are going to
>> depend on an already released version of the C++ client.
>>
>>
>> #### Automated documentation in the website
>>
>> On the Pulsar website we are auto-generating C++ documentation with the
>> Doxygen
>> tool and the Python one with Pdoc.
>>
>> Instead of just fetching the main repo code, the website build job should be
>> also fetching the new repos to run the tooling.
>>
>>
>>
>>
>>
>>
>> --
>> Matteo Merli
>> <mme...@apache.org>
>
