Hi Tison, Sorry I just missed that. Thanks for your reminder.
Thanks, Yunze > On Sep 20, 2022, at 16:35, tison <wander4...@gmail.com> wrote: > > Hi Yunze, > >> Just wondering if there is a way to retain the git history in the > pulsar-client-cpp directory? > > Matteo's proposal already write: > >> git filter-repo --subdirectory-filter pulsar-client-cpp > > So you will retain the git history. > > Best, > tison. > > > Yunze Xu <y...@streamnative.io.invalid> 于2022年9月20日周二 16:27写道： > >> LGTM. I also listed the related files outside the pulsar-client-cpp >> directory recently: >> >> - pulsar-common/src/main/proto/PulsarApi.proto: the Pulsar binary >> proto file >> - src/gen-pulsar-version-macro.py: generate the internal version info >> - pulsar-client/src/test/proto/*.proto: test the protobuf native >> schema feature >> >> It would not be a complicated job for that. Just wondering if there is >> a way to retain the git history in the pulsar-client-cpp directory? >> >> Thanks, >> Yunze >> >> >> >> >>> On Sep 20, 2022, at 07:25, Matteo Merli <mme...@apache.org> wrote: >>> >>> https://github.com/apache/pulsar/issues/17724 >>> >>> >>> >>> ## Motivation >>> >>> Pulsar C++ code base is in the same main repository for the Pulsar >> project. >>> >>> While the decision was the right one at the time, there is a >>> considerable overhead >>> in keeping the C++ client in its current position. >>> >>> ### Issues with the current approach >>> >>> The Pulsar repository has grown a lot in size and number of active >> developers. >>> >>> 1. The frequency of changes in various parts of the codebase has >> increased to a >>> point where the amount of resources dedicated to CI is very >> significant. >>> >>> Every change in Java code will trigger the CI jobs for the C++ >>> client and every >>> change in the C++ client will do the same. >>> >>> During a CI job we are building the C++ client multiple times: >>> 1. For C++ and Python client tests >>> 2. To build Python wheels to be included in the pulsar Docker >>> images (for supporting >>> Pulsar functions) >>> >>> 2. The release process for Pulsar has become very complex and >>> requires building a >>> large number of binaries for C++ and Python clients. This has >>> become too much >>> of a burden during the course of a Pulsar release. >>> >>> >>> ## Goal >>> >>> Decouple the development of C++ and Python client libraries from the >> development >>> of the core components of Pulsar in Java. >>> >>> >>> ## Changes >>> >>> ### Repositories >>> >>> 1. Move the C++ client code to a new repository >>> `github.com/apache/pulsar-client-c++` >> <http://github.com/apache/pulsar-client-c++> >>> 2. Move the Python client code to a new repository >>> `github.com/apache/pulsar-client-python` >> <http://github.com/apache/pulsar-client-python> >>> >>> The change will be done without losing any history, extracting a >>> sub-directory into >>> a new Git repository. >>> >>> ``` >>> git filter-repo --subdirectory-filter pulsar-client-cpp >>> ``` >>> >>> ### Release process >>> >>> The release process will be split in multiple parts: >>> >>> 1. the main Pulsar release will only contain the Java parts (server >>> distribution >>> and Java client library) >>> 2. The C++ client will have its own release schedule and versioning >>> 3. The Python client will have its own release schedule and versioning >>> >>> #### Versioning >>> >>> Both C++ and Python clients will continue with their own individual >> versioning. >>> >>> In order to not break anything or cause more confusion, we would need to >> use >>> a new version that is bigger than the current version (2.11.x). >>> >>> The suggestion is to start the new releases for both C++ and Python from >> 3.0.0. >>> >>> >>> #### Existing branches >>> >>> Existing branches of Pulsar, where the C++ client will still be in the >> same main >>> the repository and will be receiving bug fixes in their current location. >>> >>> The different location of the new C++ code will make the cherry-picking >> process >>> slightly more painful in the short term, though it will even out in long >> term. >>> >>> >>> ### Projects dependencies >>> >>> #### C++/Python --> Pulsar >>> >>> Both C++ and Python unit/integration tests are designed to run against >>> a standalone >>> instance of Pulsar broker. In the current form, they're using the >> `master` code >>> that is built to run the tests. >>> >>> After the split, the unit tests will use a Docker image of Pulsar. We >>> can use a few >>> different images to test for compatibility >>> 1. Latest stable (eg: 2.10.1) >>> 2. Nightly (Pulsar Docker image published every day from master branch) >>> >>> #### Pulsar --> Python >>> >>> To create a Pulsar image, we are now building the Python client wheel >>> file and then >>> installing it at build time. >>> >>> Instead, we are going to include a wheel file for a version of the >> Python client >>> that has been already released. >>> >>> #### Python --> C++ >>> >>> The Python client library is just a wrapper on top of the C++ client. >>> Today these >>> are built together, with Python wrapper code residing in a >>> sub-directory of C++ client >>> code, and compiled using the same CMake build script. >>> >>> By separating the Python client into a different repository, we are >> going to >>> depend on an already released version of the C++ client. >>> >>> >>> #### Automated documentation in the website >>> >>> On the Pulsar website we are auto-generating C++ documentation with the >> Doxygen >>> tool and the Python one with Pdoc. >>> >>> Instead of just fetching the main repo code, the website build job >> should be >>> also fetching the new repos to run the tooling. >>> >>> >>> >>> >>> >>> >>> -- >>> Matteo Merli >>> <mme...@apache.org> >> >>
