Hi Yunze, > Just wondering if there is a way to retain the git history in the pulsar-client-cpp directory?
Matteo's proposal already write: > git filter-repo --subdirectory-filter pulsar-client-cpp So you will retain the git history. Best, tison. Yunze Xu <y...@streamnative.io.invalid> 于2022年9月20日周二 16:27写道： > LGTM. I also listed the related files outside the pulsar-client-cpp > directory recently: > > - pulsar-common/src/main/proto/PulsarApi.proto: the Pulsar binary > proto file > - src/gen-pulsar-version-macro.py: generate the internal version info > - pulsar-client/src/test/proto/*.proto: test the protobuf native > schema feature > > It would not be a complicated job for that. Just wondering if there is > a way to retain the git history in the pulsar-client-cpp directory? > > Thanks, > Yunze > > > > > > On Sep 20, 2022, at 07:25, Matteo Merli <mme...@apache.org> wrote: > > > > https://github.com/apache/pulsar/issues/17724 > > > > > > > > ## Motivation > > > > Pulsar C++ code base is in the same main repository for the Pulsar > project. > > > > While the decision was the right one at the time, there is a > > considerable overhead > > in keeping the C++ client in its current position. > > > > ### Issues with the current approach > > > > The Pulsar repository has grown a lot in size and number of active > developers. > > > > 1. The frequency of changes in various parts of the codebase has > increased to a > > point where the amount of resources dedicated to CI is very > significant. > > > > Every change in Java code will trigger the CI jobs for the C++ > > client and every > > change in the C++ client will do the same. > > > > During a CI job we are building the C++ client multiple times: > > 1. For C++ and Python client tests > > 2. To build Python wheels to be included in the pulsar Docker > > images (for supporting > > Pulsar functions) > > > > 2. The release process for Pulsar has become very complex and > > requires building a > > large number of binaries for C++ and Python clients. This has > > become too much > > of a burden during the course of a Pulsar release. > > > > > > ## Goal > > > > Decouple the development of C++ and Python client libraries from the > development > > of the core components of Pulsar in Java. > > > > > > ## Changes > > > > ### Repositories > > > > 1. Move the C++ client code to a new repository > > `github.com/apache/pulsar-client-c++` > <http://github.com/apache/pulsar-client-c++> > > 2. Move the Python client code to a new repository > > `github.com/apache/pulsar-client-python` > <http://github.com/apache/pulsar-client-python> > > > > The change will be done without losing any history, extracting a > > sub-directory into > > a new Git repository. > > > > ``` > > git filter-repo --subdirectory-filter pulsar-client-cpp > > ``` > > > > ### Release process > > > > The release process will be split in multiple parts: > > > > 1. the main Pulsar release will only contain the Java parts (server > > distribution > > and Java client library) > > 2. The C++ client will have its own release schedule and versioning > > 3. The Python client will have its own release schedule and versioning > > > > #### Versioning > > > > Both C++ and Python clients will continue with their own individual > versioning. > > > > In order to not break anything or cause more confusion, we would need to > use > > a new version that is bigger than the current version (2.11.x). > > > > The suggestion is to start the new releases for both C++ and Python from > 3.0.0. > > > > > > #### Existing branches > > > > Existing branches of Pulsar, where the C++ client will still be in the > same main > > the repository and will be receiving bug fixes in their current location. > > > > The different location of the new C++ code will make the cherry-picking > process > > slightly more painful in the short term, though it will even out in long > term. > > > > > > ### Projects dependencies > > > > #### C++/Python --> Pulsar > > > > Both C++ and Python unit/integration tests are designed to run against > > a standalone > > instance of Pulsar broker. In the current form, they're using the > `master` code > > that is built to run the tests. > > > > After the split, the unit tests will use a Docker image of Pulsar. We > > can use a few > > different images to test for compatibility > > 1. Latest stable (eg: 2.10.1) > > 2. Nightly (Pulsar Docker image published every day from master branch) > > > > #### Pulsar --> Python > > > > To create a Pulsar image, we are now building the Python client wheel > > file and then > > installing it at build time. > > > > Instead, we are going to include a wheel file for a version of the > Python client > > that has been already released. > > > > #### Python --> C++ > > > > The Python client library is just a wrapper on top of the C++ client. > > Today these > > are built together, with Python wrapper code residing in a > > sub-directory of C++ client > > code, and compiled using the same CMake build script. > > > > By separating the Python client into a different repository, we are > going to > > depend on an already released version of the C++ client. > > > > > > #### Automated documentation in the website > > > > On the Pulsar website we are auto-generating C++ documentation with the > Doxygen > > tool and the Python one with Pdoc. > > > > Instead of just fetching the main repo code, the website build job > should be > > also fetching the new repos to run the tooling. > > > > > > > > > > > > > > -- > > Matteo Merli > > <mme...@apache.org> > >
