Sanjeev, Matteo, > Will these > additional repositories build against the latest pulsar release? Or will > they build against say a nightly build of master?
> testing dependents modules against the latest version, Here is my original proposal - "The three new repositories will run their Github Action based CI tests. The CI tests will use the latest build from the main pulsar repo. The pulsarbot can be enhanced to add a common action to install Pulsar client dependencies from main repo before running any actions." I was proposing adding a common github action which fetches the latest code from main repo and build the libraries and install them before running other actions. So it ensures all the three repos in-sync with main repo. This approach is easy to start and we can reuse existing Github workflows when those components are moved to separated repos. If the built time for those new repos is still a concern, we can find a way to improve later. Let me know what you think. - Sijie On Thu, Apr 9, 2020 at 9:42 PM Matteo Merli <matteo.me...@gmail.com> wrote: > +1 > > I think there are still some unknowns, but they will only be > discovered once the effort is started. > > Regarding, testing dependents modules against the latest version, we > could rely on publishing snapshot Jars (which we already do on a daily > basis) and snapshot Docker images. It's not perfect, but it would be > easier to setup until we can get a more rigorous daily CI pipeline > that cascades the artifacts across builds. In any case, we should have > a ~daily build against the "latest" master in order to avoid > regressions. > > > -- > Matteo Merli > <matteo.me...@gmail.com> > > On Thu, Apr 9, 2020 at 1:11 PM Sanjeev Kulkarni <sanjee...@gmail.com> > wrote: > > > > Thanks Sijie for writing this proposal. > > I whole-heartedly agree with this proposal. Not only does it reduce > Pulsar > > build time and make our lives as pulsar developers simple, it will also > > make connector/adaptor contributions less frictionless for contributors. > No > > longer will developers need to know about or care to compile the core > > pulsar. This could result in more contributions from non-core developers. > > I took a look at the document and it doesn't address one point. Will > these > > additional repositories build against the latest pulsar release? Or will > > they build against say a nightly build of master? > > Thanks! > > > > On Thu, Apr 9, 2020 at 12:57 PM Sijie Guo <si...@apache.org> wrote: > > > > > Hi all, > > > > > > I chatted with some committers a few months ago about the build time > of the > > > current main repo. I'd like to raise a discussion of moving connectors, > > > adapters, and presto-connector to separate repositories so that we can > > > reduce the build time of the main repo and improve the review process > for > > > individual components. > > > > > > I wrote the proposal in PIP-62. Please review it and let me know what > you > > > think. > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-62:-Move-connectors,-adapters-and-Pulsar-Presto-to-separate-repositories > > > > > > Thanks, > > > Sijie > > > >
