👍 Best Regards.
Jia Zhai Beijing, China Mobile: +86 15810491983 On Mon, Apr 13, 2020 at 2:40 AM Sijie Guo <guosi...@gmail.com> wrote: > It seems there are no objections to this PIP. I am proceeding to create 3 > new repos to kick off the effort. > > https://github.com/apache/pulsar-connectors > https://github.com/apache/pulsar-adapters > https://github.com/apache/pulsar-presto > > - Sijie > > > On Sat, Apr 11, 2020 at 10:35 AM Sanjeev Kulkarni <sanjee...@gmail.com> > wrote: > > > +1 > > > > On Thu, Apr 9, 2020 at 11:54 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > Sanjeev, Matteo, > > > > > > > Will these > > > > additional repositories build against the latest pulsar release? Or > > will > > > > they build against say a nightly build of master? > > > > > > > testing dependents modules against the latest version, > > > > > > Here is my original proposal - "The three new repositories will run > their > > > Github Action based CI tests. The CI tests will use the latest build > from > > > the main pulsar repo. The pulsarbot can be enhanced to add a common > > action > > > to install Pulsar client dependencies from main repo before running any > > > actions." > > > > > > I was proposing adding a common github action which fetches the latest > > code > > > from main repo and build the libraries and install them before running > > > other actions. > > > So it ensures all the three repos in-sync with main repo. This approach > > is > > > easy to start and we can reuse existing Github workflows when those > > > components are moved to separated repos. If the built time for those > new > > > repos is still a concern, we can find a way to improve later. > > > > > > Let me know what you think. > > > > > > - Sijie > > > > > > > > > On Thu, Apr 9, 2020 at 9:42 PM Matteo Merli <matteo.me...@gmail.com> > > > wrote: > > > > > > > +1 > > > > > > > > I think there are still some unknowns, but they will only be > > > > discovered once the effort is started. > > > > > > > > Regarding, testing dependents modules against the latest version, we > > > > could rely on publishing snapshot Jars (which we already do on a > daily > > > > basis) and snapshot Docker images. It's not perfect, but it would be > > > > easier to setup until we can get a more rigorous daily CI pipeline > > > > that cascades the artifacts across builds. In any case, we should > have > > > > a ~daily build against the "latest" master in order to avoid > > > > regressions. > > > > > > > > > > > > -- > > > > Matteo Merli > > > > <matteo.me...@gmail.com> > > > > > > > > On Thu, Apr 9, 2020 at 1:11 PM Sanjeev Kulkarni <sanjee...@gmail.com > > > > > > wrote: > > > > > > > > > > Thanks Sijie for writing this proposal. > > > > > I whole-heartedly agree with this proposal. Not only does it reduce > > > > Pulsar > > > > > build time and make our lives as pulsar developers simple, it will > > also > > > > > make connector/adaptor contributions less frictionless for > > > contributors. > > > > No > > > > > longer will developers need to know about or care to compile the > core > > > > > pulsar. This could result in more contributions from non-core > > > developers. > > > > > I took a look at the document and it doesn't address one point. > Will > > > > these > > > > > additional repositories build against the latest pulsar release? Or > > > will > > > > > they build against say a nightly build of master? > > > > > Thanks! > > > > > > > > > > On Thu, Apr 9, 2020 at 12:57 PM Sijie Guo <si...@apache.org> > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I chatted with some committers a few months ago about the build > > time > > > > of the > > > > > > current main repo. I'd like to raise a discussion of moving > > > connectors, > > > > > > adapters, and presto-connector to separate repositories so that > we > > > can > > > > > > reduce the build time of the main repo and improve the review > > process > > > > for > > > > > > individual components. > > > > > > > > > > > > I wrote the proposal in PIP-62. Please review it and let me know > > what > > > > you > > > > > > think. > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-62:-Move-connectors,-adapters-and-Pulsar-Presto-to-separate-repositories > > > > > > > > > > > > Thanks, > > > > > > Sijie > > > > > > > > > > > > > > > >
