Without a PIP, the knowledge scatters out into many different places and many different conversations. a PIP is better since other people in the community can participate in the development if they are interested in helping this.
- Sijie On Sun, Dec 1, 2019 at 8:53 PM Ali Ahmed <alahmed...@gmail.com> wrote: > Sure for unit test the plan is relatively simple. I have isolated a core > stable base which we can activate as a checkin condition. > > > https://github.com/apache/pulsar/pull/5642/files#diff-00053b72be028281c6f9a446fbf91ef5 > > The unstable tests will be need to rewritten before being removed from > exclusion we have benefits of dedicated machines for testing in github so > we can give dedicated ports an an example to avoid contention. > This should take about a week a two or work. > > Integration tests are a bigger issue I don't think we can stabilize the > current tests easily, we probably want to reorg things, move > external elements like spark, storm, flink and various io connectors in a > different module so we have lighter test load that we can harden against > failures. > > I can compile this into a PIP if community want's that. > > -Ali > > > > On Sun, Dec 1, 2019 at 8:37 PM Sijie Guo <guosi...@gmail.com> wrote: > > > Ali, > > > > Thank you for driving this. Overall looks good. > > > > I am fine with most of the proposal. However I would like to see what is > > the plan to migrate the remaining java tests and integration tests. > > In your current email, there are a lot of unknowns regarding java unit > > tests and integration tests. I would guess putting your proposal into > > a PIP so that everyone can review the PIP before voting it. > > > > Thanks, > > Sijie > > > > > > On Sat, Nov 30, 2019 at 4:39 PM Ali Ahmed <alahmed...@gmail.com> wrote: > > > > > Hello everyone , > > > > > > I am doing experiments with github CI and I believe we are ready to > vote > > on > > > the transition plan. > > > Briefly the current jenkins env is problematic there is resources > > > contention, limited parallelism and lack of control to do experiments. > > > > > > The first version of the github actions ci is ready to merge. It has > > > additional benefits that it won't trigger on documentation changes. > > > https://github.com/apache/pulsar/pull/5642 > > > > > > They were things tried but removed like caching artifacts, since the > > cache > > > size is too small to use. > > > > > > From the current jobs in the workflow the cpp tests is stable enough to > > > swap with the jenkins one we can merge this and request apache infra to > > > disable the cpp jenkins job requirement. > > > I am experimenting with running unit tests and integration test in > > parallel > > > groups with subset of tests. It's not ready yet to be a checkin > > condition, > > > we need to iterate and improve it further. > > > > > > In summary the goal is to merge the current pr as is, swap the jenkins > > cpp > > > job and then further along move the unit test and integration test > > workflow > > > as they stabilize. > > > > > >
