Thanks Ali work for this, overall looks good to me. Currently, Jenkins does not check the unit tests and code specifications related to Go, as follows:
- pulsar-client-go - pulsar-functions-go In GitHub action CI. Can we consider integrating them? https://github.com/apache/pulsar/issues/3774 <https://github.com/apache/pulsar/issues/3774> -- Thanks Xiaolong Ran > 在 2019年12月3日，上午8:52，Ali Ahmed <alahmed...@gmail.com> 写道： > > Here is PIP document > > https://docs.google.com/document/d/13QKvfTj4-AMt3j7JTSE7LcyiXBK0Z0EGzp5HgMwpoj4/edit# > > On Sun, Dec 1, 2019 at 9:13 PM Sijie Guo <guosi...@gmail.com> wrote: > >> Without a PIP, the knowledge scatters out into many different places and >> many different conversations. >> a PIP is better since other people in the community can participate in the >> development if they are interested in helping this. >> >> >> - Sijie >> >> On Sun, Dec 1, 2019 at 8:53 PM Ali Ahmed <alahmed...@gmail.com> wrote: >> >>> Sure for unit test the plan is relatively simple. I have isolated a core >>> stable base which we can activate as a checkin condition. >>> >>> >>> >> https://github.com/apache/pulsar/pull/5642/files#diff-00053b72be028281c6f9a446fbf91ef5 >>> >>> The unstable tests will be need to rewritten before being removed from >>> exclusion we have benefits of dedicated machines for testing in github so >>> we can give dedicated ports an an example to avoid contention. >>> This should take about a week a two or work. >>> >>> Integration tests are a bigger issue I don't think we can stabilize the >>> current tests easily, we probably want to reorg things, move >>> external elements like spark, storm, flink and various io connectors in a >>> different module so we have lighter test load that we can harden against >>> failures. >>> >>> I can compile this into a PIP if community want's that. >>> >>> -Ali >>> >>> >>> >>> On Sun, Dec 1, 2019 at 8:37 PM Sijie Guo <guosi...@gmail.com> wrote: >>> >>>> Ali, >>>> >>>> Thank you for driving this. Overall looks good. >>>> >>>> I am fine with most of the proposal. However I would like to see what >> is >>>> the plan to migrate the remaining java tests and integration tests. >>>> In your current email, there are a lot of unknowns regarding java unit >>>> tests and integration tests. I would guess putting your proposal into >>>> a PIP so that everyone can review the PIP before voting it. >>>> >>>> Thanks, >>>> Sijie >>>> >>>> >>>> On Sat, Nov 30, 2019 at 4:39 PM Ali Ahmed <alahmed...@gmail.com> >> wrote: >>>> >>>>> Hello everyone , >>>>> >>>>> I am doing experiments with github CI and I believe we are ready to >>> vote >>>> on >>>>> the transition plan. >>>>> Briefly the current jenkins env is problematic there is resources >>>>> contention, limited parallelism and lack of control to do >> experiments. >>>>> >>>>> The first version of the github actions ci is ready to merge. It has >>>>> additional benefits that it won't trigger on documentation changes. >>>>> https://github.com/apache/pulsar/pull/5642 >>>>> >>>>> They were things tried but removed like caching artifacts, since the >>>> cache >>>>> size is too small to use. >>>>> >>>>> From the current jobs in the workflow the cpp tests is stable enough >> to >>>>> swap with the jenkins one we can merge this and request apache infra >> to >>>>> disable the cpp jenkins job requirement. >>>>> I am experimenting with running unit tests and integration test in >>>> parallel >>>>> groups with subset of tests. It's not ready yet to be a checkin >>>> condition, >>>>> we need to iterate and improve it further. >>>>> >>>>> In summary the goal is to merge the current pr as is, swap the >> jenkins >>>> cpp >>>>> job and then further along move the unit test and integration test >>>> workflow >>>>> as they stabilize. >>>>> >>>> >>> >>
