Absolutely we launch 30 jobs in parallel per pr so it should not be a problem, once we finish migration of current jobs we can start the process of adding new ones.
-Ali On Tue, Dec 3, 2019 at 6:39 PM xiaolong ran <ranxiaolong...@gmail.com> wrote: > Thanks Ali work for this, overall looks good to me. > > Currently, Jenkins does not check the unit tests and code specifications > related > to Go, as follows: > > - pulsar-client-go > - pulsar-functions-go > > > In GitHub action CI. Can we consider integrating them? > https://github.com/apache/pulsar/issues/3774 < > https://github.com/apache/pulsar/issues/3774> > > > -- > Thanks > Xiaolong Ran > > > > > 在 2019年12月3日，上午8:52，Ali Ahmed <alahmed...@gmail.com> 写道： > > > > Here is PIP document > > > > > https://docs.google.com/document/d/13QKvfTj4-AMt3j7JTSE7LcyiXBK0Z0EGzp5HgMwpoj4/edit# > > > > On Sun, Dec 1, 2019 at 9:13 PM Sijie Guo <guosi...@gmail.com> wrote: > > > >> Without a PIP, the knowledge scatters out into many different places and > >> many different conversations. > >> a PIP is better since other people in the community can participate in > the > >> development if they are interested in helping this. > >> > >> > >> - Sijie > >> > >> On Sun, Dec 1, 2019 at 8:53 PM Ali Ahmed <alahmed...@gmail.com> wrote: > >> > >>> Sure for unit test the plan is relatively simple. I have isolated a > core > >>> stable base which we can activate as a checkin condition. > >>> > >>> > >>> > >> > https://github.com/apache/pulsar/pull/5642/files#diff-00053b72be028281c6f9a446fbf91ef5 > >>> > >>> The unstable tests will be need to rewritten before being removed from > >>> exclusion we have benefits of dedicated machines for testing in github > so > >>> we can give dedicated ports an an example to avoid contention. > >>> This should take about a week a two or work. > >>> > >>> Integration tests are a bigger issue I don't think we can stabilize the > >>> current tests easily, we probably want to reorg things, move > >>> external elements like spark, storm, flink and various io connectors > in a > >>> different module so we have lighter test load that we can harden > against > >>> failures. > >>> > >>> I can compile this into a PIP if community want's that. > >>> > >>> -Ali > >>> > >>> > >>> > >>> On Sun, Dec 1, 2019 at 8:37 PM Sijie Guo <guosi...@gmail.com> wrote: > >>> > >>>> Ali, > >>>> > >>>> Thank you for driving this. Overall looks good. > >>>> > >>>> I am fine with most of the proposal. However I would like to see what > >> is > >>>> the plan to migrate the remaining java tests and integration tests. > >>>> In your current email, there are a lot of unknowns regarding java unit > >>>> tests and integration tests. I would guess putting your proposal into > >>>> a PIP so that everyone can review the PIP before voting it. > >>>> > >>>> Thanks, > >>>> Sijie > >>>> > >>>> > >>>> On Sat, Nov 30, 2019 at 4:39 PM Ali Ahmed <alahmed...@gmail.com> > >> wrote: > >>>> > >>>>> Hello everyone , > >>>>> > >>>>> I am doing experiments with github CI and I believe we are ready to > >>> vote > >>>> on > >>>>> the transition plan. > >>>>> Briefly the current jenkins env is problematic there is resources > >>>>> contention, limited parallelism and lack of control to do > >> experiments. > >>>>> > >>>>> The first version of the github actions ci is ready to merge. It has > >>>>> additional benefits that it won't trigger on documentation changes. > >>>>> https://github.com/apache/pulsar/pull/5642 > >>>>> > >>>>> They were things tried but removed like caching artifacts, since the > >>>> cache > >>>>> size is too small to use. > >>>>> > >>>>> From the current jobs in the workflow the cpp tests is stable enough > >> to > >>>>> swap with the jenkins one we can merge this and request apache infra > >> to > >>>>> disable the cpp jenkins job requirement. > >>>>> I am experimenting with running unit tests and integration test in > >>>> parallel > >>>>> groups with subset of tests. It's not ready yet to be a checkin > >>>> condition, > >>>>> we need to iterate and improve it further. > >>>>> > >>>>> In summary the goal is to merge the current pr as is, swap the > >> jenkins > >>>> cpp > >>>>> job and then further along move the unit test and integration test > >>>> workflow > >>>>> as they stabilize. > >>>>> > >>>> > >>> > >> > >
