It can take days to get a jenkins pass, we should be focused on migrating what we have first adding new workflows on a stable base is easy to do.
-Ali On Tue, Dec 3, 2019 at 8:04 PM Sijie Guo <guosi...@gmail.com> wrote: > Ali, Xiaolong, > > Instead of waiting for the whole pull request to pass, how about adding > those go unit tests and code style verification as new Github Action? > > So we can start moving towards Github Action. > > Thoughts? > > Thanks, > Sijie > > On Tue, Dec 3, 2019 at 7:54 PM Ali Ahmed <alahmed...@gmail.com> wrote: > > > Absolutely we launch 30 jobs in parallel per pr so it should not be a > > problem, once we finish migration of current jobs we can start the > process > > of adding new ones. > > > > -Ali > > > > On Tue, Dec 3, 2019 at 6:39 PM xiaolong ran <ranxiaolong...@gmail.com> > > wrote: > > > > > Thanks Ali work for this, overall looks good to me. > > > > > > Currently, Jenkins does not check the unit tests and code > specifications > > > related > > > to Go, as follows: > > > > > > - pulsar-client-go > > > - pulsar-functions-go > > > > > > > > > In GitHub action CI. Can we consider integrating them? > > > https://github.com/apache/pulsar/issues/3774 < > > > https://github.com/apache/pulsar/issues/3774> > > > > > > > > > -- > > > Thanks > > > Xiaolong Ran > > > > > > > > > > > > > 在 2019年12月3日，上午8:52，Ali Ahmed <alahmed...@gmail.com> 写道： > > > > > > > > Here is PIP document > > > > > > > > > > > > > > https://docs.google.com/document/d/13QKvfTj4-AMt3j7JTSE7LcyiXBK0Z0EGzp5HgMwpoj4/edit# > > > > > > > > On Sun, Dec 1, 2019 at 9:13 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > >> Without a PIP, the knowledge scatters out into many different places > > and > > > >> many different conversations. > > > >> a PIP is better since other people in the community can participate > in > > > the > > > >> development if they are interested in helping this. > > > >> > > > >> > > > >> - Sijie > > > >> > > > >> On Sun, Dec 1, 2019 at 8:53 PM Ali Ahmed <alahmed...@gmail.com> > > wrote: > > > >> > > > >>> Sure for unit test the plan is relatively simple. I have isolated a > > > core > > > >>> stable base which we can activate as a checkin condition. > > > >>> > > > >>> > > > >>> > > > >> > > > > > > https://github.com/apache/pulsar/pull/5642/files#diff-00053b72be028281c6f9a446fbf91ef5 > > > >>> > > > >>> The unstable tests will be need to rewritten before being removed > > from > > > >>> exclusion we have benefits of dedicated machines for testing in > > github > > > so > > > >>> we can give dedicated ports an an example to avoid contention. > > > >>> This should take about a week a two or work. > > > >>> > > > >>> Integration tests are a bigger issue I don't think we can stabilize > > the > > > >>> current tests easily, we probably want to reorg things, move > > > >>> external elements like spark, storm, flink and various io > connectors > > > in a > > > >>> different module so we have lighter test load that we can harden > > > against > > > >>> failures. > > > >>> > > > >>> I can compile this into a PIP if community want's that. > > > >>> > > > >>> -Ali > > > >>> > > > >>> > > > >>> > > > >>> On Sun, Dec 1, 2019 at 8:37 PM Sijie Guo <guosi...@gmail.com> > wrote: > > > >>> > > > >>>> Ali, > > > >>>> > > > >>>> Thank you for driving this. Overall looks good. > > > >>>> > > > >>>> I am fine with most of the proposal. However I would like to see > > what > > > >> is > > > >>>> the plan to migrate the remaining java tests and integration > tests. > > > >>>> In your current email, there are a lot of unknowns regarding java > > unit > > > >>>> tests and integration tests. I would guess putting your proposal > > into > > > >>>> a PIP so that everyone can review the PIP before voting it. > > > >>>> > > > >>>> Thanks, > > > >>>> Sijie > > > >>>> > > > >>>> > > > >>>> On Sat, Nov 30, 2019 at 4:39 PM Ali Ahmed <alahmed...@gmail.com> > > > >> wrote: > > > >>>> > > > >>>>> Hello everyone , > > > >>>>> > > > >>>>> I am doing experiments with github CI and I believe we are ready > to > > > >>> vote > > > >>>> on > > > >>>>> the transition plan. > > > >>>>> Briefly the current jenkins env is problematic there is resources > > > >>>>> contention, limited parallelism and lack of control to do > > > >> experiments. > > > >>>>> > > > >>>>> The first version of the github actions ci is ready to merge. It > > has > > > >>>>> additional benefits that it won't trigger on documentation > changes. > > > >>>>> https://github.com/apache/pulsar/pull/5642 > > > >>>>> > > > >>>>> They were things tried but removed like caching artifacts, since > > the > > > >>>> cache > > > >>>>> size is too small to use. > > > >>>>> > > > >>>>> From the current jobs in the workflow the cpp tests is stable > > enough > > > >> to > > > >>>>> swap with the jenkins one we can merge this and request apache > > infra > > > >> to > > > >>>>> disable the cpp jenkins job requirement. > > > >>>>> I am experimenting with running unit tests and integration test > in > > > >>>> parallel > > > >>>>> groups with subset of tests. It's not ready yet to be a checkin > > > >>>> condition, > > > >>>>> we need to iterate and improve it further. > > > >>>>> > > > >>>>> In summary the goal is to merge the current pr as is, swap the > > > >> jenkins > > > >>>> cpp > > > >>>>> job and then further along move the unit test and integration > test > > > >>>> workflow > > > >>>>> as they stabilize. > > > >>>>> > > > >>>> > > > >>> > > > >> > > > > > > > > >
