Hi Auto Team,
I've read the CI related wiki page at https://wiki.opnfv.org/display/AUTO/CI+for+Auto and I would like to discuss with you a few thoughts based on my experience with CI from OPNFV vswitchperf project. * I agree, that a dedicated POD should be reserved for Auto CI. Based on the length of the job execution and the content of VERIFY & MERGE jobs, it might be required to introduce an additional (2nd) POD in the future. I would prefer a bare metal POD to virtual one to simplify the final setup and thus improve its robustness. * Based on the configuration, daily jobs can be triggered daily or "daily if there was any commit since the last CI run" (pollSCM trigger). The second setup can easy the load at CI POD - useful in case that POD is shared among several projects or jobs, e.g. more compute power is left for VERIFY/MERGE jobs. I suppose that in case of Auto the frequency will be really daily to constantly verify OPNFV & ONAP (master branch) functionality even if Auto repository was not modified. In case of stable release it doesn't make sense to run it on daily basis as OPNFV & ONAP versions will be fixed. * I agree with the split of functionality between job definition yaml file (in releng repo) and real job "body" inside a Auto repo. This will add a flexibility to the job configuration and it will also allow us to verify new changes as part of verify job during review process before the changes are merged. For example, VSPERF project uses a CI related script ci/build-vsperf.sh, which based on the parameter executes daily, verify or merge job actions. This script is then called from jenkins yaml file (jjb/vswitchperf/vswitchperf.yaml) with appropriate parameter. * In case of auto CI, it might be useful to split CI job into the set of dependent jobs, where following job will be executed only in case that previous job has succeeded. This will simplify analysis of job failures as it will be visible for the first sight if the failure is caused by Auto TCs or by platform installers. The history of these jobs will give us a simple overview of "sub-jobs" stability. E.g. auto-daily-master |---> auto-install-opnfv |----> auto-install-onap |-----> auto-daily-tests example 2: auto-verify-master |---> auto-install-opnfv |----> auto-install-onap |-----> auto-verify code validation... OK doc validation .... OK sanity TC .... OK * It might be useful to prepare some output filtering. Otherwise it would be difficult to read the jenkins job console log during analysis of CI run (especially a failure). It means to suppress console output of successful steps or tests and print only simplified results (as shown in auto-verify example above). This approach expects, that full logs are either accessible (i.e. kept for a while) at POD or dumped into jenkins job console output in case that error is detected. * there are three basic job types used in OPNFV - DAILY, VERIFY (triggered by gerrit for every pushed patch or its change) and MERGE (triggered by gerrit after the merge of the patch). I suppose that in case of AUTO, VERIFY & MERGE jobs can share the content. * Do you plan to store any artifacts "generated" by CI job into OPNFV artifactory? For example, vswitchperf is pushing results into OPNFV results database (I've seen, that AUTO is also going to push to results DB ) and after that it stores appropriate log files and test report into artifacts.opnfv.org. Does it make sense to store logs from OPNFV/ONAP installation or execution of Auto TCs into artifactory too? * It might be useful to easily switch/test different OPNFV/ONAP versions. For example a "master" CI script can import a versions.sh file where user/tester can specify OPNFV and ONAP branches or commit IDs to be used. * Analogically to previous point, also the OPNFV installer and ONAP deployment methods can be specified in the (same?) file. Please consider these thoughts as an "outsider" point of view at Auto CI, as I'm new to the Auto project. It is highly possible that some of these points were already discussed or even solved already. In that case please don't hesitate to point out the sources I should check. Based on the discussion, I'll take care about CI auto wiki page updates, if it will be needed. Thank you, Martin
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
