Dvorapa added a comment.
In T186208#4998959 <https://phabricator.wikimedia.org/T186208#4998959>, @hashar wrote: > Hello, this got raised to #release-engineering-team <https://phabricator.wikimedia.org/tag/release-engineering-team/> meeting last Monday, thought I was busy with other random duties. > This week will be easier so I gotta review the request but in short Wikimedia CI does not support Travis or AppVeyor at all. I understand. Some other devs already told me you were all busy last week and you are busy in general. > For Travis, it is heavily tight to github and we cant reproduce that with Gerrit and our CI (Jenkins/Zuul/Docker). That being said, most of the time it is very straightforward to rewrite the travis logic to a Jenkins job. The basic thing why we are running Travis and Appveyor is to run unittests and nosetests and some others on multiple devices, like 32bit, 64bit, Win, Linux, Mac, Python 2.7, 3.4, 3.5, 3.6, 3.7 and 3.8. I mean various combinations of those and various presets (pwb env variables on/off, several Wikimedia wikis). The logic of both services are really close to one another, so it should be ok to merge the main points in Wikimedia CI. Also in my opinion there is no need to test py2.7, 3.4 and also perhaps 32bit anymore (as these are slowly ending their career). > AppVeyor I don't think I have ever heard of it. But I guess it is a third party closed source software and we would probably not be able to reproduce it. The config and the logs could be examined to get the idea of what these services are doing: https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/.travis.yml https://travis-ci.org/wikimedia/pywikibot/builds https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/.appveyor.yml https://ci.appveyor.com/project/ladsgroup/pywikibot-g4xqx > Note: we used to have a Jenkins job that ran the pywikibot tests against the #beta-cluster-infrastructure <https://phabricator.wikimedia.org/tag/beta-cluster-infrastructure/> (901f445d1058f55d8ceae23cff4cf7b58537a778 <https://phabricator.wikimedia.org/rCICF901f445d1058f55d8ceae23cff4cf7b58537a778> (T100903 <https://phabricator.wikimedia.org/T100903>). > ) but it was broken/never completed and I have removed it T188256 <https://phabricator.wikimedia.org/T188256>. The job was doing: > > export PYWIKIBOT2_DIR=$WORKSPACE > export TOX_TESTENV_PASSENV=PY_COLORS > export PY_COLORS=1 > tox -e venv -- pip install --upgrade ndg-httpsclient > tox -e venv -- python -m generate_family_file 'http://en.wikipedia.beta.wmflabs.org/' 'wpbeta' 'y' > tox -e venv -- python -m generate_user_files -dir:$WORKSPACE -family:wpbeta -lang:en -v > echo "console_encoding='utf8'" >> $WORKSPACE/user-config.py > tox -e venv -- pip install -rdev-requirements.txt > tox -e venv -- python setup.py nosetests --tests tests --verbosity=2 -a '"family=wpbeta,code=en"' > > > We can look at moving that as a script directly into `pywikibot/core.git`, add a new testenv in tox (eg: `integration-beta`) and then simply have a Jenkins job that invokes that testenv. Then move forward and fix it :) I see, the point was to test the every new MediaWiki version coming up. We run some beta-cluster tests on Travis and Appveyor currently, so it should be easy to mimick it. It should be ok to add a script or a plaintext list of commands into the repo. The main issue (as you can see by the priority) is this one, the formerly known Magul's quick Travis test. Every Gerrit patchset is tested by unittest, nosetest, doctest, some other similar stuff, but all on one platform with one exact presets. The Travis/Appveyor tests are quite heavy and time-consuming (They are running in several environments to compare), so there is no need to run them on every patchset. But if we added Magul back then into reviewers, he/his tool run the unittests on Travis on demand. This is what we currently miss the most I think as Travis and Appveyor after merge are still working for us (although we have no access to the Wikimedia Appveyor account and perhaps neither Wikimedia has), so there is no need to hurry with moving completely to Wikimedia CI. Also we run beta-cluster tests there, so also not a priority I think. The easy solution for the top priority Magul's test replacement would be something like this: Someone enters a magic keyword in the Gerrit patchset comment -> Wikimedia CI registers it and creates a pull request to GitHub with the code (perhaps using GH API, also closes it immediately as it wil not be needed on GitHub) -> and GitHub-Travis-Appveyor handles the rest of the work. The harder solution would be to mimick Travis/Appveyor in Wikimedia CI as you said, then when someone enters a magic keyword in the Gerrit patchset comment or someone +2 the patchset -> Wikimedia CI registers it and runs the tox on predefined set of environments including several Python versions, three major OS platforms and several presets. TASK DETAIL https://phabricator.wikimedia.org/T186208 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hashar, Dvorapa Cc: hashar, Liuxinyu970226, Legoktm, Framawiki, Dalba, Dvorapa, Magul, Aklapper, Xqt, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Liudvikas, mys_721tx, jayvdb, dduvall, Masti, Alchimista, Rxy, greg
_______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
