On 2021-03-31 05:45, Adi Roiban wrote:
On Tue, 30 Mar 2021, 22:23 Kyle Altendorf, <s...@fstab.net> wrote:
On 2021-03-30 15:24, Glyph wrote:
On Mar 30, 2021, at 7:57 AM, Kyle Altendorf <s...@fstab.net> wrote:
Hi All,
Has anyone contacted GitHub to see if they would be willing to
increase the parallelism limit in Actions? My understanding is that
we maintain two CI systems (GitHub Actions and Azure Pipelines) for
the sake of more parallelism. While perhaps worthwhile, this
doesn't
seem fun. Maybe GitHub would be willing to help out.
Not as far as I know. Do you want to give it a shot?
That was the plan. Submitted. I'll let you all know.
Thanks.
On a related note I am working to reduce the general duration of
Twisted CI jobs.
This should result in keeping the jobs slot busy for less time and
allowing more total jobs per day.
The first step was to enable parallel trial tests. Use both CPUs
available to the CI VMs.
This was already done for Linux and MacOS.
Distributed trial is not yet supported on Windows, but I am working on
it.
I already have a working distributed trial for Windows on my system.
It needs various fixed and I have submitted smaller PR for review.
Just getting distrial to work on Windows is not all...as we need to
update and clean the Twsited test to reduce the side effects.
Linux and macOS are more relaxed in terms of file access and for
example you can delete a file, even if its opened by another process or
by the same process.
Windows is not happy about that.
And then there are the 2 pypy jobs taking 30 minutes each - working to
fix it here https://github.com/twisted/twisted/pull/1543/
And then there are also general CI jobs improvements.
For example we can save about 40 seconds if we decide to stop sending
coverage to coveralls and only use codecov.io
Another example is save another 1 minute by stop using `tox --notests`
and replace it with something like
https://github.com/twisted/python-info-action to show the dependencies.
With `tox --notest` we now create the wheels and install the
dependencies twice for each job.
Would `tox --skip-pkg-install` cut it for the second invocation?
https://github.com/ericaltendorf/plotman/pull/66/files#diff-b803fcb7f17ed9235f1e5cb1fcd2f5d3b2838429d4368ae4c57ce4436577f03fR145
There is an extra 1 minute required for each job to generate the
coverage XML file. We can download all the coverage artifacts and
generate the XML only once.
I did coverage collection from all jobs for pymodbus. Looks like I
still had the XML report generating in each job, but that could
presumably be shifted. Anyways, I'm sure several of us could do this
off the top of our heads but in case it is of any interest here's what I
did. Or maybe you just point out something silly I did there and should
learn from. :]
https://github.com/riptideio/pymodbus/pull/592/files
Cheers,
-kyle
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
