Re: [Twisted-Python] GitHub Actions parallelism limit increase

[Adi Roiban]

On Tue, 6 Apr 2021 at 19:58, Kyle Altendorf  wrote:

> On 2021-04-01 09:36, Adi Roiban wrote:
>
>
Thanks Kyle for the GitHub communication.

I have checked Twisted billing and we have GitHub Teams enabled.
So yes, 60x concurrent jobs on GitHub own VMs.

GitHub billing now shows $124.00 / month to cover our Team plan but we
don't have to pay.

For reference Twisted has 41 members [0]

Upgrading to enterprise will be  $903.00 / month :)

--

We now use 13x jobs for a build - 10 - python + 3 lint/mypy/docs.
Python jobs now take a maximum of 8 minutes (on 2xCPU VMs with 4 parallel
jobs) [1] ... from the previous 30+ minutes

In the last month there were about 250 jobs so about 8 per day.

So I think that we are ok for now.

--

We have 12 extra jobs on Azure.

I think that as an experiment we can move 1 Windows and 1 macOS job from
Azure to GitHub Actions.

I would still keep Azure pipelines jobs to use those extra VMs that are
available.
Azure allows retrying a single job on failure , or all failed jobs. This is
a bit nicery for flaky tests.
In GitHub, you have to re-run the whole workflow.

-

We can get more jobs by hosting our own runners... but that is a different
story.

[snip]

Sounds good.  Sounds like exactly what I suggested.  :]  Except for
> maybe misunderstanding about what level of separation is needed for the
> interjob dependencies.
>
> Hopefully this clarifies a bit.  I think we actually intend the same
> thing.  Unless I've missed something else?
>

I guess that is best to discuss directly on the change made for a PR and
see how the final matrix will look.

Top priority for me is enabling trial parallel runs on Windows (PR waiting
for review) to use the extra CPU available on the VM.
Next is enabling some kind of PR triage helper.

[0] https://github.com/orgs/twisted/people
[1] https://github.com/twisted/twisted/actions/runs/735543893
[2]
https://dev.azure.com/twistedmatrix/twisted/_build/results?buildId=3717&view=results
-- 
Adi Roiban
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Weekly Bug Summary

[trac]

Bug summary
__
Summary for 2021-04-04 through 2021-04-11
  Opened Closed  Total Change
Enhancements:  3  1   1481 +2
Defects:   1  2   1013 -1
Tasks: 2  1 98 +1
Regressions:   0  0  0 +0
Total: 6  4   2594 +2

|== Type Changes   |== Priority Changes   |== Component Changes   
|Defect:   -1  |Normal:  +2   |Core:   +3 
|Enhancement:  +2  |Low: +0   |Mail:   -1 
|Task: +1 |Names:  -1 
  |Pb: +1 



Total Tickets
Open Tickets



New / Reopened Bugs
__
= Normal =
[#10170] Fix coverage for swinging code branches (opened by adiroiban)
taskcore   http://twistedmatrix.com/trac/ticket/10170

[#10171] Fix PYPY coverage run (opened by adiroiban) (CLOSED, fixed)
defect  core   http://twistedmatrix.com/trac/ticket/10171

[#10172] Bump required version of mypy-zope to 0.3.0 (opened by wiml)
enhancement core   http://twistedmatrix.com/trac/ticket/10172

[#10174] Drop twisted/internet/base.py _checkProcessArgs() (opened by rodrigc)
enhancement core   http://twistedmatrix.com/trac/ticket/10174

[#10175] Migrate from twisted.python.randbytes to secrets on py3.6+ ? (opened by wiml)
enhancement core   http://twistedmatrix.com/trac/ticket/10175

= Low =
[#10173] Remove py2.4 sets module support from jelly.py (opened by wiml)
taskpb http://twistedmatrix.com/trac/ticket/10173



Closed Bugs
__
= Normal =
[#3019 ] twisted.names's default getHostByName does not reliably fetch records because of its use of ANY (qtype=*) (opened by radix, closed by twm, duplicate)
defect  names  http://twistedmatrix.com/trac/ticket/3019

[#10121] Disable full pypy coverage run in a bid to stop the random failures and speed up CI (opened by adiroiban, closed by Adi Roiban , fixed)
enhancement core   http://twistedmatrix.com/trac/ticket/10121

[#10171] Fix PYPY coverage run (opened by adiroiban, closed by Adi Roiban , fixed)
defect  core   http://twistedmatrix.com/trac/ticket/10171

= Low =
[#10160] Eliminate twisted.mail.mail.DomainWithDefaultDict in favor of collections.defaultdict (opened by wiml, closed by wiml, invalid)
taskmail   http://twistedmatrix.com/trac/ticket/10160



Ticket Lifetime Stats
__
Oldest open ticket - [#50] conch command-line client doesn't work in win32 (since 2003-07-12 14:41:06).
Newest open ticket - [#10175] Migrate from twisted.python.randbytes to secrets on py3.6+ ? (since 2021-04-10 17:54:35.589739).

Mean open ticket age: 3027 days, 15:26:06.243127.
Median: 2988 days, 13:03:37.350611.
Standard deviation: 1526 days, 4:26:51.515574.
Interquartile range: 2227 days, 17:40:52.925976.

Mean time between ticket creation and ticket resolution: 615 days, 17:52:14.947192.
Median: 57 days, 0:41:16.
Standard deviation is 1025 days, 9:27:49.895859.
The interquartile range is 804 days, 5:03:48.205897.

Mean time spent in review: 101 days, 16:22:08.559415.
Median: 4 days, 20:14:03.337217.
Standard deviation: 485 days, 16:56:09.344669.
Interquartile range: 24 days, 2:14:45.082472.

Mean number of times a ticket is reviewed: 1.78190306803.
Median: 1
Standard deviation: 1.40546533024.
Interquartile range: 1.


Contributor Stats
__
In the last 4 weeks,
11 unique ticket reporters
5 unique ticket reviewers
9 unique ticket resolvers
In the last 24 weeks,
39 unique ticket reporters
9 unique ticket reviewers
15 unique ticket resolvers
In the last 48 weeks,
65 unique ticket reporters
13 unique ticket reviewers
23 unique ticket resolvers





___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python