Re: [Twisted-Python] Enable pre-commit.ci for twisted/twisted

2021-07-17 Thread Adi Roiban
On Thu, 15 Jul 2021 at 15:36, Jean-Paul Calderone 
wrote:

> On Thu, Jul 15, 2021 at 10:26 AM Adi Roiban  wrote:
>
>> Hi,
>>
>> As part of this PR https://github.com/twisted/twisted/pull/1619 Thomas
>> has requested to enable the https://pre-commit.ci  service for
>> twisted/twisted repository
>>
>> The pre-commit.ci is kind of an auto-pilot for our pre-commit
>> configuration.
>>
>> We already do post-commit checks via the pre-commit framework and I think
>> that some of the core developers have pre-commit enabled as a git hook.
>>
>> The convenience if pre-commit.ci is that if you don't have the git-hook
>> enable and you commit dirty code, pre-commit.ci will fix it for you.
>>
>> pre-commit.ci will also automatically create PR to update the pre-commit
>> dependencies.
>>
>> This all comes at the cost of allowing the external pre-commit.ci server
>> to write to twisted/twisted repository.
>>
>> pre-commit.ci was already enabled for twisted/ldaptor
>>
>> pre-commit.ci is currently enabled for twisted/twisted for testing while
>> a final decision is made.
>>
>> I am +0 on this change due to security reasons...but I do think that it
>> will reduce a bit of the frustration for first time contributors.
>>
>
> Can you elaborate on this a little bit?  First time contributors won't be
> able to push a branch to twisted/twisted.  Does giving pre-commit.ci
> write access to twisted/twisted also give it write access to forks of
> twisted/twisted?  Or maybe only to branches in forks with PRs targeting
> twisted/twisted?
>
>
External contributors can't push directly to twisted/twisted.
Their code will reach twisted/twisted vi PR... just like before

I expect that first time contributors will have their fork automatically
updated by pre-commit.ci to make the code more likely to be accepted.

For example, pre-commit will automatically run black.

I don't have any experience with pre-commit.ci  ... but this is my
expectation,



>
>> If there are no strong comments against enabling pre-commit.ci, in a
>> week, I plan to approve the PR and have the pre-commit.ci enabled for
>> the long term.
>>
>
> Thanks for raising this question with the mailing list and for supplying
> details about the motivation.  Given that there is no justification for the
> change written on the ticket or in the PR itself, this is really helpful.
>

Based on the current feedback, I will approve the PR.


-- 
Adi Roiban
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


Re: [Twisted-Python] Twisted 21.7.0 Pre-Release Announcement

2021-07-17 Thread Adi Roiban
On Fri, 16 Jul 2021 at 20:27, Glyph  wrote:

>
>
> On Jul 16, 2021, at 2:20 AM, Richard van der Hoff 
> wrote:
>
> On 16/07/2021 00:18, Glyph wrote:
>
>
>
> On Jul 15, 2021, at 9:00 AM, Richard van der Hoff 
> wrote:
>
> We can't just go and add type annotations because we need to maintain
> compatibility with older Twisted (to make it possible to package in Debian
> et al).
>
> Any suggestions for keeping mypy happy?
>
> Are you saying you need it to typecheck against older versions or just run
> against them?
>
> Ah, this gave me the clue I needed. We just need to run against them.
> Which means I can put type hints in comments, where they will be ignored at
> runtime. It's fiddly, but it will work well enough.
>
> You can also do 'from __future__ import annotations' to avoid the
> annotations getting evaluated, which might be slightly less awkward.
>
> Thanks Glyph, and thanks to Adi and Barry for your suggestions too.
>
> One particular problem I came across was the type annotation on
> inlineCallbacks. I've filed https://twistedmatrix.com/trac/ticket/10231
> about it - would appreciate thoughts.
>
> This definitely looks wrong; there should be a TypeVar in there.  Adi, I'd
> go so far as to say that this should be a release blocker, although the
> change should be fairly minimal.
>
> Richard, could you please make a proper PR for this to get CI kicked off
> and make sure the new annotation doesn't cause any failures?
>
>
I think that I will do a RC2 release candidate, since due to this issue the
GHA tests on RC1 are red - https://github.com/twisted/twisted/pull/1628
So I plan to cherry-pick those changes and do a RC2.

Regarding  the release blocker, I will block the release for any ticket
from https://twistedmatrix.com/trac/report/26

Right now, I have added https://twistedmatrix.com/trac/ticket/10231 to that
report.

So, if anyone knows how to fix https://twistedmatrix.com/trac/ticket/10231
please help to unblock the release.

Or if you think that https://twistedmatrix.com/trac/ticket/10231 should not
be a release blocker, please add your comments.

For now, I will delay the release of RC2 to wait to see what is the
resolution for #10231

Regards

-- 
Adi Roiban
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


Re: [Twisted-Python] Twisted 21.7.0 Pre-Release Announcement

2021-07-17 Thread Glyph


> On Jul 17, 2021, at 3:47 AM, Adi Roiban  wrote:
> 
> 
> 
> On Fri, 16 Jul 2021 at 20:27, Glyph  > wrote:
> 
> 
>> On Jul 16, 2021, at 2:20 AM, Richard van der Hoff > > wrote:
>> 
>> On 16/07/2021 00:18, Glyph wrote:
>> 
>>> 
>>> 
 On Jul 15, 2021, at 9:00 AM, Richard van der Hoff >>> > wrote:
 
 We can't just go and add type annotations because we need to maintain 
 compatibility with older Twisted (to make it possible to package in Debian 
 et al).
 
 Any suggestions for keeping mypy happy?
 
>>> 
>>> Are you saying you need it to typecheck against older versions or just run 
>>> against them?
>> Ah, this gave me the clue I needed. We just need to run against them. Which 
>> means I can put type hints in comments, where they will be ignored at 
>> runtime. It's fiddly, but it will work well enough.
>> 
> You can also do 'from __future__ import annotations' to avoid the annotations 
> getting evaluated, which might be slightly less awkward.
>> 
>> Thanks Glyph, and thanks to Adi and Barry for your suggestions too.
>> 
>> One particular problem I came across was the type annotation on 
>> inlineCallbacks. I've filed https://twistedmatrix.com/trac/ticket/10231 
>>  about it - would appreciate 
>> thoughts.
>> 
> 
> This definitely looks wrong; there should be a TypeVar in there.  Adi, I'd go 
> so far as to say that this should be a release blocker, although the change 
> should be fairly minimal.
> 
> Richard, could you please make a proper PR for this to get CI kicked off and 
> make sure the new annotation doesn't cause any failures?
> 
> 
> I think that I will do a RC2 release candidate, since due to this issue the 
> GHA tests on RC1 are red - https://github.com/twisted/twisted/pull/1628 
> 
> So I plan to cherry-pick those changes and do a RC2.
> 
> Regarding  the release blocker, I will block the release for any ticket from 
> https://twistedmatrix.com/trac/report/26 
> 
> 
> Right now, I have added https://twistedmatrix.com/trac/ticket/10231 
>  to that report.
> 
> So, if anyone knows how to fix https://twistedmatrix.com/trac/ticket/10231 
>  please help to unblock the 
> release.
> 
> Or if you think that https://twistedmatrix.com/trac/ticket/10231 
>  should not be a release 
> blocker, please add your comments.
> 
> For now, I will delay the release of RC2 to wait to see what is the 
> resolution for #10231

This depends largely upon your available time and the difficulty of making 
these changes, but I'd suggest doing an RC2 as soon as the fixes are ready for 
the new python versions, to ensure folks can test with those, and then do an 
RC3 as soon as the annotation fix is ready.

-g

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


Re: [Twisted-Python] twisted.web HTTPS client certificate

2021-07-17 Thread Glyph


> On Jul 14, 2021, at 7:05 AM, Barry Scott  wrote:
> 
> On Tuesday, 13 July 2021 22:48:18 BST Glyph wrote:
>> 
>>> On Jul 13, 2021, at 2:09 AM, Barry Scott  wrote:
>>> 
>>> On Monday, 12 July 2021 09:27:19 BST Glyph wrote:
 FWIW I would avoid calling the pyOpenSSL APIs for this, since hopefully 
 we'll manage to move away from OpenSSL and at least somewhat abstract away 
 the transition. 
>>> 
>>> Are you thinking to use the Windows and macOS provided crypto API and only 
>>> use openssl on Unix systems?
>>> 
>>> Barry
>> 
>> It would be nice to have a system for backends so that we could do this when 
>> certain specific situations call for it (usually related to TLS clients, 
>> rather than servers, although having both would be great), but no, the main 
>> motivation is to drop OpenSSL entirely in favor of Rustls, as recommended by 
>> the ISRG: 
>> > >.
> 
> That is a great goal for Twisted.

I'm glad you think so! I think it's a great goal for everybody, really ;-).

-g
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


Re: [Twisted-Python] Enable auto-merge for twisted/twisted

2021-07-17 Thread Glyph
Nope, I'm fully in favor. When we're done reviewing, and waiting for CI, 
reviewers should be able to set auto-merge and walk away.  Thank you for 
raising the issue!

> On Jul 14, 2021, at 5:33 AM, Adi Roiban  wrote:
> 
> Hi,
> 
> Do you see any issues if we enable the auto-merge feature for twisted/twisted?
> 
> The feature is documented here
> 
> https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/automatically-merging-a-pull-request
>  
> 
> 
> With this, in theory you don't have to wait for all the CI to be green before 
> a merge.
> You got your reviews, you fix some typos that don't require a re-review and 
> then set the merge message and your are done.
> Or there was a flaky test failure and just re-run the job and the merge will 
> be done once all the jobs are green again.
> 
> -
> 
> This can be abused if someone will enable auto-merge before the PR is 
> approved by a reviewer.
> As most of the time the reviewer will accept the PR but will request or 
> suggest minor changes.
> If the auto-merge is enabled, the PR is mergesd as soon as it is reviewed.
> 
> -
> The config looks like this:
> 
> You can allow setting pull requests to merge automatically once all required 
> reviews and status checks have passed.
> 
> Allow auto-merge
> Waits for merge requirements to be met and then merges automatically.
> 
> 
> 
> 
> There is also the option to block a merge only after all the conversations 
> are marked as resolved.
> I am using this for my project and it works ok... a bit annoying but can help 
> if you accedentally forgot to address a comment.
> 
> Cheers
> 
> 
> -- 
> Adi Roiban
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


Re: [Twisted-Python] Twisted-Infra org - twm added as maintainer

2021-07-17 Thread Glyph
Hi Adi,

This, and adding Tom to dornkirk/SSH, both sound good to me. I wish we had a 
more formal process for these various permissions but I think whatever the 
standard we would set, Tom has exceeded the bar by now.

Thanks.

> On Jul 14, 2021, at 5:27 AM, Adi Roiban  wrote:
> 
> Hi,
> 
> I tried to send this email 9 days ago... but I was not subscribed to the list 
> using this email... and have not seen the bounce message... if any.
> 
> --
> 
> I just wanted to let you know that I have added Tom Most as a "core" team 
> member to the twisted-infra organization.
> 
> This is more of an information.
> Let me know if I did something wrong
> I don't know if there is any process in place for adding new members to 
> twisted-infra org.
> 
> Tom can merge the code but without access to the actual production server 
> there is not much he can do.
> 
> I think that we can give him access to the servers to help with the 
> infrastructure.
> 
> I have not added the keys yet, but if any other member of the team think that 
> this is a good idea please check with Tom what are the preferred key/keys to 
> be added to ~/.ssh/authorized_keys
> 
> Tom's SSH Keys https://api.github.com/users/twm/keys 
> 
> 
> I can also add the keys once I get some feedback.
> 
> Cheers
> -- 
> Adi Roiban
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


[Twisted-Python] Weekly Bug Summary

2021-07-17 Thread trac



Bug summary
__
Summary for 2021-07-11 through 2021-07-18
  Opened Closed  Total Change
Enhancements:  8  2   1502 +6
Defects:   4  1   1022 +3
Tasks: 0  0 98 +0
Regressions:   2  0  2 +2
Total:15  3   2627+12

|== Type Changes|== Priority Changes   |== Component Changes 
|Defect:+3  |Highest:  +1  |Core:   +11  
|Enhancement:   +6  |Normal:  +10  |Release Management:  +1  
|Release Blocker: Regression:   +2  |Lowest:   +1
|Release Blocker: Release Process Bug:  +1   



Total Tickets
Open Tickets



New / Reopened Bugs
__
= Highest =
[#10222] awaiting a failed deferred twice fails with AttributeError (opened by graingert)
defect  core   http://twistedmatrix.com/trac/ticket/10222

= Normal =
[#10219] remove reraise related py2 compatability hacks (opened by graingert)
enhancement core   http://twistedmatrix.com/trac/ticket/10219

[#10220] fix twisted.trial.reporter frame related hacks (opened by graingert)
enhancement core   http://twistedmatrix.com/trac/ticket/10220

[#10221] add twisted.trial.unittest.mark for marking test functions in a type safe way (opened by graingert)
enhancement core   http://twistedmatrix.com/trac/ticket/10221

[#10223] creating a deferred with d.fail() always collects _debugInfo regardless of defer.setDebugging mode (opened by graingert)
defect  core   http://twistedmatrix.com/trac/ticket/10223

[#10224] python 3.10b4 (opened by graingert)
release blocker: release process bug core   http://twistedmatrix.com/trac/ticket/10224

[#10225] "wrong number of arguments" error from twisted.python.usage.options is worse than useless (opened by glyph)
defect  core   http://twistedmatrix.com/trac/ticket/10225

[#10226] if you do `from unittest import skip` in a module trial skips the whole module (opened by graingert)
enhancement core   http://twistedmatrix.com/trac/ticket/10226

[#10228] On a new final release automatically update the Read The Docs default version (opened by adiroiban)
enhancement release management http://twistedmatrix.com/trac/ticket/10228

[#10229] type annotate twisted.internet._idna to fix CI (opened by graingert)
enhancement core   http://twistedmatrix.com/trac/ticket/10229

[#10230] Fix tests after py3.8 and 3.9 updates (opened by adiroiban)
release blocker: regression core   http://twistedmatrix.com/trac/ticket/10230

[#10231] incorrect type annotation on inlineCallbacks (opened by richvdh)
release blocker: regression core   http://twistedmatrix.com/trac/ticket/10231

[#10232] TLS lowerMaximumSecurityTo off-by-one error (opened by Jon-Work)
defect  core   http://twistedmatrix.com/trac/ticket/10232

[#10233] Need API to configure TLS ECDHE curves (opened by Jon-Work)
enhancement core   http://twistedmatrix.com/trac/ticket/10233

= Lowest =
[#10227] TCP SIP Proxy (opened by adarsh-chauhan)
enhancement core   http://twistedmatrix.com/trac/ticket/10227



Closed Bugs
__
= Normal =
[#10215] run tests on 3.10.0b3 (opened by graingert, closed by graingert, fixed)
enhancement core   http://twistedmatrix.com/trac/ticket/10215

[#10212] upgrade to mypy==0.910 and mypy-zope==0.3.2 (opened by graingert, closed by graingert, fixed)
enhancement core   http://twistedmatrix.com/trac/ticket/10212

[#10211] haproxy transport wrapper returns a transport.getPeer().host of type bytes, when the type annotation is str (opened by graingert, closed by graingert, fixed)
defect  core   http://twistedmatrix.com/trac/ticket/10211



Ticket Lifetime Stats
__
Oldest open ticket - [#50] conch command-line client doesn't work in win32 (since 2003-07-12 14:41:06).
Newest open ticket - [#10233] Need API to configure TLS ECDHE curves (since 2021-07-16 10:17:40.666397).

Mean open ticket age: 3085 days, 8:12:08.958412.
Median: 3058 days, 12:50:12.124326.
Standard deviation: 1557 days, 3:22:30.406527.
Interquartile range: 2273 days, 19:02:41.004795.

Mean time between ticket creation and ticket resolution: 614 days, 2:45:21.360459.
Median: 56 days, 2:26:53.151419.
Standard deviation is 1024 days, 9:29:34.309415.
The interquartile range is 801 days, 15:10:21.961301.

Mean time spent in review: 103 days, 4:35:02.996176.
Median: 4 days, 21:15:16.
Standard deviation: 485 days, 5:35:56.336877.
Interquartil