Re: [Twisted-Python] Twisted 16.3.0 Prerelease 2 Announcement

[Cory Benfield]

> On 11 Jul 2016, at 01:45, Glyph Lefkowitz  wrote:
> 
> 
>> On Jul 9, 2016, at 10:30 AM, Paweł Miech > > wrote:
>> 
>> My question is: should user deal with this kind of stuff themselves? If some 
>> ciphers are blacklisted in HTTP2 shouldn't this be handled somewhere in 
>> Twisted?
> 
> As others have already said, this should work out of the box, and I'm not 
> sure why it isn't for you, especially that you've gone to the extra trouble 
> of building a Docker image and retrieving recent enough versions of every 
> relevant layer of the stack.
> 
> However, to answer this question generally: this should absolutely be handled 
> by Twisted.  In fact, even if we're doing the right thing already except in 
> your one configuration, we should go a step beyond and provide tooling and 
> logging to clearly explain to system operators why they won't get HTTP/2 if 
> their dependencies are out of date.

This turns out to be trickier than you’d expect.

PyOpenSSL does not expose any of the APIs for us to programmatically detect 
what ciphers are available to the OpenSSL we have installed. Cryptography 
exposes only one: SSL_get_ciphers. This is not really the one we want, because 
it lists all *possible* ciphers, rather than the ones that are actually enabled 
for a given connection. This makes it very difficult for us to conclude that 
we’d want to use HTTP/2 but we cannot because of a lack of cipher support.

Now, Twisted *could* add code to introspect the HTTP/2 TLS configuration and 
optionally terminate the connection in the same manner that Chrome does. 
Currently I’ve not done that because it’s not been hugely needed, but we could 
do that. The reality is, though, that Twisted can’t unconditionally not use 
those ciphers because it needs to support HTTP/1.1 as well as HTTP/2, and 
HTTP/1.1 does not have those same restrictions.

What would be looking for here? Out of the box, Twisted should do the very best 
it can, but right now it seems like the only thing we could do is detect when 
HTTP/2 is literally impossible to support (e.g. when there is no TLS 1.2 
support). With that said, those versions *completely* overlap with the versions 
where OpenSSL doesn’t support ALPN.

Regardless, Twisted’s default cipher ordering is appropriate for HTTP/2 (it 
prefers ECDHE AES GCM, which is what is required). So I’m not sure what more we 
could do.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 16.3.0 Prerelease 2 Announcement

[Glyph Lefkowitz]

> On Jul 11, 2016, at 3:35 AM, Cory Benfield  > wrote:
> 
>> 
>> On 11 Jul 2016, at 01:45, Glyph Lefkowitz > > wrote:
>> 
>> 
>>> On Jul 9, 2016, at 10:30 AM, Paweł Miech >> > wrote:
>>> 
>>> My question is: should user deal with this kind of stuff themselves? If 
>>> some ciphers are blacklisted in HTTP2 shouldn't this be handled somewhere 
>>> in Twisted?
>> 
>> As others have already said, this should work out of the box, and I'm not 
>> sure why it isn't for you, especially that you've gone to the extra trouble 
>> of building a Docker image and retrieving recent enough versions of every 
>> relevant layer of the stack.
>> 
>> However, to answer this question generally: this should absolutely be 
>> handled by Twisted.  In fact, even if we're doing the right thing already 
>> except in your one configuration, we should go a step beyond and provide 
>> tooling and logging to clearly explain to system operators why they won't 
>> get HTTP/2 if their dependencies are out of date.
> 
> This turns out to be trickier than you’d expect.
> 
> PyOpenSSL does not expose any of the APIs for us to programmatically detect 
> what ciphers are available to the OpenSSL we have installed. Cryptography 
> exposes only one: SSL_get_ciphers. This is not really the one we want, 
> because it lists all *possible* ciphers, rather than the ones that are 
> actually enabled for a given connection. This makes it very difficult for us 
> to conclude that we’d want to use HTTP/2 but we cannot because of a lack of 
> cipher support.

So pyOpenSSL/Cryptography doesn't have SSL_get_current_cipher anywhere?

> Now, Twisted *could* add code to introspect the HTTP/2 TLS configuration and 
> optionally terminate the connection in the same manner that Chrome does. 
> Currently I’ve not done that because it’s not been hugely needed, but we 
> could do that. The reality is, though, that Twisted can’t unconditionally not 
> use those ciphers because it needs to support HTTP/1.1 as well as HTTP/2, and 
> HTTP/1.1 does not have those same restrictions.

The main interest I think we have is to placate Chrome, to ensure it can speak 
HTTP/2 if it's possible, and to explain why it's not possible, if it's not.

> What would be looking for here? Out of the box, Twisted should do the very 
> best it can, but right now it seems like the only thing we could do is detect 
> when HTTP/2 is literally impossible to support (e.g. when there is no TLS 1.2 
> support). With that said, those versions *completely* overlap with the 
> versions where OpenSSL doesn’t support ALPN.

In the same way that we complain about service_identity perhaps we should 
complain about OpenSSL?

> Regardless, Twisted’s default cipher ordering is appropriate for HTTP/2 (it 
> prefers ECDHE AES GCM, which is what is required). So I’m not sure what more 
> we could do.

Yeah, I'm curious why the OP was having this problem.

-glyph



smime.p7s
Description: S/MIME cryptographic signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 16.3.0 Prerelease 2 Announcement

[Paweł Miech]

Thanks for input everyone!

@Cory

> right now it seems like the only thing we could do is detect when HTTP/2
is literally impossible to support (e.g. when there is no TLS 1.2 support)

This seems to suggest that Ubuntu 16.04 (the system I'm testing) does not
support ciphers required by HTTP2. But nginx article about HTTP2 lists
ubuntu as only linux like system that is able to support HTTP2 over ALPN
which is required by Chrome:
https://www.nginx.com/blog/supporting-http2-google-chrome-users/

I decided to verify tnginx statements and I tried to set up nginx with
HTTP2 on ubuntu 16.04. It turns out this is possible and it works ok. I
just followed this article here:
https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-with-http-2-support-on-ubuntu-16-04
This means that in principle Ubuntu 16.04 should be able to support HTTP2
and it has required TLS ciphers.

So the problem here is not about lack of OS support.

Looking into this nginx article they recommend two things that are part of
manual setup which (maybe?) are required?

1) They say ciphers should be set to ssl_ciphers
EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;

This long string does not mean much to me, but reading email from Amber
again I see it differs slightly from what she says Twisted uses. But one
thing I'm wondering about is how do you guys know which ciphers are set in
Twisted? Looking into source code of DefaultOpenSSLContextFactory I see
context is created here:
https://github.com/twisted/twisted/blob/3455a902fb15e732ee43b59f4d82a66b105351ba/twisted/internet/ssl.py#L107
I dont see any point where there is a call that sets ciphers. Maybe this is
done somewhere else? I tried grepping source for string mentioned by Amber
but cant find it.

2) they ask user to generate DHE key and provide this to nginx
configuration.

When I compare my nginx with Twisted using openssl I see that ciphers in
response differ.

For example this is what my nginx cipher is:

> openssl s_client -connect localhost:443
...
...
...
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256

And here is cipher for Twisted

> openssl s_client -connect localhost:8080

New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384

When I check HTTP2 cipher black list I see AES256-GCM-SHA384 is there on
this list see here https://http2.github.io/http2-spec/#BadCipherSuites





2016-07-11 21:22 GMT+02:00 Glyph Lefkowitz :

>
> On Jul 11, 2016, at 3:35 AM, Cory Benfield  wrote:
>
>
> On 11 Jul 2016, at 01:45, Glyph Lefkowitz  wrote:
>
>
> On Jul 9, 2016, at 10:30 AM, Paweł Miech  wrote:
>
> My question is: should user deal with this kind of stuff themselves? If
> some ciphers are blacklisted in HTTP2 shouldn't this be handled somewhere
> in Twisted?
>
>
> As others have already said, this *should* work out of the box, and I'm
> not sure why it isn't for you, especially that you've gone to the extra
> trouble of building a Docker image and retrieving recent enough versions of
> every relevant layer of the stack.
>
> However, to answer this question generally: this should *absolutely* be
> handled by Twisted.  In fact, even if we're doing the right thing already
> except in your one configuration, we should go a step beyond and provide
> tooling and logging to clearly explain to system operators why they won't
> get HTTP/2 if their dependencies are out of date.
>
>
> This turns out to be trickier than you’d expect.
>
> PyOpenSSL does not expose any of the APIs for us to programmatically
> detect what ciphers are available to the OpenSSL we have installed.
> Cryptography exposes only one: SSL_get_ciphers. This is not really the one
> we want, because it lists all *possible* ciphers, rather than the ones that
> are actually enabled for a given connection. This makes it very difficult
> for us to conclude that we’d want to use HTTP/2 but we cannot because of a
> lack of cipher support.
>
>
> So pyOpenSSL/Cryptography doesn't have SSL_get_current_cipher anywhere?
>
> Now, Twisted *could* add code to introspect the HTTP/2 TLS configuration
> and optionally terminate the connection in the same manner that Chrome
> does. Currently I’ve not done that because it’s not been hugely needed, but
> we could do that. The reality is, though, that Twisted can’t
> unconditionally not use those ciphers because it needs to support HTTP/1.1
> as well as HTTP/2, and HTTP/1.1 does not have those same restrictions.
>
>
> The main interest I think we have is to placate Chrome, to ensure it can
> speak HTTP/2 if it's possible, and to explain why it's not possible, if
> it's not.
>
> What would be looking for here? Out of the box, Twisted should do the very
> best it can, but right now it seems like the only thing we could do is
> detect when HTTP/2 is literally impossible to support (e.g. when there is
> no TLS 1.2 support). With that said, those versions *completely* overlap
> with the versions where OpenSSL doesn’t support ALPN.
>
>
> In the same way that we

[Twisted-Python] PyPy buildbot, less failed tests

[Craig Rodrigues]

I would like to acknowledge the contributions of Ville Skyttä who submitted
several patches to close open file descriptors in various Twisted tests.

I looked at these two PyPy builds:

https://buildbot.twistedmatrix.com/builders/ubuntu16.04-pypy5/builds/207
52 Failed tests (20 failures, 32 errors)

https://buildbot.twistedmatrix.com/builders/ubuntu16.04-pypy5/builds/208
32 Failed tests (15 failures, 17 errors)

This reduction is directly due to Ville's contributions.
Thank you Ville!!

--
Craig
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Trial running tests on Windows Python 3

[Craig Rodrigues]

I have committed some fixes to trunk, and now trial is running tests on
Windows Python 3.

This buildbot is one example build with Python 3.5:
https://buildbot.twistedmatrix.com/builders/win2012r2-64-py3.5/builds/12

There are some test failures, but a lot of things are passing.  One nice
thing to see in this build is
that there are DeprecationWarnings where str is used when bytes should be
used,
and also ResourceWarnings for tests which are not cleanly shutting down
sockets.

--
Craig
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 16.3.0 Prerelease 2 Announcement

[Craig Rodrigues]

On Mon, Jul 11, 2016 at 2:04 PM, Paweł Miech  wrote:

> 1) They say ciphers should be set to ssl_ciphers
> EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
>
> This long string does not mean much to me, but reading email from Amber
> again I see it differs slightly from what she says Twisted uses. But one
> thing I'm wondering about is how do you guys know which ciphers are set in
> Twisted? Looking into source code of DefaultOpenSSLContextFactory I see
> context is created here:
> https://github.com/twisted/twisted/blob/3455a902fb15e732ee43b59f4d82a66b105351ba/twisted/internet/ssl.py#L107
> I dont see any point where there is a call that sets ciphers. Maybe this is
> done somewhere else? I tried grepping source for string mentioned by Amber
> but cant find it.
>

In an earlier e-mail you mentioned that you were using Python 3.  Is that
still true?

In the Windows Python 3 build which was recently enabled, I saw these
warnings:

c:\buildslave\win2012r2-64-py3_5\Twisted\twisted\internet\_sslverify.py:1799:
DeprecationWarning: str for cipher_list is no longer accepted, use
bytes
c:\buildslave\win2012r2-64-py3_5\Twisted\twisted\internet\_sslverify.py:1656:
DeprecationWarning: str for buf is no longer accepted, use bytes
c:\buildslave\win2012r2-64-py3_5\Twisted\twisted\internet\_sslverify.py:1660:
DeprecationWarning: str for cipher_list is no longer accepted, use
bytes

I am not sure if this is related to your problem, but it struck me that you
mentioned a problem
with ciphers, and I saw this warning just now.

--
Craig
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 16.3.0 Prerelease 2 Announcement

[Glyph Lefkowitz]

> On Jul 11, 2016, at 4:42 PM, Craig Rodrigues  wrote:
> 
> In an earlier e-mail you mentioned that you were using Python 3.  Is that 
> still true?

Seconded - it would be very interesting to know if switching to python 2 fixes 
your issue. :)___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python