Re: [Twisted-Python] [Twisted-web] upcoming changes to twistedmatrix.com mail infrastructure

[Steve Waterbury]

On 03/16/2016 03:53 PM, Glyph wrote:



On Mar 16, 2016, at 12:06 PM, Phil Mayers mailto:p.may...@imperial.ac.uk>> wrote:



Good luck with the migration.


Thanks!  And thanks for your questions, I was worried I put a ton of
work into that email only for it to land in the void :).


FWIW, Glyph, I read *all* your emails from beginning to end -- both
because they are always educational (for me) and because it
helps me in my never-ending quest to avoid doing real work.  :D
(So, another part of the void heard from ... ;)

But seriously:  thanks for all your work on this and Twisted in general,
and to you and all the Twisted minions for the continuing and
sustained high quality of Twisted!

Cheers,
Steve

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Client Application service.Service

[Tim Hughes]

Woops : accidentally sent too early

Hi all,

I am looking into twisted for writing a client application (bot style) and
most of the examples I find are for server style applications.

Looking at the finger example and
https://github.com/jdavisp3/twisted-intro/blob/master/twisted-server-3/fastpoetry.py
I am just wondering where is best to put my application logic.

The protocol is basically a ascii line style application

It feels to me that my Service should be the main business logic  and the
protocol should just be a simple send/receive something like this gist

https://gist.github.com/timhughes/f85da0c2d88ecb4ab5e3#file-twisted_application_client-py-L36

I was then planning on writing web interface that interacted with the
service to send messages and display the response.

My main issue here is how to send the message from the service via the
protocol. I cannot find any good examples of this and am beginning to think
I am conceptualising incorrectly or over thinking it. If anyone can point
me to some examples or point me in the right direction it would be awesome.
Hopefully my example makes sense.

Cheers

Tim

Tim Hughes
mailto:thug...@thegoldfish.org

Hi all,

I am looking into twisted for writing a client application (bot style) and
most of the examples I find are for server style applications.



Looking at the finger example and
https://github.com/jdavisp3/twisted-intro/blob/master/twisted-server-3/fastpoetry.py
I am just wondering where is best to put my application logic.


The protocol is basically a ascii line style application


It feels to me that my Service should be the core and the protocol should
just be something like

def send_mesage(message):










Tim Hughes
mailto:thug...@thegoldfish.org
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [Twisted-web] upcoming changes to twistedmatrix.com mail infrastructure

[Glyph]

> On Mar 16, 2016, at 12:06 PM, Phil Mayers  wrote:
> 
> On 16/03/16 18:52, Glyph wrote:
>> Over the last few months, twistedmatrix.com 's
>> mailman installation has been used increasingly frequently to execute
>> denial-of-service attacks against people's mailboxes.  This is
> 
> My sympathies; this exact problem was the reason we CAPTCHA-ised our install 
> of mailman and have to keep a very close eye on it.

Yeah.  If this were the only problem we'd probably be going that route, but 
given issues with the rest of our mail infrastructure, getting rid of it is a 
lot more satisfying :).  When I do self-service subscription I do very 
definitely plan to integrate a CAPTCHA.

> It's really a shame there's so little open-source competition in the email 
> sector these days; it all appears to have been hoovered up by Gmail, Office 
> 365 and various spam (sorry - bulk email) providers.
> 
>> There will be a couple of inconveniences immediately after the transition:
> 
> Couple of random thoughts:
> 
> Does mailgun actually contain a mailman-alike product or are you effectively 
> building one on top of it?

Mailgun does have mailing lists: 
https://documentation.mailgun.com/api-mailinglists.html

This is not really a "mailman-alike"; its feature-set is extremely minimal 
(and, as mailgun will readily tell you, using it for members-only mailing lists 
is a bit of a weird case for their product; their primary target is 
transactional application emails, like notifications of activity in a web app, 
invoices, alerts, that sort of thing).  There are some things we will miss 
(particularly archives; I'm hoping we can just pipe the messages into pipermail 
somehow); but huge amounts of Mailman's customizability are just useless fluff. 
 Some are actively bad, like mailing you all your passwords in plain text every 
month.  We don't use most of its features, and we have to explicitly disable a 
lot of them.  Many of these things are better in more recent releases, but for 
us, upgrading to a more recent release is quite a bit more work than abandoning 
it entirely.

However, despite peer-to-peer lists being a little outside Mailgun's core 
demographic, they're totally supported, and I've had a pretty good experience 
(better than mailman administration, certainly) administering a medium-sized 
mailing list using their web UI.  I do plan to build a few small tools, like a 
self-service subscription tool, using the API, but even that will be good; 
it'll make a nice little demo Klein app.

> Will the mailman-style List-X headers remain?

Yes, although for unfortunate technical reasons the values of those headers may 
change (the way lists vs. personal addresses are name-spaced on 
twistedmatrix.com is unfortunate for reasons having nothing to do with mailgun, 
but it will probably matter now whereas it didn't before).

> Will the behaviour of the list w.r.t. things like routing of To:/Cc:'ed 
> people change.

For members-post mailing lists, mailgun unconditionally sets the reply-to 
header, which is exactly the way we have mailman configured right now, so: no.

> Good luck with the migration.

Thanks!  And thanks for your questions, I was worried I put a ton of work into 
that email only for it to land in the void :).

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] pyOpenSSL 16.0.0 released

[Hynek Schlawack]

On behalf of PyCA – the Python Cryptography Authority – I’m anxious to announce 
that after almost a year since the 0.15.1 release of pyOpenSSL we’ve released 
the brand new 16.0.0.

A few organizational notes:

1. The pyopenssl-users mailing list and #pyopenssl IRC channel are deprecated.  
Please use cryptography-dev 
 and #cryptography-dev on 
Freenode where you’re much more likely to get help.
2. The version scheme switched to CalVer because a 0.x version for a 15 years 
old project is rather odd and calling it 1.0 although we don’t expect a 2.0 to 
ever happen didn’t make any sense.  pyOpenSSL is a long-running project with 
strict backward-compatibility requirements and is hence better served with a 
calendar-based version scheme.
3. Please note that some of us will be doing a TLS/HTTPS workshop at PyCon US 
2016 so if you always wanted to learn about these things first hand, make sure 
to sign up: .  We've 
opted to receive no compensation and asked the organizers to send them to 
PyLadies instead.  So you’ll be doing good while learning something!

***

Release details:

While the list of changes looks short, a lot internal work happened:

72 files changed, 15511 insertions(+), 15063 deletions(-)

We’ve done our best to not break any existing applications; including by making 
the urllib3 and Twisted test suites part of our CI.

The full changelog can be found at 
.

This is the first release under full stewardship of PyCA. We have made many 
changes to make local development more pleasing. The test suite now passes both 
on Linux and OS X with OpenSSL 0.9.8, 1.0.1, and 1.0.2. It has been moved to 
py.test, all CI test runs are part of tox and the source code has been made 
fully flake8 compliant.

We hope to have lowered the barrier for contributions significantly but are 
open to hear about any remaining frustrations.


Backward-incompatible changes:

• Python 3.2 support has been dropped. It never had significant real world 
usage and has been dropped by our main dependency cryptography. Affected users 
should upgrade to Python 3.3 or later.


Deprecations:

• The support for EGD has been removed. The only affected function 
OpenSSL.rand.egd() now uses os.urandom() to seed the internal PRNG instead. 
Please see pyca/cryptography#1636 for more background information on this 
decision. In accordance with our backward compatibility policy 
OpenSSL.rand.egd() will be removed no sooner than a year from the release of 
16.0.0.
Please note that you should use urandom for all your secure random number needs.
• Python 2.6 support has been deprecated. Our main dependency 
cryptography deprecated 2.6 in version 0.9 (2015-05-14) with no time table for 
actually dropping it. pyOpenSSL will drop Python 2.6 support once cryptography 
does.


Changes:

• Fixed OpenSSL.SSL.Context.set_session_id, OpenSSL.SSL.Connection.renegotiate, 
OpenSSL.SSL.Connection.renegotiate_pending, and 
OpenSSL.SSL.Context.load_client_ca. They were lacking an implementation since 
0.14. #422
• Fixed segmentation fault when using keys larger than 4096-bit to sign data. 
#428
• Fixed AttributeError when OpenSSL.SSL.Connection.get_app_data() was called 
before setting any app data. #304
• Added OpenSSL.crypto.dump_publickey() to dump OpenSSL.crypto.PKey objects 
that represent public keys, and OpenSSL.crypto.load_publickey() to load such 
objects from serialized representations. #382
• Added OpenSSL.crypto.dump_crl() to dump a certificate revocation list out to 
a string buffer. #368
• Added OpenSSL.SSL.Connection.get_state_string() using the OpenSSL binding 
state_string_long. #358
• Added support for the socket.MSG_PEEK flag to OpenSSL.SSL.Connection.recv() 
and OpenSSL.SSL.Connection.recv_into(). #294
• Added OpenSSL.SSL.Connection.get_protocol_version() and 
OpenSSL.SSL.Connection.get_protocol_version_name(). #244
• Switched to utf8string mask by default. OpenSSL formerly defaulted to a 
T61String if there were UTF-8 characters present. This was changed to default 
to UTF8String in the config around 2005, but the actual code didn't change it 
until late last year. This will default us to the setting that actually works. 
To revert this you can call 
OpenSSL.crypto._lib.ASN1_STRING_set_default_mask_asc(b"default"). #234

***

For PyCA,
Hynek Schlawack


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Cannot perform IPv6 Link-Local Multicast on Twisted

[Glyph]

> On Mar 16, 2016, at 6:41 AM, Shantanoo Desai  
> wrote:
> 
> I am trying to use the .joinGroup() by passing the IPv6 link-local multicast 
> add. ff02::1 but get DNS lookup failure error.
> 
> I am trying to use the UDP multicast example on the twistedmatrix.com 
>  website and simply replacing the IPv4 multicast 
> address with the IPv6 one.
> 
> I have also posted a query on StackOverflow: 
> http://stackoverflow.com/questions/36034258/ipv6-link-local-multicast-on-twisted
>  
> 
> 
> I am newbie on Twisted and some suggestions would be appreciated. 

I haven't had time to fully investigate, but unfortunately, this is probably a 
hard-coded limitation of Twisted, currently.  The right thing for you to do 
would be to contribute a patch to Twisted that fixes this; please file a ticket 
for that if there isn't one already.

But, in the interests both of telling you how you might write that patch, and 
also how to work around it right now, let me explain: 'listenMulticast' 
constructs a MulticastPort which has an addressFamily of AF_INET (i.e.: IPv4).  
In order to join an IPv6 group, you would need one that supports IPv6.

The right way to do this within Twisted is to set the address family the way 
that twisted.internet.tcp._BaseTCPClient does; check if the input 'interface' 
is a literal ipv4 or ipv6 address and set the self.addressFamily accordingly.

However, assuming you don't care about Windows, as a quick workaround (this is 
NOT a good long-term solution; at some point in the distant future, 
`twisted.internet.udp´ will hopefully disappear as an importable module, since 
you should always be doing this sort of thing via the reactor), you could do 
something like this:

from socket import AF_INET6
from twisted.internet.udp import MulticastPort

class MyMulticastPort(MulticastPort, object):
addressFamily = AF_INET6

def listenMulticast(reactor, port, protocol, interface='', maxPacketSize=8192,
listenMultiple=False):
p = MyMulticastPort(port, protocol, interface, maxPacketSize, reactor,
listenMultiple)
p.startListening()
return p

Note that this is totally untested, which is why I'm answering here and not 
Stack Overflow :-).  Let me know if it works!

-glyph

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [Twisted-web] upcoming changes to twistedmatrix.com mail infrastructure

[Phil Mayers]

On 16/03/16 18:52, Glyph wrote:

Over the last few months, twistedmatrix.com 's
mailman installation has been used increasingly frequently to execute
denial-of-service attacks against people's mailboxes.  This is


My sympathies; this exact problem was the reason we CAPTCHA-ised our 
install of mailman and have to keep a very close eye on it.


It's really a shame there's so little open-source competition in the 
email sector these days; it all appears to have been hoovered up by 
Gmail, Office 365 and various spam (sorry - bulk email) providers.



There will be a couple of inconveniences immediately after the transition:


Couple of random thoughts:

Does mailgun actually contain a mailman-alike product or are you 
effectively building one on top of it?


Will the mailman-style List-X headers remain?

Will the behaviour of the list w.r.t. things like routing of To:/Cc:'ed 
people change.


Good luck with the migration.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] upcoming changes to twistedmatrix.com mail infrastructure

[Glyph]

Over the last few months, twistedmatrix.com's mailman installation has been 
used increasingly frequently to execute denial-of-service attacks against 
people's mailboxes.  This is accomplished by sending huge numbers of 
subscription requests to our website, which in turn sends huge numbers of 
confirmation emails to their inbox.  Based on some information that some 
targeted users have sent me, I now believe that this is to cause those users' 
mail quotas to be exceeded so that password reset or login notification emails 
won't reach them.

This has been going on for some time, but the frequency and severity of the 
attacks seem to be increasing; I only recently realized that this was 
considerably worse than an annoyance for those affected.  I now have at least 1 
confirmed report of this attack being a part of a (partially successful) 
identity theft.

This isn't the only problem we have with email:

We're running our own infrastructure which puts load on our already 
beyond-overloaded volunteer system administration team.
Despite running our own infrastructure, we are not dogfooding Twisted at all in 
the process, so we're not even learning anything useful from the pain; "exim is 
bad" is a lesson we've already learned many times, we do not need to keep 
learning it.
Given how hard it is for us to upgrade Mailman in our current system, we aren't 
even dogfooding our fellow community project terribly well.
Our infrastructure runs on the same host as the website and the buildmaster, 
overloading a very creaky system.
In addition to mailing lists, we run a mail forwarder.  Our server's sender 
reputation is ... not great.  We don't have SPF records, we don't do DKIM, and 
we don't provide authenticated SMTP for users, so emails just come from 
"wherever" when they are sent from, e.g. 'gl...@twistedmatrix.com' :-).

In order to address this, as soon as I can reasonably manage to do so, I will 
be moving Twisted's email infrastructure to mailgun.com , 
a product that I've been successfully using for a range of personal domains (in 
particular, the divmod.com  email forwarder - yes, I still 
operate that, when the Twisted community promises you an email address for life 
you get it ;-)).  Additionally, Mailgun uses a bunch of Twisted within their 
infrastructure, so (although we won't be operating it) we will actually be 
dogfooding considerably more.

(Mailgun is a product of my employer, Rackspace, but they've given us a 
generous open source discount so there's no conflict of interest; the Twisted 
project won't be spending money on this.)

There will be a couple of inconveniences immediately after the transition:

At first, there will be no self-service subscription to mailing lists any more. 
 If you want to subscribe, you'll have to send a message to 
twisted-python-ow...@twistedmatrix.com 
 and the list administrator 
(right now, probably just me) will manually add your address.  (Self-service 
unsubscription will still be possible.)
I'm not sure if I'll be able to keep the list archives at 
https://twistedmatrix.com/pipermail/  
updated, at least at first.  I would encourage everyone to use 
http://news.gmane.org/gmane.comp.python.twisted 
 and 
http://news.gmane.org/gmane.comp.python.twisted.web 
 in the meanwhile.
Speaking of the contents of that sad URL, many disused mailing lists will be 
deleted.  I doubt anyone will notice since there haven't been any posts to most 
of them in many years.
If you presently send email from a twistedmatrix.com 
 address, you will probably want to start using the 
mailgun forwarder so that your messages will have nice shiny DKIM/SPF headers; 
I suspect you may start having more deliverability problems than you already do 
once other mail servers notice that we have said records if you're not using 
them.  I'll distribute SMTP credentials via GPG-encrypted email to everyone I'm 
aware of who uses such an address.

There will be considerable benefits though:

For those of you with @twistedmatrix.com  addresses, 
Mailgun operates a pretty conservative low-pass spam filter, but in looking at 
the analytics from my own personal domains, it really helps a lot and it is 
definitely more effective than the setup we've got right now.
Deliverability and mail-sending performance should be much improved; messages 
should arrive faster because they will be quarantined or deferred-bounced by 
major senders like GMail et. al. far less often, because we'll be forwarding 
less spam and legitimate messages will have appropriate anti-spam headers.
Trac will get faster at certain times because email DoSes should stop hitting 
the server.
Administrative overhead will decrease; we can just stop maintaining em