[Twisted-Python] ANN: Eliot 0.8, the logging system with causality

[Itamar Turner-Trauring]

 
Most logging systems can tell you what happened; Eliot tells you _why_
it happened:

$ python linkcheck.py | eliot-tree
4c42a789-76f5-4f0b-b154-3dd0e3041445
+-- check_links@1/started
 `-- urls: [u'http://google.com', u'http://nosuchurl']
 +-- download@2,1/started
 `-- url: http://google.com
 +-- download@2,2/succeeded
 +-- download@3,1/started
 `-- url: http://nosuchurl
 +-- download@3,2/failed
 |-- exception: requests.exceptions.ConnectionError
 |-- reason: ('Conn aborted', gaierror(-2, 'Name unknown'))
 +-- check_links@4/failed
 |-- exception: exceptions.ValueError
 |-- reason: ('Conn aborted.', gaierror(-2, 'Name unknown'))
 And here's the code that generated these logs (eliot-tree [1] was used
to render the output):

import sys
from eliot import start_action, to_file
import requests
to_file(sys.stdout)

def check_links(urls):
 with start_action(action_type="check_links", urls=urls):
 for url in urls:
 try:
 with start_action(action_type="download", url=url):
 response = requests.get(url)
 response.raise_for_status()
 except Exception as e:
 raise ValueError(str(e))

check_links(["http://google.com";], ["http://nosuchurl";])

Interested? Read more at https://eliot.readthedocs.org/ [2].

Eliot is released under the Apache License 2 by ClusterHQ [3], the
Container Data People. We're hiring! [4]
 

Links:
--
[1] https://warehouse.python.org/project/eliot-tree/
[2] https://eliot.readthedocs.org/
[3] https://clusterhq.com
[4] https://clusterhq.com/careers/
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] ANN: Eliot 0.8, the logging system with causality

[Judy Craig]

This is an automatic reply. I am away from the office for the summer, returning 
on Tuesday, August 25, 2015.  I will be happy to get back to you then.  

Enjoy your summer!

Judy Craig
Student Services

This email and any files transmitted with it may be confidential and are 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error please notify the system 
manager. If you are not
the named addressee you should not disseminate, distribute or copy this email.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Blacklisting hosts

[Chris Norman]

Hello,
> On 9 Aug 2015, at 17:07, Cory Benfield  wrote:
> 
> 
>> On 8 Aug 2015, at 08:07, Chris Norman  wrote:
>> 
>> Hi all,
>> I am using Twisted to make a game server. I want to be able to ban IP 
>> addresses. Currently I check if the host is in a blacklist, and if it is, 
>> call abortConnection on the transport. It works fine, but I'm thinking there 
>> should be a better way, to actively refuse the connection in the first place?
> 
> I am not aware of any hook in the BSD socket API that lets you refuse a 
> connection entirely. Generally, you put a socket into ‘listen’ mode 
> (indicating to the OS that you’ll accept new connections), and then you call 
> accept() to get the new connection. In fact, the OS will accept the 
> connection even before you call accept(): it’ll do it asynchronously, and you 
> will just get the FD for the connection. IIRC Windows has a winsock specific 
> thing that might do what you want, but that’s pretty platform specific and 
> probably doesn’t actually prevent the connection getting established anyway.
> 
> If you really want to never allow the connection at all, you’ll probably want 
> to program iptables (or some other firewall if you aren’t on Linux) to do the 
> packet filtering for you. A combination of iptables and ipsets will get you a 
> high-performance IP address blacklist that will drop all packets before they 
> ever reach your application.

Thanks for that. I was sort of hoping for a Pythonic solution that doesn't rely 
on SubProcess ETC, particularly as I want this server to run on any OS you 
throw at it. Thanks for the idea though, I'll certainly use that if I get 
something that little Python can't handle.
> 
> Cory
> 
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Blacklisting hosts

[Chris Norman]

Hello,

> On 10 Aug 2015, at 03:32, Glyph  wrote:
> 
>> 
>> On Aug 9, 2015, at 9:07 AM, Cory Benfield > > wrote:
>> 
>> 
>>> On 8 Aug 2015, at 08:07, Chris Norman >> > wrote:
>>> 
>>> Hi all,
>>> I am using Twisted to make a game server. I want to be able to ban IP 
>>> addresses. Currently I check if the host is in a blacklist, and if it is, 
>>> call abortConnection on the transport. It works fine, but I'm thinking 
>>> there should be a better way, to actively refuse the connection in the 
>>> first place?
>> 
>> I am not aware of any hook in the BSD socket API that lets you refuse a 
>> connection entirely. Generally, you put a socket into ‘listen’ mode 
>> (indicating to the OS that you’ll accept new connections), and then you call 
>> accept() to get the new connection. In fact, the OS will accept the 
>> connection even before you call accept(): it’ll do it asynchronously, and 
>> you will just get the FD for the connection. IIRC Windows has a winsock 
>> specific thing that might do what you want, but that’s pretty platform 
>> specific and probably doesn’t actually prevent the connection getting 
>> established anyway.
>> 
>> If you really want to never allow the connection at all, you’ll probably 
>> want to program iptables (or some other firewall if you aren’t on Linux) to 
>> do the packet filtering for you. A combination of iptables and ipsets will 
>> get you a high-performance IP address blacklist that will drop all packets 
>> before they ever reach your application.
> 
> 
> There is a shortcut in Twisted, at least, although it does not actually 
> refuse the initial connection for the reasons listed above; you can examine 
> the "addr" passed to IProtocolFactory.buildProtocol and return None.

This is perfect, thanks. It would have been better to refuse the connection 
entirely, but as Corey said, I can use iptables if I get desperate.

> 
> -glyph
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com 
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python 
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python