[Twisted-Python] Twisted Endpoints and WebSocket / Autobahn

[Tobias Oberstein]

Hi,

I am working on Twisted Endpoint support in Autobahn 
https://github.com/tavendo/AutobahnPython.

So far, I made Autobahn able to talk WebSocket _over_ arbitrary Twisted 
Endpoints, e.g. WebSocket over Unix domain sockets works. This is already quite 
nifty.

Is there anything like Twisted Endpoints for Processes and/or Serial? Like:

endpoint = serverFromString(reactor,"process:program=/usr/local/bin/myprogram")

or

endpoint = serverFromString(reactor,"serial:port=/dev/tty1:baudrate=115000")

That would allow to talk WebSocket over stdin/stdout to a program or a serial 
conneced device. Both can be useful in certain situations.

I'd also like to add the other: creating an Twisted Endpoint _from_ an Autobahn 
server/client to be able to talk any (stream) protocol _over_ WebSocket (like 
SSH or VNC over WebSocket).

endpoint = 
serverFromString(reactor,"websocket:tcp:80:url=ws://myhostname.com/somepath")

I guess I need to provide an implementation of
http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.ITransport.html

What else? Is there some docs or recipe?

Thanks!
/Tobias


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Queries about connecting to a XML-RPC server over IPv6

[Amit Saha]

- Original Message -
> From: "Amit Saha" 
> To: "Twisted general discussion" 
> Sent: Tuesday, December 10, 2013 11:59:27 AM
> Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server 
> over IPv6
> 
> 
> 
> - Original Message -
> > From: "Phil Mayers" 
> > To: twisted-python@twistedmatrix.com
> > Sent: Monday, December 9, 2013 7:33:17 PM
> > Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server
> > over IPv6
> > 
> > On 09/12/13 05:28, Amit Saha wrote:
> > 
> > > proxy = Proxy('http://localhost6:8000')
> > > proxy.callRemote('my_proxy_method').addCallbacks(printValue, printError)
> > >
> > >
> > > When I run it, i get "No route to host: 101, Network is unreachable".
> > >
> > > However, 'curl -6 localhost:8000' succeeds. What could be going on here?
> > 
> > The IPv6 support in Twisted is very new - check your version even *has*
> > it - and it's still a work in progress. You might find that you can't do
> > this. I'd have to read the code to be sure and I don't have time right
> > now, but my guess is that t.w.xmlrpc isn't "getaddrinfo"-ised so won't
> > connect to IPv6 names.
> > 
> > If your version of Twisted supports it, you could probably work around
> > this by sub-classing Proxy and calling reactor.connectTCP('::1', 8000)
> > to attach the protocol yourself.
> 
> Thanks for your reply, Phil. So, I basically modified twisted/web/xmlrpc.py
> so that
> when it was calling connectTCP(), i substituted '::1' in place of self.host
> and yes the method is called successfully.
> 
> I will consider your other hints and see if there is an easy way to work
> around this
> in my particular use case.

I finally ended up doing this:

(Sorry for the +)

+class ProxyIPv6(xmlrpc.Proxy):
+
+def __init__(self, url, **kwargs):
+
+xmlrpc.Proxy.__init__(self, url, kwargs)
+# resolve the LC's host/port to IPv6 address
+# and overrise self.host
+retries = 0
+while retries < 5:
+try:
+self.host = socket.getaddrinfo(self.host, self.port,
+   socket.AF_INET6, 0, 
socket.SOL_TCP)[0][4][0]
+except:
+# IPv6 look up failed
+log.debug('Failed to look up IPv6 address for LC. Sleeping for 
5s')
+time.sleep(5)
+retries += 1
+else:
+log.info('Resolved IPv6 address of the LC.')
+break


I have the retry loop since I saw that the IPv6 name resolution was failing 
once, but never happened
after that. The particular use case I am currently applying this is in a daemon 
process which starts on boot
on a IPv4 and IPv6 system. Then I disable IPv4 and the communication with the 
XML-RPC proxy server
happens only over IPv6.

Thanks for the pointers earlier.
-Amit.

-- 
Amit Saha 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Queries about connecting to a XML-RPC server over IPv6

[Phil Mayers]

On 12/12/13 12:23, Amit Saha wrote:



- Original Message -

From: "Amit Saha" 
To: "Twisted general discussion" 
Sent: Tuesday, December 10, 2013 11:59:27 AM
Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server over 
IPv6



- Original Message -

From: "Phil Mayers" 
To: twisted-python@twistedmatrix.com
Sent: Monday, December 9, 2013 7:33:17 PM
Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server
over IPv6

On 09/12/13 05:28, Amit Saha wrote:


proxy = Proxy('http://localhost6:8000')
proxy.callRemote('my_proxy_method').addCallbacks(printValue, printError)


When I run it, i get "No route to host: 101, Network is unreachable".

However, 'curl -6 localhost:8000' succeeds. What could be going on here?


The IPv6 support in Twisted is very new - check your version even *has*
it - and it's still a work in progress. You might find that you can't do
this. I'd have to read the code to be sure and I don't have time right
now, but my guess is that t.w.xmlrpc isn't "getaddrinfo"-ised so won't
connect to IPv6 names.

If your version of Twisted supports it, you could probably work around
this by sub-classing Proxy and calling reactor.connectTCP('::1', 8000)
to attach the protocol yourself.


Thanks for your reply, Phil. So, I basically modified twisted/web/xmlrpc.py
so that
when it was calling connectTCP(), i substituted '::1' in place of self.host
and yes the method is called successfully.

I will consider your other hints and see if there is an easy way to work
around this
in my particular use case.


I finally ended up doing this:


Just to point out this is obviously a pretty hackish way of doing it; 
for starters, getaddrinfo() is blocking, so you should use deferToThread 
at minimum.


You might want to look at:

http://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/internet/endpoints.py#L707

...for inspiration.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Queries about connecting to a XML-RPC server over IPv6

[Amit Saha]

- Original Message -
> From: "Phil Mayers" 
> To: twisted-python@twistedmatrix.com
> Sent: Thursday, December 12, 2013 10:33:21 PM
> Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server 
> over IPv6
> 
> On 12/12/13 12:23, Amit Saha wrote:
> >
> >
> > - Original Message -
> >> From: "Amit Saha" 
> >> To: "Twisted general discussion" 
> >> Sent: Tuesday, December 10, 2013 11:59:27 AM
> >> Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC server
> >> over IPv6
> >>
> >>
> >>
> >> - Original Message -
> >>> From: "Phil Mayers" 
> >>> To: twisted-python@twistedmatrix.com
> >>> Sent: Monday, December 9, 2013 7:33:17 PM
> >>> Subject: Re: [Twisted-Python] Queries about connecting to a XML-RPC
> >>> server
> >>> over IPv6
> >>>
> >>> On 09/12/13 05:28, Amit Saha wrote:
> >>>
>  proxy = Proxy('http://localhost6:8000')
>  proxy.callRemote('my_proxy_method').addCallbacks(printValue, printError)
> 
> 
>  When I run it, i get "No route to host: 101, Network is unreachable".
> 
>  However, 'curl -6 localhost:8000' succeeds. What could be going on here?
> >>>
> >>> The IPv6 support in Twisted is very new - check your version even *has*
> >>> it - and it's still a work in progress. You might find that you can't do
> >>> this. I'd have to read the code to be sure and I don't have time right
> >>> now, but my guess is that t.w.xmlrpc isn't "getaddrinfo"-ised so won't
> >>> connect to IPv6 names.
> >>>
> >>> If your version of Twisted supports it, you could probably work around
> >>> this by sub-classing Proxy and calling reactor.connectTCP('::1', 8000)
> >>> to attach the protocol yourself.
> >>
> >> Thanks for your reply, Phil. So, I basically modified
> >> twisted/web/xmlrpc.py
> >> so that
> >> when it was calling connectTCP(), i substituted '::1' in place of
> >> self.host
> >> and yes the method is called successfully.
> >>
> >> I will consider your other hints and see if there is an easy way to work
> >> around this
> >> in my particular use case.
> >
> > I finally ended up doing this:
> 
> Just to point out this is obviously a pretty hackish way of doing it;
> for starters, getaddrinfo() is blocking, so you should use deferToThread
> at minimum.

Indeed it is very hackish. It is basically the first thing that helped me solve 
the problem.
> 
> You might want to look at:
> 
> http://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/internet/endpoints.py#L707
> 
> ...for inspiration.

Thanks, I will take a look and hopefully can modify my approach.

Best,
Amit.

-- 
Amit Saha 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Reliable way to check if Twisted has IPv6 support?

[Amit Saha]

Hello,

I have a code path similar to:

if Twisted has IPv6:
  # do this
else:
  # do that

I came up with this: 

Using 'getattr' to get a function which wouldn't exist if there was no IPv6 
address as follows:

>>> from twisted.internet import abstract
>>> getattr(abstract, 'isIPv6Address')


Is there a more reliable way (which works with the Twisted-8.0+) to check this?

Thanks,
Amit.


-- 
Amit Saha 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python