I did some testing between a twisted xmlrpc ssl server and a twisted client
with lots of simultaneous requests.
I'm using windows (don't have the choice...), python 2.7, twisted 11.1 and both
server and client are on the same computer... (Xeon 2Ghz 3GB of RAM)
The client is configured to get a number of "Hello" answers and if there is a
problem on the connection, it will try again with a calllater of 1second
If the number of requests is low (<100), there is no problem.
With 200 requests, there is some errors "Connection to the other side was lost
in a non-clean fashion." but the client can still get all the answers by
retrying.
If I try to increase the number of requests, the number of retries is rising
very fast and at some point the client is unable to get any answer (with errors
like "User timeout caused connection failure")
100 request - no retries 0.688 s
200 requests - 37 retries 1.5s
500 requests - 395 retries 4.2s
750 requests - 1205 retries - 36.4s
1500 requests - too much
In the client the time is measured between the start of the reactor and the
reception of each answer.
What I don't understand is that it seems no answer is parsed before all the
messages are received.
For example for the 750 requests, the first message is parsed after 32.1s
My questions are the following:
- what is happening ?
- why are all messages parsed at the end ? Priority of the reactor ?
- is it possible to limit the number of incoming connections globally and/or
for each client ? How ? Same for outgoing connections in the client.
- to be able to see what is going on, is it possible to print the number of
connections (counting connectionMade and connectionLost or is there a better
method?), and the number of callbacks waiting in the reactor ?
- is there a maximum number of callbacks waiting in the reactor ?
The code I used is below.
Modified from
http://twistedmatrix.com/pipermail/twisted-python/2007-May/015357.html
Thanks for your help.
##
xmlrpc_ssl_helloworld_server.py
##
from twisted.web import xmlrpc, server, http
from twisted.internet import reactor, ssl
from twisted.web.xmlrpc import withRequest
class Engine:
def __init__(self):
self.clientWS_ip_address = "127.0.0.1"
self.clientWS_port = 8010
self.clientWS =
ClientWebServiceListener(self.clientWS_ip_address,self.clientWS_port,self)
self.clientWS.start()
class ClientWebServiceListener():
class XmlRpcFunctions(xmlrpc.XMLRPC):
@withRequest
def xmlrpc_helloworld(self,request):
return 'Hello ' +
request.channel.transport.getPeerCertificate().get_subject().commonName
def __init__(self,ip_address,port,engine):
self.engine = engine
self.ip_address = ip_address
self.port = port
def start(self):
ctx = self.makeSSLContext(myKey='server.pem',trustedCA='cacert.pem')
self.listening_port = reactor.listenSSL(self.port,
server.Site(self.XmlRpcFunctions()) ,ctx)
def makeSSLContext(self,myKey,trustedCA):
'''Returns an ssl Context Object
@param myKey a pem formated key and certificate with for my current host
the other end of this connection must have the cert from the CA that
signed this key
@param trustedCA a pem formated certificat from a CA you trust
you will only allow connections from clients signed by this CA
and you will only allow connections to a server signed by this CA
'''
fd = open(myKey,'r')
theCert = ssl.PrivateCertificate.loadPEM(fd.read())
fd.close()
fd = open(trustedCA,'r')
theCA = ssl.Certificate.loadPEM(fd.read())
fd.close()
ctx = theCert.options(theCA)
# Now the options you can set look like Standard OpenSSL Library options
# The SSL protocol to use, one of SSLv23_METHOD, SSLv2_METHOD,
# SSLv3_METHOD, TLSv1_METHOD. Defaults to TLSv1_METHOD.
ctx.method = ssl.SSL.TLSv1_METHOD
# If True, verify certificates received from the peer and fail
# the handshake if verification fails. Otherwise, allow anonymous
# sessions and sessions with certificates which fail validation.
ctx.verify = True
# Depth in certificate chain down to which to verify.
ctx.verifyDepth = 1
# If True, do not allow anonymous sessions.
ctx.requireCertification = True
# If True, do not re-verify the certificate on session resumption.
ctx.verifyOnce = True
# If True, generate a new key whenever ephemeral DH parameters are used
# to prevent small subgroup attacks.
ctx.enableSingleUseKeys = True
# If True, set a session ID on each context. This allows a shortened
# handshake to be used when a known client reconnects.
ctx.enableSessions = True
# If True, enable various non-spec protocol fixes for broken
# SSL implementations.
ctx.fixBrokenPeer
