On 07/09/2013 09:04 AM, Rob Meijer wrote:
Hi everyone,
I'm working on what is just my second project using Twisted-Web, so I'm
still a relative newbee on the subject.
I'm working on a project that uses Twisted Web as a simple authorization
proxy. All requests to my proxy contain an authorization-token and are
either handled by the proxy, or are relayed to an other server. For all
GET stuff and small POST stuff this is not a problem. When I want to
process large POST requests however, I run into my limits of understanding
how Twisted Web actually works.
1) I figured out that next to the 'process' in my request handler, I need
to also overload handleContentChunk, parse the form body-parts in the
first chunk myself and open a proxy connection (self.agent.request) if the
authorization token checks out.
2) When it comes to appending the data received in handleContentChunk, and
if needed throttling the client if the server couldn't keep up, I can't
figure out how to connect handleContentChunk and my self.agent.request
instance.
You probably want to read up on the producer/consumer stuff in Twisted.
In particular if you're using t.w.client.Agent, bodies in requests are
supplied by an IBodyProducer.
http://twistedmatrix.com/documents/current/web/howto/client.html
Essentially, you need an IBodyProducer that maps to the incoming
transport via request, which I guess would look something like this:
class RequestProducer(object):
implements(IBodyProducer)
def __init__(self, request):
self.req = request
def startProducing(self, consumer):
self.d = defer.Deferred()
self.consumer = consumer
return d
def pauseProducing(self):
self.req.transport.pauseProducing()
def stopProducing(self):
# FIXME: what to do here...
self.req.transport.loseConnection()
def finish():
self.d.callback(None)
...and you'll have code like this on the request object:
def gotLength(self, length):
self.bodyprod = RequestProducer(self)
if length:
self.bodyprod.length = length
else:
self.bodyprod.length = twisted.web.iweb.UNKNOWN_LENGTH
self.out_req = self.agent.request(
'GET', url, headers, bodyprod
)
def handleContentChunk(self, data):
...
if data_to_be_forwarded:
self.bodyprod.consumer.write(data)
if some_done_condition:
self.bodyprod.finish()
3) When the token does not check out, or the connection to the server
fails, it remains a mystery to me how I should throw an error in such a
way that it allows me to send a proper error message to the client, while
This is sort of a problem with HTTP. The client will probably keep
sending the data.
The best you can do is write an HTTP error to the transport then throw
the connection away, or blackhole all future content chunks.
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
