Hey, Thanks for your reply! The json data should never be too long so I'm not worried about the memory usage, I need the whole json object to start working anyway realistically - I was more concerned about blocking reading the data from the network - specifically the request.content.read(), if the client happens to be sending it very slowly this would block everything up right? - or would this not be an issue?
Maybe because I have fairly small content bodies I wont have to worry? Payl On 4 September 2013 22:45, Glyph <gl...@twistedmatrix.com> wrote: > > On Sep 4, 2013, at 10:48 AM, Paul Wiseman <poal...@gmail.com> wrote: > > > I'm still pretty new to twisted and feel I'm slowly getting the hang of > it, enough to realise that this line of code is probably going to block and > not do me any favours, the line is the first line in a resource.Resource > render_POST. > > > > json_request = json.loads(request.content.read()) > > > > The resource.Resource is a child of another resource which is passed to > server.Site which is passed to internent.TCPServer. > > > > The problem is I can't work out how I can read the post data from the > request in an async way. > > > > I have a feeling I need to implement a protocol, I'm guessing a > LineReceiver but I can't figure out how I'd tie that in with my current > solution or specifically how the LineReceiver would even read > asynchronously to be honest.. > > > > Maybe the read is fine? I need the whole post data to do anything useful > I guess as I can't string to a json decoder that I'm aware of. Just it will > block everything up while I read, which shouldn't be long but I guess I'm > bound to the speed of the person posting. > > > > Thanks all! > > If you're parsing a JSON object, you're going to be representing the whole > thing in memory at the end of the interaction regardless, even if you > parsed it and buffered it in an event-driven way. > > That means you need to keep this data relatively small no matter what; if > it's arbitrarily large, you are going to start swapping anyway. > > So probably, just doing the blocking parse is fine. You might be able to > save a *little* memory by parsing it as it comes in, but you're also going > to have to write your own JSON parser, which is probably going to take more > programmer time than you will ever save in execution time by this marginal > memory reduction :-). Better would be to spend that effort enforcing > stringent resource limits so that you will give up reading before you ever > get to the parse in the case where it's big enough to cause a problem. > > -glyph > > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
