Thanks for the detailed reply.
1.
twistd.bat web -n --path=%path% --profile=stats_obj --profiler=cProfile
--savestats
is NOT working for twisted 9.0, --profile and --profiler are both not
recognized.
2.
Sounds like I start to work until all the data are received, as the content
are multipart/mixed typed pretty large content, I wanna start to find
boundaries and save part of them ASAP. Any clue?
3.
Not only one file, but many files in a multipart/mixed typed body which
totally reach a 50MB/s speed.
I made a tests which doing a multipart/mixed request, containing 50000 * 1KB
small files. Assume socket.sendall(data) is called
at 00:00
render_POST start: 00:03
process_multipart_content_start: 00:03
process_multipart_content_done:00:11
write_all_small_files_in_one_file_start :00:11
write_all_small_files_in_one_file_done:00:13
respones_given:00:13
Yes, it takes 13s to get it done. I can only make the
process_multipart_content spend less time by combining those small files in
several larger files but I still don't know why 3s are spend before
render_POST.
On Tue, Dec 29, 2009 at 2:02 AM, Reza Lotun <rlo...@gmail.com> wrote:
> Hi,
>
> > Currently I'm running twisted web via this command line:
> > twistd.bat web -n --path=%path%
> > ...
> > 1. How can I turn on the --profile option?
>
> You should just be able to do something like:
>
> twistd.bat web -n --path=%path% --profile=stats_obj
> --profiler=cProfile --savestats
>
> Then after a duration of time pounding the server with requests you can
> stop
> it and introspect the stats_obj file:
>
> python
> >> import pstats
> >> s = pstats.Stats('stats_obj')
> >> s.sort_stats('time').print_stats()
>
> This will give you an ordered list of functions where the most time has
> been
> spent. It's also useful to sort by cumulative time. For more information on
> how to introspect stats objects see the Python documentation for the pstats
> and cProfile module.
>
> > 2. Any way to know how many millisecond elapsed between "request
> received"
> > and render_POST call?
>
> You'll have to measure this yourself I think. A glance at the twisted
> codebase informs me:
> - In twisted/web/http.py incoming requests are handled by an
> HTTPChannel (the protocol from the http server factory)
> - Incoming data is processed by the lineReceived method which creates
> a Request object
> - When all content for a request is received the allContentsReceived
> method is invoked, which in turn calls the requestReceived method on
> the Request object, which does some internal processing like parsing a
> POST-encoded body if there is one. This then calls the process method
> on the Request object
> - In twisted/web/server.py the Request subclass' process method then
> finds the resource being requested and calls the render method
>
> I imagine you could modify the code in the above places to set global
> variables by calling time.time(), and dumping them out at some point.
>
> > 3. If there are a bunch of files need to be uploaded (say at 50MB/s speed
> )
> > and save by server, Should I write the file directly in render_POST or
> using
> > deferred call?
>
> Do you mean the server will be *receiving* a file in the HTTP request
> body? You can access the request body by looking at request.content
> which exposes a File-like API (that is, threat it like you would an
> open file in Python).
>
> If you want to *send* a file via Twisted, take a look at:
> http://twistedmatrix.com/documents/current/core/howto/producers.html
> and
>
> http://twistedmatrix.com/documents/9.0.0/api/twisted.protocols.basic.FileSender.html
>
> Cheers,
> Reza
>
> --
> Reza Lotun
> mobile: +44 (0)7521 310 763
> email: rlo...@gmail.com
> work: r...@tweetdeck.com
> twitter: @rlotun
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
--
>: ~
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
