Just looking over the httperf command, Kuba used --num-calls=1 This
would not be an accurate real-world test because it creates a new
connection for every request whereas most browsers span requests over
only a few connections. Nicholas Piel's test used --num-calls=10 for
testing HTTP/1.1 servers.
In my own test, the difference (on Windows) between 1 and 10 yields a
~2.5x increase in requests per second. I don't have a readily
accessible Linux right now. Kuba, please run these numbers again with
--num-calls=10.
-tim
On 3/19/2010 8:49 AM, Timothy Farrell wrote:
This is a different test than the one I presented. The test I
presented was run on Windows with one instance and tested with
ApacheBench. I've looked at httperf a little and it seems to be a
more realistic test than ApacheBench.
Due to the nature of how Rocket handles listening sockets, it is a
little slower at accepting connections compared to Cherrypy. Nicholas
Piel's test handles 10 requests per connection whereas Apachebench
would handle 1000. So there will be a difference by virtue of the
difference in fresh connections. This could explain why Rocket is
slower with 4 instances, but that being the case it should also be
slower with one instance (even though they hit some arbitrary external
wall) which is inconclusive at this point.
I'd be curious which version Nicholas Piel tested. I just fixed a
performance issue yesterday for linux. If he tested prior to that
version (1.0.2) then yes, it would appear much slower.
Are these numbers consistent with Tim numbers? Could this be dues to a
different memory usage?
Note that my tests were run on Windows. I'm not sure what Cherrypy's
bottleneck on Windows is, but Rocket is not subject to it on that
platform. Also, Rocket uses less memory (by almost 2MB) than Cherrypy
on Windows 7. I haven't looked at memory usage in Linux but due to
Rocket's less-custom code-base we should see a similarly smaller
memory usage amount.
In the 4-instance test, this is not a use-case I'd considered yet. As
previously mentioned, Rocket is slower at accepting connections. If
pound was closing the connection (HTTP 1.0 behavior and some HTTP 1.1
proxies) after every request, this could explain why Rocket comes up
slower.
Some other things to consider:
- Kuba, how many processor cores are on your test machine? Having
more processes than processors will hurt Rocket more than Cherrypy.
- It seems that you are testing this against web2py (notice how all
the responses are 3xx), perhaps you should just test the servers
themselves for now. If that's not the case, may we see the invocation
code?
In the bigger picture, there are some other matters to consider:
- Who will likely run web2py with the build-in webserver? New users
testing things out or devs running relatively small jobs.
- What platforms will those run on? Windows in the former. The
latter is anyone's guess. (Let's not start a rabbit-trail about which
operating system is better, just consider which one most students run.)
So here are some things to consider in this situation:
- Rocket measures (not horribly) slower than Cherrypy on Linux with 4
instances running. How common of a situation is this?
- Rocket is not affected by a major concurrency issue with
single-instance Cherrypy on Windows.
I think going forward we should figure out which one is truly faster
as a single-instance on Linux. I wouldn't be surprised if Rocket is
slightly slower than Cherrypy but it should not be vastly slower. The
goal of Rocket was not to be faster than Cherrypy but to be more
concurrent. So far that's true for Windows and inconclusive on
Linux. I don't have access to a Mac, but I would be surprised if Macs
performed differently than Linux.
Anyone know how to identify that wall that both servers are hitting on
Linux?
-tim
On 3/19/2010 5:36 AM, Kuba Kucharski wrote:
Are these numbers consistent with Tim numbers? Could this be dues to a
different memory usage?
1. Tim?
2. I have a lot of free memory while testing
I wrote email to an author of the blog entry about wsgi webserver
benchmarks - Nicholas Piël
http://nichol.as/benchmark-of-python-web-servers
In short he says:
make sure you do not use ab
yes, in my tests I use httperf
make sure you are running from other machine with limits also tweaked
this is done like that by me
use recompiled httperf
done already
this also comes from him:
"I did a quick benchmark after being pointed to Rocket and I could not
see the same performance advantage for Rocket over CherryPy, more the
opposite."
--
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web...@googlegroups.com.
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en.
