I've noticed that there appears to be a significant performance hit when using CURLOPT_READFUNCTION. This issue seems to be platform dependent as I've only been able to get poor performance on Linux (Amazon Linux m1.large 64-bit ami-0358ce33) across multiple versions of cURL and PHP. I've not seen any performance issues on my Mac running PHP 5.3.15 and cURL 7.21.4.
When sending PUT requests containing a 10 byte body (testing123) to a node.js server (others have reported issues with Jetty as well) using CURLOPT_READFUNCTION, the read and write times returned from CURLINFO_SPEED_UPLOAD and CURLINFO_SPEED_DOWNLOAD are very poor: ~833 upload and 1333 download. If you send the same request using CURLOPT_CUSTOMREQUEST => PUT and send the body using CURLOPT_POSTFIELDS then the transfer times are significantly improved: ~34,000 upload and ~55,000 download. Note: In both tests, I disabled the Expect: 100-Continue header by setting an "Expect:" header in CURLOPT_HTTPHEADER. I am also utilizing persistent HTTP connections by using the same multi handle and computing the average upload and download times across many different requests. I wrote a very simple test script that demonstrates the performance issue: https://gist.github.com/mtdowling/6059009. You'll need to have a node.js server running to handle the requests. I've written up a simple bash script that will install PHP, node.js, start the test server, and run the performance test: https://gist.github.com/anonymous/6059035. Thinking that this might be an issue with a specific version of cURL or PHP, I manually compiled different versions of PHP and cURL and ran the performance tests. There was no improvement using the version combination I had success with on my mac or using the latest version of cURL (7.31) and PHP (5.5.1). This does not appear to be version dependent. Here are the results of that testing: https://github.com/guzzle/guzzle/issues/349#issuecomment-21284834 I ran strace on the PHP script and found that using CURLOPT_POSTFIELDS appears to send the headers and the entire payload before receiving anything from the server, while CURLOPT_READFUNCTION appears to send the request headers, receive the response headers, then sends the body afterwards. I've provided the strace output below. For brevity and easier comprehension, I removed the various calls to "clock_gettime(CLOCK_MONOTONIC, {17579, 343661534}) = 0". CURLOPT_READFUNCTION strace: sendto(3, "PUT /guzzle-server/perf HTTP/1.1"..., 78, MSG_NOSIGNAL, NULL, 0) = 78 poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}, {fd=3, events=POLLOUT|POLLWRNORM}], 2, 0) = 1 ([{fd=3, revents=POLLOUT|POLLWRNORM}]) sendto(3, "testing123", 10, MSG_NOSIGNAL, NULL, 0) = 10 select(4, [3], [], [], {1, 0}) = 1 (in [3], left {0, 964661}) poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=3, revents=POLLIN|POLLRDNORM}]) recvfrom(3, "HTTP/1.1 200 OK\r\nContent-Length:"..., 16384, 0, NULL, NULL) = 116 poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout) CURLOPT_POSTFIELDS strace: sendto(3, "PUT /guzzle-server/perf HTTP/1.1"..., 137, MSG_NOSIGNAL, NULL, 0) = 137 poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=3, revents=POLLIN|POLLRDNORM}]) recvfrom(3, "HTTP/1.1 200 OK\r\nContent-Length:"..., 16384, 0, NULL, NULL) = 116 poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout) The loop used to execute the curl_multi handles is very simple and can be found in the test script at https://gist.github.com/mtdowling/6059009#file-readfuction_perf-php-L5. Does anyone have any insight on why I'm seeing such a performance hit? Is there some way I can get better performance, perhaps by rearranging my CURLOPT_* options or changing my loop that executes the cURL handles? Based on the strace output, I would assume that this is a cURL issue and not a PHP issue. Please let me know if I can supply any additional information to help troubleshoot. Thanks, Michael ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
