On Tue, Apr 26, 2011 at 10:49:46AM -0700, Cliff Frey wrote: > Also, a completely different bug: ?If you want to support potentially > infinite POST streams from clients (imagine that you wanted to > implement word-count as an http server, where they POST a document, > and you return the word count) then clients can run your server out of > memory by sending one very large chunk. ?I think that the > evhttp_request_set_chunked_cb callback should be called on every read, > not just when a complete HTTP chunk has been read. ?I have made a > patch that does this, but I worry that maybe some user out there > depends on the only-read-complete-http-chunk behavior. >
A while back I went and did some looking for a proper way to handle such situations. The first method I came up with would allow a callback to be executed when a full set of headers (before any post data) is recvd. This would allow you to set the chunk callback function which normally can only be used by a client. https://github.com/ellzey/Libevent/commit/36218640b3b19fc94a75e2e691fc41ccda81224c Another approach I tried was to extend the request handling to allow for your normal http callback to get partial chunks of data. When you twiddle the streaming flag, you would need to check evhttp_request's flags for EVHTTP_REQ_PARTIAL_STREAM. If it is set, you can write the output wherever, and the buffer will be drained transparently afterwards. https://github.com/ellzey/Libevent/commit/4ead4de1da726d801c7506c7af648809cd7284f9 These both work in theory, but if I remember correctly have bugs. Picking the "right way" to deal with this issue is hard. The current handler seems clunky to work with, and hacky to modify. This is primarily due to feature creep.. I think the right balance of API transparency, good resource management and abstractions would be introducing more and more user defined callbacks. Such a method would allow a developer to "hook" into all parts of the request cycle. While I support these types of changes, I often wonder if the better solution to these problems is to gut the httpd API and create a more robust and abstracted replacement. *prepares for flogging* Libevent's http API was created for JIT services, not a apache/nginx/iis replacement. But the reality is, developers have shown that the system should be just as robust as the real ones. A project to keep an eye on (one I have started using after dealing with the many faults of the libevent http engine) is https://github.com/ry/http-parser *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
