On Tue, Apr 26, 2011 at 1:49 PM, Cliff Frey <cl...@meraki.com> wrote: > (apologies if this is a duplicate post) > > It might be clearer to just look at my patches and commit messages > https://github.com/clifffrey/Libevent/commits/http_oom_prevention > But I've summarized my problems/fixes below. Please let me know if > you would rather I submit a github pull request, or attach patches to > the email, or submit bugs/features on the website.
Pull requests are just fine. My first suggestion here would be that I'd be a lot more comfortable here with some unit tests. > I want to send a very large (potentially infinite) file to a slow HTTP > client, and I don't want the server to run out of memory while doing > it (imagine I want someone to use one GET request to receive a very > busy stream-of-events, and if they cannot keep up, I want to just skip > some events rather than infinitely-buffering them). > > To do this, I need to be able to get a callback when > evhttp_connection->bufev's write buffer is empty (or close to empty). > I could imagine an implementation where you can explicitly check the > length and explicitly get a callback when the length falls below some > threshold. Another option is to change write_chunk to make a callback > when the write buffer is completely empty. This is what I > implemented, but I am open to other suggestions. If you're interested on the write buffer getting close to empty, the evhttp connection can have a write callback and set a write low-water mark on the bufev so that the callback will get invoked when it falls below a threshold. I'm not too picky about the implementation here as a first cut so long as the API remains sane and supportable in the future. > Also, a completely different bug: If you want to support potentially > infinite POST streams from clients (imagine that you wanted to > implement word-count as an http server, where they POST a document, > and you return the word count) then clients can run your server out of > memory by sending one very large chunk. I think that the > evhttp_request_set_chunked_cb callback should be called on every read, > not just when a complete HTTP chunk has been read. I have made a > patch that does this, but I worry that maybe some user out there > depends on the only-read-complete-http-chunk behavior. Perhaps a flag of some kind could be set so that anybody depending on the old behavior can get it? -- Nick *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
