On Wed, May 11, 2011 at 10:12 AM, Cliff Frey <cl...@meraki.com> wrote: > I spent some more time thinking about this, and I have a few use cases that > it might be worth keeping in mind. > > It would be nice if the API made it easy to hook up a received HTTP request > (where we are the server) to an outgoing HTTP request (where we are the > client) in order to make building an HTTP proxy very simple. Not a huge > deal if this isn't super simple though. > It should be easy to hook up a zlib compressor/decompressor to the HTTP > library for supporting compressed transfers (with Accept-Encoding).
Yay. > > I'm not sure who should be responsible for allocating the bufferevent > objects. If there was an easy way to connect two existing bufferevent > objects to one another (like bufferevent_pair_new, but for existing ones), > then it would be fine for the http library to provide the bufferevent > objects, but it would probably be more flexible if the caller provided the > bufferevent objects (but then I am unsure who should be responsible for > cleanup of them). > So, in any case, it boils down to something like > Option 1 (from my previous email) > // used instead of evhttp_set_gen_cb, cb is called as soon as the request > // is received, before the content of the request has been received. > void evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request > *, struct bufferevent *bev, void *), void *arg); > // this returns a bufferevent that can be used to send a reply to the client > struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, int > code, const char *reason); > Option 2 > // used instead of evhttp_set_gen_cb, cb is called as soon as the request > // is received, before the content of the request has been received. > // To received the contents of the message, a bufferevent * must be > returned, > // which the evhttp library will fill with the request data, respecting the > // highwater mark. Once BEV_EVENT_EOF has been received, the evhttp library > // will not access the bufferevent * again, and it is up to the user to free > // the bufferevent. > struct bufferevent *evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct > evhttp_request *, void *), void *arg); > // evhttp will free bev after receiving BEV_EVENT_EOF presumably BEV_EVENT_ERROR or BEV_EVENT_TIMEOUT as well > void evhttp_send_reply_bev(struct evhttp_request *req, int code, const char > *reason, struct bufferevent *bev); > > I think that I like option 2 better. > Cliff I like option 2 as well; as you say, it's very easy for proxy servers to use. Somehow I'd avoided reading up too much on bufferevents before; I just did so and see that they're perfect for this. One caveat is that if not all aspects of bufferevents work with evhttp, that should be documented. > > On Tue, May 10, 2011 at 11:36 PM, Cliff Frey <cl...@meraki.com> wrote: >>> >>> So I like the idea of getting read-by-read notification of data as it >>> arrives. I'm not thrilled with the idea of changing default behavior >>> wrt what counts as a read, though. Either a flag or another kind of >>> callback would make me much more comfortable about the change here. >> >> I will try and make a new version of the read-breakup change that adds a >> flag of some sort. >>> >>> I think that the evhttp_send_reply_chunk_with_cb behavior might be a >>> little screwy. Suppose that you send one chunk with a cb, then send >>> another chunk with cb == NULL. The second time you call >>> evhttp_send_reply_chunk_with_cb, it won't clear out the cb, so the >>> first cb will get called for the second chunk. That can't be >>> as-intended, right? >>> >>> I *think* that some part of this could be done by exposing the >>> bufferevent more, but like you I'm a little unclear on the details >>> here. Pausing and unpausing could (I think) be done with >>> bufferevent_enable and disable on the user's part; throttling could be >>> done with the rate-limiting functionality and the high-water/low-water >>> functionality; and did you know that you can have more than one >>> callback on an evbuffer? You can get into lots of fun with that. >>> >>> But on consideration, this code is pretty darn nice. I'm hoping other >>> evhttp users will have a look at it too, but on the whole I am pretty >>> happy about merging it into 2.1. >> >> On the whole I agree with you that my proposed API is a little bit screwy. >> It does get the job done in a fairly simple manner though... which is why I >> did it. In any case, here is an attempt at a cleaner API, but I wanted to >> get feedback from the list on it. This API very-much just builds on the >> current API (although I imagine that in the actual change to http.c, much of >> the current API would be implemented in terms of the new API...) >> Receiving HTTP requests as an HTTP server: >> >> Current interface >> Useful for simple servers, impossible to receive infinite/streaming >> requests >> set a maximum incoming request size (to avoid OOM) >> set a callback that gets a complete request: >> evhttp_set_gencb(evhttp *, void (*cb)(struct evhttp_request *, void >> *), void *arg); >> Proposed interface >> Leave the current interface for simple servers, but add another callback >> that supports receiving streaming requests. If this other callback is >> registered, then it takes precedence over the old one. >> This is called as soon as a request is received, before the body of the >> request has been received (if any) >> >> evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request *, >> struct bufferevent *bev, void *), void *arg); >> >> If the request has content-length 0, then bev is NULL. If 'cb' sets the >> highwater mark of bev, then the http library will attempt to respect that, >> and will stop reading from the underlying TCP connection if bev is full. Is >> is up to 'cb' to register callbacks on bev that will receive future events >> (including EOF for the end of the request) >> >> Sending an HTTP response as an HTTP server: >> >> Current interface: provides two decent options, does not support flow >> control when sending possibly-infinite streams to slow clients (i.e. very >> frequent stock ticker symbol updates over a persistent connection to a very >> very slow client) >> either >> evhttp_send_reply(struct evhttp_request *req, int code, const char >> *reason, struct evbuffer *databuf); >> or >> evhttp_send_reply_start(struct evhttp_request *req, int code, const char >> *reason); >> evhttp_send_reply_chunk(struct evhttp_request *req, struct evbuffer >> *databuf); >> evhttp_send_reply_chunk_with_cb(struct evhttp_request *req, struct >> evbuffer *databuf, void (*cb)(struct evhttp_request *, void *), void *arg); >> Proposed interface: add a third option that allows for flow control >> struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, >> int code, const char *reason); >> It is the caller's responsibility to fill the returned bufferevent with >> the data to be sent to the client, and to eventually >> bufferevent_flush(BEV_FINISHED) on it. At this point, the caller should >> never access the bufferevent or evhttp_request again (i.e. the http library >> will free it when finished). The evhttp library would only empty this >> bufferevent when the underlying TCP connection's bufferevent was not too >> full (probably with a configurable highwater mark defining full, i.e. >> evhttp_connection_set_xmit_highwater or something) >> >> I believe that a similar set of additions could be made with the >> http-client API, but I wanted to get feedback on this potential server API >> first. It would likely involve incompatible changes to the evhttp_request >> structure, but people are not supposed to depend on that anyways. >> Cliff > -- Scott Lamb <http://www.slamb.org/> *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
