On Wed, May 11, 2011 at 11:21 AM, Scott Lamb <sl...@slamb.org> wrote: > On Wed, May 11, 2011 at 10:12 AM, Cliff Frey <cl...@meraki.com> wrote: >> I spent some more time thinking about this, and I have a few use cases that >> it might be worth keeping in mind. >> >> It would be nice if the API made it easy to hook up a received HTTP request >> (where we are the server) to an outgoing HTTP request (where we are the >> client) in order to make building an HTTP proxy very simple. Not a huge >> deal if this isn't super simple though. >> It should be easy to hook up a zlib compressor/decompressor to the HTTP >> library for supporting compressed transfers (with Accept-Encoding). > > Yay. > >> >> I'm not sure who should be responsible for allocating the bufferevent >> objects. If there was an easy way to connect two existing bufferevent >> objects to one another (like bufferevent_pair_new, but for existing ones), >> then it would be fine for the http library to provide the bufferevent >> objects, but it would probably be more flexible if the caller provided the >> bufferevent objects (but then I am unsure who should be responsible for >> cleanup of them). >> So, in any case, it boils down to something like >> Option 1 (from my previous email) >> // used instead of evhttp_set_gen_cb, cb is called as soon as the request >> // is received, before the content of the request has been received. >> void evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request >> *, struct bufferevent *bev, void *), void *arg); >> // this returns a bufferevent that can be used to send a reply to the client >> struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, int >> code, const char *reason); >> Option 2 >> // used instead of evhttp_set_gen_cb, cb is called as soon as the request >> // is received, before the content of the request has been received. >> // To received the contents of the message, a bufferevent * must be >> returned, >> // which the evhttp library will fill with the request data, respecting the >> // highwater mark. Once BEV_EVENT_EOF has been received, the evhttp library >> // will not access the bufferevent * again, and it is up to the user to free >> // the bufferevent. >> struct bufferevent *evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct >> evhttp_request *, void *), void *arg); >> // evhttp will free bev after receiving BEV_EVENT_EOF > > presumably BEV_EVENT_ERROR or BEV_EVENT_TIMEOUT as well > >> void evhttp_send_reply_bev(struct evhttp_request *req, int code, const char >> *reason, struct bufferevent *bev); >> >> I think that I like option 2 better. >> Cliff > > I like option 2 as well; as you say, it's very easy for proxy servers > to use. Somehow I'd avoided reading up too much on bufferevents > before; I just did so and see that they're perfect for this. One > caveat is that if not all aspects of bufferevents work with evhttp, > that should be documented.
Also, sorry if this is obvious to someone who's actually used bufferevents, but a question: how would the error behavior work? * If the client side fails, it probably indicates that via BEV_EVENT_ERROR|BEV_EVENT_READING? (Presumably even if the error at the TCP layer was on a write call.) * If the server side fails, BEV_EVENT_ERROR|BEV_EVENT_WRITING? If the bufferevent in question came from a client request, how do both the client evhttp request and the user code get error notification? (Maybe the least surprising thing would be if the user could freely call bufferevent_setcb without worrying about the client request? It probably should be made clear if the client request still exists when this callback is called.) * If the user code wants to abort a request in progress (single or linked), how would it do so? > >> >> On Tue, May 10, 2011 at 11:36 PM, Cliff Frey <cl...@meraki.com> wrote: >>>> >>>> So I like the idea of getting read-by-read notification of data as it >>>> arrives. I'm not thrilled with the idea of changing default behavior >>>> wrt what counts as a read, though. Either a flag or another kind of >>>> callback would make me much more comfortable about the change here. >>> >>> I will try and make a new version of the read-breakup change that adds a >>> flag of some sort. >>>> >>>> I think that the evhttp_send_reply_chunk_with_cb behavior might be a >>>> little screwy. Suppose that you send one chunk with a cb, then send >>>> another chunk with cb == NULL. The second time you call >>>> evhttp_send_reply_chunk_with_cb, it won't clear out the cb, so the >>>> first cb will get called for the second chunk. That can't be >>>> as-intended, right? >>>> >>>> I *think* that some part of this could be done by exposing the >>>> bufferevent more, but like you I'm a little unclear on the details >>>> here. Pausing and unpausing could (I think) be done with >>>> bufferevent_enable and disable on the user's part; throttling could be >>>> done with the rate-limiting functionality and the high-water/low-water >>>> functionality; and did you know that you can have more than one >>>> callback on an evbuffer? You can get into lots of fun with that. >>>> >>>> But on consideration, this code is pretty darn nice. I'm hoping other >>>> evhttp users will have a look at it too, but on the whole I am pretty >>>> happy about merging it into 2.1. >>> >>> On the whole I agree with you that my proposed API is a little bit screwy. >>> It does get the job done in a fairly simple manner though... which is why I >>> did it. In any case, here is an attempt at a cleaner API, but I wanted to >>> get feedback from the list on it. This API very-much just builds on the >>> current API (although I imagine that in the actual change to http.c, much of >>> the current API would be implemented in terms of the new API...) >>> Receiving HTTP requests as an HTTP server: >>> >>> Current interface >>> Useful for simple servers, impossible to receive infinite/streaming >>> requests >>> set a maximum incoming request size (to avoid OOM) >>> set a callback that gets a complete request: >>> evhttp_set_gencb(evhttp *, void (*cb)(struct evhttp_request *, void >>> *), void *arg); >>> Proposed interface >>> Leave the current interface for simple servers, but add another callback >>> that supports receiving streaming requests. If this other callback is >>> registered, then it takes precedence over the old one. >>> This is called as soon as a request is received, before the body of the >>> request has been received (if any) >>> >>> evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request *, >>> struct bufferevent *bev, void *), void *arg); >>> >>> If the request has content-length 0, then bev is NULL. If 'cb' sets the >>> highwater mark of bev, then the http library will attempt to respect that, >>> and will stop reading from the underlying TCP connection if bev is full. Is >>> is up to 'cb' to register callbacks on bev that will receive future events >>> (including EOF for the end of the request) >>> >>> Sending an HTTP response as an HTTP server: >>> >>> Current interface: provides two decent options, does not support flow >>> control when sending possibly-infinite streams to slow clients (i.e. very >>> frequent stock ticker symbol updates over a persistent connection to a very >>> very slow client) >>> either >>> evhttp_send_reply(struct evhttp_request *req, int code, const char >>> *reason, struct evbuffer *databuf); >>> or >>> evhttp_send_reply_start(struct evhttp_request *req, int code, const char >>> *reason); >>> evhttp_send_reply_chunk(struct evhttp_request *req, struct evbuffer >>> *databuf); >>> evhttp_send_reply_chunk_with_cb(struct evhttp_request *req, struct >>> evbuffer *databuf, void (*cb)(struct evhttp_request *, void *), void *arg); >>> Proposed interface: add a third option that allows for flow control >>> struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, >>> int code, const char *reason); >>> It is the caller's responsibility to fill the returned bufferevent with >>> the data to be sent to the client, and to eventually >>> bufferevent_flush(BEV_FINISHED) on it. At this point, the caller should >>> never access the bufferevent or evhttp_request again (i.e. the http library >>> will free it when finished). The evhttp library would only empty this >>> bufferevent when the underlying TCP connection's bufferevent was not too >>> full (probably with a configurable highwater mark defining full, i.e. >>> evhttp_connection_set_xmit_highwater or something) >>> >>> I believe that a similar set of additions could be made with the >>> http-client API, but I wanted to get feedback on this potential server API >>> first. It would likely involve incompatible changes to the evhttp_request >>> structure, but people are not supposed to depend on that anyways. >>> Cliff >> > > > > -- > Scott Lamb <http://www.slamb.org/> > -- Scott Lamb <http://www.slamb.org/> *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
