Re: [libmicrohttpd] Complete request on fd becoming ready
On the MHD side, it may depend on connection volume and your hardware. You could use the 'thread per connection' mode, which would allow the thread to wait for a response without blocking other connections. Alternatively, you could use MHD's 'suspend/resume' functionality to reduce the number of MHD threads (and thus free up the primary MHD thread to service other connections/requests). If you don't use the 'thread per connection' mode, I strongly suspect you're going to want/need to use at least 1 other thread to service the binary protocol side of things. I'm not certain, but I don't think MHD has a mechanism where you can 'service it repeatedly' from a main loop (as is common in some embedded systems or real-time applications). If your hardware can support it, I'd really recommend adding one or more threads to handle the back-end. (C++ might make that easier than straight C, by the way.) For application structure, there are lots of ways to do it. That really doesn't have much to do with MHD. Off the top of my head, I'd consider using condition variables, wait/notify, and some kind of 'result pointer' to transfer data between threads. I do this all the time (in C++ on Linux - not sure how hard that would be to do in C or other platforms). You could also just use 2 blocking queues and have a thread on each side to process the data from one queue and feed stuff to the other queue for the other side to handle. It really depends on how many simultaneous requests you want to process. If traffic/request volume is sufficiently low, the simplest thing to do is use MHD's 'thread per connection' mode and just do the back-end I/O in that thread. Send the request, wait for the response, and send the result back out MHD to the caller. Just make sure the back-end is thread-safe and/or the MHD side waits for a request to finish before sending another (even if it's simply by using a mutex). It's hard to know what to suggest without a lot more info. In short - this sounds a lot more like a threading problem than any problem you might have with MHD. Hope that helps. Ken On Sun, Mar 26, 2017 at 8:44 PM, Hein-Pieter van Braam wrote: > Hello all, > > I am writing an MHD application where the MHD HTTP server acts as a > proxy between a HTTP REST API and a different binary protocol on the > backend. My MHD application opens a new TCP connection to the backend, > translates the request, and waits for an answer. > > The waiting on an answer part would normally be blocking, and reading > the MHD manual I can't come up with a way of using async io here. What > would an MHD application look like that creates response objects from > (another) polling loop, and where do I even place this loop? > > I have been thinking of a variety of convoluted things with threads, > then I figured it'd probably be better to ask the experts :) > > Thanks! > > - HP > >
Re: [libmicrohttpd] Complete request on fd becoming ready
Hi! Thank you for your reply, I think my original question wasn't very well posed. Let me try to rephrase it. In the epoll + threadpool configuration of MHD, MHD itself uses an epoll() type loop to service fds that are ready to send or receive traffic from clients. I/O is non-blocking and when MHD is waiting on the kernel to finish sending traffic back to the client, or waiting on traffic from the client, the fd goes back into the wait list of epoll. What I'm hoping to achieve is to receive traffic from my backend in a similar manner. So I'm hoping for something like this: * MHD mainloop accept()'s a new connection * MHD pushes this new connection to a thread on its threadpool * MHD runs the request handler for the connection * Request handler parses the HTTP request and constructs a request in the binary protocol. * Request handler opens a non-blocking connection to the backend server. < .. magic .. > * The request gets sent to the backend server once the connection to the backend becomes ready for writing. < .. more magic .. > * Once the listening FD for the backend connection becomes ready, create a response object and send it to the client to have MHD then use its normal async loop to service the client. What I'm thinking of doing now is to do something like this in the request handler: * Send the parsed request to a separate thread that has its own epoll() loop, along with struct MHD_Connection *connection. * Return MHD_OK * In the separate thread, deal with the request like normal, when the backend socket becomes ready for reading construct the MHD_Response and call MHD_queue_response (connection, MHD_HTTP_OK, response); * Destroy the response object Now, I'm not entirely sure if this will work for a variety of reasons. I don't know if it's safe toe call MHD_queue_response() in this manner for instance. I was also wondering if there was a way of combining these loops so I don't have to do it this way. Maybe if external select does what I think it does I could use that as my programs' main loop instead of MHD's, but I don't know if that will actually be any better. On Mon, 2017-03-27 at 08:33 -0400, Kenneth Mastro wrote: > > If traffic/request volume is sufficiently low, the simplest thing to > do is use MHD's 'thread per connection' mode and just do the back-end > I/O in that thread. Send the request, wait for the response, and > send the result back out MHD to the caller. Just make sure the back- > end is thread-safe and/or the MHD side waits for a request to finish > before sending another (even if it's simply by using a mutex). It's > hard to know what to suggest without a lot more info. This is more or less what I'm doing now. Doing blocking I/O to the backend service in a thread per connection. > In short - this sounds a lot more like a threading problem than any > problem you might have with MHD. It's not so much a 'problem' as that I'm trying to figure out how to optimize my program so it can be fully asynchronous. > > > On Sun, Mar 26, 2017 at 8:44 PM, Hein-Pieter van Braam > wrote: > > Hello all, > > > > I am writing an MHD application where the MHD HTTP server acts as a > > proxy between a HTTP REST API and a different binary protocol on > > the > > backend. My MHD application opens a new TCP connection to the > > backend, > > translates the request, and waits for an answer. > > > > The waiting on an answer part would normally be blocking, and > > reading > > the MHD manual I can't come up with a way of using async io here. > > What > > would an MHD application look like that creates response objects > > from > > (another) polling loop, and where do I even place this loop? > > > > I have been thinking of a variety of convoluted things with > > threads, > > then I figured it'd probably be better to ask the experts :) > > > > Thanks! > > > > - HP > > > > > >
Re: [libmicrohttpd] Complete request on fd becoming ready
See inline comments, below. Ken On Mon, Mar 27, 2017 at 9:04 AM, Hein-Pieter van Braam wrote: > Hi! > > Thank you for your reply, I think my original question wasn't very well > posed. Let me try to rephrase it. > In the epoll + threadpool configuration of MHD, MHD itself uses an > epoll() type loop to service fds that are ready to send or receive > traffic from clients. I/O is non-blocking and when MHD is waiting on > the kernel to finish sending traffic back to the client, or waiting on > traffic from the client, the fd goes back into the wait list of epoll. > > What I'm hoping to achieve is to receive traffic from my backend in a > similar manner. So I'm hoping for something like this: > > * MHD mainloop accept()'s a new connection > * MHD pushes this new connection to a thread on its threadpool > * MHD runs the request handler for the connection > * Request handler parses the HTTP request and constructs a request in > the binary protocol. > * Request handler opens a non-blocking connection to the backend > server. > < .. magic .. > > * The request gets sent to the backend server once the connection to > the backend becomes ready for writing. > < .. more magic .. > > * Once the listening FD for the backend connection becomes ready, > create a response object and send it to the client to have MHD then use > its normal async loop to service the client. > I'm not entirely sure what 'magic' you need. I may not understand your problem well enough, though. It sounds like you want to use MHD's suspend/resume functionality to allow MHD to use epoll internally to service connections, and then feed it responses when the response is ready. I have not used MHD's suspend/resume code personally (I use the 'thread per connection' mode), but I know many people have. >From what I recall, the gist is: * You get a request from the caller. That causes MHD to call your code to process it. * You do what you need to get things going, and then 'suspend' the connection. * This allows MHD to use that thread to continue to process incoming requests. * Once the result is ready, you call the 'resume' function to allow you to do stuff with that connection again. * At this point I THINK MHD calls your callback again, thus giving you the option to queue a response. (I.e., you don't just queue it from any old thread. You have to wait for one of MHD's threads to service you for that particular connection.) I'm a bit hazy on the details and/or logistics of the above, but I think that's about right. I personally can't help more with that, but I think searching the mailing list archive might be helpful. I've seen messages roll through here in the past couple years that talk about this. Maybe somebody with more experience in this area can chime in. What I'm thinking of doing now is to do something like this in the > request handler: > > * Send the parsed request to a separate thread that has its own epoll() > loop, along with struct MHD_Connection *connection. > * Return MHD_OK > * In the separate thread, deal with the request like normal, when the > backend socket becomes ready for reading construct the MHD_Response and > call MHD_queue_response (connection, MHD_HTTP_OK, response); > * Destroy the response object > > Now, I'm not entirely sure if this will work for a variety of reasons. > I don't know if it's safe toe call MHD_queue_response() in this manner > for instance. > See my above comment. It sounds like you want to use the suspend/resume functionality and you're just wondering how to get it to work. Valid question. I'm not sure how robust the example code for suspend/resume is. (I remember thinking it wasn't impressively clear and/or wasn't really available at the time, but I only gave it cursory attention. The mailing list archive may be of more help.) > I was also wondering if there was a way of combining these loops so I > don't have to do it this way. Maybe if external select does what I > think it does I could use that as my programs' main loop instead of > MHD's, but I don't know if that will actually be any better. > Fair enough. I don't know if you can service other things in an MHD external epoll. (I don't see why not, but I have never used MHD's external select stuff.) If you have the horsepower (and it won't take much), keeping the loops separate may be a bit simpler. I.e., skip the external select. Let MHD do its thing and you do your thing. You have the problem of getting the data back to the caller either way and will still need to use suspend/resume since you shouldn't block the MHD threads. (Or just use thread per connection, of course. You can do whatever you want in those threads because MHD doesn't need them to service other connections with them.) > > On Mon, 2017-03-27 at 08:33 -0400, Kenneth Mastro wrote: > > > > If traffic/request volume is sufficiently low, the simplest thing to > > do is use MHD's 'thread per connection' mode and just do the back-end > > I/O in tha
[libmicrohttpd] why does not libmicrohttpd support for resume from break-point??
why does not libmicrohttpd support for resume from break-point?? how do?? Please give some suggestions??
[libmicrohttpd] feature request: give a possibility to re-assign callbacks
Hello,
1. Subj.
2. Why is this should be useful?
2.1. When you're actively using suspend/resume functionality and later
trying to shutdown the MHD daemon, then last one, by default, uses
default callbacks (which were given at the startup) for _resumed_
connections (as docs says).
By good intentions, such callbacks must be transformed into stubs,
that is, { return MHD_NO; } to complete the task properly. Currently,
it is possible to wait, I may say, weeks if there are thousands of
active requests.
2.2. It gives a possibility to write, I have no good words for this
except next, the polymorph-like servers. For instance, it may be funny
(and may be be a serious work) to load an external shared library and
then re-assign callbacks to the functions which the library contains.
Either, are there any alternatives about how to do tasks above I don't know?
P.S. Yes, I know, that the task must be done properly, that is, there
are no active connections, the daemon should be in quince-mode, or
connections must be blocked by a firewall and so on. This is not a
problem.
Thanks!
--
With Best Regards,
Vitaliy V. Tokarev
Re: [libmicrohttpd] MHD_quiesce_daemon() question
On 03/26/2017 10:36 PM, Evgeny Grin wrote:
> On 26.03.2017 8:33, silvioprog wrote:
>> I found the following related message:
>>
>> https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
>>
>> I've used a similar logic, but with item X below, because I need to wait
>> the client processing:
>>
>> 1) MHD_quiesce_daemon()
>> *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
>> 2) MHD_stop_daemon()
>> 3) close()
>>
>> Real implementation:
>>
>> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) {
>> MHD_socket fd;
>> if (srv && srv->listening) {
>> fd = MHD_quiesce_daemon(srv->mhd);
>> if (!force)
>> while (MHD_get_daemon_info(srv->mhd,
>> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0)
>> usleep(1000 * 500); //TODO: allow to use external callback
>> MHD_stop_daemon(srv->mhd);
>> if (fd != MHD_INVALID_SOCKET)
>> close(fd);
>> srv->listening = false;
>> return true;
>> }
>> return false;
>> }
>>
>>
>> Calling it with bf_httpsrv_shutdown(srv, false) the server stops waiting
>> for clients processing.
>>
>> So, what do you think about the logic above? Should it be improved?!
>>
>> Thanks in advance for any suggestions!
>
> If you don't check returned value from MHD_quiesce_daemon(), you may
> later found that you didn't quiesced daemon, so move check right after
> MHD_quiesce_daemon() and added error handling.
> If you didn't set connection timeout, connection may live indefinitely.
> Moreover, even with connection timeout, clients may continue processing
> on same HTTP 1.1 connections with new requests indefinitely.
> Furthermore, even with HTTP 1.0 and connection timeout hypothetical
> client may read answer very slow with results in very long unpredictable
> closure of connection.
> So: yes, you code will work but may result in very long (hours, for
> example) or even indefinitely long daemon shutdown.
>
And while Evgeny is 100% correct, let me point out the opposite concern:
0.5 s can still be an eternity (think: shell scripts, automated tests,
etc.) and that you ideally should use MHD_OPTION_NOTIFY_CONNECTION to
notify main() that you are "done". For example by doing a semaphore-down
operation/IPC write in main() and a semaphore-up()/IPC read in the
callback IF you are past quiesce and info tells you that you are the
last connection.
signature.asc
Description: OpenPGP digital signature
[libmicrohttpd] Reading data from named pipe
I have an API that reads a subset of data from an archive and writes it to a FILE*. I can use this API to write this data subset to a temporary file via mkstemp() and fdopen(). I am currently using MHD_create_response_from_callback() to read chunks of bytes from this temporary file and write them to the web client. This works well on small files. To avoid the I/O cost of writing a larger temporary file, and then reading from it, I set up a named pipe via mkfifo() and a FILE* that points to it. When I have MHD_create_response_from_callback() try to read from this named pipe, in the same way that it is set up to read from the temporary file, the server and client hang. If this is possible, what is the correct way to serve data from a named pipe? Regards, Alex
Re: [libmicrohttpd] feature request: give a possibility to re-assign callbacks
On 27.03.2017 17:29, Vitaliy T wrote:
> 1. Subj.
Which callbacks would you like to reassign?
> 2. Why is this should be useful?
>
> 2.1. When you're actively using suspend/resume functionality and later
> trying to shutdown the MHD daemon, then last one, by default, uses
> default callbacks (which were given at the startup) for _resumed_
> connections (as docs says).
>
> By good intentions, such callbacks must be transformed into stubs,
> that is, { return MHD_NO; } to complete the task properly. Currently,
> it is possible to wait, I may say, weeks if there are thousands of
> active requests.
What prevents to return "MHD_NO" early from the same callback when required?
> 2.2. It gives a possibility to write, I have no good words for this
> except next, the polymorph-like servers. For instance, it may be funny
> (and may be be a serious work) to load an external shared library and
> then re-assign callbacks to the functions which the library contains.
>
> Either, are there any alternatives about how to do tasks above I don't know?
Simple (pseudo code):
int callback(void *user_data)
{
struct my_data * const info = (struct my_data *) user_data;
switch(info->callback_type)
{
case use_callback_a: return call_callback_a(user_data);
case use_callback_b: return call_callback_b(user_data);
default:
return info->current_callback(user_data);
}
}
Will this work for you?
--
Wishes,
Evgeny
Re: [libmicrohttpd] why does not libmicrohttpd support for resume from break-point？
Hi! Could you clarify, what do you mean by "resume from break-point"? -- Best Wishes, Evgeny Grin On 27.03.2017 5:21, 星**光 wrote: > why does not libmicrohttpd support for resume from break-point？ how > do？ Please give some suggestions？
Re: [libmicrohttpd] feature request: give a possibility to re-assign callbacks
On 27 March 2017 at 23:19, Evgeny Grin wrote: > Which callbacks would you like to reassign? At least, MHD_AccessHandlerCallback (a.k.a DH). > What prevents to return "MHD_NO" early from the same callback when required? Nothing. But on each request we do at least one useless check. > Simple (pseudo code): [...] > Will this work for you? Yes, it will work for me. But I asked about "the right way" :) Your code also uses de-referencing, which also consumes CPU. Anyway, I will live fine with such a workaround. It can be added to the MHD's tutorial. Thanks! -- With Best Regards, Vitaliy V. Tokarev
Re: [libmicrohttpd] MHD_quiesce_daemon() question
Thanks for replying Evgeny, you and Christian always saving my day. :-)
I read and re-read your and Christian answer, and finally I found a
possible way to never lock the server at shutdown: adding an option to exit
the server after configurable attempts. It can solve the timeout problem
too, because I can't ensure that the programmer have configured it (anyway
my library will assume 15 seconds by default). So, the code below applies
your tips and implements the attempts approach:
bool bf_httpsrv_shutdown(struct bf_httpsrv *srv) {
MHD_socket fd;
uint8_t shutdown_attempts; /* Max 256. */
if (srv && srv->listening) {
fd = MHD_quiesce_daemon(srv->mhd);
if (fd != MHD_INVALID_SOCKET)
close(fd);
else
_BF_LOG("Server quiesce failed.\n");
if (srv->forced_shutdown) {
shutdown_attempts = 1;
while (MHD_get_daemon_info(srv->mhd,
MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0) {
if (shutdown_attempts >= srv->shutdown_attempts) { /*
Default srv->shutdown_attempts is 10. */
_BF_LOG("Forced server shutdown.\n");
exit(EINTR);
}
shutdown_attempts++;
sleep(1);
}
}
MHD_stop_daemon(srv->mhd);
srv->listening = false;
return true;
}
return false;
}
Unfortunately if exit was called the MHD_stop_daemon() will be not called,
probably raising some memory leak, but current I have no idea how it could
be solved.
Feel totally free to point improvements about this new version. :-)
On Sun, Mar 26, 2017 at 5:36 PM, Evgeny Grin wrote:
> On 26.03.2017 8:33, silvioprog wrote:
> > I found the following related message:
> >
> > https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
> >
> > I've used a similar logic, but with item X below, because I need to wait
> > the client processing:
> >
> > 1) MHD_quiesce_daemon()
> > *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
> > 2) MHD_stop_daemon()
> > 3) close()
> >
> > Real implementation:
> >
> > bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) {
> > MHD_socket fd;
> > if (srv && srv->listening) {
> > fd = MHD_quiesce_daemon(srv->mhd);
> > if (!force)
> > while (MHD_get_daemon_info(srv->mhd,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0)
> > usleep(1000 * 500); //TODO: allow to use external
> callback
> > MHD_stop_daemon(srv->mhd);
> > if (fd != MHD_INVALID_SOCKET)
> > close(fd);
> > srv->listening = false;
> > return true;
> > }
> > return false;
> > }
> >
> >
> > Calling it with bf_httpsrv_shutdown(srv, false) the server stops waiting
> > for clients processing.
> >
> > So, what do you think about the logic above? Should it be improved?!
> >
> > Thanks in advance for any suggestions!
>
> If you don't check returned value from MHD_quiesce_daemon(), you may
> later found that you didn't quiesced daemon, so move check right after
> MHD_quiesce_daemon() and added error handling.
> If you didn't set connection timeout, connection may live indefinitely.
> Moreover, even with connection timeout, clients may continue processing
> on same HTTP 1.1 connections with new requests indefinitely.
> Furthermore, even with HTTP 1.0 and connection timeout hypothetical
> client may read answer very slow with results in very long unpredictable
> closure of connection.
> So: yes, you code will work but may result in very long (hours, for
> example) or even indefinitely long daemon shutdown.
>
> --
> Best Wishes,
> Evgeny Grin
>
--
Silvio Clécio
Re: [libmicrohttpd] MHD_quiesce_daemon() question
Thanks a lot for your great idea, dude. I certainly will use it very soon.
:-)
I took a look at IPCs, but now, I would like something that stops receiving
new clients and forces the current ones to finish their processing.
Probably I may use IPCs (or any existing feature in MHD) to send some
signal to the clients warning that the server will be finished in N
seconds, so the clients could finish their processing immediately.
On Mon, Mar 27, 2017 at 1:00 AM, Christian Grothoff
wrote:
> On 03/26/2017 10:36 PM, Evgeny Grin wrote:
> > On 26.03.2017 8:33, silvioprog wrote:
> >> I found the following related message:
> >>
> >> https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
> >>
> >> I've used a similar logic, but with item X below, because I need to wait
> >> the client processing:
> >>
> >> 1) MHD_quiesce_daemon()
> >> *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
> >> 2) MHD_stop_daemon()
> >> 3) close()
> >>
> >> Real implementation:
> >>
> >> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) {
> >> MHD_socket fd;
> >> if (srv && srv->listening) {
> >> fd = MHD_quiesce_daemon(srv->mhd);
> >> if (!force)
> >> while (MHD_get_daemon_info(srv->mhd,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0)
> >> usleep(1000 * 500); //TODO: allow to use external
> callback
> >> MHD_stop_daemon(srv->mhd);
> >> if (fd != MHD_INVALID_SOCKET)
> >> close(fd);
> >> srv->listening = false;
> >> return true;
> >> }
> >> return false;
> >> }
> >>
> >>
> >> Calling it with bf_httpsrv_shutdown(srv, false) the server stops waiting
> >> for clients processing.
> >>
> >> So, what do you think about the logic above? Should it be improved?!
> >>
> >> Thanks in advance for any suggestions!
> >
> > If you don't check returned value from MHD_quiesce_daemon(), you may
> > later found that you didn't quiesced daemon, so move check right after
> > MHD_quiesce_daemon() and added error handling.
> > If you didn't set connection timeout, connection may live indefinitely.
> > Moreover, even with connection timeout, clients may continue processing
> > on same HTTP 1.1 connections with new requests indefinitely.
> > Furthermore, even with HTTP 1.0 and connection timeout hypothetical
> > client may read answer very slow with results in very long unpredictable
> > closure of connection.
> > So: yes, you code will work but may result in very long (hours, for
> > example) or even indefinitely long daemon shutdown.
> >
>
> And while Evgeny is 100% correct, let me point out the opposite concern:
> 0.5 s can still be an eternity (think: shell scripts, automated tests,
> etc.) and that you ideally should use MHD_OPTION_NOTIFY_CONNECTION to
> notify main() that you are "done". For example by doing a semaphore-down
> operation/IPC write in main() and a semaphore-up()/IPC read in the
> callback IF you are past quiesce and info tells you that you are the
> last connection.
>
>
--
Silvio Clécio
