On 12/06/2012 02:58 AM, Sebastiano Merlino wrote:
Hi Christian,
thanks for your reply. I implemented a "multi-thread external select"
using your suggestion of providing the same external non blocking
socket to multiple daemons.
My question now is, once the content_reader_callbacks returns 0
(because it has not still the data), the file descriptor seems to be
removed from the library's fd_sets. Have I to store the file
descriptor somewhere and re-insert it inside the write fd_set manually
or there is another way?
MHD will put it back for you -- once your content reader callback
returns non-zero! You are waiting on some OTHER kind of event
(presumably, after all, you wanted to produce the data asynchronously!),
which you put into the select. Once that OTHER event produces data, you
call MHD_run again and it will again call your callback (without the FD
being in 'select'); once your callback returns non-zero, MHD will put
the socket back into the FD set.
Happy hacking!
Christian
Thanks,
Sebastiano
2012/12/5 Christian Grothoff <[email protected]
<mailto:[email protected]>>
On 12/04/2012 07:59 PM, Sebastiano Merlino wrote:
Hi all,
I am a big fan of the library and I realized several projects
using it.
I am now trying to write a proxy using libmicrohttpd and curl;
at the moment, I simply do a curl_easy_perform to the real
service when I receive a request. I am using an
INTERNAL_SELECT mode to manage requests with a pool of
threads. I prefer to avoid THREAD_PER_CONNECTION to avoid
explosions in terms of thread number and EXTERNAL_SELECT cause
the limitation to a single thread.
Whenever I receive a call I have to wait the real server
response in order to be able to give a response to the client,
and when I have a lot of concurrent requests the pool is
exhausted and connections are queued; so I am planning to use
the curl multi interface to avoid the blocking on curl request
and use an async io system to enhance parallelism.
Is it possible to delay my response (from libmirohttpd) to the
time when I receive the response from curl without completely
block one thread of the pool?
Am I completely blind and is there another and more simple
system to do what I am trying to do?
Well, the 'simple' answer is the one I believe you already have
given: use 'EXTERNAL_SELECT' mode, which allows you to return 0
from the content reader callback to say "no data yet" and to
integrate MHD with a curl-multi event loop. This is what we do in
an HTTP(S) socks proxy in GNUnet, so I can safely say that it does
work. THREAD_PER_CONNECTION and curl_easy should also work, and
--- unless you are on an embedded system --- I'm not sure the
number of threads is a major issue (especially as with any
select-based approach, you're limited to FD_SETSIZE (typically
1024) sockets and thus <512 connections). So if we set the same
concurrency limit for the thread-based approach, we need 512
threads, which is not all that scary given modern memory sizes on
servers/desktops.
So anyway, as for scaling, I would suspect that the
EXTERNAL_SELECT setup will first be limited by your FD_SETSIZE
(unless you raised it) and not by the CPU being limited to a
single thread. So from an engineering point of view, I would
urge you try that and only optimize further if you have
established a need to do so via benchmarks.
Now, suppose you're on an embedded multi-core system with
low-power cores, and your benchmarks tell you that multi-threading
is needed; or maybe you do something else that is really expensive
with the data or have some larger FD_SETSIZE. What to do in those
cases?
Right now my answer would be to start multiple MHD-daemons, using
the *same* listen socket. If you open the listen socket yourself
and pass it to MHD, you can have one listen socket be shared among
multiple MHD daemon handles. So basically, you open one listen
socket, start a bunch of threads, and have each of them run an
external select loop around a per-thread MHD-daemon handle. This
would essentially create an 'external select thread pool' mode.
You will need to put the 'listen' socket into non-blocking mode
(to avoid blocking one of the threads on a race for the socket ---
note that the threads racing for the socket is _intended_ in this
design, and the same race is used in the "internal select'-based
thread-pool of MHD).
While I have not tried this 'external-select thread pool', I'm
reasonably confident that it would work. Still, if you do get it
to work, I'm sure the list would appreciate an update ;-).
My 2 cents
Happy hacking!
Christian
