Re: [libmicrohttpd] [PATCH]: try_grow_read_buffer() failed
On 17.03.2017 4:31, Vitaliy T wrote:
> The question is not closed.
>
> Applied to recent commits from
> a10c64c79337431924830db9e82f6c1c70994942 and early ones.
>
> I have started deeper investigation of the problem. It leads to the
> MHD_pool_reallocate() function from src/microhttpd/memorypool.c
>
> I have found that the next check is interpreted by MHD differently,
> depended on who or what is calling the function:
>
> 273 if ((pool->pos + asize >= pool->pos) &&
> 274 (pool->pos + asize <= pool->end))
> 275 {
>
> 1. Sometimes the same check is ignored by MHD.
> 2. And if MHD_pool_reallocate() was called by try_grow_read_buffer()
> the INTERNAL_ERROR will be send.
>
> For example, I got next values:
> pool: 0x0x8001dd50
> pool->pos: 66565
> asize: 67376
> pool->end: 130976
>
> First time this "error" was ingored, and later the same pool address
> became "bad".
>
> Can't say which function is exactly ignores that MHD_pool_reallocate()
> was "failed". All I know that it is not fatal for the server. Am I
> right? :)
No, it's not correct.
It's not fatal as long as buffer for incoming data still have some space
(even if failed to grow) or MHD do not need to read data from remote any
more.
It's fatal if MHD need to read remote data and no space left in buffer.
Pretty simple.
> So, either we need to fix MHD_pool_reallocate(), either to fix
> try_grow_read_buffer() or even fix the problem differently.
>
Will answer to your next email.
--
Best Wishes,
Evgeny Grin
Re: [libmicrohttpd] [PATCH]: MHD_connection_update_event_loop_info sends INTERNAL_ERROR for suspended connections
On 17.03.2017 5:42, Vitaliy T wrote: > Inside of MHD_connection_handle_read () src/microhttpd/connection.c is > missing check against of the return value for the > try_grow_read_buffer() call. It's just ignored. And now without a fear > I may say that the INTERNAL_ERROR will be send for a connection that > was resumed and then suspended again (probably few times). > > Here is the log of the server: > > +0300 2017-03-17 05:31:34.254320 * Connection from 127.0.0.1 port > 47824 [33/49728] > +0300 2017-03-17 05:31:34.261671 * Connection from 127.0.0.1 port 48848 > +0300 2017-03-17 05:31:34.272435 ! 127.0.0.1 port 47824: suspend > +0300 2017-03-17 05:31:34.279515 ! 127.0.0.1 port 46800: uploaded `big2' > +0300 2017-03-17 05:31:34.280380 ! 127.0.0.1 port 47824: resumed > +0300 2017-03-17 05:31:34.280917 ! 127.0.0.1 port 48848: uploading `big3' > +0300 2017-03-17 05:31:34.281012 * Connection 127.0.0.1 port 46800 closed: OK > does not fit #2: > pool: 0x0x14206d50 > pool->pos: 66565 > asize: 67376 > pool->end: 130976 > FAIL: try_grow_read_buffer > FAIL: MHD_connection_handle_read > 127.0.0.1 port 47824 > +0300 2017-03-17 05:31:34.281732 ! 127.0.0.1 port 47824: suspend > does not fit #2: > pool: 0x0x14206d50 > pool->pos: 66565 > asize: 67376 > pool->end: 130976 > FAIL: try_grow_read_buffer > ERROR: 127.0.0.1 port 47824 > Error processing request (HTTP response code is 500 > (`Internal server error tle>Some programmer needs to study the manual more > carefully.')). Closi > ng connection. > +0300 2017-03-17 05:31:34.378595 ! 127.0.0.1 port 48848: uploaded `big3' > +0300 2017-03-17 05:31:34.384555 ! 127.0.0.1 port 47824: resumed > +0300 2017-03-17 05:31:34.384716 * Connection 127.0.0.1 port 48848 closed: OK > +0300 2017-03-17 05:31:34.384857 * Connection 127.0.0.1 port 47824 closed: OK > > As we can see the connection 127.0.0.1 port 47824 was suspended, > resumed and then again suspended. > > The server was run in epoll mode with 1 thread. > DEFAULT_HTTPD_CONNECTION_MEMORY_LIMIT = 128k > DEFAULT_HTTPD_CONNECTION_MEMORY_INCREMENT = 4k > Both suspend & resume operations are called from DH. > > Hope, it will help. To get the full picture, could you also monitor value pointed by 'upload_data_size' before and after your MHD_AccessHandlerCallback is called? Please note that even if you suspend connection, you must process at least some data (preferably - all data) and decrement value of 'upload_data_size'. -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] [PATCH]: MHD_connection_update_event_loop_info sends INTERNAL_ERROR for suspended connections
On 17.03.2017 10:03, Evgeny Grin wrote: > On 17.03.2017 5:42, Vitaliy T wrote: >> As we can see the connection 127.0.0.1 port 47824 was suspended, >> resumed and then again suspended. >> >> The server was run in epoll mode with 1 thread. >> DEFAULT_HTTPD_CONNECTION_MEMORY_LIMIT = 128k >> DEFAULT_HTTPD_CONNECTION_MEMORY_INCREMENT = 4k >> Both suspend & resume operations are called from DH. >> >> Hope, it will help. > > To get the full picture, could you also monitor value pointed by > 'upload_data_size' before and after your MHD_AccessHandlerCallback is > called? > > Please note that even if you suspend connection, you must process at > least some data (preferably - all data) and decrement value of > 'upload_data_size'. Processing logic was updated: while connection is suspended overflown buffer will not cause connection to fail (as it will not generate busy-waiting for suspended connection), but once connection is resumed - connection will fail if application will not read data from buffer in callback. -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] [PATCH]: MHD_connection_update_event_loop_info sends INTERNAL_ERROR for suspended connections
On 17.03.2017 15:10, Vitaliy T wrote: > Hello, > > On 17 March 2017 at 10:03, Evgeny Grin wrote: >> Please note that even if you suspend connection, you must process at >> least some data (preferably - all data) and decrement value of >> 'upload_data_size'. > > Yes, this was my mistake. Before suspending a connection I have to > forgot about decrementing upload_data_size. > > I have questions: > > 1) Due the async nature of the MHD, how can I keep a connection in > resume state _without_ reading data in DH? > > I mean, I can't say in which moment I can resume the connection with > guarantee to read the data. I have to create some kind of queue with > tracking which connection is could read data and which ones could not. > And more importantly that I must guarantee that if I resumed a > connection and it must read full data. > > Things getting complicated. > > Current DH logic looks like that: > > 1. I have to initialize internal structure per request, then it will > be stored to con_cls and I return MHD_YES. > 2. On the second entry into DH, I do process request: either upload a > file, either suspend a connection. > 3. When uploading the file has been finished I am resuming the next > connection (if it is). > 4. Even after resuming the connection there is a chance that a new > connection will be proceeded before the "resumed" connection. > > I can track if the connection was resumed and give it a higher > priority. But if so I have to suspend _all_ new connections before > doing something useful. > I will think about it, but, IMHO, it is wrong that I can't > suspend/resume connections when I want without reading buffers. > > 2) Is it possible to do so that the chain > suspend-resume-suspend-resume will not affect read buffers? Until I > will "say" to MHD that I really read a data and we can continue in the > normal mode. Because, by you words I can't resume a connection if I > don't know will be it suspended again or not. I can resume the > connection only if I give guarantees that it will be proceeded as > usual (read all data). Looks like you want to blindly resume connection and check whether to process it only in callback handler. I suggest to move check for "readiness to be processed" to some separate function. Then instead of blindly resuming connection and immediately suspending it again - resume connection only when you will be sure that it's ready to be processed. You don't need to make this check only inside callback function. If you suspend connection and resume it later - connection conditions are not changed, you will not get any new information in callback. So you can store required information in your internal structures and check it *before* resuming, not *after* resuming. Resume/suspend repeated cycles without processing of information - is ineffective and wastes a lot of resources. -- Best Wishes, Evgeny Grin PS If you will check master on official git, you may find support of your ineffective way of processing.
Re: [libmicrohttpd] [PATCH]: MHD_connection_update_event_loop_info sends INTERNAL_ERROR for suspended connections
On 17.03.2017 15:59, Vitaliy T wrote:
> On 17 March 2017 at 15:43, Evgeny Grin wrote:
>> You don't need to make this check only inside callback function. If you
>> suspend connection and resume it later - connection conditions are not
>> changed, you will not get any new information in callback. So you can
>> store required information in your internal structures and check it
>> *before* resuming, not *after* resuming.
>
> Seems to be I got the point. Thanks for the advice.
>
>> Resume/suspend repeated cycles without processing of information - is
>> ineffective and wastes a lot of resources.
>
> There is a little problem. If I would proceed the data and right after
> that I suspend the connection, then I must keep this (read) data
> somewhere, because a new data already is arrived. This leads to next
> problem: more suspended connections means that the app. will keep even
> more data (one proceeded, one in MHD buffers). IMHO, 1) the
> application becomes overcomplicated; 2) there is even more wasting of
> resources.
You are storing connection handler somewhere anyway.
Declare:
struct my_conn_info
{
const MHD_connection * conn;
enum req_method {req_UNK, req_GET, req_POST, req_HEAD, req_OTHER} method;
const char * url;
unsigned int timeout;
int other_data1;
char other_data2;
} my_connection
and use it instead of MHD connection handler for you internal functions.
Pretty simple and take only a few more bytes per connection.
It's really much lighter comparing to calling heavy systems call like
send/recv/select.
And you can use it with callbacks as well (as con_cls). Anyway you need
to store connection specific data while you incrementally processing
incoming data.
>
> Either, I chose the wrong tool :)
This is a free software. You always have a choice. :)
>> PS If you will check master on official git, you may find support of
>> your ineffective way of processing.
>
> Yes, I have tested the patch. As I suggested early, it's works
> flawlessly. It is harmless, even if you say that is "wrong way" :)
> Ok, lets finish at this point, I must finish my project. Later, I will
> give you a feedback.
>
Good to know that it's working.
Send more reports! And thanks for that report.
--
Best Wishes,
Evgeny Grin
Re: [libmicrohttpd] misc patches to current git master
Patch for docs applied.
libintl is not always required. For example, on GNU/Linux it usually
provided by glibc.
--
Best Wishes,
Evgeny Grin
On 22.03.2017 20:47, Vitaliy T wrote:
> 1. libmicrohttpd.texi:1649: @node seen before @end deftypefun
>
> diff --git a/doc/libmicrohttpd.texi b/doc/libmicrohttpd.texi
> index 1d6438d9..b6e933f1 100644
> --- a/doc/libmicrohttpd.texi
> +++ b/doc/libmicrohttpd.texi
> @@ -1616,6 +1616,7 @@ the right options for this call.
>
> @deftypefun int MHD_get_fdset2 (struct MHD_Daemon *daemon, fd_set *
> read_fd_set, fd_set * write_fd_set, fd_set * except_fd_set, int
> *max_fd, unsigned int fd_setsize)
> Like @code{MHD_get_fdset()}, except that you can manually specify the
> value of FD_SETSIZE used by your application.
> +@end deftypefun
>
>
> @deftypefun int MHD_get_timeout (struct MHD_Daemon *daemon, unsigned
> long long *timeout)
>
>
> 2. src/examples/msgs_i18n.c should be linked against libintl
>
> diff --git a/src/examples/msgs_i18n.c b/src/examples/msgs_i18n.c
> index 2d8eb566..b6b98909 100755
> --- a/src/examples/msgs_i18n.c
> +++ b/src/examples/msgs_i18n.c
> @@ -32,7 +32,7 @@
> * mkdir -p src/examples/locale/pt_BR/LC_MESSAGES
> * mv libmicrohttpd.mo libmicrohttpd.po src/examples/locale/pt_BR/LC_MESSAGES
> * cd src/examples/
> - * gcc -o msgs_i18n msgs_i18n.c -lmicrohttpd
> + * gcc -o msgs_i18n msgs_i18n.c -lmicrohttpd -lintl
> * export LANGUAGE=pt_BR
> * ./msgs_i18n
> * # it may print: Opção inválida 4196490! (Você terminou a lista com
> MHD_OPTION_END?)
>
>
> Thanks!
>
Re: [libmicrohttpd] MHD_quiesce_daemon() question
On 26.03.2017 8:33, silvioprog wrote:
> I found the following related message:
>
> https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
>
> I've used a similar logic, but with item X below, because I need to wait
> the client processing:
>
> 1) MHD_quiesce_daemon()
> *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
> 2) MHD_stop_daemon()
> 3) close()
>
> Real implementation:
>
> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) {
> MHD_socket fd;
> if (srv && srv->listening) {
> fd = MHD_quiesce_daemon(srv->mhd);
> if (!force)
> while (MHD_get_daemon_info(srv->mhd,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0)
> usleep(1000 * 500); //TODO: allow to use external callback
> MHD_stop_daemon(srv->mhd);
> if (fd != MHD_INVALID_SOCKET)
> close(fd);
> srv->listening = false;
> return true;
> }
> return false;
> }
>
>
> Calling it with bf_httpsrv_shutdown(srv, false) the server stops waiting
> for clients processing.
>
> So, what do you think about the logic above? Should it be improved?!
>
> Thanks in advance for any suggestions!
If you don't check returned value from MHD_quiesce_daemon(), you may
later found that you didn't quiesced daemon, so move check right after
MHD_quiesce_daemon() and added error handling.
If you didn't set connection timeout, connection may live indefinitely.
Moreover, even with connection timeout, clients may continue processing
on same HTTP 1.1 connections with new requests indefinitely.
Furthermore, even with HTTP 1.0 and connection timeout hypothetical
client may read answer very slow with results in very long unpredictable
closure of connection.
So: yes, you code will work but may result in very long (hours, for
example) or even indefinitely long daemon shutdown.
--
Best Wishes,
Evgeny Grin
Re: [libmicrohttpd] feature request: give a possibility to re-assign callbacks
On 27.03.2017 17:29, Vitaliy T wrote:
> 1. Subj.
Which callbacks would you like to reassign?
> 2. Why is this should be useful?
>
> 2.1. When you're actively using suspend/resume functionality and later
> trying to shutdown the MHD daemon, then last one, by default, uses
> default callbacks (which were given at the startup) for _resumed_
> connections (as docs says).
>
> By good intentions, such callbacks must be transformed into stubs,
> that is, { return MHD_NO; } to complete the task properly. Currently,
> it is possible to wait, I may say, weeks if there are thousands of
> active requests.
What prevents to return "MHD_NO" early from the same callback when required?
> 2.2. It gives a possibility to write, I have no good words for this
> except next, the polymorph-like servers. For instance, it may be funny
> (and may be be a serious work) to load an external shared library and
> then re-assign callbacks to the functions which the library contains.
>
> Either, are there any alternatives about how to do tasks above I don't know?
Simple (pseudo code):
int callback(void *user_data)
{
struct my_data * const info = (struct my_data *) user_data;
switch(info->callback_type)
{
case use_callback_a: return call_callback_a(user_data);
case use_callback_b: return call_callback_b(user_data);
default:
return info->current_callback(user_data);
}
}
Will this work for you?
--
Wishes,
Evgeny
Re: [libmicrohttpd] why does not libmicrohttpd support for resume from break-point？
Hi! Could you clarify, what do you mean by "resume from break-point"? -- Best Wishes, Evgeny Grin On 27.03.2017 5:21, 星**光 wrote: > why does not libmicrohttpd support for resume from break-point？ how > do？ Please give some suggestions？
Re: [libmicrohttpd] MHD_quiesce_daemon() question
Fixed version of you code:
bool bf_httpsrv_shutdown(struct bf_httpsrv *srv) { MHD_socket fd;
uint8_t shutdown_attempts; /* Max 256. */ if (srv && srv->listening) {
fd = MHD_quiesce_daemon(srv->mhd); if (fd != MHD_INVALID_SOCKET) {
srv->listening = false; close(fd); if (! srv->forced_shutdown) {
shutdown_attempts = 1; while (MHD_get_daemon_info(srv->mhd,
MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0) { if
(shutdown_attempts >= srv->shutdown_attempts) { /* Default
srv->shutdown_attempts is 10. */ _BF_LOG("Forced server shutdown.\n");
break; } shutdown_attempts++; sleep(1); } } } else _BF_LOG("Server
quiesce failed.\n"); MHD_stop_daemon(srv->mhd); return true; } return
false; }
--
Best Wishes,
Evgeny Grin
On 28.03.2017 4:46, silvioprog wrote:
> Thanks for replying Evgeny, you and Christian always saving my day. :-)
> I read and re-read your and Christian answer, and finally I found a
> possible way to never lock the server at shutdown: adding an option to
> exit the server after configurable attempts. It can solve the timeout
> problem too, because I can't ensure that the programmer have
> configured it (anyway my library will assume 15 seconds by default).
> So, the code below applies your tips and implements the attempts approach:
> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv) { MHD_socket fd;
> uint8_t shutdown_attempts; /* Max 256. */ if (srv && srv->listening) {
> fd = MHD_quiesce_daemon(srv->mhd); if (fd != MHD_INVALID_SOCKET)
> close(fd); else _BF_LOG("Server quiesce failed.\n"); if
> (srv->forced_shutdown) { shutdown_attempts = 1; while
> (MHD_get_daemon_info(srv->mhd,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0) { if
> (shutdown_attempts >= srv->shutdown_attempts) { /* Default
> srv->shutdown_attempts is 10. */ _BF_LOG("Forced server shutdown.\n");
> exit(EINTR); } shutdown_attempts++; sleep(1); } }
> MHD_stop_daemon(srv->mhd); srv->listening = false; return true; }
> return false; }
> Unfortunately if exit was called the MHD_stop_daemon() will be not
> called, probably raising some memory leak, but current I have no idea
> how it could be solved.
> Feel totally free to point improvements about this new version. :-)
> On Sun, Mar 26, 2017 at 5:36 PM, Evgeny Grin <mailto:k...@yandex.ru>> wrote:
>
> On 26.03.2017 8:33, silvioprog wrote: > I found the following
> related message: > >
> https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
> <https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html>
> > > I've used a similar logic, but with item X below, because I
> need to wait > the client processing: > > 1) MHD_quiesce_daemon()
> > *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
> > 2) MHD_stop_daemon() > 3) close() > > Real implementation: > >
> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) { >
>MHD_socket fd; > if (srv && srv->listening) { > fd
> = MHD_quiesce_daemon(srv->mhd); > if (!force) >
> while (MHD_get_daemon_info(srv->mhd,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0) >
>usleep(1000 * 500); //TODO: allow to use external
> callback > MHD_stop_daemon(srv->mhd); > if (fd !=
> MHD_INVALID_SOCKET) > close(fd); >
> srv->listening = false; > return true; > } >
> return false; > } > > > Calling it with bf_httpsrv_shutdown(srv,
> false) the server stops waiting > for clients processing. > > So,
> what do you think about the logic above? Should it be improved?! >
> > Thanks in advance for any suggestions! If you don't check
> returned value from MHD_quiesce_daemon(), you may later found that
> you didn't quiesced daemon, so move check right after
> MHD_quiesce_daemon() and added error handling. If you didn't set
> connection timeout, connection may live indefinitely. Moreover,
> even with connection timeout, clients may continue processing on
> same HTTP 1.1 connections with new requests indefinitely.
> Furthermore, even with HTTP 1.0 and connection timeout
> hypothetical client may read answer very slow with results in very
> long unpredictable closure of connection. So: yes, you code will
> work but may result in very long (hours, for example) or even
> indefinitely long daemon shutdown. -- Best Wishes, Evgeny Grin
>
>
> --
> Silvio Clécio
Re: [libmicrohttpd] Using MHD_UpgradeResponseHandle without including internal.h
Hi John, MHD_UpgradeResponseHandle is intentionally opaque. It's not designed for direct usage of internal members. If you define it locally, you'll almost likely to break compatibility with future version. MHD uses a lot of opaque structures which are internally changed from version to version. But as long as application uses only public official API, those changes don't break compatibility. Usually MHD has specific API to get information about opaque handle. What do you want to get from internal members? -- Best Wishes, Evgeny Grin On 31.03.2017 0:58, John Duncan wrote: > I noticed that microhttpd.h only contains only forward defintions of the > MHD_UpgradeResponseHandle structure. The file internal.h contains the > actual structure definition with all members. > > When I try to include the internal.h file directly, I get tons of > compilation errors leading me to believe that I'm not supposed to be > including it. > > My question is, when building my own applications, how am I supposed to > access member portions of the MHD_UpgradeResponseHandle structure > without including internal.h. Am I expected to define the structure > within my own project headers and use the provided > MHD_UpgradeResponseHandle pointer as somewhat of an opaque type? > > I don't mind doing this, I just want to make sure this is the correct > approach. > > Thanks; > ~JD
Re: [libmicrohttpd] MHD_create_post_processor: recommended buffer size for large posts
Hi Silvio, Sidenote: SVN is not updated any more, latest version is on git: https://gnunet.org/git/libmicrohttpd.git/tree/doc/examples/largepost.c "512" is not recommended size, it's just suitable size for example. Optimal value depends on OS, architecture, amount of available RAM. You should experiment to find out what is optimal for you. -- Best Wishes, Evgeny Grin On 31.03.2017 6:17, silvioprog wrote: > Hello dudes, > > Looking at MHD examples, it seems the recommended buffer size to specify > on MHD_create_post_processor() for large uploads is 512: > > https://gnunet.org/svn/libmicrohttpd/doc/examples/largepost.c > <https://gnunet.org/svn/libmicrohttpd/doc/examples/largepost.c> > > However, what means large for you? About 100 MB, 1 GB, 10 GB or greater?! > > I have a structure that I need to support ~ 5 GB, so I'm planning to use > 1024, because I want fast transfer using less memory and CPU possible. > In that case, is 1024 an ideal size? > > Thank you! > > P.S.: we should specify the size (in MB or GB) that we consider large on > the example `largepost.c`. > > -- > Silvio Clécio
Re: [libmicrohttpd] integration with my websockets
Hi José, On 11.04.2017 19:55, José Bollo wrote: > On Tue, 11 Apr 2017 18:29:04 +0200 > José Bollo wrote: > >> I just found time now to integrate the work you did last summer >> for upgrading connections. Seems to work out of the box, but... >> >> First a remark: I was previously trying to allocate ressources for the >> web socket before to reply to the upgrading connection. That is not >> more possible except if I create the websocket without socket. That is >> a detail but a not pleasant detail. Could you explain a bit more do you want to implement and why it is not possible? >> Third, do you plan to provide a new version soon? Yes, watch this list. :) -- Best Wishes, Evgeny Grin
[libmicrohttpd] GNU libmicrohttpd 0.9.53 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.53. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). This release include a large number of fixes and new features as well. Most noticeable changes: * HTTP "Upgrade" is deeply reworked and tested on all supported platforms * Added MHD_USE_AUTO and MHD_USE_AUTO_INTERNAL_THREAD flags for automatic selection of polling function depending on platform capabilities and requested mode - especially useful for multiplatform application * Renamed some of MHD flags, now they are self-explainable and more obvious, like MHD_USE_INTERNAL_POLLING_THREAD instead of MHD_USE_SELECT_INTERNALLY; old names are still supported * Darwin, *BSD and Solaris support improved Other changes: * Added 'configure' parameter for building without "Upgrade" support * Added support for suspending connection in thread-per-connection mode * Added HTTP "Upgrade" simple example * Added special handling of "fast" connections with small request and response - they are fully processed in one turn * Added detection and rejection of invalid combinations of daemon flags * Added detection of invalid combination of daemon option with flags * Added support of getting connection timeout value by MHD_get_connection_info() * Added ability to get actual daemon flags by MHD_get_daemon_info() * Added usage of "calloc()" where supported * Added support for more efficient setting of thread names on some platforms (Solaris, NetBSD etc.) * Added support for SO_EXCLBIND on Solaris * Added usage of SO_REUSEADDR (in addition to SO_REUSEPORT) for MHD_OPTION_LISTENING_ADDRESS_REUSE options * Reworked handling connection in epoll mode so all active connections are processed exactly one time per turn * Changed monitoring for closure: now connection are read only when more incoming data is required, all connection are monitored for OOB data (which treated as error) * except_fd_set now is not optional now for MHD_get_fdset(), MHD_get_fdset2() and MHD_run_from_select() * Improved connection processing in epoll mode: now connection can process both read and write each turn * Updated HTTP response codes to latest: now MHD return code MHD_HTTP_REQUEST_HEADER_FIELDS_TOO_LARGE (431) instead of old MHD_HTTP_REQUEST_ENTITY_TOO_LARGE (413) for very long header. * Updated and added all missing standard and additional HTTP methods * Updated and added all missing standard HTTP headers names (and headers categories) * Added more tests to test-suite * Added real run-test detection instead of guessing of triggering select() by shutdown of listening socked * Added support for chunk size > 16 MB (up to 2^64-1), ignored chunk extensions instead of triggering an error * Converted many run-time 'strlen()' to compile-time calculations * Many data races fixes and other thread-safety improvements * Some fixes in suspend-resume data races * Eliminated calling application on suspended connection (which before could happen under certain circumstances) * Fixed unintentional usage of SO_REUSEADDR on W32 * Fixed MHD_OPTION_LISTENING_ADDRESS_REUSE on Linux kernels before 3.9 * Fixed sending responses backed by files not supported by sendfile() * Fixed thread safety for responses backed by file FD * Fixed handling of more than 128 ongoing connection and 128 incoming connections in epoll mode * Resolved busy-waiting under certain conditions in HTTPS mode * Fixed wrong value returned by MHD_get_timeout() * Fixed reset of timeout on resumed connections. * Fixed never-expired timeouts on HTTPS connections. * Fixed fail if connection suspended and resumed by application several times without processing incoming data * Many other fixes, improvements and optimisations. You can download GNU libmicrohttpd from * ftp://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at http://www.gnu.org/software/libmicrohttpd/. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] GNU libmicrohttpd 0.9.53 released
Thanks for report! This could happen with ancient Linux kernels (which is not supported already). Other platforms are unaffected. Fixed by 01df8861c19a203cea9ef26570893c2b9c0e48b2. -- Best Wishes, Evgeny Grin On 12.04.2017 16:02, Anthony G. Basile wrote: > On 4/11/17 5:53 PM, Evgeny Grin wrote: >> I'm glad to announce the release of GNU libmicrohttpd 0.9.53. >> >> * Reworked handling connection in epoll mode so all active connections >> are processed exactly one time per turn >> > libmicrohttpd 0.9.53 fails to build if EPOLL_SUPPORT is not available. > At line 2255 of src/microhttpd/daemon.c there a reference to > connection->epoll_state which is not shielded by #ifdef EPOLL_SUPPORT > and causes the following build error: > > make[3]: Entering directory > '/var/tmp/portage/net-libs/libmicrohttpd-0.9.53/work/libmicrohttpd-0.9.53/src/microhttpd' > /bin/sh ../../libtool --tag=CC --mode=compile x86_64-pc-linux-gnu-gcc > -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I../../src/include > -I../../src/microhttpd -I/usr/include/p11-kit-1 -DBUILDING_MHD_LIB=1 > -fvisibility=hidden -pthread -I/usr/include/p11-kit-1 -O2 -pipe > -fno-strict-aliasing -c -o libmicrohttpd_la-daemon.lo `test -f > 'daemon.c' || echo './'`daemon.c > libtool: compile: x86_64-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H > -I. -I../.. -I../../src/include -I../../src/microhttpd > -I/usr/include/p11-kit-1 -DBUILDING_MHD_LIB=1 -fvisibility=hidden > -pthread -I/usr/include/p11-kit-1 -O2 -pipe -fno-strict-aliasing -c > daemon.c -fPIC -DPIC -o .libs/libmicrohttpd_la-daemon.o > daemon.c: In function ‘send_param_adapter’: > daemon.c:2255:23: error: ‘struct MHD_Connection’ has no member named > ‘epoll_state’ > connection->epoll_state &= ~MHD_EPOLL_STATE_WRITE_READY; >^ > make[3]: *** [Makefile:1056: libmicrohttpd_la-daemon.lo] Error 1 > make[3]: Leaving directory > '/var/tmp/portage/net-libs/libmicrohttpd-0.9.53/work/libmicrohttpd-0.9.53/src/microhttpd' > make[2]: *** [Makefile:414: all-recursive] Error 1 > make[2]: Leaving directory > '/var/tmp/portage/net-libs/libmicrohttpd-0.9.53/work/libmicrohttpd-0.9.53/src' > make[1]: *** [Makefile:545: all-recursive] Error 1 > make[1]: Leaving directory > '/var/tmp/portage/net-libs/libmicrohttpd-0.9.53/work/libmicrohttpd-0.9.53' > make: *** [Makefile:450: all] Error 2 >
Re: [libmicrohttpd] 0.9.53 breaks MHD_USE_POLL?
Hi Lorenzo, Thanks for report! We introduced more checks in release 0.9.53 to make MHD more foolproof. Now MHD reject invalid combinations of flags instead of starting with undefined behaviour. Applications are advised to set explicitly flag MHD_USE_INTERNAL_POLLING_THREAD when using MHD_USE_THREAD_PER_CONNECTION. Currently MHD_USE_THREAD_PER_CONNECTION flag also enforce implicit setting of MHD_USE_INTERNAL_POLLING_THREAD. I fixed check to improve backward compatibility with MHD_USE_THREAD_PER_CONNECTION and MHD_USE_POLL without MHD_USE_INTERNAL_POLLING_THREAD. If application will be used with new version of MHD, I strongly advise to use MHD_USE_AUTO instead of MHD_USE_POLL or MHD_USE_EPOLL (unless you have specific reason to use poll() or epoll). With MHD_USE_AUTO you will get best performance on any platform. If you know any rejected valid combination of flags - let me know. We'll probably make a bugfix release soon. -- Best Wishes, Evgeny Grin On 24.04.2017 16:38, Lorenzo Miniero wrote: > Hi all, > > I'm using MHD in my server and it works great. Recently I've been told, > though, that some people couldn't get it to work. Somebody pointed out > that it only seems to happen on recent versions, so I tried 0.9.53 (the > latest on my Fedora) and that seems indeed to be the case. > > Specifically, by fiddling a bit with the code it looks like the cause of > the failure is the MHD_USE_POLL. Check this sample test code as an example: > > https://gist.github.com/lminiero/5bd6795f2534b9c523c90603ec23816a > > The code as it is results in a failure for me (daemon=NULL). If I remove > the MHD_USE_POLL, it works instead. The same code was definitely working > on the previous version of MHD my Fedora shipped (0.9.46), but not sure > if 0.9.53 itself is the cause or any version in between. > > At first I thought it could be a problem with flag names, as I read in > the Changelog that some changed, but even in that case, the Changelog > also says old names are still supported for backwards compatibilty. > Besides, apparently MHD_USE_POLL is not one of the affected ones. > > I also considered the possibility that it may be an issue with how the > Fedora 25 rpm was built, as MHD_USE_POLL is supposed to fail if poll() > is not available (#ifndef HAVE_POLL) and so it might simply be a matter > of a broken check there, but I'm not convinced that's it, as the same > issue was reported on different distros (e.g., Arch Linux) and systems > (Mac OS). So if it really is a broken check, it seems to be broken in > more than one place. > > Do you have any insight on what may be the problem here? > > Thanks! > Lorenzo signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] 0.9.53 breaks MHD_USE_POLL?
Hi Lorenzo,
On 26.04.2017 11:56, Lorenzo Miniero wrote:
> 2017-04-25 18:36 GMT+02:00 Evgeny Grin <mailto:k...@yandex.ru>>:
>
> Hi Lorenzo,
>
> Thanks for report!
> We introduced more checks in release 0.9.53 to make MHD more
> foolproof.
> Now MHD reject invalid combinations of flags instead of starting with
> undefined behaviour.
> Applications are advised to set explicitly flag
> MHD_USE_INTERNAL_POLLING_THREAD when using
> MHD_USE_THREAD_PER_CONNECTION.
> Currently MHD_USE_THREAD_PER_CONNECTION flag also enforce implicit
> setting of MHD_USE_INTERNAL_POLLING_THREAD.
> I fixed check to improve backward compatibility with
> MHD_USE_THREAD_PER_CONNECTION and MHD_USE_POLL without
> MHD_USE_INTERNAL_POLLING_THREAD.
>
> If application will be used with new version of MHD, I strongly advise
> to use MHD_USE_AUTO instead of MHD_USE_POLL or MHD_USE_EPOLL
> (unless you
> have specific reason to use poll() or epoll). With MHD_USE_AUTO
> you will
> get best performance on any platform.
>
> If you know any rejected valid combination of flags - let me know.
> We'll
> probably make a bugfix release soon.
>
>
> thanks for the quick response!
>
> MHD_USE_AUTO does indeed seem to be the easiest fix, here, although it
> looks like a recent addition (last November?) and so might not be
> available in deployments that still ship an older version of the
> library. Adding MHD_USE_INTERNAL_POLLING_THREAD where MHD_USE_POLL is
> specified might do the trick too, but that also was only recently
> added. It is available on both my Fedora and on some 16.04 Ubuntu
> boxes we have (which are on 0.9.33), but not sure if the same can be
> said for all the people using our code at the moment, so I'll have to
> fix the code so that it can "adapt".
The old (and confusing) name of MHD_USE_INTERNAL_POLLING_THREAD was
MHD_USE_SELECT_INTERNALLY. It *should *be used even with MHD_USE_POLL if
you want to run MHD internal threads.
So, pretty simple fix is to use MHD_USE_POLL | MHD_USE_SELECT_INTERNALLY
| MHD_USE_THREAD_PER_CONNECTION combination.
It is compilable with old MHD versions as well as with new versions.
You can simplify even more and use predefined combination:
MHD_USE_POLL_INTERNALLY | MHD_USE_THREAD_PER_CONNECTION (new name is
MHD_USE_POLL_INTERNAL_THREAD | MHD_USE_THREAD_PER_CONNECTION).
>
> Using an #ifdef to see if those flags exist and react accordingly
> wouldn't work, as they're an enum and not defines. I guess that one
> way to do that without involving configure checks could be looking
> at MHD_VERSION: if it's higher than X, then we can use MHD_USE_AUTO,
> otherwise we do what worked for us so far. Not sure what X is, though:
> do you know the programmatic version number of when the feature was added?
>
You found answer already, as I see.
I could suggest you to use following macro to make code compilable with
older MHD version on all platforms:
#if MHD_VERSION < 0x00095208
# define MHD_USE_AUTO 0
#endif /* MHD_VERSION < 0x00095208 */
daemon = MHD_start_daemon (MHD_USE_AUTO | MHD_USE_THREAD_PER_CONNECTION
| MHD_USE_SELECT_INTERNALLY,
port, NULL, NULL, &handler, &h_cls, MHD_OPTION_END);
In this case older MHD versions will use select() and never versions
will use best available polling functions (select(), poll(), epoll).
Alternatively, if your application is used only on platforms where
poll() is available:
#if MHD_VERSION < 0x00095208
# define MHD_USE_AUTO MHD_USE_POLL
#endif /* MHD_VERSION < 0x00095208 */
daemon = MHD_start_daemon (MHD_USE_AUTO | MHD_USE_THREAD_PER_CONNECTION
| MHD_USE_SELECT_INTERNALLY,
port, NULL, NULL, &handler, &h_cls, MHD_OPTION_END);
In this case older MHD versions will use poll() and never versions will
use best available polling functions (select(), poll(), epoll).
--
Best Wishes,
Evgeny Grin
Re: [libmicrohttpd] 0.9.53 breaks MHD_USE_POLL?
On 26.04.2017 13:54, Lorenzo Miniero wrote: > Thanks! So, just to clarify, when doing MHD_USE_THREAD_PER_CONNECTION, > even with MHD_USE_AUTO you still need to specify > MHD_USE_SELECT_INTERNALLY (or the newer > MHD_USE_INTERNAL_POLLING_THREAD), or does MHD_USE_AUTO also > automatically sets the correct threading model? Asking because the way > I was addressing this in a pull request on my project was simply to do a: > > #if MHD_VERSION >= 0x00095208 > MHD_USE_THREAD_PER_CONNECTION | MHD_USE_AUTO | MHD_USE_DUAL_STACK, > #else > MHD_USE_THREAD_PER_CONNECTION | MHD_USE_POLL | MHD_USE_DUAL_STACK, > #endif > > which seems to be working as expected on 0.9.53. > It better to specify MHD_USE_SELECT_INTERNALLY explicitly. If it is not specified, MHD_USE_THREAD_PER_CONNECTION will use MHD_USE_SELECT_INTERNALLY implicitly. Version 0.9.53 fails with poll and without MHD_USE_SELECT_INTERNALLY because check of invalid combinations of flags is done *before* adding automatically MHD_USE_SELECT_INTERNALLY. It was fixed in git master. I recommend you to always use MHD_USE_SELECT_INTERNALLY when you specify MHD_USE_THREAD_PER_CONNECTION. Or use predefined combination of flags: #if MHD_VERSION >= 0x00095208 MHD_USE_THREAD_PER_CONNECTION | MHD_USE_AUTO_INTERNAL_THREAD | MHD_USE_DUAL_STACK, #else MHD_USE_THREAD_PER_CONNECTION | MHD_USE_POLL_INTERNALLY | MHD_USE_DUAL_STACK, #endif I also recommend to use MHD_USE_DEBUG flag when you developing something. Actually, it is not a "debug", it is a "log". -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] 3xx code is incorrect reason phrase with rfc
On 26.04.2017 5:41, hyunho wrote: > I am using MHD 0.9.53 version are working to create a cache server. > > Reason Phrase from 303 onwards found to be wrong with the rfc. > > For example, 304 is defined as '304 Not Modified' in the rfc, but server > response using MHD version 0.9.53 responds with '304 See Other' Hi! Thanks for report! Shift in numbers of reason strings was introduced with update of reason codes. Although the reason string must be ignored by client and only code number must be used (https://tools.ietf.org/html/rfc7230#section-3.1.2 ), obviously it is not convenient to use unmatched strings. Fixed in git master and added test to prevent accidental shift in future. -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] How to close all upgraded connections when shutting down MHD_Daemon?
Hi Nicolas, On 04.05.2017 3:18, Nicolas Mora wrote: > I'm currently working on an implementation of a websocket manager with > MHD and it's getting pretty good so far. > > The problem I have is when the daemon is shut down and there are open > connections. The documentation on MHD_quiesce_daemon says that "Note > that the caller is responsible for closing the returned socket;" which > is close to what I have, but since a websocket can be left open > indefinitely, I need to know if the daemon needs to stop. MHD_quiesce_daemon() prevent MHD from accepting new connections but allow to continue processing with current connections. The returned socket is listening socket. > Is there a signal or an event or any other way for the upgrade_handler > function to know that a MHD_quiesce_daemon or a MHD_stop_daemon has > been sent ? You can track quiesced state of MHD in some global variable (if you are using single MHD instance) or in local variable associated with MHD instance. You will need to update this variable when you quiesced MHD. In upgrade_handler you can check current state of MHD by checking variable. Keep in mind that MHD will not accept new connection after MHD_quiesce_daemon(), so upgrade_handler will be called after MHD_quiesce_daemon() if connection was accepted before MHD_quiesce_daemon() but not yet "upgraded". MHD_stop_daemon() will close all connections including "upgraded" connections. So you can't use "upgraded" connections after MHD_stop_daemon(). Could you explain a bit more situation that you need to solve? Do you want to implement graceful MHD shutdown after closing all upgraded connections? -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] How to close all upgraded connections when shutting down MHD_Daemon?
>> MHD_stop_daemon() will close all connections including "upgraded" >> connections. So you can't use "upgraded" connections after >> MHD_stop_daemon(). >> > The problem I have is that MHD_stop_daemon() doesn't close all the > connections. Instead, it looks like it waits for the active connections > to close by themselves and I have the following log message: > Initiated daemon shutdown while "upgraded" connection was not closed. > Do you use HTTP or HTTPS? You must not call MHD_stop_daemon() while you still have some not closed "upgraded" connections. May be we should add more alarming message in case of such API violation. >> Could you explain a bit more situation that you need to solve? >> Do you want to implement graceful MHD shutdown after closing all >> upgraded connections? >> > I guess what I'd rather have is for the upgraded connections to be > closed by MHD when MHD_stop_daemon() is called, so the upgrade callback > could interpret it easily. > Silvio's suggestion might work too, some signal like a > "MHD_DAEMON_INFO_WAS_QUIESCED" or, in my case > "MHD_DAEMON_INFO_WAS_STOPPED". > > My current implementation is the following. > When a websocket upgrade is called by the client, the websocket callback > is executed: > https://github.com/babelouest/ulfius/blob/2.0/src/ulfius.c#L432 > Then the program loops on listening to the socket > https://github.com/babelouest/ulfius/blob/2.0/src/u_websocket.c#L115 > > But in my example program, when the user stops the program by pressing > the key while connections are active, the program doesn't stop > and waits for the open websockets to close. > https://github.com/babelouest/ulfius/blob/2.0/example_programs/websocket_example/websocket_example.c Could you provide a minimal example? -- Wishes, Evgeny
Re: [libmicrohttpd] How to close all upgraded connections when shutting down MHD_Daemon?
On 04.05.2017 16:41, Nicolas Mora wrote: >> >> Do you use HTTP or HTTPS? > Eventually, since it's a framework for both http and https, I will use > both protocols. Behavior in undefined situation can differ in HTTP and HTTPS modes. MHD_stop_daemon() should close all upgraded connections in HTTPS mode. >> You must not call MHD_stop_daemon() while you still have some not closed >> "upgraded" connections. >> May be we should add more alarming message in case of such API violation. > I understand. > What I have thought of is to store a signature of all opened upgraded > connections in an array, then before calling MHD_stop_daemon(), going > through each connection, and close them. I guess that's the most clean > way to do it. Yes, it's most correct way. >> Could you provide a minimal example? >> > The file > https://github.com/babelouest/ulfius/blob/2.0/example_programs/websocket_example/websocket_example.c > provides a minimal example of what I intend to do, although it's still > in progress. >From MHD point of view it's not minimal - a lot of functions and variables that not required to trigger this behavior. -- Wishes, Evgeny
Re: [libmicrohttpd] How to close all upgraded connections when shutting down MHD_Daemon?
Same for "pending data". Moreover MHD_DAEMON_INFO_PENDING_DATA is confusing and not correct. Zero in data_already_pending doesn't mean that no data is pending. Zero only means that "no data for immediately process". May be socket is not ready for sending more data, but more data is pending. And mapping "bool" to not bool is not correct. union MHD_DaemonInfo doesn't have "bool" member and currently we do not want to expose "bool" in public header. On some platforms "bool" can be implemented as "char". -- Best Wishes, Evgeny Grin On 04.05.2017 17:20, Christian Grothoff wrote: > You forgot to update doc/libmicrohttpd.texi, otherwise looks OK even > though I'm not convinced SHUTDOWN/QUESTCED are useful: an application > can trivially track those itself, so these two are definitively just API > bloat. > > On 05/04/2017 03:59 PM, silvioprog wrote: >> Done. So dudes, what do you think about this attached patch? >> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [PATCH] Check response existence on upgrade
Thanks! Applied. -- Best Wishes, Evgeny Grin On 04.05.2017 22:58, José Bollo wrote: > When a connection upgrade is requested and when the > request sent an error reply, the response is most of > the time already sent when the test on > connection->response->upgrade_handler is made, leading > to dereferencing NULL. > > Two possibilities exist: > > NULL == connection->response || NULL == > connection->response->upgrade_handler > > or > > NULL != connection->response && NULL == > connection->response->upgrade_handler > > The first is prefered because it is probably safer to close the > connection in that case. > > Change-Id: Ie6e7fc165f7fe3635ade0952bb34a0b937d38716 > Signed-off-by: José Bollo > --- > src/microhttpd/connection.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/microhttpd/connection.c b/src/microhttpd/connection.c > index 4753d6e..91a4492 100644 > --- a/src/microhttpd/connection.c > +++ b/src/microhttpd/connection.c > @@ -882,7 +882,7 @@ keepalive_possible (struct MHD_Connection > *connection) #ifdef UPGRADE_SUPPORT > if ( (MHD_str_equal_caseless_ (end, > "upgrade")) && > - (NULL == connection->response->upgrade_handler) ) > + (NULL == connection->response || NULL == > connection->response->upgrade_handler) ) >return MHD_NO; > #endif /* UPGRADE_SUPPORT */ > >
Re: [libmicrohttpd] issue with suspend/resume
Hi José, Shouldn't be any change. Could you provide minimal example? -- Best Wishes, Evgeny Grin On 05.05.2017 17:06, José Bollo wrote: > Hi > > Was there a change in how suspend/resume works? > > I observe that responses are no more send after resuming. > > Best regards > José Bollo > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] About an HTTPS connection Upgraded
Didn't check your code (and your code is not minimal example!), but I suspect that you call MHD_stop_daemon() before you finish working with "upgraded" sockets. You should call MHD_upgrade_action() with MHD_UPGRADE_ACTION_CLOSE for each "upgraded" socket after you finished with socket and call MHD_stop_daemon() only after you finished working with all HTTP and "upgraded" sockets operations. -- Best Wishes, Evgeny Grin On 07.05.2017 23:20, Nicolas Mora wrote: > > > Le 2017-05-07 à 15:42, Christian Grothoff a écrit : >> Eh, what is wrong is that you use "gnutls_record_send". MHD will do >> this for you, you should just use "send/recv", even in HTTPS mode! In >> fact, your code (short of MHD_start_daemon) can be exactly the same for >> HTTP and HTTPS. >> > Argh, I tried that at first and it didn't work, so I used the wrong > path, my bad. > > Any I retried it without any gnutls call on the MHD_sock and I still > have problems reading and writing the socket, nothing is transmitted, > although the handshake is correct because the client says that the > websocket is open, and the send and receive functions are > > When the server is shut down, I have the following error messages: > Failed to forward to remote client 53 bytes of data received from > application: daemon shut down > Failed to signal resume via inter-thread communication channel. > > The updated code can be found here: > https://github.com/babelouest/ulfius/blob/2.0b/src/u_websocket.c > > Any clue? > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Wrong 304 Response Message
Right, descriptions for HTTP codes 304-308 were shifted in 0.9.53. Already reported and fixed: http://lists.gnu.org/archive/html/libmicrohttpd/2017-04/msg00024.html Anyway, clients should not use text description, only numerical code should be used. -- Best Wishes, Evgeny Grin On 08.05.2017 6:05, Markus Mäkelä wrote: > Hi, > > I'm generating HTTP 304 Not Modified responses with libmicrohttpd and > the actual responses returned seem to have the wrong message: > > [markusjm@localhost core]$ curl -I -H "If-Modified-Since: Mon, 08 May 17 > 02:57:43 GMT" localhost:8989/v1/servers/server1 > HTTP/1.1 304 See Other > Connection: Keep-Alive > Date: Mon, 08 May 17 02:58:41 GMT > > It seems like the message for 303 See Other is used instead of the 304 > Not Modified one. I'm using libmicrohttpd version 0.9.53. > > Markus > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] upgraded connections remain suspended when using MHD_USE_THREAD_PER_CONNECTION
Hi Dom, Could you provide a bit more information: used OS (W32, GNU/Linux, FreeBSD, something else...), MHD version? Minimal example also could help. -- Wishes, Evgeny On 09.05.2017 14:12, Dom F wrote: > Hello, > > I start the daemon with these flags: > MHD_USE_THREAD_PER_CONNECTION | MHD_USE_DEBUG | MHD_USE_POLL | > MHD_USE_ITC | MHD_USE_INTERNAL_POLLING_THREAD | MHD_ALLOW_UPGRADE > > When I'm finished with an upgraded connection I call: > MHD_upgrade_action(urh, MHD_UPGRADE_ACTION_CLOSE) > > This leaves urh->was_closed and urh->clean_ready as true and returns > MHD_YES. > > MHD_upgrade_action() also calls MHD_resume_connection() but > resume_suspended_connections() in daemon.c is never called. > > So upgraded connections add to daemon->suspended_connections_tail and > daemon->connections increases until it hits daemon->connection_limit > then my application starts reporting this: > > Server reached connection limit. Closing inbound connection. > > Anyone else seeing the same? > > Dom >
Re: [libmicrohttpd] issue with suspend/resume
On 10.05.2017 10:31, José Bollo wrote: > On Fri, 5 May 2017 21:09:21 +0300 > Evgeny Grin wrote: >> Shouldn't be any change. >> Could you provide minimal example? > > > My feeling is that this is a typical use case and it should work with > only one call to MHD_run. I also think that providing a way to resume a > connection and sending the pending data without having to call MHD_run > would help (but is not required). > Thanks for clarifying and example. MHD do not provide any obligation how many MHD_run() call should be passed before something is processed. You example works with older version only by coincidence. If you change polling function from epoll to poll() or select(), it will require more MHD_run() calls. Moreover, latest versions sending responses with fewer MHD_run() calls, especially in epoll mode. If you provide full response right at time - it will be send immediately, no need to use MHD_run() more times. However I'll make some changes to provide early resume in external poll mode. You could try MHD from git master tomorrow. -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] upgraded connections remain suspended when using MHD_USE_THREAD_PER_CONNECTION
On 10.05.2017 11:21, Dom F wrote: > I'm using MHD versions 0.9.54 and latest from git on both FreeBSD 11 and > Linux (I think CentOS 7 with v3.10 kernel). > No extra args to ./configure on Linux. > HTTPS, gnutls and libgcrypt related args to ./configure on FreeBSD. > > My MHD_UpgradeHandler creates a new thread, immediately detached, which > does the following: > > Put socket into blocking mode > Loop doing read() and processing received data until error, EOF or > "close" websocket frame > Call MHD_upgrade_action(urh, MHD_UPGRADE_ACTION_CLOSE) > Exit thread > > It is also possible that I might call "shutdown(sock, SHUT_RDWR)" from > another thread to cause above loop to exit. By your shutdown() of socket you breaking MHD internal processing. It mostly relevant to HTTPS processing, didn't recheck HTTP code. But I suggest you do not shutdown "upgraded" socket. Probably we need to update documentation to reflect it. Could you try without shutdown() of socket? -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] issue with suspend/resume
You already can try new version from git master.
--
Best Wishes,
Evgeny Grin
On May 11, 2017 10:13:47 AM GMT+03:00, "José Bollo" wrote:
>On Wed, 10 May 2017 21:57:17 +0300
>Evgeny Grin wrote:
>
>> On 10.05.2017 10:31, José Bollo wrote:
>> > On Fri, 5 May 2017 21:09:21 +0300
>> > Evgeny Grin wrote:
>> >> Shouldn't be any change.
>> >> Could you provide minimal example?
>> >
>> >
>> > My feeling is that this is a typical use case and it should work
>> > with only one call to MHD_run. I also think that providing a way to
>> > resume a connection and sending the pending data without having to
>> > call MHD_run would help (but is not required).
>> >
>
>Hi Evgeny,
>
>> Thanks for clarifying and example.
>
>Thank you also for MHD!
>
>> MHD do not provide any obligation how many MHD_run() call should be
>> passed before something is processed.
>
>But calling it at least one time make sense. On the example, there is
>no threading but in real there is. At some point when the request it
>treated, the connection is resume (in some thread) and a call MHD_run
>is piped. It is a good design: I can make serial calls to MHD_run that
>can't be reentering and I can process the request/connection in some
>other thread.
>
>> You example works with older version only by coincidence.
>
>8'{ gloups
>
>> If you
>> change polling function from epoll to poll() or select(), it will
>> require more MHD_run() calls.
>
>I made a test with a long reply and it required more call...
>
>> Moreover, latest versions sending responses with fewer MHD_run()
>> calls, especially in epoll mode. If you provide full response right
>> at time - it will be send immediately, no need to use MHD_run() more
>> times.
>
>That is not clear to me. Sometime, I don't need to suspend the
>connection and in that case, the response is sent immediately, yes. But
>when suspend/resume is requested, I never have never seen the
>response immediately sent.
>
>> However I'll make some changes to provide early resume in external
>> poll mode. You could try MHD from git master tomorrow.
>
>Quiet impatient... If I can help, tell me.
>
>Best regards
>José
Re: [libmicrohttpd] issue with suspend/resume
On 11.05.2017 10:13, José Bollo wrote: Shouldn't be any change. Could you provide minimal example? >>> >>> >>> My feeling is that this is a typical use case and it should work >>> with only one call to MHD_run. I also think that providing a way to >>> resume a connection and sending the pending data without having to >>> call MHD_run would help (but is not required). >>> > >> MHD do not provide any obligation how many MHD_run() call should be >> passed before something is processed. > > But calling it at least one time make sense. On the example, there is > no threading but in real there is. At some point when the request it > treated, the connection is resume (in some thread) and a call MHD_run > is piped. It is a good design: I can make serial calls to MHD_run that > can't be reentering and I can process the request/connection in some > other thread. "At least one time" - correct approach. May be more calls are required. MHD require a lot of internal processing and to handle it correctly with external polling you must also call MHD_get_timeout(). MHD also can run poll functions in internal threads, external poll could provide better result if you intensively intermix polling and processing MHD sockets and some other sockets. If you are using MHD internal polling you could not care about MHD timeouts, runs and processing. MHD will handle all those things automatically. > >> If you >> change polling function from epoll to poll() or select(), it will >> require more MHD_run() calls. > > I made a test with a long reply and it required more call... Correct, long reply always require many calls. >> Moreover, latest versions sending responses with fewer MHD_run() >> calls, especially in epoll mode. If you provide full response right >> at time - it will be send immediately, no need to use MHD_run() more >> times. > > That is not clear to me. Sometime, I don't need to suspend the > connection and in that case, the response is sent immediately, yes. But > when suspend/resume is requested, I never have never seen the > response immediately sent. Starting from 0.9.53 implemented "fast track" for small (received by single recv() and response is provided immediately) HTTP requests: they are handled in one turn. Suspend/resume always required additional processing. >> However I'll make some changes to provide early resume in external >> poll mode. You could try MHD from git master tomorrow. > > Quiet impatient... If I can help, tell me. Pushes to master several minutes ago. Please ignore previous announcement - by some reason commit was not pushed to master. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] issue with suspend/resume
On 11.05.2017 15:26, José Bollo wrote:
> First, I just tried the latest commit
> (787bfd1859db22c58271f287dfcb505c0052edd3): Do not add any "Connection"
> headers for "upgrade" connections.
>
> My issue was still here but...
Strange, resuming connection are now resumed before any other processing.
Are you sure that there are no difference at all?
>> "At least one time" - correct approach.
>> May be more calls are required.
>
> But there is no way to get a status "should MHD_run be called again?".
> Oh "MHD_get_timeout" is maybe the answer! Does it return 0 in case of
> "loop again please"? It seems that yes.
>
> Test...
>
> Works by replacing MHD_run(h) by
>
> do { MHD_run(h); } while(MHD_get_timeout(h, &to) == MHD_YES && !to);
>
> To be honest, I was not handling the timeout because I had no idea that
> it had to drive a such loop. I was thinking it wasn't usefull when
> external epoll was used. I'm not sure to need it for something
> different of the above loop, detecting lazy transmitters can be treated
> lazyly IMHO.
Zero timeout means "need more immediately processing", for example if
more data is still waiting in network buffers.
No-zero timeouts usually generated when you set connection timeout so
MHD is able to close timed-out connections.
--
Wishes,
Evgeny
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] issue with suspend/resume
On 11.05.2017 16:13, José Bollo wrote: > On Thu, 11 May 2017 15:49:43 +0300 > Evgeny Grin wrote: > >> On 11.05.2017 15:26, José Bollo wrote: >>> First, I just tried the latest commit >>> (787bfd1859db22c58271f287dfcb505c0052edd3): Do not add any >>> "Connection" headers for "upgrade" connections. >>> >>> My issue was still here but... >> >> Strange, resuming connection are now resumed before any other >> processing. Are you sure that there are no difference at all? > > No difference: even the header is not sent. OK, I'll check it. > PS. A little question that dont really need answer now: why to not integrate > the test and the loop in MHD_run? Currently timing of MHD_run() is roughly predictable: it's o(n) + const, where n - number of connections. If such kind of loop will be inserted into MHD_run() than timing will be completely unpredictable: if some client will send data fast enough than MHD will process data again and again (hypothetically - infinitely). Application which use MHD in external polling mode, usually run some other processing in the same thread. With this loop, thread may process only MHD data and spend very small part of time for other tasks. -- Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] issue with suspend/resume
Hi José, On 11.05.2017 16:13, José Bollo wrote: >>> First, I just tried the latest commit >>> (787bfd1859db22c58271f287dfcb505c0052edd3): Do not add any >>> "Connection" headers for "upgrade" connections. >>> >>> My issue was still here but... >> >> Strange, resuming connection are now resumed before any other >> processing. Are you sure that there are no difference at all? > > No difference: even the header is not sent. Could you try with current git master? Processing should be improved after 6da279752e54c616edcb485ea637234101447c9: resume_suspended_connections(): update states of connection after resuming -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] upgraded connections remain suspended when using MHD_USE_THREAD_PER_CONNECTION
On 11.05.2017 13:50, Dom F wrote: > OK, I replaced my calls to shutdown() with a debug message and turned on > MHD_USE_THREAD_PER_CONNECTION again. > Currently I am seeing approx. 55 active websocket threads but > daemon->connections is up to around 1450. > Looking at my logfile there were only about 4 times my code would have > called shutdown() anyway. > Also, I'm NOT using MHD_USE_TLS or any HTTPS-related MHD flag. > (daemon->options is 42061) > > Hope this helps narrow down the issue! I can also very easily test if > you want to throw more suggestions or patches my way. > Thanks for report and additional information! Closure of upgraded connection in poll() and select() mode with thread-per-connection is fixed in git master. Could you check an updated MHD from git? -- Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Recursive lock in GIT
Hi Markus,
Thanks for report!
Could you provide more information? Why connection was closed right
after resume?
--
Wishes,
Evgeny
On 21.05.2017 13:52, Markus Doppelbauer wrote:
> Hello,
>
> I have updated "0.9.53" to GIT - maybe a week ago.
> There is a recursive lock to "daemon->cleanup_connection_mutex"
>
>
>
> __owner=6065 holds the lock - but tries to lock again in "connection.c:2837"
>
> The callstack:
> "resume_suspended_connections()" at daemon.c:2908
> "MHD_connection_handle_idle()" at connection.c:3344
> "cleanup_connection()" at connection.c:2837
>
> The mutex is locked in "daemon.c":"resume_suspended_connections()" - frame 5
> The mutex is locked again in "connection.c":"cleanup_connection()" - frame 3
>
>
>
> (gdb) thread apply all bt
>
> Thread 4 (Thread 0x7f4166ef8700 (LWP 6065)):
> #0 __lll_lock_wait () at
> ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
> #1 0x003c0c0095d8 in _L_lock_854 () from /lib64/libpthread.so.0
> #2 0x003c0c0094a7 in __pthread_mutex_lock (mutex=0x36e4868) at
> pthread_mutex_lock.c:61
> #3 0x004998c4 in cleanup_connection (connection=0x7f4158000e50)
> at /.../microhttpd/connection.c:2837
> #4 0x0049b830 in MHD_connection_handle_idle
> (connection=0x7f4158000e50) at /.../microhttpd/connection.c:3344
> #5 0x0049e659 in resume_suspended_connections
> (daemon=0x36e4720) at /.../microhttpd/daemon.c:2908
> #6 0x004a404c in MHD_epoll (daemon=0x36e4720, may_block=1) at
> /.../microhttpd/daemon.c:4238
> #7 0x004a4825 in MHD_select_thread (cls=0x36e4720) at
> /.../microhttpd/daemon.c:4542
> #8 0x003c0c007aa1 in start_thread (arg=0x7f4166ef8700) at
> pthread_create.c:301
> #9 0x003c0bce8bcd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
>
> Thread 2 (Thread 0x7f41417fb700 (LWP 22899)):
> #0 __lll_lock_wait () at
> ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
> #1 0x003c0c0095d8 in _L_lock_854 () from /lib64/libpthread.so.0
> #2 0x003c0c0094a7 in __pthread_mutex_lock (mutex=0x36e4868) at
> pthread_mutex_lock.c:61
> #3 0x0049eb16 in MHD_resume_connection
> (connection=0x7f41580015e0) at /.../microhttpd/daemon.c:2808
> [snip: processes request]
>
> Thread 1 (Thread 0x7f41aeef68a0 (LWP 6058)):
> [snip: main() thread]
>
>
>
> (gdb) thread 2
> (gdb) frame 3
> (gdb) p daemon->cleanup_connection_mutex
> $2 = {__data = {__lock = 2, __count = 0, __owner = 6065, __nusers = 1,
> __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
> __size = "\002\000\000\000\000\000\000\000\261\027\000\000\001",
> '\000' , __align = 2}
>
>
>
> (gdb) thread 4
> (gdb) frame 3
> (gdb) p daemon->cleanup_connection_mutex
> $3 = {__data = {__lock = 2, __count = 0, __owner = 6065, __nusers = 1,
> __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
> __size = "\002\000\000\000\000\000\000\000\261\027\000\000\001",
> '\000' , __align = 2}
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] Recursive lock in GIT
IE 4.01 on Win95? Cool. :) Should be fixed in master. Could you check it? -- Best Wishes, Evgeny Grin On 21.05.2017 16:30, Markus Doppelbauer wrote: > Dear Evgeny, > > Maybe because the request-headers contain "Connection: close"? > This is what I could determine with `gdb`: > > (gdb) frame 4 > (gdb) p *connection->headers_received > > GET / HTTP/1.1 > Host: server.name > Connection: close > Content-Length: 4 > Accept-Encoding: gzip > Accept: */* > Content-Type: application/x-www-form-urlencoded > User-Agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows 95) > Cache-Control: no-cache > > Thanks a lot > Markus > > > > Am Sonntag, den 21.05.2017, 15:21 +0300 schrieb Evgeny Grin: >> Hi Markus, >> >> Thanks for report! >> Could you provide more information? Why connection was closed right >> after resume? >> signature.asc Description: OpenPGP digital signature
[libmicrohttpd] GNU libmicrohttpd 0.9.55 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.55. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). This is bugfix release. Most noticeable changes since version 0.9.53: * Fixed minor issues uncovered by a security audit sponsored by the Secure Open Source Fund, a Mozilla initiative to support key pieces of open source software with security audits. For more information see https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#GNU_libmicrohttpd SOS is part of Mozilla Open Source Support (MOSS) - read more at https://mozilla.org/moss * Fixed build on GNU/Linux without epoll support * Improved backward compatibility with applications which use specific combination of flags * Fixed wrong text in HTTP reason strings for HTTP result codes 303-308 * Added new option MHD_OPTION_STRICT_FOR_CLIENT * Added connection info type MHD_CONNECTION_INFO_REQUEST_HEADER_SIZE Most noticeable changes since unannounced version 0.9.54: * Fixed adding unwanted "Transfer-Encoding: chunked" and "Transfer-Encoding: chunked" to response headers when "upgrade" is used. * Fixed null dereference with upgrade response when error sent to client instead of response * Fixed case-sensitive match in MHD_get_response_header(), now used case-insensitive match * Fixed detection of "Connection: " tokens when several tokens are used within single header line * Fixed closing of "upgraded" connection in poll() mode with thread-per-connection * Fixes leak of response when "upgrade" failed * Fixed build of examples when building with disabled "upgrade" * Fixed possible null dereference in HTTPS test * Fixed returned pointer for MHD_CONNECTION_INFO_CONNECTION_SUSPENDED * Fixed cleanup of suspended connection * Other minor fixes * Removed some dead code * Improved connection processing after resuming in epoll mode. * Optimized detection of "Keep-Alive/Closed" * Other minor optimizations You can download GNU libmicrohttpd from * ftp://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at http://www.gnu.org/software/libmicrohttpd/. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [GSoC Update] Week 2
On 13.06.2017 19:37, Darshit Shah wrote: >> >> Things which would be done in the coming week: >> >> * Finished on wget_test_start_server() in order to call Libmicrohttpd as >> service for wget_test(). Problems and questions need to be resolved: >> - Decide what the best threading model for Libmicrohttpd. Currently >> using >> MHD_USE_INTERNALLY_POLLING_THREAD which use external select. I >> still check >> the comparison with legacy code that use Wget2 API wget_thread_start. > Choose any mechanism that uses select(). We can change the threading > model at a later stage if it turns out to be a bottleneck. `epoll` is > Linux-only and even `poll` isn't always available, so as long as you > choose a `select` based implementation, it should be fine for now. You can use MHD_is_feature_supported() with MHD_FEATURE_POLL and MHD_FEATURE_EPOLL to check for supported polling functions. Alternatively, with latest versions of MHD you can use MHD_USE_AUTO and MHD will choose select(), poll() or epoll-based polling automatically. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [GSoC Update] Week 2
On 17.06.2017 11:36, Tim Rühsen wrote:
> On Samstag, 17. Juni 2017 13:53:06 CEST Didik Setiawan wrote:
>> On Tue, Jun 13, 2017 at 06:37:27PM +0200, Darshit Shah wrote:
>>>> - http_server_port still hardcoded.
>>>
>>> This is important. The port should be a random number. Usually, passing 0
>>> in the port number makes the kernel choose an open one for you.
>>>
>>> Having a randomised port is important to ensure that multiple runs don't
>>> step on each other.
>>
>> I still don't know how to accomplish this. Maybe, it just my understanding
>> that when I pass 0 in MHD port number, the result is still 0.
>> Another approach, when I look into the old code, it generate port number by
>> calling wget_tcp_get_local_port(). But, I need to call wget_tcp_init() and
>> wget_tcp_listen() respectively in order to get proper result.
>> Conclusion, do I need to use existing wget_tcp_get_local_port() to get the
>> port, or maybe there is a function in MHD to do that?
>
> All you need is the socket descriptor. How to call getsockname() +
> getnameinfo() to retrieve the port number you see in libwget/net.c/
> wget_tcp_get_local_port().
>
> If MHD doesn't have such a function, either try to get the socket descriptor
> or extend MHD with a small function (similar code as in
> wget_tcp_get_local_port).
There are several ways to implement it.
First one:
Initialise socket externally and start listen() on it. You can use any
bind and port detection function.
Pass socket FD to MHD_start_daemon() by MHD_OPTION_LISTEN_SOCKET.
Second one:
Use MHD_start_daemon() with "0" as port number then use
MHD_get_daemon_info() with MHD_DAEMON_INFO_LISTEN_FD to get listen
socket FD. Use any port detection.
Third one:
Implemented two day ago, works with MHD_VERSION >= 0x00095501 and on
platforms that support getsockname().
Use MHD_start_daemon() with "0" as port number then use
MHD_get_daemon_info() with MHD_DAEMON_INFO_BIND_PORT to get port number.
You can combine second and third methods:
---
int port_num;
if(0) {}
#if MHD_VERSION >= 0x00095501
else if (MHD_NO !=
MHD_is_feature_supported (MHD_FEATURE_AUTODETECT_BIND_PORT))
{
const union MHD_DaemonInfo *dinfo;
dinfo = MHD_get_daemon_info (d, MHD_DAEMON_INFO_BIND_PORT);
if (NULL == dinfo || 0 == dinfo->port)
{
/* Insert code to handle error. */
return -1;
}
port_num = (int)dinfo->port;
}
#endif /* MHD_VERSION >= 0x00095501 */
else
{
const union MHD_DaemonInfo *dinfo;
MHD_socket sock_fd;
dinfo = MHD_get_daemon_info (d, MHD_OPTION_LISTEN_SOCKET);
if (NULL == dinfo)
{
/* Insert code to handle error. */
return -1;
}
sock_fd = dinfo->listen_fd;
/* Insert code to use port detection on provided socket. */
}
---
Note that socket type used by MHD functions is 'MHD_socket'.
MHD_socket is 'int' on POSIX platforms (Linux, *BSD, Unix, Darwin) and
'SOCKET' on Windows platform (excluding Cygwin, where MHD_socket is 'int').
--
Best Wishes,
Evgeny Grin
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] [GSoC Update] Week 2
On 17.06.2017 22:22, Didik Setiawan wrote:
> On Sat, Jun 17, 2017 at 01:30:57PM +0300, Evgeny Grin wrote:
>>
>
>> You can combine second and third methods:
>
> I use this combination.
>
>> ---
>> int port_num;
>>
>> if(0) {}
>> #if MHD_VERSION >= 0x00095501
>> else if (MHD_NO !=
>> MHD_is_feature_supported (MHD_FEATURE_AUTODETECT_BIND_PORT))
>> {
>> const union MHD_DaemonInfo *dinfo;
>> dinfo = MHD_get_daemon_info (d, MHD_DAEMON_INFO_BIND_PORT);
>> if (NULL == dinfo || 0 == dinfo->port)
>> {
>> /* Insert code to handle error. */
>> return -1;
>> }
>> port_num = (int)dinfo->port;
>> }
>> #endif /* MHD_VERSION >= 0x00095501 */
>> else
>> {
>> const union MHD_DaemonInfo *dinfo;
>> MHD_socket sock_fd;
>> dinfo = MHD_get_daemon_info (d, MHD_OPTION_LISTEN_SOCKET);
>
> Don't you mean:
>
> dinfo = MHD_get_daemon_info(d, MHD_DAEMON_INFO_LISTEN_FD);
>
> Because when I tried using MHD version < 0.9.55 the result is not as
> expected.
>
>> if (NULL == dinfo)
>> {
>> /* Insert code to handle error. */
>> return -1;
>> }
>> sock_fd = dinfo->listen_fd;
>> /* Insert code to use port detection on provided socket. */
>> }
>> ---
Right, correct value is MHD_DAEMON_INFO_LISTEN_FD.
--
Wishes,
Evgeny
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] Add 'manywarnings' module from Wget2
On 18.06.2017 13:45, Tim Rühsen wrote: > maybe you like this... > > "Add m4/wget_manywarnings > > This enables (if switched on) basically all warnings for gcc and clang. > Exceptions are explicitly controlled in configure.ac. > > To enable by default: create a file '.manywarnings' > Enable explicitly : ./configure --enable-manywarnings > Disable explicitly: ./configure --disable-manywarnings > > New warnings of new compiler versions are automatically added. > This module works much faster than the 'manywarnings' module from gnulib. > > Created for developers and Continuous Integration." > > I just added the WARN_FLAGS to src/microhttpd/Makefile.am, but if you like it > add it also to the other Makefile.am. > > It might need some further tuning in configure.ac to enable/disable warnings > (I > just copied the code from Wget2 were we ignore certain warning). Hi Tim, While idea is generally looks good, proposed implementation do not suit well from my point of view. Code is not self-explainable, so it should be well-documented. For compatibility with future versions, code should test compiler acceptance for added flags. Otherwise software may fail to build with some new GCC/clang version. Test for "gcc" is not correct: case $CC in *gcc*) CCNAME="gcc";; *clang*) CCNAME="clang";; esac This fail on Darwin (where 'gcc' is symlink for 'clang') and on any other platform with such symlink. Moreover, 'gcc' could match directory part. For example CC='/opt/compilers/gcc-like/clang-4.0/bin/clang' will be detected as 'gcc'. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] POST file content without the form method
26.06.2017, 16:35, "alexander.he...@a1.net" : > I've used the largepost.c example and implemented the upload. When I perform > a test using cUrl, everything works fine. > > curl -F "payload=http://username:password@localhost:9090 > > However requirements changed and I need to change the logic from using the > form post to posting the file content directly without having a "key". Doing > this in cUrl looks like: > > curl -d "@remove.xml" http://username:password@localhost:9090 > > Assume the signature of my iteratePost looks like this: > > static int iteratePost(void *conInfoCls, > enum MHD_ValueKind kind, > const char *key, > const char *filename, > const char *content_type, > const char *transfer_encoding, > const char *data, > uint64_t off, > size_t size); > > When i post the file content directly, some part of the files content is in > the key and some in the data variables. It would be possible to handle this; > however, if have a "=" character in the first section of the file, it seems > that libmicrohttpd is removing it. So lets assume the file remove.xml has the > following line at the beginning: > > > > What is receive in the iteratePost is: > > > Hence, the content is invalid. > > Is there a way in the library to handle this case? Hi Alex, If I understand you correctly, you are trying to upload some arbitrary data to MHD. Correct? POST is designed to handle uploading of filled forms with data in format 'name=value'. It cannot handle some random data. If you want to push content of any local file, you should use PUT. src/testcurl/test_put.c test_put_chunked.c and test_large_put.c may give you some inspiration. -- Best Wishes, Evgeny Grin
Re: [libmicrohttpd] HTTP/2 Support on GNU Libmcrohttpd
On 08.07.2017 0:37, Didik Setiawan wrote: > I just reiterate a question about is there any plan to make GNU Libmicrohttpd > to > support HTTP/2? > > If not, may I bother if we are willing to propose new change about that? > Adding > HTTP/2 support to Libmicrohttpd. I know this can be categorized as non trivial > task, since for me myself, there are still many unresolved tasks regarding > GSoC > 2017. But, if you agree, I will start work in this topic once my tasks > completed, in the near weeks. I'm new to HTTP/2 and how to implementing RFC > into > real software so any guide and pointers are really appreciated. > > I open this question because in Wget2 project, there is a requirement about > testing Wget2 HTTP/2 code. Currently, I working on a topic of integrating > Libmicrohttpd into Wget2 as http server. Unfortunately, Libmicrohttpd is not > supporting HTTP/2 yet. And it will be nice if Libmicrohttpd has support on it, > so both of the project could have benefit from it. Hi Didik, Implementing of HTTP/2 in appreciated but not yet planned. HTTP/1.1 is enough for many current tasks and will be enough in foreseeable future, so HTTP/2 is not high priority. HTTP/2 is much more complex comparing to HTTP/1.1. It require multiplexing of data streams and encoding and decoding headers. HTTP/2 will also require significant rise of amount of code and larger resulting binary. Many functions, useful for HTTP/1.0 and HTTP/1.1 can't be reused for HTTP/2 or will need rewrite. We consider small binary size as advantage of MHD so HTTP/2-related code must be implemented as disableable by configure switch. Patches are always welcome but for so large change it will need a separate branch with atomic commits and deep review. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Draft of CMake build script
Hi Štefan,
Thanks for sharing this.
Our main build system is GNU autotools and it will remain our main build
system in foreseeable future. It is used currently on GNU/Linux,
Windows, *BSD, Darwin, Solaris, OpenIndiana and many other platforms.
As exclusion, we also maintain MSVC projects, but maintaining them is
pretty simple as some information is reused from autotools.
By using CMake we could get rid of MSVC projects, but maintaining CMake
is way harder then MSVC project in current state. Moreover, it will be
harder to provide same level of flexibility for MSVC configuration as we
currently provide by native projects.
Switching from MSVC projects to CMake will increase amount of
maintenance tasks and do not provide currently additional benefits, so
currently we will not going to maintain it in official repository.
--
Best Wishes,
Evgeny Grin
On 18.07.2017 18:52, Štefan Bellus wrote:
> Hi all
>
> I prepare the CMakeLIsts.txt file.
> I try to convert autotools project to CMake one.
> The advantage is the same build system on linux and windows.
> It is draft because the CMake builds only library (no documentation and
> examples, no correct rc file on windows)
>
>
> Please review or at least give me know if maintainer of libmicrohttpd
> wants to also support CMake build scripts.
>
> The CMakeLists.txt start
> #
> # Cmake instructions file
> #
> cmake_minimum_required(VERSION 3.1)
>
> include(CheckSymbolExists)
> include(CheckIncludeFiles)
> include(CheckCSourceCompiles)
> include(CheckStructHasMember)
> include(CheckFunctionExists)
> include(CheckCCompilerFlag)
> include(CheckCSourceRuns)
>
> project(libmicrohttpd LANGUAGES C VERSION 0.9.55)
>
> set(ENABLE_BAUTH yes CACHE BOOL "Enable basic Auth support")
> set(ENABLE_DAUTH yes CACHE BOOL "Enable digest Auth support")
> set(ENABLE_EPOLL yes CACHE BOOL "Enable epoll")
> set(ENABLE_POLL yes CACHE BOOL "Enable poll")
> set(ENABLE_ERROR_MSGS yes CACHE BOOL "Enable support for error
> messages")
> set(ENABLE_POSTPROCESSOR yes CACHE BOOL "Enable postprocessor support")
> set(ENABLE_HTTPS yes CACHE BOOL "Enable HTTPS support")
> set(DISABLE_THREAD_NAMES no CACHE BOOL "Disable setting name on
> generated threads")
> set(ENABLE_UPGRADEyes CACHE BOOL "Enable Upgrade support")
> set(MHD_ITC autoCACHE STRING "For inter process
> communication choose one of pipe, socketpair, eventfd, auto")
>
> macro(FindNSS)
> # - Try to find the NSS library
> # Once done this will define
> #
> # NSS_FOUND - system has mozilla-nss lib
> # NSS_INCLUDE_DIRS - the mozilla-nss include directories
> # NSS_LDFLAGS - Link these to use mozilla-nss
> # NSS_CFLAGS_OTHER - Compiler switches required for using NSS
> #
> # Copyright (c) 2006, Laurent Montel, <mailto:mon...@kde.org>>
> #
> # Redistribution and use is allowed according to the terms of the BSD
> license.
> # For details see the accompanying COPYING-CMAKE-SCRIPTS file.
>
> if(NSS_INCLUDE_DIRS AND NSS_LDFLAGS)
>
> # in cache already
> SET(NSS_FOUND TRUE)
>
> else()
> if(NOT WIN32)
> find_package(PkgConfig)
> pkg_search_module(NSS nss)
> endif(NOT WIN32)
>
> if (NSS_FOUND)
> if (NOT Nss_FIND_QUIETLY)
> message(STATUS "Found NSS: ${NSS_LDFLAGS}")
> endif (NOT Nss_FIND_QUIETLY)
> else (NSS_FOUND)
> if (Nss_FIND_REQUIRED)
> message(FATAL_ERROR "Could NOT find NSS")
> endif (Nss_FIND_REQUIRED)
> endif (NSS_FOUND)
>
> mark_as_advanced(NSS_INCLUDE_DIRS NSS_LDFLAGS NSS_CFLAGS_OTHER
> NSS_VERSION)
>
> endif()
> endmacro()
>
> macro(FindLibGCrypt)
> # - Try to find the libgcrypt library
> # Once done this will define
> #
> # LIBGCRYPT_FOUND - system has libgcrypt
> # LIBGCRYPT_INCLUDE_DIR - the libgcrypt include directory
> # LIBGCRYPT_LIBRARIES - Link these to use libgcrypt
>
> # Copyright (c) 2010 Joris Guisson <mailto:joris.guis...@gmail.com>>
> # Copyright (c) 2007 Charles Connell <mailto:char...@connells.org>> (This was based upon FindKopete.cmake)
> #
> # Redistribution and use is allowed according to the terms of the
> BSD license.
> # For details see the accompanying COPYING-CMAKE-SCRIPTS file.
>
> if(LIBGCRYPT_INCLUDE_DIR AND LIBGCRYPT_LIBRARIES)
>
> # read from cache
> set(LIBGCRYPT_FOUND TRUE)
>
> else(LIBGCRYPT_INCLUDE_DIR AND LIBGCRYPT_LIBRARIES)
>
Re: [libmicrohttpd] Draft of CMake build script
On 20.07.2017 10:22, Štefan Bellus wrote: > Sorry That I do no reply on mails, but I did not received any even if I > am subscribed. > I only noticed your reply in archive. Try to check "spam". >> By using CMake we could get rid of MSVC projects, but maintaining > CMake is way harder then MSVC project in current state > > Yes. The benefit will be achieved only if you use only CMake. No > autotools, MSVC projects. > I convert the autotools to CMake to check if possible to replace it with > CMake. > I found out that it is possible. It take me 2 days to convert it. But it > not waste of time. I learn how to configure sources in CMake and I learn > also how autotools configure sources. I suggest you to put your result to some public repo. May be others will find your CMake setup useful. >> What about meson? > > It is cool. The syntax is nice. They keep the idea of describing the > build in general language and > let the back-end (CMake or meson) to generate specific commands for > underlying build systems. > The syntax of this general language is much better in meson. > I votes for meson :) > I am little bit skeptic because more and more people start to use CMake > nowadays. But meson is written in python and a lot of more people can > use python as C/CPP (CMake is written in C/CPP) and contribute to meson. > I guess if meson also introduce some package manager for C (something > like Java maven repositories or C# Nuget) they will win. > > I am a maintainer of project that depends on your library. I would love > if you can provide same build system on all platforms. It really make > thinks easier. As I said, you can try to maintain your CMake configuration. If there will be mass demand for CMake, we could come back to this question. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] HTTPS errors
Hi Štefan, Could you provide more details? MHD version, GnuTLS version, OS type and version, text output of failed tests? Of course it is not OK, HTTPS should works fine. -- Best Wishes, Evgeny Grin On 15.08.2017 19:42, Štefan Bellus wrote: > Hi > > I tune configuration and add test, examples target. I have following > test results. > The following tests FAILED: > 39 - tls_options (Failed) > 43 - https_get_select (Failed) > 44 - https_session_info (Failed) > Also I have trouble when I start src/examples/demo_https (it is build > with autotools). The server reports "Error: received handshake message > out of context" and chrome reports ERR_EMPTY_RESPONSE. > > Is it OK or I should investigate more? signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] HTTPS errors
All tests are passed on GNU/Linux (i686, x86_64, SPARC, ARM), macOS, Windows (MinGW and MSVC), FreeBSD, OpenBSD, NetBSD, Solaris. One or two tests are failing on GNU Hurd. If you are using git: autoreconf -v /configure make all make check If you are using released tarball, skip first step. -- Best Wishes, Evgeny Grin On 15.08.2017 22:55, Štefan Bellus wrote: > Ok. > > If it is working on your pc then I will provide for you the requested info. > > When I build with autotools then NO tests are built. > > Please could you give me some hint how to build them? > > > Typed with thumbs. > > On Aug 15, 2017 20:52, "Evgeny Grin" <mailto:k...@yandex.ru>> wrote: > > Hi Štefan, > > Could you provide more details? > MHD version, GnuTLS version, OS type and version, text output of failed > tests? > > Of course it is not OK, HTTPS should works fine. > > -- > Best Wishes, > Evgeny Grin > > On 15.08.2017 19:42, Štefan Bellus wrote: > > Hi > > > > I tune configuration and add test, examples target. I have following > > test results. > > The following tests FAILED: > > 39 - tls_options (Failed) > > 43 - https_get_select (Failed) > > 44 - https_session_info (Failed) > > Also I have trouble when I start src/examples/demo_https (it is build > > with autotools). The server reports "Error: received handshake message > > out of context" and chrome reports ERR_EMPTY_RESPONSE. > > > > Is it OK or I should investigate more? > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] HTTPS errors
Looks like you are using outdated and unsupported version of RHEL. Actual version in RHEL 6 is 6.9. Version 6.8 currently in Extended support. Most probably it is caused by outdated GnuTLS version. 2.x versions are known to have problems. However I'll try to reproduce it on CentOS when I'll have some spare time. -- Best Wishes, Evgeny Grin On 16.08.2017 12:03, Štefan Bellus wrote: > Hi > > I run test compiled with autotools and get SAME (return value 77 in your > driver means skip and in cmake it means fail) result. > MHD version 0.9.55 > GnuTLS version 2.8.5 > OS type and version RedHat 6.4 (uname -a -> Linux buildvm > 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 > x86_64 GNU/Linux) > > Making check in microhttpd > PASS: test_str_compare > PASS: test_str_to_value > PASS: test_str_token > PASS: test_http_reasons > PASS: test_shutdown_select > PASS: test_shutdown_poll > PASS: test_daemon > PASS: test_upgrade > PASS: test_upgrade_tls > PASS: test_postprocessor > PASS: test_postprocessor_large > PASS: test_postprocessor_amp > > Testsuite summary for GNU Libmicrohttpd 0.9.55 > > # TOTAL: 12 > # PASS: 12 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 0 > # XPASS: 0 > # ERROR: 0 > > Making check in testcurl > Making check in . > PASS: test_start_stop > PASS: test_get > PASS: test_get_sendfile > PASS: test_urlparse > PASS: test_delete > PASS: test_put > PASS: test_process_headers > PASS: test_process_arguments > PASS: test_parse_cookies > PASS: test_large_put > PASS: test_get11 > PASS: test_get_sendfile11 > PASS: test_put11 > PASS: test_large_put11 > PASS: test_large_put_inc11 > PASS: test_long_header > PASS: test_long_header11 > PASS: test_get_chunked > PASS: test_put_chunked > PASS: test_iplimit11 > PASS: test_termination > PASS: test_timeout > PASS: test_callback > PASS: perf_get > PASS: test_get_response_cleanup > PASS: test_quiesce > PASS: test_concurrent_stop > PASS: perf_get_concurrent > PASS: test_quiesce_stream > PASS: test_post > PASS: test_postform > PASS: test_post_loop > PASS: test_post11 > PASS: test_postform11 > PASS: test_post_loop11 > PASS: test_digestauth > PASS: test_digestauth_with_arguments > > Testsuite summary for GNU Libmicrohttpd 0.9.55 > > # TOTAL: 37 > # PASS: 37 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 0 > # XPASS: 0 > # ERROR: 0 > > Making check in https > Making check in . > SKIP: test_tls_options > PASS: test_https_multi_daemon > PASS: test_https_get > FAIL: test_https_get_select > PASS: test_https_get_parallel > SKIP: test_https_get_parallel_threads > SKIP: test_https_session_info > PASS: test_https_time_out > PASS: test_tls_authentication > PASS: test_empty_response > > Testsuite summary for GNU Libmicrohttpd 0.9.55 > > # TOTAL: 10 > # PASS: 6 > # SKIP: 3 > # XFAIL: 0 > # FAIL: 1 > # XPASS: 0 > # ERROR: 0 > ==== > See src/testcurl/https/test-suite.log > Please report to libmicrohttpd@gnu.org <mailto:libmicrohttpd@gnu.org> > > > Best regards > Stevo > > On Tue, Aug 15, 2017 at 11:14 PM, Evgeny Grin <mailto:k...@yandex.ru>> wrote: > > All tests are passed on GNU/Linux (i686, x86_64, SPARC, ARM), macOS, > Windows (MinGW and MSVC), FreeBSD, OpenBSD, NetBSD, Solaris. > One or two tests are failing on GNU Hurd. > > If you are using git: > autoreconf -v > /configure > make all > make check > > If you are using released tarball, skip first step. > > -- > Best Wishes, > Evgeny Grin > > On 15.08.2017 22:55, Štefan Bellus wrote: > > Ok. > > > > If it is working on your pc then I will provide for you the requested > info. > > > > When I build with autotools then NO tests are built. > > > > Please could you give me some hint how to build them? > > >
Re: [libmicrohttpd] HTTPS errors
Tested on CentOS 6.9 with current updates. MHD 0.9.55, GnuTLS 2.12.23 (from distribution repo). No test fails. -- Best Wishes, Evgeny Grin On 16.08.2017 14:57, Evgeny Grin wrote: > Looks like you are using outdated and unsupported version of RHEL. > Actual version in RHEL 6 is 6.9. Version 6.8 currently in Extended support. > Most probably it is caused by outdated GnuTLS version. 2.x versions are > known to have problems. > However I'll try to reproduce it on CentOS when I'll have some spare time. > > -- Best Wishes, Evgeny Grin On 16.08.2017 12:03, Štefan Bellus wrote: >> Hi >> >> I run test compiled with autotools and get SAME (return value 77 in your >> driver means skip and in cmake it means fail) result. >> MHD version 0.9.55 >> GnuTLS version 2.8.5 >> OS type and version RedHat 6.4 (uname -a -> Linux buildvm >> 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 >> x86_64 GNU/Linux) >> >> >> Making check in https >> Making check in . >> SKIP: test_tls_options >> PASS: test_https_multi_daemon >> PASS: test_https_get >> FAIL: test_https_get_select >> PASS: test_https_get_parallel >> SKIP: test_https_get_parallel_threads >> SKIP: test_https_session_info >> PASS: test_https_time_out >> PASS: test_tls_authentication >> PASS: test_empty_response >> >> Testsuite summary for GNU Libmicrohttpd 0.9.55 >> >> # TOTAL: 10 >> # PASS: 6 >> # SKIP: 3 >> # XFAIL: 0 >> # FAIL: 1 >> # XPASS: 0 >> # ERROR: 0 >> >> See src/testcurl/https/test-suite.log >> Please report to libmicrohttpd@gnu.org <mailto:libmicrohttpd@gnu.org> >> >> >> Best regards >> Stevo >> >> On Tue, Aug 15, 2017 at 11:14 PM, Evgeny Grin > <mailto:k...@yandex.ru>> wrote: >> >> All tests are passed on GNU/Linux (i686, x86_64, SPARC, ARM), macOS, >> Windows (MinGW and MSVC), FreeBSD, OpenBSD, NetBSD, Solaris. >> One or two tests are failing on GNU Hurd. >> >> If you are using git: >> autoreconf -v >> /configure >> make all >> make check >> >> If you are using released tarball, skip first step. >> >> -- >> Best Wishes, >> Evgeny Grin >> >> On 15.08.2017 22:55, Štefan Bellus wrote: >> > Ok. >> > >> > If it is working on your pc then I will provide for you the requested >> info. >> > >> > When I build with autotools then NO tests are built. >> > >> > Please could you give me some hint how to build them? >> > >> > >> > Typed with thumbs. >> > >> > On Aug 15, 2017 20:52, "Evgeny Grin" > <mailto:k...@yandex.ru> >> > <mailto:k...@yandex.ru <mailto:k...@yandex.ru>>> wrote: >> > >> > Hi Štefan, >> > >> > Could you provide more details? >> > MHD version, GnuTLS version, OS type and version, text output >> of failed >> > tests? >> > >> > Of course it is not OK, HTTPS should works fine. >> > >> > -- >> > Best Wishes, >> > Evgeny Grin >> > >> > On 15.08.2017 19:42, Štefan Bellus wrote: >> > > Hi >> > > >> > > I tune configuration and add test, examples target. I have >> following >> > > test results. >> > > The following tests FAILED: >> > > 39 - tls_options (Failed) >> > > 43 - https_get_select (Failed) >> > > 44 - https_session_info (Failed) >> > > Also I have trouble when I start src/examples/demo_https (it >> is build >> > > with autotools). The server reports "Error: received >> handshake message >> > > out of context" and chrome reports ERR_EMPTY_RESPONSE. >> > > >> > > Is it OK or I should investigate more? >> > >> >> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL
Hi Denis, Thanks for report! Minimal example could help a lot. In the meantime, could you provide more details how do you process incoming data? Do you use MHD_USE_ERROR_LOG flag for daemon? What's the output on stderr? Also you could build MHD from source, modify definition of PUT_SIZE in src/testcurl/test_large_put.c and run 'make check' -- Wishes, Evgeny On 04.09.2017 7:35, Denis Dowling wrote: > Hi All, > > > > I recently updated some code to use version 0.9.55 of libmicrohttpd. > After the change I noticed that larger (~100kB) HTTPS requests are > blocking forever. During the change I swapped from using the option > MHD_USE_POLL to MHD_USE_AUTO. I can see from an strace that > libmicrohttpd is now using epoll() for the dispatching. If I switch > back to MHD_USE_POLL then everything works fine again. If I use HTTP > requests then everything is fine so it is something to do with the > combination of HTTPS and epoll(). Other options are > MHD_USE_INTERNAL_POLLING_THREAD | MHD_USE_SUSPEND_RESUME | MHD_USE_TLS. > > > > Has anyone seen anything similar? I am running on CentOS7.3. The same > issue also appears on Redhat7.4. I will try to modify one of the example > applications in libmicrohttpd to demonstrate the issue. signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL
Hi Denis, Reproduced on CentOS 7 with current git master. Will look into it. -- Best Wishes, Evgeny Grin On 05.09.2017 4:54, Denis Dowling wrote: > Hi Evgeny, > > It is actually the GET that is causing the problem. > > I have been able to duplicate as follows: > - Testing on CentOS7.3.1611. Everything up to date according to yum. > - Start with a clean unpack of libmicrohttpd-0.9.55 > - Edit the https_fileserver_example.c file and change the > MHD_USE_THREAD_PER_CONNECTION option to MHD_USE_EPOLL. Leave the other > options as they are. > - make https_fileserver_example > - In the same directory create 2 files of different sizes (1kB and 100kB) > dd if=/dev/urandom of=small.dat bs=1024 count=1 > dd if=/dev/urandom of=big.dat bs=1024 count=100 > - Start https_fileserver_example >./https_fileserver_example > - In another window use wget to grab both files > wget --no-check-certificate https://localhost:/small.dat > wget --no-check-certificate https://localhost:/big.dat > > In my case the big.dat download freezes at 16384 bytes downloaded. If I > change from MHD_USE_EPOLL to MHD_USE_POLL it works fine. If I use > MHD_USE_AUTO it hangs as the system detects epoll() and uses it. > > Regards, > Denis > > -Original Message- > From: libmicrohttpd > [mailto:libmicrohttpd-bounces+denis.dowling=hsd.com...@gnu.org] On Behalf Of > Evgeny Grin > Sent: Monday, 4 September 2017 8:11 PM > To: libmicrohttpd@gnu.org > Subject: Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL > > Hi Denis, > > Thanks for report! > Minimal example could help a lot. > > In the meantime, could you provide more details how do you process incoming > data? > Do you use MHD_USE_ERROR_LOG flag for daemon? What's the output on stderr? > > Also you could build MHD from source, modify definition of PUT_SIZE in > src/testcurl/test_large_put.c and run 'make check' > > -- > Wishes, > Evgeny > > On 04.09.2017 7:35, Denis Dowling wrote: >> Hi All, >> >> >> >> I recently updated some code to use version 0.9.55 of libmicrohttpd. >> After the change I noticed that larger (~100kB) HTTPS requests are >> blocking forever. During the change I swapped from using the option >> MHD_USE_POLL to MHD_USE_AUTO. I can see from an strace that >> libmicrohttpd is now using epoll() for the dispatching. If I switch >> back to MHD_USE_POLL then everything works fine again. If I use HTTP >> requests then everything is fine so it is something to do with the >> combination of HTTPS and epoll(). Other options are >> MHD_USE_INTERNAL_POLLING_THREAD | MHD_USE_SUSPEND_RESUME | MHD_USE_TLS. >> >> >> >> Has anyone seen anything similar? I am running on CentOS7.3. The same >> issue also appears on Redhat7.4. I will try to modify one of the >> example applications in libmicrohttpd to demonstrate the issue. > > > > > The information contained in this message and any attachments is strictly > confidential and intended solely for the use of the intended recipient(s). > The copyright in this communication belongs to (HSD). This message and any > attachments are confidential and may be subject to legal or other > professional privilege. Any confidentiality or privilege is not waived or > lost because this e-mail has been sent to you by mistake. If you have > received this transmission in error, please notify HSD on +61 3 9875 5900 > immediately and destroy all copies of this e-mail. The contents of this email > message may not be quoted, copied, reproduced or published in part or in > whole, without the written authorisation of HSD. > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Semantics of MHD_run_from_select
Hi Martin, On 3 Sep 2017 16:08:22 +0200, Martin Steuer wrote: > Hi, > > What is the expected semantics of calling MHD_run_from_select with a MHD > configured to use epoll in combination with an application side event loop? > > I am diagnosing a problem with systemd-journal-remote which utilizes MHD > to receive journal entries from remote hosts. Currently, when > transferring a bigger journal, the remote side suddenly stops because it > seems not to receive ACKs anymore. This is because MHD_run_from_select > is left even if a connections' fd is still read-ready and > systemd-journal-remote will call epoll_wait on the corresponding fd. If > now the remote side does not send any more data epoll_wait will not be > left because of the edge triggered nature of the descriptor. > > With the internal event loop, MHD has access to the state of a > connection and thus can decide if it'll have to call epoll_wait. For > external event loops however that's not true. So my answer to the above > question would be: MHD_run_from_select should never be left if there is > any more data available to be processed by the application. For edge > triggered epoll this is the only safe way of doing it I can think of. > > As a proof of concept I tried a simple workaround (see attached patch) > which seems to solve the problem (only works for reads), but I am sure > that this solution is neither complete nor nice. > > What's your opinion on this? Looks like you do not use MHD_get_timeout() to get timeout value for your external select(). If MHD has more data to send and any socket is still ready for sending in "epoll mode", you will get zero timeout value. This is effectively same result as you get in "select mode" - select() function will return immediately when any socket is already ready. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Semantics of MHD_run_from_select
Hi Martin, I've updated doxy in git master to make usage of MHD_get_timeout() more clearly necessary. Looking on systemd code, I think that simplest way to correct systemd behaviour is to add special timerfd to polling set and update timer after each run of MHD. -- Best Wishes, Evgeny Grin On 07.09.2017 23:46, Martin Steuer wrote: > Thanks Evgeny, > > I verified the results from MHD_get_timeout and they correctly indicate > not to wait. I will have a look if systemd's code can be adapted to use > this. > > Martin > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL
Hi Denis, Problem was fixed in git master. Could you recheck on your side? -- Best Wishes, Evgeny Grin On 05.09.2017 23:22, Evgeny Grin wrote: > Hi Denis, > > Reproduced on CentOS 7 with current git master. > Will look into it. > > -- Best Wishes, Evgeny Grin On 05.09.2017 4:54, Denis Dowling wrote: >> Hi Evgeny, >> >> It is actually the GET that is causing the problem. >> >> I have been able to duplicate as follows: >> - Testing on CentOS7.3.1611. Everything up to date according to yum. >> - Start with a clean unpack of libmicrohttpd-0.9.55 >> - Edit the https_fileserver_example.c file and change the >> MHD_USE_THREAD_PER_CONNECTION option to MHD_USE_EPOLL. Leave the other >> options as they are. >> - make https_fileserver_example >> - In the same directory create 2 files of different sizes (1kB and 100kB) >> dd if=/dev/urandom of=small.dat bs=1024 count=1 >> dd if=/dev/urandom of=big.dat bs=1024 count=100 >> - Start https_fileserver_example >>./https_fileserver_example >> - In another window use wget to grab both files >> wget --no-check-certificate https://localhost:/small.dat >> wget --no-check-certificate https://localhost:/big.dat >> >> In my case the big.dat download freezes at 16384 bytes downloaded. If I >> change from MHD_USE_EPOLL to MHD_USE_POLL it works fine. If I use >> MHD_USE_AUTO it hangs as the system detects epoll() and uses it. >> >> Regards, >> Denis >> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL
Hi Denis, Thanks for confirming. -- Best Wishes, Evgeny Grin On 11.09.2017 3:29, Denis Dowling wrote: > Hi Evgeny, > > I have checked my application with the latest git master version of > libmicrohttpd and your change fixes the epoll lockup problem. > > Thanks for your help. > > Regards, > Denis > > -Original Message- > From: libmicrohttpd > [mailto:libmicrohttpd-bounces+denis.dowling=hsd.com...@gnu.org] On Behalf Of > Evgeny Grin > Sent: Saturday, 9 September 2017 5:08 AM > To: libmicrohttpd@gnu.org > Subject: Re: [libmicrohttpd] libmicrohttpd 0.9.55 https and MHD_USE_EPOLL > > Hi Denis, > > Problem was fixed in git master. > Could you recheck on your side? > > -- > Best Wishes, > Evgeny Grin > > On 05.09.2017 23:22, Evgeny Grin wrote: >> Hi Denis, >> >> Reproduced on CentOS 7 with current git master. >> Will look into it. >> >> -- Best Wishes, Evgeny Grin On 05.09.2017 4:54, Denis Dowling wrote: >>> Hi Evgeny, >>> >>> It is actually the GET that is causing the problem. >>> >>> I have been able to duplicate as follows: >>> - Testing on CentOS7.3.1611. Everything up to date according to yum. >>> - Start with a clean unpack of libmicrohttpd-0.9.55 >>> - Edit the https_fileserver_example.c file and change the >>> MHD_USE_THREAD_PER_CONNECTION option to MHD_USE_EPOLL. Leave the other >>> options as they are. >>> - make https_fileserver_example >>> - In the same directory create 2 files of different sizes (1kB and 100kB) >>> dd if=/dev/urandom of=small.dat bs=1024 count=1 >>> dd if=/dev/urandom of=big.dat bs=1024 count=100 >>> - Start https_fileserver_example >>>./https_fileserver_example >>> - In another window use wget to grab both files >>> wget --no-check-certificate https://localhost:/small.dat >>> wget --no-check-certificate https://localhost:/big.dat >>> >>> In my case the big.dat download freezes at 16384 bytes downloaded. If I >>> change from MHD_USE_EPOLL to MHD_USE_POLL it works fine. If I use >>> MHD_USE_AUTO it hangs as the system detects epoll() and uses it. >>> >>> Regards, >>> Denis >>> > > > > > The information contained in this message and any attachments is strictly > confidential and intended solely for the use of the intended recipient(s). > The copyright in this communication belongs to (HSD). This message and any > attachments are confidential and may be subject to legal or other > professional privilege. Any confidentiality or privilege is not waived or > lost because this e-mail has been sent to you by mistake. If you have > received this transmission in error, please notify HSD on +61 3 9875 5900 > immediately and destroy all copies of this e-mail. The contents of this email > message may not be quoted, copied, reproduced or published in part or in > whole, without the written authorisation of HSD. > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 20.09.2017 12:16, Svein Olav Bjerkeset wrote:
> I am trying to build libmicrohttpd-0.9.55 with support for https on
> RHEL5. As I can not even get the demo_https server to work (server demo
> which uses plain http, works as intended), there must be something wrong
> with libmicrohttpd itself or one or more of the libraries it uses.
>
> Running the demo_https server, it seems to start just fine. However when
> I try to use curl to connect, it just hangs for a long time after which
> it comes back with "curl: (52) Empty reply from server".
>
> I does not seem like the "generate_page" callback function is ever
> called. I have tried to add the following statement as the first
> statement, but I see no output:
>
> printf ("U=%s, M=%s\n", url, method);
>
> Occasionally, I get a response, with part of the response headers
> garbled, indicating that memory was overwritten somewhere. In this case
> curl still hangs after receiving the response and I do get output from
> the printf-statement above.
>
> As RHEL5 are missing a lot of libraries, or have outdated versions, I
> have built most of the dependent libraries myself. Here is a list of the
> libraries I have built and linked with microhttpd, and the versions I used:
>
> gnutls: 3.6.0 (also tried 3.5.15 with same result)
> gcrypt: 1.8.0
> libgpg-error: 1.27
> p11-kit: 0.23.2
> tasn1: 4.12
> nettle: 3.3
> ffi: 3.2.1
> gmp: 6.1.2
>
> Any idea on where to start tracking down this issue?
>
> Best regards,
> Svein Olav Bjerkeset
Hi Svein,
First of all, could you use another name in email? Otherwise it may
confuse other users.
Did you use curl on the same machine?
If you use curl came from RHEL5 repo than your curl could be outdated or
could be linked with outdated SSL lib.
As first step I suggest to check connection from another machine with
modern curl. Also try wget.
Broken response headers may indicate broken TLS lib or broken TLS decoding.
--
Best Wishes,
Evgeny Grin
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 25.09.2017 9:55, microhttpd wrote:
> Den 2017-09-21 15:33, skrev Evgeny Grin:
>> On 20.09.2017 12:16, Svein Olav Bjerkeset wrote:
>>> I am trying to build libmicrohttpd-0.9.55 with support for https on
>>> RHEL5. As I can not even get the demo_https server to work (server demo
>>> which uses plain http, works as intended), there must be something wrong
>>> with libmicrohttpd itself or one or more of the libraries it uses.
>>>
>>> Running the demo_https server, it seems to start just fine. However when
>>> I try to use curl to connect, it just hangs for a long time after which
>>> it comes back with "curl: (52) Empty reply from server".
>>>
>>> I does not seem like the "generate_page" callback function is ever
>>> called. I have tried to add the following statement as the first
>>> statement, but I see no output:
>>>
>>> printf ("U=%s, M=%s\n", url, method);
>>>
>>> Occasionally, I get a response, with part of the response headers
>>> garbled, indicating that memory was overwritten somewhere. In this case
>>> curl still hangs after receiving the response and I do get output from
>>> the printf-statement above.
>>>
>>> As RHEL5 are missing a lot of libraries, or have outdated versions, I
>>> have built most of the dependent libraries myself. Here is a list of the
>>> libraries I have built and linked with microhttpd, and the versions I
>>> used:
>>>
>>> gnutls: 3.6.0 (also tried 3.5.15 with same result)
>>> gcrypt: 1.8.0
>>> libgpg-error: 1.27
>>> p11-kit: 0.23.2
>>> tasn1: 4.12
>>> nettle: 3.3
>>> ffi: 3.2.1
>>> gmp: 6.1.2
>>>
>>> Any idea on where to start tracking down this issue?
>>>
>>> Best regards,
>>> Svein Olav Bjerkeset
>>
>> Hi Svein,
>>
>> First of all, could you use another name in email? Otherwise it may
>> confuse other users.
>>
>> Did you use curl on the same machine?
>> If you use curl came from RHEL5 repo than your curl could be outdated or
>> could be linked with outdated SSL lib.
>> As first step I suggest to check connection from another machine with
>> modern curl. Also try wget.
>>
>> Broken response headers may indicate broken TLS lib or broken TLS
>> decoding.
>
> Sorry for the email name. It should be corrected now.
Still the same: microhttpd
> I have tried both curl and whether, and they both fail. The results are
> the same for local and remote connections. I have tried from a RHEL 6.9
> server with modern curl/wget and it still fails.
Latest git contains some fixes for epoll modes.
Could you fetch source from official git on other platform (autoconf on
RHEL5 is too old), run "autoreconf -i && ./configure && make dist-gzip".
Then build from resulted tarball on your RHEL5 machine?
Most probably GnuTLS is not yet well polished on your machine.
For testing connectivity I'd recommend to use RHEL/CentOS 7 or latest
Debian. Also same platforms are recommended to fetch sources from git.
--
Best Wishes,
Evgeny Grin
signature.asc
Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 27.09.2017 10:37, Svein Olav Bjerkeset wrote: >> Latest git contains some fixes for epoll modes. >> Could you fetch source from official git on other platform (autoconf on >> RHEL5 is too old), run "autoreconf -i && ./configure && make dist-gzip". >> Then build from resulted tarball on your RHEL5 machine? >> >> Most probably GnuTLS is not yet well polished on your machine. >> >> For testing connectivity I'd recommend to use RHEL/CentOS 7 or latest >> Debian. Also same platforms are recommended to fetch sources from git. > > I presume it was libmicrohhtpd you wanted me to check out from git and > not one of the support libraries? I have now tried that, without any > luck. It still behaves as before. I have also tried to use curl and wget > from a RHEL7 server as suggested. Results are still the same. > > Here is the configure command I used: > # export CPPFLAGS="-I/opt/microhttpd/include" > # export LDFLAGS="-L/opt/microhttpd/lib64 -Wl,-rpath,/opt/microhttpd/lib64" > # ./configure --prefix=/opt/microhttpd --libdir=/opt/microhttpd/lib64 > --with-gnutls --enable-https=yes --with-libcurl=/opt/svv-dev > > (in /opt/svv-dev is a modern curl, version 7.55.1) > > I also tried --disable-epoll without any noticable change. GNU libmicrohttpd use libcurl only for testing. So, if you already have latest libcurl, you can run "make check" with MHD. But I suspect that problem comes from outdated RHEL5 GnuTLS lib. MHD on git master is able to use modern GnuTLS without gcrypt. As next step I suggest to build latest GnuTLS on RHEL5 and build MHD with modern GnuTLS. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 29.09.2017 17:47, Svein Olav Bjerkeset wrote: >> Great. Please inform the developer mailinglists of GNU nettle and GnuTLS >> about the regression(s). > > I will post about the segfault problem in GnuTLS 3.6.0 on the GnuTLS list. > > Regarding the dowgrade from 3.5.15 to 3.3.26, I really do not know > wheter the problem is with the newer GnuTLS or the way microhttpd uses it. > > As for nettle, I have no reason to belive either version has a bug, but > GnuTLS 3.3.26 required 2.7.1. Tried to reproduce on CentOS 5.11. Seems that nettle require newer GMP lib than comes with system. Do not think that it is very practical to have so many rebuilt libs to run something. MHD itself works fine, so I believe that problem is on GnuTLS side. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 29.09.2017 21:17, Svein Olav Bjerkeset wrote: >> Tried to reproduce on CentOS 5.11. >> Seems that nettle require newer GMP lib than comes with system. > > I built GMP 6.1.2 in the setup that finally worked. > > As I ended up building almost all the dependencies, except for basic > stuff like posix threads, it would not come as a big surprise to me if > the combination of newest MHD and GnuTLS would fail in the same way on > more modern platforms as well. > > If anyone would like a copy of my build environment for further testing, > please let me know. I will settle for the working version I have got at > this point as far as RHEL5 is concerned. If you could share scripts for rebuild or exact build steps - I'll try to reproduce. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 29.09.2017 23:02, Svein Olav Bjerkeset wrote: >> If you could share scripts for rebuild or exact build steps - I'll try >> to reproduce. > > Build script is attached. It requires the source files to be unpacked in > the build root. > > I just tried it on a new CentOS 5 vm, and now it works, even with > 3.6.0/2.7.1. In other words, I am unable to reproduce the error on this vm. Did your RHEL5 machine has all latest 5-series updates, including latest RHEL5 kernel? Or may be one of rebuilt libs was compiled against new lib, but have missing '-rpath' so used at runtime with wrong lib? This could explain broken data and crashes. -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5
On 30.09.2017 20:12, Svein Olav Bjerkeset wrote: >> Great. Please inform the developer mailinglists of GNU nettle and GnuTLS >> about the regression(s). > > Got the following response on the GnuTLS devel list: > >>> That seems to be the same case as the issue fixed by the following >>> merge request: >>> https://gitlab.com/gnutls/gnutls/merge_requests/491 > Thanks for informing! Did you try GnuTLS version from git to get everything working? -- Best Wishes, Evgeny Grin signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] using other TLS libraries
Hi Gauthier, I've done impressive work. We have very similar plans for integrated support for multiple TLS backends. However, I believe that we should start porting on updated MHD API (see microhttpd2.h). One of the goals of new API is to make MHD more TLS-lib independent. In the meantime, you could criticize new API and propose enhancements. -- Best Wishes, Evgeny Grin On 18.10.2017 17:01, Gauthier Haderer wrote: > Hello, > > I recently made changes to abstract the TLS API needed by MHD. And I > added support for OpenSSL. Could someone review my changes and tell me > if this is something that could be part of an official version? > > You will find more information here: > https://gnunet.org/bugs/view.php?id=4917#c12475 > > Thank you, > > > Gauthier signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Semantics of MHD_run_from_select
It was fixed in git master. -- Best Wishes, Evgeny Grin 28.10.2017, 00:09, "Martin Steuer" : > Hi Evgeny, > > After a lot of delay I reported the problem to systemd with your > suggested solution [1]. However now I noticed that for SSL connections > the initial problem still persisted. > I see that under this condition MHD_get_timeout always returns MHD_NO. > > Do you have an idea why this might happen? > > Thanks again, > Martin > > [1] https://github.com/systemd/systemd/issues/7130
Re: [libmicrohttpd] Please add support for RFC7616 (Digest auth with sha-2 instead of md5)
Hi Tim, Nice idea. We could add it to our wish list and implement it when time permits. Patches are welcome and could speed up things. -- Best Wishes, Evgeny Grin 28.10.2017, 21:44, "Tim Rühsen" : > Hi, > > curl just added support for RFC7616, and I just opened an issue for Wget2 as > well. > > Since wget2 uses MHD for it's test suite, support for that RFC in MHD would be > great. > > Basically, it's just sha-2 instead of md5 hashing, so should be straight > forward. > > WDYT ? > > Regards, Tim
Re: [libmicrohttpd] macOS build failure for libmicrohttpd 0.9.56
Thanks for report! Fixed build with optimisation for size. -- Best Wishes, Evgeny Grin 25.11.2017 12:52, ilove zfs пишет: > Full build log: > https://gist.github.com/ilovezfs/d43a15376aef661fcb91ee145f7a9841 > > The error is > > libtool: compile: /usr/bin/clang -DHAVE_CONFIG_H -I. -I../.. > -I../../src/include -I../../src/microhttpd -DBUILDING_MHD_LIB=1 > -F/usr/local/Frameworks -fvisibility=hidden -pthread -no-cpp-precomp > -fno-common -Os -w -pipe -march=native -mmacosx-version-min=10.11 > -fno-strict-aliasing -c mhd_str.c -fno-common -DPIC -o > .libs/libmicrohttpd_la-mhd_str.o > mhd_str.c:721:47: error: use of undeclared identifier 'todigitvalue'; > did you mean 'toxdigitvalue'? > toxdigitvalue : todigitvalue; > ^~~~ > toxdigitvalue > mhd_str.c:191:1: note: 'toxdigitvalue' declared here > toxdigitvalue (char c) > ^ > 1 error generated. > make[2]: *** [libmicrohttpd_la-mhd_str.lo] Error 1 > make[1]: *** [install-recursive] Error 1 > make: *** [install-recursive] Error 1 > > Ref: https://github.com/Homebrew/homebrew-core/pull/21032 signature.asc Description: OpenPGP digital signature
[libmicrohttpd] GNU libmicrohttpd 0.9.57 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.57. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). Most noticeable changes since version 0.9.55: * Huge internal refactoring of handling connections with most improvement for HTTPS connections. * Added support for debug asserts (configure parameter '--enable-asserts'). * Added ability to bind to free available port with automatic detection of resulting port number (useful for automated testing of applications). * Dropped support for Symbian (was broken a long time ago). * Portability fixes for GNU Hurd. * Dropped dependency on libgcrypt with new GnuTLS versions. * Added feature check for MHD ability to suppress SIGPIPE signals on MHD connections. * Added internal check for using correct by application when providing response. WARNING: badly designed apps may stop working. * Fixed sending and receiving large data over HTTPS in epoll mode. * Fixed most compiler warnings for '-Wall -Wextra'. * Used chunks for sendfile() transfers, improved distribution of CPU resources per connection. * Ability to build demos without libmagic. * Added support for sendfile() on FreeBSD, Solaris/OpenIndiana and Darwin with additional optimizations on FreeBSD 11. * Fixed and improved GnuTLS detection in configure. * Added MHD_free() for freeing MHD-allocated memory on application side, useful for W32 with several CRT libs. * Added special tricks in configure to support Solaris/OpenIndiana. * Other portability fixes. * Minor fixes. * Various multi-thread improvements. This is bugfix release. Most noticeable changes since version 0.9.56: * Fixed builds with compiler optimization for size. * Corrected names in W32 DLL resources. * Improved Cygwin support. * Improved and clarified configure summary message. * Added configure parameter to disable sendfile() usage. * Removed need on all platforms to explicitly initialize libmicrohttpd. You can download GNU libmicrohttpd from * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at https://www.gnu.org/software/libmicrohttpd/. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] GNU libmicrohttpd 0.9.57 released
Hi Markus,
Seems that current version is more correct.
Notification callback is called while connection is still in cleanup
(not fully closed). Supposed that application will do cleanup related to
this specific connection, and, after application-side cleanup, MHD
preform cleanup on its side.
Number of connection must decremented only after connection is cleaned
and removed from list of connections.
MHD cleanup should be preformed only after cleanup on application side.
For single threaded MHD, you can use interpret "1" for number of
connection as "closing last connection". For multi-threaded MHD any
assumptions about number current connections are not thread-safe.
I'll check whether documentation should be clarified.
--
Wishes,
Evgeny
01.12.2017 21:29, Markus Doppelbauer пишет:
> Hello,
>
> I have a problem with this commit:
> https://github.com/Karlson2k/libmicrohttpd/commit/b93ddc6b5ebb76fbe0a4beb73901ad2c4c7e8ca8#diff-c5e0aecb44767e9dda5a88f49b24a3c5
>
> void NotifyConnectionCallback( ... )
> {
> const union MHD_DaemonInfo *daemoninfo = MHD_get_daemon_info( daemon,
> MHD_DAEMON_INFO_CURRENT_CONNECTIONS );
> if( toe == MHD_CONNECTION_NOTIFY_CLOSED && daemoninfo->num_connections
> == 0 )
> {
> ... do something if all connections are closed ...
> }
> }
>
> "daemoninfo->num_connections" never becomes zero - because it is
> decremented after the "MHD_CONNECTION_NOTIFY_CLOSED" event.
>
> Thanks a lot
> Markus
>
>
>
> Am Montag, den 27.11.2017, 23:57 +0300 schrieb Evgeny Grin:
>> Dear all,
>>
>>
>> I'm glad to announce the release of GNU libmicrohttpd 0.9.57.
>>
>> GNU libmicrohttpd is a small C library that is supposed to make it easy
>> to run an HTTP server as part of another application. GNU libmicrohttpd
>> is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU
>> libmicrohttpd is fast, portable and has a simple API and (without TLS
>> support and other optional features) a small binary size (~32k).
>>
>> Most noticeable changes since version 0.9.55:
>> * Huge internal refactoring of handling connections with most
>> improvement for HTTPS connections.
>> * Added support for debug asserts (configure parameter
>> '--enable-asserts').
>> * Added ability to bind to free available port with automatic detection
>> of resulting port number (useful for automated testing of
>> applications).
>> * Dropped support for Symbian (was broken a long time ago).
>> * Portability fixes for GNU Hurd.
>> * Dropped dependency on libgcrypt with new GnuTLS versions.
>> * Added feature check for MHD ability to suppress SIGPIPE signals on
>> MHD connections.
>> * Added internal check for using correct by application when providing
>> response. WARNING: badly designed apps may stop working.
>> * Fixed sending and receiving large data over HTTPS in epoll mode.
>> * Fixed most compiler warnings for '-Wall -Wextra'.
>> * Used chunks for sendfile() transfers, improved distribution of
>> CPU resources per connection.
>> * Ability to build demos without libmagic.
>> * Added support for sendfile() on FreeBSD, Solaris/OpenIndiana and
>> Darwin with additional optimizations on FreeBSD 11.
>> * Fixed and improved GnuTLS detection in configure.
>> * Added MHD_free() for freeing MHD-allocated memory on application
>> side, useful for W32 with several CRT libs.
>> * Added special tricks in configure to support Solaris/OpenIndiana.
>> * Other portability fixes.
>> * Minor fixes.
>> * Various multi-thread improvements.
>>
>> This is bugfix release.
>> Most noticeable changes since version 0.9.56:
>> * Fixed builds with compiler optimization for size.
>> * Corrected names in W32 DLL resources.
>> * Improved Cygwin support.
>> * Improved and clarified configure summary message.
>> * Added configure parameter to disable sendfile() usage.
>> * Removed need on all platforms to explicitly initialize libmicrohttpd.
>>
>>
>> You can download GNU libmicrohttpd from
>>
>> * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors.
>> * Our git repository at https://gnunet.org/git/libmicrohttpd.git
>>
>> Please report bugs to our bugtracker at https://gnunet.org/bugs/.
>>
>> The documentation (including a reference manual and tutorial) can be
>> found at https://www.gnu.org/software/libmicrohttpd/.
>>
>>
signature.asc
Description: OpenPGP digital signature
[libmicrohttpd] GNU libmicrohttpd 0.9.58 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.58. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). This is bugfix release. Changes since version 0.9.57: * Better support for automatic port detection for sockets provided by MHD_OPTION_LISTEN_SOCKET option. * Documentation installation fixes. * Removed usage of 3DES mode from test-suite to avoid test failures on modern platforms. * Minor compiler warning fix. You can download GNU libmicrohttpd from * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at https://www.gnu.org/software/libmicrohttpd/. -- Wishes, Evgeny
Re: [libmicrohttpd] Release 0.9.58, but configuration summary is 0.9.55
Hi, Make sure that you try sources from official FTP https://ftp.gnu.org/gnu/libmicrohttpd/ I've just rechecked and it is 0.9.58: configure: GNU libmicrohttpd 0.9.58 Configuration Summary: I tried both https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.58.tar.gz and https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz with same result. -- Best Wishes, Evgeny Grin 08.12.2017 5:20, silvioprog пишет: > Hi dudes. > > I've upgraded my MHD copy and after configuring it I've got the > following summary: > > === > *configure: libmicrohttpd 0.9.55 Configuration Summary:* > Cross-compiling: no > Operating System: mingw32 > Threading lib: w32 > Use thread names: no > Inter-thread comm: socketpair > libcurl (testing): no, many unit tests will not run > Target directory: /mingw32 > Shutdown of listening socket > trigger select: no > Use debug asserts: no > Messages: yes > Basic auth.: yes > Digest auth.: yes > HTTP "Upgrade": yes > Postproc: yes > HTTPS support: no (disabled) > poll support: no > epoll support: no > sendfile used: no > build docs: no > build examples: no > > configure: > License : LGPL or eCos > === > > I might be wrong, but it should report *0.9.58*. 😶 > > -- > Silvio Clécio signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Compile Error
Hi Jong How do you define 'response' and 'ethbuf08'? 'reponse' must be defined as 'struct MHD_Response *response;' 'ethbuf08' should be defined list some array or pointer, for example 'char ethbuf08[8];' You provided not enough information to give you precise answer. -- Best Wishes, Evgeny Grin 07.12.2017 23:23, Jong Han wrote: > Hello, > > Please check thid error Message > > > sync.c:268:9: error: cannot convert to a pointer type > response = MHD_create_response_from_buffer (8, (void > *)ethbuf08, MHD_RESPMEM_PERSISTENT); > === > > I don't know why this is a problem. > > Thanks. > > Jong Han > jongha...@gmail.com <mailto:jongha...@gmail.com> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Release 0.9.58, but configuration summary is 0.9.55
Hi Silvio, If I get you correctly, you are asking about proper way to run configure. I did not use AX_SUBDIRS_CONFIGURE. I could tell you that if you are using tarball, then you do not need to run autoreconf or bootstrap as configure is already generated. -- Best Wishes, Evgeny Grin 08.12.2017 19:46, silvioprog пишет: > Hum... so it seems a problem related to my environment. :-/ I'm going > to check again it tonight. > > I'm configuring MHD in my project with AX_SUBDIRS_CONFIGURE ...: > > AM_COND_IF([WINDOWS], [LDFLAGS="-static -lrpcrt4"], [CFLAGS=""]) # > I've declared 'CFLAGS=""' just because when I build my project > removing it I've got a crash in my app initialization and I can't > understand why :-( > > AX_SUBDIRS_CONFIGURE([libmicrohttpd], [[--enable-static=yes], > [--enable-shared=no], > [--disable-doc], > [--disable-examples], > [--disable-curl], > [--enable-https=no], > [--enable-asserts=no]]) > > ... because I want to distribute only one DLL file, without any extra > DLL files > (libgcc_s_dw2-1.dll, libwinpthread-1.dll, libmicrohttpd-12.dll etc.). > Do you think I can use AX_SUBDIRS_CONFIGURE() to configure MHD? I'm > not sure it is runs the MHD's bootstrap or call something like autoreconf. > > > On Fri, Dec 8, 2017 at 5:03 AM, Evgeny Grin <mailto:k...@yandex.ru>> wrote: > > Hi, > > Make sure that you try sources from official FTP > https://ftp.gnu.org/gnu/libmicrohttpd/ > <https://ftp.gnu.org/gnu/libmicrohttpd/> > > I've just rechecked and it is 0.9.58: > configure: GNU libmicrohttpd 0.9.58 Configuration Summary: > > I tried both > https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.58.tar.gz > <https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.58.tar.gz> > and > https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz > <https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz> > with same result. > > -- > Best Wishes, > Evgeny Grin > > 08.12.2017 5:20, silvioprog пишет: >> Hi dudes. >> >> I've upgraded my MHD copy and after configuring it I've got the >> following summary: >> >> === >> *configure: libmicrohttpd 0.9.55 Configuration Summary:* >> Cross-compiling: no >> Operating System: mingw32 >> Threading lib: w32 >> Use thread names: no >> Inter-thread comm: socketpair >> libcurl (testing): no, many unit tests will not run >> Target directory: /mingw32 >> Shutdown of listening socket >> trigger select: no >> Use debug asserts: no >> Messages: yes >> Basic auth.: yes >> Digest auth.: yes >> HTTP "Upgrade": yes >> Postproc: yes >> HTTPS support: no (disabled) >> poll support: no >> epoll support: no >> sendfile used: no >> build docs: no >> build examples: no >> >> configure: >> License : LGPL or eCos >> === >> >> I might be wrong, but it should report *0.9.58*. 😶 >> >> -- >> Silvio Clécio > > > > > -- > Silvio Clécio signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Release 0.9.58, but configuration summary is 0.9.55
For git submodule you need to call 'autoreconf -i' after each update. -- Best Wishes, Evgeny Grin 08.12.2017 20:21, silvioprog пишет: > Hi dude! :-) > > I'm using git sub-module upgrading it each new MHD release, however > tarball can be a better option with AX_SUBDIRS_CONFIGURE(). > > Thanks for the tip! :-) > > On Fri, Dec 8, 2017 at 2:15 PM, Evgeny Grin <mailto:k...@yandex.ru>> wrote: > > Hi Silvio, > > If I get you correctly, you are asking about proper way to run > configure. > I did not use AX_SUBDIRS_CONFIGURE. I could tell you that if you > are using tarball, then you do not need to run autoreconf or > bootstrap as configure is already generated. > > -- > Best Wishes, > Evgeny Grin > > 08.12.2017 19:46, silvioprog пишет: >> Hum... so it seems a problem related to my environment. :-/ I'm >> going to check again it tonight. >> >> I'm configuring MHD in my project with AX_SUBDIRS_CONFIGURE ...: >> >> AM_COND_IF([WINDOWS], [LDFLAGS="-static -lrpcrt4"], [CFLAGS=""]) >> # I've declared 'CFLAGS=""' just because when I build my project >> removing it I've got a crash in my app initialization and I can't >> understand why :-( >> >> AX_SUBDIRS_CONFIGURE([libmicrohttpd], [[--enable-static=yes], >> [--enable-shared=no], >> [--disable-doc], >> [--disable-examples], >> [--disable-curl], >> [--enable-https=no], >> [--enable-asserts=no]]) >> >> ... because I want to distribute only one DLL file, without any >> extra DLL files >> (libgcc_s_dw2-1.dll, libwinpthread-1.dll, libmicrohttpd-12.dll >> etc.). Do you think I can use AX_SUBDIRS_CONFIGURE() to configure >> MHD? I'm not sure it is runs the MHD's bootstrap or call >> something like autoreconf. >> >> >> On Fri, Dec 8, 2017 at 5:03 AM, Evgeny Grin > <mailto:k...@yandex.ru>> wrote: >> >> Hi, >> >> Make sure that you try sources from official FTP >> https://ftp.gnu.org/gnu/libmicrohttpd/ >> <https://ftp.gnu.org/gnu/libmicrohttpd/> >> >> I've just rechecked and it is 0.9.58: >> configure: GNU libmicrohttpd 0.9.58 Configuration Summary: >> >> I tried both >> https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.58.tar.gz >> <https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.58.tar.gz> >> and >> https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz >> <https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-latest.tar.gz> >> with same result. >> >> -- >> Best Wishes, >> Evgeny Grin >> >> 08.12.2017 5:20, silvioprog пишет: >>> Hi dudes. >>> >>> I've upgraded my MHD copy and after configuring it I've got >>> the following summary: >>> >>> === >>> *configure: libmicrohttpd 0.9.55 Configuration Summary:* >>> Cross-compiling: no >>> Operating System: mingw32 >>> Threading lib: w32 >>> Use thread names: no >>> Inter-thread comm: socketpair >>> libcurl (testing): no, many unit tests will not run >>> Target directory: /mingw32 >>> Shutdown of listening socket >>> trigger select: no >>> Use debug asserts: no >>> Messages: yes >>> Basic auth.: yes >>> Digest auth.: yes >>> HTTP "Upgrade": yes >>> Postproc: yes >>> HTTPS support: no (disabled) >>> poll support: no >>> epoll support: no >>> sendfile used: no >>> build docs: no >>> build examples: no >>> >>> configure: >>> License : LGPL or eCos >>> === >>> >>> I might be wrong, but it should report *0.9.58*. 😶 >>> >>> -- >>> Silvio Clécio >> >> >> -- >> Silvio Clécio > > > -- > Silvio Clécio signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Server reached connection limit. Closing inbound connection. how to increase the limits?
In addition, consider using MHD_OPTION_CONNECTION_TIMEOUT so old connection will be dropped automatically. -- Best Wishes,Evgeny Grin 20.01.2018, 20:53, "Christian Grothoff" :Hi Pascal,There are several "limits" that may apply.(1) MHD_OPTION_CONNECTION_LIMIT -- usually 1020 (or FD_SETSIZE - 4) islikely the first one you may hit. You can pass a different value toMHD_start_daemon() in the varargs;(2) FD_SETSIZE is another limit you are likely to hit if you use theselect() event loop; tricky to bump, you ought to switch to poll/epollinstead.(3) ulimit (-n) may also impose a 1024 limit on your processHappy hacking!ChristianOn 01/20/2018 08:56 AM, Menaxerius wrote: Hello, I'm using microlibhttpd for our Miningpool Server application. So far so good. But due to more and more active miners is reaching I start reaching this limit. Can someone explain me where those limit is set and how to increase it? Thanks in advance. Regards Pascal
Re: [libmicrohttpd] Unable to connect to libmicrohttpd in embedded system.
Hi Marcel,
Why do you think that you have some race condition? You are using single
thread, unlikely it could produce any internal race condition.
With recent versions of MHD, I suggest to use MHD_USE_AUTO.
In your case it could be MHD_USE_AUTO | MHD_USE_INTERNAL_POLLING_THREAD
| MHD_USE_DEBUG combination.
'daemon->connections_tail' should be NULL while MHD do not serve any
connection (only listening).
I suggest you to try to debug 'MHD_accept_connection ()' - this function
should accept new connection.
Alternatively, you could try to run any of provided examples to ensure
that MHD function properly.
--
Best Wishes,
Evgeny Grin
22.01.2018 1:09, Marcel Rutten пишет:
> Thanks, Silvio, it would be great if you could send me that working
> code. Probably saves a lot of time :-). My environment is an
> openembedded tree, as published by the manufacturer of the thermostat,
> back in 2012. Parts of it are really old, so sometimes I have to do
> quite some patching to backport newer code to that tree and toolchain.
> Nevertheless, your working code is probably fairly easy to build.
> libmicrohttpd wasn't so hard in terms of passing it through this
> toolchain. Getting it to work is a different story :-).
>
> In the meantime, I ploughed through a lot of routines, working my way
> up from the routines that have daemon->default_handler (which is
> called from ozwcp) as argument.
> I have tested the following options for the server:
>
> MHD_USE_EPOLL | MHD_USE_INTERNAL_POLLING_THREAD | MHD_USE_DEBUG
>
> This ends up in a race condition in MHD_epoll(), as called in
> MHD_polling_thread().
>
> MHD_USE_POLL | MHD_USE_INTERNAL_POLLING_THREAD | MHD_USE_DEBUG
>
> gives a race condition in MHD_poll()
>
> and finally:
> |
> MHD_USE_INTERNAL_POLLING_THREAD | MHD_USE_DEBUG
> This gives a race condition in MHD_select(). The race conditions at
> least explain why the client keeps trying, while nothing gets actually
> loaded. Only after killing the daemon, the connection is reset. I did
> some follow-up on the race condition in MHD_select(), and found that
> the connections in the daemon struct are not filled:
> (excerpt from internal_run_from_select() ):
> ... (FD_ISSET (ds, read_fd_set)) ) (void) MHD_accept_connection
> (daemon); if (0 == (daemon->options & MHD_USE_THREAD_PER_CONNECTION))
> { /* do not have a thread per connection, process all connections now
> */ prev = daemon->connections_tail; < prev = 0x00 while (NULL !=
> (pos = prev)) < pos is also 0x00, so the loop never starts,
> resulting in a new call to MHD_select(), and so on and on .. { prev =
> pos->prev; ds = pos->socket_fd; ...
> Both current and previous connections are NULL pointers, which is not
> good :-(, but it already narrows the issue down to a much smaller part
> of the daemon. So the next question is: where are these connections
> filled, and how do I check if it's done properly?
> |
>
> On 21 January 2018 at 22:28, silvioprog <mailto:silviop...@gmail.com>> wrote:
>
> Hello Marcel, welcome to the list!
>
> I used MHD on ARM in a small HTTP server testing for
> Android:
> https://lists.gnu.org/archive/html/libmicrohttpd/2017-02/msg00014.html
> <https://lists.gnu.org/archive/html/libmicrohttpd/2017-02/msg00014.html>
> . This link contains an attached picture showing the results. I
> think I have the sources in my old machine.
>
> I don't know if my environment can be useful for you, but I built
> MHD for ARM via integrated Android Studio's CMake, and used MHD in
> Java project with JNI. I can search and send sources if they could
> be useful for you ...
>
> On Sun, Jan 21, 2018 at 7:48 AM, Marcel Rutten
> mailto:mcmrut...@gmail.com>> wrote:
>
> Hi, I'm new to this list, and I have been trying to get
> libmicrohttpd to work on a linux-based smart thermostat.
>
> To no avail, so far. I should say that I'm not an expert on
> web connections, but fairly fluent in C.
>
> The thermostat has a Freescale imx 27 processor, (ARM-926 EJS)
> and has an openembedded image, developed around 2011, with
> linux kernel 2.6.26. The reason I want to get libmicrohttpd
> working on it, is because it features a zwave controller,
> which may be used to control TRVs (Thermostatic Radiator
> Valves), a feature which is not available in the standard
> firmware of the thermostat. The zwave interface can be
> operated through openzwave, and its web interface ozwcp (open
> zwave control panel). ozwcp is linked against libmicrohttpd to
> provide the web interface.
>
&g
Re: [libmicrohttpd] Download page status: OFF
Hi Silvio, Works fine for me. Both URLs.Note: FTP is deprecated. -- Best Wishes,Evgeny Grin 15.08.2018, 07:25, "silvioprog" :Hi MHD users. It seems the download page is offline. Could you test it?: https://ftp.gnu.org/gnu/libmicrohttpd/ the to the FTP: ftp://ftp.gnu.org/gnu/libmicrohttpd/ I've tried to access the page from two different browsers (and DNSs), but reached timeout in both. :-( Thank you! --Silvio Clécio
Re: [libmicrohttpd] Hang with MHD_OPTION_THREAD_POOL_SIZE on Windows
Hi Jonathan, Why do you this that flag WSA_FLAG_OVERLAPPED should fix something?Looking at documentation, it should change nothing to non-overlapped I/O.https://docs.microsoft.com/en-us/windows/desktop/winsock/overlapped-i-o-and-event-objects-2 -- Best Wishes,Evgeny Grin 08.12.2018, 05:19, "Jonathan McDougall" :I've been having problems with MHD_OPTION_THREAD_POOL_SIZE on Windows.I started with 0.9.55 from vcpkg, but then changed the portfile to use0.9.61 instead. Both give the same results.Connections to the server hang intermittently. More threads in the poolseems to make it hang more often. I think I was able to trace it back toa blocking call to accept() in MHD_accept_connection().What I'm seeing is that all threads block on a select() call inMHD_select(). When a connection comes in, *multiple threads* wake upat the same time and end up in MHD_accept_connection(). Some of themseem to then block on accept().Repeated calls of curl eventually works, but it can take a dozen callsbefore one goes through with 8 threads in the pool. Threads that areblocked in accept() seem to be able to eventually wake up and accept aconnection.I'm attaching a short repro below. Executing something like'curl http://127.0.0.1:8080/a' usually hangs. There's nothing special inthe code, I ripped it out of test_get.c. In fact, test_get.c itselfhangs in testMultithreadedPoolGet().I'm using Visual Studio 2019 Preview (16.0 P1) on Windows 10. I'mreproducing this on both x86 and x64.#include #include int echo(void* cls, struct MHD_Connection* connection, const char* url,const char* method, const char* version,const char* upload_data, size_t* upload_data_size,void** unused){static int ptr;struct MHD_Response* response;int ret;if (&ptr != *unused){*unused = &ptr;return MHD_YES;}*unused = NULL;response = MHD_create_response_from_buffer(strlen(url), (void*)url, MHD_RESPMEM_MUST_COPY);ret = MHD_queue_response(connection, MHD_HTTP_OK, response);MHD_destroy_response(response);return ret;}int main(){const unsigned int count = 8;struct MHD_Daemon* d = MHD_start_daemon(MHD_USE_INTERNAL_POLLING_THREAD,8080, NULL, NULL, &echo, NULL,MHD_OPTION_THREAD_POOL_SIZE, count,MHD_OPTION_END);getc(stdin);MHD_stop_daemon(d);return 0;}-- Jonathan McDougall
Re: [libmicrohttpd] [Windows] Build fails on MinGW-w64 (0.9.62)
Hi, It was fixed already in Git master by 1616caa17764c878bc18c6344a0f7f60b2cb596c -- Best Wishes,Evgeny Grin 28.12.2018, 09:08, "silvioprog" :Hello, I've tried to compile the latest stable version on MinGW-w64, but it raises the following error at build time (cut relevant parts only): wget --continue --content-disposition https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.62.tar.gztar -zxvf libmicrohttpd-0.9.62.tar.gzcd libmicrohttpd-0.9.62/ && mkdir build && cd build/../configure --enable-static=yes --enable-shared=no --enable-messages=yes --enable-https=yes --enable-asserts=no --enable-coverage=no --disable-httpupgrade --disable-dauth --disable-doc --disable-examples --disable-curl ... configure: GNU libmicrohttpd 0.9.62 Configuration Summary: Target directory: /mingw32 Cross-compiling: no Operating System: mingw32 Shutdown of listening socket trigger select: no Inter-thread comm: socketpair poll support: yes epoll support: no sendfile used: no HTTPS support: yes (using libgnutls) Threading lib: w32 Use thread names: no Use debug asserts: no Messages: yes Gettext: yes Basic auth.: yes Digest auth.: no HTTP "Upgrade": no Postproc: yes Build docs: no Build examples: no Test with libcurl: no, many unit tests will not run configure: HTTPS subsystem configuration: License : LGPL only configure: WARNING: This will be a custom build with missing symbols. Do NOT use this build in a distribution. Building with these kinds of configure options is only for custom builds for embedded systems. Now the make: make ... ../../../src/microhttpd/daemon.c: In function 'thread_main_handle_connection':../../../src/microhttpd/daemon.c:1781:17: error: array type has incomplete element type 'struct pollfd' struct pollfd p[1 + EXTRA_SLOTS]; ^../../../src/microhttpd/daemon.c:1843:29: error: 'POLLIN' undeclared (first use in this function); did you mean 'POINT'? p[0].events = POLLIN; ^~ POINT../../../src/microhttpd/daemon.c:1843:29: note: each undeclared identifier is reported only once for each function it appears inIn file included from ../../../src/microhttpd/internal.h:72:0, from ../../../src/microhttpd/daemon.c:32:../../../src/microhttpd/mhd_sockets.h:340:27: warning: implicit declaration of function 'WSAPoll'; did you mean 'WSANtohl'? [-Wimplicit-function-declaration] # define MHD_sys_poll_ WSAPoll ^../../../src/microhttpd/daemon.c:1846:23: note: in expansion of macro 'MHD_sys_poll_' if (0 > MHD_sys_poll_ (p, ^../../../src/microhttpd/mhd_sockets.h:362:38: error: 'POLLRDBAND' undeclared (first use in this function); did you mean 'POLLIN'? # define MHD_POLL_EVENTS_ERR_DISC POLLRDBAND ^../../../src/microhttpd/daemon.c:2012:32: note: in expansion of macro 'MHD_POLL_EVENTS_ERR_DISC' p[0].events |= POLLIN | MHD_POLL_EVENTS_ERR_DISC; ^~~~../../../src/microhttpd/daemon.c:2015:23: error: 'POLLOUT' undeclared (first use in this function); did you mean 'POLLIN'? p[0].events |= POLLOUT | MHD_POLL_EVENTS_ERR_DISC; ^~~ POLLIN../../../src/microhttpd/daemon.c:2055:39: error: 'POLLERR' undeclared (first use in this function); did you mean 'POLLOUT'? (0 != (p[1].revents & (POLLERR | POLLHUP | POLLIN))) ) ^~~ POLLOUT../../../src/microhttpd/daemon.c:2055:49: error: 'POLLHUP' undeclared (first use in this function); did you mean 'POLLOUT'? (0 != (p[1].revents & (POLLERR | POLLHUP | POLLIN))) ) ^~~ POLLOUTIn file included from ../../../src/microhttpd/daemon.c:44:0:../../../src/microhttpd/daemon.c: In function 'MHD_poll_all':../../../src/microhttpd/daemon.c:3736:30: error: invalid application of 'sizeof' to incomplete type 'struct pollfd' sizeof (struct pollfd)); ^../../../src/microhttpd/mhd_compat.h:79:38: note: in definition of macro 'MHD_calloc_' #define MHD_calloc_(n,s) calloc((n),(s)) ^../../../src/microhttpd/daemon.c:3754:3: error: invalid use of undefined type 'struct pollfd' p[poll_server].fd = ls; ^../../../src/microhttpd/daemon.c:3754:3: error: dereferencing pointer to incomplete type 'struct pollfd'../../../s
Re: [libmicrohttpd] Binding to a specifc address on Windows
HI Daniel, Please clarify, whether do you want to use MHD on Windows natively or with MSys POSIX transformation layer?It is two different tasks. -- Best Wishes,Evgeny Grin 05.03.2019, 17:21, "Daniel Kraft" :Hi!I'm using libmicrohttpd indirectly through libjson-rpc-cpp. Currently,this starts the HTTP server to always listen on all interfaces. I wouldlike to change this so that the server binds only to localhost (or moregenerally to a specific address).According to the documentation, I can use MHD_OPTION_SOCK_ADDRESS toachieve this. It works fine for me on GNU/Linux, and someone from myteam was also able to compile the code (with some tweaks) on Windowswith MSYS.However, the option expects the binding address as "struct sockaddr",which as far as I can tell is POSIX and not standard C. Thus, I wonderif that is the correct way to bind microhttpd to a specific address onWindows (which microhttpd states to support officially). Is it, or am Imissing something here?Thank you very much!Yours,Daniel --https://www.domob.eu/OpenPGP: 1142 850E 6DFF 65BA 63D6 88A8 B249 2AC4 A733 0737Namecoin: id/domob -> https://nameid.org/?name=domob--3.6.0: Bar-Pri-Ran-Rog-Sam-Val-WizTo go: Arc-Cav-Hea-Kni-Mon-Tou
Re: [libmicrohttpd] From LGPL 3.0 to LGPL 2.1?
16.05.2019 0:28, Christian Grothoff wrote: > On 5/15/19 1:37 AM, silvioprog wrote: >> Hello friends. >> >> After updating my git, I found the following commit log: >> >> /"Updated README and COPYING/ >> /Note: library code is licensed under LGPLv2.1+ or eCOS terms// >> /Come testsuite programs are licensed under GPLv3 terms."/ >> >> Could you explain the main reasons? (Any advantages/disadvantages?) >> >> I would like to understand it because I'm going to upgrade MHD in my >> project. > > To clarify, this is not really a change in license, AFAIK it was just > clarified in the README. MHD has always been under LGPLv2.1+, and due to > certain requests we dual-licensed under GPL+eCOS exception some years > ago for the subset the code that doesn't touch GnuTLS. > > So an upgrade should not change anything for your project. > That's absolutely correct. Library code was always provided under dual LGPLv2.1+ and eCOS licenses. If you link MHD with GnuTLS, then you should use LGPLv2.1+ license. Separate licenses for testsuite programs do not change anything for library itself. LGPLv2.1+ licensed code could be modified and reused under LGPLv2.1+, LGPLv3+, GPLv2+ and GPLv3(+). LGPLv3+ licensed code could be modified and reused under LGPLv3+ and GPLv3(+) only. MHD is licensed under LGPLv2.1+ terms, which allow wider reuse of code. Additional eCOS license further increase flexibility. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Access to OSX machine?
Hi ng0, Check personal email for solution. -- Wishes, Evgeny 20.05.2019 17:22, n...@n0.is wrote: > Hi, > > I need access to an OSX machine for tracing syscalls done > by libmicrohttpd and written tests, and > analyzing them with dtrace or something equivalent to > ktruss / truss on OSX. > > Can anyone on this list provide me with temporary access > to an OSX machine capable of performing these tasks, > ideally 24/7 usable (it's difficult for me to plan when > exactly I will work on this, but if I have to I will. It's > just that access without time constrains would be easier). > > Cheers, > ng0 > > n...@n0.is transcribed 478 bytes: >> Hi all, >> >> I am ng0 (the listing says N. Gillmann but I prefer to go by ng0), >> and I was accepted into the Google Summer of Code program this >> year to work on libmicrohttpd. >> In short, my project is focused on syscall/setsockopt optmization on >> various platforms >> (Debian Linux, FreeBSD 12-RELEASE-p3, NetBSD-CURRENT), and working on the >> testsuite for the libmicrohttpd2 API. >> >> I just wanted to briefly introduce myself to the list. >> >> >> I look forward to working with you. >> >> >> Cheers, >> ng0 >> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [PATCH] Fix more 'address already in use' issues
Hi Tim, Thanks for patches. However whole testsuite is not designed to be run in parallel. Testsuite individual tests use all available CPU cores so when used in parallel, tests could produce unpredictable wrong results. -- Wishes, Evgeny 28.05.2019 11:51, Tim Rühsen wrote: > Three more tests fail in parallel 'make check'. The attached patch fixes > all three. > > Regards, Tim signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [PATCH] Fix test_upgrade/test_upgrade_tls running in parallel
On which system did you test it? Looks like MHD_FEATURE_AUTODETECT_BIND_PORT is not available on you OS, which is pretty strange as it should works on all modern platforms. -- Wishes, Evgeny 28.05.2019 11:43, Tim Rühsen wrote: > Hi, > > test_upgrade_tls always fails here (make check on multi-core). Attached > patch fixes it. > > Regards, Tim signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [PATCH] Fix test_upgrade/test_upgrade_tls running in parallel
Could you provide config.log, please? 28 мая 2019 г. 23:39:18 GMT+03:00, "Tim Rühsen" пишет: >On 28.05.19 21:07, Evgeny Grin wrote: >> On which system did you test it? >> Looks like MHD_FEATURE_AUTODETECT_BIND_PORT is not available on you >OS, >> which is pretty strange as it should works on all modern platforms. >> > >Debian unstable, daily updated. > >With those patches, all works for me, reproducible. (Even with parallel >make check). > >Regards, Tim -- Best wishes, Evgeny Grin
Re: [libmicrohttpd] [PATCH] Fix test_upgrade/test_upgrade_tls running in parallel
Hi Tim, Thanks for details. I'll fix this case. However, it is not advisable to use any -Werror= flags for configure, as it may break some tests. For example, if you will use -Werror=extra-semi clang flag then you will break most of compile and link tests. I'd recommend you to run configure without any -Werror flags, but redefine CGLAGS with -Werror parameters for make invocation. -- Wishes, Evgeny 29.05.2019 12:45, Tim Rühsen wrote: > On 5/28/19 9:07 PM, Evgeny Grin wrote: >> On which system did you test it? >> Looks like MHD_FEATURE_AUTODETECT_BIND_PORT is not available on you OS, >> which is pretty strange as it should works on all modern platforms. >> > MHD_config.h: > /* Define if you have usable `getsockname' function. */ > /* #undef MHD_USE_GETSOCKNAME */ > > config.log: > configure:18603: checking whether getsockname is declared > configure:18603: result: yes > configure:18629: checking for getsockname > configure:18629: result: yes > configure:18635: checking whether getsockname() is usable > ... > conftest.c: In function 'main': > conftest.c:165:3: error: implicit declaration of function 'close' > [-Werror=implicit-function-declaration] > 165 | close (sckt); > | ^ > conftest.c:165:3: warning: nested extern declaration of 'close' > [-Wnested-externs] > cc1: some warnings being treated as errors > configure:18730: $? = 1 > configure: program exited with status 1 > configure:18742: result: no > > > I am using > $ gcc-9 --version > gcc-9 (Debian 9.1.0-1) 9.1.0 > > with a lot of warning flags - though I don't use -Werror explicitly. > > It's your configure.ac code that doesn't play well here. > Why are you not just using "AC_CHECK_FUNCS([getsockname])" ? Or use > gnulib as portability layer ? > > Regards, Tim > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] HTTP and HTTPs parallel
Hello Sandra, Yes, you have to start two daemons, one for HTTP and another one for HTTPS. You can configure them with identical options, including callbacks. However, they will not share same port. Yes, you can enforce port sharing by MHD_OPTION_LISTENING_ADDRESS_REUSE, but it will be unpredictable which daemon will pick-up connection. In addition, it is not advisable to serve both types of traffic (HTTPS and HTTP) on the same port as it may be used as security weakness. -- Wishes, Evgeny 30.05.2019 16:53, Sandra Gilge wrote: > Hallo, > is ist possible to implement A libmicrohttpd WEB-Server that supports > HTTP AND HTTPs with one daemon? > I was able to setup a HTTP Webserver and also a HTTPS Web server. But > it then always answers only for HTTP or HTTPS but not both. > Do I have to call MHD_start_daemon twice with the different settings? > Thanks > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] [PATCH] Fix test_upgrade/test_upgrade_tls running in parallel
30.05.2019 16:45, Tim Rühsen wrote: >> Thanks for details. >> I'll fix this case. > Thank you ! Check updated git master, please. > >> However, it is not advisable to use any -Werror= flags for configure, as >> it may break some tests. >> For example, if you will use -Werror=extra-semi clang flag then you will >> break most of compile and link tests. >> I'd recommend you to run configure without any -Werror flags, but >> redefine CGLAGS with -Werror parameters for make invocation. >> > My CFLAGS are > > ac_configure_args = 'CFLAGS=-g -Og -Wall -Wextra -Wall -Wextra > -Wformat=2 -Walloc-zero -Walloca -Wbad-function-cast -Wcomments > -Wdate-time -Wdouble-promotion -Wduplicated-branches -Wduplicated-cond > -Werror-implicit-function-declaration -Wfloat-conversion -Wfloat-equal > -Wformat -Wformat-signedness -Wformat-truncation -Winit-self > -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations > -Wmissing-format-attribute -Wmissing-include-dirs -Wmissing-prototypes > -Wmultichar -Wnested-externs -Wnormalized -Wold-style-definition > -Woverlength-strings -Wpedantic -Wpointer-arith -Wredundant-decls > -Wscalar-storage-order -Wshift-overflow -Wstrict-prototypes > -Wstringop-overflow -Wswitch-default -Wswitch-enum -Wundef > -Wunused-const-variable -Wvariadic-macros -Wwrite-strings > -Warray-bounds=2 -Wnormalized=nfc -Wshift-overflow=2 > -Wunused-const-variable=2 -Wformat-overflow=2 -Wformat-truncation=1 > -Wstringop-overflow=2 -fdiagnostics-color=always -Wno-cast-function-type' > > So there is this -Werror-implicit-function-declaration. The CFLAGS are > automatically generated by using 'gcc -Q --help=warnings,C' and some > scripting within a m4 file. This command lists > -Werror-implicit-function-declaration for gcc-4.6 and higher. I wonder > if this is a gcc bug that I run into - since only warnings have been > requested. Looks like a typo in GCC built-in help description. Option '-Werror-implicit-function-declaration' (with minus sign) actually must by '-Werror=implicit-function-declaration' (with equal sign). When you request all warnings, this option is not filtered out by '-Werror=' template and used as error flag. -- Wishes, Evgney signature.asc Description: OpenPGP digital signature
[libmicrohttpd] GNU libmicrohttpd 0.9.64 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.64. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). Notable changes since version 0.9.63: * Extended API to support GET keys and values with binary zeros. * HTTP methods, headers and status codes were updated from official registries. * Improved speed of headers keys lookup by comparison length of keys as first step. * Fixed build for GNU Hurd. * Optimized MD5 calculation (used for Digest authentication), greatly improved speed on W32. * Improved autoassigned port detection on *BSD platforms. * Finally added ability to avoid SIGPIPE on Solaris 11.4, improved support for NetBSD 7+. * Rewrote SHA256 calculation from scratch to avoid change of LGPL version for MHD. Calculation speed improved. * Added flags MHD_USE_INSECURE_TLS_EARLY_DATA and MHD_USE_POST_HANDSHAKE_AUTH_SUPPORT to control TLS lib behaviour. * Fixed and improved MS VC support. * Tests in testsuite fixed and improved. Added more tests. * Improved detection of various functions in configure. * Various minor fixes in tests and examples. This release is signed with new key EA81 2DBE FA5A 7EF1 7DA8 F2C1 460A 317C 3326 D2AE You can download GNU libmicrohttpd from * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at https://www.gnu.org/software/libmicrohttpd/. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
[libmicrohttpd] GNU libmicrohttpd 0.9.65 released
Dear all, I'm glad to announce the release of GNU libmicrohttpd 0.9.65. GNU libmicrohttpd is a small C library that is supposed to make it easy to run an HTTP server as part of another application. GNU libmicrohttpd is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU libmicrohttpd is fast, portable and has a simple API and (without TLS support and other optional features) a small binary size (~32k). This is a bugfix release. Notable changes since version 0.9.64: * Largely reworked, fixed and updated code for managing of connections' memory pools. * Memory on W32 is cleared more securely. * Used less memory for connection due to reuse existing allocated buffers instead of allocation new additional buffers. * Better handled connection's memory shortage situation. * Error responses are sent now even if no buffer space is left available. You can download GNU libmicrohttpd from * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. * Our git repository at https://gnunet.org/git/libmicrohttpd.git Please report bugs to our bugtracker at https://gnunet.org/bugs/. The documentation (including a reference manual and tutorial) can be found at https://www.gnu.org/software/libmicrohttpd/. -- Wishes, Evgeny signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] GNU libmicrohttpd 0.9.65 released
Hi Tim, It was missing accidentally, thanks for pointing my attention on it. Tag was pushed. -- Wishes, Evgeny *From:* Tim Rühsen *Sent:* Tuesday, July 9, 2019, 14:10 UTC+3 *Subject:* [libmicrohttpd] GNU libmicrohttpd 0.9.65 released > Thanks for the new release ! > > Could you push the git tags to the repo(s) ? > > Regards, Tim > > On 7/5/19 10:26 PM, Evgeny Grin wrote: >> Dear all, >> >> >> I'm glad to announce the release of GNU libmicrohttpd 0.9.65. >> >> GNU libmicrohttpd is a small C library that is supposed to make it easy >> to run an HTTP server as part of another application. GNU libmicrohttpd >> is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU >> libmicrohttpd is fast, portable and has a simple API and (without TLS >> support and other optional features) a small binary size (~32k). >> >> This is a bugfix release. >> Notable changes since version 0.9.64: >> * Largely reworked, fixed and updated code for managing of connections' >> memory pools. >> * Memory on W32 is cleared more securely. >> * Used less memory for connection due to reuse existing allocated buffers >> instead of allocation new additional buffers. >> * Better handled connection's memory shortage situation. >> * Error responses are sent now even if no buffer space is left available. >> >> You can download GNU libmicrohttpd from >> >> * https://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors. >> * Our git repository at https://gnunet.org/git/libmicrohttpd.git >> >> Please report bugs to our bugtracker at https://gnunet.org/bugs/. >> >> The documentation (including a reference manual and tutorial) can be >> found at https://www.gnu.org/software/libmicrohttpd/. >> >> signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] feature request: change malloc/calloc/free functions
Hi Nicolas, In addition to Christian's answer, I'd like to point your attention to the fact that for your case no patch is needed. Just use 'MHD_create_response_from_buffer_with_free_callback()' and you will be able to use any custom 'free()' function. For strings allocated by MHD, you should use 'MHD_free()'. So memory management in MHD and in application is already isolated, you can you any allocation routines as long as you will correctly use MHD API. -- Wishes, Evgney 21.07.2019 22:28, Nicolas Mora wrote: > I happened to see that MHD doesn't allow to use different > malloc/calloc/free functions than the one provided by libc. > > This would be useful if the underlying app using MHD uses different > allocating functions like Hoard: http://www.hoard.org/ > More specifically, when using MHD_RESPMEM_MUST_FREE in a response > allocated with a different malloc() function, there would be problems. > > I can submit a patch for it. > Basically, I'd do it by adding functions like this: > > void MHD_set_alloc_funcs(MHD_malloc_t malloc_fn, MHD_calloc_t calloc_fn, > MHD_free_t free_fn); > void MHD_get_alloc_funcs(MHD_malloc_t * malloc_fn, MHD_calloc_t * > calloc_fn, MHD_free_t * free_fn); > > I didn't see any use of realloc() in the source code, so I wouldn't > allow to change it. > > Then, all internal call to malloc()/calloc()/free() would be replaced by > MHD_malloc()/MHD_calloc()/MHD_free() > > How about that? Any feedback? > > /Nicolas > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] feature request: change malloc/calloc/free functions
Don't we have now duplication with 'MHD_create_response_from_buffer_with_free_callback()'? Shouldn't we remove one of implementations? -- Wishes, Evgeny 22.07.2019 12:51, Christian Grothoff пишет: > Thanks, that looks good --- modulo re-using the value 0 in the > enumeration ;-), and lacking an update to the handbook/ChangeLog. I've > made those fixes and pushed the result to Git: > > 3d1b941137f9d8379e6e67d5abd57be5ae6ebe1a > > Happy hacking! > > Christian > > On 7/22/19 4:33 AM, Nicolas Mora wrote: >> Sorry, the patch was inverted, here is a clean patch >> >> Le 19-07-21 à 22 h 30, Nicolas Mora a écrit : >>> Hello again, >>> >>> In fact, there already was the skeleton in MHD to provide what I mean. I >>> attached a patch fil for the function MHD_set_response_options that >>> allows to override free() for the response buffer. >>> The change is quite simple in fact... >>> >>> I didn't add tests to validate this patch because I wouldn't know where >>> to put them. >>> But with this change, I don't need to use MHD_RESPMEM_MUST_COPY anymore! >>> >>> What do you think? >>> >>> /Nicolas >>> >>> Le 19-07-21 à 16 h 10, Nicolas Mora a écrit : Hello, Thanks for the feedback. I totally agree with you on the performance side. I myself don't use different malloc()/realloc()/free() than the libc one. The only reason I make this request is because of a bug a user of my framework had because of my use of MHD_RESPMEM_MUST_FREE by default which caused problems if the response is allocated with a malloc() function incompatible with the libc one. The workaround I found was to use MHD_RESPMEM_MUST_COPY instead, then free the bdy response after calling MHD_create_response_from_buffer. This works fine that some memory resource is wasted (not for long but still). If not for the specific malloc/free function, would it be possible to specify the free() function to deallocate the response body when using MHD_RESPMEM_MUST_FREE? Like an option you could pass to MHD_Connection * inside the MHD_AccessHandlerCallback function? Le 19-07-21 à 15 h 44, Christian Grothoff a écrit : > Hi Nicolas, > > Thanks for the proposal, but I don't think this kind of patch belongs > into MHD. malloc() performance is not critical for MHD at all, as MHD > hardly uses malloc(): We mostly use our own custom memory pool, usually > on top of mmap(), to avoid fragmentation issues and to limit memory > consumption per TCP connection. > > So I doubt you'd get _any_ performance delta by using Hoard. If I am > wrong and you do have MHD-specific performance measurements that show > that this is not premature optimization, please share them! > > Please also consider that there are allocation functions like > strdup()/strndup(), and mixing allocators (malloc going to Hoard, > strdup() to libc) is likely to end in disaster on free(). So in my view > the only _good_ place to add the functions you propose (or Hoard itself) > would be (GNU) libc. > > Happy hacking! > > Christian > > On 7/21/19 9:28 PM, Nicolas Mora wrote: >> I happened to see that MHD doesn't allow to use different >> malloc/calloc/free functions than the one provided by libc. >> >> This would be useful if the underlying app using MHD uses different >> allocating functions like Hoard: http://www.hoard.org/ >> More specifically, when using MHD_RESPMEM_MUST_FREE in a response >> allocated with a different malloc() function, there would be problems. >> >> I can submit a patch for it. >> Basically, I'd do it by adding functions like this: >> >> void MHD_set_alloc_funcs(MHD_malloc_t malloc_fn, MHD_calloc_t calloc_fn, >> MHD_free_t free_fn); >> void MHD_get_alloc_funcs(MHD_malloc_t * malloc_fn, MHD_calloc_t * >> calloc_fn, MHD_free_t * free_fn); >> >> I didn't see any use of realloc() in the source code, so I wouldn't >> allow to change it. >> >> Then, all internal call to malloc()/calloc()/free() would be replaced by >> MHD_malloc()/MHD_calloc()/MHD_free() >> >> How about that? Any feedback? >> >> /Nicolas >> > > signature.asc Description: OpenPGP digital signature
Re: [libmicrohttpd] Using MHD_USE_INTERNAL_POLLING_THREAD without MHD_OPTION_THREAD_POOL_SIZE
Hi Nicolas, You should provide pool size with MHD_OPTION_THREAD_POOL_SIZE only if you are not using "thread per connection"."Thread pool" mode handle all connections by specified number of threads."Thread per connection" run one thread for watching for new incoming connection and separate thread for each connection. "Thread per connection" is less efficient as it may try to run more threads than number of available logical CPU cores, but allow to implement response callbacks in lazy way, when callback may block processing for a long period of time.In "thread pool" (and "Internal polling thread") modes response callbacks must not block. If you need noticeable amount of time to generate a response, you should suspend connection and resume it when response is ready. -- Best Wishes,Evgeny Grin 06.08.2019, 20:04, "Nicolas Mora" :When using the flag MHD_USE_THREAD_PER_CONNECTION with MHD_start_daemon,MHD recommends to explicitly enable flag MHD_USE_INTERNAL_POLLING_THREADtoo.Is there a default thread pool size if we don't use the flagMHD_OPTION_THREAD_POOL_SIZE? Should we set it to an arbitrary value? Orshould I assume MHD will provide the right pool size according to thesystem it's running on?
Re: [libmicrohttpd] test_post_loop11
Reasons should be be pretty simple: buffers flushing do not work, each request require 0.2 s to finish. Multiply by number of requests. -- Best Wishes,Evgeny Grin 17.10.2019, 19:54, "ng0" :Hi,on at least NetBSD this is one of at least 2 testswhich take their good time to finish. Does anyonehave an idea what's happening there so that thetest can be improved if necessary/possible?Thanks
Re: [libmicrohttpd] tsearch updates from NetBSD source
This could cause a problem on platforms where size of unsigned long is smaller than size of pointer. Windows x64 is obvious example. -- Best Wishes,Evgeny Grin 27.11.2019, 14:47, "ng0" :Hi,is this okay for merge? I took it from our (NetBSD) tree,compared to what libmicrohttpd contains.diff --git a/src/lib/tsearch.c b/src/lib/tsearch.cindex 78f37608..2d374cad 100644--- a/src/lib/tsearch.c+++ b/src/lib/tsearch.c@@ -12,6 +12,9 @@ #include "tsearch.h" #include +#ifndef __UNCONST+#define __UNCONST(a) ((void *)(unsigned long)(const void *)(a))+#endif typedef struct node {@@ -21,7 +24,7 @@ typedef struct node } node_t;-/* $NetBSD: tsearch.c,v 1.5 2005/11/29 03:12:00 christos Exp $ */+/* $NetBSD: tsearch.c,v 1.7 2012/06/25 22:32:45 abs Exp $ */ /* find or insert datum into search tree */ void * tsearch (const void *vkey, /* key to be located */@@ -47,10 +50,10 @@ tsearch (const void *vkey, /* key to be located */ } q = malloc (sizeof(node_t)); /* T5: key not found */- if (q)+ if (q != 0) { /* make new node */ *rootp = q; /* link new node to old */- q->key = vkey; /* initialize new node */+ q->key = __UNCONST(vkey); /* initialize new node */ q->llink = q->rlink = NULL; } return q;
Re: [libmicrohttpd] tsearch updates from NetBSD source
Let's stay with current version. '!= 0' will not add any improvement.Unpatched version gives maintenance simplicity. -- Best Wishes,Evgeny Grin 27.11.2019, 16:53, "ng0" :Okay, thanks.What about the != 0 addition, could this be problematic as well?
Re: [libmicrohttpd] running in main thread
You can also use very portable and universal solution: mhd = MHD_start_daemon(MHD_USE_AUTO, while(1){ fd_set rs; fd_set ws; fd_set es; MHD_socket maxsock; struct timeval tv; struct timeval* tvp; unsigned long long tov; FD_ZERO (&rs); FD_ZERO (&ws); FD_ZERO (&es); maxsock = MHD_INVALID_SOCKET; if (MHD_NO == MHD_get_fdset (d, &rs, &ws, &es, &maxsock)) break; if (MHD_NO == MHD_get_timeout(mhd, &tov)) tvp = NULL; else { tv.tv_sec = tov / 1000; tv.tv_usec = (tov % 1000) * 1000; tvp = &tvp; } select (maxsock+ 1, &rs, &ws, &es, tvp); if (MHD_NO == MHD_run_from_select (d, &rs, &ws, &es) break;} Untested.Add error checking and types casting. -- Best Wishes,Evgeny Grin 03.12.2019, 00:32, "Christian Grothoff" :On 12/2/19 10:18 PM, José Bollo wrote: Well, that's trivial. Please consider the attached fragment (not tested, but should be very close to what you need, modulo error handling). I know it is trivial (my implementation below) but I advocate to offer a facility that does the job. It avoids tricky aspects like EPOLL and get_daemon_info (is working for windows???)AFAIK on the Windows Subsystem for Linux it should work, and everythingelse is a mess to do in a portable way. Regardless, these loops areusually pretty simple, hence I do still think that they don't belonginto the library. What we could do is at sample loops into thedocumentation (tutorial?).
Re: [libmicrohttpd] Unable to find specified file minimal_example.c
Hi Bryan, You should ask Ubuntu/Debian package maintainers about files provided by .deb packages.MHD itself provides all mentioned files. -- Best Wishes,Evgeny Grin 22.12.2019, 05:44, "Bryan White" :I have libmicrohttpd[10|-dev|-dbg] version 0.9.44 installed under Ubuntu 16.04 LTS.> locate minimal_exampledoes not find it. > locate hellobrowserfinds/home/XXX/libmicrohttpd_copy/examples/hellobrowser.c/usr/share/doc/libmicrohttpd-dev/examples/hellobrowser.c RegardsBryan WhiteDirector, Brycom Data LtdTel: +64(9)627 1416Mob: +64(21)176 7222 On Sat, 21 Dec 2019 at 22:39, Christian Grothoff <groth...@gnunet.org> wrote:Hi Bryan,I don't know where you were looking, but it is in the TGZ releases oflibmicrohttpd, and in our Git in the src/examples/ directory, as thedocumentation says:* In Git:https://git.gnunet.org/libmicrohttpd.git/tree/src/examples/minimal_example.c* Official release:http://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.69.tar.gz(unpack, go to src/examples/minimal_example.c).Happy hacking!ChristianOn Sat, 2019-12-21 at 16:58 +1300, Bryan White wrote:> At https://www.gnu.org/software/libmicrohttpd/tutorial.html> It says "Both of these issues you will find addressed in the official> minimal_example.c residing in the src/examples directory of the MHD> package"> But I can't find it. Please advise where I can find it, or its new> name.> Thank you.>> Regards>> Bryan White> Director, Brycom Data Ltd> Tel: +64(9)627 1416> Mob: +64(21)176 7222
