Re: [libmicrohttpd] Websockets

[Christian Grothoff]

On 03/18/2017 01:44 AM, John Duncan wrote:
> Websockets and RFC6455.  I noticed the test_upgrade.c unit test code
> doesn't provide the accept handshake key combination hashing specified on
> page 8 of the RFC, in the section "opening handshake."
> 
> Apparently we're supposed to combine the websocket key with a predesignated
> static GUID, hash it, and send it back.  Without doing this, current
> firefox refuses to accept a websocket connection to MHD.
> 
> 
> My question is, should I write this functionality myself or are there plans
> for MHD to implement it in the future with macros or such?  It seems very
> easy to implement but I don't want to implement my own code if the library
> has plans to implement this functionality for users directly.  Don't want
> to duplicate efforts etc.

Right now, the expectation is that you need to write this yourself, at
least I have no plans to do more in terms of API than what we have today
with respect to "HTTP Upgrade".

Happy hacking!

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] '-lrt' not resolving undefined reference to clock_gettime?

[Christian Grothoff]

glibc < 2.17 requires -lrt for clock_gettime(), older versions do not.
I've added "-lrt" in Git commit ed6509bf6ca46e39e3514680bfb81216e2a825bf
but without testing (as I don't have an ancient glibc on this system and
I am on a train...).  Please let me know if it does not work...

Happy hacking!

Christian

On 03/20/2017 09:55 AM, Alex Reynolds wrote:
> I am compiling a static binary ("my_project") that links in a static
> compilation of the libmicrohttpd.a library, along with a couple other
> libraries.
> 
> On some platforms, this appears to work. 
> 
> On others, like a default image of Linux that is used in Travis CI
> (Ubuntu 12.04.5 LTS, gcc 4.6.3), I get the following compilation error
> related to libmicrohttpd:
> 
> /third-party/libmicrohttpd/lib/libmicrohttpd.a(libmicrohttpd_la-mhd_mono_clock.o):
> In function `MHD_monotonic_sec_counter_init':
> /third-party/libmicrohttpd/src/microhttpd/mhd_mono_clock.c:160:
> undefined reference to `clock_gettime'
> /third-party/libmicrohttpd/src/microhttpd/mhd_mono_clock.c:180:
> undefined reference to `clock_gettime'
> /third-party/libmicrohttpd/src/microhttpd/mhd_mono_clock.c:191:
> undefined reference to `clock_gettime'
> /third-party/libmicrohttpd/src/microhttpd/mhd_mono_clock.c:202:
> undefined reference to `clock_gettime'
> /third-party/libmicrohttpd/lib/libmicrohttpd.a(libmicrohttpd_la-mhd_mono_clock.o):
> In function `MHD_monotonic_sec_counter':
> /third-party/libmicrohttpd/src/microhttpd/mhd_mono_clock.c:315:
> undefined reference to `clock_gettime'
> collect2: ld returned 1 exit status
> make: *** [my_project] Error 1
> 
> From reading Stack Overflow, this appears to be related to linking the
> Real Time library (`-lrt`).
> 
> I have tried compiling my static binary with the addition of
> `-Wl,--no-as-needed -lrt` to my LIBS variable, which is positioned in
> the build statement after the objects are compiled, e.g.:
> 
> ...
> gcc -g -Wall -Wextra -std=c99 -D__USE_POSIX -D__STDC_CONSTANT_MACROS
> -D__STDINT_MACROS -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE=1 -O3
> -pthread -static -static-libgcc -I/usr/include
> -I/third-party/bzip2
> -I/third-party/libmicrohttpd/include
> -L"/third-party/bzip2"
> -L"/third-party/libmicrohttpd/lib" my_project.o -o my_project
> mt19937.a -Wl,--no-as-needed -lrt -lm -lbz2 -lmicrohttpd
> ...
> 
> However, this patch did not work. I still get the same fatal build errors. 
> 
> As the example build statement shows, I can incorporate other static
> libraries into my project's binary -- just not libmicrohttpd. I'm not
> sure what else to try and was wondering if others might have ideas.
> 
> In Travis CI, I would also get similar errors about undefined references
> to Pthreads calls, which I fixed by adding `-pthread` to my build flags,
> as well as undefined references to functions in libgcrypt and GnuTLS
> libraries, which I fixed by adding `--without-libgcrypt
> --without-gnutls` to the libmicrohttpd ./configure options. 
> 
> Is there a similar modification I can make to my build or linking
> process to resolve the clock_gettime() call errors, so that I can
> incorporate the static libmicrohttpd.a library? 
> 
> Thanks for any advice!
> 
> Regards,
> Alex
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [PATCH]: MHD_connection_update_event_loop_info sends INTERNAL_ERROR for suspended connections

[Christian Grothoff]

On 03/17/2017 01:10 PM, Vitaliy T wrote:
> Hello,
> 
> On 17 March 2017 at 10:03, Evgeny Grin  wrote:
>> Please note that even if you suspend connection, you must process at
>> least some data (preferably - all data) and decrement value of
>> 'upload_data_size'.
> 
> Yes, this was my mistake. Before suspending a connection I have to
> forgot about decrementing upload_data_size.

Hmm. As long as you suspend inside of the DH, we should just allow you
to not decrement upload_data_size.  I've checked Git master, and there
it is clearly OK (line 2020: if (... && (! suspended)) error();).  Did
not check if this was just recently fixed.

> I have questions:
> 
> 1) Due the async nature of the MHD, how can I keep a connection in
> resume state _without_ reading data in DH?

That you must not do, as it would mean running at 100% CPU in select().
> I mean, I can't say in which moment I can resume the connection with
> guarantee to read the data. I have to create some kind of queue with
> tracking which connection is could read data and which ones could not.
> And more importantly that I must guarantee that if I resumed a
> connection and it must read full data.

What you can do is simply "optimistically" resume a connection when
there is a _chance_ that you might be ready, an if it turns out you were
wrong, instantly suspend it again.

> Things getting complicated.
> 
> Current DH logic looks like that:
> 
> 1. I have to initialize internal structure per request, then it will
> be stored to con_cls and I return MHD_YES.
> 2. On the second entry into DH, I do process request: either upload a
> file, either suspend a connection.
> 3. When uploading the file has been finished I am resuming the next
> connection (if it is).
> 4. Even after resuming the connection there is a chance that a new
> connection will be proceeded before the "resumed" connection.
> 
> I can track if the connection was resumed and give it a higher
> priority. But if so I have to suspend _all_ new connections before
> doing something useful.
> I will think about it, but, IMHO, it is wrong that I can't
> suspend/resume connections when I want without reading buffers.

I think you can do so (now), and if it is really something that changed
between the time you resumed and the time MHD called you, it should be
acceptable (and in particular not result in busy waiting).

> 2) Is it possible to do so that the chain
> suspend-resume-suspend-resume will not affect read buffers?

We should definitively implement it like that.

I've put in the required logic to avoid MHD freaking out about the
"event loop without network activity" in
4a77c8eba2e3e4857dabb12fbb52920b0eb98a59




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] MHD_quiesce_daemon() question

[Christian Grothoff]

On 03/26/2017 10:36 PM, Evgeny Grin wrote:
> On 26.03.2017 8:33, silvioprog wrote:
>> I found the following related message:
>>
>> https://lists.gnu.org/archive/html/libmicrohttpd/2014-09/msg00012.html
>>
>> I've used a similar logic, but with item X below, because I need to wait
>> the client processing:
>>
>> 1) MHD_quiesce_daemon()
>> *X) while (info.num_connections > 0) sleep(0.5s) # pseudo code*
>> 2) MHD_stop_daemon()
>> 3) close()
>>
>> Real implementation:
>>
>> bool bf_httpsrv_shutdown(struct bf_httpsrv *srv, bool force) {
>> MHD_socket fd;
>> if (srv && srv->listening) {
>> fd = MHD_quiesce_daemon(srv->mhd);
>> if (!force)
>> while (MHD_get_daemon_info(srv->mhd, 
>> MHD_DAEMON_INFO_CURRENT_CONNECTIONS)->num_connections > 0)
>> usleep(1000 * 500); //TODO: allow to use external callback
>> MHD_stop_daemon(srv->mhd);
>> if (fd != MHD_INVALID_SOCKET)
>> close(fd);
>> srv->listening = false;
>> return true;
>> }
>> return false;
>> }
>>
>>
>> Calling it with bf_httpsrv_shutdown(srv, false) the server stops waiting
>> for clients processing.
>>
>> So, what do you think about the logic above? Should it be improved?!
>>
>> Thanks in advance for any suggestions!
> 
> If you don't check returned value from MHD_quiesce_daemon(), you may
> later found that you didn't quiesced daemon, so move check right after
> MHD_quiesce_daemon() and added error handling.
> If you didn't set connection timeout, connection may live indefinitely.
> Moreover, even with connection timeout, clients may continue processing
> on same HTTP 1.1 connections with new requests indefinitely.
> Furthermore, even with HTTP 1.0 and connection timeout hypothetical
> client may read answer very slow with results in very long unpredictable
> closure of connection.
> So: yes, you code will work but may result in very long (hours, for
> example) or even indefinitely long daemon shutdown.
> 

And while Evgeny is 100% correct, let me point out the opposite concern:
0.5 s can still be an eternity (think: shell scripts, automated tests,
etc.) and that you ideally should use MHD_OPTION_NOTIFY_CONNECTION to
notify main() that you are "done". For example by doing a semaphore-down
operation/IPC write in main() and a semaphore-up()/IPC read in the
callback IF you are past quiesce and info tells you that you are the
last connection.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Reading data from named pipe

[Christian Grothoff]

On 03/27/2017 08:03 PM, Alex Reynolds wrote:
> I have an API that reads a subset of data from an archive and writes it to
> a FILE*. I can use this API to write this data subset to a temporary file
> via mkstemp() and fdopen().
> 
> I am currently using MHD_create_response_from_callback() to read chunks of
> bytes from this temporary file and write them to the web client. This works
> well on small files.
> 
> To avoid the I/O cost of writing a larger temporary file, and then reading
> from it, I set up a named pipe via mkfifo() and a FILE* that points to it.
> When I have MHD_create_response_from_callback() try to read from this named
> pipe, in the same way that it is set up to read from the temporary file,
> the server and client hang.
> 
> If this is possible, what is the correct way to serve data from a named
> pipe?

The answer depends on the style of event loop you are using.

For external select, do a non-blocking (!) read from it, and return
whatever you got to MHD via response_from_callback(), returning 0 if the
pipe is still operational but you have no more data. To keep the event
loop going, you need to manually add your pipe to the read-set, so that
select/poll/epoll unblock as needed.

For thread-per-connection, use a blocking (!) read from the pipe, and
always return data to MHD.

For thread-pool / internal select, you would have to suspend the
connection when the pipe runs dry, and then have some _other_ thread
check for the pipe becoming again available and calling resume on the
connection. This one is usually messy.


If you want to go experimental, you could try to get
MHD_create_response_from_fd64() to work and use fileno() on your FILE*
to get the underlying FD.  I have _never_ tested this, and suspect it'll
blow up in your face the moment the 'fd' blocks as MHD expects it to be
a file and not a pipe. But, maybe it is possible to hack MHD to handle
blocking 'sendfile()' --- I am pretty sure it does not manage this case
today --- and thereby unify the 3 cases above, simplify the client and
improve performance.  Still, this is more like adding a medium-size
feature to MHD than the "correct way" you asked for, so don't take this
as advice for how to do it, but more as a pointer in case you ever need
something even better and have too much time ;-).




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Using MHD_UpgradeResponseHandle without including internal.h

[Christian Grothoff]

Hi John,

You're expected to pass data via "cls", you have no influence on
"extra_in".  "extra_in" is there in case MHD "accidentally" read already
more data than the HTTP header from the Web socket, thereby making it
available to you (you are to treat it as if you read it from 'sock').

So in your example where there is "NULL", just pass some pointer and
you'll be given it in the "cls" argument.

Happy hacking!

Christian

On 04/01/2017 01:26 AM, John Duncan wrote:
> I was trying to pass custom non-global scoped data to my upgrade_cb for
> websockets (upgrade_cb is the name of the routine in test_upgrade.c).  I
> noticed that the opaque MHD_UpgradeResponseHandle structure was passed as a
> pointer parameter to the callback, so I was trying to investigate its
> members to get some insight.  That's when I realized that the members were
> not exported.  Your answer provides the insight I needed regarding opacity,
> however I still need to pass in custom data to the upgrade callback.
> 
> The callback has the following parameters, and I'm guessing I'm supposed to
> be using extra_in to pass data into the callback:
> 
> static void upgrade_cb
> (
> void *cls,
> struct MHD_Connection *connection,
> void *con_cls,
> const char *extra_in,
> size_t extra_in_size,
> MHD_socket sock,
> struct MHD_UpgradeResponseHandle *urh
> );
> 
> However, the only time I see upgrade_cb referenced in test_upgrade.c is
> when it's used as follows:
> 
> resp = MHD_create_response_for_upgrade (&upgrade_cb, NULL);
> 
> Do I pass data in through the second parameter?
> 
> Basically, my question boils down to:
> 
> How do I attach my own data so that when the callback is triggered I can
> pass through my own pointers/structures to the callback without utilizing
> globally scoped variables?
> 
> Thanks;
> ~JD
> 
> 
> 
> 
> On Thu, Mar 30, 2017 at 8:32 PM, Evgeny Grin  wrote:
> 
>> Hi John,
>>
>> MHD_UpgradeResponseHandle is intentionally opaque. It's not designed for
>> direct usage of internal members.
>> If you define it locally, you'll almost likely to break compatibility
>> with future version.
>> MHD uses a lot of opaque structures which are internally changed from
>> version to version. But as long as application uses only public official
>> API, those changes don't break compatibility.
>>
>> Usually MHD has specific API to get information about opaque handle.
>> What do you want to get from internal members?
>>
>> --
>> Best Wishes,
>> Evgeny Grin
>>
>> On 31.03.2017 0:58, John Duncan wrote:
>>> I noticed that microhttpd.h only contains only forward defintions of the
>>> MHD_UpgradeResponseHandle structure.  The file internal.h contains the
>>> actual structure definition with all members.
>>>
>>>  When I try to include the internal.h file directly, I get tons of
>>> compilation errors leading me to believe that I'm not supposed to be
>>> including it.
>>>
>>> My question is, when building my own applications, how am I supposed to
>>> access member portions of the MHD_UpgradeResponseHandle structure
>>> without including internal.h.  Am I expected to define the structure
>>> within my own project headers and use the provided
>>> MHD_UpgradeResponseHandle pointer as somewhat of an opaque type?
>>>
>>> I don't mind doing this, I just want to make sure this is the correct
>>> approach.
>>>
>>> Thanks;
>>> ~JD
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] integration with my websockets

[Christian Grothoff]

Jose,

Have you considered doing

  if upgrade-requested-for(req) then
 my_cls = create-websocket-for(req)
 if my_cls created then
  MHD_create_response_for_upgrade (&my_handler, my_cls);
 else
  reply "internal error"
 end
  else
 
  end

Basically, use the "upgrade_handler_cls" argument to pass along any kind
of state you need for your upgrade handling.  That way, you
can do pretty much any initialization/setup work you require before you
queue the response object and thus lock yourself into some particular
type of response.

As far as I see it, the only limitation here is that you cannot touch
the socket itself. (However, technically you can touch it as well using
MHD_CONNECTION_INFO_CONNECTION_FD for HTTP-only connections, but it's
usually not a good idea to hack around with that.)


Happy hacking!

-Christian

On 04/11/2017 07:23 PM, José Bollo wrote:
>> Could you explain a bit more do you want to implement and why it is
>> not possible?
> 
> This pseudo-code is no more possible:
> 
>   if upgrade-requested-for(req) then
>  handler = create-websocket-for(req)
>  if handler created then
>   reply "switch protocol"
>  else
>   reply "internal error"
>  end
>   else
>  
>   end
> 
> Instead you may write:
> 
>   if upgrade-requested-for(req) then
>  reply "switch protocol" with callback
>   else
>  
>   end
> 
> and in the callback
> 
>   handler = create-websocket-for(req)
>   if handler created then
>   cool
>   else
>   not cool, too late for sending an internal error
>   end
> 
> But as written there is something not cool cool
> So the following version is dandling with the hole
> 
>   if upgrade-requested-for(req) then
>  handler = create-websocket-lazy
>  if handler created then
>   reply "switch protocol" with callback
>  else
>   reply "internal error"
>  end
>   else
>  
>   end
> 
> and in the callback
> 
>   attach-websocket-socket(...)
> 
>>
 Third, do you plan to provide a new version soon?  
>>
>> Yes, watch this list. :)
>>
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] How to access raw request bytes sent to a request handler (and total request size)?

[Christian Grothoff]

Hi John,

The raw request data is unavailable as MHD processed it in-place
(zero-copy!) to give the parsed headers to the application.

However, it should be relatively easy to modify MHD to just return the
number of bytes in the header, if that would be sufficient?

Happy hacking!

Christian

On 04/18/2017 02:50 AM, John Duncan wrote:
> I'd like to be able to access the raw (decrypted) request data recieved
> and total request size from within a request handler if it's available. 
> I'd like to use it for data rate limiting  with regards to individual
> request handlers.  (eg.  one request handler has a 8kbs limit, one has a
> 16kbs limit, that kinda thing)
> 
> I know you can use MHD_get_connection_values to get access to all
> component data, but I'd prefer to get the raw sent request if possible
> as it'd be very easy to just use the request length as an additive value
> to calculate the bytes-per-second average for any given request handler.
> 
> The alternative is using iterators with MHD_get_connection_values, which
> come out to be less efficient.
> 
> 
> Thanks!
> ~JD



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] How to close all upgraded connections when shutting down MHD_Daemon?

[Christian Grothoff]

You forgot to update doc/libmicrohttpd.texi, otherwise looks OK even
though I'm not convinced SHUTDOWN/QUESTCED are useful: an application
can trivially track those itself, so these two are definitively just API
bloat.

On 05/04/2017 03:59 PM, silvioprog wrote:
> Done. So dudes, what do you think about this attached patch?
> 


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] About an HTTPS connection Upgraded

[Christian Grothoff]

On 05/07/2017 09:20 PM, Nicolas Mora wrote:
> Hello,
> 
> I'm still working on a websocket implementation based on MHD, the non
> secure websocket works fine, but the secure websocket doesn't work yet.
> 
> I'm having problems with the socket and the tls session. I get my
> inspiration from the test_upgrade.c file but I can't have a valid tls
> session anyway.
> 
> Inside the upgrade callback function, I use the same sequence as in the
> example program to initiate the tls session and the result tells me it's
> connected:
> https://github.com/babelouest/ulfius/blob/2.0/src/u_websocket.c#L127
> 
> But when it comes to reading or sending data, I got nothing or errors,
> whether the socket is in blocking mode or not.
> 
> The read/write functions are at the end of the file:
> https://github.com/babelouest/ulfius/blob/2.0/src/u_websocket.c#L720
> 
> Does anyone see what's wrong with my code?

Eh, what is wrong is that you use "gnutls_record_send".  MHD will do
this for you, you should just use "send/recv", even in HTTPS mode!  In
fact, your code (short of MHD_start_daemon) can be exactly the same for
HTTP and HTTPS.




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] About an HTTPS connection Upgraded

[Christian Grothoff]

On 05/07/2017 10:20 PM, Nicolas Mora wrote:
> 
> Any I retried it without any gnutls call on the MHD_sock and I still
> have problems reading and writing the socket, nothing is transmitted,

Hmm.  Try 'strace' on IO operations (only, i.e. "strace -e
trace=network,ipc,desc").  When you 'send' to the Web socket, it should
be writing to a socketpair(), followed by MHD 'recv'ing from that
socketpair() and then GnuTLS ultimately send'ing to the actual TCP
socket. Equivalent for 'recv()'.

One possible cause is that you somehow fail to trigger the MHD event
loop, but your example is too complex for me to tell quickly.  Also, I'd
not exclude the possibility of a bug, given that "Upgrade" is still
relatively new (and you didn't write which version of MHD you are using...).



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] How to access raw request bytes sent to a request handler (and total request size)?

[Christian Grothoff]

Dear John,

Just wanted to let you know that this has now been implemented.

See:
https://gnunet.org/git/libmicrohttpd.git/commit/?id=76cf7d7f5877a3ce0f3bb131aa6e6f420d5b98eb

Happy hacking!

Christian

On 04/26/2017 12:02 AM, John Duncan wrote:
> That would be great, thanks!
> 


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Sleeping a request thread

[Christian Grothoff]

Hi,

Sleeping in THREAD_PER_CONNECTION mode is fine --- if you want to really
just throttle the response generation and basically create latency.

-Christian

On 05/19/2017 12:22 AM, Miguel Sancho wrote:
> Hi,
> using the MHD_USE_THREAD_PER_CONNECTION mode,
> are there any issues in "sleeping" the request thread some seconds
> waiting for response? any known drawback in MHD?
> 
> for example to wait 4s:
>  usleep(400)
>  MHD_queue_response (..)
> 
> Thanks



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] socket_context not set (libmicrohttpd-0.9.50)

[Christian Grothoff]

Hi Miguel,

There are two contexts, the "socket context" and the "request context".
The access handler is given the "request context", to get the socket
context there we would have to extend the MHD_get_connection_info() API
to allow you to get it during the access handler.  I guess that's an
oversight in the API, as we should make it possible for you to get the
access context. I'll put it on my list of things to fix.

-Christian

On 06/01/2017 05:11 PM, Miguel Sancho wrote:
> Hi,
> finding the following issue (libmicrohttpd-0.9.50),
> I would like to check whether this is already known.
> 
> Seems that socket_context is not set in #MHD_AccessHandlerCallback.
> 
>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context object
> is created
>   * In #MHD_AccessHandlerCallback, *socket_context is a NULL pointer
>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context object
> is created
> 
> Code sample:
> /const MHD_ConnectionInfo *connectionInfo =
>  
> MHD_get_connection_info(connection,MHD_CONNECTION_INFO_SOCKET_CONTEXT);
> if (*(connectionInfo->socket_context == NULL))
> {
> printf("ERROR\n");
> }
> /
> Any idea?
> Thanks
> Miguel


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] socket_context not set (libmicrohttpd-0.9.50)

[Christian Grothoff]

Sorry, was mistaken, the API does already exist as it should. The
argument you need to pass is aptly named:
MHD_CONNECTION_INFO_SOCKET_CONTEXT.

-Christian

On 06/01/2017 05:33 PM, Christian Grothoff wrote:
> Hi Miguel,
> 
> There are two contexts, the "socket context" and the "request context".
> The access handler is given the "request context", to get the socket
> context there we would have to extend the MHD_get_connection_info() API
> to allow you to get it during the access handler.  I guess that's an
> oversight in the API, as we should make it possible for you to get the
> access context. I'll put it on my list of things to fix.
> 
> -Christian
> 
> On 06/01/2017 05:11 PM, Miguel Sancho wrote:
>> Hi,
>> finding the following issue (libmicrohttpd-0.9.50),
>> I would like to check whether this is already known.
>>
>> Seems that socket_context is not set in #MHD_AccessHandlerCallback.
>>
>>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context object
>> is created
>>   * In #MHD_AccessHandlerCallback, *socket_context is a NULL pointer
>>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context object
>> is created
>>
>> Code sample:
>> /const MHD_ConnectionInfo *connectionInfo =
>>  
>> MHD_get_connection_info(connection,MHD_CONNECTION_INFO_SOCKET_CONTEXT);
>> if (*(connectionInfo->socket_context == NULL))
>> {
>> printf("ERROR\n");
>> }
>> /
>> Any idea?
>> Thanks
>> Miguel


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] socket_context not set (libmicrohttpd-0.9.50)

[Christian Grothoff]

On 06/02/2017 04:23 PM, Miguel Sancho wrote:
> Christian, thanks for your answer,
> I need to recheck, already doing the following in the
> #MHD_AccessHandlerCallback and the *socket_context appears to be NULL:
> /MHD_get_connection_info(connection,MHD_CONNECTION_INFO_SOCKET_CONTEXT);/
> 
> Regards
> 
> 2017-06-01 17:11 GMT+02:00 Miguel Sancho  >:
> 
> Hi,
> finding the following issue (libmicrohttpd-0.9.50),
> I would like to check whether this is already known.
> 
> Seems that socket_context is not set in #MHD_AccessHandlerCallback.
> 
>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context
> object is created
>   * In #MHD_AccessHandlerCallback, *socket_context is a NULL pointer
>   * In #MHD_NotifyConnectionCallback (STARTED), **socket_context
> object is created
> 
> Code sample:
> /const MHD_ConnectionInfo *connectionInfo =
>  
> MHD_get_connection_info(connection,MHD_CONNECTION_INFO_SOCKET_CONTEXT);
> if (*(connectionInfo->socket_context == NULL))
> {
> printf("ERROR\n");
> }
> /
> Any idea?

Eh, what is that "*" doing in the "if" line?  That looks very wrong.  Try:

if (NULL == connectionInfo->socket_context)
  {
printf("ERROR\n");
  }

Also, what code do you use in the connection notification callback to
set the socket_contest?



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] socket_context not set (libmicrohttpd-0.9.50)

[Christian Grothoff]

On 06/05/2017 06:01 PM, Miguel Sancho wrote:
> Christian,
> 
>> Eh, what is that "*" doing in the "if" line?
> 
> reason is that in definition of the MHD_ConnectionInfo union, I see
> socket_context defined as "void**", maybe it should be just "void*"

It is indeed "void *" in more recent versions of MHD.

>> Also, what code do you use in the connection notification callback to
> set the socket_contest?
> 
> MyContext *contextPtr = new MyContext();
> *socket_context = contextPtr;

Ok, that  looks correct.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [GSoC Update] Week 1

[Christian Grothoff]

Maybe I missed something, but why should the MHD dependency become
_mandatory_? Why not conditionally-compile and run the tests only if MHD
is present?  MHD does the same for libcurl.  Making the dependency
optional also avoids the obvious possibility of circular dependencies if
we ever were to add libwget2-based tests to MHD :-).

On 06/05/2017 06:46 PM, Didik Setiawan wrote:
>  * With Libmicrohttpd becomes mandatory package to install before building 
> Wget2
>binary, there must be proper warning about this requirement, otherwise the
>building process will fail. I have add oneliner information into README.md.
>Please notice me if there are some other places where I can add warning 
> like
>this.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [GSoC Update] Week 2

[Christian Grothoff]

On 06/13/2017 05:42 AM, Didik Setiawan wrote:
>- Last check failed when the test try to resolve URL with question mark.
>  E.g: "/subdir1/subpage1.html?query¶m", when I debug, it return just
>  "/subdir1/subpage1.html" so the result is 404 not found. I also check 
> using
>  logging example source code provided in Libmicrohttpd tutorial [4]. When 
> I
>  access using http client such as Wget2 and Firefox, the result is still 
> the
>  same. The URL result omit the query part. Need to confirm to 
> Libmicrohttpd
>  side about this, whether it is intended behaviour or not.

Yes, that's intended, for URL parameters/arguments you need to use
MHD_get_connection_values() with kind=MHD_GET_ARGUMENT_KIND to inspect them.

Happy hacking!

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Processing single request with data from heap

[Christian Grothoff]

MHD_create_response_from_callback() is the API you are looking for.
src/examples/chunked_example.c (and various others) is one example for
how this API is used. Admittedly, it's not in the tutorial (yet), and I
agree it would be good to add it.

On 06/18/2017 10:38 AM, Alex Reynolds wrote:
> I have some request handler code that sends the contents of a regular
> file to the client. However, the issue is that my program must first
> write a regular file to the file system, and this is time consuming for
> very large files. My program (and the client) must wait for the entire
> file to be generated, before my program can go back and read the file to
> send any data to the client.
> 
> As an addition to the excellent tutorials already available, I'd like to
> ask if there could be a tutorial written that demonstrates how to
> process a single request for data in chunks, where the source of the
> data is not a regular file, but a heap-allocated buffer of some size,
> which is repeatedly filled to some content length (up to or smaller than
> its size) and sent to the client in asynchronous fashion, until all data
> are processed.
> 
> Regards,
> Alex



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] "updates to proposed API based on discussions with EG"

[Christian Grothoff]

On 08/21/2017 07:25 PM, silvioprog wrote:
> Hello duddes. :-)
>
> I'm very exited with this news and I'm following each new commit. I
> would be glad to read these discussions because I've learned a lot with
> them.

Well, to be honest I am more frank (both in terms of admitting that I'm
wrong as well as insisting I'm right) when I am not sending a message to
a mailinglist with 100+ participants. As a very frank discussion was
called for, I ran the initial discussions with Evgeny in private.
However, I do think it is time to add more brains, as the discussions
between Evgeny and me seem to have largely converged. (We don't agree
100%, but IMO the delta is pretty small and it would be unwise to bias
the discussion here by exposing it.)

So let's finally open the discussion here!

As Silvio observed by watching Git, Evgeny and I have been discussing
what the next MHD API should look like. Basically, the existing API has
evolved to be almost 100% backwards compatible since MHD 0.0.0, and it
is time to fix my mistakes from the past: I am sure we can create an API
that is more intuitive, easier to use, more secure to use, and will even
improve performance. Furthermore, we want the migration from the
existing API to the new one to be painless: (1) support a superset of
the features, (2) have the library implement both API-styles for a while
(years!) so that applications can migrate to the new style when it
becomes available on all the platforms they care about.

With these high level goals in mind, Evgeny and I analyzed issues with
the current API and wrote a new one. You can find it in Git under
"src/include/microhttpd2.h" (that's not what the header will be called
in the end, I plan to merge it with microhttpd.h once it is actually
implemented).

So, if you care about the future of MHD, please have a look, read our
comments, study the API, and provide us with constructive feedback!

Thanks!

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] web site: libevhtp has a new github url

[Christian Grothoff]

Thanks, fixed now (sorry for the delay, was on the move). -Christian

On 07/20/2017 07:42 AM, Allan Wind wrote:
> https://github.com/criticalstack/libevhtp
> 
> 
> /Allan


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Build libmicrohttpd with HTTPS to powerpc

[Christian Grothoff]

Hi Avner,

We never supported 'gcrypt-only'. We always required GnuTLS, just some
very long time ago we tried to include parts of the GnuTLS code with
MHD. However, that version never worked well with HTTPS/TLS, so it is
definitively not advisable to go there.  If you cannot build GnuTLS, you
could:

1) Ask the GnuTLS mailinglist for help,
2) Disable TLS support for MHD if you do not need it; or,
3) Put MHD-logic behind a reverse proxy (i.e. Apache) that
   does support TLS

Happy hacking!

Christian

On 09/03/2017 09:23 PM, Avner Flesch wrote:
> Hi,
> 
> I am trying to build libmicrohttpd for powerpc (ppc_6xx) host.
> I remember that a few years ago, in order to support HTTPS you could use
> GnuTls lib or gcrypt lib - I am having trouble to build GnuTls (because
> of all the dependencies). What is the last libmicrohttpd version that
> can support HTTPS with gcrypt only?
> 
> Thanks
> 
> Avner



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Build libmicrohttpd with HTTPS to powerpc

[Christian Grothoff]

On 09/13/2017 07:18 PM, Avner Flesch wrote:
> Hi,
> 
> Thanks for the answer.
> Anyway I did it and manage to build libmicrohttpd with HTTPS
> but, I get a core dump in MHD_start_daemon (only with HTTPS support)
> Now I am checking maybe something is wrong with my arguments, but
> another direction I am checking is the depended libraries - let's say I
> am using the last libmicrohttpd version 0.9.55, are there recommended
> versions of gcrypt and gnutls to use? and other libraries?

Well, mostly I'd check the recent CVE's for libgcrypt before deploying
in production ;-).  That said, I'm not aware of any version *building*
but then not working. So if it compiles, you should be OK (modulo using
the MHD API correctly...).

My guess: you passed some arguments required for TLS initialization
incorrectly.

Happy hacking!

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Misc patch (MHD_TLS_CONNECTION_INIT not declared)

[Christian Grothoff]

Fixed, thanks for reporting! -Christian

On 09/27/2017 11:17 AM, Maru Berezin wrote:
> Hello, in commit 243e8fcd6054e4c0d2964b0d4b29e0c15861498d (5 Jun 2017), the 
> definition of MHD_TLS_CONNECTION_INIT was deleted.
> 
> Please find attached a patch.
> 
> Regards,
> maru
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Building libmicrohttpd-0.9.55 on RHEL5

[Christian Grothoff]

Great. Please inform the developer mailinglists of GNU nettle and GnuTLS
about the regression(s).

On 09/29/2017 03:24 PM, Svein Olav Bjerkeset wrote:
>> But I suspect that problem comes from outdated RHEL5 GnuTLS lib.
>> MHD on git master is able to use modern GnuTLS without gcrypt. As next
>> step I suggest to build latest GnuTLS on RHEL5 and build MHD with modern
>> GnuTLS.
> 
> Success! I managed to get a working version by reverting to nettle 2.7.1
> and gnutls 3.3.26. The other libraries are at the same version as listen
> in my first post in this thread.
> 
> Both my own code and the demo_https server works with this combination
> of libraries.
> 
> Best regards,
> Svein Olav Bjerkeset
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] call_connection_handler (connection); /* "final" call */ is called twice

[Christian Grothoff]

Well, I'm pretty sure there is nothing _wrong_ about your code. The MHD
API doesn't actually specify whether it will call you one last time
before the timeout, but what happens right now is that the timeout
triggers the select() to unblock, and MHD goes over all "pending"
connections, and _afterwards_ collects those that have timed out. We
_may_ change that behavior in the future (not that I have specific
plans), so you probably shouldn't rely upon it.

My 2 cents

Christian

On 10/06/2017 06:14 PM, Decebal Epuran wrote:
> I am developing a web server that mainly handles POST requests (json).
> I'm getting the "final call" (*upload_data_size == 0) when all the POST
> data is transfered to my buffer and I'm ready to parse the json data and
> do additional work before send the reply.
> If I'm going over the timeout limit (MHD_OPTION_CONNECTION_TIMEOUT)
> without sending any reply I'm getting another "final call"
> (*upload_data_size == 0) before the MHD_OPTION_NOTIFY_COMPLETED call.
> Is this behavior by design or something is not right in my code?
> 
> Thanks



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Is there something like MHD_free() ?

[Christian Grothoff]

Hi Tim,

We definitively don't have MHD_free() today.

I'm not aware of any API call where MHD returns a pointer to the
application that the application is expected to free.

There is one case in reverse, where for a response the application can
give a pointer to MHD which MHD will then free(), namely
MHD_RESPMEM_MUST_FREE.  Thus, using this may be unsafe on W32 as here
the application allocates and MHD frees.  However, this is purely
intended as an optimization to avoid a copy.

Happy hacking!

Christian

On 10/09/2017 04:38 PM, Tim Rühsen wrote:
> Hi,
> 
> a library included free() function is basically needed on systems where
> the library malloc heap is different from the caller's malloc heap,
> which happens on Windows when the library is a separate DLL.
> 
> 
> So, is there (or could we have) something like
> 
> void MHD_free(void *ptr)
> {
> free(ptr);
> }
> 
> ?
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Is there something like MHD_free() ?

[Christian Grothoff]

On 10/09/2017 07:24 PM, Tim Rühsen wrote:
> Hi Christian,
> 
> On Montag, 9. Oktober 2017 17:46:32 CEST Christian Grothoff wrote:
>> Hi Tim,
>>
>> We definitively don't have MHD_free() today.
>>
>> I'm not aware of any API call where MHD returns a pointer to the
>> application that the application is expected to free.
> 
> MHD_basic_auth_get_username_password ?
> 
> The docs say the pass and user should be free'd:
> 
> 10.1 Using Basic Authentication
> 
> Function: char * MHD_basic_auth_get_username_password (struct MHD_Connection 
> *connection, char** password)

Ah, yes, good point, I tend to forget about the auth APIs ;-).

In that case, I guess it would make sense to add MHD_free(), unless
there is a better way?



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Is there something like MHD_free() ?

[Christian Grothoff]

Ok, added and documented in 181ac3c5..6c4c13ef. -Christian

On 10/09/2017 09:41 PM, Tim Rühsen wrote:
> No better way comes to my mind. Most C libraries have a similar function out 
> of the same reason. It would be consistent to have it as well.
> Think of gnutls_free, idna_free, idn2_free, psl_free, ...
> 
> Regards, Tim



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] using other TLS libraries

[Christian Grothoff]

Hi Gauthier,

I agree with Evgeny, good job. However, there is one (modest) change I
would like to see, which is that the TLS backend(s) should be loaded via
dlopen()/dlsym()/libltdl instead of hard-linked with MHD. That way, we
could actually have one "libmicrohttpd.so" binary installed and the
application can choose which (of the available, possibly none) of the
TLS backends it wants to use. If we do not do this, we end up with the
libcurl situation where you never know what TLS backend is available,
and if you do need a specific one, you're in hell.

That said, I think your refactoring is _exactly_ the right step in that
direction and it should take very little to go from there where I would
like to see this go.

Finally, while I agree with Evgeny that we want to see microhttpd2.h as
the new API, I disagree that this needs to wait for the microhttpd2-API
to be finalized or even implemented. I think if someone adds the
dlopen() capability to this, it could be merged rather quickly. The main
issue with dlopen() is the logic needed to reliably identify the
(relocatable) path to the plugin/shared library, which is _always_ very
messy IMO. (see gnunet/src/util/{os_installation.c,plugin.c} for how
I've hacked up solutions for this in the past).


Happy hacking!

Christian

On 10/18/2017 04:01 PM, Gauthier Haderer wrote:
> Hello,
> 
> I recently made changes to abstract the TLS API needed by MHD. And I
> added support for OpenSSL. Could someone review my changes and tell me
> if this is something that could be part of an official version?
> 
> You will find more information here:
> https://gnunet.org/bugs/view.php?id=4917#c12475
> 
> Thank you,
> 
> 
> Gauthier




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] connections reply nothing when using libmicrohttpd

[Christian Grothoff]

Hi,

Might it be that you are using 'select()' in your event loop? (The rest
of the answer does not apply if you do not, but it is almost certain
that this is your problem).  There, the FD_SIZE limit applies (not just
the ulimit), so once you hit near 1000 FDs open from mmap, the server
won't be able to accept() incoming connections and pass them to select()
anymore.  As MHD accepted(), then got a "bad" (high) FD, its only choice
is to close() the FD, which leads exactly to the behavior your describe.


In this case, you have three possible solutions:
1) change to poll or epoll, also improving performance and scalabiltiy
2) Increase your ulimit (as you did presumably in point 4) and then
   use 'dup2' for the files you open for mmap() to move the FD into
   the range *above* 1024.  Make sure to close() the file FDs below 1024
   afterwards.  That way, MHD can keep using the low FDs for the TCP
   sockets.
3) Depending on your situation, you might also be able to simply
   open the file, call mmap(), then *close* the file but not call
   munmap() until the response is done.  That way, you don't burn an
   FD at all and don't need to do either (1) or (2).


Note that once you have more than 1000 concurrent network connections,
only (1) will really fix your problems.


Happy hacking!

Christian

On 10/21/2017 10:38 AM, Yatong Zhang wrote:
> Hi Christian,
> I built a http server with libmicrohttpd and it serves as a file content
> server. My http server opens lots of files using 'mmap' and the files
> increase daily. At first the server works fine but recently when the
> files opened about more than 1000, it begins to reply nothing. curl
> indicates that 'Server returned nothing (no headers, no data)' or
> 'Failure when receiving data from the peer‘.  So,
> 1. When the files opened are more then 1000, server sometimes close
> connections immediately. It's 'sometimes', not 'always'
> 2. The files are opened with mmap.
> 3. If the files are reduced to about 950, server works fine.
> 4. I have modified system limits, such as number opened files, but don't
> help.
> Here the limits of the server process:
> 
> cat /proc/10758/limits
> Limit Soft Limit   Hard Limit  
> Units
> Max cpu time  unlimited    unlimited   
> seconds  
> Max file size unlimited    unlimited   
> bytes
> Max data size unlimited    unlimited   
> bytes
> Max stack size    8388608  unlimited   
> bytes
> Max core file size    0    0   
> bytes
> Max resident set  unlimited    unlimited   
> bytes
> Max processes 514482   514482  
> processes
> Max open files    65535    65535   
> files
> Max locked memory 65536    65536   
> bytes
> Max address space unlimited    unlimited   
> bytes
> Max file locks    unlimited    unlimited   
> locks
> Max pending signals   514482   514482  
> signals  
> Max msgqueue size 819200   819200  
> bytes
> Max nice priority 0    0   
> Max realtime priority 0    0   
> Max realtime timeout  unlimited    unlimited    us 
> 
> 
> I have tuned all the parameters I can find including 'sysctl' stuff,
> nothing helps. I know this must have something to do with the number of
> opened files but I don't know why and how to fix it. So would you please
> help and give me some advices?
> Thanks very much.
> 
> Best regards,
> 
> Yatong Zhang



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] using other TLS libraries

[Christian Grothoff]

Yes, that is exactly what I would like to see, and AFAIK Evgeny has
started to hack on it.

-Christian

On 10/25/2017 04:20 PM, Gauthier Haderer wrote:
> Hi Christan,
> 
> I'm not sure I really understand your point.
> 
> Regarding the detection of TLS backend's availability, one can use the
> function MHD_TLS_is_feature_supported() to test if a backend is present.
> 
> Of course, if MHD was built without support for a TLS backend, a client
> won't be able to use it unless it rebuilds MHD with support enabled. If
> your intent is to be able to add this support without the rebuild
> requirement, the only option is to add a TLS plugin mecanism. This would
> mean to publish the TLS API for plugin creators, basically "tls.h" plus
> plugin glue. And maybe deliver a few standard plugins for GnuTLS,
> OpenSSL, LibreSSL along with MHD or in another package.
> 
> Is this your idea?
> 
> 
> Gauthier
>  
> 
> Hi Gauthier,
> I agree with Evgeny, good job. However, there is one (modest) change I
> would like to see, which is that the TLS backend(s) should be loaded via
> dlopen()/dlsym()/libltdl instead of hard-linked with MHD. That way, we
> could actually have one "libmicrohttpd.so" binary installed and the
> application can choose which (of the available, possibly none) of the
> TLS backends it wants to use. If we do not do this, we end up with the
> libcurl situation where you never know what TLS backend is available,
> and if you do need a specific one, you're in hell.
> That said, I think your refactoring is _exactly_ the right step in that
> direction and it should take very little to go from there where I would
> like to see this go.
> Finally, while I agree with Evgeny that we want to see microhttpd2.h as
> the new API, I disagree that this needs to wait for the microhttpd2-API
> to be finalized or even implemented. I think if someone adds the
> dlopen() capability to this, it could be merged rather quickly. The main
> issue with dlopen() is the logic needed to reliably identify the
> (relocatable) path to the plugin/shared library, which is _always_ very
> messy IMO. (see gnunet/src/util/{os_installation.c,plugin.c} for how
> I've hacked up solutions for this in the past).
> Happy hacking!
> Christian
> On 10/18/2017 04:01 PM, Gauthier Haderer wrote:
> >/Hello,
> />
> //>/I recently made changes to abstract the TLS API needed by MHD. And I
> />/added support for OpenSSL. Could someone review my changes and
> tell me
> />/if this is something that could be part of an official version?
> />
> //>/You will find more information here:
> />/https://gnunet.org/bugs/view.php?id=4917#c12475
> />
> //>/Thank you,
> />
> //>
> //>/Gauthier/
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] any plan to support http 2.0?

[Christian Grothoff]

Eh, no, microhttpd2.h has _nothing_ to do with HTTP 2.0. It is simply
about building a better interface for the existing HTTP 1.x.

In fact, I personally do not have plans to work on HTTP 2.0-support.

Happy hacking!

Christian

On 10/26/2017 01:23 AM, silvioprog wrote:
> Hi Yuyong.
> 
> An interface for HTTP2 was sent to the trunk and it is under development:
> 
> https://github.com/Karlson2k/libmicrohttpd/blob/master/src/include/microhttpd2.h
> 
> Please take a look at this topic too:
> 
> http://lists.gnu.org/archive/html/libmicrohttpd/2017-08/msg00013.html
> 
> Cheers.
> 
> On Wed, Oct 25, 2017 at 12:55 PM, Yuyong Zhang
> mailto:yuyong.zh...@casa-systems.com>>
> wrote:
> 
> Hi,
> 
> __ __
> 
> There are requirements in one project I am working on to support
> http 2.0, especially server push functionalities.
> 
> __ __
> 
> Is http 2.0 supported? If not, are there a release plan to support
> http 2.0?
> 
> __ __
> 
> Thanks a lot.
> 
> __ __
> 
> Regards,
> 
> __ __
> 
> Yuyong
> 
> 
> -- 
> Silvio Clécio



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] How to detect when the client closes the connection in libmicrohttpd

[Christian Grothoff]

You could use MHD_CONNECTION_INFO_CONNECTION_FD to obtain the socket of
a client, and then call poll/select/epoll on that yourself. Running
'recv' is likely to have unintended side-effects ;-).

Happy hacking!

Christian


On 11/02/2017 08:40 AM, Andrew Oseredchuk wrote:
> Hello,
> 
> Im writing server application based on libmicrohttpd. Clients send
> requests to server and server run a new process that is responsible for
> responce. Server process needs some time to prepare responce. During
> this time client could close connection.
> 
> Is where a way how server/libmicrohttpd could detect connection abort
> and stop server process execution? Or is where a way we can get directly
> to libmicrohttpd socket to run "recv" on it in some background thread?
> 
> Thank you in advance,
> Andrew



signature.asc
Description: OpenPGP digital signature

[libmicrohttpd] GNU libmicrohttpd 0.9.56 released

[Christian Grothoff]

Dear all,

I'm glad to announce the release of GNU libmicrohttpd 0.9.56.

GNU libmicrohttpd is a small C library that is supposed to make it easy
to run an HTTP server as part of another application. GNU libmicrohttpd
is fully HTTP 1.1 compliant and supports IPv6. Finally, GNU
libmicrohttpd is fast, portable and has a simple API and (without TLS
support and other optional features) a small binary size (~32k).

This is bugfix release.
Most noticeable changes since version 0.9.55:
* Fixed receiving large requests in TLS mode with epoll
* Various portability fixes / improvements
* added support for sendmail() on FreeBSD
* added MHD_free() to allow proper free()-ing of username/password
  on W32.
* removed non-functional support for Symbian
* various minor thread-safety improvements
* various minor optimizations and code refactoring


You can download GNU libmicrohttpd from

* ftp://ftp.gnu.org/gnu/libmicrohttpd/ and all GNU FTP mirrors.
* Our git repository at https://gnunet.org/git/libmicrohttpd.git

Please report bugs to our bugtracker at https://gnunet.org/bugs/.

The documentation (including a reference manual and tutorial) can be
found at http://www.gnu.org/software/libmicrohttpd/.


Happy hacking!

Christian




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Warnings from libmicrohttpd.h

[Christian Grothoff]

Eh, yes.

Should be fixed in 98cd69d1..867c58ad, please report if not.

Happy hacking!

Christian

On 01/03/2018 12:10 PM, Tim Rühsen wrote:
> Hi,
> 
> I see these annoying from the user-included file libmicrohttpd.h (latest
> git master). Could you fix it, please ?
> 
> /usr/oms/src/wget2/x86_64-w64-mingw32/include/microhttpd.h:215:5:
> warning: "__clang_major__" is not defined, evaluates to 0 [-Wundef]
>  #if __clang_major__+0 >= 5 || \
>  ^~~
> /usr/oms/src/wget2/x86_64-w64-mingw32/include/microhttpd.h:216:42:
> warning: "__clang_major__" is not defined, evaluates to 0 [-Wundef]
>(!defined(__apple_build_version__) && (__clang_major__+0  > 3 ||
> (__clang_major__+0 == 3 && __clang_minor__ >= 3))) || \
>   ^~~
> /usr/oms/src/wget2/x86_64-w64-mingw32/include/microhttpd.h:216:69:
> warning: "__clang_major__" is not defined, evaluates to 0 [-Wundef]
>(!defined(__apple_build_version__) && (__clang_major__+0  > 3 ||
> (__clang_major__+0 == 3 && __clang_minor__ >= 3))) || \
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

[libmicrohttpd] Fwd: [IMPORTANT] Ideas for Summer of Code 2018

[Christian Grothoff]

Dear all,

Please contribute ideas for GSoC students (especially those you might be
able to supervise) around GNU libmicrohttpd.

We should collect first on the mailinglist and then send the finished
HTML to Guiseppe.

Happy hacking!

Christian   


 Forwarded Message 
Subject: [IMPORTANT] Ideas for Summer of Code 2018
Date: Tue, 09 Jan 2018 11:06:53 +0100
From: Giuseppe Scrivano 
Reply-To: summer-of-c...@gnu.org
To: GNU Summer Of Code 

Hi!

Google is accepting applications for the next Summer of Code and as
usual we are going to apply for it. José, Darshit and myself are going
to manage the presence of GNU.

We should start thinking about a list of ideas for the next Summer of
Code and potential mentors.

This is the list of ideas we had last year:

  http://www.gnu.org/software/soc-projects/ideas-2017.html

Is there anything left undone that can be reused this year?

Please start sending ideas to add to the page and link them to the
potential mentors that can help with it.

Please send the html snippet for the ideas page, something like:

-
http://www.gnu.org/software/foo";;>GNU FOO

GNU FOO maintains their list of ideas for GSOC in an external
  webpage: http://foo.org/wanted.html";;>http://foo.org/wanted.html.

-

so that it can be embedded in the page as it is.

Regards,
Giuseppe




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Server reached connection limit. Closing inbound connection. how to increase the limits?

[Christian Grothoff]

Hi Pascal,

There are several "limits" that may apply.

(1) MHD_OPTION_CONNECTION_LIMIT -- usually 1020 (or FD_SETSIZE - 4) is
likely the first one you may hit.  You can pass a different value to
MHD_start_daemon() in the varargs;

(2) FD_SETSIZE is another limit you are likely to hit if you use the
select() event loop; tricky to bump, you ought to switch to poll/epoll
instead.

(3) ulimit (-n) may also impose a 1024 limit on your process

Happy hacking!

Christian

On 01/20/2018 08:56 AM, Menaxerius wrote:
> Hello, I'm using microlibhttpd for our Miningpool Server application.
> 
> So far so good. But due to more and more active miners is reaching I
> start reaching this limit.
> 
> Can someone explain me where those limit is set and how to increase it? 
> 
> Thanks in advance.
> 
> Regards
> 
> Pascal



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Unable to connect to libmicrohttpd in embedded system.

[Christian Grothoff]

-1 can be *normal*, as MHD calls 'accept()' (possibly via multiple
threads when using a thread pool) until there are no more clients.  You
should check what errno says.

Also, in case accept4() is buggy on your platform, it is an
optimization; so you could #undef HAVE_ACCEPT4 if accept() +
setsockopt() works better. But I've not yet heard about a buggy
accept4() implementation...

Happy hacking!

Christian

On 01/22/2018 11:32 PM, Marcel Rutten wrote:
> Found it. accept4() returns an invalid socket.
> I will carry on tomorrow, it's almost midnight in my time zone ...



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [PATCH] code style: remove tabs from source code

[Christian Grothoff]

Hi Moritz,


Removing tabs might be desirable, but this patch destroys indentation
(see stark example below), which is not.

My 2 cents.

Happy hacking!

Christian

On 01/24/2018 02:23 PM, Moritz Warning wrote:
>if (url == NULL)
>   {
> - MHD_stop_daemon (d);
> - return 16;
> +  MHD_stop_daemon (d);
> +  return 16;
>   }



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] MHD and PVS-Studio Analysis

[Christian Grothoff]

Hi silvioprog,

One of the issues was clearly real (fixed in Git), the other three I
don't get.  Looking at the code, the flag can clearly go in either
direction, and the incremented dest pointer cannot be NULL (was checked
just a few lines above).

Regardless, running such tools is always a good idea and generally
helpful, so I would definitively appreciate a full run and reports (or
patches, if the reports turn out to be well-founded).

Happy hacking!

Christian

On 01/31/2018 11:30 PM, silvioprog wrote:
> Hi dudes.
> 
> PVS-Studio [1]  is a Russian tool
> (Windows/Linux) for bug detection in the source code of programs written
> in C/C++.
> 
> The tool have been issued bugs in many popular projects like LLVM/GCC,
> cURL, OpenSSL, Doxygen, OpenCV, GTK, glibc, tor etc. The full list is
> available at PVS-Studio's inspections page [2]
> . It has helped the authors to
> fix the reported issues [3]
> .
> 
> I have been using the free version of PVS-Studio for analyzing open
> source projects I use, then it reported issues [4]
>  in "Medium
> level" in four MHD files
> . The attached
> tar.gz contains a few pictures showing the testing result and text files
> containing respective links to access each issue explanation.
> 
> If you agree with fixing those issues, I can run a full test in all MHD
> files and share all the reported issues (I fix the ones possible for me).
> 
> [1] PVS-Studio page, < https://www.viva64.com/en/pvs-studio
>  >
> [2] PVS-Studio inspections, < https://www.viva64.com/en/
> inspections
>  >
> [3] cURL fixes based on PVS-Studio issuing,
> < https://github.com/curl/curl/search?q=pvs&type=Issues
>  >
> [4] first MHD report using PVS-Studio free for open source projects,
> < https://duallsistemas.com.br/download/pvs_mhd/fullhtml
>  >
> 
> --
> Silvio Clécio



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] "Failed to join a thread": Race condition when closing the connection?

[Christian Grothoff]

Hi Jan-Benedict,

This is very strange, the loop should not fix this, as pthread_join
should simply block (not race!) until the thread is done.  In fact, I
generally think the right answer to ESRCH would be to die, as to me this
indicates some kind of memory corruption or other severe invariant
violation.

Now, given that you mentioned changes related to popen()-logic in your
own code, I wonder if the change in your _application_ logic related to
fork() may be interoperating badly with threads.  In particular, after
you fork(), all of the "other" threads will be gone, so if you fork()
and then continue any MHD-interaction related to the threads spawned by
MHD, that is likely to be, eh, problematic --- and may show up with an
ESRCH.  However, that doesn't quite explain to me why putting this in a
loop with sleeps might fix it. (But I don't know enough about your code.)

Regardless, the loop/sleep is a very, very wrong fix, and I strongly
suspect the problem is in your code (or how you use the MHD API, in
conjunction with fork()).

My 2 cents.

Happy hacking!

Christian

On 02/01/2018 06:21 PM, Jan-Benedict Glaw wrote:
> Testing it a bit, I found out that pthread_join() returns 3
> (== ESRCH), which shall happen when "No thread with the ID thread
> could be found." (man 3 pthread_join). I would have expected
> "EINVAL thread is not a joinable thread." though...
> 
>   However, putting the pthread_join() or MHD_join_thread_() into a
> small loop with a few msec delay in between fixed it, with an
> occassional hint that it took a second try to reap the thread.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Any MHD_ContentReaderCallback Example Code

[Christian Grothoff]

On 02/04/2018 03:19 AM, lingweicai wrote:
> Dear all:
> 
> I want to create responses with strings variables, for example , html
> start tag, html title, css, js, head1, content, html end tag , etc.
> 
> Can I use this callback function instead of
> MHD_create_response_from_buffer?

In principle you can use MHD_create_response_from_callback() always
instead of _from_buffer, as _from_callback is more general. Whether it
makes sense in your case I cannot judge.

> I have not found any example codes on the callback.

Look here: src/examples/fileserver_example_dirs.c



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Clang build fails

[Christian Grothoff]

Hi,

Build issue should be fixed in Git now.

IIRC, our CI (Buildbot) is not (yet) setup to build with clang, but also
it doesn't send out proper notifications on failures in the first place,
so it is right now of limited use.

Help with setting up better CIs would always be welcome, albeit I
personally prefer using our "own" infrastructure (we have machines)
instead of depending on some corporate hosting.  Generally, I would like
to end up with a setup where the entire CI configuration is also in a
Git repo and can be easily collaboratively improved. So if you have time
and energy to make that happen, great ;-)

Happy hacking!

Christian



On 02/21/2018 10:17 AM, Tim Rühsen wrote:
> Hi,
> 
> currently, clang can't build latest master. It because of a missing
> prototype for MHD_init() in src/lib/init.c.
> 
> Clang errors at that point while gcc just emits a warning.
> 
> Anyways, this breaks building the dependency chain for Wget2 fuzzing :-(
> 
> 
> Do you make a CI build before pushing to master ? If you want me to help
> to setup a CI on Gitlab, let me know.
> 
> 
> init.c:56:5: warning: implicit declaration of function 'MHD_init' is
> invalid in C99 [-Wimplicit-function-declaration]
> MHD_init ();
> ^
> init.c:98:1: error: conflicting types for 'MHD_init'
> MHD_init (void)
> ^
> init.c:56:5: note: previous implicit declaration is here
> MHD_init ();
> ^
> 1 warning and 1 error generated.
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Clang build fails

[Christian Grothoff]

On 02/21/2018 12:15 PM, Tim Rühsen wrote:
>> Generally, I would like
>> to end up with a setup where the entire CI configuration is also in a
>> Git repo and can be easily collaboratively improved. So if you have time
>> and energy to make that happen, great ;-)
> 
> Gitlab allows exactly that,  but you have your own CI already running
> and the missing parts are maybe just a piece of configuration.
> So it has to be your initiative.
> 
> If you like to give Gitlab a chance, let me know and I surely can help.
> You can also have your own Gitlab instance without gitlab.com involved.
> And BTW, you can use gitlab-ci-multi-runner to use your own machines /
> OSes as CI runners. The gitlab.com CI hosting is limited to docker, but
> hey - it's free to use. And since the CI setup is developed in a git
> repo as well, nothing is lost, even if gitlab.com dies.

Well, as long as we can easily export the result from gitlab.com to
eventually move it to independent infrastructure, there's nothing wrong
with starting a CI system there first. So please do give it a shot!

Would it be useful to create a new mailinglist for CI notifications for
this, or should we just use this list?




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] 回复:Re: Any MHD_ContentReaderCallback Example Code

[Christian Grothoff]

Hi lingweicai,

As you cannot 'update' cls, just use malloc a

struct MyContext {
  struct MyList *head_ptr;
  // possibly more stuff here
} *ctx;

and pass 'ctx' as 'cls' instead of passing 'head_ptr' directly.
That way, you can easily update 'head_ptr'.

Happy hacking!

Christian

On 02/21/2018 12:48 PM, lingweicai wrote:
> 
> Thanks Christian.
> 
>  
> 
> Can I create a MHD_response, by the crc callback to print a circleq ( a
> circle queue,generated with Macro functions /sys/queue.h ), my crc
> callback is like this :
> 
>  
> 
> ssize_t
> crc (void *cls, uint64_t pos, char *buf, size_t max)
> {
>     struct circleq_user  * cq_user_ptr = cls; 
> 
> // if last entry, end the crc 
>     if ( cq_user_ptr->entries.cqe_next == (void *) &head )
> 
>     return MHD_CONTENT_READER_END_OF_STREAM;
> 
> 
> // pointer of next entry to assigned to cls 
>     cls  = cq_user_ptr->entries.cqe_next ;
> 
> // generate buff content
> 
>     return snprintf (buf, 256,
>    "%s",
>    cq_user_ptr->user_mem_list_ptr->u_base.u_name ,
>    cq_user_ptr->user_mem_list_ptr->u_base.u_name
>     );
> } 
> 
>  
> 
> when my circle-head pointer as the parameter of cls for crc, the above
> code always print the first entry of my circle queue. Can I assign the
> next pointer of
> "cls  = cq_user_ptr->entries.cqe_next ; " 
> 
> to re-callback the crc ? it looks this does not work. how can I transfer
> the "next pointer" of my list as the parameter to the next call crc ?
> 
> 
> Thanks,
> 
> Forrest Ling wei cai.
> 
> 
>  
> 
>  
> 
> 
>  
> 
>  
> 
>  
> 
> - 原始邮件 -
> 
>  
> 
> *发件人：*Christian Grothoff
> 
> *发送时间：*2018-02-05 21:58:17
> 
> *收件
> 人：*lingweicai;libmicrohttpd
> 
> *主 题：*Re: Any MHD_ContentReaderCallback Example Code
> 
> On 02/04/2018 03:19 AM, lingweicai wrote:
>> Dear all:
>>
>> I want to create responses with strings variables, for example , html
>> start tag, html title, css, js, head1, content, html end tag , etc.
>>
>> Can I use this callback function instead of
>> MHD_create_response_from_buffer?
> 
> In principle you can use MHD_create_response_from_callback() always
> instead of _from_buffer, as _from_callback is more general. Whether it
> makes sense in your case I cannot judge.
> 
>> I have not found any example codes on the callback.
> 
> Look here: src/examples/fileserver_example_dirs.c
> 
> 
> 
> 
> <https://mail.sohu.com/activity/prettyemail/views/>
> 
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Gitlab CI still fails

[Christian Grothoff]

These should be fixed (largely warnings about memory leaks in testcases,
one relating to an actual issue with MHD not calling connection
completed handler for upgraded connections).

Happy hacking!

Christian

On 02/27/2018 10:18 AM, Tim Rühsen wrote:
> Hi,
> 
> two tests still fail because of memory leaks. It's reproducible with
> 
> CC=gcc CFLAGS="-O0 -g -ggdb3 -fno-omit-frame-pointer -fsanitize=address"
> ./configure
> 
> make clean
> 
> make check
> 
> ...
> 
> FAIL: test_upgrade
> FAIL: test_upgrade_tls
> ...
> 
> 
> cat src/microhttpd/test_upgrade.log
> 
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Another CI failure ...

[Christian Grothoff]

Fixed in Git f4154e6b..4507f419. -Christian

On 02/27/2018 10:30 AM, Tim Rühsen wrote:
> Two more usages of strcpy() annoy the clang static analyzer:
> 
> $ make clean
> 
> $ scan-build-3.9 -v -enable-checker security,nullability --status-bugs
> make -j$(nproc)
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] make distcheck fails

[Christian Grothoff]

Merged, but applied small additional correction as I do not want the new
headers to be installed in systems yet (too early).

Happy hacking!

Christian
p.s.: I'm aware of the test_upgrade test failure, it's tricky, so will
take a bit more time to fix properly.

On 02/28/2018 11:55 AM, Tim Rühsen wrote:
> There now is a fix for 'make distcheck' and the clang's scan-build in
> branch gitlab/ci-test (if you like to merge that).
> 
> 
> With Best Regards, Tim
> 
> 
> 
> On 02/28/2018 09:50 AM, Tim Rühsen wrote:
>> $ ./bootstrap && ./configure --enable-asserts && make clean
>>
>> $ make distcheck
>>
>>
>> In file included from ../../../../src/lib/action_from_response.c:25:0:
>> ../../../../src/lib/internal.h:32:10: fatal error: microhttpd2.h: No
>> such file or directory
>>  #include "microhttpd2.h"
>>   ^~~
>>
>>
>> With Best Regards, Tim
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Possible bug on macOS Sierra

[Christian Grothoff]

Try casting the 8*1024*1024 to a "size_t" explicitly. I suspect on your
Mac-Machine, you have maybe a 64-bit size_t and then things go very
wrong if you pass a 32-bit argument.

Note that the microhttpd2-API that I've started to hack on is supposed
to categorically eliminate such API-usage bugs, but it'll take quite
some time to be implemented (main lesson I've learned so far: the change
is HUGE).

Happy hacking!

Christian

On 02/28/2018 12:05 PM, Giovanni Mascellani wrote:
> Dear developers of microhttpd,
> 
> trying to port an application using microhttpd to macOS, I hit a strange
> behaviour. Could you help me understanding whether it is my fault or a
> hint of a bug?
> 
> Consider the attached file micro.c, which is the example on [1] with a
> few modifications in the parameters of MHD_start_daemon. If I compile it
> on Linux (Debian unstable with distributed microhttpd), it works fine.
> 
>  [1] https://www.gnu.org/software/libmicrohttpd/
> 
> If I compile it on macOS Sierra (with microhttpd 0.9.59 from brew), the
> compilation succeeds, but when I execute:
> 
> $ ./micro 1234
> Failed to create listen thread: Resource temporarily unavailable
> 
> However, if I switch MHD_OPTION_NOTIFY_COMPLETED and
> MHD_OPTION_THREAD_STACK_SIZE (together with their arguments), or if I
> remove either of the two, then it compiles and runs correctly. Is this
> the intended behaviour?
> 
> (to my actual case it matters very few, since switching back the two
> options is perfectly fine; however, this seems funny, and I thought that
> you might want to check it thoroughly, if this is not meant to happen)
> 
> Please Cc me when replying, I am not subscribed to the list.
> 
> Thanks for your work on microhttpd!
> 
> Giovanni.
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Request: stricter checks for client input

[Christian Grothoff]

Should be implemented in 785ae52b..5717a9ec, feedback welcome... -Christian

On 03/01/2018 09:21 AM, Tim Rühsen wrote:
> Hi,
> 
> could MHD_OPTION_STRICT_FOR_CLIENT be more strict regarding spaces in
> the 'resource' of HTTP requests, please ?
> 
> Currently MHD accepts spaces (instead %20 resp. +) and converts them to
> + for the return value of MHD_get_connection_values().
> 
> 
> This makes a test for wget2 succeed though wget2 doesn't send a correct
> request line. (Only on older versions of MHD this test fails).
> 
> The appropriate code seems to be in parse_initial_message_line():
> /* Search from back to accept misformed URI with space */
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] test_upgrade aborts (and thus FAILS)

[Christian Grothoff]

Should be fixed now. Tough one, fortunately unlikely to affect most
people: for some threading models, upgraded connections didn't call the
per-request clean up function, and for all multi-threaded models during
daemon stop, we might call the per-request clean up function for
not-yet-finished connections from a different thread (namely the main
thread instead of the worker).  Testcase checked for that, and
aborted... ;-). -Christian

On 02/28/2018 09:46 AM, Tim Rühsen wrote:
> Reported by Gitlab CI and reproducible here (amd64):
> 
> $ ./bootstrap && ./configure --enable-asserts && make clean
> 
> $ TESTS_ENVIRONMENT="valgrind --trace-children=yes" make check
> 
> 
> src/microhttpd/test_upgrade.c FAILs, log file contains:
> 
> ==2363== Process terminating with default action of signal 6 (SIGABRT)
> ==2363==at 0x561C6A0: raise (raise.c:51)
> ==2363==by 0x561DCF6: abort (abort.c:90)
> ==2363==by 0x10A1E1: notify_completed_cb (test_upgrade.c:457)
> ==2363==by 0x51A8A45: MHD_connection_close_ (connection.c:950)
> ==2363==by 0x51B33D6: resume_suspended_connections (daemon.c:2793)
> ==2363==by 0x51BA34A: MHD_stop_daemon (daemon.c:6342)
> ==2363==by 0x10AEE5: test_upgrade (test_upgrade.c:1090)
> ==2363==by 0x10B43D: main (test_upgrade.c:1240)
> 
> 
> With Best Regards, Tim
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Three runtime errors in tests with UBSAN

[Christian Grothoff]

On 02/27/2018 10:39 AM, Tim Rühsen wrote:
> $ CC=gcc CFLAGS="-O0 -g -ggdb3 -fno-omit-frame-pointer
> -fsanitize=undefined" ./configure
> 
> $ make clean
> $ make check
> $ grep runtime src/*/*.log
> 
> src/microhttpd/test_upgrade.log:test_upgrade.c:1075:32: runtime error:
> member access within misaligned address 0x560144586014 for type 'const
> union MHD_DaemonInfo', which requires 8 byte alignment
> 
> src/microhttpd/test_upgrade_tls.log:test_upgrade.c:1075:32: runtime
> error: member access within misaligned address 0x55cea9a95014 for type
> 'const union MHD_DaemonInfo', which requires 8 byte alignment

I'm not sure I agree with these two, as while we return a 'union' with
an 8-byte alignment, the only valid *field* we return is 16-bit, and the
union overall is read-only.  So reading the 16-bit field should be
perfectly safe from an alignment perspective (it is 16-bit aligned), and
everything else is a really bad idea (TM) anyway, as it would read garbage.

We could theoretically fix this, but to avoid ABI breakage this would
require using more memory (effectively reserving a fresh union for each
possible return case) and I do not see how this would gain anything for
anyone except making the sanitizer happy.

A better solution is of course the upcoming microhttpd2-API, which
avoids this issue entirely (by not returning a union).

For now, I've just hacked the testcase to memcpy() the port, which
doesn't actually improve the code but eliminates the warning. 8-)

> src/testcurl/test_put_chunked.log:test_put_chunked.c:107:16: runtime
> error: null pointer passed as argument 1, which is declared to never be null

Fixed.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] MinGW build failures

[Christian Grothoff]

Might be fixed in f60a569e..a7b25581, please let me know if not (or
rather Evgeny, as he'll know better what to do ;-)).

Happy hacking!

Christian

On 03/04/2018 10:34 AM, Tim Ruehsen wrote:
> Hi,
> 
> just saw that the wget2 MinGW build breaks since a while. Could you
> have a look ?
> 
> With Best Regards, Tim
> 
> 
> In file included from internal.h:32:0,
>  from action_continue.c:25:
> ../../src/include/microhttpd2.h: In function ‘MHD_daemon_create’:
> ../../src/include/microhttpd2.h:277:26: error: unknown type name
> ‘__THROW’
>  #define MHD_NONNULL(...) __THROW __nonnull((__VA_ARGS__))
>   ^
> ../../src/include/microhttpd2.h:1485:3: note: in expansion of macro
> ‘MHD_NONNULL’
>MHD_NONNULL(1);
>^~~
> ../../src/include/microhttpd2.h:277:44: error: expected declaration
> specifiers or ‘...’ before ‘(’ token
>  #define MHD_NONNULL(...) __THROW __nonnull((__VA_ARGS__))
> ^
> ../../src/include/microhttpd2.h:1485:3: note: in expansion of macro
> ‘MHD_NONNULL’
>MHD_NONNULL(1);
>^~~
> In file included from ../../src/include/mhd_options.h:34:0,
>  from internal.h:30,
>  from action_continue.c:25:
> ../../MHD_config.h:543:21: error: expected declaration specifiers
> before ‘__attribute__’
>  #define _MHD_EXTERN __attribute__((visibility("default")))
> __declspec(dllexport) extern
>  ^
> ../../src/include/microhttpd2.h:1496:1: note: in expansion of macro
> ‘_MHD_EXTERN’
>  _MHD_EXTERN enum MHD_StatusCode
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] MinGW build failures

[Christian Grothoff]

On 03/04/2018 12:47 PM, Tim Ruehsen wrote:
> Am Sonntag, den 04.03.2018, 11:40 +0100 schrieb Christian Grothoff:
>> Might be fixed in f60a569e..a7b25581, please let me know if not (or
>> rather Evgeny, as he'll know better what to do ;-)).
> 
> Thanks, the reported issues are fixed but now stopping at
> 
> In file included from action_continue.c:25:0:
> internal.h:842:23: error: field ‘celi’ has incomplete type
>enum MHD_EpollState celi;
>^~~~
> make[3]: *** [Makefile:626: libmicrohttpd2_la-action_continue.lo]
> Fehler 1
> make[3]: Verzeichnis „/home/tim/src/wget2/libmicrohttpd/src/lib“ wird
> verlassen

Fixed.

> and
> 
> ddaemon_start.c:116:23: note: each undeclared identifier is reported
> only once for each function it appears in
> daemon_options.c:26:10: fatal error: dlfcn.h: No such file or directory
>  #include 
>   ^
> compilation terminated.

I've added a configure-check for this (e3ff8946..77398cdb).  To build
this part of the logic on W32, you need to install dlfcn-win32:

https://stackoverflow.com/questions/1397022/mingw-3-4-5-missing-dlfcn-h




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Suspend/resume with single thread and external epoll not sending response

[Christian Grothoff]

Dear Bob,

I've analyzed your code, and the issue is on your end: you simply didn't
set the timeout correctly. When using an external event loop, it is
mandatory that you ask MHD for the timeout using MHD_get_timeout() and
use that with select/poll/epoll.  Then, you must call MHD_run() once
whenever epoll() returns (including timeouts!).

In your case, MHD would have given you a timeout of 0, but you used
infinity instead, with predictable results...

I've attached a corrected version of the code.

Happy hacking!

Christian

On 03/02/2018 08:26 PM, Robert D Kocisko wrote:
> First, thanks for your amazing work on MHD!
> 
> This question is a near duplicate of the 2014 message thread from Tom
> Cornell entitled "Trouble getting a response sent from a separate worker
> thread (with external select)".  However, I am not using separate worker
> threads--everything is in one thread and so I don't think the
> recommendations found in that thread apply to my scenario.  
> 
> Basically after receiving a request from an HTTP client, I want to be
> able to do some asynchronous 'work' which is really just waiting on
> another process such as a database engine to calculate and return the
> result which, when complete, I will forward back to the client.  This is
> all done in one thread using epoll, so I don't want any blocking and I
> don't want any busy loops.  MHD's external epoll support combined with
> suspend/resume fits into this architecture perfectly, but there's a
> problem: after resuming the connection and queueing the data, the
> headers are sent to the client immediately, but the body of the response
> does not get sent until another client request arrives.  
> 
> Anyway, to make this all concrete, I've put together a small working
> example (below) which shows the problem.  This is built against the
> latest dev rev (7f1dbb2) on elementaryOS (which is Ubuntu 16.04).  Every
> time a request comes in it suspends the connection and starts a 1 second
> timer which, when it expires, resumes the connection.  When the
> connection is resumed the response is queued (simply echos the request
> url).  I realize this example leaks timer fds and doesn't clean up
> properly but it successfully demonstrates the problem.
> 
> I have experimented with calling MHD_run() twice after
> MHD_resume_connection() rather than the once required by the docs, and
> that does seem to work, but that seems extremely hacky and I'm not sure
> if twice is enough (why twice and not three times?).  I've skimmed the
> source code looking for obvious answers but none are readily apparent to me.
> 
> At this point I'm pretty sure that this is a bug with MHD but am I
> missing something?  
> 
> Thanks!
> Bob Kocisko
> 
> -
> 
> #include "platform.h"
> #include 
> #include 
> #include 
> 
> #define TIMEOUT_INFINITE -1
> 
> struct Request {
>   struct MHD_Connection *connection;
>   int timerfd;
> };
> 
> int epfd;
> struct epoll_event evt;
> 
> static int
> ahc_echo (void *cls,
>           struct MHD_Connection *connection,
>           const char *url,
>           const char *method,
>           const char *version,
>           const char *upload_data, size_t *upload_data_size, void **ptr)
> {
>   struct MHD_Response *response;
>   int ret;
>   struct Request* req;
>   struct itimerspec ts;
>   (void)url;               /* Unused. Silent compiler warning. */
>   (void)version;           /* Unused. Silent compiler warning. */
>   (void)upload_data;       /* Unused. Silent compiler warning. */
>   (void)upload_data_size;  /* Unused. Silent compiler warning. */
> 
>   req = *ptr;
>   if (!req)
>   {
> 
>     req = malloc(sizeof(struct Request));
>     req->connection = connection;
>     req->timerfd = 0;
>     *ptr = req;
>     return MHD_YES;
>   }
> 
>   if (req->timerfd)
>   {
>     // send response (echo request url)
>     response = MHD_create_response_from_buffer (strlen (url),
>                                                 (void *) url,
>                                                 MHD_RESPMEM_MUST_COPY);
>     ret = MHD_queue_response (connection, MHD_HTTP_OK, response);
>     MHD_destroy_response (response);
>     return ret;
>   }
>   else
>   {
>     // create timer and suspend connection
>     req->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK);
>     if (-1 == req->timerfd)
>     {
>       printf("timerfd_create: %s", strerror(errno));
>       return MHD_NO;
>     }
>     evt.events = EPOLLIN;
>     evt.data.ptr = req;
>     if (-1 == epoll_ctl(epfd, EPOLL_CTL_ADD, req->timerfd, &evt))
>     {
>       printf("epoll_ctl: %s", strerror(errno));
>       return MHD_NO;
>     }
>     ts.it_value.tv_sec = 1;
>     ts.it_value.tv_nsec = 0;
>     ts.it_interval.tv_sec = 0;
>     ts.it_interval.tv_nsec = 0;
>     if (-1 == timerfd_settime(req->timerfd, 0, &ts, NULL))
>     {
>       printf("timerfd_settime: %s", strerror(errno));
>       return MHD_NO;
>     }
> 
>     MHD_suspend_connection(connection);

Re: [libmicrohttpd] Suspend/resume with single thread and external epoll not sending response

[Christian Grothoff]

Done: 85913afa..7d7ccbcd

Thanks for the suggestion!

-Christian

On 03/13/2018 07:09 PM, Robert D Kocisko wrote:
> Hi Christian,
> 
> Thank you!  Very cool to see the example working.  Funny, I
> implemented MHD_get_timeout in the main loop in my project but in all
> my tests it always returned MHD_NO, so I assumed it was not applicable
> to my use case and didn't include it in the example I wrote.  But I
> didn't make the connection that it *also* had to be called after
> MHD_resume_connection until your response here.  Would you consider
> the attached patch?   It adds one line to the docs that I'm quite sure
> would have pointed me in the right direction had it been there before.
> 
> Thanks!
> Bob
> 
> On Sun, Mar 11, 2018 at 1:11 PM, Christian Grothoff  
> wrote:
>>
>> Dear Bob,
>>
>> I've analyzed your code, and the issue is on your end: you simply didn't
>> set the timeout correctly. When using an external event loop, it is
>> mandatory that you ask MHD for the timeout using MHD_get_timeout() and
>> use that with select/poll/epoll.  Then, you must call MHD_run() once
>> whenever epoll() returns (including timeouts!).
>>
>> In your case, MHD would have given you a timeout of 0, but you used
>> infinity instead, with predictable results...
>>
>> I've attached a corrected version of the code.
>>
>> Happy hacking!
>>
>> Christian
>>
>> On 03/02/2018 08:26 PM, Robert D Kocisko wrote:
>>> First, thanks for your amazing work on MHD!
>>>
>>> This question is a near duplicate of the 2014 message thread from Tom
>>> Cornell entitled "Trouble getting a response sent from a separate worker
>>> thread (with external select)".  However, I am not using separate worker
>>> threads--everything is in one thread and so I don't think the
>>> recommendations found in that thread apply to my scenario.
>>>
>>> Basically after receiving a request from an HTTP client, I want to be
>>> able to do some asynchronous 'work' which is really just waiting on
>>> another process such as a database engine to calculate and return the
>>> result which, when complete, I will forward back to the client.  This is
>>> all done in one thread using epoll, so I don't want any blocking and I
>>> don't want any busy loops.  MHD's external epoll support combined with
>>> suspend/resume fits into this architecture perfectly, but there's a
>>> problem: after resuming the connection and queueing the data, the
>>> headers are sent to the client immediately, but the body of the response
>>> does not get sent until another client request arrives.
>>>
>>> Anyway, to make this all concrete, I've put together a small working
>>> example (below) which shows the problem.  This is built against the
>>> latest dev rev (7f1dbb2) on elementaryOS (which is Ubuntu 16.04).  Every
>>> time a request comes in it suspends the connection and starts a 1 second
>>> timer which, when it expires, resumes the connection.  When the
>>> connection is resumed the response is queued (simply echos the request
>>> url).  I realize this example leaks timer fds and doesn't clean up
>>> properly but it successfully demonstrates the problem.
>>>
>>> I have experimented with calling MHD_run() twice after
>>> MHD_resume_connection() rather than the once required by the docs, and
>>> that does seem to work, but that seems extremely hacky and I'm not sure
>>> if twice is enough (why twice and not three times?).  I've skimmed the
>>> source code looking for obvious answers but none are readily apparent to me.
>>>
>>> At this point I'm pretty sure that this is a bug with MHD but am I
>>> missing something?
>>>
>>> Thanks!
>>> Bob Kocisko
>>>
>>> -
>>>
>>> #include "platform.h"
>>> #include 
>>> #include 
>>> #include 
>>>
>>> #define TIMEOUT_INFINITE -1
>>>
>>> struct Request {
>>>   struct MHD_Connection *connection;
>>>   int timerfd;
>>> };
>>>
>>> int epfd;
>>> struct epoll_event evt;
>>>
>>> static int
>>> ahc_echo (void *cls,
>>>   struct MHD_Connection *connection,
>>>   const char *url,
>>>   const char *method,
>>>   const char *version,
>>>   const char *upload_data, size_

Re: [libmicrohttpd] Suspend/resume with single thread and external epoll not sending response

[Christian Grothoff]

Thanks for agreeing.

Added as src/examples/suspend_resume_epoll.c in 7d7ccbcd..8abd74f3.

Happy hacking!

Christian

On 03/13/2018 07:11 PM, Robert D Kocisko wrote:
> I agree with Silvio, it would be great to have this example included
> in the official examples.
> 
> On Sun, Mar 11, 2018 at 2:11 PM, silvioprog  wrote:
>> Hello dudes.
>>
>> It would be nice to attach this example (or Christian's explanation) in
>> MHD's docs/examples. :-)
>>
>> Thank you!
>>
>> On Sun, Mar 11, 2018 at 2:11 PM, Christian Grothoff 
>> wrote:
>>>
>>> Dear Bob,
>>>
>>> I've analyzed your code, and the issue is on your end: you simply didn't
>>> set the timeout correctly. When using an external event loop, it is
>>> mandatory that you ask MHD for the timeout using MHD_get_timeout() and
>>> use that with select/poll/epoll.  Then, you must call MHD_run() once
>>> whenever epoll() returns (including timeouts!).
>>>
>>> In your case, MHD would have given you a timeout of 0, but you used
>>> infinity instead, with predictable results...
>>>
>>> I've attached a corrected version of the code.
>>>
>>> Happy hacking!
>>>
>>> Christian
>>>
>>> On 03/02/2018 08:26 PM, Robert D Kocisko wrote:
>>>> First, thanks for your amazing work on MHD!
>>>>
>>>> This question is a near duplicate of the 2014 message thread from Tom
>>>> Cornell entitled "Trouble getting a response sent from a separate worker
>>>> thread (with external select)".  However, I am not using separate worker
>>>> threads--everything is in one thread and so I don't think the
>>>> recommendations found in that thread apply to my scenario.
>>>>
>>>> Basically after receiving a request from an HTTP client, I want to be
>>>> able to do some asynchronous 'work' which is really just waiting on
>>>> another process such as a database engine to calculate and return the
>>>> result which, when complete, I will forward back to the client.  This is
>>>> all done in one thread using epoll, so I don't want any blocking and I
>>>> don't want any busy loops.  MHD's external epoll support combined with
>>>> suspend/resume fits into this architecture perfectly, but there's a
>>>> problem: after resuming the connection and queueing the data, the
>>>> headers are sent to the client immediately, but the body of the response
>>>> does not get sent until another client request arrives.
>>>>
>>>> Anyway, to make this all concrete, I've put together a small working
>>>> example (below) which shows the problem.  This is built against the
>>>> latest dev rev (7f1dbb2) on elementaryOS (which is Ubuntu 16.04).  Every
>>>> time a request comes in it suspends the connection and starts a 1 second
>>>> timer which, when it expires, resumes the connection.  When the
>>>> connection is resumed the response is queued (simply echos the request
>>>> url).  I realize this example leaks timer fds and doesn't clean up
>>>> properly but it successfully demonstrates the problem.
>>>>
>>>> I have experimented with calling MHD_run() twice after
>>>> MHD_resume_connection() rather than the once required by the docs, and
>>>> that does seem to work, but that seems extremely hacky and I'm not sure
>>>> if twice is enough (why twice and not three times?).  I've skimmed the
>>>> source code looking for obvious answers but none are readily apparent to
>>>> me.
>>>>
>>>> At this point I'm pretty sure that this is a bug with MHD but am I
>>>> missing something?
>>>>
>>>> Thanks!
>>>> Bob Kocisko
>>>>
>>>> -
>>>>
>>>> #include "platform.h"
>>>> #include 
>>>> #include 
>>>> #include 
>>>>
>>>> #define TIMEOUT_INFINITE -1
>>>>
>>>> struct Request {
>>>>   struct MHD_Connection *connection;
>>>>   int timerfd;
>>>> };
>>>>
>>>> int epfd;
>>>> struct epoll_event evt;
>>>>
>>>> static int
>>>> ahc_echo (void *cls,
>>>>   struct MHD_Connection *connection,
>>>>   const char *url,
>>>>   const char *me

Re: [libmicrohttpd] Advise about upgrade "facility"

[Christian Grothoff]

On 03/16/2018 03:40 PM, José Bollo wrote:
> Hi all,
> 
> I searched last two days a "double free" bug that appeared using latest
> master for my implementation of websockets using MHD upgrade "facility".
> 
> After bisecting, it appears that the commit 
> 
>>> commit 0ebdff94d461abd4328cf45a6281c15139a045eb
>>> Author: Christian Grothoff 
>>> Date:   Fri Mar 2 21:45:17 2018 +0100
>>>
>>> fix test_upgrade transient failures (#5189)
> 
> introduces the change.
> 
> Before it, I had to free manually the resource attached to the
> connection. After it, freeing it in the MHD_OPTION_NOTIFY_COMPLETED
> callback "as usual" works (didn't work before). This lead in my code to
> a dual free of resources.

Yep, that's why it was a bug ;-).

> For me using the callback of MHD_OPTION_NOTIFY_COMPLETED in all cases
> (after upgrade or not) is produce the lightest code.
> 
> 2 questions:
> 
>  - is it possible to increase the version to 0x00095901 for testing the
>case and adapt my code?
> 
> #define MHD_VERSION   0x00095901

Eh, the MHD_VERSION is already on 0x00095902, and yes, it should be safe
to teset for that increase.

>  - in my case, the resource is not used and was released in the upgrade
>callback. Is there a way to force the con_cls of a connection? If
>yes I can release the resources earlier.

This is not possible with the current API.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] How to proper fix error: "ISO C90 does not support ‘long long’"?

[Christian Grothoff]

Changing 'long long' will break ABI compatibility. So this is not
something you want to do for a mere warning in response to you trying
out compiler flags to try settings nobody has needed for 20+ years.

On 03/24/2018 04:30 AM, silvioprog wrote:
> Christian,
> 
> I'm using the following compilers:
> 
> 1) clang version 5.0.1-svn325091-1~exp1 (branches/release_50)
> 
> 2) arm-linux-androideabi-gcc (Linaro GCC 4.7-2014.04-1~dev) 4.7.4
> 
> 3) gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
> 
> clang and arm-linux-androideabi-gcc compiles MHD fine, but gcc raises
> the following error:
> 
> [snip]
> /libmicrohttpd/include/microhttpd.h:277:46: error: ISO C90 does not
> support ‘long long’ [-Werror=long-long]
>  #define MHD_UNSIGNED_LONG_LONG unsigned long long
>                                               ^
> /libmicrohttpd/include/microhttpd.h:2348:4: note: in expansion of macro
> ‘MHD_UNSIGNED_LONG_LONG’
>     MHD_UNSIGNED_LONG_LONG *timeout);
>     ^
> cc1: all warnings being treated as errors
> 
> I fixed it using the attached patch, but I'm not sure if it is the
> proper way to fix it.
> 
> WDYT about this change?
> 
> --
> Silvio Clécio



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] New project: microhttpd.h or microhttpd2.h?

[Christian Grothoff]

On 03/25/2018 04:05 AM, silvioprog wrote:
> Hello.
> 
> I'm going to create a new project from scratch that uses MHD as main
> HTTP library, so I'm free to choose a new API. However, I have some
> questions that may help me to choose between microhttpd.h and microhttpd2.h:
> 
> * can I use the new API in production? (I'm following MHD changes, but
> I'm not sure about the new API development status)

No. It compiles, but there are chunks of code missing to even get the
minimum things to work.

> * will the previous API still receiving new features and updates?
> (specially for websockets)

If necessary, sure. I expect to maintain both APIs for years in parallel.

> * is there a tutorial or reference to study the new API? (this two
> documents of the previous API helped me a lot: 1

Not yet.

>  / 2
> )
> * is there examples or tests (even drafts) showing how to use the new API?

Not yet.

> I'm OK to study (and contribute) the new API and I'm inclined to choose
> it because its simpler/friendly functions, MHD_Action etc., but if you
> don't recommend to use it for while I'm happy using the current stable
> API anyway. ;-)

Always happy for contributions, and translating existing (old API)
testcases, examples and documentation to 'new API' would definitively be
a good starting point, but be aware that even if done correctly, the
code could not yet possibly work.




signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] libmicrohttpd installation on Solaris 10

[Christian Grothoff]

Dear Marek,

Thanks for your report.  As for the "test_options" failure, I wonder
what happens on Solaris 10 if you "bind()" a socket to the port 0.
Given the error message, it seems that the semantics are different from
those on other platforms: on GNU/Linux, bind()ing to 0 means "pick a
random free port".  If on Solaris this is simply illegal, that would
explain the test failure.

I don't have a Solaris 10 box. Can you confirm this is the case? If so,
we need to document this limitation on Solaris (bind-to-random-port not
supported) and modify the test to "skip" that check.

Happy hacking!

Christian

On 04/17/2018 09:18 PM, Marek, James wrote:
> FAIL: test_options
> 
> ==
> 
>  
> 
> Failed to bind to port 0: Cannot assign requested address
> 
> running test: ip addr option [fail]
> 
> FAIL test_options (exit status: 1)
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Questions about libmicrohttpd

[Christian Grothoff]

On 04/29/2018 03:14 PM, Austin Einter wrote:
> Dear All
> I am very new to libmicrohttpd.
> I am planning to use libmicrohttpd in one of my project.
> 
> I have basic two questions
> 
> 1) Can I send HTTP responses in an asynch manner. 
> Example - Say I get a GET request in my libmicrohttpd server.
> Say answer_to_connection callback got called. I do not want to send
> response immediately. I will interact with my application server (which
> is in a separate machine). After my interaction with my application
> server I want to send response for GET request. Is it possible? If so
> please help how to achieve it.

Yes, this is possible. In thread-per-connection mode, you can just block
in the callback. If you use an event loop, you must call
MHD_suspend_connection(), and then MHD_resume_connection() once you are
ready to serve the reply.

> 
> 2)I know libmicrohttpd is a server. Is it possible to use it as a
> client, like send a GET or POST request to another web server.

Nope, but there are libraries like libwget2 and libcurl for that.
git.taler.net/twister contains an example for implementing a proxy
combining MHD and libgnurl.

Best regards,

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] POST data processing

[Christian Grothoff]

On 05/17/2018 06:52 AM, Austin Einter wrote:
> Hello
> I am using libmicrohttpd and facing issue with POST processing.
> 
> The POST message received contains xml or json data. libmicrohttpd does
> not support xml / json. So I can not use post process or iterate
> callbacks for body data processing. In fact I tried and it does not work
> (iterate callbacks not called even when I call post process).
> 
> So I am left with two options.
> 
> 1) Either write body parser, where I need to take care of content-length
> case, chunked data case etc. 

Nope, that is _always_ take care of by MHD. You just need to feed the
"upload_data" you get in your MHD_AccessHandlerCallback as you receive
it to your XML or JSON parser, either incrementally or as a whole.

> Is there anyway in libmicrohttpd, I can collect the entire POST message
> as it comes / received at socket level?

Sure.  You should look at GNU Taler (https://git.taler.net/), the
exchange and the merchant-backend both use MHD to receive JSON-formatted
uploads.  The core logic for this is actually from/in GNUnet
(https://gnunet.org/git/, gnunet.git, src/json/json_mhd.c).  So if you
are happy with libjansson, you can link against libgnunetjson (like
Taler does) and use that code to parse JSON.

XML usage would be analogous (and this is ~200 LOC only).

Happy hacking!

Christian


0xE29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] POST data processing

[Christian Grothoff]

Dear Austin,

Please look at the code samples I pointed you towards. If you have XML
or JSON POST data, you must not use the MHD_PostProcessor API at all.

Happy hacking!

Christian

On 05/18/2018 09:25 AM, Austin Einter wrote:
> Dear All
> I am getting data in access content callback.
> I can do my own parsing.
> 
> To get body data, independent chunks through iterate callback (NOT
> access content callback), I have issues.
> 
> In some cases, post process create fails (returns null).
> In some cases, post process is created, but when I call post process
> api, iterate callback is not called.
> 
> Are you telling that
> 1) always post process create will succeed (irrespective of xml / json /
> binary body)
> 2) if post process is created, on calling post process api, iterate
> callback is called.
> 
> Once I get sometime (sorry, but really occupied), will put a small code
> and share.
> 
> Regards
> Austin
> 
> 
> 
> On Thu, May 17, 2018 at 8:24 PM, Kenneth Mastro
> mailto:kenneth.mas...@gmail.com>> wrote:
> 
> Unless something broke in a very recent version, POST definitely
> works with JSON data.  I do it all the time.  I could be wrong, but
> I suspect MHD does not care about the data type at all.
> 
> I very strongly suspect the problem is in your code or your test. 
> Since it works with form data, are you sure your request is getting
> sent as a proper 'POST' when you're trying with XML or JSON?
> 
> Just a thought - are you just trying to use MHD's post-processor to
> process the data like you do with form data?  In that case - no, MHD
> does not PARSE the JSON or XML (like it can with form data).  You
> have to do that yourself or use a third party library (but either
> way, you have to capture the data from MHD and do something with it
> on your own).
> 
> 
> Ken
> 
> 
> On Thu, May 17, 2018 at 10:33 AM, Austin Einter
> mailto:austin.ein...@gmail.com>> wrote:
> 
> Hello
> When I tried, if content type is xml, post processor crate
> failed (it returned null).
> When content type is binary, it created post processor, however
> when I call post process api, the iterate callback not called.
> 
> That made me to search google.
> Somewhere I read it does not support xml and json.
> 
> Data I am getting in access content callback, however my iterate
> callback is not called.
> Please note that same code is working for text data (file upload
> or form data).
> 
> Thanks
> Austin
> 
> 
> 
> 
> On Thu, May 17, 2018 at 6:56 PM, silvioprog
> mailto:silviop...@gmail.com>> wrote:
> 
> Hello Austin,
> 
> As Christian explained, via "upload_data" you can receive
> any payload kind. I'm writting a library which maps MHD
> callbacks to "objects" and it needs to support the following
> features:
> 
> 1. receive one or more files *on demand*(to receive large
> files about 5 GB+ without "frying" CPU/RAM) in same request
> (files via form-data);
> 2. receive form fields (HTML-form fields via
> x-www-form-urlencoded);
> 3. 1 and 2 (both files and fields via form-data);
> 4. payload contents (raw-data, json/xml and so on via
> application/);
> 5. handlers to create customized 'body-parser' if none of
> the above options fits.
> 
> it seems the option 4 above is the one you are looking for.
> For example, supposing you need to send a JSON
> "{"foo":"bar"}" to the server, in the client:
> 
> curl -X POST -H "Content-Type: application/json" -d
> '{"foo":"bar"}' http://localhost:8080
> 
> and in a minimal snipped server example, you can receive it as:
> 
> ...
> 
> static void req_cb(void *cls, struct bk_httpreq *req, struct
> bk_httpres *res) {
>     struct bk_str *payload = bk_httpreq_payload(req);
>     printf("Payload string: %s; Size: %zd",
> bk_str_content(payload), bk_str_length(payload));
>     ...
> }
> 
> int main(void) {
>     struct bk_httpsrv *srv = bk_httpsrv_new(req_cb, NULL);
>     bk_httpsrv_listen(srv, 8080, false);
>     ...
> }
> 
> ...
> 
> then it prints "Payload string: {"foo":"bar"}; Size: 13" in
> the terminal. The "payload" variable above is an instance of
> the object "bk_str" which contains useful "methods" for
> string handling like "content", "length", "clear", "printf"
> and more.
> 
> If you want to take a look how it was implemented, please
> use the branch "*new_api*" and specifically the line 78 from
> bk_h

Re: [libmicrohttpd] Using GnuTLS with GNUTLS_CRD_PSK and AES-256-GCM, SHA384

[Christian Grothoff]

Dear Tal,

If you look at daemon.c:580, you can see that so far we only implemented
the CRD_CERTIFICATE type.  There is no fundamental reason why PSK could
not be supported, but the respective data structures would need to be
added to 'struct MHD_Daemon' and the case filled in here with the right
call to gnutls.

I've never used the PSK API of GnuTLS, so some (likely small) digging
might be required to figure out what exactly should be called.  Also,
another MHD_OPTION_HTTPS_PSK_KEY or something like that would likely be
needed.

I don't mind adding support, but I'm pretty busy so unless I receive a
patch (or say a donation to GNUnet e.V, too small for "real" funding)
this is unlikely to happen super-quickly.


Happy hacking!

Christian

On 05/22/2018 07:58 PM, Tal Maoz (tmaoz) wrote:
> Hey guys,
> 
>  
> 
> I hope someone can help with this as it is pretty urgent.
> 
> I’m trying to build a simple secure server over libmicrohttpd.
> 
> I compiled version 0.9.59 with libgnutls 3.5.18.
> 
> I’m trying to use TLS-PSK with AES-256-GCM and SHA384 but I get an error:
> 
> Ø  Error: invalid credentials type 4 specified.
> 
>  
> 
> My code:
> 
>  
> 
> *daemon = MHD_start_daemon(MHD_USE_THREAD_PER_CONNECTION | MHD_USE_SSL |
> MHD_USE_DEBUG,*
> 
> *   arguments.port_arg,*
> 
> *   NULL,*
> 
> *   NULL,*
> 
> *   &request_handler,*
> 
> *   NULL,*
> 
> *   MHD_OPTION_CONNECTION_TIMEOUT, 256,*
> 
> *   MHD_OPTION_HTTPS_CRED_TYPE,
> GNUTLS_CRD_PSK,*
> 
> *   MHD_OPTION_HTTPS_PRIORITIES,
> "NONE:+AES-256-GCM:+SHA384",*
> 
> *   MHD_OPTION_HTTPS_MEM_KEY, key_pem,*
> 
> *   MHD_OPTION_HTTPS_MEM_CERT, cert_pem,*
> 
> *   MHD_OPTION_END);*
> 
>  
> 
> I looked into the source code of libmicrohttps and in
> microhttpd/daemon.c:576 I see that, for some reason, if anything other
> than GNUTLS_CRD_CERTIFICATE is used, this error message is given. The
> documentation says:
> 
>  
> 
> MHD_OPTION_HTTPS_CRED_TYPE
> 
> Daemon credentials type. Either certificate or anonymous, this option
> should be followed by one of the values listed in "enum
> gnutls_credentials_type_t".
> 
> Any Idea on what to do with this? Is there some other config I need to
> use/change for this to work? When I remove the
> “*MHD_OPTION_HTTPS_CRED_TYPE*“ and “*MHD_OPTION_HTTPS_PRIORITIES*” , the
> server works (but not in the mode I need, obviously).
> 
>  
> 
> If this is simply not supported, any idea if there are any plans to ever
> support this?
> 
>  
> 
> Any help will be very much appreciated.
> 
>  
> 
> Thanks,
> 
>  
> 
> Tal
> 
>  
> 
> http://www.cisco.com/c/dam/m/en_us/employee-connection/signaturetool/images/banners/Photography/banner7.png
> 
>  
> 
>   
> 
> *Tal Maoz*
> 
> Senior Software Engineer
> 
> CTAO Innovation Group
> 
> tm...@cisco.com 
> 
> Tel: *+972-2-5886289*
> 
>   
> 
> *Cisco Systems Israel Ltd.*
> 
> 5 Shlomo Halevi Street
> 
> Har Hotzvim High Tech Park
> 
> Jerusalem
> 
> 9777019
> 
> Israel
> 
> Cisco.com 
> 
>   
> 
> http://www.cisco.com/c/dam/m/en_us/signaturetool/images/linkedin-16x16.png
> 
> 
>  
> 
> http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif Think before
> you print.
> 
>  
> 
>  
> 
>  
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] undefined reference to *

[Christian Grothoff]

You're missing "-lmicrohttpd" on the gcc command line.

On 07/19/2018 07:49 PM, Nick Leli wrote:
> # gcc -o microhttp microhttp.c -L/usr/local/lib/* -I/usr/local/include
> /tmp/cc7sUAvI.o: In function `ahc_echo':



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

On 08/23/2018 06:21 PM, Santos Das wrote:
> Hi,
> 
> Can anyone please confirm if libmicrohttpd supports the Async handling?

Yes, it supports asynchronous processing.

> Also, any information on the performance of libmicrohttps will be useful.

Let's just say if used properly, it will never be anywhere near the
bottleneck. We know people running 100,000 requests/s on a single system
with MHD years ago. Quite likely to do more today, but very difficult to
test: when benchmarking, you will always be benchmarking your clients,
your network or your application logic long before hitting any MHD
threshold (on sane operating systems).



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

On 08/23/2018 08:36 PM, Santos Das wrote:
> Hi,
> 
> Thank you for the quick reply. I have one more question, please help in
> clarifying -
> 
> Can "libmicrohttpd" run as a part of the application event loop or is it
> *_mandatory _*for the  "libmicrohttpd"  to have its own event loop?

It can run as part of the application's event loop.

You can find a simple example here:

https://git.taler.net/exchange.git/tree/src/bank-lib/fakebank.c

The application's event loop here is easily identified as it is the
GNUNET_SCHEDULER*-API.

The code shows integration both with the select() and epoll()-based
versions of MHD.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

On 08/23/2018 09:09 PM, Santos Das wrote:
> Hi,
> 
> Thanks a lot. Is there any significant performance impact when it runs
> in this mode ?

Not inherently, but you don't get the thread-pool feature that way (at
least not without quite a bit of work by you!).  My real
high-performance setups use the thread pool (because my application
logic requires some CPU time, not so much for MHD), but of course you
could hack up your own.  But just FYI: that's not trivial.

> Has anyone compared its performance against "mongoose " ?

I'm not aware of a direct 1:1 comparison, but as I said, it is very hard
to benchmark the HTTP logic itself.

> I''m evaluating a http library for my C++ based application
> microservices .  I am evaluating libraries that can provide our C++
> workers with native HTTP capabilities. 
> 
> 
> I’m defining native as:
> 
>   * No additional threads other than those spawned by the application
> when it chooses to use blocking calls (entirely optional)
>   * No additional queueing
>   * Native sockets interface with full use of listen, accept, bind,
> connect, recv, send.
> 
> 
> Do you think libmicrohttpd is the best choice or is there any better
> alternatives for me ?

Well, we have a list of alternatives on the MHD web site. But naturally
few people on this list would think they are as useful for _their_
usecases. ;-)

Note that there is some C++ wrapper around MHD here:
 https://github.com/etr/libhttpserver

But I don't know if it is any good (never tried).

Happy hacking!

Christian

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

On 08/24/2018 04:12 PM, Santos Das wrote:
> Hi,
> 
> Thanks for the prompt reply. I have few more questions., kindly see if
> you could help.
> 
> a. Does this have a transaction scoreboarding capability (e.g., to match
> up asynchronous operations)?   The use of the passed argument *cls seems
> to imply this?   That is, when you get an inbound transation you
> allocate a tracking object in application space and hand its pointer to
> libhttpd so that it can give it back to you later (i.e., you are giving
> the response and it asynchronously asks for the next chunk of bytes in
> the response).

I'm not sure I'd call this scoreboarding, but yes, the 'cls' is there so
that the application can associate state with the request for later
continuations.

> b. Does it support chunk-encoding of responses?

Yes.

> c. Does MHD has inbuilt JSON encode/decode capabilities ?

No, but easy to add by linking against libjanson.

-Christian

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

On 08/24/2018 08:05 PM, Santos Das wrote:
> Thank you once again for the prompt reply.
> 
> I have few more questions. Are you aware of any project which has
> integrated libmicrohttpd along with libcurl without losing any
> functionality of MHD ? I really want to run in the single threaded model
> as I mentioned above.

GNUnet and GNU Taler (in Twister) do this for implementing an http
proxy, basically HTTP server via MHD, then libcurl for the HTTP client:

https://git.taler.net/twister.git/

I don't see why you think integrating with libcurl would result in a
loss of functionality.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

On 08/29/2018 08:44 AM, Kunal Ekawde wrote:
> 3. Just after #2, when curl_multi_info_read is successful ,
> MHD_queue_response is invoked, but it returns 0 and the easy_handle
> state is 1.
> 
> Can we send the response to client only from the MHD callback handler ?

Yes. You must call MHD_resume_connection (you should have called suspend
before), and then queue once the callback is invoked.

> or something is missing in above ?

Nope, just "bad timing" of the call.



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

HTTP 1.1 processes requests in-order. Clients may send us additional
requests on the same TCP connection, but MHD will only ever look at a
subsequent request after processing the previous request is complete.

I hope this clarifies.

On 08/29/2018 09:43 AM, Kunal Ekawde wrote:
> Ahh ok, getting it now, I wasn't using suspend/resume, but if client
> pipelines the requests, we would have suspended the connection, will it
> be handled ?
> Appreciate you help.
> 
> Thanks,
> Kunal
> 
> On Wed, Aug 29, 2018 at 12:47 PM Christian Grothoff  <mailto:groth...@gnunet.org>> wrote:
> 
> On 08/29/2018 08:44 AM, Kunal Ekawde wrote:
> > 3. Just after #2, when curl_multi_info_read is successful ,
> > MHD_queue_response is invoked, but it returns 0 and the easy_handle
> > state is 1.
> >
> > Can we send the response to client only from the MHD callback
> handler ?
> 
> Yes. You must call MHD_resume_connection (you should have called suspend
> before), and then queue once the callback is invoked.
> 
> > or something is missing in above ?
> 
> Nope, just "bad timing" of the call.
> 
> 
> 
> -- 
> ~Kunal


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

Hi Santos,

MHD simply doesn't read the next request (at least not intentionally,
bits may end up in our read buffer but won't be looked at). But the
kernels may already buffer subsequent requests.

Happy hacking!

Christian

On 09/03/2018 10:30 AM, Santos Das wrote:
> Hi  Christian ,
> 
> How does this work with pipe-lining? My understanding was that the
> server would still read the request but will maintain the order of
> response. But, from your email it seems the server would also not read
> the second request before sending a reply to the first request> ?
> 
> Is this correct?
> 
> thanks, santos
> 
> On Mon, Sep 3, 2018 at 12:41 PM Christian Grothoff  <mailto:groth...@gnunet.org>> wrote:
> 
> HTTP 1.1 processes requests in-order. Clients may send us additional
> requests on the same TCP connection, but MHD will only ever look at a
> subsequent request after processing the previous request is complete.
> 
> I hope this clarifies.
> 
> On 08/29/2018 09:43 AM, Kunal Ekawde wrote:
> > Ahh ok, getting it now, I wasn't using suspend/resume, but if client
> > pipelines the requests, we would have suspended the connection,
> will it
> > be handled ?
> > Appreciate you help.
> >
> > Thanks,
> > Kunal
> >
> > On Wed, Aug 29, 2018 at 12:47 PM Christian Grothoff
> mailto:groth...@gnunet.org>
> > <mailto:groth...@gnunet.org <mailto:groth...@gnunet.org>>> wrote:
> >
> >     On 08/29/2018 08:44 AM, Kunal Ekawde wrote:
> >     > 3. Just after #2, when curl_multi_info_read is successful ,
> >     > MHD_queue_response is invoked, but it returns 0 and the
> easy_handle
> >     > state is 1.
> >     >
> >     > Can we send the response to client only from the MHD callback
> >     handler ?
> >
> >     Yes. You must call MHD_resume_connection (you should have
> called suspend
> >     before), and then queue once the callback is invoked.
> >
> >     > or something is missing in above ?
> >
> >     Nope, just "bad timing" of the call.
> >
> >
> >
> > --
> > ~Kunal
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

Dear Kunal,

Yes, you can do all this, including raw epoll, no threads and storing
connection details in '*cls'.  I just don't have an example that does
all this at once ;-)

Happy hacking!

Christian

On 08/27/2018 11:15 AM, Kunal Ekawde wrote:
> Hi,
> 
> With ref to:
> https://lists.gnu.org/archive/html/libmicrohttpd/2018-08/msg00015.html ,
> I gone through twister, but looks like this is sync operation. The curl
> operation is performed sync in same thread. I wanted to know if there is
> a way to make it async, so basically on receiving the data callback
> (entire data), we should be able to store the connection details (cls ?)
> so that server can do some other task, like may be fetch from DB. When
> server is ready with response, it should be able to call
> MHD_queue_response with the stored connection details. When it breaks to
> fetch from DB, it could also handle other incoming requests. 
> Let me know If I'm missing something here or if there is any pointers
> w.r.t use of MHD in this way preferably using raw epoll instead of GNUnet.
> 
> Threading model interested is : Application is single threaded with its
> own epoll, MHD not to create any threads.
> 
> Thanks,
> ~Kunal



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Question on Async support

[Christian Grothoff]

Hi!

There is no official support for HTTP2 at this time, and there is no
timeline for adding support either.  The easiest way to get HTTP2
support would be to run your MHD behind a reverse proxy (Apache/nginx)
that supports HTTP2.

Happy hacking!

Christian

On 08/24/2018 09:24 PM, Santos Das wrote:
> Hi,
> 
> What is the plan of supporting http2 for MHD? 
> 
> I saw this project and the comments. Can you please comment ?. The
> libcurl supports http2 with 3rd party nghttp2. So, wondering what is the
> similar support for MHD ? Is there any recommended project for the same ?
> 
> https://github.com/maru/libmicrohttpd-http2
> 
> https://lists.gnu.org/archive/html/libmicrohttpd/2016-08/msg00032.html
> 
> 
> On Fri, Aug 24, 2018 at 11:59 PM Christian Grothoff  <mailto:groth...@gnunet.org>> wrote:
> 
> On 08/24/2018 08:05 PM, Santos Das wrote:
> > Thank you once again for the prompt reply.
> >
> > I have few more questions. Are you aware of any project which has
> > integrated libmicrohttpd along with libcurl without losing any
> > functionality of MHD ? I really want to run in the single threaded
> model
> > as I mentioned above.
> 
> GNUnet and GNU Taler (in Twister) do this for implementing an http
> proxy, basically HTTP server via MHD, then libcurl for the HTTP client:
> 
> https://git.taler.net/twister.git/
> 
> I don't see why you think integrating with libcurl would result in a
> loss of functionality.
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [possible feature request?] Loading key/cert from file [gnutls_certificate_set_x509_key_file2()]

[Christian Grothoff]

Hi Silvio,

I don't like the proposed feature very much, it's something an
application developer can reasonably easily write themselves, it would
introduce a dependency on file system operations to MHD, and it would
include a lot of different possible error types (file not found, access
permissions wrong, etc.) that would have to be passed back to the
application.

That said, adding a good _example_ (reference code) for how to do this
in the documentation would be totally fine.

My 2 cents!

Happy hacking!

Christian
p.s.: sorry for the delay, life's been very busy...

On 07/29/2018 05:39 AM, silvioprog wrote:
> Hello dudes.
> 
> I tried to pass the key/cert files to the MHD library, however, it
> allows only passing via memory (using
> gnutls_certificate_set_x509_key_mem2()):
> 
>   MHD_OPTION_HTTPS_MEM_KEY, "private key content",
>   MHD_OPTION_HTTPS_MEM_CERT, "certificate content"
> 
> but it would be nice to use the GnuTLS's builtin functions to load the
> key/cert files: gnutls_certificate_set_x509_key_file2(). Advantage: the
> programmer just pass the path of the key/cert instead of writing own
> unsafe functions for file loading. So, what do you thing about
> to add those new options?:
> 
>   MHD_OPTION_HTTPS_FILE_KEY, "key.pem",
>   MHD_OPTION_HTTPS_FILE_CERT, "cert.pem",
>   MHD_OPTION_HTTPS_FILE_TRUST, "ca.pem"
> 
> Thank you!
> 
> -- 
> Silvio Clécio



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Web Socket disconnect behaviour

[Christian Grothoff]

On 9/3/18 6:09 PM, Paul Topley wrote:
> 
> This all works great, but there is one particular situation im not sure
> how to handle.  When there is an unexpected disconnect in the socket
> between client and server, which isnt instigated by either (lets say the
> server crashes, ethernet disconnect etc), im not sure how to
> re-establish the connection between then two.
> 
> 
> I have tried firing off another upgrade response from the server when a
> socket disconnect is detected, but the same socket int which had just
> died is passed back to my handler, which subsequently returns zero when
> I call recv to get data.
> 
> 
> Is there anything else I should be doing?
> 

You need to initiate another HTTP request from the client, again do the
upgrade and then handle resumption within your application protocol.

Happy hacking!

Christian



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

In this case, MHD would use an eventfd instead of a pipe *if* the
platform supports it (i.e. on GNU/Linux).

On 09/06/2018 08:11 AM, Kunal Ekawde wrote:
> Hi Christian,
> 
> Can you please confirm if suspend/resume wont use pipe (lfor IPC)?
> As in manual it's mentioned as:
> "Enables using MHD_suspend_connection and MHD_resume_connection,
> as performing these calls requires some additional pipes to be created,
> and code not using these calls should not pay the cost."
> 
> But couldn't see in src code.
> My application is using external epoll w/o any MHD threads.
> 
> Thanks,
> Kunal
> 
> On Wed, Sep 5, 2018 at 9:22 PM Kunal Ekawde  <mailto:kunaleka...@gmail.com>> wrote:
> 
> Thanks Christian, yes I got it done :)
> 
> On Wed, Sep 5, 2018 at 3:05 PM Christian Grothoff
> mailto:groth...@gnunet.org>> wrote:
> 
> Dear Kunal,
> 
> Yes, you can do all this, including raw epoll, no threads and
> storing
> connection details in '*cls'.  I just don't have an example that
> does
> all this at once ;-)
> 
> Happy hacking!
> 
> Christian
> 
> On 08/27/2018 11:15 AM, Kunal Ekawde wrote:
> > Hi,
> >
> > With ref to:
> >
> https://lists.gnu.org/archive/html/libmicrohttpd/2018-08/msg00015.html
> ,
> > I gone through twister, but looks like this is sync operation.
> The curl
> > operation is performed sync in same thread. I wanted to know
> if there is
> > a way to make it async, so basically on receiving the data
> callback
> > (entire data), we should be able to store the connection
> details (cls ?)
> > so that server can do some other task, like may be fetch from
> DB. When
> > server is ready with response, it should be able to call
> > MHD_queue_response with the stored connection details. When it
> breaks to
> > fetch from DB, it could also handle other incoming requests. 
> > Let me know If I'm missing something here or if there is any
> pointers
> > w.r.t use of MHD in this way preferably using raw epoll
> instead of GNUnet.
> >
> > Threading model interested is : Application is single threaded
> with its
> > own epoll, MHD not to create any threads.
> >
> > Thanks,
> > ~Kunal
> 
> 
> 
> -- 
> ~Kunal
> 
> 
> 
> -- 
> ~Kunal


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Aync handling & message response

[Christian Grothoff]

On 09/06/2018 08:19 AM, Kunal Ekawde wrote:
> 
> Can you please provide some information on 'MHD_USE_TURBO' as I couldn't
> find any info in manual.

From the manual:

@item MHD_USE_TURBO
@cindex performance
Enable optimizations to aggressively improve performance.

Currently, the optimizations this option enables are based on
opportunistic reads and writes.  Bascially, MHD will simply try to
read or write or accept on a socket before checking that the socket is
ready for IO using the event loop mechanism.  As the sockets are
non-blocking, this may fail (at a loss of performance), but generally
MHD does this in situations where the operation is likely to succeed,
in which case performance is improved.  Setting the flag should generally
be safe (even though the code is slightly more experimental).  You may
want to benchmark your application to see if this makes any difference
for you.




0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] DELETE with message body

[Christian Grothoff]

Yes. With a few exceptions MHD doesn't really care about which HTTP
method it is an just passes everything to the application.

On 09/12/2018 05:13 AM, Santos Das wrote:
> Can  MHD handle the a msg body associated with a delete method? Can it
> pass it to the application when a message body is received in the DELETE
> method?
> 
> thanks, santos



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

If you suspend a connection, you are responsible to schedule some job to
resume it, after which MHD's timeout counter will start again (and of
course your response logic could then also just force MHD to close the
connection once MHD asks for response data).  MHD timeouts are there to
detect (idle) clients that just keep connections open, not to prevent
the server from suspending a connection.

If the client closes a connection, it'll be detected if/when we try to
send data to the client and handled appropriately.

Happy hacking!

Christian

On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> If the connection is suspended and cases of:
> a. client closes the connection before response is sent.
> b. server app doesn't respond at all post suspend.
>  
> both cases there is no notification by MHD even if connection timeout is
> specified and notify callback. How can these cases be handled ?
> 
> Thanks,
> Kunal
> 
> On Mon, Sep 17, 2018 at 9:02 PM Santos Das  > wrote:
> 
> Hi,
> 
> Just to add more, my application is single threaded and I am running
> MHD inside that. I don't create additional threads. I use suspend
> and resume as suggested earlier as I do async processing.
> 
> So, I am thinking how can we implement flow control in this scenario..
> 
> Thanks, Santos
> 
> 
> 
> On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > wrote:
> 
> Hi,
> 
> Could you kindly let me know how can we implement the flow
> control ? Protection against the badly behaving clients.
> 
> 
> HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's 
> flow control 
> 
> mechanisms to resolve temporary overloads, rather than terminating 
> connections with 
> 
> the expectation that clients will retry. The latter technique can 
> exacerbate network 
> 
> congestion. 
> 
>  
> 
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> 
> 
> Thanks, Santos
> 
> 
> 
> 
> -- 
> ~Kunal


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

Hi!

Good news: You don't need to do anything.  MHD will maintain persistent
TCP connections and use TCP's flow control mechanisms automatically.
You can _force_ MHD to not keep connections open by adding "Connection:
Close" to your HTTP response header manually, but by default MHD will do
the right thing.

This is independent of your threading model or the use of suspend/resume.

Happy hacking!

Christian

On 09/17/2018 05:33 PM, Santos Das wrote:
> Hi,
> 
> Just to add more, my application is single threaded and I am running MHD
> inside that. I don't create additional threads. I use suspend and resume
> as suggested earlier as I do async processing.
> 
> So, I am thinking how can we implement flow control in this scenario..
> 
> Thanks, Santos
> 
> 
> 
> On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > wrote:
> 
> Hi,
> 
> Could you kindly let me know how can we implement the flow control ?
> Protection against the badly behaving clients.
> 
> 
> HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's 
> flow control 
> 
> mechanisms to resolve temporary overloads, rather than terminating 
> connections with 
> 
> the expectation that clients will retry. The latter technique can 
> exacerbate network 
> 
> congestion. 
> 
>  
> 
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> 
> 
> Thanks, Santos
> 
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 12:23 PM, Kunal Ekawde wrote:
> Thanks Christian,
> So as a solution we need to have a timer for such cases and on timer
> expiry , resume the connection post which as data is not available,
> either error respond or close connection.

Yes, basically if you suspend to perform some asynchronous activity that
needs a timeout, you need to manage that timeout.

> If we close the connection, we
> shall loose the pending requests on that connection right ?

Yes, if you "close" i.e. by returning "MHD_NO".

> and if we
> error respond, we shall get pending messages on that connection right
> (number of pending messages would depend on
> MHD_OPTION_CONNECTION_MEMORY_LIMIT?)?

If you respond, other requests on the connection will then be processed,
but not limited by the MEMORY_LIMIT.  MHD won't use more RAM per
connection than allowed by MEMORY_LIMIT, but the OSes for client and
server negotiate a TCP buffer which is really what limits network
transmissions.  And the client just won't be able to transmit more
requests than what that TCP buffer allows until you resume, but in
theory the number of requests that can be pipelined is unlimited.

> If the case was client side disconnection, post resume, we should get
> notify callback with reason 'MHD_REQUEST_TERMINATED_WITH_ERROR' ? so we
> can do cleanup right ?

Yes.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 01:31 PM, Santos Das wrote:
> Hi Christian,
> 
> Thanks a lot. I have few questions in this regard -
> 
> a.    When we put the connection in suspend state, can we pass a timeout
> at the expiry of which the connection will be automatically resumed?

No.

> b.    For asynchronous communication , i.e the application received the
> request but is not able to send the response now (it needs to do some
> work may be creating another request to another endpoint) , is it
> mandatory to use suspend-resume only or there is some other mechanism
> that we can use ?

If you use 'thread per connection', you could also simply block.

> c.   How does the keep alive works in the HTTP server library ? 

MHD takes care of it, it adds keep-alive headers and processes them
_unless_ explicitly overridden by the application.

> Could you please explain how the TCP's flow control mechanism is
> helping in this case. A few lines would be great, please.

TCP flow control requires the kernel to provide message buffers for
retransmission (sender) and reordering (receiver). So those buffers can
and will be used by the kernel (on any OS) if the client sends
additional requests that the server is not yet processing.

> Sorry, lot of questions. But, please see if you could help in this.
> 
> On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff  <mailto:groth...@gnunet.org>> wrote:
> 
> If you suspend a connection, you are responsible to schedule some job to
> resume it, after which MHD's timeout counter will start again (and of
> course your response logic could then also just force MHD to close the
> connection once MHD asks for response data).  MHD timeouts are there to
> detect (idle) clients that just keep connections open, not to prevent
> the server from suspending a connection.
> 
> If the client closes a connection, it'll be detected if/when we try to
> send data to the client and handled appropriately.
> 
> Happy hacking!
> 
> Christian
> 
> On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> > If the connection is suspended and cases of:
> > a. client closes the connection before response is sent.
> > b. server app doesn't respond at all post suspend.
> >  
> > both cases there is no notification by MHD even if connection
> timeout is
> > specified and notify callback. How can these cases be handled ?
> >
> > Thanks,
> > Kunal
> >
> > On Mon, Sep 17, 2018 at 9:02 PM Santos Das  <mailto:santos@gmail.com>
> > <mailto:santos@gmail.com <mailto:santos@gmail.com>>> wrote:
> >
> >     Hi,
> >
> >     Just to add more, my application is single threaded and I am
> running
> >     MHD inside that. I don't create additional threads. I use suspend
> >     and resume as suggested earlier as I do async processing.
> >
> >     So, I am thinking how can we implement flow control in this
> scenario..
> >
> >     Thanks, Santos
> >
> >
> >
> >     On Mon, Sep 17, 2018 at 8:22 PM Santos Das
> mailto:santos@gmail.com>
> >     <mailto:santos@gmail.com <mailto:santos@gmail.com>>>
> wrote:
> >
> >         Hi,
> >
> >         Could you kindly let me know how can we implement the flow
> >         control ? Protection against the badly behaving clients.
> >
> >
> >         HTTP/1.1 servers SHOULD maintain persistent connections
> and use TCP's flow control
> >
> >         mechanisms to resolve temporary overloads, rather than
> terminating connections with
> >
> >         the expectation that clients will retry. The latter
> technique can exacerbate network
> >
> >         congestion.
> >
> >          
> >
> >         https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> >
> >
> >         Thanks, Santos
> >
> >
> >
> >
> > --
> > ~Kunal
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:22 PM, Kunal Ekawde wrote:
> Thanks for prompt response, few more doubts :)
> 1. Yeah, timers per connection on each request doesn't seems good, need
> to figure that out.
> 2. When you say ' MHD won't use more RAM per connection than allowed by
> MEMORY_LIMIT' , will MHD read from TCP buffer only 'MEMORY_LIMIT' data ?
> is it the recv call buffer size ? when will remaining data be read ?

When the application has finished processing the data that was read and
thus created the necessary free space for more.



0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:35 PM, Santos Das wrote:
> Thanks Christian.
> 
> I have a question on "MHD_OPTION_CONNECTION_MEMORY_LIMIT" . I went
> through the manual. How does this affect the send/recv buffer size per
> client connection. What does it eventually translate to?

It does not directly translate. Basically, the send/recv buffer for the
application payload is whatever is left after MHD allocated storage for
the HTTP header. So if you have a 32k buffer and a 1k header, you may
have 30+ kb for send/recv of payload, but if you have a 32k buffer and
an 8k HTTP header, you might have only 24k for the send/recv buffer.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:43 PM, Santos Das wrote:
> Thanks Christian. Looks like you answered this question for Kunal Ekwade
> already. Thank you. 
> 
> But, still not clear from your reply, is it equal to the receive buffer
> size ?

MHD uses "whatever is left" (minus some reserves depending on what phase
processing is in) -- first for the receive buffer, and later for the
send buffer (re-using space from the receive buffer).  In both cases it
is minus headers (headers sent and headers received!) and management
data structures.

The objective is simple: we must not use more than the memory limit per
connection, and within that constraint we make the buffers as big as we
(safely) can.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys

Re: [libmicrohttpd] Libmicrohttpd my program cannot be run background

[Christian Grothoff]

Dear Forrest,

I don't know how this could _not_ work, but it's certainly not an issue
with the MHD library.  If you're using one of our demos, our use of
'getchar()' to wait for a signal to terminate might be the cause. In
this case, you could just replace 'getchar()' with while(1) sleep(1); to
ensure that the process continues after stdin is closed.

My 2 cents

Christian

On 09/22/2018 04:25 PM, lingweicai wrote:
> Dear,
> 
> I am to run my program with libmicrohttpd in background on Linux, with
> nohup myprogram end with ‘&’.
> 
> But it always fails.
> 
> How can I solve this problem?
> 
> Thank you
> 
> Forrest
> 
> ---原始邮件---
> 
> *发件人:*
> Christian Grothoff
> *时　间:* 2018/09/18 20:43:58
> *收件人:*
> 
> *抄　送:*
> undefined
> *主　题:*
> Re: [libmicrohttpd] Regarding Flow control
> 
> On 09/18/2018 02:35 PM, Santos Das wrote:
>> Thanks Christian.
>>
>> I have a question on "MHD_OPTION_CONNECTION_MEMORY_LIMIT" . I went
>> through the manual. How does this affect the send/recv buffer size per
>> client connection. What does it eventually translate to?
> 
> It does not directly translate. Basically, the send/recv buffer for the
> application payload is whatever is left after MHD allocated storage for
> the HTTP header. So if you have a 32k buffer and a 1k header, you may
> have 30+ kb for send/recv of payload, but if you have a 32k buffer and
> an 8k HTTP header, you might have only 24k for the send/recv buffer.
> 
> 发自搜狐邮箱客户端
> 
> 
> 
> 
> <https://a.app.qq.com/o/simple.jsp?pkgname=com.sohu.mail.client.cordova>
> 
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Web Socket disconnect behaviour

[Christian Grothoff]

Sorry, but I don't have one. Maybe someone else does...

On 10/09/2018 04:41 AM, silvioprog wrote:
> 
> @Christian, just for curiosity, is there any document or example showing
> MHD + WS? I can't provide it since I'm not familiar with WS, but it
> would be nice if someone could provide one for us. 😅
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] make MHD behave as a HTTP 1.0 server

[Christian Grothoff]

Looks good, pushed as f7ab8b59..9199e5aa (with updates to documentation).

Happy hacking!

Christian

On 10/15/18 9:03 PM, Gauthier Haderer wrote:
> Hi,
> 
> I attached a patch which adds a response flag to force version to HTTP
> 1.0 in
> responses but still maintaining connection management.
>     
> The existing MHD_RF_HTTP_VERSION_1_0_ONLY flag already changes MHD's
> behavior to apply HTTP 1.0 rules for connection management. When
> enabled, MHD sends a response using the same version as used in the
>  request (is this normal?).
>     
>  What I want is MHD responding as a HTTP 1.0 server with support for
>  connection management headers would do. This is what the
>  MHD_RF_HTTP_VERSION_1_0_RESPONSE response flag is for.
>     
>  You can even combine it with MHD_RF_HTTP_VERSION_1_0_ONLY to change the
> response's HTTP version while maintaining strict compliance with HTTP
> 1.0 regarding connection management.
>     
> This solution is not perfect as this flag is set on the response which
> is created after header processing. So MHD will behave as a HTTP 1.1
> server until the response is queued. It means that an invalid HTTP 1.1
> request will fail even if the response is sent with HTTP 1.0 and the
> request would be valid if interpreted with this version. For example,
> this request will fail in strict mode:
>     
> GET /dummy HTTP/1.1
>     
> as the Host header is missing and is mandatory in HTTP 1.1, but it
> should succeed when interpreted with HTTP 1.0.
>     
> I don't think this is a big issue in practice. Besides, being able to
> change the HTTP version on a response basis is really convenient when
> using MHD in a test framework where I need to validate a client against
> HTTP 1.1 AND HTTP 1.0 as I can start a single server for the whole set
> of tests to run.
> 
> Do you think this is something you could consider integrating mainstream?
> 
> Regards,
> 
> 
> Gauthier



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] [PATCH] Fix build issue when parent dir is an automake project dir

[Christian Grothoff]

Pushed as suggested in 9199e5aa..41ac9325

On 10/18/2018 04:35 PM, Tim Rühsen wrote:
> Building fails if the parent directory is an automake project dir:
> 
> $ ./bootstrap
> libtoolize: putting auxiliary files in '..'.
> libtoolize: copying file '../ltmain.sh'
> libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
> libtoolize: copying file 'm4/libtool.m4'
> libtoolize: copying file 'm4/ltoptions.m4'
> libtoolize: copying file 'm4/ltsugar.m4'
> libtoolize: copying file 'm4/ltversion.m4'
> libtoolize: copying file 'm4/lt~obsolete.m4'
> configure.ac:61: installing '../compile'
> configure.ac:67: error: required file '../config.rpath' not found
> configure.ac:26: installing '../missing'
> doc/examples/Makefile.am: installing '../depcomp'
> autoreconf: automake failed with exit status: 1
> 
> The fix is to specify AC_CONFIG_AUX_DIR before AM_INIT_AUTOMAKE.
> 
> Regards, Tim
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Build issue on MINGW

[Christian Grothoff]

Hi Tim,

The issue is that in line Makefile.am:10, we have "noinst_" instead of
"lib_" because the code is far from complete and it must not yet be
installed. That causes the dlname='' issue you describe.

The best fix I can propose: I'm adding an option (--enable-experimental)
which disables building src/lib/ by default, so that users that don't
care about the experimental code don't get disrupted like this.

Happy hacking!

Christian

On 10/18/2018 04:45 PM, Tim Rühsen wrote:
> Hi,
> 
> the build on MinGW fails in lib/src/ due to
> 
> @echo Creating $@ and libmicrohttpd2.exp by $(DLLTOOL)... && \
> dll_name=`$(EGREP) -o dlname=\'.+\' libmicrohttpd2.la` && \
> 
> libmicrohttpd2.la contains
>   dlname=''
> and thus the egrep (or grep -E) fails.
> 
> I am on Debian unstable and use this sequence in a script:
> 
> unset CC
> PREFIX=x86_64-w64-mingw32
> export INSTALLDIR="$PWD/$PREFIX"
> export PKG_CONFIG_PATH=$INSTALLDIR/lib/pkgconfig
> export CPPFLAGS="-I$INSTALLDIR/include"
> export LDFLAGS="-L$INSTALLDIR/lib"
> export CFLAGS="-O2 -Wall -Wno-format"
> 
> git clone --recursive https://github.com/dlfcn-win32/dlfcn-win32.git
> cd dlfcn-win32
> ./configure --prefix=$PREFIX --cc=$PREFIX-gcc
> make
> cp -p libdl.a ../$PREFIX/lib/
> cp -p dlfcn.h ../$PREFIX/include/
> cd ..
> 
> git clone --recursive https://gnunet.org/git/libmicrohttpd.git
> cd libmicrohttpd
> ./bootstrap
> ./configure --build=x86_64-pc-linux-gnu --host=$PREFIX
> --prefix=$INSTALLDIR --disable-doc --disable-examples --enable-shared
> make clean
> make -j$(nproc)
> cd ..
> 
> 
> Regards, Tim
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Keeping the TCP connection open

[Christian Grothoff]

If you do not set a timeout or do something else explicit to close the
connection, MHD will try to keep the connection open. The rest depends
on the HTTP client, which you cannot force to keep the connection open.

On 10/20/2018 08:57 AM, Santos Das wrote:
> Hi,
> 
> We are using a HTTP 1.1. 
> 
> Is there a way we can keep the client connection open for ever (not
> through the keep alive time out) till it is closed explicitly ?
> 
> Thanks, Santos 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] TCP connection callbacks - problem

[Christian Grothoff]

Dear Daniel,

You didn't mention which thread mode you were using, but manual
inspection suggests your issue should occur in thread-per-connection
modes (only).  In that case, I've pushed a (likely) fix in
b55cd46b..47281241 to Git master.

Please test and report back!

Happy hacking!

Christian

On 10/19/2018 03:53 PM, Daniel Bujnik wrote:
> Hello,
> 
> I am using libmicrohttpd with MHD_OPTION_NOTIFY_CONNECTION parameter to
> receive notifications for TCP connection opening/closing. Notifications
> for opening case are coming through as expected, however I am
> experiencing problems with closing case, where I am NOT getting the
> callback when my client terminates, though I can see client FIN packet
> in tcpdump on the server side and connection stops appearing in netstat.
> 
> Eventually the closing notification is being delivered by libmicrohttpd
> but that is just after a new TCP connection is opened. The sequence of
> callbacks I am seeing is as followed:
> 
> 1) callback with MHD_CONNECTION_NOTIFY_STARTED - TCP connection  A opened
> 2) callback with MHD_CONNECTION_NOTIFY_STARTED - new TCP connection B is
> opened
> 3)  callback with MHD_CONNECTION_NOTIFY_CLOSED - TCP connection  A closed
> 
> Of course I would expect nr 3 to come before nr 2. I am using nc to
> open/close sockets on client end.
> 
> I would appreciate any comments on the described issue. Is that behavior
> correct? If not is there a way to make it right?
> 
> Kind regards,
> Daniel
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Build issue on MINGW

[Christian Grothoff]

Hi Tim,

Well, the --enable-experimental switch was already added to Git master
when I sent the e-mail.  As for doing features in new branches, that is
sometimes done, but in this case the specific goal was to expose the
code to a broad range of build systems to see if there were any compile
errors on some platforms -- intentionally long before putting the code
near production, hence 'noinst' (and building it _last_, so even if it
failed, the rest would just work). Still, I think the
--enable-experimental flag is reasonable at this time.

Happy hacking!

Christian

On 10/23/2018 10:28 AM, Tim Rühsen wrote:
> Hi Christian,
> 
> sorry for answering so late :-(
> 
> Such a switch would indeed be useful if one wants to test/use the latest
> master branch.
> 
> In my case it is using latest master as dependency for CI Testing of
> Wget2. This includes a MinGW CI runner.
> 
> Of course I can checkout the latest release tag, but atm I try to find
> issues in MHD development early.
> 
> There is also the possibility to slightly change the development cycle
> of MHD:
> - Add new a feature in an own branch
> - Push that branch to Gitlab.com to test the code via CI (we already
> created CI runners), optionally create an MR (merge request) for
> discussion/collaboration.
> - When CI succeeds: merging of the feature branch into master (directly
> or via Gitlab UI)
> 
> That way you keep the master branch clean and tested.
> 
> Whatever you decide, let me know and I'll change my build scripts :-)
> 
> Regards, Tim
> 
> 
> On 10/18/18 5:34 PM, Christian Grothoff wrote:
>> Hi Tim,
>>
>> The issue is that in line Makefile.am:10, we have "noinst_" instead of
>> "lib_" because the code is far from complete and it must not yet be
>> installed. That causes the dlname='' issue you describe.
>>
>> The best fix I can propose: I'm adding an option (--enable-experimental)
>> which disables building src/lib/ by default, so that users that don't
>> care about the experimental code don't get disrupted like this.
>>
>> Happy hacking!
>>
>> Christian
>>
>> On 10/18/2018 04:45 PM, Tim Rühsen wrote:
>>> Hi,
>>>
>>> the build on MinGW fails in lib/src/ due to
>>>
>>> @echo Creating $@ and libmicrohttpd2.exp by $(DLLTOOL)... && \
>>> dll_name=`$(EGREP) -o dlname=\'.+\' libmicrohttpd2.la` && \
>>>
>>> libmicrohttpd2.la contains
>>>   dlname=''
>>> and thus the egrep (or grep -E) fails.
>>>
>>> I am on Debian unstable and use this sequence in a script:
>>>
>>> unset CC
>>> PREFIX=x86_64-w64-mingw32
>>> export INSTALLDIR="$PWD/$PREFIX"
>>> export PKG_CONFIG_PATH=$INSTALLDIR/lib/pkgconfig
>>> export CPPFLAGS="-I$INSTALLDIR/include"
>>> export LDFLAGS="-L$INSTALLDIR/lib"
>>> export CFLAGS="-O2 -Wall -Wno-format"
>>>
>>> git clone --recursive https://github.com/dlfcn-win32/dlfcn-win32.git
>>> cd dlfcn-win32
>>> ./configure --prefix=$PREFIX --cc=$PREFIX-gcc
>>> make
>>> cp -p libdl.a ../$PREFIX/lib/
>>> cp -p dlfcn.h ../$PREFIX/include/
>>> cd ..
>>>
>>> git clone --recursive https://gnunet.org/git/libmicrohttpd.git
>>> cd libmicrohttpd
>>> ./bootstrap
>>> ./configure --build=x86_64-pc-linux-gnu --host=$PREFIX
>>> --prefix=$INSTALLDIR --disable-doc --disable-examples --enable-shared
>>> make clean
>>> make -j$(nproc)
>>> cd ..
>>>
>>>
>>> Regards, Tim
>>>
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] MHD_start_daemon and testing

[Christian Grothoff]

Yes.

On 10/26/18 4:23 PM, Erik Smith wrote:
> Is the server guaranteed to be listening after MHD_start_daemon
> returns?  I want to be able to be able to implement quick client/server
> unit testing without the need to poll the server to see if it's up.
> 
> erik
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] TCP connection callbacks - problem

[Christian Grothoff]

Hi Daniel,

Thanks, but no worries: this one has my attention, I just need to find
the time to look into it.  Oh, in terms of what would help: automated
testcases (especially those ready for integration with 'make check') are
always very helpful for speeding up things...

Happy hacking!

Christian

On 10/29/18 8:09 AM, Daniel Bujnik wrote:
> Hi Christian,
> 
> I have created a mantis bug for the issue I had described hoping it will
> get some more attention:
> https://gnunet.org/bugs/view.php?id=5463
> 
> Please let me know if you have any questions or would like me to test
> some more fixes.
> 
> Kind regards,
> Daniel
> 
> 
> 
> 
> 
> 
> On Sat, 20 Oct 2018 at 11:46, Christian Grothoff  <mailto:groth...@gnunet.org>> wrote:
> 
> Dear Daniel,
> 
> You didn't mention which thread mode you were using, but manual
> inspection suggests your issue should occur in thread-per-connection
> modes (only).  In that case, I've pushed a (likely) fix in
> b55cd46b..47281241 to Git master.
> 
> Please test and report back!
> 
> Happy hacking!
> 
> Christian
> 
> On 10/19/2018 03:53 PM, Daniel Bujnik wrote:
> > Hello,
> >
> > I am using libmicrohttpd with MHD_OPTION_NOTIFY_CONNECTION
> parameter to
> > receive notifications for TCP connection opening/closing.
> Notifications
> > for opening case are coming through as expected, however I am
> > experiencing problems with closing case, where I am NOT getting the
> > callback when my client terminates, though I can see client FIN packet
> > in tcpdump on the server side and connection stops appearing in
> netstat.
> >
> > Eventually the closing notification is being delivered by
> libmicrohttpd
> > but that is just after a new TCP connection is opened. The sequence of
> > callbacks I am seeing is as followed:
> >
> > 1) callback with MHD_CONNECTION_NOTIFY_STARTED - TCP connection  A
> opened
> > 2) callback with MHD_CONNECTION_NOTIFY_STARTED - new TCP
> connection B is
> > opened
> > 3)  callback with MHD_CONNECTION_NOTIFY_CLOSED - TCP connection  A
> closed
> >
> > Of course I would expect nr 3 to come before nr 2. I am using nc to
> > open/close sockets on client end.
> >
> > I would appreciate any comments on the described issue. Is that
> behavior
> > correct? If not is there a way to make it right?
> >
> > Kind regards,
> > Daniel
> >
>