Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

If you suspend a connection, you are responsible to schedule some job to
resume it, after which MHD's timeout counter will start again (and of
course your response logic could then also just force MHD to close the
connection once MHD asks for response data).  MHD timeouts are there to
detect (idle) clients that just keep connections open, not to prevent
the server from suspending a connection.

If the client closes a connection, it'll be detected if/when we try to
send data to the client and handled appropriately.

Happy hacking!

Christian

On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> If the connection is suspended and cases of:
> a. client closes the connection before response is sent.
> b. server app doesn't respond at all post suspend.
>  
> both cases there is no notification by MHD even if connection timeout is
> specified and notify callback. How can these cases be handled ?
> 
> Thanks,
> Kunal
> 
> On Mon, Sep 17, 2018 at 9:02 PM Santos Das  > wrote:
> 
> Hi,
> 
> Just to add more, my application is single threaded and I am running
> MHD inside that. I don't create additional threads. I use suspend
> and resume as suggested earlier as I do async processing.
> 
> So, I am thinking how can we implement flow control in this scenario..
> 
> Thanks, Santos
> 
> 
> 
> On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > wrote:
> 
> Hi,
> 
> Could you kindly let me know how can we implement the flow
> control ? Protection against the badly behaving clients.
> 
> 
> HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's 
> flow control 
> 
> mechanisms to resolve temporary overloads, rather than terminating 
> connections with 
> 
> the expectation that clients will retry. The latter technique can 
> exacerbate network 
> 
> congestion. 
> 
>  
> 
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> 
> 
> Thanks, Santos
> 
> 
> 
> 
> -- 
> ~Kunal


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

Hi!

Good news: You don't need to do anything.  MHD will maintain persistent
TCP connections and use TCP's flow control mechanisms automatically.
You can _force_ MHD to not keep connections open by adding "Connection:
Close" to your HTTP response header manually, but by default MHD will do
the right thing.

This is independent of your threading model or the use of suspend/resume.

Happy hacking!

Christian

On 09/17/2018 05:33 PM, Santos Das wrote:
> Hi,
> 
> Just to add more, my application is single threaded and I am running MHD
> inside that. I don't create additional threads. I use suspend and resume
> as suggested earlier as I do async processing.
> 
> So, I am thinking how can we implement flow control in this scenario..
> 
> Thanks, Santos
> 
> 
> 
> On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > wrote:
> 
> Hi,
> 
> Could you kindly let me know how can we implement the flow control ?
> Protection against the badly behaving clients.
> 
> 
> HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's 
> flow control 
> 
> mechanisms to resolve temporary overloads, rather than terminating 
> connections with 
> 
> the expectation that clients will retry. The latter technique can 
> exacerbate network 
> 
> congestion. 
> 
>  
> 
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> 
> 
> Thanks, Santos
> 
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Kunal Ekawde]

Thanks Christian,
So as a solution we need to have a timer for such cases and on timer expiry
, resume the connection post which as data is not available, either error
respond or close connection. If we close the connection, we shall loose the
pending requests on that connection right ? and if we error respond, we
shall get pending messages on that connection right (number of pending
messages would depend on MHD_OPTION_CONNECTION_MEMORY_LIMIT?) ?
If the case was client side disconnection, post resume, we should get
notify callback with reason 'MHD_REQUEST_TERMINATED_WITH_ERROR' ? so we can
do cleanup right ?

Thanks,
Kunal


On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff 
wrote:

> If you suspend a connection, you are responsible to schedule some job to
> resume it, after which MHD's timeout counter will start again (and of
> course your response logic could then also just force MHD to close the
> connection once MHD asks for response data).  MHD timeouts are there to
> detect (idle) clients that just keep connections open, not to prevent
> the server from suspending a connection.
>
> If the client closes a connection, it'll be detected if/when we try to
> send data to the client and handled appropriately.
>
> Happy hacking!
>
> Christian
>
> On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> > If the connection is suspended and cases of:
> > a. client closes the connection before response is sent.
> > b. server app doesn't respond at all post suspend.
> >
> > both cases there is no notification by MHD even if connection timeout is
> > specified and notify callback. How can these cases be handled ?
> >
> > Thanks,
> > Kunal
> >
> > On Mon, Sep 17, 2018 at 9:02 PM Santos Das  > > wrote:
> >
> > Hi,
> >
> > Just to add more, my application is single threaded and I am running
> > MHD inside that. I don't create additional threads. I use suspend
> > and resume as suggested earlier as I do async processing.
> >
> > So, I am thinking how can we implement flow control in this
> scenario..
> >
> > Thanks, Santos
> >
> >
> >
> > On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > > wrote:
> >
> > Hi,
> >
> > Could you kindly let me know how can we implement the flow
> > control ? Protection against the badly behaving clients.
> >
> >
> > HTTP/1.1 servers SHOULD maintain persistent connections and use
> TCP's flow control
> >
> > mechanisms to resolve temporary overloads, rather than
> terminating connections with
> >
> > the expectation that clients will retry. The latter technique
> can exacerbate network
> >
> > congestion.
> >
> >
> >
> > https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> >
> >
> > Thanks, Santos
> >
> >
> >
> >
> > --
> > ~Kunal
>


-- 
~Kunal

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 12:23 PM, Kunal Ekawde wrote:
> Thanks Christian,
> So as a solution we need to have a timer for such cases and on timer
> expiry , resume the connection post which as data is not available,
> either error respond or close connection.

Yes, basically if you suspend to perform some asynchronous activity that
needs a timeout, you need to manage that timeout.

> If we close the connection, we
> shall loose the pending requests on that connection right ?

Yes, if you "close" i.e. by returning "MHD_NO".

> and if we
> error respond, we shall get pending messages on that connection right
> (number of pending messages would depend on
> MHD_OPTION_CONNECTION_MEMORY_LIMIT?)?

If you respond, other requests on the connection will then be processed,
but not limited by the MEMORY_LIMIT.  MHD won't use more RAM per
connection than allowed by MEMORY_LIMIT, but the OSes for client and
server negotiate a TCP buffer which is really what limits network
transmissions.  And the client just won't be able to transmit more
requests than what that TCP buffer allows until you resume, but in
theory the number of requests that can be pipelined is unlimited.

> If the case was client side disconnection, post resume, we should get
> notify callback with reason 'MHD_REQUEST_TERMINATED_WITH_ERROR' ? so we
> can do cleanup right ?

Yes.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Santos Das]

Hi Christian,

Thanks a lot. I have few questions in this regard -

a.When we put the connection in suspend state, can we pass a timeout at
the expiry of which the connection will be automatically resumed?

b.For asynchronous communication , i.e the application received the
request but is not able to send the response now (it needs to do some work
may be creating another request to another endpoint) , is it mandatory to
use suspend-resume only or there is some other mechanism that we can use ?

c.   How does the keep alive works in the HTTP server library ?

Sorry, lot of questions. But, please see if you could help in this.

On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff 
wrote:

> If you suspend a connection, you are responsible to schedule some job to
> resume it, after which MHD's timeout counter will start again (and of
> course your response logic could then also just force MHD to close the
> connection once MHD asks for response data).  MHD timeouts are there to
> detect (idle) clients that just keep connections open, not to prevent
> the server from suspending a connection.
>
> If the client closes a connection, it'll be detected if/when we try to
> send data to the client and handled appropriately.
>
> Happy hacking!
>
> Christian
>
> On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> > If the connection is suspended and cases of:
> > a. client closes the connection before response is sent.
> > b. server app doesn't respond at all post suspend.
> >
> > both cases there is no notification by MHD even if connection timeout is
> > specified and notify callback. How can these cases be handled ?
> >
> > Thanks,
> > Kunal
> >
> > On Mon, Sep 17, 2018 at 9:02 PM Santos Das  > > wrote:
> >
> > Hi,
> >
> > Just to add more, my application is single threaded and I am running
> > MHD inside that. I don't create additional threads. I use suspend
> > and resume as suggested earlier as I do async processing.
> >
> > So, I am thinking how can we implement flow control in this
> scenario..
> >
> > Thanks, Santos
> >
> >
> >
> > On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > > wrote:
> >
> > Hi,
> >
> > Could you kindly let me know how can we implement the flow
> > control ? Protection against the badly behaving clients.
> >
> >
> > HTTP/1.1 servers SHOULD maintain persistent connections and use
> TCP's flow control
> >
> > mechanisms to resolve temporary overloads, rather than
> terminating connections with
> >
> > the expectation that clients will retry. The latter technique
> can exacerbate network
> >
> > congestion.
> >
> >
> >
> > https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> >
> >
> > Thanks, Santos
> >
> >
> >
> >
> > --
> > ~Kunal
>

Re: [libmicrohttpd] Regarding Flow control

[Santos Das]

Hi Christian,

Thank you. Could you please explain how the TCP's flow control mechanism is
helping in this case. A few lines would be great, please.

thanks, santos

On Tue, Sep 18, 2018 at 3:18 PM Christian Grothoff 
wrote:

> Hi!
>
> Good news: You don't need to do anything.  MHD will maintain persistent
> TCP connections and use TCP's flow control mechanisms automatically.
> You can _force_ MHD to not keep connections open by adding "Connection:
> Close" to your HTTP response header manually, but by default MHD will do
> the right thing.
>
> This is independent of your threading model or the use of suspend/resume.
>
> Happy hacking!
>
> Christian
>
> On 09/17/2018 05:33 PM, Santos Das wrote:
> > Hi,
> >
> > Just to add more, my application is single threaded and I am running MHD
> > inside that. I don't create additional threads. I use suspend and resume
> > as suggested earlier as I do async processing.
> >
> > So, I am thinking how can we implement flow control in this scenario..
> >
> > Thanks, Santos
> >
> >
> >
> > On Mon, Sep 17, 2018 at 8:22 PM Santos Das  > > wrote:
> >
> > Hi,
> >
> > Could you kindly let me know how can we implement the flow control ?
> > Protection against the badly behaving clients.
> >
> >
> > HTTP/1.1 servers SHOULD maintain persistent connections and use
> TCP's flow control
> >
> > mechanisms to resolve temporary overloads, rather than terminating
> connections with
> >
> > the expectation that clients will retry. The latter technique can
> exacerbate network
> >
> > congestion.
> >
> >
> >
> > https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> >
> >
> > Thanks, Santos
> >
> >
>

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 01:31 PM, Santos Das wrote:
> Hi Christian,
> 
> Thanks a lot. I have few questions in this regard -
> 
> a.    When we put the connection in suspend state, can we pass a timeout
> at the expiry of which the connection will be automatically resumed?

No.

> b.    For asynchronous communication , i.e the application received the
> request but is not able to send the response now (it needs to do some
> work may be creating another request to another endpoint) , is it
> mandatory to use suspend-resume only or there is some other mechanism
> that we can use ?

If you use 'thread per connection', you could also simply block.

> c.   How does the keep alive works in the HTTP server library ? 

MHD takes care of it, it adds keep-alive headers and processes them
_unless_ explicitly overridden by the application.

> Could you please explain how the TCP's flow control mechanism is
> helping in this case. A few lines would be great, please.

TCP flow control requires the kernel to provide message buffers for
retransmission (sender) and reordering (receiver). So those buffers can
and will be used by the kernel (on any OS) if the client sends
additional requests that the server is not yet processing.

> Sorry, lot of questions. But, please see if you could help in this.
> 
> On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff  > wrote:
> 
> If you suspend a connection, you are responsible to schedule some job to
> resume it, after which MHD's timeout counter will start again (and of
> course your response logic could then also just force MHD to close the
> connection once MHD asks for response data).  MHD timeouts are there to
> detect (idle) clients that just keep connections open, not to prevent
> the server from suspending a connection.
> 
> If the client closes a connection, it'll be detected if/when we try to
> send data to the client and handled appropriately.
> 
> Happy hacking!
> 
> Christian
> 
> On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> > If the connection is suspended and cases of:
> > a. client closes the connection before response is sent.
> > b. server app doesn't respond at all post suspend.
> >  
> > both cases there is no notification by MHD even if connection
> timeout is
> > specified and notify callback. How can these cases be handled ?
> >
> > Thanks,
> > Kunal
> >
> > On Mon, Sep 17, 2018 at 9:02 PM Santos Das  
> > >> wrote:
> >
> >     Hi,
> >
> >     Just to add more, my application is single threaded and I am
> running
> >     MHD inside that. I don't create additional threads. I use suspend
> >     and resume as suggested earlier as I do async processing.
> >
> >     So, I am thinking how can we implement flow control in this
> scenario..
> >
> >     Thanks, Santos
> >
> >
> >
> >     On Mon, Sep 17, 2018 at 8:22 PM Santos Das
> mailto:santos@gmail.com>
> >     >>
> wrote:
> >
> >         Hi,
> >
> >         Could you kindly let me know how can we implement the flow
> >         control ? Protection against the badly behaving clients.
> >
> >
> >         HTTP/1.1 servers SHOULD maintain persistent connections
> and use TCP's flow control
> >
> >         mechanisms to resolve temporary overloads, rather than
> terminating connections with
> >
> >         the expectation that clients will retry. The latter
> technique can exacerbate network
> >
> >         congestion.
> >
> >          
> >
> >         https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> >
> >
> >         Thanks, Santos
> >
> >
> >
> >
> > --
> > ~Kunal
> 


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Kunal Ekawde]

Thanks for prompt response, few more doubts :)
1. Yeah, timers per connection on each request doesn't seems good, need to
figure that out.
2. When you say ' MHD won't use more RAM per connection than allowed by
MEMORY_LIMIT' , will MHD read from TCP buffer only 'MEMORY_LIMIT' data ? is
it the recv call buffer size ? when will remaining data be read ?

Thanks

On Tue, Sep 18, 2018 at 4:37 PM Christian Grothoff 
wrote:

> On 09/18/2018 12:23 PM, Kunal Ekawde wrote:
> > Thanks Christian,
> > So as a solution we need to have a timer for such cases and on timer
> > expiry , resume the connection post which as data is not available,
> > either error respond or close connection.
>
> Yes, basically if you suspend to perform some asynchronous activity that
> needs a timeout, you need to manage that timeout.
>
> > If we close the connection, we
> > shall loose the pending requests on that connection right ?
>
> Yes, if you "close" i.e. by returning "MHD_NO".
>
> > and if we
> > error respond, we shall get pending messages on that connection right
> > (number of pending messages would depend on
> > MHD_OPTION_CONNECTION_MEMORY_LIMIT?)?
>
> If you respond, other requests on the connection will then be processed,
> but not limited by the MEMORY_LIMIT.  MHD won't use more RAM per
> connection than allowed by MEMORY_LIMIT, but the OSes for client and
> server negotiate a TCP buffer which is really what limits network
> transmissions.  And the client just won't be able to transmit more
> requests than what that TCP buffer allows until you resume, but in
> theory the number of requests that can be pipelined is unlimited.
>
> > If the case was client side disconnection, post resume, we should get
> > notify callback with reason 'MHD_REQUEST_TERMINATED_WITH_ERROR' ? so we
> > can do cleanup right ?
>
> Yes.
>


-- 
~Kunal

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:22 PM, Kunal Ekawde wrote:
> Thanks for prompt response, few more doubts :)
> 1. Yeah, timers per connection on each request doesn't seems good, need
> to figure that out.
> 2. When you say ' MHD won't use more RAM per connection than allowed by
> MEMORY_LIMIT' , will MHD read from TCP buffer only 'MEMORY_LIMIT' data ?
> is it the recv call buffer size ? when will remaining data be read ?

When the application has finished processing the data that was read and
thus created the necessary free space for more.



0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Regarding Flow control

[Santos Das]

Thanks Christian.

I have a question on "MHD_OPTION_CONNECTION_MEMORY_LIMIT" . I went through
the manual. How does this affect the send/recv buffer size per client
connection. What does it eventually translate to?

MHD_OPTION_CONNECTION_MEMORY_LIMIT - Maximum memory size per connection
(followed by a size_t). The default is 32 kB (32*1024 bytes) as defined by
the internal constant MHD_
POOL_SIZE_DEFAULT. Values above 128k are unlikely to result in much
benefit, as half of the memory will be typically used for IO, and TCP
buffers are unlikely to support window sizes above 64k on most systems.

On Tue, Sep 18, 2018 at 5:28 PM Christian Grothoff 
wrote:

> On 09/18/2018 01:31 PM, Santos Das wrote:
> > Hi Christian,
> >
> > Thanks a lot. I have few questions in this regard -
> >
> > a.When we put the connection in suspend state, can we pass a timeout
> > at the expiry of which the connection will be automatically resumed?
>
> No.
>
> > b.For asynchronous communication , i.e the application received the
> > request but is not able to send the response now (it needs to do some
> > work may be creating another request to another endpoint) , is it
> > mandatory to use suspend-resume only or there is some other mechanism
> > that we can use ?
>
> If you use 'thread per connection', you could also simply block.
>
> > c.   How does the keep alive works in the HTTP server library ?
>
> MHD takes care of it, it adds keep-alive headers and processes them
> _unless_ explicitly overridden by the application.
>
> > Could you please explain how the TCP's flow control mechanism is
> > helping in this case. A few lines would be great, please.
>
> TCP flow control requires the kernel to provide message buffers for
> retransmission (sender) and reordering (receiver). So those buffers can
> and will be used by the kernel (on any OS) if the client sends
> additional requests that the server is not yet processing.
>
> > Sorry, lot of questions. But, please see if you could help in this.
> >
> > On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff  > > wrote:
> >
> > If you suspend a connection, you are responsible to schedule some
> job to
> > resume it, after which MHD's timeout counter will start again (and of
> > course your response logic could then also just force MHD to close
> the
> > connection once MHD asks for response data).  MHD timeouts are there
> to
> > detect (idle) clients that just keep connections open, not to prevent
> > the server from suspending a connection.
> >
> > If the client closes a connection, it'll be detected if/when we try
> to
> > send data to the client and handled appropriately.
> >
> > Happy hacking!
> >
> > Christian
> >
> > On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
> > > If the connection is suspended and cases of:
> > > a. client closes the connection before response is sent.
> > > b. server app doesn't respond at all post suspend.
> > >
> > > both cases there is no notification by MHD even if connection
> > timeout is
> > > specified and notify callback. How can these cases be handled ?
> > >
> > > Thanks,
> > > Kunal
> > >
> > > On Mon, Sep 17, 2018 at 9:02 PM Santos Das  > 
> > > >>
> wrote:
> > >
> > > Hi,
> > >
> > > Just to add more, my application is single threaded and I am
> > running
> > > MHD inside that. I don't create additional threads. I use
> suspend
> > > and resume as suggested earlier as I do async processing.
> > >
> > > So, I am thinking how can we implement flow control in this
> > scenario..
> > >
> > > Thanks, Santos
> > >
> > >
> > >
> > > On Mon, Sep 17, 2018 at 8:22 PM Santos Das
> > mailto:santos@gmail.com>
> > > >>
> > wrote:
> > >
> > > Hi,
> > >
> > > Could you kindly let me know how can we implement the flow
> > > control ? Protection against the badly behaving clients.
> > >
> > >
> > > HTTP/1.1 servers SHOULD maintain persistent connections
> > and use TCP's flow control
> > >
> > > mechanisms to resolve temporary overloads, rather than
> > terminating connections with
> > >
> > > the expectation that clients will retry. The latter
> > technique can exacerbate network
> > >
> > > congestion.
> > >
> > >
> > >
> > > https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
> > >
> > >
> > > Thanks, Santos
> > >
> > >
> > >
> > >
> > > --
> > > ~Kunal
> >
>

Re: [libmicrohttpd] Regarding Flow control

[Santos Das]

Thanks Christian. Looks like you answered this question for Kunal Ekwade
already. Thank you.

But, still not clear from your reply, is it equal to the receive buffer
size ?




On Tue, Sep 18, 2018 at 6:05 PM Santos Das  wrote:

> Thanks Christian.
>
> I have a question on "MHD_OPTION_CONNECTION_MEMORY_LIMIT" . I went through
> the manual. How does this affect the send/recv buffer size per client
> connection. What does it eventually translate to?
>
> MHD_OPTION_CONNECTION_MEMORY_LIMIT - Maximum memory size per connection
> (followed by a size_t). The default is 32 kB (32*1024 bytes) as defined by
> the internal constant MHD_
> POOL_SIZE_DEFAULT. Values above 128k are unlikely to result in much
> benefit, as half of the memory will be typically used for IO, and TCP
> buffers are unlikely to support window sizes above 64k on most systems.
>
> On Tue, Sep 18, 2018 at 5:28 PM Christian Grothoff 
> wrote:
>
>> On 09/18/2018 01:31 PM, Santos Das wrote:
>> > Hi Christian,
>> >
>> > Thanks a lot. I have few questions in this regard -
>> >
>> > a.When we put the connection in suspend state, can we pass a timeout
>> > at the expiry of which the connection will be automatically resumed?
>>
>> No.
>>
>> > b.For asynchronous communication , i.e the application received the
>> > request but is not able to send the response now (it needs to do some
>> > work may be creating another request to another endpoint) , is it
>> > mandatory to use suspend-resume only or there is some other mechanism
>> > that we can use ?
>>
>> If you use 'thread per connection', you could also simply block.
>>
>> > c.   How does the keep alive works in the HTTP server library ?
>>
>> MHD takes care of it, it adds keep-alive headers and processes them
>> _unless_ explicitly overridden by the application.
>>
>> > Could you please explain how the TCP's flow control mechanism is
>> > helping in this case. A few lines would be great, please.
>>
>> TCP flow control requires the kernel to provide message buffers for
>> retransmission (sender) and reordering (receiver). So those buffers can
>> and will be used by the kernel (on any OS) if the client sends
>> additional requests that the server is not yet processing.
>>
>> > Sorry, lot of questions. But, please see if you could help in this.
>> >
>> > On Tue, Sep 18, 2018 at 3:12 PM Christian Grothoff > > > wrote:
>> >
>> > If you suspend a connection, you are responsible to schedule some
>> job to
>> > resume it, after which MHD's timeout counter will start again (and
>> of
>> > course your response logic could then also just force MHD to close
>> the
>> > connection once MHD asks for response data).  MHD timeouts are
>> there to
>> > detect (idle) clients that just keep connections open, not to
>> prevent
>> > the server from suspending a connection.
>> >
>> > If the client closes a connection, it'll be detected if/when we try
>> to
>> > send data to the client and handled appropriately.
>> >
>> > Happy hacking!
>> >
>> > Christian
>> >
>> > On 09/17/2018 09:01 PM, Kunal Ekawde wrote:
>> > > If the connection is suspended and cases of:
>> > > a. client closes the connection before response is sent.
>> > > b. server app doesn't respond at all post suspend.
>> > >
>> > > both cases there is no notification by MHD even if connection
>> > timeout is
>> > > specified and notify callback. How can these cases be handled ?
>> > >
>> > > Thanks,
>> > > Kunal
>> > >
>> > > On Mon, Sep 17, 2018 at 9:02 PM Santos Das > > 
>> > > >>
>> wrote:
>> > >
>> > > Hi,
>> > >
>> > > Just to add more, my application is single threaded and I am
>> > running
>> > > MHD inside that. I don't create additional threads. I use
>> suspend
>> > > and resume as suggested earlier as I do async processing.
>> > >
>> > > So, I am thinking how can we implement flow control in this
>> > scenario..
>> > >
>> > > Thanks, Santos
>> > >
>> > >
>> > >
>> > > On Mon, Sep 17, 2018 at 8:22 PM Santos Das
>> > mailto:santos@gmail.com>
>> > > >>
>> > wrote:
>> > >
>> > > Hi,
>> > >
>> > > Could you kindly let me know how can we implement the flow
>> > > control ? Protection against the badly behaving clients.
>> > >
>> > >
>> > > HTTP/1.1 servers SHOULD maintain persistent connections
>> > and use TCP's flow control
>> > >
>> > > mechanisms to resolve temporary overloads, rather than
>> > terminating connections with
>> > >
>> > > the expectation that clients will retry. The latter
>> > technique can exacerbate network
>> > >
>> > > congesti

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:35 PM, Santos Das wrote:
> Thanks Christian.
> 
> I have a question on "MHD_OPTION_CONNECTION_MEMORY_LIMIT" . I went
> through the manual. How does this affect the send/recv buffer size per
> client connection. What does it eventually translate to?

It does not directly translate. Basically, the send/recv buffer for the
application payload is whatever is left after MHD allocated storage for
the HTTP header. So if you have a 32k buffer and a 1k header, you may
have 30+ kb for send/recv of payload, but if you have a 32k buffer and
an 8k HTTP header, you might have only 24k for the send/recv buffer.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys

Re: [libmicrohttpd] Regarding Flow control

[Christian Grothoff]

On 09/18/2018 02:43 PM, Santos Das wrote:
> Thanks Christian. Looks like you answered this question for Kunal Ekwade
> already. Thank you. 
> 
> But, still not clear from your reply, is it equal to the receive buffer
> size ?

MHD uses "whatever is left" (minus some reserves depending on what phase
processing is in) -- first for the receive buffer, and later for the
send buffer (re-using space from the receive buffer).  In both cases it
is minus headers (headers sent and headers received!) and management
data structures.

The objective is simple: we must not use more than the memory limit per
connection, and within that constraint we make the buffers as big as we
(safely) can.


0x939E6BE1E29FC3CC.asc
Description: application/pgp-keys