Re: [libmicrohttpd] MHD_create_post_processor: recommended buffer size for large posts

[silvioprog]

Oops, wrong link at Google search, I forgot SVN was deprecated. ^^' (it
should be disabled)

Basically I want to offer general value for best performance on Android
(smartphones) and Linux/Windows (servers), so after your explanation I'm
going to use it:

#if defined(__ANDROID__)
# define BF_HTTPSRV_POST_BUF_SIZE 1024 /* 1 kB */
#elif defined(WIN32)
# define BF_HTTPSRV_POST_BUF_SIZE 32768 /* 32 kB */
#else
# define BF_HTTPSRV_POST_BUF_SIZE 65536 /* 64 kB */
#endif


Thanks dude! :-)

On Fri, Mar 31, 2017 at 3:42 AM, Evgeny Grin  wrote:

> Hi Silvio,
>
> Sidenote: SVN is not updated any more, latest version is on git:
> https://gnunet.org/git/libmicrohttpd.git/tree/doc/examples/largepost.c
>
"512" is not recommended size, it's just suitable size for example.
>
> Optimal value depends on OS, architecture, amount of available RAM.
> You should experiment to find out what is optimal for you.
>
> --
> Best Wishes,
> Evgeny Grin
>
> On 31.03.2017 6:17, silvioprog wrote:
> > Hello dudes,
> >
> > Looking at MHD examples, it seems the recommended buffer size to specify
> > on MHD_create_post_processor() for large uploads is 512:
> >
> > https://gnunet.org/svn/libmicrohttpd/doc/examples/largepost.c
> > 
> >
> > However, what means large for you? About 100 MB, 1 GB, 10 GB or greater?!
> >
> > I have a structure that I need to support ~ 5 GB, so I'm planning to use
> > 1024, because I want fast transfer using less memory and CPU possible.
> > In that case, is 1024 an ideal size?
> >
> > Thank you!
> >
> > P.S.: we should specify the size (in MB or GB) that we consider large on
> > the example `largepost.c`.
> >
> > --
> > Silvio Clécio
>

-- 
Silvio Clécio

Re: [libmicrohttpd] Using MHD_UpgradeResponseHandle without including internal.h

[John Duncan]

I was trying to pass custom non-global scoped data to my upgrade_cb for
websockets (upgrade_cb is the name of the routine in test_upgrade.c).  I
noticed that the opaque MHD_UpgradeResponseHandle structure was passed as a
pointer parameter to the callback, so I was trying to investigate its
members to get some insight.  That's when I realized that the members were
not exported.  Your answer provides the insight I needed regarding opacity,
however I still need to pass in custom data to the upgrade callback.

The callback has the following parameters, and I'm guessing I'm supposed to
be using extra_in to pass data into the callback:

static void upgrade_cb
(
void *cls,
struct MHD_Connection *connection,
void *con_cls,
const char *extra_in,
size_t extra_in_size,
MHD_socket sock,
struct MHD_UpgradeResponseHandle *urh
);

However, the only time I see upgrade_cb referenced in test_upgrade.c is
when it's used as follows:

resp = MHD_create_response_for_upgrade (&upgrade_cb, NULL);

Do I pass data in through the second parameter?

Basically, my question boils down to:

How do I attach my own data so that when the callback is triggered I can
pass through my own pointers/structures to the callback without utilizing
globally scoped variables?

Thanks;
~JD




On Thu, Mar 30, 2017 at 8:32 PM, Evgeny Grin  wrote:

> Hi John,
>
> MHD_UpgradeResponseHandle is intentionally opaque. It's not designed for
> direct usage of internal members.
> If you define it locally, you'll almost likely to break compatibility
> with future version.
> MHD uses a lot of opaque structures which are internally changed from
> version to version. But as long as application uses only public official
> API, those changes don't break compatibility.
>
> Usually MHD has specific API to get information about opaque handle.
> What do you want to get from internal members?
>
> --
> Best Wishes,
> Evgeny Grin
>
> On 31.03.2017 0:58, John Duncan wrote:
> > I noticed that microhttpd.h only contains only forward defintions of the
> > MHD_UpgradeResponseHandle structure.  The file internal.h contains the
> > actual structure definition with all members.
> >
> >  When I try to include the internal.h file directly, I get tons of
> > compilation errors leading me to believe that I'm not supposed to be
> > including it.
> >
> > My question is, when building my own applications, how am I supposed to
> > access member portions of the MHD_UpgradeResponseHandle structure
> > without including internal.h.  Am I expected to define the structure
> > within my own project headers and use the provided
> > MHD_UpgradeResponseHandle pointer as somewhat of an opaque type?
> >
> > I don't mind doing this, I just want to make sure this is the correct
> > approach.
> >
> > Thanks;
> > ~JD
>
>

Re: [libmicrohttpd] Using MHD_UpgradeResponseHandle without including internal.h

[Christian Grothoff]

Hi John,

You're expected to pass data via "cls", you have no influence on
"extra_in".  "extra_in" is there in case MHD "accidentally" read already
more data than the HTTP header from the Web socket, thereby making it
available to you (you are to treat it as if you read it from 'sock').

So in your example where there is "NULL", just pass some pointer and
you'll be given it in the "cls" argument.

Happy hacking!

Christian

On 04/01/2017 01:26 AM, John Duncan wrote:
> I was trying to pass custom non-global scoped data to my upgrade_cb for
> websockets (upgrade_cb is the name of the routine in test_upgrade.c).  I
> noticed that the opaque MHD_UpgradeResponseHandle structure was passed as a
> pointer parameter to the callback, so I was trying to investigate its
> members to get some insight.  That's when I realized that the members were
> not exported.  Your answer provides the insight I needed regarding opacity,
> however I still need to pass in custom data to the upgrade callback.
> 
> The callback has the following parameters, and I'm guessing I'm supposed to
> be using extra_in to pass data into the callback:
> 
> static void upgrade_cb
> (
> void *cls,
> struct MHD_Connection *connection,
> void *con_cls,
> const char *extra_in,
> size_t extra_in_size,
> MHD_socket sock,
> struct MHD_UpgradeResponseHandle *urh
> );
> 
> However, the only time I see upgrade_cb referenced in test_upgrade.c is
> when it's used as follows:
> 
> resp = MHD_create_response_for_upgrade (&upgrade_cb, NULL);
> 
> Do I pass data in through the second parameter?
> 
> Basically, my question boils down to:
> 
> How do I attach my own data so that when the callback is triggered I can
> pass through my own pointers/structures to the callback without utilizing
> globally scoped variables?
> 
> Thanks;
> ~JD
> 
> 
> 
> 
> On Thu, Mar 30, 2017 at 8:32 PM, Evgeny Grin  wrote:
> 
>> Hi John,
>>
>> MHD_UpgradeResponseHandle is intentionally opaque. It's not designed for
>> direct usage of internal members.
>> If you define it locally, you'll almost likely to break compatibility
>> with future version.
>> MHD uses a lot of opaque structures which are internally changed from
>> version to version. But as long as application uses only public official
>> API, those changes don't break compatibility.
>>
>> Usually MHD has specific API to get information about opaque handle.
>> What do you want to get from internal members?
>>
>> --
>> Best Wishes,
>> Evgeny Grin
>>
>> On 31.03.2017 0:58, John Duncan wrote:
>>> I noticed that microhttpd.h only contains only forward defintions of the
>>> MHD_UpgradeResponseHandle structure.  The file internal.h contains the
>>> actual structure definition with all members.
>>>
>>>  When I try to include the internal.h file directly, I get tons of
>>> compilation errors leading me to believe that I'm not supposed to be
>>> including it.
>>>
>>> My question is, when building my own applications, how am I supposed to
>>> access member portions of the MHD_UpgradeResponseHandle structure
>>> without including internal.h.  Am I expected to define the structure
>>> within my own project headers and use the provided
>>> MHD_UpgradeResponseHandle pointer as somewhat of an opaque type?
>>>
>>> I don't mind doing this, I just want to make sure this is the correct
>>> approach.
>>>
>>> Thanks;
>>> ~JD
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: [libmicrohttpd] Using MHD_UpgradeResponseHandle without including internal.h

[John Duncan]

Well, that's fantastic!  Thanks man.

~JD

On Fri, Mar 31, 2017 at 5:08 PM, Christian Grothoff 
wrote:

> Hi John,
>
> You're expected to pass data via "cls", you have no influence on
> "extra_in".  "extra_in" is there in case MHD "accidentally" read already
> more data than the HTTP header from the Web socket, thereby making it
> available to you (you are to treat it as if you read it from 'sock').
>
> So in your example where there is "NULL", just pass some pointer and
> you'll be given it in the "cls" argument.
>
> Happy hacking!
>
> Christian
>
> On 04/01/2017 01:26 AM, John Duncan wrote:
> > I was trying to pass custom non-global scoped data to my upgrade_cb for
> > websockets (upgrade_cb is the name of the routine in test_upgrade.c).  I
> > noticed that the opaque MHD_UpgradeResponseHandle structure was passed
> as a
> > pointer parameter to the callback, so I was trying to investigate its
> > members to get some insight.  That's when I realized that the members
> were
> > not exported.  Your answer provides the insight I needed regarding
> opacity,
> > however I still need to pass in custom data to the upgrade callback.
> >
> > The callback has the following parameters, and I'm guessing I'm supposed
> to
> > be using extra_in to pass data into the callback:
> >
> > static void upgrade_cb
> > (
> > void *cls,
> > struct MHD_Connection *connection,
> > void *con_cls,
> > const char *extra_in,
> > size_t extra_in_size,
> > MHD_socket sock,
> > struct MHD_UpgradeResponseHandle *urh
> > );
> >
> > However, the only time I see upgrade_cb referenced in test_upgrade.c is
> > when it's used as follows:
> >
> > resp = MHD_create_response_for_upgrade (&upgrade_cb, NULL);
> >
> > Do I pass data in through the second parameter?
> >
> > Basically, my question boils down to:
> >
> > How do I attach my own data so that when the callback is triggered I can
> > pass through my own pointers/structures to the callback without utilizing
> > globally scoped variables?
> >
> > Thanks;
> > ~JD
> >
> >
> >
> >
> > On Thu, Mar 30, 2017 at 8:32 PM, Evgeny Grin  wrote:
> >
> >> Hi John,
> >>
> >> MHD_UpgradeResponseHandle is intentionally opaque. It's not designed for
> >> direct usage of internal members.
> >> If you define it locally, you'll almost likely to break compatibility
> >> with future version.
> >> MHD uses a lot of opaque structures which are internally changed from
> >> version to version. But as long as application uses only public official
> >> API, those changes don't break compatibility.
> >>
> >> Usually MHD has specific API to get information about opaque handle.
> >> What do you want to get from internal members?
> >>
> >> --
> >> Best Wishes,
> >> Evgeny Grin
> >>
> >> On 31.03.2017 0:58, John Duncan wrote:
> >>> I noticed that microhttpd.h only contains only forward defintions of
> the
> >>> MHD_UpgradeResponseHandle structure.  The file internal.h contains the
> >>> actual structure definition with all members.
> >>>
> >>>  When I try to include the internal.h file directly, I get tons of
> >>> compilation errors leading me to believe that I'm not supposed to be
> >>> including it.
> >>>
> >>> My question is, when building my own applications, how am I supposed to
> >>> access member portions of the MHD_UpgradeResponseHandle structure
> >>> without including internal.h.  Am I expected to define the structure
> >>> within my own project headers and use the provided
> >>> MHD_UpgradeResponseHandle pointer as somewhat of an opaque type?
> >>>
> >>> I don't mind doing this, I just want to make sure this is the correct
> >>> approach.
> >>>
> >>> Thanks;
> >>> ~JD
> >>
> >>
> >
>
>