Re: [PATCH 3 of 3] QUIC: path MTU discovery
Hi, On Mon, May 01, 2023 at 08:58:55PM +0400, Sergey Kandaurov wrote: > > > On 28 Mar 2023, at 18:51, Roman Arutyunyan wrote: > > > > # HG changeset patch > > # User Roman Arutyunyan > > # Date 1679993500 -14400 > > # Tue Mar 28 12:51:40 2023 +0400 > > # Branch quic > > # Node ID 13d43a278510f131101c7b19d87455a0171ebe2f > > # Parent c686c97f4abd6e1ca9a2cc2324d5a24f3d035c58 > > QUIC: path MTU discovery. > > > > MTU selection starts by probing the maximum allowed MTU first. After that, > > binary search is used to find the path MTU. > > > > Maximum allowed MTU is calculated as the minimum of max_udp_payload for > > client > > and server, and local interface MTU. > > > > diff --git a/auto/unix b/auto/unix > > --- a/auto/unix > > +++ b/auto/unix > > @@ -448,6 +448,54 @@ ngx_feature_test="setsockopt(0, IPPROTO_ > > . auto/feature > > > > > > +# IP packet fragmentation flags > > + > > +ngx_feature="IP_DONTFRAG" > > +ngx_feature_name="NGX_HAVE_IP_DONTFRAG" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test="getsockopt(0, IPPROTO_IP, IP_DONTFRAG, NULL, 0)" > > +. auto/feature > > + > > + > > +ngx_feature="IPV6_DONTFRAG" > > +ngx_feature_name="NGX_HAVE_IPV6_DONTFRAG" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test="getsockopt(0, IPPROTO_IPV6, IPV6_DONTFRAG, NULL, 0)" > > +. auto/feature > > + > > + > > +# Linux MTU flags > > + > > +ngx_feature="IP_PMTUDISC_DO" > > +ngx_feature_name="NGX_HAVE_IP_PMTUDISC_DO" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test="getsockopt(0, IPPROTO_IP, IP_PMTUDISC_DO, NULL, 0)" > > +. auto/feature > > + > > + > > +ngx_feature="IPV6_PMTUDISC_DO" > > +ngx_feature_name="NGX_HAVE_IPV6_PMTUDISC_DO" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test="getsockopt(0, IPPROTO_IPV6, IPV6_PMTUDISC_DO, NULL, 0)" > > +. auto/feature > > + > > + > > ngx_feature="TCP_DEFER_ACCEPT" > > ngx_feature_name="NGX_HAVE_DEFERRED_ACCEPT" > > ngx_feature_run=no > > @@ -920,6 +968,19 @@ ngx_feature_test="int i = FIONREAD; prin > > . auto/feature > > > > > > +ngx_feature="ioctl(SIOCGIFMTU)" > > +ngx_feature_name="NGX_HAVE_SIOCGIFMTU" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test="int i = SIOCGIFMTU; struct ifreq ifr; > > + ifr.ifr_name[0] = 'e'; printf(\"%d\", i)" > > +. auto/feature > > + > > + > > ngx_feature="struct tm.tm_gmtoff" > > ngx_feature_name="NGX_HAVE_GMTOFF" > > ngx_feature_run=no > > @@ -1002,3 +1063,17 @@ ngx_feature_test='struct addrinfo *res; > > if (getaddrinfo("localhost", NULL, NULL, &res) != 0) > > return 1; > > freeaddrinfo(res)' > > . auto/feature > > + > > + > > +ngx_feature="getifaddrs()" > > +ngx_feature_name="NGX_HAVE_GETIFADDRS" > > +ngx_feature_run=no > > +ngx_feature_incs="#include > > + #include > > + #include " > > +ngx_feature_path= > > +ngx_feature_libs= > > +ngx_feature_test='struct ifaddrs *ifaddr; > > + if (getifaddrs(&ifaddr) != 0) return 1; > > + freeifaddrs(ifaddr)' > > +. auto/feature > > diff --git a/src/core/ngx_connection.c b/src/core/ngx_connection.c > > --- a/src/core/ngx_connection.c > > +++ b/src/core/ngx_connection.c > > @@ -1010,6 +1010,74 @@ ngx_configure_listening_sockets(ngx_cycl > > } > > > > #endif > > + > > +#if (NGX_HAVE_IP_PMTUDISC_DO) > > + > > +if (ls[i].quic && ls[i].sockaddr->sa_family == AF_INET) { > > +value = 1; > > + > > +if (setsockopt(ls[i].fd, IPPROTO_IP, IP_PMTUDISC_DO, > > + (const void *) &value, sizeof(int)) > > +== -1) > > +{ > > +ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, > > + "setsockopt(IP_PMTUDISC_DO) " > > + "for %V failed, ignored", > > + &ls[i].addr_text); > > +} > > +} > > + > > +#elif (NGX_HAVE_IP_DONTFRAG) > > + > > +if (ls[i].quic && ls[i].sockaddr->sa_family == AF_INET) { > > +value = 1; > > + > > +if (setsockopt(ls[i].fd, IPPROTO_IP, IP_DONTFRAG, > > + (const void *) &value, sizeof(int)) > > +== -1) > > +{ > > +ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, > > + "setsockopt(IP_DONTFRAG) " > > + "for %V failed, ignor
Re: [PATCH 3 of 3] QUIC: path MTU discovery
> On 8 May 2023, at 16:15, Roman Arutyunyan wrote: > > Hi, > > On Mon, May 01, 2023 at 08:58:55PM +0400, Sergey Kandaurov wrote: >> >>> On 28 Mar 2023, at 18:51, Roman Arutyunyan wrote: >>> >>> # HG changeset patch >>> # User Roman Arutyunyan >>> # Date 1679993500 -14400 >>> # Tue Mar 28 12:51:40 2023 +0400 >>> # Branch quic >>> # Node ID 13d43a278510f131101c7b19d87455a0171ebe2f >>> # Parent c686c97f4abd6e1ca9a2cc2324d5a24f3d035c58 >>> QUIC: path MTU discovery. >>> [..] >> >> I'd move the dontfrag part to a separate change for clarity. >> It can be seen as a foundation for succeeding PLPMTUD work >> not strictly related to it. >> (Further, PLPMTUD is an optional feature, while dontfrag >> is a MUST per RFC 9000, section 14.) > > You're right. Attached is a separate patch for this. > > [..] > I think it's fine. -- Sergey Kandaurov ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: [PATCH] Configure: introduced --without-libxslt
Hello! On Sat, May 06, 2023 at 06:03:15PM -0700, Dmitry Volyntsev wrote: > On 06.05.2023 16:12, Maxim Dounin wrote: > > > On Fri, May 05, 2023 at 11:05:09PM -0700, Dmitry Volyntsev wrote: > > > >> # HG changeset patch > >> # User Dmitry Volyntsev > >> # Date 1683353037 25200 > >> # Fri May 05 23:03:57 2023 -0700 > >> # Node ID 4891e0920d7c0e89def28694686e34294c69acf1 > >> # Parent b71e69247483631bd8fc79a47cc32b762625b1fb > >> Configure: introduced --without-libxslt. > >> > >> This allows to explicitly disable libxslt discovery by > >> nginx and nginx addons. > >> > >> diff --git a/auto/lib/conf b/auto/lib/conf > >> --- a/auto/lib/conf > >> +++ b/auto/lib/conf > >> @@ -29,8 +29,21 @@ if [ $USE_ZLIB = YES ]; then > >> . auto/lib/zlib/conf > >> fi > >> > >> -if [ $USE_LIBXSLT != NO ]; then > >> +if [ $USE_LIBXSLT != NO -a $USE_LIBXSLT != DISABLED ]; then > >> . auto/lib/libxslt/conf > >> + > >> +else > >> +if [ $USE_LIBXSLT = DISABLED -a $HTTP = YES -a $HTTP_XSLT = YES ]; > >> then > >> + > >> +cat << END > >> + > >> +$0: error: the HTTP ngx_http_xslt_module requires the libxslt library. > >> +You can either disable the module by using --without-http_xslt_module > >> +option or you have to enable the libxslt support. > >> + > >> +END > >> +exit 1 > >> +fi > >> fi > >> > >> if [ $USE_LIBGD != NO ]; then > >> diff --git a/auto/modules b/auto/modules > >> --- a/auto/modules > >> +++ b/auto/modules > >> @@ -277,7 +277,7 @@ if [ $HTTP = YES ]; then > >> . auto/module > >> fi > >> > >> -if [ $HTTP_XSLT != NO ]; then > >> +if [ $HTTP_XSLT != NO -a $USE_LIBXSLT != DISABLED ]; then > >> ngx_module_name=ngx_http_xslt_filter_module > >> ngx_module_incs= > >> ngx_module_deps= > >> diff --git a/auto/options b/auto/options > >> --- a/auto/options > >> +++ b/auto/options > >> @@ -363,6 +363,8 @@ use the \"--with-mail_ssl_module\" optio > >> --with-openssl=*)OPENSSL="$value" ;; > >> --with-openssl-opt=*)OPENSSL_OPT="$value" ;; > >> > >> +--without-libxslt) USE_LIBXSLT=DISABLED ;; > >> + > >> --with-md5=*) > >> NGX_POST_CONF_MSG="$NGX_POST_CONF_MSG > >> $0: warning: the \"--with-md5\" option is deprecated" > > > > The only "without" configure option for libraries we use is > > "--without-pcre", and it is disables PCRE library usage in the > > nginx core (notably, regular expressions in server names and > > locations). > > > > In contrast, the XSLT library is only used by the xslt filter > > module, and the natural way to disable its usage is to don't > > enable the module. Similarly, OpenSSL, zlib, GD, and GeoIP > > libraries are enabled by the corresponding modules, and not > > enabled when the modules are not enabled. > > > > It is not clear why XSLT should be different, and how this is > > expected to be used. > > That is correct that XSLT library is only used for xslt filter. > Nevertheless LIBXSLT is supported in ngx_module_libs for 3rd-party > nginx modules, so for XSLT there are may be other uses outside of > nginx core. Unlike OPENSSL and ZLIB which are essential parts of any > modern nginx build, LIBXSLT is less commonly used and users > sometimes want to disable LIBXSLT altogether. As for the other libraries, the expected approach to disable the XSLT library usage is to disable modules which depend on it. > What is the suggested way for a user to build 3rd-party module when > he/she wants to specifically disable a part which needs LIBXSLT? > > Alternatively there can be a way for nginx user building nginx to > somehow signal to a 3rd-party module configure script what he/she wants. As currently implemented, nginx expects an addon module to list libraries it depends on. If a module normally depends on a library, but can be configured to do not depend on it, it is something to be configured on the module side. A readily available solutions would be to pre-configure the module somehow, or use an environment variable to provide module-specific options. We can consider implementing a way to provide module-specific configure options, but I don't think I remember [m]any requests for this. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel
enable request_auth module to send auth service error message body when it is allowed
# HG changeset patch # User Davood Falahati <0x0dav...@gmail.com> # Date 1683588448 -7200 # Tue May 09 01:27:28 2023 +0200 # Node ID 0977f155bc2d288eedf006033b9a5094d0e8098f # Parent b71e69247483631bd8fc79a47cc32b762625b1fb let request_auth_module pass auth body when it is allowed diff -r b71e69247483 -r 0977f155bc2d src/http/modules/ngx_http_auth_request_module.c --- a/src/http/modules/ngx_http_auth_request_module.c Mon May 01 19:16:05 2023 +0400 +++ b/src/http/modules/ngx_http_auth_request_module.c Tue May 09 01:27:28 2023 +0200 @@ -13,6 +13,7 @@ typedef struct { ngx_str_t uri; ngx_array_t *vars; +ngx_flag_tenable; } ngx_http_auth_request_conf_t; @@ -62,6 +63,12 @@ NGX_HTTP_LOC_CONF_OFFSET, 0, NULL }, +{ ngx_string("send_auth_body"), + NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF | NGX_CONF_TAKE1, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_auth_request_conf_t, enable), + NULL }, ngx_null_command }; @@ -106,6 +113,9 @@ ngx_http_post_subrequest_t*ps; ngx_http_auth_request_ctx_t *ctx; ngx_http_auth_request_conf_t *arcf; +ngx_list_t *hs; +ngx_buf_t *b; +ngx_chain_t out, *in; arcf = ngx_http_get_module_loc_conf(r, ngx_http_auth_request_module); @@ -141,6 +151,36 @@ if (ctx->status == NGX_HTTP_UNAUTHORIZED) { sr = ctx->subrequest; +if (arcf->enable) { + +r->headers_out.content_type = sr->headers_out.content_type; + +hs = &sr->headers_out.headers; + +r->headers_out.headers = *hs; + +b = ngx_calloc_buf(r->pool); +if (b == NULL) { + return NGX_ERROR; +} + +r->headers_out.status = ctx->status; + +b->last_buf = 1; +b->last_in_chain = 1; +b->memory = 1; + +out.buf = b; +out.next = NULL; + +in = ctx->subrequest->out; +in->next = &out; + +ngx_http_send_header(r); + +return ngx_http_output_filter(r, in); +} + h = sr->headers_out.www_authenticate; if (!h && sr->upstream) { @@ -323,6 +363,8 @@ conf->vars = NGX_CONF_UNSET_PTR; +conf->enable = NGX_CONF_UNSET; + return conf; } @@ -335,6 +377,7 @@ ngx_conf_merge_str_value(conf->uri, prev->uri, ""); ngx_conf_merge_ptr_value(conf->vars, prev->vars, NULL); +ngx_conf_merge_value(conf->enable, prev->enable, 0); return NGX_CONF_OK; } ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel
keep response body of the subrequest inside the memory and use it if send_auth_body is set
# HG changeset patch # User Davood Falahati <0x0dav...@gmail.com> # Date 1683593026 -7200 # Tue May 09 02:43:46 2023 +0200 # Node ID 1053357966cda6a0902b748a9b4b8a214b36ccd4 # Parent b71e69247483631bd8fc79a47cc32b762625b1fb keep response body of the subrequest inside the memory and use it if send_auth_body is set diff -r b71e69247483 -r 1053357966cd src/http/modules/ngx_http_auth_request_module.c --- a/src/http/modules/ngx_http_auth_request_module.c Mon May 01 19:16:05 2023 +0400 +++ b/src/http/modules/ngx_http_auth_request_module.c Tue May 09 02:43:46 2023 +0200 @@ -13,6 +13,7 @@ typedef struct { ngx_str_t uri; ngx_array_t *vars; +ngx_flag_tenable; } ngx_http_auth_request_conf_t; @@ -62,6 +63,12 @@ NGX_HTTP_LOC_CONF_OFFSET, 0, NULL }, +{ ngx_string("send_auth_body"), + NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF | NGX_CONF_TAKE1, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_auth_request_conf_t, enable), + NULL }, ngx_null_command }; @@ -106,6 +113,9 @@ ngx_http_post_subrequest_t*ps; ngx_http_auth_request_ctx_t *ctx; ngx_http_auth_request_conf_t *arcf; +ngx_list_t *hs; +ngx_buf_t *b; +ngx_chain_t out, *in; arcf = ngx_http_get_module_loc_conf(r, ngx_http_auth_request_module); @@ -141,6 +151,36 @@ if (ctx->status == NGX_HTTP_UNAUTHORIZED) { sr = ctx->subrequest; +if (arcf->enable) { + +r->headers_out.content_type = sr->headers_out.content_type; + +hs = &sr->headers_out.headers; + +r->headers_out.headers = *hs; + +b = ngx_calloc_buf(r->pool); +if (b == NULL) { + return NGX_ERROR; +} + +r->headers_out.status = ctx->status; + +b->last_buf = 1; +b->last_in_chain = 1; +b->memory = 1; + +out.buf = b; +out.next = NULL; + +in = sr->out; +in->next = &out; + +ngx_http_send_header(r); + +return ngx_http_output_filter(r, in); +} + h = sr->headers_out.www_authenticate; if (!h && sr->upstream) { @@ -191,9 +231,12 @@ ps->handler = ngx_http_auth_request_done; ps->data = ctx; - +/* +* response body is being kept in memory and client won't receive it +* use subrequest->out to access the chain buffer +*/ if (ngx_http_subrequest(r, &arcf->uri, NULL, &sr, ps, -NGX_HTTP_SUBREQUEST_WAITED) +NGX_HTTP_SUBREQUEST_IN_MEMORY) != NGX_OK) { return NGX_ERROR; @@ -209,8 +252,6 @@ return NGX_ERROR; } -sr->header_only = 1; - ctx->subrequest = sr; ngx_http_set_ctx(r, ctx, ngx_http_auth_request_module); @@ -323,6 +364,8 @@ conf->vars = NGX_CONF_UNSET_PTR; +conf->enable = NGX_CONF_UNSET; + return conf; } @@ -335,6 +378,7 @@ ngx_conf_merge_str_value(conf->uri, prev->uri, ""); ngx_conf_merge_ptr_value(conf->vars, prev->vars, NULL); +ngx_conf_merge_value(conf->enable, prev->enable, 0); return NGX_CONF_OK; } ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: enable request_auth module to send auth service error message body when it is allowed
Hello! On Tue, May 09, 2023 at 01:40:18AM +0200, Davood Falahati wrote: > # HG changeset patch > # User Davood Falahati <0x0dav...@gmail.com> > # Date 1683588448 -7200 > # Tue May 09 01:27:28 2023 +0200 > # Node ID 0977f155bc2d288eedf006033b9a5094d0e8098f > # Parent b71e69247483631bd8fc79a47cc32b762625b1fb > let request_auth_module pass auth body when it is allowed > > diff -r b71e69247483 -r 0977f155bc2d > src/http/modules/ngx_http_auth_request_module.c > --- a/src/http/modules/ngx_http_auth_request_module.c Mon May 01 19:16:05 > 2023 +0400 > +++ b/src/http/modules/ngx_http_auth_request_module.c Tue May 09 01:27:28 > 2023 +0200 > @@ -13,6 +13,7 @@ > typedef struct { > ngx_str_t uri; > ngx_array_t *vars; > +ngx_flag_tenable; > } ngx_http_auth_request_conf_t; > > > @@ -62,6 +63,12 @@ >NGX_HTTP_LOC_CONF_OFFSET, >0, >NULL }, > +{ ngx_string("send_auth_body"), > + NGX_HTTP_MAIN_CONF | NGX_HTTP_SRV_CONF | NGX_HTTP_LOC_CONF | > NGX_CONF_TAKE1, > + ngx_conf_set_flag_slot, > + NGX_HTTP_LOC_CONF_OFFSET, > + offsetof(ngx_http_auth_request_conf_t, enable), > + NULL }, > >ngx_null_command > }; > @@ -106,6 +113,9 @@ > ngx_http_post_subrequest_t*ps; > ngx_http_auth_request_ctx_t *ctx; > ngx_http_auth_request_conf_t *arcf; > +ngx_list_t *hs; > +ngx_buf_t *b; > +ngx_chain_t out, *in; > > arcf = ngx_http_get_module_loc_conf(r, ngx_http_auth_request_module); > > @@ -141,6 +151,36 @@ > if (ctx->status == NGX_HTTP_UNAUTHORIZED) { > sr = ctx->subrequest; > > +if (arcf->enable) { > + > +r->headers_out.content_type = sr->headers_out.content_type; > + > +hs = &sr->headers_out.headers; > + > +r->headers_out.headers = *hs; > + > +b = ngx_calloc_buf(r->pool); > +if (b == NULL) { > + return NGX_ERROR; > +} > + > +r->headers_out.status = ctx->status; > + > +b->last_buf = 1; > +b->last_in_chain = 1; > +b->memory = 1; > + > +out.buf = b; > +out.next = NULL; > + > +in = ctx->subrequest->out; > +in->next = &out; > + > +ngx_http_send_header(r); > + > +return ngx_http_output_filter(r, in); > +} > + > h = sr->headers_out.www_authenticate; > > if (!h && sr->upstream) { > @@ -323,6 +363,8 @@ > > conf->vars = NGX_CONF_UNSET_PTR; > > +conf->enable = NGX_CONF_UNSET; > + > return conf; > } > > @@ -335,6 +377,7 @@ > > ngx_conf_merge_str_value(conf->uri, prev->uri, ""); > ngx_conf_merge_ptr_value(conf->vars, prev->vars, NULL); > +ngx_conf_merge_value(conf->enable, prev->enable, 0); > > return NGX_CONF_OK; > } Thanks for the patch. It is, however, is not going to work for at least two reasons: 1. The ctx->subrequest->out is only available when there is a NGX_HTTP_SUBREQUEST_IN_MEMORY flag (and implies various restrictions). 2. The auth subrequst is created with the sr->header_only flag set, so the will be no response body available in at all. Futher, it might not be a good idea to copy all headers from the subrequest while not providing various links and pointers from the r->headers_out structure. This is going to break various filter modules, such as charset filter (which uses r->headers_out.charset, r->headers_out.override_charset, r->headers_out.content_encoding), sub filter (as testing content type uses r->headers_out.content_type_len), and many more things. Note well that "enable" isn't a good name for a field responsible for an optional feature. Similarly, "send_auth_body" does not look self-explanatory. Overall, please also take a look at http://nginx.org/en/docs/contributing_changes.html for some basic hints on how to submit patches. Most notably, it might be a good idea outline the use case for the feature you are trying to introduce and why existing features are not enough for this use case. The design of the module generally suggests that the custom response body, if needed, can be provided using the error_page directive, much like with other auth modules. Hope this helps. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: keep response body of the subrequest inside the memory and use it if send_auth_body is set
Hello! On Tue, May 09, 2023 at 02:45:36AM +0200, Davood Falahati wrote: > # HG changeset patch > # User Davood Falahati <0x0dav...@gmail.com> > # Date 1683593026 -7200 > # Tue May 09 02:43:46 2023 +0200 > # Node ID 1053357966cda6a0902b748a9b4b8a214b36ccd4 > # Parent b71e69247483631bd8fc79a47cc32b762625b1fb > keep response body of the subrequest inside the memory and use it if > send_auth_body is set Please see the response to your previous patch. Please also note that when sending updated versions of a patch, it is usually a good idea to make sure they are properly threaded. When using hg email, use "--in-reply-to " to ensure correct threading. [...] -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx-devel