Http2 enable on all virtul host settings automatically

2017-09-14 Thread Junaid Malik 
Hello Guys,

We recently upgraded Nginx from nginx/1.9.12 - nginx/1.13.2, details of 
nginx/1.13.2 supported modules are given below. We are facing problem of 
automatic enabling of HTTP2 protocol on bsa1.example.com as we only enabled 
http2 on dsa1.example.com. Nginx configurations of both sites are given below.

Supported urls of different Nginx configurations are given below respectively
1 - 
https://dsa1.example.com/forums/user_avatar/www.example.com/cooltahir/25/1497380_1.png
2 - 
https://bsa1.example.com/blog/wp-content/plugins/ultimate-responsive-image-slider/css/slider-pro.css?ver=4.6.1


Site to verify Http2 protocol
 https://tools.keycdn.com/http2-test


--
nginx version
--
nginx version: nginx/1.13.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
built with OpenSSL 1.0.2k  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx 
--modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf 
--error-log-path=/var/log/nginx/error.log 
--http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid 
--lock-path=/var/run/nginx.lock 
--http-client-body-temp-path=/var/cache/nginx/client_temp 
--http-proxy-temp-path=/var/cache/nginx/proxy_temp 
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp 
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp 
--http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx 
--with-compat --with-file-aio --with-threads --with-http_addition_module 
--with-http_auth_request_module --with-http_dav_module --with-http_flv_module 
--with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module 
--with-http_random_index_module --with-http_realip_module 
--with-http_secure_link_module --with-http_slice_module --with-http_ssl_module 
--with-openssl=openssl-1.0.2k --with-http_stub_status_module 
--with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module 
--with-stream --with-stream_realip_module --with-stream_ssl_module 
--with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall 
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong 
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' 
--with-ld-opt=


---
dsa.conf
---
proxy_cache_path /var/www/example_dsa/ levels=2:2:2 keys_zone=pakwheels-dsa:50m 
max_size=300m inactive=525600m loader_files=400;

server {
listen   80;
listen 443 ssl http2;
# Enable SSL

#ssl_certificate /etc/nginx/certs/pakwheels_with_subdomains.pem;
#ssl_certificate_key /etc/nginx/certs/example_with_subdomains.key;

ssl_certificate 
/etc/nginx/certs/pakwheels_with_subdomains_renew_28_august.pem;
ssl_certificate_key 
/etc/nginx/certs/example_with_subdomains_renew_28_august.key;


ssl_session_timeout 10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES;
ssl_prefer_server_ciphers on;




server_name dsa1.example.com dsa2.example.com dsa3.example.com 
dsa4.example.com;



   rewrite ^/forums/forums/(.*)$ /forums/$1 permanent;

   location / {
gzip on;
gzip_min_length 100;
gzip_types text/plain text/xml application/xml text/css text/javascript 
application/javascript application/x-javascript text/x-component 
application/json application/xhtml+xml application/rss+xml application/atom+xml 
application/vnd.ms-fontobject image/svg+xml application/x-font-ttf 
font/opentype application/octet-stream;
gzip_comp_level 1;
gzip_disable "MSIE [1-6]\.";
expires 12M;

# ProxySettings
proxy_cache_lock off;
proxy_set_header Accept-Encoding "";
add_header X-Cache $upstream_cache_status;
add_header 'Access-Control-Allow-Origin' '*';
#proxy_ignore_headers Vary;
proxy_ignore_headers Set-Cookie;
resolver 213.133.100.100 213.133.99.99 213.133.98.98;
set $backend www.example.com;
proxy_pass https://$backend$request_uri;
#proxy_set_header Authorization "Basic 
cGFrYm9hcmQ6M3YzbnR1cjNzMDA3";
#proxy_pass_header  Authorization;
proxy_pass_header P3P;
proxy_cache_min_uses 1;
proxy_cache pakwheels-dsa;
proxy_cache_valid 200 365d;
proxy_cache_valid any 2s;
proxy_cache_key pwstatic.pakwheels0$uri$is_args$args;

proxy_intercept_errors on;
error_page 403 = @no_image;
error_page 404 = @no_image;
error_page 400 = @no_image;

proxy_hide_header   x-amz-id-2;
proxy_hide_header   x-amz-request-id;
# END ProxySettings

}

   location @no_image {
return 404 '';
add_header Content

Re: Http2 enable on all virtul host settings automatically

2017-09-14 Thread Maxim Dounin 
Hello!

On Thu, Sep 14, 2017 at 11:06:30AM +, Junaid Malik wrote:

> We recently upgraded Nginx from nginx/1.9.12 - nginx/1.13.2, 
> details of nginx/1.13.2 supported modules are given below. We 
> are facing problem of automatic enabling of HTTP2 protocol on 
> bsa1.example.com as we only enabled http2 on dsa1.example.com. 
> Nginx configurations of both sites are given below.

HTTP/2 is enabled on a listening socket, not in a particular 
server block.  Quoting http://nginx.org/r/listen:

: The http2 parameter (1.9.5) configures the port to accept HTTP/2 connections.
: Normally, for this to work the ssl parameter should be specified as well, but
: nginx can also be configured to accept HTTP/2 connections without SSL.

-- 
Maxim Dounin
http://nginx.org/
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Too many redirects

2017-09-14 Thread tseveendorj 

Hi,
I configured http (www, non-www) to https (non-www) and https (www) to 
https (non-www). It is working fine. But I need to add geoip redirection 
in location / but I got Too many redirect.


This server is behind load balancer. LB is redirecting 80 to 81 and 443 
to 80.

server {
listen  81;
server_name www.example.com example.com;
return  301 https://example.com$request_uri;
}
server {
listen  80;
server_name www.example.com;
return  301 https://example.com$request_uri;
}
server {
listen   80; ## listen for ipv4; this line is default and implied
server_name example.com;

...
}

I added geoip redirection in

nginx.conf
geoip_country /usr/share/GeoIP/GeoIP.dat;

sites-enabled/example.com

location / {

# AWS load balancer access log off
if ($ignore_ua) {
access_log off;
return 200;
}

index index.html;
if ($geoip_country_code != "JP") { return 301 
https://example.com/en/; }

}
location = /en/ {
index index.html;
try_files $uri $uri/ =404;
}

I'm trying to if request from other than JP request go to /en/ if not 
/index.html


BR,
Tseveen





___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

nginx cache growing well above max_size threshold

2017-09-14 Thread Lucas Rolff 
Hi guys,


I have a minor question, so I have an nginx box using proxy_cache, it has a key 
zone of 40 gigabyte (so it can cache 320 million files), a max_size of 1500 
gigabyte for the cache and the inactive set to 30 days.


However we experience that nginx goes well above the defined limit - in our 
case the max size is 1500 gigabyte, but the cache directory takes goes well 
above 1700 gigabyte.

There's a total of 42.000.000 files currently on the system, meaning the 
average filesize is about 43 kilobyte.


Normally I know that nginx can go slightly above the limit, until the cache 
manager purges the files, but it stays at about 1700 gigabyte constantly unless 
we manually clear out the size.


I see there's a change in 1.13.1 that ignores long locked cache entries, is it 
possible that this bugfix actually fixes above issue?

Upgrading is rather time consuming and we have to ensure nginx versions across 
the platform, so I wonder if anyone has some pointers if the above bugfix would 
maybe solve our issue. (currently the custom nginx version is based on nginx 
1.10.3).


Best Regards,
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: nginx cache growing well above max_size threshold

2017-09-14 Thread Maxim Dounin 
Hello!

On Thu, Sep 14, 2017 at 04:34:09PM +, Lucas Rolff wrote:

> I have a minor question, so I have an nginx box using 
> proxy_cache, it has a key zone of 40 gigabyte (so it can cache 
> 320 million files), a max_size of 1500 gigabyte for the cache 
> and the inactive set to 30 days.
> 
> However we experience that nginx goes well above the defined 
> limit - in our case the max size is 1500 gigabyte, but the cache 
> directory takes goes well above 1700 gigabyte.
> 
> There's a total of 42.000.000 files currently on the system, 
> meaning the average filesize is about 43 kilobyte.
> 
> Normally I know that nginx can go slightly above the limit, 
> until the cache manager purges the files, but it stays at about 
> 1700 gigabyte constantly unless we manually clear out the size.
> 
> I see there's a change in 1.13.1 that ignores long locked cache 
> entries, is it possible that this bugfix actually fixes above 
> issue?
> 
> Upgrading is rather time consuming and we have to ensure nginx 
> versions across the platform, so I wonder if anyone has some 
> pointers if the above bugfix would maybe solve our issue. 
> (currently the custom nginx version is based on nginx 1.10.3).

https://trac.nginx.org/nginx/ticket/1163

TL;DR:

This behaviour indicate there is a problem somewhere, likely 
socket leaks or process crashes.  Reports suggests it might be 
related to HTTP/2.  The change in 1.13.1 don't fix the root cause, 
but will allow nginx to keep cache under max_size regardless of 
the problem.

-- 
Maxim Dounin
http://nginx.org/
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: nginx cache growing well above max_size threshold

2017-09-14 Thread Lucas Rolff 
Okay cool, I'll give it a try

In our case we do not run http2 on the machines since haproxy runs in front as 
well (which doesn't support http2)

I'll also try enable a bit more verbose logging on one of the machines to see 
what the logs say

Thanks a lot Maxim!

Best regards,
Lucas Rolff

Get Outlook for iOS

From: nginx  on behalf of Maxim Dounin 

Sent: Thursday, September 14, 2017 6:55:57 PM
To: nginx@nginx.org
Subject: Re: nginx cache growing well above max_size threshold

Hello!

On Thu, Sep 14, 2017 at 04:34:09PM +, Lucas Rolff wrote:

> I have a minor question, so I have an nginx box using
> proxy_cache, it has a key zone of 40 gigabyte (so it can cache
> 320 million files), a max_size of 1500 gigabyte for the cache
> and the inactive set to 30 days.
>
> However we experience that nginx goes well above the defined
> limit - in our case the max size is 1500 gigabyte, but the cache
> directory takes goes well above 1700 gigabyte.
>
> There's a total of 42.000.000 files currently on the system,
> meaning the average filesize is about 43 kilobyte.
>
> Normally I know that nginx can go slightly above the limit,
> until the cache manager purges the files, but it stays at about
> 1700 gigabyte constantly unless we manually clear out the size.
>
> I see there's a change in 1.13.1 that ignores long locked cache
> entries, is it possible that this bugfix actually fixes above
> issue?
>
> Upgrading is rather time consuming and we have to ensure nginx
> versions across the platform, so I wonder if anyone has some
> pointers if the above bugfix would maybe solve our issue.
> (currently the custom nginx version is based on nginx 1.10.3).

https://trac.nginx.org/nginx/ticket/1163

TL;DR:

This behaviour indicate there is a problem somewhere, likely
socket leaks or process crashes.  Reports suggests it might be
related to HTTP/2.  The change in 1.13.1 don't fix the root cause,
but will allow nginx to keep cache under max_size regardless of
the problem.

--
Maxim Dounin
http://nginx.org/
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: nginx cache growing well above max_size threshold

2017-09-14 Thread Jeff Dyke 
You can actually can run H/2 through HAProxy, using ALPN to determine if
the client understands H/2  I have the following (snippet of a) config that
sends to different nginx ports based on the ALPN response.

frontend https
  mode tcp
  bind 0.0.0.0:443 ssl crt /etc/haproxy/certs alpn h2,http/1.1 ecdhe
secp384r1
  http-response set-header Strict-Transport-Security "max-age=1600;
includeSubDomains; preload;"
  timeout http-request 10s
  #send all HTTP/2 traffic to a specific backend
  use_backend http2-nodes if { ssl_fc_alpn -i h2 }
  #send HTTP/1.1 and HTTP/1.0 to default, which don't speak HTTP/2
  default_backend http1-nodes

backend http1-nodes
  mode http
  balance roundrobin
  default-server inter 1s fall 2

  server web01 10.1.1.12:80 check send-proxy
  server web03 10.1.1.14:80 check send-proxy

backend http2-nodes
  mode tcp
  balance roundrobin
  default-server inter 1s fall 2

  server web01 10.1.1.12:81 check send-proxy
  server web03 10.1.1.14:81 check send-proxy

Sounds like you may not want to complicate this system ATM, but just
throwing it out there.  It's worked really well for me, i have had i live
for about a year.


On Thu, Sep 14, 2017 at 1:09 PM, Lucas Rolff  wrote:

> Okay cool, I'll give it a try
>
> In our case we do not run http2 on the machines since haproxy runs in
> front as well (which doesn't support http2)
>
> I'll also try enable a bit more verbose logging on one of the machines to
> see what the logs say
>
> Thanks a lot Maxim!
>
> Best regards,
> Lucas Rolff
>
> Get Outlook for iOS 
> --
> *From:* nginx  on behalf of Maxim Dounin <
> mdou...@mdounin.ru>
> *Sent:* Thursday, September 14, 2017 6:55:57 PM
> *To:* nginx@nginx.org
> *Subject:* Re: nginx cache growing well above max_size threshold
>
> Hello!
>
> On Thu, Sep 14, 2017 at 04:34:09PM +, Lucas Rolff wrote:
>
> > I have a minor question, so I have an nginx box using
> > proxy_cache, it has a key zone of 40 gigabyte (so it can cache
> > 320 million files), a max_size of 1500 gigabyte for the cache
> > and the inactive set to 30 days.
> >
> > However we experience that nginx goes well above the defined
> > limit - in our case the max size is 1500 gigabyte, but the cache
> > directory takes goes well above 1700 gigabyte.
> >
> > There's a total of 42.000.000 files currently on the system,
> > meaning the average filesize is about 43 kilobyte.
> >
> > Normally I know that nginx can go slightly above the limit,
> > until the cache manager purges the files, but it stays at about
> > 1700 gigabyte constantly unless we manually clear out the size.
> >
> > I see there's a change in 1.13.1 that ignores long locked cache
> > entries, is it possible that this bugfix actually fixes above
> > issue?
> >
> > Upgrading is rather time consuming and we have to ensure nginx
> > versions across the platform, so I wonder if anyone has some
> > pointers if the above bugfix would maybe solve our issue.
> > (currently the custom nginx version is based on nginx 1.10.3).
>
> https://trac.nginx.org/nginx/ticket/1163
>
> TL;DR:
>
> This behaviour indicate there is a problem somewhere, likely
> socket leaks or process crashes.  Reports suggests it might be
> related to HTTP/2.  The change in 1.13.1 don't fix the root cause,
> but will allow nginx to keep cache under max_size regardless of
> the problem.
>
> --
> Maxim Dounin
> http://nginx.org/
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

empty user-agent and logs

2017-09-14 Thread Christos Chatzaras 
curl -A "-" https://hostname/index.php 

and

curl -A "" https://hostname/index.php 

are logged with:

xxx.xxx.xxx.xxx - - [14/Sep/2017:22:47:09 +0300] "GET /index.php HTTP/1.1" 200 
26039 "-" "-"

There is not difference if there is an empty user-agent or a user-agent with a 
dash.

Any idea it shows the empty user agents with dash?___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Too many redirects

2017-09-14 Thread Francis Daly 
On Thu, Sep 14, 2017 at 11:54:23PM +0800, tseveendorj wrote:

Hi there,

> location / {
> index index.html;
> if ($geoip_country_code != "JP") { return 301
> https://example.com/en/; }
> }

> location = /en/ {

Change that to "location /en/ {" or "location ^~ /en/ {".

> index index.html;
> try_files $uri $uri/ =404;
> }
> 
> I'm trying to if request from other than JP request go to /en/ if
> not /index.html

You request /en/, which is processed in the second location. That does
an internal redirect to /en/index.html, which is processed in the first
location and returns a 301 redirect to /en/, so you have a loop.

Change "location =" to break the loop, so that the request to
/en/index.html is not handled in the first location.

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: empty user-agent and logs

2017-09-14 Thread Francis Daly 
On Thu, Sep 14, 2017 at 10:57:39PM +0300, Christos Chatzaras wrote:

Hi there,

> xxx.xxx.xxx.xxx - - [14/Sep/2017:22:47:09 +0300] "GET /index.php HTTP/1.1" 
> 200 26039 "-" "-"
> 
> There is not difference if there is an empty user-agent or a user-agent with 
> a dash.
> 
> Any idea it shows the empty user agents with dash?

In the "common log format" widely used by web servers, the "hyphen" in the
output indicates that the requested piece of information is not available.

nginx does this in the function ngx_http_log_variable() in the file
src/http/modules/ngx_http_log_module.c. If the difference between empty
and dash matters to you, that is where you would probably change it.

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Too many redirects

2017-09-14 Thread Tseveendorj Ochirlantuu 
Thank you Francis Daly. It works.

On 15 Sep 2017 5:45 am, "Francis Daly"  wrote:

On Thu, Sep 14, 2017 at 11:54:23PM +0800, tseveendorj wrote:

Hi there,

> location / {
> index index.html;
> if ($geoip_country_code != "JP") { return 301
> https://example.com/en/; }
> }

> location = /en/ {

Change that to "location /en/ {" or "location ^~ /en/ {".

> index index.html;
> try_files $uri $uri/ =404;
> }
>
> I'm trying to if request from other than JP request go to /en/ if
> not /index.html

You request /en/, which is processed in the second location. That does
an internal redirect to /en/index.html, which is processed in the first
location and returns a 301 redirect to /en/, so you have a loop.

Change "location =" to break the loop, so that the request to
/en/index.html is not handled in the first location.

f
--
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: No Upstream Proxy Headers

2017-09-14 Thread Francis Daly 
On Mon, Sep 04, 2017 at 08:46:48PM +, Kilian Ries wrote:

Hi there,

> i'm running a nginx (version: nginx/1.13.1) with two vhosts with exact the 
> same configuration. The only difference is the upstream section: each vhosts 
> points to a different upstream server / ip.
> 

"nginx -T" will show the full configuration used.

That might help show other differences in the running configs.

> proxy_set_header X-Real-IP   $proxy_protocol_addr;
> proxy_set_header X-Forwarded-For $proxy_protocol_addr;

> vhost_1 works without any problem and i can see both proxy_headers in the 
> tcpdump and in my upstream-apache access-logs.

"Normal" web clients don't speak the proxy protocol.

Can you describe a complete test case that someone else can use to see
the problem you see?

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Modules needed to reverse proxy

2017-09-14 Thread Francis Daly 
On Thu, Sep 07, 2017 at 02:37:24AM +0200, Jose Carlos Sánchez wrote:

Hi there,

> Hi, i need to recompile nginx to include modsecurity module and i want to 
> take advantage of not compiling unnecessary modules. 

> Someone has the list of modules needed to do reverse proxy? 

./configure --help | grep -o -- --without'[^ ]'*

Use all of those except for --without-http (because you want to listen
for incoming http) and --without-http_proxy_module (because you want to
reverse-proxy to an upstream server).

After that, "nginx -t" will tell you if your config needs any other
modules -- for example, if you use "location ~", you might not want to
exclude pcre.

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx