Describe HN: Nginx Total Well-known Configuration

Nginx total worthwhile configuration

Nginx configs. Not the most highly tremendous, productive or the supreme one. Objective worthwhile configs, which I’d like to notion in default nginx packages out of the box 😆
Bonus: fail2ban, filebeat, dockerfile and docker-price configs for nginx 🙂

Motivation: I in truth were the employ of nginx for the final 5 years on the least, and I configured it in truth for a complete bunch setups of 30+ firms and startups: sites, apps, websockets, proxies, load balancing, from just a few up to 1k rps, and plenty others… And I’m a piece bit upset by the official nginx wiki.
The final drop used to be this weblog publish within the official weblog:
this publish doesn’t provide a complete solution, half of these solutions can even be incorporated into nginx configs or snippets by default,
and one of the most varied solutions, equivalent to disabling access logging, in my ogle are the execrable observe 😆

At the same time there are plenty beautiful documentation and supreme practices:
nginx scientific doctors,
digitalocean config generator,
mozilla ssl supreme practices,
and plenty others…

And there are also some more attention-grabbing initiatives and examples:

• nginx-admins-handbook
  Big complete recordsdata, must be taught for any nginx admin.
• html5-boilerplate nginx configs
  Most traditional sequence of configuration snippets.
• nginx-boilerplate
  One other one total boilerplate.
• elasticweb/nginx-configs
  Assortment of Nginx configs for preferred CMS/CMF/Frameworks per PHP.
• openbridge/nginx
  Docker image, but I luxuriate in now not checked it effectively yet, their configs require further nginx modules and setup
  and it would possibly now not be true copied to the in vogue nginx setup. On the opposite hand, that you just can employ it with docker.
  Also I price now not accept as true with nginx microcache for every station, notion identified traps.
• hub.docker.com/_/nginx
  Legit nginx docker image and scientific doctors.

So right here I’m attempting to position collectively all (my) beautiful patterns and data, and arrange it as merely as that you just can dispute when put next with advanced examples above. So somebody will be ready to repeat this configs and rep a lovely nginx setup out of the box 🙂

You will be ready to vote for my feature requests in official docker-nginx repo:

• [Feature Request] Evolved default settings
• [Feature Request] Custom envsubst for templating with default values

Time track:

• Filipp Frizzy 48.23h

Toughen

You will be ready to enhance this or any varied of my initiatives

• by sending your PRs with enhancing my configs or texts 😂
• by sending me donations:
  • 
  • donationalerts.com/r/filipp_frizzy
  • ETH 0xCD9fC1719b9E174E911f343CA2B391060F931ff7
  • BTC bc1q8fhsj24f5ncv3995zk9v3jhwwmscecc6w0tdw3

Configs

Predominant configs

Nearly all sections moved from main nginx.conf into conf.d directory:

• basic.conf
  Total settings, mime kinds, charset, index, timeouts, open file cache, and plenty others…
• cache.conf
  Fastcgi, Proxy and Uwsgi cache setup, notion identified traps forward of the employ of 😉
• gzip.conf
  Gzip and gzip static
• log_format.conf
  Prolonged log codecs
• real_ip.conf
  Allow X-Forwarded-For header from native networks and cloudflare
• request_id.conf
  Add X-Put a matter to of-ID header into each request for tracing and debugging
• security.conf
  Security settings and headers
• ssl.conf
  SSL supreme observe from mozilla

Snippets

Templates and entails. You will be ready to also employ config generator from digitalocean 🙂

• corps.consist of.template
  Template of corps politic for more than one subdomains setup
• default.conf
  Instance of default config with nginx_status, let’s encrypt test and redirect to https
• fastcgi.consist of
  Encompass for php locations: fastcgi parameters, timeouts and cache instance
• headers.consist of
  Encompass with all headers, notion identified traps
• protected_locations.consist of
  Encompass with protected locations with ‘squawk all’
• proxy.consist of
  Encompass for proxy locations: proxy headers, parameters, timeouts and cache instance
• referer.consist of.template
  Template of referer protection for cases whilst you concurents employ your fail2ban protection in opposition to you, notion identified traps
• resolver.conf.template
  Encompass for dynamic dns resolving, notion identified traps
• station.conf.template
  Template of total station configuration
• static_location.consist of
  Encompass with save for static recordsdata

Dockerfile

Dockerfile instance with fabricate args, configs copying and customised envsubst template engine

Docker-price

docker-price.yml instance for nginx

Fail2ban

You will be ready to employ fail2ban for banning some bots even on the motivate of load balancer.
nginx-squawk action will add squawk ; into /and plenty others/nginx/conf.d/banned.conf and reload nginx.

Warning: your rotten competitors can employ your protection like fail2ban in opposition to you, test identified traps 😉

Files for copying:

fail2ban/detention heart.native=> /and plenty others/fail2ban/detention heart.native
fail2ban/action-nginx-squawk.conf=> /and plenty others/fail2ban/action.d/nginx-squawk.conf
fail2ban/filter-magento.conf=> /and plenty others/fail2ban/filter.d/nginx-magento.conf
fail2ban/filter-wordpress.conf=> /and plenty others/fail2ban/filter.d/nginx-wordpress.conf
fail2ban/filter-nginx-noscript.conf=> /and plenty others/fail2ban/filter.d/nginx-noscript.conf

Filebeat

Filebeat by default can now not parse extended nginx access log codecs, so that you just should serene override ingest json:
Reproduction filebeat/nginx_access_ingest.json to /usr/share/filebeat/module/nginx/access/ingest/default.json

Identified traps

Cache with default settings destroy all client particular command

If you happen to employ fastcgi, proxy or uwsgi cache with default settings like

http {

    proxy_cache_path /tmp/cache stages=1:2 keys_zone=mycache: 10m max_size=10g 
                idle=60m use_temp_path=off;

    server {
        listen 80;
        proxy_cache mycache;

        save / {
            proxy_pass http://backend1;
        }

        save /some/path {
            proxy_pass http://backend2;
            proxy_cache_valid any 1m;
            proxy_cache_min_uses 3;
            proxy_cache_bypass $cookie_nocache $arg_nocache$arg_comment;
        }
    }
}

in each locations Nginx will cache every response.
So if your station has some login efficiency or shopping cart or whatever,
this would possibly also be blended and most of possibilities will rep response with command of some varied possibilities.

In this configuration I counsel caches simplest as an further tool for caching total non 200 area responses:

fastcgi_cache_valid 499 500 502 503 504 521 522 523 524 3s; # circuit breaker
fastcgi_cache_valid 404 15m; # cache Not Stumbled on for lower loading to backend
fastcgi_cache_valid 301 308 1h; # cache Eternal Redirect for lower loading to backend
fastcgi_cache_valid 302 307 5s; # cache Non eternal Redirect for lower loading to backend

# price now not cache any varied responses
fastcgi_cache_valid 200 0;
fastcgi_cache_valid any 0;

And even this one commented out in cache.conf, so that you just should serene take care of end yourself
and permit it manually for complete station or some locations.

On the opposite hand, how we are going to safely permit cache for all responses?.
And employ cache config like

fastcgi_cache_valid 401 0;
fastcgi_cache_valid any 3s;
fastcgi_cache_valid 404 15m;
fastcgi_cache_valid 301 308 1h;
fastcgi_cache_valid 200 5m;

1. The perfect
   By default, NGINX respects the Cache-Control headers from foundation servers.
   It doesn’t cache responses with Cache-Control station to Deepest, No-Cache, or
   No-Retailer or with Location-Cookie within the response header. So if your app can add Cache-Control
   header into every response – we are carried out right here 🙂 Instance

Parameters of caching would possibly additionally be station straight within the response header. This has increased priority than surroundings of caching time the employ of the directive.
- The “X-Accel-Expires” header area gadgets caching time of a response in seconds. The zero impress disables caching for a response. If the price starts with the @ prefix, it gadgets an absolute time in seconds since Epoch, up to which the response will be cached.
- If the header doesn't consist of the “X-Accel-Expires” area, parameters of caching will be station within the header fields “Expires” or “Cache-Control”.
- If the header entails the “Location-Cookie” area, this kind of response is per chance now not cached.
- If the header entails the “Range” area with the actual impress “*”, this kind of response is per chance now not cached (1.7.7). If the header entails the “Range” area with yet one more impress, this kind of response will be cached taking into story the corresponding request header fields (1.7.7).
Processing of 1 or more of these response header fields can even be disabled the employ of the fastcgi_ignore_headers directive.

ngx_http_fastcgi_module

2. Essentially the most true
   If you happen to app can store cache in an external cache database
   like redis or memcached, that you just can employ Nginx
   redis or
   memcached
   modules rather then nginx cache for every caching and rushing up your station.

3. Essentially the most complex
   You will be ready to verify URI and cookies by nginx itself, but right here is spellbinding
   and add a multitude into your configs and likelihood of errors. There would possibly be a lovely instance in
   the engintron configs,
   but it be under GPLv2 so I’m in a position to now not consist of it into my snippets. Also there is a piece more uncomplicated
   instance easy programs to study simplest one cookie.

4. Bonus: the lucky one
   For static command locations that you just can true permit cache with none dancing around 🙂

Adding add_header eradicate all add_header directives from guardian sections

Configuration like

add_header Name1 Value1;

save / {
    add_header Name2 Value2;

Finally create simplest Name2 header in response.
So employ add_header.conf consist of or copy all headers manually
into sections under HTTP one.

consist of /and plenty others/nginx/snippets/headers.consist of

DNS resolving and cache in Docker, Kubernetes and varied dynamic environments

By default, as NGINX starts up or reloads its configuration,
it queries a DNS server to resolve backend dns records.
The DNS server returns the list of backend IPs,
and NGINX makes employ of the default Spherical Robin algorithm to load balance requests amongst them.
NGINX chooses the DNS server from the OS configuration file /and plenty others/resolv.conf.
This implies is the least versatile capacity to price carrier discovery and has the following further drawbacks:

• If the area title can’t be resolved, NGINX fails to start or reload its configuration.
• NGINX caches the DNS records till the next restart or configuration reload, ignoring the records’ TTL values.

For dynamic dns resolving in docker, k8s and varied dynamic environments,
you should serene station the Enviornment Name in a Variable and add resolver directive
to explicitly specify the title server
as NGINX doesn’t consult with /and plenty others/resolv.conf in this case.

resolver 127.0.0.1 legitimate=10s;

server {
    save / {
        station $backend backends.instance.com;
        proxy_pass http://$backend;
    }
}

You will be ready to configure and consist of resolver.conf snippet for arrange resolver alternate choices:

consist of /and plenty others/nginx/snippets/resolver.conf

Fail2ban and any varied protection can even be weak in opposition to you

Not simplest that incorrectly configured protection will block legitimate customers,
even merely configured protection like fail2ban, in particular with botsearch-total filter,
can even be weak for assault to you. Shall we mumble, you competitors can add to their sites one thing like

“>

Then legitimate user after consult with to the their station will be robotically blocked to your station 😆
You will be ready to battle with this observe the employ of http_referer, notion snippets/referer.consist of.template template 😉
Warning: I in truth luxuriate in now not examined this code yet

Default templating engine in official docker image can now not proceed variables with default values like ${var:-$DEFAULT}

By default nginx in docker employ GNU envsubst
that can now not proceed variables with default values.
You will be ready to employ as a substitute a8m envsubst or stephenc envsub,
first one already has a prebuilded binary for x86_64 arch, test the Dockerfile in this repo 😉

Entails like /*.conf are processed within the alphabetic assert

Here’s notable for nginx in docker as all configs will be found in one dir

Errors like failed (24: Too many open recordsdata) or worker_connections exceed open file resource limit

Ache with limit of open recordsdata (ulimit -n)

You will be ready to trade it

• systemd
  Add into /and plenty others/systemd/machine/nginx.d/override.conf

[Service]
LimitNOFILE=100000

• dilapidated init machine
  Alternate /and plenty others/default/nginx

• docker-price

ulimits:
  nproc: 65535
  nofile:
    comfy: 100000
    spellbinding: 100000

Maybe you should serene also trade /and plenty others/security/limits.conf

nginx           spellbinding    nofile          100000
nginx           comfy    nofile          100000
www-data        spellbinding    nofile          100000
www-data        comfy    nofile          100000

and /and plenty others/sysctl.conf

Nginx fabricate data

Docker

nginx version: nginx/1.17.9
built by gcc 8.3.0 (Debian 8.3.0-6)
built with OpenSSL 1.1.1d  10 Sep 2019
TLS SNI enhance enabled
configure arguments: 
--prefix=/and plenty others/nginx 
--sbin-path=/usr/sbin/nginx 
--modules-path=/usr/lib/nginx/modules 
--conf-path=/and plenty others/nginx/nginx.conf 
--error-log-path=/var/log/nginx/error.log 
--http-log-path=/var/log/nginx/access.log 
--pid-path=/var/trip/nginx.pid 
--lock-path=/var/trip/nginx.lock 
--http-client-body-temp-path=/var/cache/nginx/client_temp 
--http-proxy-temp-path=/var/cache/nginx/proxy_temp 
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp 
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp 
--http-scgi-temp-path=/var/cache/nginx/scgi_temp 
--user=nginx 
--group=nginx 
--with-compat 
--with-file-aio 
--with-threads 
--with-http_addition_module 
--with-http_auth_request_module 
--with-http_dav_module 
--with-http_flv_module 
--with-http_gunzip_module 
--with-http_gzip_static_module 
--with-http_mp4_module 
--with-http_random_index_module 
--with-http_realip_module 
--with-http_secure_link_module 
--with-http_slice_module 
--with-http_ssl_module 
--with-http_stub_status_module 
--with-http_sub_module 
--with-http_v2_module 
--with-mail 
--with-mail_ssl_module 
--with-circulation 
--with-stream_realip_module 
--with-stream_ssl_module 
--with-stream_ssl_preread_module 
--with-cc-decide='-g -O2 
-fdebug-prefix-contrivance=/data/builder/debuild/nginx-1.17.9/debian/debuild-rotten/nginx-1.17.9=. 
-fstack-protector-tough -Wformat -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -fPIC' 
--with-ld-decide='-Wl,-z,relro -Wl,-z,now -Wl,--as-wished -pie'

Ubuntu 18.04 fabricate data

nginx version: nginx/1.14.0 (Ubuntu)
built with OpenSSL 1.1.1  11 Sep 2018
TLS SNI enhance enabled
configure arguments: 
--with-cc-decide='-g -O2 -fdebug-prefix-contrivance=/fabricate/nginx-GkiujU/nginx-1.14.0=. 
-fstack-protector-tough -Wformat -Werror=format-security 
-fPIC -Wdate-time -D_FORTIFY_SOURCE=2' 
--with-ld-decide='-Wl,-Bsymbolic-capabilities -Wl,-z,relro -Wl,-z,now -fPIC' 
--prefix=/usr/share/nginx 
--conf-path=/and plenty others/nginx/nginx.conf 
--http-log-path=/var/log/nginx/access.log 
--error-log-path=/var/log/nginx/error.log 
--lock-path=/var/lock/nginx.lock 
--pid-path=/trip/nginx.pid 
--modules-path=/usr/lib/nginx/modules 
--http-client-body-temp-path=/var/lib/nginx/body 
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi 
--http-proxy-temp-path=/var/lib/nginx/proxy 
--http-scgi-temp-path=/var/lib/nginx/scgi 
--http-uwsgi-temp-path=/var/lib/nginx/uwsgi 
--with-debug 
--with-pcre-jit 
--with-http_ssl_module 
--with-http_stub_status_module 
--with-http_realip_module 
--with-http_auth_request_module 
--with-http_v2_module 
--with-http_dav_module 
--with-http_slice_module 
--with-threads 
--with-http_addition_module 
--with-http_geoip_module=dynamic 
--with-http_gunzip_module 
--with-http_gzip_static_module 
--with-http_image_filter_module=dynamic 
--with-http_sub_module 
--with-http_xslt_module=dynamic 
--