NGINX for Staff Engineers
Everything from the event loop up to edge architecture — with the configuration, the failure modes, and the answers interviewers are actually probing for.
Junior answers describe what a directive does. Staff answers describe what happens to the system when it's wrong — connection exhaustion, thundering herds on cache expiry, retry storms during a partial upstream outage, config reloads that silently drop nothing but double memory. Every section below has a "what breaks" angle. Lead with that.
00Mental model & positioning
NGINX is a single-binary, event-driven, C-based network proxy. It is not "a web server that also proxies" — in modern architectures it is a proxy that also happens to serve files well.
The five jobs NGINX does in production
| Role | What it means | Typical placement |
|---|---|---|
| Static file server | sendfile/mmap straight from page cache to socket, zero app involvement | Rarely alone now — CDN does this |
| Reverse proxy | Terminates client conn, opens/reuses upstream conn, buffers, rewrites headers | In front of every app tier |
| Load balancer (L7) | Distributes across an upstream pool with health tracking | Edge tier + internal service tier |
| TLS terminator | Offloads handshakes/crypto; often the only TLS-aware component | Edge, or sidecar |
| Traffic policy point | Rate limits, auth subrequests, header injection, canary splits, caching | Edge / API gateway tier |
Why it beat Apache — the C10K story
Apache's prefork/worker MPM allocated a process or thread per connection. Each idle keep-alive connection cost ~1–8 MB of RSS and a scheduler slot. 10,000 concurrent connections meant 10,000 threads: context-switch thrash, memory exhaustion, and latency that degraded superlinearly.
NGINX inverted this: a small fixed number of single-threaded workers, each running a non-blocking event loop over epoll (Linux) / kqueue (BSD). Connection state is a small heap struct (~1 KB), not a stack. Memory scales with active connections, not threads. 100k idle keep-alives on one box is routine.
"NGINX trades programming model simplicity for constant memory per connection. The cost is that any blocking operation inside a worker stalls every connection on that worker — which is why disk I/O gets pushed to a thread pool (aio threads) and why you never embed slow synchronous logic in Lua on the request path."
Where NGINX sits in a modern stack
Editions you must be able to distinguish
| Edition | Notes |
|---|---|
| OSS nginx | Free. Passive health checks only, no native active probes, no dynamic upstream API, no session persistence beyond hashing. |
| PLUS NGINX Plus | Commercial. Active health checks, sticky cookie/learn, live activity dashboard + JSON stats API, dynamic reconfig API (add/remove upstreams without reload), cache purge API, JWT auth, dynamic modules. |
| FORK OpenResty | nginx + LuaJIT + ecosystem. Lets you write request logic in Lua at each phase. Basis of Kong. |
| FORK Angie / Freenginx | Community forks after the F5/Nginx Inc. governance split; Angie adds active health checks and Prometheus output for free. |
"On OSS I get passive health checks via max_fails/fail_timeout, which only eject a node after real user requests fail. If I need pre-emptive ejection I either buy Plus, run Angie, or put the health-check responsibility in the cloud LB / service discovery layer and re-render the upstream block."
01Process & event architecture
Process model
- Master runs as root only to bind privileged ports and read certs/keys, then supervises. It never touches request data.
- Workers drop to an unprivileged user (
user nginx;). Each is single-threaded and owns an event loop. Setworker_processes auto;→ one per CPU core. - Cache loader runs at startup, walks the cache dir, and populates the shared-memory key index. Cache manager runs continuously enforcing
max_sizeandinactiveeviction. - Shared memory zones are the only cross-worker state. This is why rate limits are approximately global rather than per-worker — but the counters live in a fixed-size slab that can fill up.
The event loop
Each worker loop iteration:
- Compute the nearest timer expiry (red-black tree of timers).
epoll_wait()with that timeout → get a batch of ready fds.- For each ready fd, dispatch its read/write handler — a state machine that does a non-blocking chunk of work and returns.
- Fire expired timers (request timeouts, keepalive expiry, resolver TTL).
- Run posted events (deferred work queued by handlers).
Never block a worker. A single read() on a cold file, a synchronous DNS lookup, a slow Lua HTTP call with a blocking socket, or an expensive regex over a huge body stalls every other connection pinned to that worker. Symptom: p99 latency spikes on a subset of requests with no upstream latency to explain it, and worker_processes-shaped periodicity in the latency histogram.
Non-blocking disk I/O
# Offload blocking file reads to a thread pool so the loop keeps spinning thread_pool default threads=32 max_queue=65536; location /video/ { aio threads=default; # async read via pool directio 16m; # bypass page cache for big files sendfile off; # sendfile and directio are mutually exclusive output_buffers 2 1m; }
This is the pattern for large-media serving: without it, one cold 1 GB read blocks the worker for the duration of the seek+read.
Connection acceptance & the thundering herd
| Mechanism | Behaviour | When to use |
|---|---|---|
accept_mutex on | Workers take turns accepting — avoids all workers waking on one connection. Adds latency (serialized accept). | Legacy / low connection rate. Default is off since 1.11.3. |
EPOLLEXCLUSIVE | Kernel wakes only one waiter. Used automatically on Linux 4.5+. | Default modern behaviour. |
reuseport on listen | Each worker gets its own listen socket + kernel accept queue; kernel hashes the 4-tuple to a queue. | Highest connection-rate workloads. Best throughput, but imbalance possible with long-lived conns. |
listen 443 ssl reuseport; # declare on ONE server block per addr:port only http2 on;
reuseport must appear on exactly one listen for a given address:port; repeating it in multiple server blocks is a config error. Also: during a reload, connections in a departing worker's dedicated accept queue can be dropped — a known trade-off of reuseport.
Memory model
- Per-connection pool: small arena allocator, freed wholesale at connection close — near-zero fragmentation, no GC pauses.
- Per-request pool: same idea, freed at request end.
- Shared slab zones: fixed size, declared in config.
zone=name:10m. Roughly 16k$binary_remote_addrstates per MB forlimit_req; ~8k cache keys per MB forproxy_cache_path.
When a limit_req zone fills, NGINX evicts the LRU entry — under a distributed attack with many source IPs, your zone thrashes and the limiter becomes ineffective. When a proxy_cache keys zone fills, NGINX starts force-evicting cache entries regardless of max_size, and you'll see the disk cache never reach its configured size. Size the keys zone first, then max_size.
02Request processing phases
This is the highest-signal architecture question in an NGINX interview. Modules register handlers into ordered phases; understanding the order explains almost every "why did my directive not apply?" bug.
| # | Phase | What runs here |
|---|---|---|
| 1 | POST_READ | realip module — rewrites $remote_addr from X-Forwarded-For before anything else sees it. |
| 2 | SERVER_REWRITE | rewrite/set directives at server{} level. |
| 3 | FIND_CONFIG | Location matching happens here. Not pluggable. |
| 4 | REWRITE | rewrite/set/if inside location{}. |
| 5 | POST_REWRITE | Internal-redirect check; loops back to FIND_CONFIG if the URI changed (max 10 cycles). |
| 6 | PREACCESS | limit_req, limit_conn, degradation. Rate limiting happens before auth. |
| 7 | ACCESS | allow/deny, auth_basic, auth_request, auth_jwt. |
| 8 | POST_ACCESS | Evaluates satisfy any|all across the access modules. |
| 9 | PRECONTENT | try_files, mirror, internal redirects. (Called TRY_FILES pre-1.13.) |
| 10 | CONTENT | Exactly one content handler: proxy_pass, fastcgi_pass, grpc_pass, static file handler, return, stub_status… |
| 11 | LOG | access_log. Runs even if the client disconnected mid-response. |
Response bodies then pass through the output filter chain (in order): ssi → sub_filter → gzip/brotli → chunked → writer. Headers pass through the header filter chain (add_header, headers_more, CORS).
"Why does my add_header disappear on a 502?" — Because core add_header only applies to a whitelist of status codes (200, 201, 204, 206, 301, 302, 303, 304, 307, 308) unless you add the always flag. Use add_header X-Foo bar always; or the third-party headers_more module.
"Why does my add_header in a child location wipe out the parent's?" — add_header is array-inheriting: a child context that defines any add_header replaces the whole inherited set. Same for proxy_set_header. This causes real security incidents (HSTS/CSP silently dropped in one location).
"Why is my auth bypassed for rate-limited clients?" — It isn't; the reverse is true. limit_req is PREACCESS, so a 429 is returned before auth runs. That's usually what you want (cheap rejection) but means unauthenticated attackers consume your limit-zone slots.
Internal redirects
location / { try_files $uri $uri/ /index.html; # each fallback re-enters FIND_CONFIG } location @fallback { # named location: only reachable internally proxy_pass http://legacy_backend; } location /api/ { error_page 404 = @fallback; # "=" makes it adopt the fallback's status proxy_pass http://api_backend; }
X-Accel-Redirect is the same machinery driven by the upstream: your app authorizes a download, then returns an empty 200 with X-Accel-Redirect: /protected/file.zip, and NGINX serves the file from an internal location. The app never streams bytes.
location /protected/ { internal; # 404 for direct client requests alias /var/data/files/; }
03Configuration model
Context hierarchy
# ── main (global) ───────────────────────────── user nginx; worker_processes auto; worker_rlimit_nofile 65535; pid /run/nginx.pid; error_log /var/log/nginx/error.log warn; load_module modules/ngx_http_brotli_filter_module.so; events { worker_connections 16384; multi_accept on; use epoll; } http { # ── L7 HTTP ── include mime.types; default_type application/octet-stream; upstream app { server 10.0.1.10:8080; } server { # virtual host listen 443 ssl; server_name api.example.com; location /v1/ { # URI-scoped policy if ($request_method = POST) { ... } # avoid — see gotchas } } } stream { # ── L4 TCP/UDP ── upstream pg { server 10.0.2.5:5432; } server { listen 5432; proxy_pass pg; } }
Inheritance rules — the part people get wrong
| Directive type | Rule | Examples |
|---|---|---|
| Scalar | Child overrides parent; otherwise inherited. | proxy_read_timeout, client_max_body_size, root |
| Array | Child replaces the entire array if it defines even one entry. No merging. | add_header, proxy_set_header, fastcgi_param, set_real_ip_from, limit_req |
| Action | Not inherited at all — only applies where declared. | rewrite, return, try_files, proxy_pass |
server { add_header Strict-Transport-Security "max-age=31536000" always; add_header X-Frame-Options DENY always; location /api/ { add_header X-API-Version "2"; ← HSTS AND X-Frame-Options ARE NOW GONE for /api/ } }
Fix: repeat all headers in the child, or centralise them in an include snippets/security-headers.conf; that you include in every location, or use more_set_headers from headers_more which merges.
Variables
| Variable | Meaning / gotcha |
|---|---|
$uri | Normalized, decoded URI after internal rewrites. No query string. |
$request_uri | Original, raw, undecoded URI including query string. Use for logging/redirect fidelity. |
$args / $arg_name | Query string / a specific query param. |
$host | Host header lowercased, or server_name if absent. $http_host is the raw header. |
$remote_addr | Peer IP — rewritten by realip module if configured. |
$proxy_add_x_forwarded_for | $http_x_forwarded_for, $remote_addr — appends, doesn't replace. Spoofable if you don't sanitize. |
$request_time | Total time from first byte read to last byte written — includes client upload & slow-client download. |
$upstream_response_time | Time upstream took. Comma/colon-separated list if retries or multiple upstreams occurred. |
$upstream_connect_time | TCP+TLS setup to upstream. 0.000 when a keepalive conn was reused — great signal for pool health. |
$upstream_cache_status | MISS · HIT · EXPIRED · STALE · UPDATING · REVALIDATED · BYPASS |
$request_id | Auto-generated 32-hex-char ID. Propagate it as your trace correlation ID. |
$connection / $connection_requests | Connection serial number / how many requests rode that keep-alive connection. |
$ssl_protocol, $ssl_cipher, $ssl_session_reused | TLS posture per request — log these to prove TLS-version migration progress. |
$request_time - $upstream_response_time is your client-side latency. If that gap is large and upstream time is flat, the problem is the network/client (mobile users, slow uploads), not your app. This single derived metric ends a lot of "the API is slow" arguments.
04server & location matching
Step 1 — choosing the server block
- Filter by
listen— most specific IP:port match wins (10.0.0.1:80beats*:80). - Among those, match
server_nameagainst theHostheader, in this precedence:- Exact:
api.example.com - Leading wildcard, longest first:
*.example.com - Trailing wildcard:
www.example.* - Regex, first match in config order:
~^(?<sub>.+)\.example\.com$
- Exact:
- No match → the
default_serverfor that listen, else the first server block defined.
server { # catch-all: unknown Host, direct IP hits, scanners listen 80 default_server; listen 443 ssl default_server; ssl_reject_handshake on; # 1.19.4+: refuse TLS for unknown SNI server_name _; return 444; # nginx-special: close connection, no response }
Without this, an attacker sending Host: internal-admin can land on whichever server block happens to be first — a classic Host-header routing bypass.
Step 2 — choosing the location
| Order | Syntax | Semantics |
|---|---|---|
| 1 | location = /exact | Exact match. If it hits, search stops immediately. Fastest — use for /healthz, /favicon.ico. |
| 2 | location ^~ /prefix/ | Prefix match that, if it's the longest prefix, skips regex evaluation entirely. |
| 3 | location /prefix/ | Plain prefix. Longest match is remembered but regexes get a chance to override. |
| 4 | location ~ regex / ~* regex | Case-sensitive / insensitive regex. First match in file order wins — order matters, length does not. |
| 5 | fallback | The longest stored prefix match from step 3. |
# Worked example — request: GET /images/logo.png location / { ... } # prefix, len 1 → remembered location /images/ { ... } # prefix, len 8 → longest, remembered location ~* \.(png|jpg)$ { ... } # regex → EVALUATED, MATCHES → WINS # Now add ^~ and the regex never runs: location ^~ /images/ { ... } # → WINS, regexes skipped
Regex locations are evaluated linearly per request. On a config with 200 regex locations you're doing up to 200 PCRE runs per request. Restructure to prefix locations with ^~, or put the discriminator into map (hash lookup, O(1)).
root vs alias — the eternal confusion
# root: location path is APPENDED to root location /static/ { root /var/www; # /static/a.js → /var/www/static/a.js } # alias: location path is REPLACED by alias location /static/ { alias /var/www/assets/; # /static/a.js → /var/www/assets/a.js }
If the location lacks a trailing slash but the alias has one — location /static { alias /var/www/assets/; } — then GET /static../etc/passwd resolves to /var/www/assets../etc/passwd → /var/www/etc/passwd. Always keep trailing slashes symmetric between location and alias. This is a real, repeatedly-exploited CVE class.
map — the right tool for conditional logic
# map builds a hash table, evaluated lazily, no "if" needed map $http_user_agent $is_bot { default 0; ~*(googlebot|bingbot|crawler) 1; } map $request_method $limit_key { default ""; # empty key = limit not applied POST $binary_remote_addr; PUT $binary_remote_addr; } map $http_upgrade $connection_upgrade { # canonical websocket map default upgrade; '' close; } # Multi-input decisions: chain maps or use a composite key map "$host:$uri" $backend { default app_default; ~^api\. api_pool; }
Inside location{}, only return and rewrite ... last are guaranteed safe inside if. Anything else produces undefined behaviour because if creates an implicit nested location with its own config context. Notorious example:
location /x { set $a 1; if ($http_x) { set $a 2; } # creates nested ctx proxy_pass http://$a; # may see the WRONG value / segfault paths }
Replace with map, or try_files, or move the branch to the upstream. Say "I use map for value selection and if only for return" and you've signalled seniority.
05Static content serving
server { root /var/www/site; # ── zero-copy path: file → socket without userspace ── sendfile on; tcp_nopush on; # with sendfile: fill full packets before sending (TCP_CORK) tcp_nodelay on; # disable Nagle on keepalive conns → low latency for small responses # ── cache open file descriptors + stat() results ── open_file_cache max=200000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; # cache 404s too location ^~ /assets/ { expires 1y; add_header Cache-Control "public, immutable"; access_log off; gzip_static on; # serve pre-built app.js.gz if present brotli_static on; # serve pre-built app.js.br if present } # ── SPA fallback ── location / { try_files $uri $uri/ /index.html; } location = /index.html { add_header Cache-Control "no-cache"; # revalidate the shell every load } }
Compression
gzip on; gzip_comp_level 5; # 5 is the knee; 9 costs 3× CPU for ~2% size gzip_min_length 256; # below MTU, compression is a net loss gzip_proxied any; gzip_vary on; # MUST — emits "Vary: Accept-Encoding" for correct CDN caching gzip_types text/plain text/css application/json application/javascript application/xml image/svg+xml application/wasm; # NOTE: text/html is ALWAYS compressed and must not be listed brotli on; # dynamic module; ~15-20% better than gzip for text brotli_comp_level 4;
Compressing already-compressed formats (jpg, png, mp4, zip, woff2) burns CPU for zero gain. And historically, compressing responses that mix secrets with attacker-controlled input enables BREACH/CRIME-style attacks — mitigate with CSRF tokens that change per request, or disable compression on sensitive JSON endpoints.
Byte-range & large files
location /downloads/ { aio threads; sendfile on; sendfile_max_chunk 2m; # prevents one huge file monopolising a worker max_ranges 1; # mitigate multi-range amplification DoS limit_rate_after 10m; # first 10MB at full speed… limit_rate 2m; # …then throttle to 2MB/s per connection }
06Reverse proxy deep dive
The single most-asked config detail: proxy_pass trailing slash
| location | proxy_pass | Request /api/users/1 becomes |
|---|---|---|
/api/ | http://backend (no URI) | /api/users/1 — full original path passed |
/api/ | http://backend/ (URI = "/") | /users/1 — the matched prefix is stripped |
/api/ | http://backend/v2/ | /v2/users/1 — prefix replaced |
/api/ | http://backend/v2 (no trailing /) | /v2users/1 — concatenated, almost always a bug |
If proxy_pass contains any URI part (even just /), the matched location prefix is replaced by it. If it contains only scheme+host, the original URI passes through untouched.
Two extra traps: (1) a proxy_pass with a URI part is illegal inside a regex location or a named location — you must use rewrite instead. (2) Using a variable in proxy_pass (proxy_pass http://$backend;) changes behaviour: it disables startup-time resolution, requires a resolver, and drops the URI unless you append one explicitly.
A production-grade proxy block, annotated
location /api/ { proxy_pass http://api_pool; # ── protocol ──────────────────────────────────────────── proxy_http_version 1.1; # REQUIRED for keepalive + chunked + websockets proxy_set_header Connection ""; # clear inherited "close" so pooling works # ── identity headers ──────────────────────────────────── proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Request-ID $request_id; # trace correlation # ── timeouts (tune per endpoint, not globally) ────────── proxy_connect_timeout 2s; # TCP handshake only — keep SHORT proxy_send_timeout 30s; # between successive writes to upstream proxy_read_timeout 30s; # between successive reads — NOT total time # ── buffering ─────────────────────────────────────────── proxy_buffering on; proxy_buffer_size 8k; # must hold the full response HEADER proxy_buffers 16 8k; # body buffers, per connection proxy_busy_buffers_size 16k; proxy_max_temp_file_size 256m; # 0 = never spill to disk # ── request body ──────────────────────────────────────── client_max_body_size 25m; # else 413 client_body_buffer_size 256k; proxy_request_buffering on; # off = stream uploads straight through # ── failover ──────────────────────────────────────────── proxy_next_upstream error timeout http_502 http_503 http_504; proxy_next_upstream_tries 2; proxy_next_upstream_timeout 5s; # ── hide leaky upstream headers ───────────────────────── proxy_hide_header X-Powered-By; proxy_hide_header Server; }
Buffering: the concept interviewers dig into
# Streaming endpoint (SSE / LLM tokens) location /stream { proxy_pass http://app; proxy_buffering off; proxy_cache off; proxy_read_timeout 1h; chunked_transfer_encoding on; add_header X-Accel-Buffering no; # upstream can also send this header to opt out }
X-Accel-Buffering: no trickYour upstream can disable buffering per response by emitting this header — so you keep buffering on globally and let streaming endpoints opt out. Cleaner than maintaining a list of streaming paths in nginx config.
A 502 with upstream sent too big header while reading response header from upstream in error.log means the response headers exceeded proxy_buffer_size. Common cause: a huge Set-Cookie (session bloat) or many CORS/debug headers. Fix: raise proxy_buffer_size 16k; and ask why your headers are 8 KB.
Timeouts — the mental model
| Directive | Clock resets when… | Typical value |
|---|---|---|
client_header_timeout | Reading request headers | 10s (Slowloris defence) |
client_body_timeout | Each body read op | 10–30s |
send_timeout | Each write to client | 10–60s |
keepalive_timeout | Idle client connection | 65s (must exceed LB idle timeout) |
proxy_connect_timeout | N/A — hard cap, max 75s | 1–3s |
proxy_read_timeout | Each read from upstream | Per-endpoint: 5s API, 300s report |
proxy_read_timeout 30s does not cap a request at 30s. If the upstream dribbles one byte every 29 seconds, the request runs forever. For a true deadline you need the app to enforce it, or use limit_rate/Lua timers. Interviewers love this one.
If your AWS ALB/NLB idle timeout is 60s and NGINX keepalive_timeout is 60s, you get a race: NGINX closes a pooled connection at the exact moment the LB sends a request on it → intermittent 502s at low traffic. Rule: NGINX keepalive_timeout > LB idle timeout at every hop, and the same going upstream (NGINX's upstream keepalive_timeout must be shorter than the app server's idle timeout).
Non-HTTP upstreams
location ~ \.php$ { include fastcgi_params; fastcgi_pass unix:/run/php-fpm.sock; # unix socket beats TCP on localhost fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_split_path_info ^(.+\.php)(/.+)$; fastcgi_buffers 16 16k; } location /grpc/ { grpc_pass grpc://grpc_pool; } location /uwsgi/ { include uwsgi_params; uwsgi_pass unix:/run/uwsgi.sock; } location /scgi/ { include scgi_params; scgi_pass 127.0.0.1:9000; }
DNS resolution — a genuine production hazard
# PROBLEM: this resolves ONCE at startup/reload and caches forever. # If the upstream is an ELB/ClusterIP whose IP rotates → traffic to a dead IP. upstream api { server api.internal.example.com:443; } # FIX A (OSS): force re-resolution via a variable + resolver resolver 10.0.0.2 valid=10s ipv6=off; location /api/ { set $api_host api.internal.example.com; proxy_pass https://$api_host$request_uri; # note: must re-append URI } # FIX B (Plus): resolve=… on the upstream server, keeps pooling + LB upstream api { zone api 64k; server api.internal.example.com:443 resolve; keepalive 64; }
Fix A costs you the upstream block — no load-balancing policy, no keepalive pool, no passive health tracking; you're leaning on DNS round-robin. The clean answer at staff level is: don't resolve DNS in NGINX at all. Use service discovery (Consul-template, k8s Endpoints controller, or the NGINX Plus API) to render/patch the upstream block, so NGINX always has concrete IPs.
07Load balancing
upstream api_pool { zone api_pool 64k; # shared memory → state shared ACROSS WORKERS least_conn; # algorithm (declare BEFORE servers) server 10.0.1.10:8080 weight=3 max_fails=3 fail_timeout=10s max_conns=200; server 10.0.1.11:8080 weight=1; server 10.0.1.12:8080 backup; # only used when all primaries are down server 10.0.1.13:8080 down; # admin-disabled, kept for hash stability server 10.0.1.14:8080 slow_start=30s; /* PLUS: ramp weight after recovery */ keepalive 64; # idle conns kept PER WORKER keepalive_requests 1000; keepalive_timeout 60s; }
zone, everything is per-workerNo zone ⇒ each worker keeps its own copy of upstream state. With 16 workers and max_conns=10, you actually allow 160 concurrent connections to that backend. Same for max_fails — a backend must fail 3× per worker to be fully ejected. Always declare zone in OSS 1.9.0+.
Algorithms
| Directive | How it picks | Use when | Watch out for |
|---|---|---|---|
| (default) round robin | Weighted RR | Homogeneous, short, uniform requests | Ignores actual load; one slow node still gets its share |
least_conn | Fewest active connections (weight-adjusted) | Variable request durations — the common real-world case | A node that fails instantly shows 0 conns and attracts all traffic ("black hole"). Pair with passive health checks. |
ip_hash | Hash of first 3 octets of IPv4 | Legacy sticky sessions | NAT/CGNAT ⇒ huge groups land on one node. Adding a node reshuffles nearly everything. |
hash $key | Hash of an arbitrary key | Sharding by tenant/user/URL | Same reshuffle problem without consistent |
hash $key consistent | Ketama ring | Cache-tier fan-out | Only ~1/N keys move when a node changes — this is the answer for cache affinity |
random two least_conn | Pick 2 at random, take the less loaded | Many NGINX instances in front of shared backends | The "power of two choices" — avoids the herd effect where every LB independently picks the same "least loaded" node |
least_time PLUS | Lowest avg response time × conns | Heterogeneous hardware | Plus only |
random two existsWith one LB, least_conn is optimal. With 50 NGINX instances each independently computing "least loaded", they all pick the same idle backend simultaneously and hammer it — then all switch away together. Oscillation. Power-of-two-choices injects randomness that provably keeps max load within O(log log n) of average while eliminating the herd. Same reasoning as Envoy's default LB policy.
Upstream keepalive — the biggest single latency win
upstream app { server ...; keepalive 64; } location / { proxy_pass http://app; proxy_http_version 1.1; ← without this, HTTP/1.0, no keepalive proxy_set_header Connection ""; ← without this, "Connection: close" is forwarded }
Missing either line silently disables pooling. Symptom: $upstream_connect_time is non-zero on every request, and you see TIME_WAIT socket buildup on the NGINX box (ss -s). A new TCP+TLS handshake per request adds 1–3 RTTs — often 5–40 ms of pure waste.
keepalive is per worker. With worker_processes 16; keepalive 64; you may hold up to 1024 idle connections to that upstream from one NGINX box. Multiply by the number of NGINX boxes and check it against the backend's max connections / file descriptor limit. Rule of thumb: keepalive ≈ 2 × (peak RPS × avg upstream latency) / worker_processes.
Session persistence
# OSS: hash on an app-provided cookie or header upstream app { hash $cookie_sessionid consistent; server ...; } # PLUS: nginx issues and tracks its own cookie upstream app { sticky cookie srv_id expires=1h domain=.example.com path=/ httponly secure; }
"Sticky sessions are a smell — they make deploys, autoscaling, and node failure user-visible. My default is stateless backends with session state in Redis/JWT. I'd only use stickiness for a legacy app I can't refactor, for local in-process caches where affinity is a performance optimization (not a correctness requirement), or for WebSocket connections that are inherently pinned."
08Health checks & resilience
Passive (OSS)
upstream app { zone app 64k; # 3 failures within a 10s window → mark down for 10s, then try ONE probe request server 10.0.1.10:8080 max_fails=3 fail_timeout=10s; } # What counts as a "failure" is defined by: proxy_next_upstream error timeout http_502 http_503 http_504;
Note the coupling: fail_timeout is both the window in which failures are counted and the ejection duration. And recovery is "try one live request" — real users pay for the probe.
Active (Plus / Angie)
location / { proxy_pass http://app; health_check interval=5s fails=3 passes=2 uri=/healthz match=ok; } match ok { status 200; header Content-Type ~ application/json; body ~ '"status":\s*"ok"'; }
- Liveness ≠ readiness. Liveness = "is the process alive" (restart me if not). Readiness = "should I get traffic" (drain me if not). NGINX should probe readiness.
- Don't make health checks deep. If
/healthzchecks the database, a single DB blip makes every node report unhealthy simultaneously → NGINX ejects the entire pool → total outage, when degraded service was possible. Check only what this instance controls. - Graceful shutdown sequence: on SIGTERM the app should (1) flip readiness to fail, (2) keep serving in-flight requests for
drain_time> probe interval × failure threshold, (3) then exit. Without step 2 you drop requests on every deploy. - Retry safety.
proxy_next_upstreamdefaults includeerror timeout. A timeout on aPOST /chargemay have already succeeded upstream — retrying double-charges. Setproxy_next_upstream error timeout non_idempotent;only if you mean it, and prefer idempotency keys in the app. - Retry amplification. With
proxy_next_upstream_tries 3, a backend brownout turns 1× traffic into 3× traffic exactly when the tier is already struggling. Cap tries at 2, addproxy_next_upstream_timeout, and rely on the client for anything more.
Circuit-breaker-ish behaviour
upstream app { zone app 64k; server a:8080 max_conns=100 max_fails=2 fail_timeout=15s; server b:8080 max_conns=100 max_fails=2 fail_timeout=15s; queue 100 timeout=3s; /* PLUS: queue instead of instant 502 when max_conns is hit */ }
max_conns is your bulkhead: it caps concurrency per backend so one slow endpoint can't consume the whole app's thread pool. In OSS without queue, exceeding it returns 502 immediately — so pair it with a friendly error_page.
Graceful degradation
server { proxy_intercept_errors on; error_page 500 502 503 504 /maintenance.html; location = /maintenance.html { root /var/www/errors; internal; add_header Retry-After 30 always; } # Serve stale cache instead of an error — best-in-class degradation location / { proxy_cache zone1; proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; proxy_cache_background_update on; proxy_pass http://app; } }
09Caching
# ── http{} context ────────────────────────────────────────── proxy_cache_path /var/cache/nginx/api levels=1:2 # dir fan-out: /a/bc/<md5> — avoids huge dirs keys_zone=api_cache:100m # ~800k keys (≈8k keys per MB) max_size=20g # disk ceiling (soft; manager enforces) inactive=60m # evict if UNREQUESTED this long (independent of TTL) use_temp_path=off; # write in place, avoid cross-device copy location /api/ { proxy_cache api_cache; proxy_cache_key "$scheme$request_method$host$request_uri"; proxy_cache_methods GET HEAD; # ── TTLs per status ── proxy_cache_valid 200 301 302 10m; proxy_cache_valid 404 1m; # negative caching stops 404 storms proxy_cache_valid any 1m; # ── stampede control ── proxy_cache_lock on; # only ONE request populates a cold key proxy_cache_lock_timeout 5s; proxy_cache_lock_age 5s; proxy_cache_min_uses 2; # don't cache one-hit-wonders (protects disk) # ── stale serving = availability ── proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; proxy_cache_background_update on; # serve stale NOW, refresh async proxy_cache_revalidate on; # conditional GET w/ If-Modified-Since / ETag # ── bypass rules ── proxy_cache_bypass $http_authorization $cookie_session $arg_nocache; proxy_cache_no_cache $http_authorization $cookie_session; # ── ignore upstream cache-busting headers (use with care) ── proxy_ignore_headers Cache-Control Expires Set-Cookie X-Accel-Expires; add_header X-Cache-Status $upstream_cache_status always; proxy_pass http://api_pool; }
proxy_cache_bypass vs proxy_cache_no_cache
| Directive | Effect when the variable is non-empty and ≠ "0" |
|---|---|
proxy_cache_bypass | Don't read from cache (go to upstream) — but the response may still be stored. |
proxy_cache_no_cache | Don't write to cache — but a cached copy may still be served. |
You almost always want both for authenticated traffic. Setting only bypass is a well-known cause of cache poisoning where one user's private response gets served to everyone.
- Omit
$hostfrom the key on a multi-tenant proxy → tenant A sees tenant B's data. - Include
$http_cookie→ cardinality explosion, ~0% hit rate. - Ignore
Vary: Accept-Encoding→ gzip body sent to a client that didn't ask for it (garbled page). - Query-param order (
?a=1&b=2vs?b=2&a=1) creates two keys. Normalize withmap/Lua or accept the miss. - Unkeyed attacker-controlled headers that the upstream reflects (e.g.
X-Forwarded-Host) → web cache poisoning. Either key on them or strip them at the edge.
Microcaching — the highest-ROI trick
# Cache even "dynamic, personalized" pages for ONE second. # 5,000 rps on a hot endpoint → 1 rps hits your app. 99.98% offload. location / { proxy_cache micro; proxy_cache_valid 200 1s; proxy_cache_lock on; proxy_cache_use_stale updating error timeout; proxy_cache_background_update on; proxy_pass http://app; }
The insight: for anonymous, read-heavy traffic, one second of staleness is invisible to users but collapses a traffic spike by three orders of magnitude. This is the standard answer to "how would you survive a front-page traffic spike without scaling the app tier?"
Cache stampede / thundering herd
Purging
# PLUS: native location ~ /purge(/.*) { allow 10.0.0.0/8; deny all; proxy_cache_purge api_cache "$scheme$request_method$host$1"; } # OSS options: # a) ngx_cache_purge third-party module # b) compute md5 of the cache key and rm the file: # echo -n "httpsGETexample.com/path" | md5sum # → /var/cache/nginx/api/<last-char>/<2 chars before>/<md5> # c) versioned cache keys — bump a $cache_version prefix on deploy (cleanest)
"I avoid purge entirely. I put a version token in the cache key (build SHA, or a per-tenant epoch stored in a map) so a 'purge' is a key-namespace rotation — instantaneous, atomic, and works across an arbitrary fleet of cache nodes with no fan-out API call. Old entries age out via inactive."
10TLS, HTTP/2 and HTTP/3
server { listen 443 ssl reuseport; listen [::]:443 ssl; http2 on; # 1.25.1+ (was `listen … http2`) listen 443 quic reuseport; # HTTP/3 over QUIC (UDP), 1.25.0+ add_header Alt-Svc 'h3=":443"; ma=86400'; # advertise h3 to clients ssl_certificate /etc/ssl/fullchain.pem; # leaf + intermediates, in order ssl_certificate_key /etc/ssl/privkey.pem; # ── protocol & ciphers ── ssl_protocols TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers off; # modern advice: trust client order (TLS1.3) ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256: ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384; ssl_ecdh_curve X25519:prime256v1; # ── session resumption (huge CPU + RTT saver) ── ssl_session_cache shared:SSL:50m; # ~200k sessions; SHARED across workers ssl_session_timeout 1d; ssl_session_tickets on; ssl_session_ticket_key /etc/ssl/ticket1.key; # rotate; share across the fleet ssl_session_ticket_key /etc/ssl/ticket2.key; # older key, decrypt-only # ── OCSP stapling: server fetches revocation proof so clients don't have to ── ssl_stapling on; ssl_stapling_verify on; ssl_trusted_certificate /etc/ssl/chain.pem; resolver 1.1.1.1 8.8.8.8 valid=300s; # ── TLS 1.3 0-RTT: 1 fewer RTT, but REPLAYABLE ── ssl_early_data on; proxy_set_header Early-Data $ssl_early_data; # app must reject non-idempotent 0-RTT add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always; }
Handshake cost — why termination is worth it
Ticket keys. If you don't set ssl_session_ticket_key, each NGINX instance generates its own random key and never rotates it while it runs. Consequence: (a) no resumption across instances behind an LB → handshake storms; (b) a long-lived key that, if stolen, decrypts all past traffic. Fix: distribute rotating key files (e.g. hourly, keep 2–3 generations) via your secrets system.
0-RTT replay. An attacker can capture and replay early data. Never let a 0-RTT request perform a state-changing action — gate on the Early-Data: 1 header in the app and return 425 Too Early for non-idempotent methods.
Multiple certs & SNI
# Dual RSA + ECDSA on one server block — client negotiates the best it supports ssl_certificate /etc/ssl/ecdsa.crt; ssl_certificate_key /etc/ssl/ecdsa.key; ssl_certificate /etc/ssl/rsa.crt; ssl_certificate_key /etc/ssl/rsa.key; # Cert selection by SNI variable (avoids thousands of server blocks) ssl_certificate /etc/ssl/certs/$ssl_server_name.crt; ssl_certificate_key /etc/ssl/certs/$ssl_server_name.key; # (variables in ssl_certificate disable the cert cache → use ssl_certificate_cache in 1.27.4+)
mTLS (client certificates)
ssl_client_certificate /etc/ssl/ca.pem; ssl_verify_client on; # or `optional` to allow both and branch ssl_verify_depth 2; ssl_crl /etc/ssl/revoked.crl; proxy_set_header X-Client-DN $ssl_client_s_dn; proxy_set_header X-Client-Verify $ssl_client_verify; # SUCCESS | FAILED:reason | NONE proxy_set_header X-Client-Serial $ssl_client_serial;
HTTP/2 & HTTP/3 notes worth stating
| Aspect | HTTP/1.1 | HTTP/2 | HTTP/3 (QUIC) |
|---|---|---|---|
| Transport | TCP | TCP | UDP |
| Multiplexing | None (6 conns/host) | Streams over 1 conn | Streams over 1 conn |
| Head-of-line blocking | App layer | Still at TCP layer — one lost packet stalls all streams | Eliminated (per-stream loss recovery) |
| Header compression | None | HPACK | QPACK |
| Handshake | TCP+TLS: 2–3 RTT | 2–3 RTT | 1 RTT, or 0-RTT resumed |
| Conn migration | No | No | Yes (Connection ID survives IP change — wifi→cellular) |
1. NGINX speaks HTTP/2 to clients, but by default proxies to upstreams over HTTP/1.1. That's usually fine and actually desirable (connection pooling). Use grpc_pass when you need real h2 upstream.
2. Domain sharding, sprite sheets, and inlining are anti-patterns under HTTP/2 — they defeat multiplexing and caching. If you migrate to h2 without removing sharding you can get slower.
# HTTP/2 tuning http2_max_concurrent_streams 128; keepalive_requests 1000; # h2 counts each STREAM as a request — raise this # CVE-2023-44487 "Rapid Reset": mitigated in 1.25.3; also cap streams + keepalive_time
11Rate limiting & traffic shaping
Three independent limiters
# ── http{} ───────────────────────────────────────────────── # 1. REQUEST RATE — leaky bucket, per key limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s; limit_req_zone $server_name zone=persrv:10m rate=1000r/s; limit_req_zone $http_x_api_key zone=perkey:20m rate=100r/s; # 2. CONCURRENT CONNECTIONS limit_conn_zone $binary_remote_addr zone=conn_perip:10m; # 3. BANDWIDTH — per connection # (limit_rate / limit_rate_after, set in location or via $limit_rate) limit_req_status 429; # default is 503 — 429 is semantically correct limit_conn_status 429; limit_req_log_level warn; location /api/ { limit_req zone=perip burst=20 nodelay; limit_req zone=persrv burst=200; # multiple limits ALL apply; strictest wins limit_conn conn_perip 20; limit_rate 1m; }
How the leaky bucket actually behaves
$binary_remote_addrnot$remote_addr— 4 bytes vs 15, ~4× more entries per MB of zone.- Behind a CDN/LB,
$remote_addris the proxy. You must configurereal_ipfirst, or key on$http_cf_connecting_ip/ the correct XFF position. - For APIs, key on the API key or tenant ID, not IP — IP-based limits punish shared NAT and don't stop distributed abuse.
- Use
mapto produce an empty key to exempt traffic (empty key ⇒ limit not applied). This is how you whitelist internal ranges.
# Whitelist internal networks + trusted partners from the limiter geo $limit_exempt { default 0; 10.0.0.0/8 1; 192.168.0.0/16 1; } map $limit_exempt $limit_key { 0 $binary_remote_addr; # limited 1 ""; # exempt } limit_req_zone $limit_key zone=perip:10m rate=10r/s;
Tiered limits by plan
map $http_x_api_key $plan { default free; ~^pk_pro_ pro; ~^pk_ent_ ent; } map $plan $free_key { free $http_x_api_key; default ""; } map $plan $pro_key { pro $http_x_api_key; default ""; } limit_req_zone $free_key zone=free:10m rate=10r/s; limit_req_zone $pro_key zone=pro:10m rate=500r/s; location /api/ { limit_req zone=free burst=20 nodelay; limit_req zone=pro burst=1000 nodelay; }
NGINX's limiter is per-instance. With 10 edge nodes and rate=10r/s, a client can actually do ~100 r/s. Options: (a) divide the limit by fleet size and accept imprecision, (b) put the limiter behind a consistent-hash L4 layer so a given key always lands on the same NGINX node, (c) use a shared counter (Redis via OpenResty / Envoy's global ratelimit service) — at the cost of a network round trip per request, (d) accept NGINX as coarse DoS protection and enforce exact quotas in the application where you already have the tenant context.
Also note NGINX returns 429 with no Retry-After or X-RateLimit-* headers by default. Add them: add_header Retry-After 1 always; — API consumers need them.
12Security hardening & WAF
Baseline hardening
server_tokens off; # hide version in Server header + error pages more_clear_headers Server; # headers_more: remove it entirely # Body / header size caps (DoS surface) client_max_body_size 10m; client_body_buffer_size 128k; client_header_buffer_size 1k; large_client_header_buffers 4 8k; client_body_timeout 10s; client_header_timeout 10s; # Method allow-list if ($request_method !~ ^(GET|HEAD|POST|PUT|PATCH|DELETE|OPTIONS)$) { return 405; } # Block hidden files, backups, VCS location ~ /\.(?!well-known) { deny all; access_log off; } location ~* \.(bak|old|sql|env|git|swp)$ { deny all; } # Security headers — repeat in EVERY location that sets any add_header add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always; add_header X-Content-Type-Options "nosniff" always; add_header X-Frame-Options "DENY" always; add_header Referrer-Policy "strict-origin-when-cross-origin" always; add_header Content-Security-Policy "default-src 'self'; frame-ancestors 'none'" always; add_header Permissions-Policy "geolocation=(), camera=(), microphone=()" always;
real_ip — get this wrong and every IP-based control is bypassable
set_real_ip_from 10.0.0.0/8; # ONLY list proxies you actually control set_real_ip_from 173.245.48.0/20; # e.g. Cloudflare ranges real_ip_header X-Forwarded-For; real_ip_recursive on; # walk XFF right-to-left, skipping trusted IPs
If you trust 0.0.0.0/0 or blindly read the leftmost XFF entry, any client can send X-Forwarded-For: 10.0.0.1 and impersonate an internal address — defeating allow/deny, rate limits, geo rules, and audit logs. Correct approach:
- Trust only your known proxy CIDRs in
set_real_ip_from. - Enable
real_ip_recursive onso NGINX walks from the right, discarding trusted hops, and stops at the first untrusted address — that's the real client. - At the outermost edge, overwrite rather than append:
proxy_set_header X-Forwarded-For $remote_addr;so client-supplied values are discarded. - Better still, standardise on RFC 7239
Forwarded, or a signed internal header.
Access control
location /admin/ { allow 10.0.0.0/8; allow 2001:db8::/32; deny all; auth_basic "restricted"; auth_basic_user_file /etc/nginx/.htpasswd; satisfy all; # require BOTH ip and password (default). `any` = either. }
WAF options
| Option | Notes |
|---|---|
| ModSecurity 3 + OWASP CRS | The classic. Runs as an NGINX dynamic module. Powerful but CPU-heavy and false-positive-prone; expect a long tuning period in DetectionOnly mode first. |
| NGINX App Protect PLUS | F5's commercial WAF, signature + behavioural, lower overhead than ModSec. |
| Coraza | Go reimplementation of ModSecurity's rule engine; used via a connector or in Envoy/Caddy. |
| CDN-layer WAF | Cloudflare/AWS WAF. Usually the right answer — blocks upstream of your bandwidth and your CPU. |
load_module modules/ngx_http_modsecurity_module.so; server { modsecurity on; modsecurity_rules_file /etc/nginx/modsec/main.conf; }
"A WAF is a compensating control, not a fix. I run it in detection mode first, tune out false positives against real traffic, and treat any rule that fires in production as a bug report for the application. I'd rather spend the CPU on rate limiting and bot detection at the CDN and fix injection at the ORM layer."
Slowloris and friends
| Attack | Mechanism | NGINX mitigation |
|---|---|---|
| Slowloris | Many connections, headers sent 1 byte at a time | client_header_timeout 10s, limit_conn. NGINX is inherently resilient (cheap connections) but not immune. |
| Slow POST (R-U-Dead-Yet) | Body dribbled slowly | client_body_timeout, client_max_body_size, proxy_request_buffering on |
| Slow read | Client advertises tiny TCP window | send_timeout, proxy_buffering on, limit_conn |
| HTTP/2 Rapid Reset | Open+RST_STREAM flood, unbounded work | Upgrade to ≥1.25.3, http2_max_concurrent_streams, keepalive_requests |
| Range amplification | Range: bytes=0-,0-,0-… thousands of times | max_ranges 1; |
| Request smuggling | CL/TE desync between proxy and backend | Keep NGINX current; ensure backend rejects duplicate/conflicting Content-Length/Transfer-Encoding; avoid mixing proxies with different parsers |
13Auth at the edge
auth_request — delegate authz to a service
server { location /api/ { auth_request /_authz; # pull identity out of the auth response and pass it downstream auth_request_set $user_id $upstream_http_x_user_id; auth_request_set $scopes $upstream_http_x_scopes; proxy_set_header X-User-Id $user_id; proxy_set_header X-Scopes $scopes; proxy_pass http://api_pool; } location = /_authz { internal; proxy_pass http://authz_service; proxy_pass_request_body off; # auth svc doesn't need the body proxy_set_header Content-Length ""; proxy_set_header X-Original-URI $request_uri; proxy_set_header X-Original-Method $request_method; proxy_set_header Authorization $http_authorization; } error_page 401 = @login; location @login { return 302 https://auth.example.com/login?rd=$scheme://$host$request_uri; } }
Semantics: 2xx from the subrequest → allow. 401/403 → deny with that status. Anything else → 500.
Every request now costs an extra internal HTTP round trip. Mitigate with: keepalive to the authz service, a short-TTL cache of authz decisions keyed on the token (proxy_cache on /_authz with a 5–30s TTL), or moving to stateless JWT verification so no subrequest is needed at all.
JWT validation
/* NGINX PLUS — native */ location /api/ { auth_jwt "api"; auth_jwt_key_file /etc/nginx/jwks.json; # or auth_jwt_key_request for JWKS URL proxy_set_header X-Sub $jwt_claim_sub; proxy_pass http://api; }
# OSS — njs or OpenResty js_import auth.js; location /api/ { auth_request /_jwt; proxy_pass http://api; } location = /_jwt { internal; js_content auth.verify; }
CORS done correctly
map $http_origin $cors_origin { default ""; ~^https://(app|admin)\.example\.com$ $http_origin; # echo only allowed origins } location /api/ { if ($request_method = OPTIONS) { add_header Access-Control-Allow-Origin $cors_origin always; add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS" always; add_header Access-Control-Allow-Headers "Authorization, Content-Type" always; add_header Access-Control-Allow-Credentials "true" always; add_header Access-Control-Max-Age 86400 always; add_header Content-Length 0; return 204; } add_header Access-Control-Allow-Origin $cors_origin always; add_header Access-Control-Allow-Credentials "true" always; add_header Vary Origin always; ← REQUIRED or caches will serve the wrong ACAO proxy_pass http://api; }
Access-Control-Allow-Origin: * together with Allow-Credentials: true is rejected by browsers — you must echo a specific origin. And if you echo $http_origin without a Vary: Origin, any cache (NGINX's own, or a CDN) will serve origin A's ACAO header to origin B.
14Performance tuning
Capacity arithmetic
worker_processes auto; worker_cpu_affinity auto; # pin workers to cores → better cache locality worker_rlimit_nofile 200000; worker_shutdown_timeout 30s; # cap how long old workers linger after reload events { worker_connections 65535; multi_accept on; # drain the whole accept queue per loop iteration use epoll; } http { sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 75s; keepalive_requests 10000; reset_timedout_connection on; # RST instead of FIN → free memory immediately server_names_hash_bucket_size 128; types_hash_max_size 4096; access_log /var/log/nginx/access.log main buffer=64k flush=5s; open_file_cache max=200000 inactive=20s; }
Kernel tuning (/etc/sysctl.conf)
net.core.somaxconn = 65535 # accept queue depth (listen backlog ceiling) net.core.netdev_max_backlog = 65535 net.ipv4.tcp_max_syn_backlog = 65535 net.ipv4.ip_local_port_range = 1024 65535 # ephemeral ports for upstream conns net.ipv4.tcp_tw_reuse = 1 # reuse TIME_WAIT for OUTGOING conns (safe) net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_slow_start_after_idle = 0 # don't reset cwnd on idle keepalives net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq net.ipv4.tcp_fastopen = 3 fs.file-max = 2097152 # and in nginx: listen 443 ssl backlog=65535; # must be ≤ somaxconn
tcp_tw_recycleIt breaks connections from clients behind NAT (timestamp-based rejection) and was removed in Linux 4.12. tcp_tw_reuse is the safe one, and it only affects outbound connections — which is exactly the NGINX→upstream direction.
Ephemeral port exhaustion
A proxy opening a fresh TCP connection per upstream request burns one ephemeral port per request, each lingering in TIME_WAIT for 60s. With ~28k usable ports that caps you around 470 rps per upstream IP. Symptoms: connect() failed (99: Cannot assign requested address).
- Fix #1 (do this): upstream
keepalivepooling — removes the problem entirely. - Fix #2: widen
ip_local_port_range, enabletcp_tw_reuse. - Fix #3: more upstream IPs/ports — the 4-tuple includes the destination, so N backends give you N× the ports.
- Fix #4: Unix domain sockets for same-host upstreams (php-fpm, sidecars).
TLS CPU
- ECDSA certs are ~4–10× cheaper to sign than RSA-2048. Serve both, let clients pick.
- Session resumption converts a full handshake into a symmetric-key operation — the single biggest TLS win.
- Check
openssl speedand whether AES-NI is available; on ARM/older CPUs preferCHACHA20-POLY1305. ssl_buffer_size 4k;reduces TTFB for small responses (default 16k means the first record waits to fill).
Tuning checklist by symptom
| Symptom | Likely cause | Action |
|---|---|---|
| High CPU, low throughput | TLS handshakes; gzip level too high; regex locations | Resumption, ECDSA, gzip_comp_level 4-5, replace regex with map |
worker_connections are not enough | Slot exhaustion | Raise worker_connections + worker_rlimit_nofile; remember proxying doubles usage |
Too many open files | fd limit | worker_rlimit_nofile, systemd LimitNOFILE |
| Periodic p99 spikes | Blocking disk I/O in a worker; log flush | aio threads, access_log buffer= flush= |
| Disk I/O high on a proxy | Response bodies spilling to proxy_temp_path | Raise proxy_buffers, or set proxy_max_temp_file_size 0 |
Cannot assign requested address | Ephemeral port exhaustion | Upstream keepalive |
| Intermittent 502 at low traffic | Keepalive idle-timeout race | NGINX keepalive_timeout < upstream's idle timeout |
15Logging & observability
Structured access logs
log_format json escape=json '{' '"ts":"$time_iso8601",' '"req_id":"$request_id",' '"remote_addr":"$remote_addr",' '"host":"$host",' '"method":"$request_method",' '"uri":"$request_uri",' '"status":$status,' '"bytes_sent":$body_bytes_sent,' '"request_time":$request_time,' '"upstream_addr":"$upstream_addr",' '"upstream_status":"$upstream_status",' '"upstream_connect_time":"$upstream_connect_time",' '"upstream_header_time":"$upstream_header_time",' '"upstream_response_time":"$upstream_response_time",' '"cache":"$upstream_cache_status",' '"scheme":"$scheme",' '"tls_ver":"$ssl_protocol",' '"tls_cipher":"$ssl_cipher",' '"tls_reused":"$ssl_session_reused",' '"http_ver":"$server_protocol",' '"ua":"$http_user_agent",' '"referer":"$http_referer",' '"conn_reqs":$connection_requests '}'; access_log /var/log/nginx/access.log json buffer=64k flush=5s; access_log syslog:server=10.0.0.5:514,tag=nginx,severity=info json; # sample only 1% of successful requests, log ALL errors map $status $loggable { ~^[23] 0; default 1; } access_log /var/log/nginx/errors.log json if=$loggable;
escape=json is mandatoryWithout it, a request URI containing a quote or backslash produces malformed JSON and your log pipeline silently drops lines — often exactly the malicious requests you most wanted to see.
The four latency variables — how to read them
Metrics
# OSS: minimal built-in stats location = /nginx_status { stub_status; allow 127.0.0.1; allow 10.0.0.0/8; deny all; access_log off; } # → Active connections: 291 # server accepts handled requests # 16630948 16630948 31070465 # Reading: 6 Writing: 179 Waiting: 106 /* PLUS: rich JSON API — per-upstream, per-zone, per-cache */ location /api { api write=on; allow 10.0.0.0/8; deny all; } location = /dashboard.html { root /usr/share/nginx/html; }
- nginx-prometheus-exporter — scrapes
stub_status(or the Plus API) into Prometheus. - nginx-module-vts / Angie — per-server-zone and per-upstream metrics on OSS.
- Log-derived metrics — Vector/Fluent Bit parsing the JSON log into histograms is often the most flexible route (gives you true per-route p99).
Distributed tracing
# Propagate a correlation ID (works with any tracing backend) proxy_set_header X-Request-ID $request_id; add_header X-Request-ID $request_id always; # Native OpenTelemetry module (nginx 1.25.2+ / otel-nginx) load_module modules/ngx_otel_module.so; otel_exporter { endpoint otel-collector:4317; } otel_service_name nginx-edge; otel_trace on; otel_trace_context propagate; # inject/extract W3C traceparent
Golden signals to alert on
| Signal | Source | Alert idea |
|---|---|---|
| 5xx rate | $status | >1% over 5m, split by upstream |
p99 $upstream_response_time | log histogram | Per route, vs 7-day baseline |
| Cache hit ratio | $upstream_cache_status | Sudden drop = key change or purge storm |
| Active connections | stub_status | >70% of worker_connections capacity |
| Waiting/Reading ratio | stub_status | High Reading = slowloris-ish |
| 429/503 rate | logs | Limiter firing on legitimate traffic |
$upstream_connect_time > 0 | logs | Keepalive pool broken after a deploy |
16Zero-downtime reload & binary upgrade
Config reload — SIGHUP
- During overlap you have 2× workers and 2× memory. On a box tuned to 80% RAM, a reload under load can OOM.
- Old workers hang around for the life of their longest connection — WebSockets/SSE can keep them for hours. Set
worker_shutdown_timeout 30s;(accepting that those connections get cut). - The shared-memory zones are re-created:
limit_reqcounters andssl_session_cachereset. Frequent reloads (e.g. a k8s ingress controller reloading on every Endpoint change) effectively disable your rate limiter and cause TLS handshake storms. That's the core argument for Envoy/xDS or the NGINX Plus dynamic API in high-churn environments. proxy_cacheon disk survives, but the cache loader re-walks it.
Binary upgrade (new nginx version, no dropped connections)
kill -USR2 $(cat /run/nginx.pid) # old master renames pid→pid.oldbin, execs NEW binary # both masters now share the listen sockets kill -WINCH $(cat /run/nginx.pid.oldbin) # old workers finish & exit; old MASTER stays # verify new version is healthy, then either: kill -QUIT $(cat /run/nginx.pid.oldbin) # commit: retire old master # or roll back: kill -HUP $(cat /run/nginx.pid.oldbin) # old master respawns its workers kill -QUIT $(cat /run/nginx.pid) # retire the new one
Signal reference
| Signal | To master | To worker |
|---|---|---|
TERM/INT | Fast shutdown — drops connections | Same |
QUIT | Graceful shutdown | Graceful — finish then exit |
HUP | Reload config, graceful worker swap | — |
USR1 | Reopen log files (for logrotate) | Reopen logs |
USR2 | Upgrade binary on the fly | — |
WINCH | Gracefully shut down workers, keep master | — |
Safe deploy pipeline
# 1. render config from templates # 2. syntax + semantic validation in a throwaway container nginx -t -c /tmp/candidate/nginx.conf # 3. diff against running config, require review for security-header changes # 4. canary: reload ONE node, watch 5xx + latency for 5 min # 5. rolling reload across the fleet with a health gate between nodes # 6. automatic rollback: keep last-known-good config, reload it on SLO breach # drain a node before maintenance (health endpoint returns 503 → LB removes it) touch /etc/nginx/drain && sleep 30 && systemctl reload nginx
location = /healthz { access_log off; if (-f /etc/nginx/drain) { return 503 "draining"; } return 200 "ok"; add_header Content-Type text/plain; }
17Debugging playbook
First five commands
nginx -t # syntax check nginx -T # dump the FULLY RESOLVED config (all includes) — invaluable nginx -V # version + compile flags + which modules exist ps -eo pid,ppid,rss,cmd | grep nginx ss -s ; ss -tan state time-wait | wc -l
Turn on debug logging for one client only
# requires --with-debug (check nginx -V) events { debug_connection 203.0.113.42; } # or a CIDR error_log /var/log/nginx/debug.log debug; # This logs every phase transition for that one IP without drowning the box.
Error-log messages → root cause
| Message | Meaning & fix |
|---|---|
connect() failed (111: Connection refused) | Nothing listening upstream. Check the app, the port, and security groups. |
connect() failed (110: Connection timed out) | Packets dropped — firewall/SG/NACL, or upstream accept queue full. |
connect() failed (99: Cannot assign requested address) | Ephemeral port exhaustion. Enable upstream keepalive. |
upstream prematurely closed connection | App crashed mid-response, or the app's idle timeout < NGINX's keepalive_timeout. Classic with Node's 5s default server.keepAliveTimeout. |
upstream timed out (110) while reading response header | App is slow. Raise proxy_read_timeout only after confirming it's not a hang. |
upstream sent too big header | Raise proxy_buffer_size; investigate header bloat. |
no live upstreams while connecting | All backends passively marked down. Check max_fails/fail_timeout and whether the app is actually healthy. |
worker_connections are not enough | Raise worker_connections; remember proxying costs 2 slots. |
could not build server_names_hash | Raise server_names_hash_bucket_size (long/many server_names). |
rewrite or internal redirection cycle | A try_files/error_page loop hit the 10-cycle cap. Usually try_files $uri /index.php where /index.php doesn't exist. |
client intended to send too large body | Raise client_max_body_size. |
SSL_do_handshake() failed | Protocol/cipher mismatch, bad SNI, or a client speaking plaintext to a TLS port. |
Status codes NGINX itself generates
| Code | Who emitted it | Typical cause |
|---|---|---|
| 400 | NGINX | Malformed request, or header size > large_client_header_buffers |
| 408 | NGINX | client_header_timeout/client_body_timeout hit |
| 413 | NGINX | client_max_body_size exceeded |
| 429 | NGINX | limit_req/limit_conn (if you set limit_req_status 429) |
| 444 | NGINX (special) | You returned it — close connection, send nothing |
| 499 | NGINX (special) | Client closed the connection before NGINX responded. Means your app was too slow OR the client timed out. Not an app error — but a strong latency signal. |
| 502 | NGINX | Upstream refused/closed/sent garbage/oversized headers |
| 503 | NGINX or app | No live upstreams; or limit_* default status |
| 504 | NGINX | proxy_read_timeout or proxy_connect_timeout exceeded |
It is not a server error and won't appear in your app's error rate — but a wave of 499s at exactly 30s means a client-side timeout is firing against a slow endpoint. Alert on it separately.
Live inspection
# which upstream served each request, and how long it took tail -f access.log | jq -r '[.upstream_addr,.status,.upstream_response_time,.uri]|@tsv' # connection state distribution ss -tan | awk '{print $1}' | sort | uniq -c | sort -rn # is a worker blocked? (should be in epoll_wait almost always) strace -p $(pgrep -f 'nginx: worker' | head -1) -c -f # flame graph a hot worker perf record -F 99 -p $(pgrep -f 'nginx: worker' | head -1) -g -- sleep 30 # reproduce a request exactly as nginx sees it curl -v --resolve api.example.com:443:10.0.1.5 https://api.example.com/health openssl s_client -connect example.com:443 -servername example.com -tls1_3
18Stream module — L4 TCP/UDP proxying
stream { log_format basic '$remote_addr [$time_local] $protocol $status ' '$bytes_sent $bytes_received $session_time $upstream_addr'; access_log /var/log/nginx/stream.log basic; # ── TCP load balancing (e.g. Postgres read replicas) ── upstream pg_read { zone pg_read 64k; least_conn; server 10.0.2.11:5432 max_fails=2 fail_timeout=10s; server 10.0.2.12:5432 max_fails=2 fail_timeout=10s; } server { listen 5432; proxy_pass pg_read; proxy_connect_timeout 2s; proxy_timeout 10m; # idle timeout for the whole session proxy_socket_keepalive on; } # ── UDP (DNS, syslog, QUIC, game traffic) ── upstream dns { server 10.0.3.1:53; server 10.0.3.2:53; } server { listen 53 udp; proxy_pass dns; proxy_responses 1; # expect exactly 1 datagram back proxy_timeout 1s; } # ── TLS passthrough routed by SNI (no decryption!) ── map $ssl_preread_server_name $sni_backend { api.example.com api_tls; admin.example.com admin_tls; default default_tls; } upstream api_tls { server 10.0.4.1:443; } upstream admin_tls { server 10.0.4.2:443; } upstream default_tls { server 10.0.4.3:443; } server { listen 443; ssl_preread on; # peek at the ClientHello WITHOUT terminating proxy_pass $sni_backend; } # ── PROXY protocol: preserve client IP through an L4 hop ── server { listen 8443 proxy_protocol; # accept PROXY header from an upstream LB set_real_ip_from 10.0.0.0/8; proxy_pass backend; proxy_protocol on; # and emit it downstream } }
| http{} — L7 | stream{} — L4 | |
|---|---|---|
| Sees | Method, URI, headers, body | Bytes + TCP/UDP metadata (+ SNI via ssl_preread) |
| Can do | Path routing, caching, header rewrite, compression, auth | Port forwarding, SNI routing, TLS passthrough, DB/queue proxying |
| Cost | Parsing + buffering per request | Near-zero — just splice bytes |
| Use for | Web/API traffic | Postgres, Redis, Kafka, SMTP, MQTT, end-to-end-encrypted TLS |
"If the requirement is end-to-end encryption to the backend for compliance, I can't terminate TLS at the edge. I use stream + ssl_preread to route by SNI without decrypting — I get host-based routing and health checks while the ciphertext flows through untouched. The trade-off: no path routing, no caching, no WAF, and no L7 observability."
PROXY protocol chain
19gRPC, WebSocket, SSE
gRPC
upstream grpc_svc { zone grpc_svc 64k; server 10.0.5.1:50051; server 10.0.5.2:50051; keepalive 32; } server { listen 443 ssl; http2 on; # gRPC REQUIRES HTTP/2 end to end location /helloworld.Greeter/ { # route by fully-qualified service name grpc_pass grpc://grpc_svc; # grpcs:// for TLS to upstream grpc_set_header X-Request-ID $request_id; grpc_read_timeout 1h; # streaming RPCs are long-lived grpc_send_timeout 1h; grpc_socket_keepalive on; } # map gRPC status onto a sane error when the backend is gone error_page 502 = /grpc_unavailable; location = /grpc_unavailable { internal; add_header grpc-status 14 always; # UNAVAILABLE add_header grpc-message "upstream unavailable" always; return 204; } }
gRPC multiplexes many RPCs over one long-lived HTTP/2 connection. A connection-level L4 balancer therefore pins all of a client's RPCs to one backend forever — you get zero balancing and terrible skew after a scale-up. You need request-level (L7) balancing, which is exactly what grpc_pass gives you. Alternatives: client-side LB with a resolver (gRPC's own round_robin policy over headless DNS), a proper mesh (Envoy), or setting MAX_CONNECTION_AGE on the server so connections periodically rebalance.
WebSocket
map $http_upgrade $connection_upgrade { default upgrade; '' close; } location /ws/ { proxy_pass http://ws_backend; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header Host $host; proxy_read_timeout 3600s; # else idle sockets are killed after 60s proxy_send_timeout 3600s; proxy_buffering off; proxy_socket_keepalive on; }
Each WS connection holds a worker slot for its entire lifetime and keeps an old worker alive across every config reload. On a busy chat service that means workers accumulate. Mitigations: worker_shutdown_timeout, server-side max connection age with client reconnect-with-backoff, and putting WS on a separate NGINX pool from your HTTP traffic so reloads there don't disturb it.
Server-Sent Events
location /events { proxy_pass http://sse_backend; proxy_http_version 1.1; proxy_set_header Connection ''; proxy_buffering off; ← without this the client gets NOTHING until the end proxy_cache off; gzip off; ← gzip buffers too chunked_transfer_encoding off; proxy_read_timeout 24h; }
20OpenResty, njs & modules
Module types
| Type | Notes |
|---|---|
| Static | Compiled in at build time (./configure --with-http_v2_module). Check with nginx -V. |
| Dynamic (1.9.11+) | .so loaded with load_module in the main context. Must be built against the exact NGINX version. |
# Commonly-added modules load_module modules/ngx_http_brotli_filter_module.so; load_module modules/ngx_http_geoip2_module.so; load_module modules/ngx_http_headers_more_filter_module.so; # merge/remove headers load_module modules/ngx_http_modsecurity_module.so; load_module modules/ngx_http_js_module.so; # njs load_module modules/ngx_otel_module.so;
njs — JavaScript, first-party, lightweight
# /etc/nginx/njs/hmac.js # function verify(r) { # var sig = r.headersIn['X-Signature']; # ... compute expected ... # if (sig !== expected) { r.return(403); return; } # r.return(204); # } # export default { verify }; js_import hmac from njs/hmac.js; js_set $tenant hmac.tenantFromToken; # compute a variable in JS location = /_verify { internal; js_content hmac.verify; } location /api/ { auth_request /_verify; proxy_pass http://api; }
OpenResty / Lua — full request-phase control
lua_shared_dict ratelimit 10m; location /api/ { access_by_lua_block { -- distributed rate limit backed by Redis local redis = require "resty.redis" local red = redis:new() red:set_timeouts(100, 100, 100) -- NON-BLOCKING (cosocket) local ok = red:connect("redis", 6379) if not ok then return end -- fail OPEN, not closed local key = "rl:" .. ngx.var.http_x_api_key local n = red:incr(key) if n == 1 then red:expire(key, 1) end red:set_keepalive(10000, 100) -- return conn to pool if n > 100 then ngx.header["Retry-After"] = 1 return ngx.exit(429) end } proxy_pass http://api; }
Lua phase hooks map onto NGINX phases: set_by_lua → rewrite_by_lua → access_by_lua → content_by_lua → header_filter_by_lua → body_filter_by_lua → log_by_lua. Plus init_by_lua (master, pre-fork) and init_worker_by_lua (per worker — where you start background timers).
Use only cosocket APIs (resty.*) — they yield to the event loop. Any use of os.execute, LuaSocket, blocking file I/O, or a tight CPU loop blocks the whole worker. Also: init_by_lua runs before fork, so anything cached there is shared copy-on-write; per-worker state belongs in init_worker_by_lua, and cross-worker state in lua_shared_dict.
GeoIP
geoip2 /etc/nginx/GeoLite2-Country.mmdb { auto_reload 60m; $geoip2_country_code country iso_code; } map $geoip2_country_code $blocked { default 0; XX 1; YY 1; } server { if ($blocked) { return 403; } }
21NGINX in Kubernetes
Two different controllers — know the difference
ingress-nginx (kubernetes/ingress-nginx) | nginx-ingress (nginxinc) | |
|---|---|---|
| Maintainer | Kubernetes community | F5/NGINX Inc. |
| Base | NGINX + Lua (OpenResty) | Plain NGINX / NGINX Plus |
| Endpoint changes | Updated in Lua without a reload | Reload (OSS) or dynamic API (Plus) |
| Config surface | Huge nginx.ingress.kubernetes.io/* annotation set | VirtualServer/VirtualServerRoute CRDs |
| Note | The one most people mean by "nginx ingress". Went into maintenance mode in 2025 with InGate proposed as successor — check current status. | Cleaner CRD model, commercial support path |
# Ingress with the common annotations apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: api annotations: nginx.ingress.kubernetes.io/proxy-body-size: "25m" nginx.ingress.kubernetes.io/proxy-read-timeout: "60" nginx.ingress.kubernetes.io/proxy-connect-timeout: "3" nginx.ingress.kubernetes.io/limit-rps: "100" nginx.ingress.kubernetes.io/limit-burst-multiplier: "5" nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/backend-protocol: "GRPC" # canary: route 10% of traffic to this Ingress' backend nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "10" # or header-based canary for internal testing nginx.ingress.kubernetes.io/canary-by-header: "X-Canary" spec: ingressClassName: nginx tls: - hosts: [api.example.com] secretName: api-tls rules: - host: api.example.com http: paths: - path: /v1 pathType: Prefix backend: { service: { name: api-svc, port: { number: 80 } } }
In a large cluster, Deployments scale and pods reschedule constantly. A naive controller re-renders nginx.conf and reloads on every Endpoint change. At high churn that means reloads every few seconds, and each reload: doubles worker memory transiently, resets limit_req counters and the TLS session cache, and leaves old workers lingering on long connections. The controller can spend more time reloading than serving.
Fixes: (1) ingress-nginx's Lua endpoint updater bypasses reload for pure endpoint changes; (2) NGINX Plus dynamic upstream API; (3) move to an xDS-based proxy (Envoy / Gateway API implementations like Envoy Gateway, Istio, Contour) where config updates are streamed atomically with no process churn. That's the honest reason Envoy dominates mesh workloads.
Ingress vs Gateway API
Ingress is expressively limited, so every vendor bolted on annotations — which are unportable and untyped. Gateway API (GatewayClass / Gateway / HTTPRoute / GRPCRoute) is the successor: role-oriented (infra team owns Gateway, app team owns HTTPRoute), typed, and supports header matching, traffic splitting, and request mirroring natively. Expect a question on this if the role is platform-flavoured.
NGINX as sidecar vs mesh
NGINX can be a sidecar, but it lacks a control plane: no xDS, no automatic mTLS identity (SPIFFE), no per-service circuit-breaker telemetry, no centralised policy. That's why meshes standardised on Envoy. NGINX's sweet spot in k8s is the north-south edge; Envoy owns east-west.
22NGINX vs Envoy vs HAProxy vs the rest
| NGINX | Envoy | HAProxy | Traefik | Caddy | |
|---|---|---|---|---|---|
| Language | C | C++ | C | Go | Go |
| Config | Static file + reload | Dynamic xDS (hot, atomic) | Static + Runtime API | Auto from providers | Caddyfile / JSON API |
| Threading | Multi-process, 1 thread each | Multi-threaded, thread-per-core | Multi-threaded (single-proc) | Goroutines | Goroutines |
| L7 features | Excellent | Excellent + retries/outlier detection/shadowing built in | Excellent, superb L4 | Good | Good |
| Caching | Strong (disk cache) | Weak | Basic small-object cache | None | Plugin |
| Observability | Basic OSS (stub_status) | Best-in-class — huge stats surface, native tracing | Very good stats socket | Good | Good |
| Health checks | Passive (OSS) / Active (Plus) | Active + outlier detection | Active, free | Active | Active |
| Auto-TLS | External (certbot) | External | External | Built-in ACME | Built-in ACME |
| Best at | Edge, static, caching, TLS termination | Service mesh, dynamic environments | Pure LB, TCP, extreme reliability | Container-native routing | Simplicity, HTTPS by default |
- NGINX — the edge tier: TLS termination, response caching, static assets, simple stable routing, huge operational familiarity, tiny memory footprint. Config is a file, which means it's reviewable in git and diffable — an underrated operational property.
- Envoy — anywhere config changes frequently or you need first-class observability: service mesh, multi-tenant gateways, anything driven by a control plane. Retries with budgets, outlier detection, request shadowing, and per-route stats are built in rather than bolted on.
- HAProxy — when load balancing is the job: free active health checks, best-in-class TCP mode, exceptional stability, and a runtime API for changing servers without reload.
- Traefik/Caddy — small teams, automatic certificates, container-native discovery, low operational overhead.
And the meta-answer: "most large orgs run more than one — Envoy or a cloud LB at L4, NGINX at the caching/TLS edge, and Envoy sidecars east-west. Picking one globally is usually the wrong framing."
23Real-world architectures
A. Multi-tier edge for a high-traffic site
B. Canary / blue-green / A-B
# ── Weighted canary via split_clients (deterministic per user!) ── split_clients "$cookie_uid$remote_addr" $variant { 5% canary; 5% canary_b; * stable; } map $variant $pool { canary app_v2; canary_b app_v3; default app_v1; } # Header override wins so QA can force a variant map $http_x_force_variant $final_pool { v2 app_v2; v3 app_v3; default $pool; } location / { proxy_pass http://$final_pool; add_header X-Variant $variant always; # log & measure per variant }
split_clients hashes the key with MurmurHash2, so the same user always lands in the same bucket across requests — essential or your A/B test is noise. Include something stable (user id) in the key, not just $request_id.
# ── Blue-green: flip an included file and reload ── # /etc/nginx/active_pool.conf contains: set $active blue; include /etc/nginx/active_pool.conf; location / { proxy_pass http://app_$active; } # ── Mirror traffic to a new stack without affecting users ── location / { mirror /_shadow; mirror_request_body on; proxy_pass http://app_v1; } location = /_shadow { internal; proxy_pass http://app_v2$request_uri; proxy_connect_timeout 1s; # response is DISCARDED; client never waits on it }
Mirrored requests hit a real backend — if v2 writes to the production database you've just double-written. Point mirrors at an isolated stack with its own data store. Also, mirroring doubles your outbound connection count and the mirror subrequest still consumes a worker_connections slot.
C. API gateway pattern
upstream users_svc { zone u 64k; server users:8080; keepalive 32; } upstream orders_svc { zone o 64k; server orders:8080; keepalive 32; } server { listen 443 ssl; http2 on; server_name api.example.com; # global policy limit_req zone=perkey burst=50 nodelay; auth_request /_authz; proxy_set_header X-Request-ID $request_id; # versioned routes; note ^~ so no regex is evaluated location ^~ /v1/users/ { rewrite ^/v1/users/(.*) /$1 break; proxy_pass http://users_svc; } location ^~ /v1/orders/ { rewrite ^/v1/orders/(.*) /$1 break; proxy_pass http://orders_svc; } # cacheable public reads location ^~ /v1/catalog/ { auth_request off; proxy_cache api_cache; proxy_cache_valid 200 60s; proxy_cache_lock on; proxy_pass http://catalog_svc; } # deprecation / sunset location ^~ /v0/ { add_header Sunset "Wed, 31 Dec 2026 23:59:59 GMT" always; add_header Deprecation "true" always; proxy_pass http://legacy_svc; } location = /_authz { internal; proxy_pass http://authz; ... } }
D. Surviving a traffic spike (the "HN front page" question)
- Microcache anonymous responses for 1s → app sees ~1 rps per distinct URL.
proxy_cache_lock on+use_stale updating+background_update→ no stampede on expiry.max_connsper backend → bulkhead so the app tier can't be overwhelmed.limit_reqkeyed on IP with generous burst → shave off abusive clients only.proxy_cache_use_stale error timeout http_5xx→ if the app dies, keep serving the last good copy.- Static
error_pagefallback for anything not in cache — degraded, not down. - Push assets to the CDN with
Cache-Control: immutableand long TTLs so NGINX never sees them.
E. Multi-region failover
upstream region_pool { zone region_pool 64k; server us-east.internal:443 max_fails=2 fail_timeout=10s; # local server us-west.internal:443 backup; # DR — only on total local failure keepalive 32; } location / { proxy_pass https://region_pool; proxy_ssl_server_name on; ← REQUIRED: send SNI upstream proxy_ssl_name $host; proxy_ssl_verify on; proxy_ssl_trusted_certificate /etc/ssl/internal-ca.pem; proxy_ssl_session_reuse on; proxy_next_upstream error timeout http_502 http_503; proxy_next_upstream_tries 2; }
proxy_ssl_server_name defaults to off. Proxying to any TLS endpoint that uses SNI-based cert selection (which is every cloud LB and CDN) without it produces a handshake failure or the wrong certificate. Similarly, proxy_ssl_verify defaults to off — meaning by default NGINX does not validate upstream certificates. Turn it on for anything crossing a trust boundary.
24Top 20 gotchas — rapid fire
| # | Gotcha | Fix |
|---|---|---|
| 1 | proxy_pass trailing slash changes URI semantics | Slash ⇒ strip prefix; no slash ⇒ pass through |
| 2 | add_header in a child location wipes inherited headers | Repeat them, or use headers_more |
| 3 | add_header ignored on 4xx/5xx | Add always |
| 4 | Upstream keepalive silently off | Need both proxy_http_version 1.1 and proxy_set_header Connection "" |
| 5 | upstream DNS resolved once at startup | resolver+variable, Plus resolve, or service discovery |
| 6 | No zone ⇒ per-worker state | Always declare zone in upstreams |
| 7 | proxy_read_timeout isn't a total deadline | Enforce deadlines in the app |
| 8 | Keepalive idle-timeout races cause phantom 502s | NGINX timeout < upstream idle timeout; > downstream LB timeout |
| 9 | if inside location is undefined behaviour | Use map / try_files / return |
| 10 | alias without symmetric trailing slash ⇒ path traversal | Match trailing slashes exactly |
| 11 | Trusting all XFF ⇒ IP spoofing | Narrow set_real_ip_from + real_ip_recursive on |
| 12 | proxy_cache_bypass alone still stores private responses | Also set proxy_cache_no_cache |
| 13 | Cache key missing $host on multi-tenant | Include host and any Vary dimension |
| 14 | Reload resets limit_req counters & TLS session cache | Reduce reload frequency; dynamic upstreams |
| 15 | Missing default_server ⇒ Host-header routing surprises | Explicit catch-all returning 444 |
| 16 | proxy_ssl_server_name off by default | Turn on when proxying to TLS with SNI |
| 17 | proxy_ssl_verify off by default | Turn on + set trusted CA |
| 18 | Regex locations evaluated in file order, not by specificity | Order deliberately; prefer ^~ prefixes |
| 19 | gzip without gzip_vary ⇒ CDN serves wrong encoding | gzip_vary on |
| 20 | SSE/WebSocket blank until finished | proxy_buffering off (+ gzip off for SSE) |
25Interview Q&A bank
Click to reveal. Answers are written as you'd actually say them — with the trade-off, not just the fact.
Architecture & internals
Walk me through what happens from TCP SYN to response byte for a proxied request.
The kernel completes the handshake and queues the socket on the listen backlog. A worker's epoll reports the listening fd readable and it accept()s (or, with reuseport, the kernel already steered it to that worker's own queue). NGINX allocates a connection pool and, if TLS, drives the handshake — hopefully resumed via session cache or ticket.
Then it reads and parses the request line and headers into a request pool. It picks the server block by listen + Host, then runs the phase pipeline: POST_READ (realip rewrites $remote_addr), SERVER_REWRITE, FIND_CONFIG (location match), REWRITE, PREACCESS (rate limits), ACCESS (auth), PRECONTENT (try_files, mirror), then CONTENT where proxy_pass runs.
The proxy module selects an upstream peer per the balancing algorithm, grabs an idle keepalive connection if available, rewrites headers per proxy_set_header, and writes the request. Response headers come back into proxy_buffer_size; the body streams into proxy_buffers and spills to a temp file if it overflows. The response goes through the header filter chain and the body filter chain (ssi → sub_filter → gzip → chunked → writer) to the client socket. Finally the LOG phase fires, the request pool is freed, and the connection either goes idle for keepalive or closes.
Two things I'd call out: the upstream connection is released back to the pool as soon as the response is fully buffered — that's the point of buffering — and the whole path is one worker, non-blocking, so any blocking call here would stall every other connection on that worker.
Why is NGINX faster than Apache prefork? Where would Apache actually win?
NGINX's memory and scheduling cost scales with active work, not with connection count — a fixed set of workers multiplexing tens of thousands of connections via epoll, ~1 KB of heap per idle connection versus a whole process/thread stack.
Apache wins where you want embedded interpreters and per-request isolation: mod_php, .htaccess for shared hosting where each customer edits their own config, and a much richer module ecosystem for in-process request handling. Also, Apache's event MPM narrowed the gap considerably; the "NGINX is 10× faster" framing is a 2010 argument. Today I'd pick NGINX for proxying and static content, and I wouldn't fight to migrate a working Apache app-server setup.
What is worker_connections and how do you size it?
It's the max simultaneous connections per worker, and it counts everything: client connections, upstream connections, listening sockets, resolver sockets, and internal subrequests. So a proxied request consumes at least two slots — max_clients ≈ worker_processes × worker_connections / 2, and less if you use auth_request or mirror.
Sizing: start from peak concurrent connections (not RPS) = peak RPS × avg connection lifetime, add headroom for keepalive idle connections, divide by worker count, multiply by 2–3. Then make sure worker_rlimit_nofile ≥ worker_connections and the systemd LimitNOFILE covers it, or you'll hit "too many open files" long before the configured ceiling.
What breaks if you do blocking work in a worker?
Everything on that worker queues behind it. With 8 workers, roughly 1/8 of in-flight requests stall for the duration. The signature in metrics is p99 latency spikes with a flat $upstream_response_time — the time is being spent inside NGINX, not the app.
Sources: cold-file disk reads (fix: aio threads + directio), synchronous DNS (fix: resolver, which is async), blocking Lua (fix: cosockets only), enormous regexes with catastrophic backtracking, and huge gzip on giant bodies.
Explain shared memory zones and what happens when one fills up.
Workers are separate processes, so any state that must be consistent across them lives in an mmap'd slab allocator declared in config: limit_req_zone, limit_conn_zone, keys_zone for proxy_cache, upstream ... zone, ssl_session_cache shared:.
They're fixed size. When a limit_req zone fills, NGINX force-evicts up to a couple of LRU entries per request — under a wide distributed attack the zone thrashes and the limiter becomes useless. When a cache keys zone fills, NGINX starts evicting cache entries irrespective of max_size, so your 20 GB disk cache mysteriously plateaus at 3 GB. Rule of thumb: ~16k IP states per MB for limit zones, ~8k keys per MB for cache zones. Size the keys zone from your expected object count first.
Configuration
Explain proxy_pass with and without a trailing slash.
If proxy_pass includes any URI component — even a bare / — NGINX replaces the matched location prefix with it. If it's only scheme+host+port, the original request URI is passed through unchanged.
So with location /api/: proxy_pass http://b; sends /api/users, while proxy_pass http://b/; sends /users. Two extra rules: you can't use a URI part inside a regex or named location (use rewrite ... break instead), and if you use a variable in proxy_pass the URI is dropped unless you append $request_uri explicitly.
Order of precedence for location matching?
= exact wins outright and stops the search. Otherwise NGINX finds the longest prefix match; if that one was declared ^~, it's used and regexes are skipped. Otherwise regexes are tried in configuration file order and the first match wins. If no regex matches, the remembered longest prefix is used.
The part people miss is that regex order is file order, not specificity or length — so a broad ~ \.php$ placed above a narrow one silently shadows it.
Why is if discouraged?
Inside a location, if creates an implicit nested configuration context. Directives that aren't return or rewrite ... last can end up evaluated in the wrong context — set values can be lost, proxy_pass inside an if can bypass the parent's proxy_set_header directives entirely. It's not a bug so much as a leaky implementation detail that the maintainers documented as "IfIsEvil".
My rule: if only for return. Value selection goes through map (which is a hash lookup, lazily evaluated, and inherits cleanly), file existence through try_files, and anything genuinely conditional through map-chained variables or the upstream application.
root vs alias?
root appends the request URI to the path; alias replaces the matched location prefix with the path. location /s/ { root /var/www; } maps /s/a.js → /var/www/s/a.js; with alias /var/www/assets/; it maps to /var/www/assets/a.js.
Security note: keep trailing slashes symmetric. location /s { alias /var/www/assets/; } — no slash on the location, slash on the alias — allows /s../ to escape the directory. That's a recurring real-world CVE pattern.
Why do my security headers vanish in one location?
add_header is array-inheriting: a child context that defines any add_header discards the entire inherited array rather than merging. Same for proxy_set_header. So adding one debug header in /api/ silently drops HSTS and CSP for that path.
Fixes, in order of preference: put all headers in a snippet and include it in every location; use more_set_headers from the headers_more module which merges properly; or add a CI check that asserts the security headers are present on every route.
Load balancing & resilience
Compare the load-balancing algorithms and when you'd use each.
Round-robin for uniform, short requests. least_conn when request durations vary — which is most real systems — but be aware of the black-hole failure mode: a backend that fails instantly shows zero active connections and therefore attracts all traffic. Pair it with health checks.
ip_hash only for legacy stickiness, and it's poor: it hashes the first three IPv4 octets, so anyone behind CGNAT lands together, and adding a node reshuffles almost everything. hash $key consistent is the better sharding primitive — Ketama ring, so only ~1/N of keys move when membership changes. That's what you want for cache-node affinity.
random two least_conn is the one worth knowing: with many independent NGINX instances, plain least_conn makes them all pick the same idle backend simultaneously and oscillate. Power-of-two-choices adds randomness and provably bounds max load near the average. Envoy defaults to it for the same reason.
OSS health checks — what do you actually get, and what would you do about it?
Passive only: max_fails failures within fail_timeout marks the peer down for fail_timeout, then one real user request probes it. What counts as a failure is defined by proxy_next_upstream. So detection requires real traffic to fail first, and recovery makes a user pay for the probe.
Options at staff level: run NGINX Plus or Angie for active checks; or accept that health-checking belongs a layer up — the cloud LB and the k8s readiness probe already do it, and the Endpoints controller removes unhealthy pods from the upstream list before NGINX ever sees them. That's usually the cleanest answer: don't duplicate health checking in the proxy when the orchestrator owns it.
Your backend gets slow. Walk me through what NGINX does and how it can make things worse.
Requests pile up. Each waiting request holds a client connection and an upstream connection, so worker_connections pressure roughly doubles. When proxy_read_timeout fires, NGINX returns 504 — and if proxy_next_upstream includes timeout, it retries on another backend, multiplying load on a tier that's already struggling. That's retry amplification, and it's how a brownout becomes an outage.
Meanwhile clients time out first, giving you a wave of 499s, and any client-side retry adds another multiplier.
What I'd do: cap proxy_next_upstream_tries 2 with a proxy_next_upstream_timeout budget; set max_conns per backend as a bulkhead so excess requests fail fast instead of queueing; enable proxy_cache_use_stale so degraded means "slightly stale" rather than "500"; and make sure client retries use exponential backoff with jitter. The principle is shed load early and cheaply rather than queueing it.
Is retrying a failed POST safe?
No, not by default. proxy_next_upstream error timeout will retry a request that timed out — but a timeout means NGINX doesn't know whether the upstream processed it. Retrying a POST /payments can double-charge. NGINX guards this: non-idempotent methods aren't retried after the request was sent unless you explicitly add non_idempotent to proxy_next_upstream.
The real fix is idempotency keys in the API contract — client generates a UUID, server dedupes. Then retries are safe everywhere in the stack, not just at NGINX.
How do you do sticky sessions, and should you?
OSS: hash $cookie_sessionid consistent. Plus: sticky cookie, sticky route, or sticky learn (which tracks the app's own session cookie in a shared zone).
Should you? Usually not. Stickiness makes deploys, autoscaling and node failure user-visible, and it defeats even load distribution. I'd push session state into Redis or a signed token and keep backends stateless. Legitimate exceptions: WebSocket connections (inherently pinned anyway), in-process caches where affinity is a performance optimization rather than a correctness requirement, and legacy apps you can't refactor yet.
Caching
Design a cache key. What goes wrong if you get it wrong?
Default is $scheme$proxy_host$request_uri. I'd normally use $scheme$request_method$host$request_uri, plus any dimension the response actually varies on.
Failure modes: omit $host on a multi-tenant proxy and tenant A gets tenant B's response — a data breach, not a bug. Omit the method and a HEAD poisons the GET entry. Ignore Vary: Accept-Encoding and you hand a gzip body to a client that can't decode it. Include $http_cookie and your cardinality explodes to near-zero hit rate. And any attacker-controlled header that the upstream reflects into the response but that isn't in the key is a web-cache-poisoning vector — X-Forwarded-Host is the classic.
Also: query parameter order creates distinct keys, and tracking params like utm_* fragment the cache. Normalizing those with a map is often a bigger hit-rate win than any TTL tuning.
What is a cache stampede and how does NGINX prevent it?
A hot key expires and every concurrent request for it simultaneously misses, so all of them are forwarded upstream at once. The upstream saturates, requests time out, nothing gets stored, and the pattern repeats — a self-sustaining outage.
proxy_cache_lock on lets only the first request populate a cold key; the rest wait up to proxy_cache_lock_timeout. Better: combine with proxy_cache_use_stale updating and proxy_cache_background_update on, so waiting requests get the stale copy immediately while one background request refreshes. Upstream sees exactly one request; nobody waits.
proxy_cache_min_uses 2 is a complementary control — it stops one-hit-wonder URLs from evicting genuinely hot content.
Explain microcaching and why it's so effective.
Cache "dynamic" pages for one second. At 5,000 rps on a hot URL, the app sees 1 rps — a 99.98% reduction — and no human perceives one second of staleness on a news page or product listing.
It works because traffic spikes are concentrated: a small number of URLs take the overwhelming majority of requests. Even a sub-second TTL collapses that concentration. It's the highest return-on-effort change available for read-heavy anonymous traffic, and it requires no application change.
Caveat: it must be scoped to anonymous traffic. Add proxy_cache_bypass and proxy_cache_no_cache on session cookies and Authorization, or you'll serve one user's personalized page to everyone.
proxy_cache_bypass vs proxy_no_cache?
proxy_cache_bypass means "don't read from cache for this request" — but the response can still be written. proxy_no_cache means "don't write this response" — but a cached copy could still be served. They're independent, and for authenticated traffic you need both. Setting only bypass is a known route to leaking a private response into the shared cache.
How do you invalidate cache across a fleet of 50 NGINX nodes?
Native proxy_cache_purge is Plus-only and is per-node, so you'd need a fan-out to all 50 — which is a distributed-systems problem with partial-failure semantics you don't want.
My preferred design is versioned cache keys: put a token in the key ($cache_version$scheme$host$request_uri) sourced from a build SHA or a per-tenant epoch. "Purging" becomes bumping the token, which is atomic, instantaneous across the fleet, and needs no coordination. Stale entries age out via inactive.
If you truly need targeted purge, the honest answer is to move the cache tier to something built for it — Varnish with soft purge and surrogate keys, or a CDN with tag-based invalidation.
TLS & protocols
What actually happens in a TLS handshake and how do you make it cheap?
TLS 1.3 full handshake: ClientHello with key share → ServerHello, cert, CertificateVerify (an asymmetric signature — the expensive part) → Finished. One round trip. TLS 1.2 needs two.
Making it cheap: (1) resumption — session cache (server-side state in a shared zone) or session tickets (client holds encrypted state, so it works across the whole fleet). Resumed handshakes skip the asymmetric operation entirely. (2) ECDSA certificates, roughly 4–10× cheaper to sign than RSA-2048; serve both and let the client pick. (3) OCSP stapling so clients don't make their own revocation round trip. (4) 0-RTT for resumed connections, with the caveat that early data is replayable and must never trigger a state change.
The operational gotcha: if you don't distribute ssl_session_ticket_key files across the fleet, each instance invents its own, so resumption fails whenever the LB sends a client to a different box — and the never-rotated key undermines forward secrecy.
HTTP/2 vs HTTP/3 — what actually changes?
HTTP/2 multiplexes streams over one TCP connection with HPACK header compression. But TCP delivers bytes in order, so a single lost packet head-of-line-blocks every stream on that connection. Over lossy mobile networks HTTP/2 can be slower than HTTP/1.1 with six connections.
HTTP/3 moves to QUIC over UDP, which implements per-stream reliability — packet loss only stalls the affected stream. It also merges the transport and TLS handshakes (1 RTT, 0 RTT resumed) and adds connection migration via a Connection ID, so a phone switching wifi→cellular keeps its session.
Operationally: HTTP/3 needs UDP 443 open (many corporate networks block it), CPU cost is higher because congestion control runs in userspace, and you advertise it with Alt-Svc so clients upgrade on a subsequent connection. And note that NGINX still talks HTTP/1.1 to upstreams by default regardless — which is fine and lets you pool connections.
How do you terminate TLS but still guarantee encryption to the backend?
Re-encrypt: terminate at the edge, then proxy_pass https:// with proxy_ssl_verify on, a trusted internal CA, proxy_ssl_server_name on (it defaults to off and will break SNI-based upstreams), and proxy_ssl_session_reuse on to avoid a fresh handshake per request. Optionally mTLS with proxy_ssl_certificate so the backend can authenticate NGINX.
If compliance forbids the plaintext window entirely, don't terminate at all: use stream with ssl_preread on to route on SNI without decrypting. You keep host-based routing and health checks but lose caching, path routing, WAF and L7 metrics — that trade needs to be explicit.
Operations
What exactly happens during nginx -s reload, and what does it cost?
The master re-reads and validates the config. If it's invalid, it logs and keeps running the old config — reloads are safe in that sense. If valid, it opens any new listen sockets, forks new workers with the new config, and signals old workers with QUIT: they stop accepting, finish in-flight requests, close idle keepalives, and exit.
The costs people forget: during the overlap you have double the workers and roughly double the memory. Old workers survive as long as their longest connection — a WebSocket can keep one alive for hours unless you set worker_shutdown_timeout. And all shared memory zones are recreated, so limit_req counters reset and the TLS session cache is flushed.
That last point is the argument for dynamic config in high-churn environments: an ingress controller reloading every few seconds effectively has no working rate limiter and causes continuous TLS handshake storms.
How do you upgrade the NGINX binary with zero dropped connections?
USR2 to the master: it renames the pidfile to .oldbin and execs the new binary, which inherits the listening sockets — so two masters are now accepting on the same sockets. Then WINCH to the old master gracefully retires its workers while keeping the old master alive as a rollback option. Verify health on the new version, then QUIT the old master to commit — or HUP it to respawn its workers and QUIT the new one to roll back.
In practice, on immutable infrastructure I'd rather roll new instances behind the LB than do in-place binary upgrades — but the mechanism is worth knowing because it's what makes NGINX viable on long-lived pets.
You're paged: 502s spiking. What do you check, in order?
error.log— the exact message distinguishes the cause: "connection refused" (nothing listening), "timed out" (network/firewall or saturated accept queue), "upstream prematurely closed" (app crash or idle-timeout race), "too big header" (buffer sizing), "no live upstreams" (all peers passively ejected).- Is it all upstreams or one? Group logs by
$upstream_addr. A single bad node is a very different incident from a tier-wide failure. - Did anything deploy — app, config, or infra? Correlate with deploy timestamps first, always.
- Check
$upstream_connect_time: if it jumped from 0 to non-zero, keepalive pooling broke (someone droppedproxy_http_version 1.1), and you may be hitting ephemeral port exhaustion. - Check whether it's load-correlated or constant-rate. Constant low-rate 502s at low traffic is the classic keepalive idle-timeout race with the upstream.
- Mitigate before you finish diagnosing: enable
proxy_cache_use_stale, shed load with tighter limits, or roll back.
What's a 499 and why do you care?
An NGINX-specific code meaning the client closed the connection before NGINX sent a response. It won't appear in your application's error rate, so teams routinely miss it.
A cluster of 499s at a consistent duration — say all at exactly 30s — tells you a client-side timeout is firing against a slow endpoint. It's a latency signal disguised as a client error, and it's often the earliest warning that a backend is degrading. I alert on 499 rate separately from 5xx.
How would you make NGINX config changes safe at scale?
Treat it as code: templates in git, rendered per environment, reviewed. CI runs nginx -t in a container matching production's version and modules, plus nginx -T diffed against the currently-running config so reviewers see the fully-resolved change, not just the template edit.
Then add semantic checks that nginx -t won't catch: assert security headers exist on every route, assert no location proxies without a timeout, assert no set_real_ip_from broader than your known proxy CIDRs. Those are the changes that pass syntax check and cause incidents.
Rollout: canary one node, watch 5xx and p99 for five minutes with an automatic rollback gate, then roll the fleet with health checks between nodes. Keep last-known-good on disk so rollback is a reload, not a deploy.
Design & judgement
When would you NOT use NGINX?
High-churn dynamic environments where config changes constantly — a service mesh, or a multi-tenant gateway with per-tenant routes changing hourly. Reload churn destroys shared state and Envoy's xDS model is simply the right tool.
When you need deep per-route observability out of the box — Envoy's stats surface is in a different league from stub_status.
When you need free active health checks and pure L4 excellence — HAProxy.
When a small team wants HTTPS to just work — Caddy or Traefik with built-in ACME.
And when a managed cloud LB covers the requirement, adding an NGINX tier is operational cost with no benefit. I'd only introduce it when I need something the LB can't do: response caching, edge auth, custom rate-limit keys, header rewriting, or cloud portability.
How do you rate limit accurately across 20 NGINX instances?
You can't, with vanilla NGINX — the shared zone is per-instance, so a limit of 10 r/s becomes 200 r/s fleet-wide. Options, in increasing cost:
- Divide and accept imprecision. Set
rateto limit/N. Breaks when instance count changes or traffic distribution is uneven, but it's free and adequate for DoS protection. - Consistent-hash the key at L4 so a given API key always reaches the same NGINX node. Now per-node limits are exact per key. Costs you connection-level flexibility and creates hot spots.
- Shared counter — OpenResty + Redis with a sliding-window or token-bucket script, or Envoy's global ratelimit service. Accurate, but adds a network round trip to every request and a hard dependency you must fail open on.
- Two-tier: coarse local limits in NGINX to absorb abuse cheaply, exact quota enforcement in the application where you already have tenant context and can return meaningful
X-RateLimit-*headers.
I'd default to (4) — NGINX as a blunt instrument protecting infrastructure, the app as the precise instrument enforcing business rules.
Design zero-downtime deploys through NGINX.
The proxy side is easy; the coordination is the hard part.
Backend rollout: new instances register, pass readiness, and get added to the upstream (via service discovery re-rendering the upstream block, or the k8s Endpoints controller). Old instances receive SIGTERM, immediately flip readiness to failing, then keep serving in-flight work for longer than the health-check detection window before exiting. Skipping that drain window is the single most common cause of deploy-time 502s.
slow_start (Plus) or gradual weight ramp avoids hammering a cold JIT-compiled instance with full traffic.
For risky changes, layer split_clients canary at 1% → 5% → 25% → 100% with automated rollback on error-rate or latency regression, and use mirror to shadow production traffic at the new version beforehand — pointed at an isolated data store so you don't double-write.
NGINX-side changes go through the config pipeline described earlier. And I'd keep long-lived connections (WebSocket) on a separate NGINX pool so app deploys don't interact with reload behaviour there.
Your p99 is 2s but p50 is 30ms. How do you find out why?
First, decompose the latency in the access log: $request_time vs $upstream_connect_time vs $upstream_header_time vs $upstream_response_time.
- If
$request_timeis huge but upstream time is flat, it's client-side — slow uploads, slow downloads, mobile networks. Not an app problem. Confirm by segmenting on user agent and response size. - If
connect_timeis spiking, upstream accept queues are full or keepalive pooling broke. - If
header_time≈response_timeand both spike, the app is slow — go to app tracing, segment by route and by$upstream_addrto see whether it's one bad instance. - If
header_timeis low butresponse_timehigh, the app is streaming slowly or the body is large.
Then check whether the slow requests cluster: one upstream address (bad node), one route (bad query), one time-of-day (batch job contention), or a periodicity matching worker count (a blocked worker — look for disk I/O without aio threads). Cache status matters too: if p99 is all MISS, the tail is just "uncached" and the fix is hit-rate work, not latency work.
Where does NGINX fit if you already run a service mesh?
The mesh owns east-west: mTLS identity, per-service retries and circuit breaking, and uniform telemetry between services. NGINX owns north-south at the edge, where the concerns are different: TLS termination for public clients with real certificates, response caching, static assets, WAF, bot management, and coarse rate limiting against the internet.
Trying to make NGINX the mesh data plane means rebuilding a control plane — xDS, SPIFFE identity, dynamic config — which is why the industry converged on Envoy there. Conversely, running Envoy at the edge means giving up NGINX's disk cache. Running both, with a clear boundary, is the common and defensible answer.
26System design scenarios
1. Design the edge for 1M rps
- Anycast CDN absorbs static + caches HTML briefly; target 90%+ offload before NGINX.
- L4 (NLB/BGP-ECMP) → NGINX fleet, sized by concurrent connections not rps.
- Per node:
reuseport,worker_processes auto, ECDSA + session tickets shared fleet-wide. - Microcache +
cache_lock+use_stale→ app tier sized for cache-miss rate, not request rate. - Sampled logging (1% of 2xx, 100% of errors) — at 1M rps, logging is an I/O budget item.
- Capacity model: state connections/node, memory/connection, TLS handshakes/sec/core, and headroom for losing an AZ.
2. Multi-tenant SaaS gateway
- Route by subdomain: wildcard cert +
map $host $tenant; per-tenant certs viassl_certificatewith SNI variable or a cert-manager sidecar. - Per-tenant rate limits keyed on tenant ID, tiered by plan via chained
maps. - Cache key must include tenant — this is the one that becomes a breach.
- Per-tenant upstreams for noisy-neighbour isolation;
max_connsas the bulkhead. - Config generated from a control plane; avoid reload-per-tenant-change (batch, or use Plus API).
3. Static + SPA + API on one domain
location ^~ /assets/— hashed filenames,immutable, 1y.location = /index.html—no-cache, so deploys are picked up instantly.location ^~ /api/— proxy, no cache, CORS if cross-origin.location /—try_files $uri $uri/ /index.htmlfor client-side routing.- Precompress at build time;
gzip_static/brotli_staticso you never spend CPU per request.
4. File upload/download service
- Uploads:
client_max_body_sizeper route;proxy_request_buffering offto stream large files straight to object storage without a disk hop. - Downloads: app authorizes, returns
X-Accel-Redirectto aninternallocation — NGINX serves bytes, app worker is freed immediately. aio threads+directio+sendfile offfor large files;limit_rate_afterfor fair bandwidth.max_ranges 1against range-amplification DoS.
5. Migrate a monolith to microservices
- Strangler fig: NGINX routes path-by-path. New service gets
/v1/users/; everything else falls through to the monolith. mirrorthe new route to the new service first, compare responses offline, then flip.split_clientsfor percentage rollout keyed on user id so a user's experience is consistent.- Keep the monolith as
backupin the upstream during the first weeks. - Emit
X-Variant/X-Upstreamheaders and log them — you need per-route comparison data to justify the migration.
6. Global multi-region
- GeoDNS or anycast steers users to the nearest region.
- Regional NGINX prefers local backends, with the remote region as
backup. - Failover trade-off: cross-region latency vs availability — usually you fail over reads freely and writes only with explicit consensus about the data store.
- Beware cascading failover: if region A dies and all its traffic lands on B, B must have the headroom or you've turned one outage into two. Size for N+1 regionally.
27Cheat sheet
Commands
nginx -t # validate config nginx -T # validate + dump FULL resolved config nginx -V # version, compile flags, modules nginx -s reload # graceful reload nginx -s quit # graceful shutdown nginx -s reopen # reopen logs (logrotate) nginx -c /path/nginx.conf -p /path/prefix nginx -g "daemon off;" # foreground — for containers/systemd
Directive quick reference
| Goal | Directive |
|---|---|
| Redirect HTTP→HTTPS | return 301 https://$host$request_uri; |
| Close connection silently | return 444; |
| Serve a literal body | return 200 "ok\n"; add_header Content-Type text/plain; |
| Internal-only location | internal; |
| Strip a path prefix | rewrite ^/api/(.*)$ /$1 break; |
| Deny hidden files | location ~ /\.(?!well-known) { deny all; } |
| Basic auth | auth_basic "x"; auth_basic_user_file /etc/nginx/.htpasswd; |
| Delegate authz | auth_request /_authz; |
| Rate limit | limit_req zone=z burst=20 nodelay; |
| Cap concurrency | limit_conn zone=c 10; / upstream max_conns= |
| Throttle bandwidth | limit_rate 500k; limit_rate_after 10m; |
| Enable upstream pooling | keepalive 64; + proxy_http_version 1.1; + proxy_set_header Connection ""; |
| Serve stale on failure | proxy_cache_use_stale error timeout http_5xx; |
| Stampede protection | proxy_cache_lock on; proxy_cache_background_update on; |
| Shadow traffic | mirror /_shadow; mirror_request_body on; |
| Percentage split | split_clients "$cookie_uid" $v { 10% b; * a; } |
| WebSocket | proxy_set_header Upgrade $http_upgrade; ... Connection $connection_upgrade; |
| Disable buffering (SSE) | proxy_buffering off; gzip off; |
| Route TLS by SNI (no decrypt) | stream { ssl_preread on; proxy_pass $ssl_preread_server_name…; } |
| Preserve client IP behind LB | set_real_ip_from CIDR; real_ip_header X-Forwarded-For; real_ip_recursive on; |
Sane defaults to open with
worker_processes auto; worker_rlimit_nofile 65535; events { worker_connections 16384; multi_accept on; } http { sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 75s; keepalive_requests 1000; client_max_body_size 10m; client_header_timeout 10s; client_body_timeout 10s; server_tokens off; reset_timedout_connection on; gzip on; gzip_vary on; gzip_comp_level 5; gzip_min_length 256; ssl_protocols TLSv1.2 TLSv1.3; ssl_session_cache shared:SSL:50m; ssl_session_timeout 1d; access_log /var/log/nginx/access.log json buffer=64k flush=5s; }
287-day study plan
| Day | Focus | Hands-on |
|---|---|---|
| 1 | Architecture, event loop, phases (§1–2) | Run NGINX locally; strace a worker; watch epoll_wait. Set debug_connection for your IP and read a full request trace. |
| 2 | Config model, matching, map (§3–4) | Build a config with overlapping prefix/regex locations and predict which wins before testing. Reproduce the add_header inheritance bug. |
| 3 | Proxying, buffering, timeouts (§6) | Proxy to a deliberately slow backend. Compare $request_time vs $upstream_response_time. Toggle keepalive and watch $upstream_connect_time and ss -s. |
| 4 | Load balancing + health (§7–8) | Three backends in Docker. Kill one; observe passive ejection timing. Try each algorithm under uneven load. |
| 5 | Caching (§9) | Load-test a cold key with and without proxy_cache_lock. Measure upstream request count. Implement microcaching and record the offload ratio. |
| 6 | TLS, rate limiting, security (§10–13) | Set up TLS 1.3 + stapling; verify with openssl s_client. Tune limit_req burst/nodelay and observe the difference under wrk. |
| 7 | Ops, debugging, design (§14–28) | Break things on purpose: exhaust worker_connections, cause a 502 five different ways, and diagnose each from the error log alone. Then do §26 out loud on a whiteboard. |
- Name the failure mode, not the feature. "I'd enable
proxy_cache_lock" is fine. "Without it, a hot key expiring sends N concurrent requests upstream simultaneously, which is how a cache tier turns a traffic spike into an outage" is the answer they remember. - State the trade-off unprompted. Every directive here costs something — buffering costs TTFB, stickiness costs elasticity, retries cost amplification, WAFs cost CPU and false positives. Volunteering the cost signals you've run this in production.
- Know where NGINX ends. Rate limiting is per-instance. Health checks are passive on OSS. Reloads reset shared state. Being crisp about the boundaries — and what you'd reach for instead — is more convincing than encyclopedic directive recall.