← System Design

NGINX for Staff Engineers

Everything from the event loop up to edge architecture — with the configuration, the failure modes, and the answers interviewers are actually probing for.

event-drivenreverse proxyL4 + L7 LB TLS terminationcachingrate limiting HTTP/2 · HTTP/3gRPCk8s ingress
How to use this at staff level

Junior answers describe what a directive does. Staff answers describe what happens to the system when it's wrong — connection exhaustion, thundering herds on cache expiry, retry storms during a partial upstream outage, config reloads that silently drop nothing but double memory. Every section below has a "what breaks" angle. Lead with that.

00Mental model & positioning

NGINX is a single-binary, event-driven, C-based network proxy. It is not "a web server that also proxies" — in modern architectures it is a proxy that also happens to serve files well.

The five jobs NGINX does in production

RoleWhat it meansTypical placement
Static file serversendfile/mmap straight from page cache to socket, zero app involvementRarely alone now — CDN does this
Reverse proxyTerminates client conn, opens/reuses upstream conn, buffers, rewrites headersIn front of every app tier
Load balancer (L7)Distributes across an upstream pool with health trackingEdge tier + internal service tier
TLS terminatorOffloads handshakes/crypto; often the only TLS-aware componentEdge, or sidecar
Traffic policy pointRate limits, auth subrequests, header injection, canary splits, cachingEdge / API gateway tier

Why it beat Apache — the C10K story

Apache's prefork/worker MPM allocated a process or thread per connection. Each idle keep-alive connection cost ~1–8 MB of RSS and a scheduler slot. 10,000 concurrent connections meant 10,000 threads: context-switch thrash, memory exhaustion, and latency that degraded superlinearly.

NGINX inverted this: a small fixed number of single-threaded workers, each running a non-blocking event loop over epoll (Linux) / kqueue (BSD). Connection state is a small heap struct (~1 KB), not a stack. Memory scales with active connections, not threads. 100k idle keep-alives on one box is routine.

Interview framing

"NGINX trades programming model simplicity for constant memory per connection. The cost is that any blocking operation inside a worker stalls every connection on that worker — which is why disk I/O gets pushed to a thread pool (aio threads) and why you never embed slow synchronous logic in Lua on the request path."

Where NGINX sits in a modern stack

Client │ TLS 1.3, HTTP/2 or HTTP/3 ▼ ┌──────────────┐ Anycast, DDoS scrub, static cache │ CDN │ (Cloudflare / CloudFront / Fastly) └──────┬───────┘ ▼ ┌──────────────┐ L4, cross-AZ, TCP/TLS passthrough │ Cloud LB │ (AWS NLB / ALB, GCP LB) └──────┬───────┘ ▼ ┌─────────────────────────────────────────────┐ │ NGINX EDGE TIER (N boxes) │ │ TLS termination · HTTP/2→1.1 downgrade │ │ routing · rate limit · WAF · auth_request │ │ response cache · gzip/brotli · access log │ └──────┬──────────────────┬───────────────────┘ ▼ ▼ ┌─────────┐ ┌─────────┐ │ app A │ │ app B │ (or a service mesh below this) └─────────┘ └─────────┘

Editions you must be able to distinguish

EditionNotes
OSS nginxFree. Passive health checks only, no native active probes, no dynamic upstream API, no session persistence beyond hashing.
PLUS NGINX PlusCommercial. Active health checks, sticky cookie/learn, live activity dashboard + JSON stats API, dynamic reconfig API (add/remove upstreams without reload), cache purge API, JWT auth, dynamic modules.
FORK OpenRestynginx + LuaJIT + ecosystem. Lets you write request logic in Lua at each phase. Basis of Kong.
FORK Angie / FreenginxCommunity forks after the F5/Nginx Inc. governance split; Angie adds active health checks and Prometheus output for free.
Say this

"On OSS I get passive health checks via max_fails/fail_timeout, which only eject a node after real user requests fail. If I need pre-emptive ejection I either buy Plus, run Angie, or put the health-check responsibility in the cloud LB / service discovery layer and re-render the upstream block."

01Process & event architecture

Process model

┌──────────────────────────┐ root ────────►│ MASTER PROCESS │ reads config, binds :80/:443, │ (no request handling) │ spawns/reaps workers, handles └────────────┬─────────────┘ signals, binary upgrade │ fork() ┌──────────┬──────────┼──────────┬──────────────┐ ▼ ▼ ▼ ▼ ▼ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌──────────────┐ │worker 0│ │worker 1│ │worker 2│ │worker 3│ │ cache manager│ │ epoll │ │ epoll │ │ epoll │ │ epoll │ │ cache loader │ │ nobody │ │ nobody │ │ nobody │ │ nobody │ │ (helpers) │ └────────┘ └────────┘ └────────┘ └────────┘ └──────────────┘ └────────────┴──── shared memory zones ───────┘ (limit_req, limit_conn, proxy_cache keys, upstream zone, ssl_session_cache)

The event loop

Each worker loop iteration:

  1. Compute the nearest timer expiry (red-black tree of timers).
  2. epoll_wait() with that timeout → get a batch of ready fds.
  3. For each ready fd, dispatch its read/write handler — a state machine that does a non-blocking chunk of work and returns.
  4. Fire expired timers (request timeouts, keepalive expiry, resolver TTL).
  5. Run posted events (deferred work queued by handlers).
The cardinal rule

Never block a worker. A single read() on a cold file, a synchronous DNS lookup, a slow Lua HTTP call with a blocking socket, or an expensive regex over a huge body stalls every other connection pinned to that worker. Symptom: p99 latency spikes on a subset of requests with no upstream latency to explain it, and worker_processes-shaped periodicity in the latency histogram.

Non-blocking disk I/O

# Offload blocking file reads to a thread pool so the loop keeps spinning
thread_pool default threads=32 max_queue=65536;

location /video/ {
    aio       threads=default;   # async read via pool
    directio  16m;                # bypass page cache for big files
    sendfile  off;                # sendfile and directio are mutually exclusive
    output_buffers 2 1m;
}

This is the pattern for large-media serving: without it, one cold 1 GB read blocks the worker for the duration of the seek+read.

Connection acceptance & the thundering herd

MechanismBehaviourWhen to use
accept_mutex onWorkers take turns accepting — avoids all workers waking on one connection. Adds latency (serialized accept).Legacy / low connection rate. Default is off since 1.11.3.
EPOLLEXCLUSIVEKernel wakes only one waiter. Used automatically on Linux 4.5+.Default modern behaviour.
reuseport on listenEach worker gets its own listen socket + kernel accept queue; kernel hashes the 4-tuple to a queue.Highest connection-rate workloads. Best throughput, but imbalance possible with long-lived conns.
listen 443 ssl reuseport;   # declare on ONE server block per addr:port only
http2 on;
Gotcha

reuseport must appear on exactly one listen for a given address:port; repeating it in multiple server blocks is a config error. Also: during a reload, connections in a departing worker's dedicated accept queue can be dropped — a known trade-off of reuseport.

Memory model

Zone exhaustion

When a limit_req zone fills, NGINX evicts the LRU entry — under a distributed attack with many source IPs, your zone thrashes and the limiter becomes ineffective. When a proxy_cache keys zone fills, NGINX starts force-evicting cache entries regardless of max_size, and you'll see the disk cache never reach its configured size. Size the keys zone first, then max_size.

02Request processing phases

This is the highest-signal architecture question in an NGINX interview. Modules register handlers into ordered phases; understanding the order explains almost every "why did my directive not apply?" bug.

#PhaseWhat runs here
1POST_READrealip module — rewrites $remote_addr from X-Forwarded-For before anything else sees it.
2SERVER_REWRITErewrite/set directives at server{} level.
3FIND_CONFIGLocation matching happens here. Not pluggable.
4REWRITErewrite/set/if inside location{}.
5POST_REWRITEInternal-redirect check; loops back to FIND_CONFIG if the URI changed (max 10 cycles).
6PREACCESSlimit_req, limit_conn, degradation. Rate limiting happens before auth.
7ACCESSallow/deny, auth_basic, auth_request, auth_jwt.
8POST_ACCESSEvaluates satisfy any|all across the access modules.
9PRECONTENTtry_files, mirror, internal redirects. (Called TRY_FILES pre-1.13.)
10CONTENTExactly one content handler: proxy_pass, fastcgi_pass, grpc_pass, static file handler, return, stub_status
11LOGaccess_log. Runs even if the client disconnected mid-response.

Response bodies then pass through the output filter chain (in order): ssisub_filtergzip/brotlichunked → writer. Headers pass through the header filter chain (add_header, headers_more, CORS).

Classic phase-order questions

"Why does my add_header disappear on a 502?" — Because core add_header only applies to a whitelist of status codes (200, 201, 204, 206, 301, 302, 303, 304, 307, 308) unless you add the always flag. Use add_header X-Foo bar always; or the third-party headers_more module.

"Why does my add_header in a child location wipe out the parent's?"add_header is array-inheriting: a child context that defines any add_header replaces the whole inherited set. Same for proxy_set_header. This causes real security incidents (HSTS/CSP silently dropped in one location).

"Why is my auth bypassed for rate-limited clients?" — It isn't; the reverse is true. limit_req is PREACCESS, so a 429 is returned before auth runs. That's usually what you want (cheap rejection) but means unauthenticated attackers consume your limit-zone slots.

Internal redirects

location / {
    try_files $uri $uri/ /index.html;   # each fallback re-enters FIND_CONFIG
}

location @fallback {                     # named location: only reachable internally
    proxy_pass http://legacy_backend;
}

location /api/ {
    error_page 404 = @fallback;             # "=" makes it adopt the fallback's status
    proxy_pass http://api_backend;
}

X-Accel-Redirect is the same machinery driven by the upstream: your app authorizes a download, then returns an empty 200 with X-Accel-Redirect: /protected/file.zip, and NGINX serves the file from an internal location. The app never streams bytes.

location /protected/ {
    internal;                            # 404 for direct client requests
    alias /var/data/files/;
}

03Configuration model

Context hierarchy

# ── main (global) ─────────────────────────────
user  nginx;
worker_processes  auto;
worker_rlimit_nofile 65535;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log warn;
load_module modules/ngx_http_brotli_filter_module.so;

events {
    worker_connections 16384;
    multi_accept on;
    use epoll;
}

http {                                # ── L7 HTTP ──
    include mime.types;
    default_type application/octet-stream;

    upstream app { server 10.0.1.10:8080; }

    server {                          # virtual host
        listen 443 ssl;
        server_name api.example.com;

        location /v1/ {               # URI-scoped policy
            if ($request_method = POST) { ... }   # avoid — see gotchas
        }
    }
}

stream {                              # ── L4 TCP/UDP ──
    upstream pg { server 10.0.2.5:5432; }
    server { listen 5432; proxy_pass pg; }
}

Inheritance rules — the part people get wrong

Directive typeRuleExamples
ScalarChild overrides parent; otherwise inherited.proxy_read_timeout, client_max_body_size, root
ArrayChild replaces the entire array if it defines even one entry. No merging.add_header, proxy_set_header, fastcgi_param, set_real_ip_from, limit_req
ActionNot inherited at all — only applies where declared.rewrite, return, try_files, proxy_pass
The #1 production footgun
server {
    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header X-Frame-Options DENY always;

    location /api/ {
        add_header X-API-Version "2";   ← HSTS AND X-Frame-Options ARE NOW GONE for /api/
    }
}

Fix: repeat all headers in the child, or centralise them in an include snippets/security-headers.conf; that you include in every location, or use more_set_headers from headers_more which merges.

Variables

VariableMeaning / gotcha
$uriNormalized, decoded URI after internal rewrites. No query string.
$request_uriOriginal, raw, undecoded URI including query string. Use for logging/redirect fidelity.
$args / $arg_nameQuery string / a specific query param.
$hostHost header lowercased, or server_name if absent. $http_host is the raw header.
$remote_addrPeer IP — rewritten by realip module if configured.
$proxy_add_x_forwarded_for$http_x_forwarded_for, $remote_addr — appends, doesn't replace. Spoofable if you don't sanitize.
$request_timeTotal time from first byte read to last byte written — includes client upload & slow-client download.
$upstream_response_timeTime upstream took. Comma/colon-separated list if retries or multiple upstreams occurred.
$upstream_connect_timeTCP+TLS setup to upstream. 0.000 when a keepalive conn was reused — great signal for pool health.
$upstream_cache_statusMISS · HIT · EXPIRED · STALE · UPDATING · REVALIDATED · BYPASS
$request_idAuto-generated 32-hex-char ID. Propagate it as your trace correlation ID.
$connection / $connection_requestsConnection serial number / how many requests rode that keep-alive connection.
$ssl_protocol, $ssl_cipher, $ssl_session_reusedTLS posture per request — log these to prove TLS-version migration progress.
Staff-level diagnostic

$request_time - $upstream_response_time is your client-side latency. If that gap is large and upstream time is flat, the problem is the network/client (mobile users, slow uploads), not your app. This single derived metric ends a lot of "the API is slow" arguments.

04server & location matching

Step 1 — choosing the server block

  1. Filter by listen — most specific IP:port match wins (10.0.0.1:80 beats *:80).
  2. Among those, match server_name against the Host header, in this precedence:
    1. Exact: api.example.com
    2. Leading wildcard, longest first: *.example.com
    3. Trailing wildcard: www.example.*
    4. Regex, first match in config order: ~^(?<sub>.+)\.example\.com$
  3. No match → the default_server for that listen, else the first server block defined.
Always define an explicit default_server
server {                          # catch-all: unknown Host, direct IP hits, scanners
    listen 80 default_server;
    listen 443 ssl default_server;
    ssl_reject_handshake on;      # 1.19.4+: refuse TLS for unknown SNI
    server_name _;
    return 444;                     # nginx-special: close connection, no response
}

Without this, an attacker sending Host: internal-admin can land on whichever server block happens to be first — a classic Host-header routing bypass.

Step 2 — choosing the location

OrderSyntaxSemantics
1location = /exactExact match. If it hits, search stops immediately. Fastest — use for /healthz, /favicon.ico.
2location ^~ /prefix/Prefix match that, if it's the longest prefix, skips regex evaluation entirely.
3location /prefix/Plain prefix. Longest match is remembered but regexes get a chance to override.
4location ~ regex / ~* regexCase-sensitive / insensitive regex. First match in file order wins — order matters, length does not.
5fallbackThe longest stored prefix match from step 3.
Algorithm: 1. scan all prefix locations → remember the LONGEST match 2. if that match was declared with "=" → USE IT, stop 3. if that match was declared with "^~" → USE IT, skip regexes 4. scan regex locations in config order → FIRST match wins, use it 5. else → use the remembered prefix match
# Worked example — request: GET /images/logo.png
location /            { ... }   # prefix, len 1  → remembered
location /images/     { ... }   # prefix, len 8  → longest, remembered
location ~* \.(png|jpg)$ { ... } # regex        → EVALUATED, MATCHES → WINS

# Now add ^~ and the regex never runs:
location ^~ /images/  { ... }   # → WINS, regexes skipped
Performance note

Regex locations are evaluated linearly per request. On a config with 200 regex locations you're doing up to 200 PCRE runs per request. Restructure to prefix locations with ^~, or put the discriminator into map (hash lookup, O(1)).

root vs alias — the eternal confusion

# root: location path is APPENDED to root
location /static/ {
    root /var/www;          # /static/a.js  →  /var/www/static/a.js
}

# alias: location path is REPLACED by alias
location /static/ {
    alias /var/www/assets/;  # /static/a.js  →  /var/www/assets/a.js
}
alias traversal bug

If the location lacks a trailing slash but the alias has one — location /static { alias /var/www/assets/; } — then GET /static../etc/passwd resolves to /var/www/assets../etc/passwd/var/www/etc/passwd. Always keep trailing slashes symmetric between location and alias. This is a real, repeatedly-exploited CVE class.

map — the right tool for conditional logic

# map builds a hash table, evaluated lazily, no "if" needed
map $http_user_agent $is_bot {
    default          0;
    ~*(googlebot|bingbot|crawler)  1;
}

map $request_method $limit_key {
    default  "";                   # empty key = limit not applied
    POST     $binary_remote_addr;
    PUT      $binary_remote_addr;
}

map $http_upgrade $connection_upgrade {   # canonical websocket map
    default upgrade;
    ''      close;
}

# Multi-input decisions: chain maps or use a composite key
map "$host:$uri" $backend {
    default                 app_default;
    ~^api\.  api_pool;
}
"If Is Evil"

Inside location{}, only return and rewrite ... last are guaranteed safe inside if. Anything else produces undefined behaviour because if creates an implicit nested location with its own config context. Notorious example:

location /x {
    set $a 1;
    if ($http_x) { set $a 2; }   # creates nested ctx
    proxy_pass http://$a;             # may see the WRONG value / segfault paths
}

Replace with map, or try_files, or move the branch to the upstream. Say "I use map for value selection and if only for return" and you've signalled seniority.

05Static content serving

server {
    root /var/www/site;

    # ── zero-copy path: file → socket without userspace ──
    sendfile       on;
    tcp_nopush     on;    # with sendfile: fill full packets before sending (TCP_CORK)
    tcp_nodelay    on;    # disable Nagle on keepalive conns → low latency for small responses

    # ── cache open file descriptors + stat() results ──
    open_file_cache          max=200000 inactive=20s;
    open_file_cache_valid    30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors   on;   # cache 404s too

    location ^~ /assets/ {
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
        gzip_static on;         # serve pre-built app.js.gz if present
        brotli_static on;       # serve pre-built app.js.br if present
    }

    # ── SPA fallback ──
    location / {
        try_files $uri $uri/ /index.html;
    }
    location = /index.html {
        add_header Cache-Control "no-cache";   # revalidate the shell every load
    }
}

Compression

gzip on;
gzip_comp_level 5;          # 5 is the knee; 9 costs 3× CPU for ~2% size
gzip_min_length 256;        # below MTU, compression is a net loss
gzip_proxied any;
gzip_vary on;              # MUST — emits "Vary: Accept-Encoding" for correct CDN caching
gzip_types text/plain text/css application/json application/javascript
           application/xml image/svg+xml application/wasm;
# NOTE: text/html is ALWAYS compressed and must not be listed

brotli on;                # dynamic module; ~15-20% better than gzip for text
brotli_comp_level 4;
Don't compress everything

Compressing already-compressed formats (jpg, png, mp4, zip, woff2) burns CPU for zero gain. And historically, compressing responses that mix secrets with attacker-controlled input enables BREACH/CRIME-style attacks — mitigate with CSRF tokens that change per request, or disable compression on sensitive JSON endpoints.

Byte-range & large files

location /downloads/ {
    aio threads;
    sendfile on;
    sendfile_max_chunk 2m;      # prevents one huge file monopolising a worker
    max_ranges 1;                # mitigate multi-range amplification DoS
    limit_rate_after 10m;        # first 10MB at full speed…
    limit_rate 2m;               # …then throttle to 2MB/s per connection
}

06Reverse proxy deep dive

The single most-asked config detail: proxy_pass trailing slash

locationproxy_passRequest /api/users/1 becomes
/api/http://backend (no URI)/api/users/1 — full original path passed
/api/http://backend/ (URI = "/")/users/1 — the matched prefix is stripped
/api/http://backend/v2//v2/users/1 — prefix replaced
/api/http://backend/v2 (no trailing /)/v2users/1 — concatenated, almost always a bug
Rule

If proxy_pass contains any URI part (even just /), the matched location prefix is replaced by it. If it contains only scheme+host, the original URI passes through untouched.

Two extra traps: (1) a proxy_pass with a URI part is illegal inside a regex location or a named location — you must use rewrite instead. (2) Using a variable in proxy_pass (proxy_pass http://$backend;) changes behaviour: it disables startup-time resolution, requires a resolver, and drops the URI unless you append one explicitly.

A production-grade proxy block, annotated

location /api/ {
    proxy_pass http://api_pool;

    # ── protocol ────────────────────────────────────────────
    proxy_http_version 1.1;            # REQUIRED for keepalive + chunked + websockets
    proxy_set_header Connection "";    # clear inherited "close" so pooling works

    # ── identity headers ────────────────────────────────────
    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host  $host;
    proxy_set_header X-Forwarded-Port  $server_port;
    proxy_set_header X-Request-ID      $request_id;   # trace correlation

    # ── timeouts (tune per endpoint, not globally) ──────────
    proxy_connect_timeout 2s;           # TCP handshake only — keep SHORT
    proxy_send_timeout    30s;          # between successive writes to upstream
    proxy_read_timeout    30s;          # between successive reads — NOT total time

    # ── buffering ───────────────────────────────────────────
    proxy_buffering       on;
    proxy_buffer_size     8k;           # must hold the full response HEADER
    proxy_buffers         16 8k;        # body buffers, per connection
    proxy_busy_buffers_size 16k;
    proxy_max_temp_file_size 256m;     # 0 = never spill to disk

    # ── request body ────────────────────────────────────────
    client_max_body_size  25m;          # else 413
    client_body_buffer_size 256k;
    proxy_request_buffering on;         # off = stream uploads straight through

    # ── failover ────────────────────────────────────────────
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 5s;

    # ── hide leaky upstream headers ─────────────────────────
    proxy_hide_header X-Powered-By;
    proxy_hide_header Server;
}

Buffering: the concept interviewers dig into

proxy_buffering ON (default) upstream ──fast──► [nginx buffers] ──slow──► client ✔ upstream connection freed as soon as response is buffered ✔ protects app workers from slow clients (Slowloris on the read side) ✔ required for proxy_cache ✘ adds latency to Time-To-First-Byte for streaming ✘ spills to disk (proxy_temp_path) for large bodies → I/O proxy_buffering OFF upstream ──────────► nginx ──────────► client (synchronous, chunk by chunk) ✔ true streaming: SSE, log tailing, LLM token streams, long-poll ✘ a slow client holds an upstream worker/thread hostage ✘ caching is impossible
# Streaming endpoint (SSE / LLM tokens)
location /stream {
    proxy_pass http://app;
    proxy_buffering off;
    proxy_cache off;
    proxy_read_timeout 1h;
    chunked_transfer_encoding on;
    add_header X-Accel-Buffering no;   # upstream can also send this header to opt out
}
The X-Accel-Buffering: no trick

Your upstream can disable buffering per response by emitting this header — so you keep buffering on globally and let streaming endpoints opt out. Cleaner than maintaining a list of streaming paths in nginx config.

"upstream sent too big header"

A 502 with upstream sent too big header while reading response header from upstream in error.log means the response headers exceeded proxy_buffer_size. Common cause: a huge Set-Cookie (session bloat) or many CORS/debug headers. Fix: raise proxy_buffer_size 16k; and ask why your headers are 8 KB.

Timeouts — the mental model

DirectiveClock resets when…Typical value
client_header_timeoutReading request headers10s (Slowloris defence)
client_body_timeoutEach body read op10–30s
send_timeoutEach write to client10–60s
keepalive_timeoutIdle client connection65s (must exceed LB idle timeout)
proxy_connect_timeoutN/A — hard cap, max 75s1–3s
proxy_read_timeoutEach read from upstreamPer-endpoint: 5s API, 300s report
Read timeout is not a total-time budget

proxy_read_timeout 30s does not cap a request at 30s. If the upstream dribbles one byte every 29 seconds, the request runs forever. For a true deadline you need the app to enforce it, or use limit_rate/Lua timers. Interviewers love this one.

Idle-timeout mismatch → phantom 502s

If your AWS ALB/NLB idle timeout is 60s and NGINX keepalive_timeout is 60s, you get a race: NGINX closes a pooled connection at the exact moment the LB sends a request on it → intermittent 502s at low traffic. Rule: NGINX keepalive_timeout > LB idle timeout at every hop, and the same going upstream (NGINX's upstream keepalive_timeout must be shorter than the app server's idle timeout).

Non-HTTP upstreams

location ~ \.php$ {
    include fastcgi_params;
    fastcgi_pass unix:/run/php-fpm.sock;      # unix socket beats TCP on localhost
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_buffers 16 16k;
}

location /grpc/ { grpc_pass grpc://grpc_pool; }
location /uwsgi/ { include uwsgi_params; uwsgi_pass unix:/run/uwsgi.sock; }
location /scgi/  { include scgi_params;  scgi_pass  127.0.0.1:9000; }

DNS resolution — a genuine production hazard

# PROBLEM: this resolves ONCE at startup/reload and caches forever.
# If the upstream is an ELB/ClusterIP whose IP rotates → traffic to a dead IP.
upstream api { server api.internal.example.com:443; }

# FIX A (OSS): force re-resolution via a variable + resolver
resolver 10.0.0.2 valid=10s ipv6=off;
location /api/ {
    set $api_host api.internal.example.com;
    proxy_pass https://$api_host$request_uri;   # note: must re-append URI
}

# FIX B (Plus): resolve=… on the upstream server, keeps pooling + LB
upstream api {
    zone api 64k;
    server api.internal.example.com:443 resolve;
    keepalive 64;
}
Trade-off to state out loud

Fix A costs you the upstream block — no load-balancing policy, no keepalive pool, no passive health tracking; you're leaning on DNS round-robin. The clean answer at staff level is: don't resolve DNS in NGINX at all. Use service discovery (Consul-template, k8s Endpoints controller, or the NGINX Plus API) to render/patch the upstream block, so NGINX always has concrete IPs.

07Load balancing

upstream api_pool {
    zone api_pool 64k;          # shared memory → state shared ACROSS WORKERS

    least_conn;                  # algorithm (declare BEFORE servers)

    server 10.0.1.10:8080 weight=3 max_fails=3 fail_timeout=10s max_conns=200;
    server 10.0.1.11:8080 weight=1;
    server 10.0.1.12:8080 backup;      # only used when all primaries are down
    server 10.0.1.13:8080 down;        # admin-disabled, kept for hash stability
    server 10.0.1.14:8080 slow_start=30s;  /* PLUS: ramp weight after recovery */

    keepalive 64;                 # idle conns kept PER WORKER
    keepalive_requests 1000;
    keepalive_timeout 60s;
}
Without zone, everything is per-worker

No zone ⇒ each worker keeps its own copy of upstream state. With 16 workers and max_conns=10, you actually allow 160 concurrent connections to that backend. Same for max_fails — a backend must fail 3× per worker to be fully ejected. Always declare zone in OSS 1.9.0+.

Algorithms

DirectiveHow it picksUse whenWatch out for
(default) round robinWeighted RRHomogeneous, short, uniform requestsIgnores actual load; one slow node still gets its share
least_connFewest active connections (weight-adjusted)Variable request durations — the common real-world caseA node that fails instantly shows 0 conns and attracts all traffic ("black hole"). Pair with passive health checks.
ip_hashHash of first 3 octets of IPv4Legacy sticky sessionsNAT/CGNAT ⇒ huge groups land on one node. Adding a node reshuffles nearly everything.
hash $keyHash of an arbitrary keySharding by tenant/user/URLSame reshuffle problem without consistent
hash $key consistentKetama ringCache-tier fan-outOnly ~1/N keys move when a node changes — this is the answer for cache affinity
random two least_connPick 2 at random, take the less loadedMany NGINX instances in front of shared backendsThe "power of two choices" — avoids the herd effect where every LB independently picks the same "least loaded" node
least_time PLUSLowest avg response time × connsHeterogeneous hardwarePlus only
Staff-level nuance: why random two exists

With one LB, least_conn is optimal. With 50 NGINX instances each independently computing "least loaded", they all pick the same idle backend simultaneously and hammer it — then all switch away together. Oscillation. Power-of-two-choices injects randomness that provably keeps max load within O(log log n) of average while eliminating the herd. Same reasoning as Envoy's default LB policy.

Upstream keepalive — the biggest single latency win

upstream app { server ...; keepalive 64; }

location / {
    proxy_pass http://app;
    proxy_http_version 1.1;         ← without this, HTTP/1.0, no keepalive
    proxy_set_header Connection ""; ← without this, "Connection: close" is forwarded
}

Missing either line silently disables pooling. Symptom: $upstream_connect_time is non-zero on every request, and you see TIME_WAIT socket buildup on the NGINX box (ss -s). A new TCP+TLS handshake per request adds 1–3 RTTs — often 5–40 ms of pure waste.

Sizing

keepalive is per worker. With worker_processes 16; keepalive 64; you may hold up to 1024 idle connections to that upstream from one NGINX box. Multiply by the number of NGINX boxes and check it against the backend's max connections / file descriptor limit. Rule of thumb: keepalive ≈ 2 × (peak RPS × avg upstream latency) / worker_processes.

Session persistence

# OSS: hash on an app-provided cookie or header
upstream app {
    hash $cookie_sessionid consistent;
    server ...;
}

# PLUS: nginx issues and tracks its own cookie
upstream app {
    sticky cookie srv_id expires=1h domain=.example.com path=/ httponly secure;
}
Say this

"Sticky sessions are a smell — they make deploys, autoscaling, and node failure user-visible. My default is stateless backends with session state in Redis/JWT. I'd only use stickiness for a legacy app I can't refactor, for local in-process caches where affinity is a performance optimization (not a correctness requirement), or for WebSocket connections that are inherently pinned."

08Health checks & resilience

Passive (OSS)

upstream app {
    zone app 64k;
    # 3 failures within a 10s window → mark down for 10s, then try ONE probe request
    server 10.0.1.10:8080 max_fails=3 fail_timeout=10s;
}
# What counts as a "failure" is defined by:
proxy_next_upstream error timeout http_502 http_503 http_504;

Note the coupling: fail_timeout is both the window in which failures are counted and the ejection duration. And recovery is "try one live request" — real users pay for the probe.

Active (Plus / Angie)

location / {
    proxy_pass http://app;
    health_check interval=5s fails=3 passes=2 uri=/healthz match=ok;
}
match ok {
    status 200;
    header Content-Type ~ application/json;
    body ~ '"status":\s*"ok"';
}
Health-check design — the real interview content
  • Liveness ≠ readiness. Liveness = "is the process alive" (restart me if not). Readiness = "should I get traffic" (drain me if not). NGINX should probe readiness.
  • Don't make health checks deep. If /healthz checks the database, a single DB blip makes every node report unhealthy simultaneously → NGINX ejects the entire pool → total outage, when degraded service was possible. Check only what this instance controls.
  • Graceful shutdown sequence: on SIGTERM the app should (1) flip readiness to fail, (2) keep serving in-flight requests for drain_time > probe interval × failure threshold, (3) then exit. Without step 2 you drop requests on every deploy.
  • Retry safety. proxy_next_upstream defaults include error timeout. A timeout on a POST /charge may have already succeeded upstream — retrying double-charges. Set proxy_next_upstream error timeout non_idempotent; only if you mean it, and prefer idempotency keys in the app.
  • Retry amplification. With proxy_next_upstream_tries 3, a backend brownout turns 1× traffic into 3× traffic exactly when the tier is already struggling. Cap tries at 2, add proxy_next_upstream_timeout, and rely on the client for anything more.

Circuit-breaker-ish behaviour

upstream app {
    zone app 64k;
    server a:8080 max_conns=100 max_fails=2 fail_timeout=15s;
    server b:8080 max_conns=100 max_fails=2 fail_timeout=15s;
    queue 100 timeout=3s;    /* PLUS: queue instead of instant 502 when max_conns is hit */
}

max_conns is your bulkhead: it caps concurrency per backend so one slow endpoint can't consume the whole app's thread pool. In OSS without queue, exceeding it returns 502 immediately — so pair it with a friendly error_page.

Graceful degradation

server {
    proxy_intercept_errors on;
    error_page 500 502 503 504 /maintenance.html;

    location = /maintenance.html {
        root /var/www/errors;
        internal;
        add_header Retry-After 30 always;
    }

    # Serve stale cache instead of an error — best-in-class degradation
    location / {
        proxy_cache zone1;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_pass http://app;
    }
}

09Caching

# ── http{} context ──────────────────────────────────────────
proxy_cache_path /var/cache/nginx/api
                 levels=1:2              # dir fan-out: /a/bc/<md5> — avoids huge dirs
                 keys_zone=api_cache:100m  # ~800k keys (≈8k keys per MB)
                 max_size=20g              # disk ceiling (soft; manager enforces)
                 inactive=60m              # evict if UNREQUESTED this long (independent of TTL)
                 use_temp_path=off;         # write in place, avoid cross-device copy

location /api/ {
    proxy_cache api_cache;
    proxy_cache_key "$scheme$request_method$host$request_uri";
    proxy_cache_methods GET HEAD;

    # ── TTLs per status ──
    proxy_cache_valid 200 301 302  10m;
    proxy_cache_valid 404          1m;    # negative caching stops 404 storms
    proxy_cache_valid any          1m;

    # ── stampede control ──
    proxy_cache_lock on;                 # only ONE request populates a cold key
    proxy_cache_lock_timeout 5s;
    proxy_cache_lock_age 5s;
    proxy_cache_min_uses 2;               # don't cache one-hit-wonders (protects disk)

    # ── stale serving = availability ──
    proxy_cache_use_stale error timeout updating
                          http_500 http_502 http_503 http_504;
    proxy_cache_background_update on;     # serve stale NOW, refresh async
    proxy_cache_revalidate on;            # conditional GET w/ If-Modified-Since / ETag

    # ── bypass rules ──
    proxy_cache_bypass $http_authorization $cookie_session $arg_nocache;
    proxy_cache_no_cache $http_authorization $cookie_session;

    # ── ignore upstream cache-busting headers (use with care) ──
    proxy_ignore_headers Cache-Control Expires Set-Cookie X-Accel-Expires;

    add_header X-Cache-Status $upstream_cache_status always;
    proxy_pass http://api_pool;
}

proxy_cache_bypass vs proxy_cache_no_cache

DirectiveEffect when the variable is non-empty and ≠ "0"
proxy_cache_bypassDon't read from cache (go to upstream) — but the response may still be stored.
proxy_cache_no_cacheDon't write to cache — but a cached copy may still be served.

You almost always want both for authenticated traffic. Setting only bypass is a well-known cause of cache poisoning where one user's private response gets served to everyone.

Cache key design = the whole game
  • Omit $host from the key on a multi-tenant proxy → tenant A sees tenant B's data.
  • Include $http_cookie → cardinality explosion, ~0% hit rate.
  • Ignore Vary: Accept-Encoding → gzip body sent to a client that didn't ask for it (garbled page).
  • Query-param order (?a=1&b=2 vs ?b=2&a=1) creates two keys. Normalize with map/Lua or accept the miss.
  • Unkeyed attacker-controlled headers that the upstream reflects (e.g. X-Forwarded-Host) → web cache poisoning. Either key on them or strip them at the edge.

Microcaching — the highest-ROI trick

# Cache even "dynamic, personalized" pages for ONE second.
# 5,000 rps on a hot endpoint → 1 rps hits your app. 99.98% offload.
location / {
    proxy_cache micro;
    proxy_cache_valid 200 1s;
    proxy_cache_lock on;
    proxy_cache_use_stale updating error timeout;
    proxy_cache_background_update on;
    proxy_pass http://app;
}

The insight: for anonymous, read-heavy traffic, one second of staleness is invisible to users but collapses a traffic spike by three orders of magnitude. This is the standard answer to "how would you survive a front-page traffic spike without scaling the app tier?"

Cache stampede / thundering herd

Without proxy_cache_lock, key expires at T: T+0ms 5,000 concurrent requests all see EXPIRED T+1ms 5,000 requests forwarded to upstream simultaneously T+2s upstream is saturated → timeouts → nothing gets cached → repeat With proxy_cache_lock on + use_stale updating + background_update: T+0ms request #1 acquires lock, goes upstream requests #2..5000 are served the STALE copy immediately T+80ms fresh response stored, lock released → upstream saw exactly 1 request

Purging

# PLUS: native
location ~ /purge(/.*) {
    allow 10.0.0.0/8; deny all;
    proxy_cache_purge api_cache "$scheme$request_method$host$1";
}

# OSS options:
#  a) ngx_cache_purge third-party module
#  b) compute md5 of the cache key and rm the file:
#     echo -n "httpsGETexample.com/path" | md5sum
#     → /var/cache/nginx/api/<last-char>/<2 chars before>/<md5>
#  c) versioned cache keys — bump a $cache_version prefix on deploy (cleanest)
Preferred answer

"I avoid purge entirely. I put a version token in the cache key (build SHA, or a per-tenant epoch stored in a map) so a 'purge' is a key-namespace rotation — instantaneous, atomic, and works across an arbitrary fleet of cache nodes with no fan-out API call. Old entries age out via inactive."

10TLS, HTTP/2 and HTTP/3

server {
    listen 443 ssl reuseport;
    listen [::]:443 ssl;
    http2 on;                                # 1.25.1+ (was `listen … http2`)

    listen 443 quic reuseport;                 # HTTP/3 over QUIC (UDP), 1.25.0+
    add_header Alt-Svc 'h3=":443"; ma=86400';  # advertise h3 to clients

    ssl_certificate     /etc/ssl/fullchain.pem;   # leaf + intermediates, in order
    ssl_certificate_key /etc/ssl/privkey.pem;

    # ── protocol & ciphers ──
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;             # modern advice: trust client order (TLS1.3)
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:
                ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:
                ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
    ssl_ecdh_curve X25519:prime256v1;

    # ── session resumption (huge CPU + RTT saver) ──
    ssl_session_cache   shared:SSL:50m;         # ~200k sessions; SHARED across workers
    ssl_session_timeout 1d;
    ssl_session_tickets on;
    ssl_session_ticket_key /etc/ssl/ticket1.key;  # rotate; share across the fleet
    ssl_session_ticket_key /etc/ssl/ticket2.key;  # older key, decrypt-only

    # ── OCSP stapling: server fetches revocation proof so clients don't have to ──
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/ssl/chain.pem;
    resolver 1.1.1.1 8.8.8.8 valid=300s;

    # ── TLS 1.3 0-RTT: 1 fewer RTT, but REPLAYABLE ──
    ssl_early_data on;
    proxy_set_header Early-Data $ssl_early_data;   # app must reject non-idempotent 0-RTT

    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
}

Handshake cost — why termination is worth it

TLS 1.2 full handshake : 2 RTT + RSA/ECDSA sign (~3-5 ms CPU per handshake) TLS 1.3 full handshake : 1 RTT TLS 1.3 resumed (PSK) : 0 extra RTT TLS 1.3 + 0-RTT early data : data rides ON the first flight Session cache : server-side state, shared:SSL zone, NOT shared across hosts Session ticket: client holds encrypted state → stateless, works fleet-wide BUT if the ticket key never rotates, forward secrecy is broken
Two real incidents to be able to describe

Ticket keys. If you don't set ssl_session_ticket_key, each NGINX instance generates its own random key and never rotates it while it runs. Consequence: (a) no resumption across instances behind an LB → handshake storms; (b) a long-lived key that, if stolen, decrypts all past traffic. Fix: distribute rotating key files (e.g. hourly, keep 2–3 generations) via your secrets system.

0-RTT replay. An attacker can capture and replay early data. Never let a 0-RTT request perform a state-changing action — gate on the Early-Data: 1 header in the app and return 425 Too Early for non-idempotent methods.

Multiple certs & SNI

# Dual RSA + ECDSA on one server block — client negotiates the best it supports
ssl_certificate     /etc/ssl/ecdsa.crt;   ssl_certificate_key /etc/ssl/ecdsa.key;
ssl_certificate     /etc/ssl/rsa.crt;     ssl_certificate_key /etc/ssl/rsa.key;

# Cert selection by SNI variable (avoids thousands of server blocks)
ssl_certificate     /etc/ssl/certs/$ssl_server_name.crt;
ssl_certificate_key /etc/ssl/certs/$ssl_server_name.key;
# (variables in ssl_certificate disable the cert cache → use ssl_certificate_cache in 1.27.4+)

mTLS (client certificates)

ssl_client_certificate /etc/ssl/ca.pem;
ssl_verify_client on;            # or `optional` to allow both and branch
ssl_verify_depth 2;
ssl_crl /etc/ssl/revoked.crl;

proxy_set_header X-Client-DN     $ssl_client_s_dn;
proxy_set_header X-Client-Verify $ssl_client_verify;   # SUCCESS | FAILED:reason | NONE
proxy_set_header X-Client-Serial $ssl_client_serial;

HTTP/2 & HTTP/3 notes worth stating

AspectHTTP/1.1HTTP/2HTTP/3 (QUIC)
TransportTCPTCPUDP
MultiplexingNone (6 conns/host)Streams over 1 connStreams over 1 conn
Head-of-line blockingApp layerStill at TCP layer — one lost packet stalls all streamsEliminated (per-stream loss recovery)
Header compressionNoneHPACKQPACK
HandshakeTCP+TLS: 2–3 RTT2–3 RTT1 RTT, or 0-RTT resumed
Conn migrationNoNoYes (Connection ID survives IP change — wifi→cellular)
Two things people get wrong

1. NGINX speaks HTTP/2 to clients, but by default proxies to upstreams over HTTP/1.1. That's usually fine and actually desirable (connection pooling). Use grpc_pass when you need real h2 upstream.

2. Domain sharding, sprite sheets, and inlining are anti-patterns under HTTP/2 — they defeat multiplexing and caching. If you migrate to h2 without removing sharding you can get slower.

# HTTP/2 tuning
http2_max_concurrent_streams 128;
keepalive_requests 1000;         # h2 counts each STREAM as a request — raise this
# CVE-2023-44487 "Rapid Reset": mitigated in 1.25.3; also cap streams + keepalive_time

11Rate limiting & traffic shaping

Three independent limiters

# ── http{} ─────────────────────────────────────────────────
# 1. REQUEST RATE — leaky bucket, per key
limit_req_zone $binary_remote_addr zone=perip:10m  rate=10r/s;
limit_req_zone $server_name        zone=persrv:10m rate=1000r/s;
limit_req_zone $http_x_api_key     zone=perkey:20m rate=100r/s;

# 2. CONCURRENT CONNECTIONS
limit_conn_zone $binary_remote_addr zone=conn_perip:10m;

# 3. BANDWIDTH — per connection
#    (limit_rate / limit_rate_after, set in location or via $limit_rate)

limit_req_status  429;      # default is 503 — 429 is semantically correct
limit_conn_status 429;
limit_req_log_level warn;

location /api/ {
    limit_req  zone=perip burst=20 nodelay;
    limit_req  zone=persrv burst=200;      # multiple limits ALL apply; strictest wins
    limit_conn conn_perip 20;
    limit_rate 1m;
}

How the leaky bucket actually behaves

rate=10r/s → nginx converts this to ONE REQUEST EVERY 100ms. It is NOT "10 requests then wait a second". burst=20 queue up to 20 excess requests, RELEASE THEM AT 100ms EACH (client sees increasing latency — request 20 waits 2s) burst=20 nodelay allow 20 excess requests IMMEDIATELY, but they still consume bucket capacity that refills at 10/s → true burst tolerance burst=20 delay=10 first 10 excess pass immediately, next 10 are throttled (hybrid: instant for normal bursts, brake for abuse) no burst request #2 arriving within 100ms of #1 → 503/429 instantly. This is almost always wrong for browsers: one page load fires 20 parallel asset requests.
Choosing the key
  • $binary_remote_addr not $remote_addr — 4 bytes vs 15, ~4× more entries per MB of zone.
  • Behind a CDN/LB, $remote_addr is the proxy. You must configure real_ip first, or key on $http_cf_connecting_ip / the correct XFF position.
  • For APIs, key on the API key or tenant ID, not IP — IP-based limits punish shared NAT and don't stop distributed abuse.
  • Use map to produce an empty key to exempt traffic (empty key ⇒ limit not applied). This is how you whitelist internal ranges.
# Whitelist internal networks + trusted partners from the limiter
geo $limit_exempt {
    default          0;
    10.0.0.0/8       1;
    192.168.0.0/16   1;
}
map $limit_exempt $limit_key {
    0 $binary_remote_addr;   # limited
    1 "";                    # exempt
}
limit_req_zone $limit_key zone=perip:10m rate=10r/s;

Tiered limits by plan

map $http_x_api_key $plan {
    default       free;
    ~^pk_pro_     pro;
    ~^pk_ent_     ent;
}
map $plan $free_key { free $http_x_api_key; default ""; }
map $plan $pro_key  { pro  $http_x_api_key; default ""; }

limit_req_zone $free_key zone=free:10m rate=10r/s;
limit_req_zone $pro_key  zone=pro:10m  rate=500r/s;

location /api/ {
    limit_req zone=free burst=20 nodelay;
    limit_req zone=pro  burst=1000 nodelay;
}
Limits of NGINX rate limiting — say this unprompted

NGINX's limiter is per-instance. With 10 edge nodes and rate=10r/s, a client can actually do ~100 r/s. Options: (a) divide the limit by fleet size and accept imprecision, (b) put the limiter behind a consistent-hash L4 layer so a given key always lands on the same NGINX node, (c) use a shared counter (Redis via OpenResty / Envoy's global ratelimit service) — at the cost of a network round trip per request, (d) accept NGINX as coarse DoS protection and enforce exact quotas in the application where you already have the tenant context.

Also note NGINX returns 429 with no Retry-After or X-RateLimit-* headers by default. Add them: add_header Retry-After 1 always; — API consumers need them.

12Security hardening & WAF

Baseline hardening

server_tokens off;                       # hide version in Server header + error pages
more_clear_headers Server;                # headers_more: remove it entirely

# Body / header size caps (DoS surface)
client_max_body_size        10m;
client_body_buffer_size     128k;
client_header_buffer_size   1k;
large_client_header_buffers 4 8k;
client_body_timeout         10s;
client_header_timeout       10s;

# Method allow-list
if ($request_method !~ ^(GET|HEAD|POST|PUT|PATCH|DELETE|OPTIONS)$) { return 405; }

# Block hidden files, backups, VCS
location ~ /\.(?!well-known) { deny all; access_log off; }
location ~* \.(bak|old|sql|env|git|swp)$ { deny all; }

# Security headers — repeat in EVERY location that sets any add_header
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
add_header X-Content-Type-Options    "nosniff" always;
add_header X-Frame-Options           "DENY" always;
add_header Referrer-Policy           "strict-origin-when-cross-origin" always;
add_header Content-Security-Policy    "default-src 'self'; frame-ancestors 'none'" always;
add_header Permissions-Policy         "geolocation=(), camera=(), microphone=()" always;

real_ip — get this wrong and every IP-based control is bypassable

set_real_ip_from 10.0.0.0/8;         # ONLY list proxies you actually control
set_real_ip_from 173.245.48.0/20;    # e.g. Cloudflare ranges
real_ip_header   X-Forwarded-For;
real_ip_recursive on;                 # walk XFF right-to-left, skipping trusted IPs
The bypass

If you trust 0.0.0.0/0 or blindly read the leftmost XFF entry, any client can send X-Forwarded-For: 10.0.0.1 and impersonate an internal address — defeating allow/deny, rate limits, geo rules, and audit logs. Correct approach:

  • Trust only your known proxy CIDRs in set_real_ip_from.
  • Enable real_ip_recursive on so NGINX walks from the right, discarding trusted hops, and stops at the first untrusted address — that's the real client.
  • At the outermost edge, overwrite rather than append: proxy_set_header X-Forwarded-For $remote_addr; so client-supplied values are discarded.
  • Better still, standardise on RFC 7239 Forwarded, or a signed internal header.

Access control

location /admin/ {
    allow 10.0.0.0/8;
    allow 2001:db8::/32;
    deny all;
    auth_basic "restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;
    satisfy all;      # require BOTH ip and password (default). `any` = either.
}

WAF options

OptionNotes
ModSecurity 3 + OWASP CRSThe classic. Runs as an NGINX dynamic module. Powerful but CPU-heavy and false-positive-prone; expect a long tuning period in DetectionOnly mode first.
NGINX App Protect PLUSF5's commercial WAF, signature + behavioural, lower overhead than ModSec.
CorazaGo reimplementation of ModSecurity's rule engine; used via a connector or in Envoy/Caddy.
CDN-layer WAFCloudflare/AWS WAF. Usually the right answer — blocks upstream of your bandwidth and your CPU.
load_module modules/ngx_http_modsecurity_module.so;
server {
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/main.conf;
}
Position on WAFs

"A WAF is a compensating control, not a fix. I run it in detection mode first, tune out false positives against real traffic, and treat any rule that fires in production as a bug report for the application. I'd rather spend the CPU on rate limiting and bot detection at the CDN and fix injection at the ORM layer."

Slowloris and friends

AttackMechanismNGINX mitigation
SlowlorisMany connections, headers sent 1 byte at a timeclient_header_timeout 10s, limit_conn. NGINX is inherently resilient (cheap connections) but not immune.
Slow POST (R-U-Dead-Yet)Body dribbled slowlyclient_body_timeout, client_max_body_size, proxy_request_buffering on
Slow readClient advertises tiny TCP windowsend_timeout, proxy_buffering on, limit_conn
HTTP/2 Rapid ResetOpen+RST_STREAM flood, unbounded workUpgrade to ≥1.25.3, http2_max_concurrent_streams, keepalive_requests
Range amplificationRange: bytes=0-,0-,0-… thousands of timesmax_ranges 1;
Request smugglingCL/TE desync between proxy and backendKeep NGINX current; ensure backend rejects duplicate/conflicting Content-Length/Transfer-Encoding; avoid mixing proxies with different parsers

13Auth at the edge

auth_request — delegate authz to a service

server {
    location /api/ {
        auth_request /_authz;

        # pull identity out of the auth response and pass it downstream
        auth_request_set $user_id $upstream_http_x_user_id;
        auth_request_set $scopes  $upstream_http_x_scopes;
        proxy_set_header X-User-Id $user_id;
        proxy_set_header X-Scopes  $scopes;

        proxy_pass http://api_pool;
    }

    location = /_authz {
        internal;
        proxy_pass http://authz_service;
        proxy_pass_request_body off;               # auth svc doesn't need the body
        proxy_set_header Content-Length "";
        proxy_set_header X-Original-URI    $request_uri;
        proxy_set_header X-Original-Method $request_method;
        proxy_set_header Authorization     $http_authorization;
    }

    error_page 401 = @login;
    location @login { return 302 https://auth.example.com/login?rd=$scheme://$host$request_uri; }
}

Semantics: 2xx from the subrequest → allow. 401/403 → deny with that status. Anything else → 500.

Cost

Every request now costs an extra internal HTTP round trip. Mitigate with: keepalive to the authz service, a short-TTL cache of authz decisions keyed on the token (proxy_cache on /_authz with a 5–30s TTL), or moving to stateless JWT verification so no subrequest is needed at all.

JWT validation

/* NGINX PLUS — native */
location /api/ {
    auth_jwt "api";
    auth_jwt_key_file /etc/nginx/jwks.json;      # or auth_jwt_key_request for JWKS URL
    proxy_set_header X-Sub $jwt_claim_sub;
    proxy_pass http://api;
}
# OSS — njs or OpenResty
js_import auth.js;
location /api/ {
    auth_request /_jwt;
    proxy_pass http://api;
}
location = /_jwt { internal; js_content auth.verify; }

CORS done correctly

map $http_origin $cors_origin {
    default "";
    ~^https://(app|admin)\.example\.com$  $http_origin;   # echo only allowed origins
}

location /api/ {
    if ($request_method = OPTIONS) {
        add_header Access-Control-Allow-Origin      $cors_origin always;
        add_header Access-Control-Allow-Methods     "GET, POST, PUT, DELETE, OPTIONS" always;
        add_header Access-Control-Allow-Headers     "Authorization, Content-Type" always;
        add_header Access-Control-Allow-Credentials "true" always;
        add_header Access-Control-Max-Age           86400 always;
        add_header Content-Length 0;
        return 204;
    }
    add_header Access-Control-Allow-Origin      $cors_origin always;
    add_header Access-Control-Allow-Credentials "true" always;
    add_header Vary Origin always;    ← REQUIRED or caches will serve the wrong ACAO
    proxy_pass http://api;
}
Two CORS landmines

Access-Control-Allow-Origin: * together with Allow-Credentials: true is rejected by browsers — you must echo a specific origin. And if you echo $http_origin without a Vary: Origin, any cache (NGINX's own, or a CDN) will serve origin A's ACAO header to origin B.

14Performance tuning

Capacity arithmetic

max_clients ≈ worker_processes × worker_connections But a PROXIED request uses TWO connection slots (client + upstream): max_proxied ≈ (worker_processes × worker_connections) / 2 worker_connections also counts: listening sockets, resolver sockets, upstream keepalive idle conns, and internal subrequests (auth_request!). Hard ceiling = worker_rlimit_nofile (per worker), which must be ≥ worker_connections and ≤ the systemd LimitNOFILE for the unit.
worker_processes auto;
worker_cpu_affinity auto;          # pin workers to cores → better cache locality
worker_rlimit_nofile 200000;
worker_shutdown_timeout 30s;      # cap how long old workers linger after reload

events {
    worker_connections 65535;
    multi_accept on;               # drain the whole accept queue per loop iteration
    use epoll;
}

http {
    sendfile on; tcp_nopush on; tcp_nodelay on;
    keepalive_timeout 75s;
    keepalive_requests 10000;
    reset_timedout_connection on;   # RST instead of FIN → free memory immediately
    server_names_hash_bucket_size 128;
    types_hash_max_size 4096;
    access_log /var/log/nginx/access.log main buffer=64k flush=5s;
    open_file_cache max=200000 inactive=20s;
}

Kernel tuning (/etc/sysctl.conf)

net.core.somaxconn = 65535              # accept queue depth (listen backlog ceiling)
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535  # ephemeral ports for upstream conns
net.ipv4.tcp_tw_reuse = 1                # reuse TIME_WAIT for OUTGOING conns (safe)
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_slow_start_after_idle = 0   # don't reset cwnd on idle keepalives
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq
net.ipv4.tcp_fastopen = 3
fs.file-max = 2097152

# and in nginx:
listen 443 ssl backlog=65535;         # must be ≤ somaxconn
Never enable tcp_tw_recycle

It breaks connections from clients behind NAT (timestamp-based rejection) and was removed in Linux 4.12. tcp_tw_reuse is the safe one, and it only affects outbound connections — which is exactly the NGINX→upstream direction.

Ephemeral port exhaustion

A proxy opening a fresh TCP connection per upstream request burns one ephemeral port per request, each lingering in TIME_WAIT for 60s. With ~28k usable ports that caps you around 470 rps per upstream IP. Symptoms: connect() failed (99: Cannot assign requested address).

TLS CPU

Tuning checklist by symptom

SymptomLikely causeAction
High CPU, low throughputTLS handshakes; gzip level too high; regex locationsResumption, ECDSA, gzip_comp_level 4-5, replace regex with map
worker_connections are not enoughSlot exhaustionRaise worker_connections + worker_rlimit_nofile; remember proxying doubles usage
Too many open filesfd limitworker_rlimit_nofile, systemd LimitNOFILE
Periodic p99 spikesBlocking disk I/O in a worker; log flushaio threads, access_log buffer= flush=
Disk I/O high on a proxyResponse bodies spilling to proxy_temp_pathRaise proxy_buffers, or set proxy_max_temp_file_size 0
Cannot assign requested addressEphemeral port exhaustionUpstream keepalive
Intermittent 502 at low trafficKeepalive idle-timeout raceNGINX keepalive_timeout < upstream's idle timeout

15Logging & observability

Structured access logs

log_format json escape=json
  '{'
    '"ts":"$time_iso8601",'
    '"req_id":"$request_id",'
    '"remote_addr":"$remote_addr",'
    '"host":"$host",'
    '"method":"$request_method",'
    '"uri":"$request_uri",'
    '"status":$status,'
    '"bytes_sent":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"upstream_addr":"$upstream_addr",'
    '"upstream_status":"$upstream_status",'
    '"upstream_connect_time":"$upstream_connect_time",'
    '"upstream_header_time":"$upstream_header_time",'
    '"upstream_response_time":"$upstream_response_time",'
    '"cache":"$upstream_cache_status",'
    '"scheme":"$scheme",'
    '"tls_ver":"$ssl_protocol",'
    '"tls_cipher":"$ssl_cipher",'
    '"tls_reused":"$ssl_session_reused",'
    '"http_ver":"$server_protocol",'
    '"ua":"$http_user_agent",'
    '"referer":"$http_referer",'
    '"conn_reqs":$connection_requests
  '}';

access_log /var/log/nginx/access.log json buffer=64k flush=5s;
access_log syslog:server=10.0.0.5:514,tag=nginx,severity=info json;

# sample only 1% of successful requests, log ALL errors
map $status $loggable { ~^[23] 0; default 1; }
access_log /var/log/nginx/errors.log json if=$loggable;
escape=json is mandatory

Without it, a request URI containing a quote or backslash produces malformed JSON and your log pipeline silently drops lines — often exactly the malicious requests you most wanted to see.

The four latency variables — how to read them

$request_time ├────────────────────────────────────┤ total $upstream_connect_time ├──┤ TCP+TLS to upstream $upstream_header_time ├──────────────┤ until first byte from app $upstream_response_time ├───────────────────────┤ until last byte from app DIAGNOSIS: connect_time high → upstream saturated / no keepalive / SYN backlog full header_time ≈ response_time and both high → app is slow (compute-bound) header_time low, response_time high → app streams slowly / large body request_time >> response_time → SLOW CLIENT, not your problem connect_time == 0.000 → keepalive reuse working ✔

Metrics

# OSS: minimal built-in stats
location = /nginx_status {
    stub_status;
    allow 127.0.0.1; allow 10.0.0.0/8; deny all;
    access_log off;
}
# → Active connections: 291
#   server accepts handled requests
#    16630948 16630948 31070465
#   Reading: 6 Writing: 179 Waiting: 106

/* PLUS: rich JSON API — per-upstream, per-zone, per-cache */
location /api { api write=on; allow 10.0.0.0/8; deny all; }
location = /dashboard.html { root /usr/share/nginx/html; }

Distributed tracing

# Propagate a correlation ID (works with any tracing backend)
proxy_set_header X-Request-ID $request_id;
add_header       X-Request-ID $request_id always;

# Native OpenTelemetry module (nginx 1.25.2+ / otel-nginx)
load_module modules/ngx_otel_module.so;
otel_exporter { endpoint otel-collector:4317; }
otel_service_name nginx-edge;
otel_trace on;
otel_trace_context propagate;   # inject/extract W3C traceparent

Golden signals to alert on

SignalSourceAlert idea
5xx rate$status>1% over 5m, split by upstream
p99 $upstream_response_timelog histogramPer route, vs 7-day baseline
Cache hit ratio$upstream_cache_statusSudden drop = key change or purge storm
Active connectionsstub_status>70% of worker_connections capacity
Waiting/Reading ratiostub_statusHigh Reading = slowloris-ish
429/503 ratelogsLimiter firing on legitimate traffic
$upstream_connect_time > 0logsKeepalive pool broken after a deploy

16Zero-downtime reload & binary upgrade

Config reload — SIGHUP

nginx -t validate first — ALWAYS nginx -s reload (or: kill -HUP $(cat /run/nginx.pid)) 1. master re-reads config; if invalid → logs error, KEEPS RUNNING old config 2. master opens any NEW listen sockets 3. master forks NEW workers with the new config → they start accepting 4. master sends SIGQUIT to OLD workers: · they stop accepting new connections · they FINISH in-flight requests · they close idle keepalive connections · then exit 5. worker_shutdown_timeout caps step 4 (default: forever)
Reload is not free
  • During overlap you have 2× workers and 2× memory. On a box tuned to 80% RAM, a reload under load can OOM.
  • Old workers hang around for the life of their longest connection — WebSockets/SSE can keep them for hours. Set worker_shutdown_timeout 30s; (accepting that those connections get cut).
  • The shared-memory zones are re-created: limit_req counters and ssl_session_cache reset. Frequent reloads (e.g. a k8s ingress controller reloading on every Endpoint change) effectively disable your rate limiter and cause TLS handshake storms. That's the core argument for Envoy/xDS or the NGINX Plus dynamic API in high-churn environments.
  • proxy_cache on disk survives, but the cache loader re-walks it.

Binary upgrade (new nginx version, no dropped connections)

kill -USR2 $(cat /run/nginx.pid)   # old master renames pid→pid.oldbin, execs NEW binary
                                   # both masters now share the listen sockets
kill -WINCH $(cat /run/nginx.pid.oldbin)  # old workers finish & exit; old MASTER stays

# verify new version is healthy, then either:
kill -QUIT $(cat /run/nginx.pid.oldbin)   # commit: retire old master
# or roll back:
kill -HUP  $(cat /run/nginx.pid.oldbin)   # old master respawns its workers
kill -QUIT $(cat /run/nginx.pid)          # retire the new one

Signal reference

SignalTo masterTo worker
TERM/INTFast shutdown — drops connectionsSame
QUITGraceful shutdownGraceful — finish then exit
HUPReload config, graceful worker swap
USR1Reopen log files (for logrotate)Reopen logs
USR2Upgrade binary on the fly
WINCHGracefully shut down workers, keep master

Safe deploy pipeline

# 1. render config from templates
# 2. syntax + semantic validation in a throwaway container
nginx -t -c /tmp/candidate/nginx.conf
# 3. diff against running config, require review for security-header changes
# 4. canary: reload ONE node, watch 5xx + latency for 5 min
# 5. rolling reload across the fleet with a health gate between nodes
# 6. automatic rollback: keep last-known-good config, reload it on SLO breach

# drain a node before maintenance (health endpoint returns 503 → LB removes it)
touch /etc/nginx/drain && sleep 30 && systemctl reload nginx
location = /healthz {
    access_log off;
    if (-f /etc/nginx/drain) { return 503 "draining"; }
    return 200 "ok";
    add_header Content-Type text/plain;
}

17Debugging playbook

First five commands

nginx -t                      # syntax check
nginx -T                      # dump the FULLY RESOLVED config (all includes) — invaluable
nginx -V                      # version + compile flags + which modules exist
ps -eo pid,ppid,rss,cmd | grep nginx
ss -s ; ss -tan state time-wait | wc -l

Turn on debug logging for one client only

# requires --with-debug (check nginx -V)
events { debug_connection 203.0.113.42; }     # or a CIDR
error_log /var/log/nginx/debug.log debug;
# This logs every phase transition for that one IP without drowning the box.

Error-log messages → root cause

MessageMeaning & fix
connect() failed (111: Connection refused)Nothing listening upstream. Check the app, the port, and security groups.
connect() failed (110: Connection timed out)Packets dropped — firewall/SG/NACL, or upstream accept queue full.
connect() failed (99: Cannot assign requested address)Ephemeral port exhaustion. Enable upstream keepalive.
upstream prematurely closed connectionApp crashed mid-response, or the app's idle timeout < NGINX's keepalive_timeout. Classic with Node's 5s default server.keepAliveTimeout.
upstream timed out (110) while reading response headerApp is slow. Raise proxy_read_timeout only after confirming it's not a hang.
upstream sent too big headerRaise proxy_buffer_size; investigate header bloat.
no live upstreams while connectingAll backends passively marked down. Check max_fails/fail_timeout and whether the app is actually healthy.
worker_connections are not enoughRaise worker_connections; remember proxying costs 2 slots.
could not build server_names_hashRaise server_names_hash_bucket_size (long/many server_names).
rewrite or internal redirection cycleA try_files/error_page loop hit the 10-cycle cap. Usually try_files $uri /index.php where /index.php doesn't exist.
client intended to send too large bodyRaise client_max_body_size.
SSL_do_handshake() failedProtocol/cipher mismatch, bad SNI, or a client speaking plaintext to a TLS port.

Status codes NGINX itself generates

CodeWho emitted itTypical cause
400NGINXMalformed request, or header size > large_client_header_buffers
408NGINXclient_header_timeout/client_body_timeout hit
413NGINXclient_max_body_size exceeded
429NGINXlimit_req/limit_conn (if you set limit_req_status 429)
444NGINX (special)You returned it — close connection, send nothing
499NGINX (special)Client closed the connection before NGINX responded. Means your app was too slow OR the client timed out. Not an app error — but a strong latency signal.
502NGINXUpstream refused/closed/sent garbage/oversized headers
503NGINX or appNo live upstreams; or limit_* default status
504NGINXproxy_read_timeout or proxy_connect_timeout exceeded
A 499 spike is the most misread signal in NGINX

It is not a server error and won't appear in your app's error rate — but a wave of 499s at exactly 30s means a client-side timeout is firing against a slow endpoint. Alert on it separately.

Live inspection

# which upstream served each request, and how long it took
tail -f access.log | jq -r '[.upstream_addr,.status,.upstream_response_time,.uri]|@tsv'

# connection state distribution
ss -tan | awk '{print $1}' | sort | uniq -c | sort -rn

# is a worker blocked? (should be in epoll_wait almost always)
strace -p $(pgrep -f 'nginx: worker' | head -1) -c -f

# flame graph a hot worker
perf record -F 99 -p $(pgrep -f 'nginx: worker' | head -1) -g -- sleep 30

# reproduce a request exactly as nginx sees it
curl -v --resolve api.example.com:443:10.0.1.5 https://api.example.com/health
openssl s_client -connect example.com:443 -servername example.com -tls1_3

18Stream module — L4 TCP/UDP proxying

stream {
    log_format basic '$remote_addr [$time_local] $protocol $status '
                     '$bytes_sent $bytes_received $session_time $upstream_addr';
    access_log /var/log/nginx/stream.log basic;

    # ── TCP load balancing (e.g. Postgres read replicas) ──
    upstream pg_read {
        zone pg_read 64k;
        least_conn;
        server 10.0.2.11:5432 max_fails=2 fail_timeout=10s;
        server 10.0.2.12:5432 max_fails=2 fail_timeout=10s;
    }
    server {
        listen 5432;
        proxy_pass pg_read;
        proxy_connect_timeout 2s;
        proxy_timeout 10m;              # idle timeout for the whole session
        proxy_socket_keepalive on;
    }

    # ── UDP (DNS, syslog, QUIC, game traffic) ──
    upstream dns { server 10.0.3.1:53; server 10.0.3.2:53; }
    server {
        listen 53 udp;
        proxy_pass dns;
        proxy_responses 1;             # expect exactly 1 datagram back
        proxy_timeout 1s;
    }

    # ── TLS passthrough routed by SNI (no decryption!) ──
    map $ssl_preread_server_name $sni_backend {
        api.example.com   api_tls;
        admin.example.com admin_tls;
        default           default_tls;
    }
    upstream api_tls     { server 10.0.4.1:443; }
    upstream admin_tls   { server 10.0.4.2:443; }
    upstream default_tls { server 10.0.4.3:443; }

    server {
        listen 443;
        ssl_preread on;              # peek at the ClientHello WITHOUT terminating
        proxy_pass $sni_backend;
    }

    # ── PROXY protocol: preserve client IP through an L4 hop ──
    server {
        listen 8443 proxy_protocol;    # accept PROXY header from an upstream LB
        set_real_ip_from 10.0.0.0/8;
        proxy_pass backend;
        proxy_protocol on;            # and emit it downstream
    }
}
http{} — L7stream{} — L4
SeesMethod, URI, headers, bodyBytes + TCP/UDP metadata (+ SNI via ssl_preread)
Can doPath routing, caching, header rewrite, compression, authPort forwarding, SNI routing, TLS passthrough, DB/queue proxying
CostParsing + buffering per requestNear-zero — just splice bytes
Use forWeb/API trafficPostgres, Redis, Kafka, SMTP, MQTT, end-to-end-encrypted TLS
When you'd choose L4

"If the requirement is end-to-end encryption to the backend for compliance, I can't terminate TLS at the edge. I use stream + ssl_preread to route by SNI without decrypting — I get host-based routing and health checks while the ciphertext flows through untouched. The trade-off: no path routing, no caching, no WAF, and no L7 observability."

PROXY protocol chain

Client(1.2.3.4) → AWS NLB (proxy_protocol v2) → NGINX → app http { server { listen 443 ssl proxy_protocol; # parse the PROXY header set_real_ip_from 10.0.0.0/8; # trust only the NLB subnet real_ip_header proxy_protocol; # take client IP from PROXY, not XFF proxy_set_header X-Real-IP $proxy_protocol_addr; } } ⚠ If proxy_protocol is enabled but the peer does NOT send the header, every connection fails with "broken header". It's all-or-nothing per listener.

19gRPC, WebSocket, SSE

gRPC

upstream grpc_svc {
    zone grpc_svc 64k;
    server 10.0.5.1:50051;
    server 10.0.5.2:50051;
    keepalive 32;
}

server {
    listen 443 ssl;
    http2 on;                          # gRPC REQUIRES HTTP/2 end to end

    location /helloworld.Greeter/ {       # route by fully-qualified service name
        grpc_pass grpc://grpc_svc;         # grpcs:// for TLS to upstream
        grpc_set_header X-Request-ID $request_id;
        grpc_read_timeout 1h;             # streaming RPCs are long-lived
        grpc_send_timeout 1h;
        grpc_socket_keepalive on;
    }

    # map gRPC status onto a sane error when the backend is gone
    error_page 502 = /grpc_unavailable;
    location = /grpc_unavailable {
        internal;
        add_header grpc-status 14 always;   # UNAVAILABLE
        add_header grpc-message "upstream unavailable" always;
        return 204;
    }
}
The gRPC load-balancing trap

gRPC multiplexes many RPCs over one long-lived HTTP/2 connection. A connection-level L4 balancer therefore pins all of a client's RPCs to one backend forever — you get zero balancing and terrible skew after a scale-up. You need request-level (L7) balancing, which is exactly what grpc_pass gives you. Alternatives: client-side LB with a resolver (gRPC's own round_robin policy over headless DNS), a proper mesh (Envoy), or setting MAX_CONNECTION_AGE on the server so connections periodically rebalance.

WebSocket

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

location /ws/ {
    proxy_pass http://ws_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade    $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Host       $host;

    proxy_read_timeout  3600s;      # else idle sockets are killed after 60s
    proxy_send_timeout  3600s;
    proxy_buffering off;
    proxy_socket_keepalive on;
}
WebSockets + reloads

Each WS connection holds a worker slot for its entire lifetime and keeps an old worker alive across every config reload. On a busy chat service that means workers accumulate. Mitigations: worker_shutdown_timeout, server-side max connection age with client reconnect-with-backoff, and putting WS on a separate NGINX pool from your HTTP traffic so reloads there don't disturb it.

Server-Sent Events

location /events {
    proxy_pass http://sse_backend;
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    proxy_buffering off;           ← without this the client gets NOTHING until the end
    proxy_cache off;
    gzip off;                      ← gzip buffers too
    chunked_transfer_encoding off;
    proxy_read_timeout 24h;
}

20OpenResty, njs & modules

Module types

TypeNotes
StaticCompiled in at build time (./configure --with-http_v2_module). Check with nginx -V.
Dynamic (1.9.11+).so loaded with load_module in the main context. Must be built against the exact NGINX version.
# Commonly-added modules
load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_geoip2_module.so;
load_module modules/ngx_http_headers_more_filter_module.so;   # merge/remove headers
load_module modules/ngx_http_modsecurity_module.so;
load_module modules/ngx_http_js_module.so;                    # njs
load_module modules/ngx_otel_module.so;

njs — JavaScript, first-party, lightweight

# /etc/nginx/njs/hmac.js
#   function verify(r) {
#     var sig = r.headersIn['X-Signature'];
#     ... compute expected ...
#     if (sig !== expected) { r.return(403); return; }
#     r.return(204);
#   }
#   export default { verify };

js_import hmac from njs/hmac.js;
js_set $tenant hmac.tenantFromToken;      # compute a variable in JS

location = /_verify { internal; js_content hmac.verify; }
location /api/ { auth_request /_verify; proxy_pass http://api; }

OpenResty / Lua — full request-phase control

lua_shared_dict ratelimit 10m;

location /api/ {
    access_by_lua_block {
        -- distributed rate limit backed by Redis
        local redis = require "resty.redis"
        local red = redis:new()
        red:set_timeouts(100, 100, 100)          -- NON-BLOCKING (cosocket)
        local ok = red:connect("redis", 6379)
        if not ok then return end                -- fail OPEN, not closed
        local key = "rl:" .. ngx.var.http_x_api_key
        local n = red:incr(key)
        if n == 1 then red:expire(key, 1) end
        red:set_keepalive(10000, 100)             -- return conn to pool
        if n > 100 then
            ngx.header["Retry-After"] = 1
            return ngx.exit(429)
        end
    }
    proxy_pass http://api;
}

Lua phase hooks map onto NGINX phases: set_by_luarewrite_by_luaaccess_by_luacontent_by_luaheader_filter_by_luabody_filter_by_lualog_by_lua. Plus init_by_lua (master, pre-fork) and init_worker_by_lua (per worker — where you start background timers).

Lua rules

Use only cosocket APIs (resty.*) — they yield to the event loop. Any use of os.execute, LuaSocket, blocking file I/O, or a tight CPU loop blocks the whole worker. Also: init_by_lua runs before fork, so anything cached there is shared copy-on-write; per-worker state belongs in init_worker_by_lua, and cross-worker state in lua_shared_dict.

GeoIP

geoip2 /etc/nginx/GeoLite2-Country.mmdb {
    auto_reload 60m;
    $geoip2_country_code country iso_code;
}
map $geoip2_country_code $blocked { default 0; XX 1; YY 1; }
server { if ($blocked) { return 403; } }

21NGINX in Kubernetes

Two different controllers — know the difference

ingress-nginx (kubernetes/ingress-nginx)nginx-ingress (nginxinc)
MaintainerKubernetes communityF5/NGINX Inc.
BaseNGINX + Lua (OpenResty)Plain NGINX / NGINX Plus
Endpoint changesUpdated in Lua without a reloadReload (OSS) or dynamic API (Plus)
Config surfaceHuge nginx.ingress.kubernetes.io/* annotation setVirtualServer/VirtualServerRoute CRDs
NoteThe one most people mean by "nginx ingress". Went into maintenance mode in 2025 with InGate proposed as successor — check current status.Cleaner CRD model, commercial support path
# Ingress with the common annotations
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "25m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3"
    nginx.ingress.kubernetes.io/limit-rps: "100"
    nginx.ingress.kubernetes.io/limit-burst-multiplier: "5"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    # canary: route 10% of traffic to this Ingress' backend
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
    # or header-based canary for internal testing
    nginx.ingress.kubernetes.io/canary-by-header: "X-Canary"
spec:
  ingressClassName: nginx
  tls:
    - hosts: [api.example.com]
      secretName: api-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /v1
            pathType: Prefix
            backend: { service: { name: api-svc, port: { number: 80 } } }
The reload-churn problem — a great staff answer

In a large cluster, Deployments scale and pods reschedule constantly. A naive controller re-renders nginx.conf and reloads on every Endpoint change. At high churn that means reloads every few seconds, and each reload: doubles worker memory transiently, resets limit_req counters and the TLS session cache, and leaves old workers lingering on long connections. The controller can spend more time reloading than serving.

Fixes: (1) ingress-nginx's Lua endpoint updater bypasses reload for pure endpoint changes; (2) NGINX Plus dynamic upstream API; (3) move to an xDS-based proxy (Envoy / Gateway API implementations like Envoy Gateway, Istio, Contour) where config updates are streamed atomically with no process churn. That's the honest reason Envoy dominates mesh workloads.

Ingress vs Gateway API

Ingress is expressively limited, so every vendor bolted on annotations — which are unportable and untyped. Gateway API (GatewayClass / Gateway / HTTPRoute / GRPCRoute) is the successor: role-oriented (infra team owns Gateway, app team owns HTTPRoute), typed, and supports header matching, traffic splitting, and request mirroring natively. Expect a question on this if the role is platform-flavoured.

NGINX as sidecar vs mesh

NGINX can be a sidecar, but it lacks a control plane: no xDS, no automatic mTLS identity (SPIFFE), no per-service circuit-breaker telemetry, no centralised policy. That's why meshes standardised on Envoy. NGINX's sweet spot in k8s is the north-south edge; Envoy owns east-west.

22NGINX vs Envoy vs HAProxy vs the rest

NGINXEnvoyHAProxyTraefikCaddy
LanguageCC++CGoGo
ConfigStatic file + reloadDynamic xDS (hot, atomic)Static + Runtime APIAuto from providersCaddyfile / JSON API
ThreadingMulti-process, 1 thread eachMulti-threaded, thread-per-coreMulti-threaded (single-proc)GoroutinesGoroutines
L7 featuresExcellentExcellent + retries/outlier detection/shadowing built inExcellent, superb L4GoodGood
CachingStrong (disk cache)WeakBasic small-object cacheNonePlugin
ObservabilityBasic OSS (stub_status)Best-in-class — huge stats surface, native tracingVery good stats socketGoodGood
Health checksPassive (OSS) / Active (Plus)Active + outlier detectionActive, freeActiveActive
Auto-TLSExternal (certbot)ExternalExternalBuilt-in ACMEBuilt-in ACME
Best atEdge, static, caching, TLS terminationService mesh, dynamic environmentsPure LB, TCP, extreme reliabilityContainer-native routingSimplicity, HTTPS by default
How to answer "why would you pick X?"
  • NGINX — the edge tier: TLS termination, response caching, static assets, simple stable routing, huge operational familiarity, tiny memory footprint. Config is a file, which means it's reviewable in git and diffable — an underrated operational property.
  • Envoy — anywhere config changes frequently or you need first-class observability: service mesh, multi-tenant gateways, anything driven by a control plane. Retries with budgets, outlier detection, request shadowing, and per-route stats are built in rather than bolted on.
  • HAProxy — when load balancing is the job: free active health checks, best-in-class TCP mode, exceptional stability, and a runtime API for changing servers without reload.
  • Traefik/Caddy — small teams, automatic certificates, container-native discovery, low operational overhead.

And the meta-answer: "most large orgs run more than one — Envoy or a cloud LB at L4, NGINX at the caching/TLS edge, and Envoy sidecars east-west. Picking one globally is usually the wrong framing."

23Real-world architectures

A. Multi-tier edge for a high-traffic site

┌──────────────┐ Internet ─────────►│ CDN + DDoS │ static assets, 95% offload └──────┬───────┘ ▼ ┌──────────────┐ │ NLB (L4) │ cross-AZ, PROXY protocol └──────┬───────┘ ┌───────────┼───────────┐ ▼ ▼ ▼ ┌──────────┐┌──────────┐┌──────────┐ │ NGINX-1 ││ NGINX-2 ││ NGINX-3 │ ← TLS term, WAF, │ edge ││ edge ││ edge │ rate limit, cache, └────┬─────┘└────┬─────┘└────┬─────┘ routing, auth_request └───────────┼───────────┘ ┌──────────┼──────────┐ ▼ ▼ ▼ ┌─────────┐┌─────────┐┌─────────┐ │ web svc ││ api svc ││ ws svc │ └─────────┘└─────────┘└─────────┘ Why NGINX and not just the cloud ALB? · disk response cache (ALB has none) · auth_request / JWT at the edge · fine-grained rate limiting with custom keys · header manipulation, canary logic, X-Accel-Redirect · portable config, not cloud-locked

B. Canary / blue-green / A-B

# ── Weighted canary via split_clients (deterministic per user!) ──
split_clients "$cookie_uid$remote_addr" $variant {
    5%    canary;
    5%    canary_b;
    *     stable;
}
map $variant $pool {
    canary    app_v2;
    canary_b  app_v3;
    default   app_v1;
}
# Header override wins so QA can force a variant
map $http_x_force_variant $final_pool {
    v2       app_v2;
    v3       app_v3;
    default  $pool;
}

location / {
    proxy_pass http://$final_pool;
    add_header X-Variant $variant always;    # log & measure per variant
}

split_clients hashes the key with MurmurHash2, so the same user always lands in the same bucket across requests — essential or your A/B test is noise. Include something stable (user id) in the key, not just $request_id.

# ── Blue-green: flip an included file and reload ──
# /etc/nginx/active_pool.conf  contains:  set $active blue;
include /etc/nginx/active_pool.conf;
location / { proxy_pass http://app_$active; }

# ── Mirror traffic to a new stack without affecting users ──
location / {
    mirror /_shadow;
    mirror_request_body on;
    proxy_pass http://app_v1;
}
location = /_shadow {
    internal;
    proxy_pass http://app_v2$request_uri;
    proxy_connect_timeout 1s;
    # response is DISCARDED; client never waits on it
}
Mirror caveats

Mirrored requests hit a real backend — if v2 writes to the production database you've just double-written. Point mirrors at an isolated stack with its own data store. Also, mirroring doubles your outbound connection count and the mirror subrequest still consumes a worker_connections slot.

C. API gateway pattern

upstream users_svc  { zone u 64k; server users:8080;  keepalive 32; }
upstream orders_svc { zone o 64k; server orders:8080; keepalive 32; }

server {
    listen 443 ssl; http2 on;
    server_name api.example.com;

    # global policy
    limit_req zone=perkey burst=50 nodelay;
    auth_request /_authz;
    proxy_set_header X-Request-ID $request_id;

    # versioned routes; note ^~ so no regex is evaluated
    location ^~ /v1/users/  { rewrite ^/v1/users/(.*) /$1 break; proxy_pass http://users_svc; }
    location ^~ /v1/orders/ { rewrite ^/v1/orders/(.*) /$1 break; proxy_pass http://orders_svc; }

    # cacheable public reads
    location ^~ /v1/catalog/ {
        auth_request off;
        proxy_cache api_cache;
        proxy_cache_valid 200 60s;
        proxy_cache_lock on;
        proxy_pass http://catalog_svc;
    }

    # deprecation / sunset
    location ^~ /v0/ {
        add_header Sunset "Wed, 31 Dec 2026 23:59:59 GMT" always;
        add_header Deprecation "true" always;
        proxy_pass http://legacy_svc;
    }

    location = /_authz { internal; proxy_pass http://authz; ... }
}

D. Surviving a traffic spike (the "HN front page" question)

  1. Microcache anonymous responses for 1s → app sees ~1 rps per distinct URL.
  2. proxy_cache_lock on + use_stale updating + background_update → no stampede on expiry.
  3. max_conns per backend → bulkhead so the app tier can't be overwhelmed.
  4. limit_req keyed on IP with generous burst → shave off abusive clients only.
  5. proxy_cache_use_stale error timeout http_5xx → if the app dies, keep serving the last good copy.
  6. Static error_page fallback for anything not in cache — degraded, not down.
  7. Push assets to the CDN with Cache-Control: immutable and long TTLs so NGINX never sees them.

E. Multi-region failover

upstream region_pool {
    zone region_pool 64k;
    server us-east.internal:443 max_fails=2 fail_timeout=10s;   # local
    server us-west.internal:443 backup;                        # DR — only on total local failure
    keepalive 32;
}
location / {
    proxy_pass https://region_pool;
    proxy_ssl_server_name on;            ← REQUIRED: send SNI upstream
    proxy_ssl_name $host;
    proxy_ssl_verify on;
    proxy_ssl_trusted_certificate /etc/ssl/internal-ca.pem;
    proxy_ssl_session_reuse on;
    proxy_next_upstream error timeout http_502 http_503;
    proxy_next_upstream_tries 2;
}
The proxy_ssl SNI trap

proxy_ssl_server_name defaults to off. Proxying to any TLS endpoint that uses SNI-based cert selection (which is every cloud LB and CDN) without it produces a handshake failure or the wrong certificate. Similarly, proxy_ssl_verify defaults to off — meaning by default NGINX does not validate upstream certificates. Turn it on for anything crossing a trust boundary.

24Top 20 gotchas — rapid fire

#GotchaFix
1proxy_pass trailing slash changes URI semanticsSlash ⇒ strip prefix; no slash ⇒ pass through
2add_header in a child location wipes inherited headersRepeat them, or use headers_more
3add_header ignored on 4xx/5xxAdd always
4Upstream keepalive silently offNeed both proxy_http_version 1.1 and proxy_set_header Connection ""
5upstream DNS resolved once at startupresolver+variable, Plus resolve, or service discovery
6No zone ⇒ per-worker stateAlways declare zone in upstreams
7proxy_read_timeout isn't a total deadlineEnforce deadlines in the app
8Keepalive idle-timeout races cause phantom 502sNGINX timeout < upstream idle timeout; > downstream LB timeout
9if inside location is undefined behaviourUse map / try_files / return
10alias without symmetric trailing slash ⇒ path traversalMatch trailing slashes exactly
11Trusting all XFF ⇒ IP spoofingNarrow set_real_ip_from + real_ip_recursive on
12proxy_cache_bypass alone still stores private responsesAlso set proxy_cache_no_cache
13Cache key missing $host on multi-tenantInclude host and any Vary dimension
14Reload resets limit_req counters & TLS session cacheReduce reload frequency; dynamic upstreams
15Missing default_server ⇒ Host-header routing surprisesExplicit catch-all returning 444
16proxy_ssl_server_name off by defaultTurn on when proxying to TLS with SNI
17proxy_ssl_verify off by defaultTurn on + set trusted CA
18Regex locations evaluated in file order, not by specificityOrder deliberately; prefer ^~ prefixes
19gzip without gzip_vary ⇒ CDN serves wrong encodinggzip_vary on
20SSE/WebSocket blank until finishedproxy_buffering off (+ gzip off for SSE)

25Interview Q&A bank

Click to reveal. Answers are written as you'd actually say them — with the trade-off, not just the fact.

Architecture & internals

Walk me through what happens from TCP SYN to response byte for a proxied request.

The kernel completes the handshake and queues the socket on the listen backlog. A worker's epoll reports the listening fd readable and it accept()s (or, with reuseport, the kernel already steered it to that worker's own queue). NGINX allocates a connection pool and, if TLS, drives the handshake — hopefully resumed via session cache or ticket.

Then it reads and parses the request line and headers into a request pool. It picks the server block by listen + Host, then runs the phase pipeline: POST_READ (realip rewrites $remote_addr), SERVER_REWRITE, FIND_CONFIG (location match), REWRITE, PREACCESS (rate limits), ACCESS (auth), PRECONTENT (try_files, mirror), then CONTENT where proxy_pass runs.

The proxy module selects an upstream peer per the balancing algorithm, grabs an idle keepalive connection if available, rewrites headers per proxy_set_header, and writes the request. Response headers come back into proxy_buffer_size; the body streams into proxy_buffers and spills to a temp file if it overflows. The response goes through the header filter chain and the body filter chain (ssi → sub_filter → gzip → chunked → writer) to the client socket. Finally the LOG phase fires, the request pool is freed, and the connection either goes idle for keepalive or closes.

Two things I'd call out: the upstream connection is released back to the pool as soon as the response is fully buffered — that's the point of buffering — and the whole path is one worker, non-blocking, so any blocking call here would stall every other connection on that worker.

Why is NGINX faster than Apache prefork? Where would Apache actually win?

NGINX's memory and scheduling cost scales with active work, not with connection count — a fixed set of workers multiplexing tens of thousands of connections via epoll, ~1 KB of heap per idle connection versus a whole process/thread stack.

Apache wins where you want embedded interpreters and per-request isolation: mod_php, .htaccess for shared hosting where each customer edits their own config, and a much richer module ecosystem for in-process request handling. Also, Apache's event MPM narrowed the gap considerably; the "NGINX is 10× faster" framing is a 2010 argument. Today I'd pick NGINX for proxying and static content, and I wouldn't fight to migrate a working Apache app-server setup.

What is worker_connections and how do you size it?

It's the max simultaneous connections per worker, and it counts everything: client connections, upstream connections, listening sockets, resolver sockets, and internal subrequests. So a proxied request consumes at least two slots — max_clients ≈ worker_processes × worker_connections / 2, and less if you use auth_request or mirror.

Sizing: start from peak concurrent connections (not RPS) = peak RPS × avg connection lifetime, add headroom for keepalive idle connections, divide by worker count, multiply by 2–3. Then make sure worker_rlimit_nofileworker_connections and the systemd LimitNOFILE covers it, or you'll hit "too many open files" long before the configured ceiling.

What breaks if you do blocking work in a worker?

Everything on that worker queues behind it. With 8 workers, roughly 1/8 of in-flight requests stall for the duration. The signature in metrics is p99 latency spikes with a flat $upstream_response_time — the time is being spent inside NGINX, not the app.

Sources: cold-file disk reads (fix: aio threads + directio), synchronous DNS (fix: resolver, which is async), blocking Lua (fix: cosockets only), enormous regexes with catastrophic backtracking, and huge gzip on giant bodies.

Explain shared memory zones and what happens when one fills up.

Workers are separate processes, so any state that must be consistent across them lives in an mmap'd slab allocator declared in config: limit_req_zone, limit_conn_zone, keys_zone for proxy_cache, upstream ... zone, ssl_session_cache shared:.

They're fixed size. When a limit_req zone fills, NGINX force-evicts up to a couple of LRU entries per request — under a wide distributed attack the zone thrashes and the limiter becomes useless. When a cache keys zone fills, NGINX starts evicting cache entries irrespective of max_size, so your 20 GB disk cache mysteriously plateaus at 3 GB. Rule of thumb: ~16k IP states per MB for limit zones, ~8k keys per MB for cache zones. Size the keys zone from your expected object count first.

Configuration

Explain proxy_pass with and without a trailing slash.

If proxy_pass includes any URI component — even a bare / — NGINX replaces the matched location prefix with it. If it's only scheme+host+port, the original request URI is passed through unchanged.

So with location /api/: proxy_pass http://b; sends /api/users, while proxy_pass http://b/; sends /users. Two extra rules: you can't use a URI part inside a regex or named location (use rewrite ... break instead), and if you use a variable in proxy_pass the URI is dropped unless you append $request_uri explicitly.

Order of precedence for location matching?

= exact wins outright and stops the search. Otherwise NGINX finds the longest prefix match; if that one was declared ^~, it's used and regexes are skipped. Otherwise regexes are tried in configuration file order and the first match wins. If no regex matches, the remembered longest prefix is used.

The part people miss is that regex order is file order, not specificity or length — so a broad ~ \.php$ placed above a narrow one silently shadows it.

Why is if discouraged?

Inside a location, if creates an implicit nested configuration context. Directives that aren't return or rewrite ... last can end up evaluated in the wrong context — set values can be lost, proxy_pass inside an if can bypass the parent's proxy_set_header directives entirely. It's not a bug so much as a leaky implementation detail that the maintainers documented as "IfIsEvil".

My rule: if only for return. Value selection goes through map (which is a hash lookup, lazily evaluated, and inherits cleanly), file existence through try_files, and anything genuinely conditional through map-chained variables or the upstream application.

root vs alias?

root appends the request URI to the path; alias replaces the matched location prefix with the path. location /s/ { root /var/www; } maps /s/a.js/var/www/s/a.js; with alias /var/www/assets/; it maps to /var/www/assets/a.js.

Security note: keep trailing slashes symmetric. location /s { alias /var/www/assets/; } — no slash on the location, slash on the alias — allows /s../ to escape the directory. That's a recurring real-world CVE pattern.

Why do my security headers vanish in one location?

add_header is array-inheriting: a child context that defines any add_header discards the entire inherited array rather than merging. Same for proxy_set_header. So adding one debug header in /api/ silently drops HSTS and CSP for that path.

Fixes, in order of preference: put all headers in a snippet and include it in every location; use more_set_headers from the headers_more module which merges properly; or add a CI check that asserts the security headers are present on every route.

Load balancing & resilience

Compare the load-balancing algorithms and when you'd use each.

Round-robin for uniform, short requests. least_conn when request durations vary — which is most real systems — but be aware of the black-hole failure mode: a backend that fails instantly shows zero active connections and therefore attracts all traffic. Pair it with health checks.

ip_hash only for legacy stickiness, and it's poor: it hashes the first three IPv4 octets, so anyone behind CGNAT lands together, and adding a node reshuffles almost everything. hash $key consistent is the better sharding primitive — Ketama ring, so only ~1/N of keys move when membership changes. That's what you want for cache-node affinity.

random two least_conn is the one worth knowing: with many independent NGINX instances, plain least_conn makes them all pick the same idle backend simultaneously and oscillate. Power-of-two-choices adds randomness and provably bounds max load near the average. Envoy defaults to it for the same reason.

OSS health checks — what do you actually get, and what would you do about it?

Passive only: max_fails failures within fail_timeout marks the peer down for fail_timeout, then one real user request probes it. What counts as a failure is defined by proxy_next_upstream. So detection requires real traffic to fail first, and recovery makes a user pay for the probe.

Options at staff level: run NGINX Plus or Angie for active checks; or accept that health-checking belongs a layer up — the cloud LB and the k8s readiness probe already do it, and the Endpoints controller removes unhealthy pods from the upstream list before NGINX ever sees them. That's usually the cleanest answer: don't duplicate health checking in the proxy when the orchestrator owns it.

Your backend gets slow. Walk me through what NGINX does and how it can make things worse.

Requests pile up. Each waiting request holds a client connection and an upstream connection, so worker_connections pressure roughly doubles. When proxy_read_timeout fires, NGINX returns 504 — and if proxy_next_upstream includes timeout, it retries on another backend, multiplying load on a tier that's already struggling. That's retry amplification, and it's how a brownout becomes an outage.

Meanwhile clients time out first, giving you a wave of 499s, and any client-side retry adds another multiplier.

What I'd do: cap proxy_next_upstream_tries 2 with a proxy_next_upstream_timeout budget; set max_conns per backend as a bulkhead so excess requests fail fast instead of queueing; enable proxy_cache_use_stale so degraded means "slightly stale" rather than "500"; and make sure client retries use exponential backoff with jitter. The principle is shed load early and cheaply rather than queueing it.

Is retrying a failed POST safe?

No, not by default. proxy_next_upstream error timeout will retry a request that timed out — but a timeout means NGINX doesn't know whether the upstream processed it. Retrying a POST /payments can double-charge. NGINX guards this: non-idempotent methods aren't retried after the request was sent unless you explicitly add non_idempotent to proxy_next_upstream.

The real fix is idempotency keys in the API contract — client generates a UUID, server dedupes. Then retries are safe everywhere in the stack, not just at NGINX.

How do you do sticky sessions, and should you?

OSS: hash $cookie_sessionid consistent. Plus: sticky cookie, sticky route, or sticky learn (which tracks the app's own session cookie in a shared zone).

Should you? Usually not. Stickiness makes deploys, autoscaling and node failure user-visible, and it defeats even load distribution. I'd push session state into Redis or a signed token and keep backends stateless. Legitimate exceptions: WebSocket connections (inherently pinned anyway), in-process caches where affinity is a performance optimization rather than a correctness requirement, and legacy apps you can't refactor yet.

Caching

Design a cache key. What goes wrong if you get it wrong?

Default is $scheme$proxy_host$request_uri. I'd normally use $scheme$request_method$host$request_uri, plus any dimension the response actually varies on.

Failure modes: omit $host on a multi-tenant proxy and tenant A gets tenant B's response — a data breach, not a bug. Omit the method and a HEAD poisons the GET entry. Ignore Vary: Accept-Encoding and you hand a gzip body to a client that can't decode it. Include $http_cookie and your cardinality explodes to near-zero hit rate. And any attacker-controlled header that the upstream reflects into the response but that isn't in the key is a web-cache-poisoning vector — X-Forwarded-Host is the classic.

Also: query parameter order creates distinct keys, and tracking params like utm_* fragment the cache. Normalizing those with a map is often a bigger hit-rate win than any TTL tuning.

What is a cache stampede and how does NGINX prevent it?

A hot key expires and every concurrent request for it simultaneously misses, so all of them are forwarded upstream at once. The upstream saturates, requests time out, nothing gets stored, and the pattern repeats — a self-sustaining outage.

proxy_cache_lock on lets only the first request populate a cold key; the rest wait up to proxy_cache_lock_timeout. Better: combine with proxy_cache_use_stale updating and proxy_cache_background_update on, so waiting requests get the stale copy immediately while one background request refreshes. Upstream sees exactly one request; nobody waits.

proxy_cache_min_uses 2 is a complementary control — it stops one-hit-wonder URLs from evicting genuinely hot content.

Explain microcaching and why it's so effective.

Cache "dynamic" pages for one second. At 5,000 rps on a hot URL, the app sees 1 rps — a 99.98% reduction — and no human perceives one second of staleness on a news page or product listing.

It works because traffic spikes are concentrated: a small number of URLs take the overwhelming majority of requests. Even a sub-second TTL collapses that concentration. It's the highest return-on-effort change available for read-heavy anonymous traffic, and it requires no application change.

Caveat: it must be scoped to anonymous traffic. Add proxy_cache_bypass and proxy_cache_no_cache on session cookies and Authorization, or you'll serve one user's personalized page to everyone.

proxy_cache_bypass vs proxy_no_cache?

proxy_cache_bypass means "don't read from cache for this request" — but the response can still be written. proxy_no_cache means "don't write this response" — but a cached copy could still be served. They're independent, and for authenticated traffic you need both. Setting only bypass is a known route to leaking a private response into the shared cache.

How do you invalidate cache across a fleet of 50 NGINX nodes?

Native proxy_cache_purge is Plus-only and is per-node, so you'd need a fan-out to all 50 — which is a distributed-systems problem with partial-failure semantics you don't want.

My preferred design is versioned cache keys: put a token in the key ($cache_version$scheme$host$request_uri) sourced from a build SHA or a per-tenant epoch. "Purging" becomes bumping the token, which is atomic, instantaneous across the fleet, and needs no coordination. Stale entries age out via inactive.

If you truly need targeted purge, the honest answer is to move the cache tier to something built for it — Varnish with soft purge and surrogate keys, or a CDN with tag-based invalidation.

TLS & protocols

What actually happens in a TLS handshake and how do you make it cheap?

TLS 1.3 full handshake: ClientHello with key share → ServerHello, cert, CertificateVerify (an asymmetric signature — the expensive part) → Finished. One round trip. TLS 1.2 needs two.

Making it cheap: (1) resumption — session cache (server-side state in a shared zone) or session tickets (client holds encrypted state, so it works across the whole fleet). Resumed handshakes skip the asymmetric operation entirely. (2) ECDSA certificates, roughly 4–10× cheaper to sign than RSA-2048; serve both and let the client pick. (3) OCSP stapling so clients don't make their own revocation round trip. (4) 0-RTT for resumed connections, with the caveat that early data is replayable and must never trigger a state change.

The operational gotcha: if you don't distribute ssl_session_ticket_key files across the fleet, each instance invents its own, so resumption fails whenever the LB sends a client to a different box — and the never-rotated key undermines forward secrecy.

HTTP/2 vs HTTP/3 — what actually changes?

HTTP/2 multiplexes streams over one TCP connection with HPACK header compression. But TCP delivers bytes in order, so a single lost packet head-of-line-blocks every stream on that connection. Over lossy mobile networks HTTP/2 can be slower than HTTP/1.1 with six connections.

HTTP/3 moves to QUIC over UDP, which implements per-stream reliability — packet loss only stalls the affected stream. It also merges the transport and TLS handshakes (1 RTT, 0 RTT resumed) and adds connection migration via a Connection ID, so a phone switching wifi→cellular keeps its session.

Operationally: HTTP/3 needs UDP 443 open (many corporate networks block it), CPU cost is higher because congestion control runs in userspace, and you advertise it with Alt-Svc so clients upgrade on a subsequent connection. And note that NGINX still talks HTTP/1.1 to upstreams by default regardless — which is fine and lets you pool connections.

How do you terminate TLS but still guarantee encryption to the backend?

Re-encrypt: terminate at the edge, then proxy_pass https:// with proxy_ssl_verify on, a trusted internal CA, proxy_ssl_server_name on (it defaults to off and will break SNI-based upstreams), and proxy_ssl_session_reuse on to avoid a fresh handshake per request. Optionally mTLS with proxy_ssl_certificate so the backend can authenticate NGINX.

If compliance forbids the plaintext window entirely, don't terminate at all: use stream with ssl_preread on to route on SNI without decrypting. You keep host-based routing and health checks but lose caching, path routing, WAF and L7 metrics — that trade needs to be explicit.

Operations

What exactly happens during nginx -s reload, and what does it cost?

The master re-reads and validates the config. If it's invalid, it logs and keeps running the old config — reloads are safe in that sense. If valid, it opens any new listen sockets, forks new workers with the new config, and signals old workers with QUIT: they stop accepting, finish in-flight requests, close idle keepalives, and exit.

The costs people forget: during the overlap you have double the workers and roughly double the memory. Old workers survive as long as their longest connection — a WebSocket can keep one alive for hours unless you set worker_shutdown_timeout. And all shared memory zones are recreated, so limit_req counters reset and the TLS session cache is flushed.

That last point is the argument for dynamic config in high-churn environments: an ingress controller reloading every few seconds effectively has no working rate limiter and causes continuous TLS handshake storms.

How do you upgrade the NGINX binary with zero dropped connections?

USR2 to the master: it renames the pidfile to .oldbin and execs the new binary, which inherits the listening sockets — so two masters are now accepting on the same sockets. Then WINCH to the old master gracefully retires its workers while keeping the old master alive as a rollback option. Verify health on the new version, then QUIT the old master to commit — or HUP it to respawn its workers and QUIT the new one to roll back.

In practice, on immutable infrastructure I'd rather roll new instances behind the LB than do in-place binary upgrades — but the mechanism is worth knowing because it's what makes NGINX viable on long-lived pets.

You're paged: 502s spiking. What do you check, in order?
  1. error.log — the exact message distinguishes the cause: "connection refused" (nothing listening), "timed out" (network/firewall or saturated accept queue), "upstream prematurely closed" (app crash or idle-timeout race), "too big header" (buffer sizing), "no live upstreams" (all peers passively ejected).
  2. Is it all upstreams or one? Group logs by $upstream_addr. A single bad node is a very different incident from a tier-wide failure.
  3. Did anything deploy — app, config, or infra? Correlate with deploy timestamps first, always.
  4. Check $upstream_connect_time: if it jumped from 0 to non-zero, keepalive pooling broke (someone dropped proxy_http_version 1.1), and you may be hitting ephemeral port exhaustion.
  5. Check whether it's load-correlated or constant-rate. Constant low-rate 502s at low traffic is the classic keepalive idle-timeout race with the upstream.
  6. Mitigate before you finish diagnosing: enable proxy_cache_use_stale, shed load with tighter limits, or roll back.
What's a 499 and why do you care?

An NGINX-specific code meaning the client closed the connection before NGINX sent a response. It won't appear in your application's error rate, so teams routinely miss it.

A cluster of 499s at a consistent duration — say all at exactly 30s — tells you a client-side timeout is firing against a slow endpoint. It's a latency signal disguised as a client error, and it's often the earliest warning that a backend is degrading. I alert on 499 rate separately from 5xx.

How would you make NGINX config changes safe at scale?

Treat it as code: templates in git, rendered per environment, reviewed. CI runs nginx -t in a container matching production's version and modules, plus nginx -T diffed against the currently-running config so reviewers see the fully-resolved change, not just the template edit.

Then add semantic checks that nginx -t won't catch: assert security headers exist on every route, assert no location proxies without a timeout, assert no set_real_ip_from broader than your known proxy CIDRs. Those are the changes that pass syntax check and cause incidents.

Rollout: canary one node, watch 5xx and p99 for five minutes with an automatic rollback gate, then roll the fleet with health checks between nodes. Keep last-known-good on disk so rollback is a reload, not a deploy.

Design & judgement

When would you NOT use NGINX?

High-churn dynamic environments where config changes constantly — a service mesh, or a multi-tenant gateway with per-tenant routes changing hourly. Reload churn destroys shared state and Envoy's xDS model is simply the right tool.

When you need deep per-route observability out of the box — Envoy's stats surface is in a different league from stub_status.

When you need free active health checks and pure L4 excellence — HAProxy.

When a small team wants HTTPS to just work — Caddy or Traefik with built-in ACME.

And when a managed cloud LB covers the requirement, adding an NGINX tier is operational cost with no benefit. I'd only introduce it when I need something the LB can't do: response caching, edge auth, custom rate-limit keys, header rewriting, or cloud portability.

How do you rate limit accurately across 20 NGINX instances?

You can't, with vanilla NGINX — the shared zone is per-instance, so a limit of 10 r/s becomes 200 r/s fleet-wide. Options, in increasing cost:

  1. Divide and accept imprecision. Set rate to limit/N. Breaks when instance count changes or traffic distribution is uneven, but it's free and adequate for DoS protection.
  2. Consistent-hash the key at L4 so a given API key always reaches the same NGINX node. Now per-node limits are exact per key. Costs you connection-level flexibility and creates hot spots.
  3. Shared counter — OpenResty + Redis with a sliding-window or token-bucket script, or Envoy's global ratelimit service. Accurate, but adds a network round trip to every request and a hard dependency you must fail open on.
  4. Two-tier: coarse local limits in NGINX to absorb abuse cheaply, exact quota enforcement in the application where you already have tenant context and can return meaningful X-RateLimit-* headers.

I'd default to (4) — NGINX as a blunt instrument protecting infrastructure, the app as the precise instrument enforcing business rules.

Design zero-downtime deploys through NGINX.

The proxy side is easy; the coordination is the hard part.

Backend rollout: new instances register, pass readiness, and get added to the upstream (via service discovery re-rendering the upstream block, or the k8s Endpoints controller). Old instances receive SIGTERM, immediately flip readiness to failing, then keep serving in-flight work for longer than the health-check detection window before exiting. Skipping that drain window is the single most common cause of deploy-time 502s.

slow_start (Plus) or gradual weight ramp avoids hammering a cold JIT-compiled instance with full traffic.

For risky changes, layer split_clients canary at 1% → 5% → 25% → 100% with automated rollback on error-rate or latency regression, and use mirror to shadow production traffic at the new version beforehand — pointed at an isolated data store so you don't double-write.

NGINX-side changes go through the config pipeline described earlier. And I'd keep long-lived connections (WebSocket) on a separate NGINX pool so app deploys don't interact with reload behaviour there.

Your p99 is 2s but p50 is 30ms. How do you find out why?

First, decompose the latency in the access log: $request_time vs $upstream_connect_time vs $upstream_header_time vs $upstream_response_time.

  • If $request_time is huge but upstream time is flat, it's client-side — slow uploads, slow downloads, mobile networks. Not an app problem. Confirm by segmenting on user agent and response size.
  • If connect_time is spiking, upstream accept queues are full or keepalive pooling broke.
  • If header_timeresponse_time and both spike, the app is slow — go to app tracing, segment by route and by $upstream_addr to see whether it's one bad instance.
  • If header_time is low but response_time high, the app is streaming slowly or the body is large.

Then check whether the slow requests cluster: one upstream address (bad node), one route (bad query), one time-of-day (batch job contention), or a periodicity matching worker count (a blocked worker — look for disk I/O without aio threads). Cache status matters too: if p99 is all MISS, the tail is just "uncached" and the fix is hit-rate work, not latency work.

Where does NGINX fit if you already run a service mesh?

The mesh owns east-west: mTLS identity, per-service retries and circuit breaking, and uniform telemetry between services. NGINX owns north-south at the edge, where the concerns are different: TLS termination for public clients with real certificates, response caching, static assets, WAF, bot management, and coarse rate limiting against the internet.

Trying to make NGINX the mesh data plane means rebuilding a control plane — xDS, SPIFFE identity, dynamic config — which is why the industry converged on Envoy there. Conversely, running Envoy at the edge means giving up NGINX's disk cache. Running both, with a clear boundary, is the common and defensible answer.

26System design scenarios

1. Design the edge for 1M rps

  • Anycast CDN absorbs static + caches HTML briefly; target 90%+ offload before NGINX.
  • L4 (NLB/BGP-ECMP) → NGINX fleet, sized by concurrent connections not rps.
  • Per node: reuseport, worker_processes auto, ECDSA + session tickets shared fleet-wide.
  • Microcache + cache_lock + use_stale → app tier sized for cache-miss rate, not request rate.
  • Sampled logging (1% of 2xx, 100% of errors) — at 1M rps, logging is an I/O budget item.
  • Capacity model: state connections/node, memory/connection, TLS handshakes/sec/core, and headroom for losing an AZ.

2. Multi-tenant SaaS gateway

  • Route by subdomain: wildcard cert + map $host $tenant; per-tenant certs via ssl_certificate with SNI variable or a cert-manager sidecar.
  • Per-tenant rate limits keyed on tenant ID, tiered by plan via chained maps.
  • Cache key must include tenant — this is the one that becomes a breach.
  • Per-tenant upstreams for noisy-neighbour isolation; max_conns as the bulkhead.
  • Config generated from a control plane; avoid reload-per-tenant-change (batch, or use Plus API).

3. Static + SPA + API on one domain

  • location ^~ /assets/ — hashed filenames, immutable, 1y.
  • location = /index.htmlno-cache, so deploys are picked up instantly.
  • location ^~ /api/ — proxy, no cache, CORS if cross-origin.
  • location /try_files $uri $uri/ /index.html for client-side routing.
  • Precompress at build time; gzip_static/brotli_static so you never spend CPU per request.

4. File upload/download service

  • Uploads: client_max_body_size per route; proxy_request_buffering off to stream large files straight to object storage without a disk hop.
  • Downloads: app authorizes, returns X-Accel-Redirect to an internal location — NGINX serves bytes, app worker is freed immediately.
  • aio threads + directio + sendfile off for large files; limit_rate_after for fair bandwidth.
  • max_ranges 1 against range-amplification DoS.

5. Migrate a monolith to microservices

  • Strangler fig: NGINX routes path-by-path. New service gets /v1/users/; everything else falls through to the monolith.
  • mirror the new route to the new service first, compare responses offline, then flip.
  • split_clients for percentage rollout keyed on user id so a user's experience is consistent.
  • Keep the monolith as backup in the upstream during the first weeks.
  • Emit X-Variant/X-Upstream headers and log them — you need per-route comparison data to justify the migration.

6. Global multi-region

  • GeoDNS or anycast steers users to the nearest region.
  • Regional NGINX prefers local backends, with the remote region as backup.
  • Failover trade-off: cross-region latency vs availability — usually you fail over reads freely and writes only with explicit consensus about the data store.
  • Beware cascading failover: if region A dies and all its traffic lands on B, B must have the headroom or you've turned one outage into two. Size for N+1 regionally.

27Cheat sheet

Commands

nginx -t                 # validate config
nginx -T                 # validate + dump FULL resolved config
nginx -V                 # version, compile flags, modules
nginx -s reload          # graceful reload
nginx -s quit            # graceful shutdown
nginx -s reopen          # reopen logs (logrotate)
nginx -c /path/nginx.conf -p /path/prefix
nginx -g "daemon off;"   # foreground — for containers/systemd

Directive quick reference

GoalDirective
Redirect HTTP→HTTPSreturn 301 https://$host$request_uri;
Close connection silentlyreturn 444;
Serve a literal bodyreturn 200 "ok\n"; add_header Content-Type text/plain;
Internal-only locationinternal;
Strip a path prefixrewrite ^/api/(.*)$ /$1 break;
Deny hidden fileslocation ~ /\.(?!well-known) { deny all; }
Basic authauth_basic "x"; auth_basic_user_file /etc/nginx/.htpasswd;
Delegate authzauth_request /_authz;
Rate limitlimit_req zone=z burst=20 nodelay;
Cap concurrencylimit_conn zone=c 10; / upstream max_conns=
Throttle bandwidthlimit_rate 500k; limit_rate_after 10m;
Enable upstream poolingkeepalive 64; + proxy_http_version 1.1; + proxy_set_header Connection "";
Serve stale on failureproxy_cache_use_stale error timeout http_5xx;
Stampede protectionproxy_cache_lock on; proxy_cache_background_update on;
Shadow trafficmirror /_shadow; mirror_request_body on;
Percentage splitsplit_clients "$cookie_uid" $v { 10% b; * a; }
WebSocketproxy_set_header Upgrade $http_upgrade; ... Connection $connection_upgrade;
Disable buffering (SSE)proxy_buffering off; gzip off;
Route TLS by SNI (no decrypt)stream { ssl_preread on; proxy_pass $ssl_preread_server_name…; }
Preserve client IP behind LBset_real_ip_from CIDR; real_ip_header X-Forwarded-For; real_ip_recursive on;

Sane defaults to open with

worker_processes auto;
worker_rlimit_nofile 65535;
events { worker_connections 16384; multi_accept on; }
http {
    sendfile on; tcp_nopush on; tcp_nodelay on;
    keepalive_timeout 75s; keepalive_requests 1000;
    client_max_body_size 10m;
    client_header_timeout 10s; client_body_timeout 10s;
    server_tokens off;
    reset_timedout_connection on;
    gzip on; gzip_vary on; gzip_comp_level 5; gzip_min_length 256;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_session_cache shared:SSL:50m; ssl_session_timeout 1d;
    access_log /var/log/nginx/access.log json buffer=64k flush=5s;
}

287-day study plan

DayFocusHands-on
1Architecture, event loop, phases (§1–2)Run NGINX locally; strace a worker; watch epoll_wait. Set debug_connection for your IP and read a full request trace.
2Config model, matching, map (§3–4)Build a config with overlapping prefix/regex locations and predict which wins before testing. Reproduce the add_header inheritance bug.
3Proxying, buffering, timeouts (§6)Proxy to a deliberately slow backend. Compare $request_time vs $upstream_response_time. Toggle keepalive and watch $upstream_connect_time and ss -s.
4Load balancing + health (§7–8)Three backends in Docker. Kill one; observe passive ejection timing. Try each algorithm under uneven load.
5Caching (§9)Load-test a cold key with and without proxy_cache_lock. Measure upstream request count. Implement microcaching and record the offload ratio.
6TLS, rate limiting, security (§10–13)Set up TLS 1.3 + stapling; verify with openssl s_client. Tune limit_req burst/nodelay and observe the difference under wrk.
7Ops, debugging, design (§14–28)Break things on purpose: exhaust worker_connections, cause a 502 five different ways, and diagnose each from the error log alone. Then do §26 out loud on a whiteboard.
Three things that separate a staff answer
  1. Name the failure mode, not the feature. "I'd enable proxy_cache_lock" is fine. "Without it, a hot key expiring sends N concurrent requests upstream simultaneously, which is how a cache tier turns a traffic spike into an outage" is the answer they remember.
  2. State the trade-off unprompted. Every directive here costs something — buffering costs TTFB, stickiness costs elasticity, retries cost amplification, WAFs cost CPU and false positives. Volunteering the cost signals you've run this in production.
  3. Know where NGINX ends. Rate limiting is per-instance. Health checks are passive on OSS. Reloads reset shared state. Being crisp about the boundaries — and what you'd reach for instead — is more convincing than encyclopedic directive recall.