Mutual TLS (mTLS) from Scratch with OpenSSL and Nginx

Standard HTTPS only verifies the server. When you open a website, your browser checks that the server's TLS certificate came from a trusted Certificate Authority (CA). Once the encrypted tunnel opens, the server has no idea who you are until you hand over an application credential, like an API key, session cookie, or JWT.

If that API key leaks or an attacker finds an unauthenticated internal endpoint, your server accepts the request.

Mutual TLS (mTLS) flips this dynamic by requiring both sides to prove their identity during the cryptographic TLS handshake. The client verifies the server certificate, and the server verifies the client certificate against an internal Certificate Authority before passing a single byte of HTTP data to your backend application.

If the client presents an untrusted certificate or no certificate at all, Nginx terminates the TCP connection immediately at the TLS boundary.

How mTLS Works

In a standard TLS handshake, only the server provides a certificate. In mTLS, the server sends a CertificateRequest message during the handshake, telling the client which Certificate Authorities it trusts.

1. Setting Up the Private Certificate Authority (CA)

Do not use public commercial CAs (like Let's Encrypt) to verify client certificates in mTLS. Anyone can get a Let's Encrypt certificate for their own domain. If you configured Nginx to trust Let's Encrypt for client verification, any attacker with a valid Let's Encrypt certificate could authenticate.

You must build your own private CA.

Create a secure directory for CA assets on your administrative machine:

mkdir -p ~/pki/ca
cd ~/pki/ca
chmod 700 ~/pki/ca

Generate Root CA Private Key

Generate an elliptic-curve private key (ED25519 or ECDSA P-256):

# Generate private key for the Root CA
openssl genpkey -algorithm ED25519 -out ca.key

# Restrict file permissions
chmod 400 ca.key

Generate Root CA Certificate

Create a self-signed Root CA certificate valid for 10 years:

openssl req -x509 -new -nodes \
  -key ca.key \
  -sha256 \
  -days 3650 \
  -out ca.crt \
  -subj "/C=ID/ST=Jakarta/O=Aljabar Infra/OU=Security/CN=Aljabar Root CA"

The ca.crt file is your trust anchor. Nginx needs this file to verify incoming client certificates. Keep ca.key private on your local machine or dedicated signing server; never copy ca.key to public web servers.

2. Setting Up the Server Certificate

The server certificate proves your server identity to clients (like standard HTTPS). You have two choices:

1. Public Domain: Use Let's Encrypt via Certbot. Clients automatically trust it via system root stores.
2. Private Internal IP / Subdomain: Sign a server certificate using your private Root CA.

For production web services, standard Let's Encrypt certificates work well for the server side. For internal private APIs, generate a server cert with your CA:

# Generate server private key
openssl genpkey -algorithm ED25519 -out server.key

# Create config with Subject Alternative Name (SAN)
cat <<EOF > server_ext.cnf
[req]
prompt = no
distinguished_name = req_distinguished_name
req_extensions = v3_req

[req_distinguished_name]
C = ID
ST = Jakarta
O = Aljabar Infra
CN = api.internal.local

[v3_req]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1 = api.internal.local
DNS.2 = localhost
IP.1 = 127.0.0.1
EOF

# Generate CSR
openssl req -new -key server.key -out server.csr -config server_ext.cnf

# Sign server certificate using our Root CA
openssl x509 -req -in server.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt -days 365 \
  -extfile server_ext.cnf -extensions v3_req

3. Generating and Signing Client Certificates

Every client device, background worker, or microservice that needs access gets its own unique certificate.

Let us create a certificate for a background worker service named worker-service-01.

Step 1: Generate Client Private Key and CSR

# Generate client key
openssl genpkey -algorithm ED25519 -out client_worker1.key

# Generate CSR with specific client identity in Common Name (CN)
openssl req -new -key client_worker1.key \
  -out client_worker1.csr \
  -subj "/C=ID/ST=Jakarta/O=Engineering/OU=Workers/CN=worker-service-01"

Step 2: Create Client Extensions File

Modern TLS stacks verify that the certificate explicitly allows clientAuth:

# client_ext.cnf
basicConstraints = CA:FALSE
keyUsage = digitalSignature
extendedKeyUsage = clientAuth

Step 3: Sign Client Certificate with Root CA

openssl x509 -req -in client_worker1.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client_worker1.crt -days 365 \
  -extfile client_ext.cnf

Step 4: Export to PKCS#12 (.p12) for Browsers / Mobile Devices (Optional)

If your client is a web browser (Safari, Chrome) or an iOS/Android application, package the certificate and private key into a single password-protected PKCS#12 bundle:

openssl pkcs12 -export \
  -in client_worker1.crt \
  -inkey client_worker1.key \
  -certfile ca.crt \
  -out client_worker1.p12

4. Configuring Nginx for mTLS

Copy ca.crt to /etc/ssl/myca/ca.crt on your Nginx server.

Edit your Nginx virtual host configuration (for example /etc/nginx/sites-available/api-mtls):

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name api.internal.local;

    # Server TLS credentials (Public or Private)
    ssl_certificate /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/private/server.key;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Client Certificate Verification
    ssl_client_certificate /etc/ssl/myca/ca.crt;
    ssl_verify_client on;
    ssl_verify_depth 2;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;

        # Forward client identity to upstream application
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Pass verified TLS metadata
        proxy_set_header X-SSL-Client-Verify $ssl_client_verify;
        proxy_set_header X-SSL-Client-DN $ssl_client_s_dn;
        proxy_set_header X-SSL-Client-Serial $ssl_client_serial;
        proxy_set_header X-SSL-Client-Fingerprint $ssl_client_fingerprint;
    }
}

Understanding ssl_verify_client Modes

Nginx supports four modes for ssl_verify_client:

• on: Enforces client certificates. Nginx rejects requests without a valid certificate during handshake.
• off: Default behavior. No client certificates requested.
• optional: Requests a certificate, but continues handshake even if the client sends none. Useful if your backend serves public content alongside restricted routes based on $ssl_client_verify.
• optional_no_ca: Requests a certificate, but does not check it against ssl_client_certificate. Verification logic is offloaded to application code.

Test and reload Nginx:

nginx -t && systemctl reload nginx

5. Testing mTLS with cURL and Python

Test 1: Request Without Certificate (Expected to Fail)

curl -v https://api.internal.local/data --cacert ca.crt

Nginx terminates the connection before processing HTTP headers:

* OpenSSL SSL_connect: Connection reset by peer in connection to api.internal.local:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: Connection reset by peer

Test 2: Request With Untrusted Certificate (Expected to Fail)

If an attacker signs their own certificate using a different CA:

curl -v https://api.internal.local/data \
  --cacert ca.crt \
  --cert fake_client.crt \
  --key fake_client.key

Nginx responds with an SSL alert:

* SSL certificate problem: self-signed certificate in certificate chain
* Closing connection 0
curl: (35) error:0A000086:SSL routines::certificate verify failed

Test 3: Request With Valid Certificate (Success)

curl -v https://api.internal.local/data \
  --cacert ca.crt \
  --cert client_worker1.crt \
  --key client_worker1.key

The TLS handshake succeeds, and the backend receives the request:

* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: CN=api.internal.local
*  issuer: CN=Aljabar Root CA
< HTTP/2 200 
< content-type: application/json
{
  "status": "authenticated",
  "client": "worker-service-01"
}

6. Consuming Client Identity in Backend Code

Because Nginx validated the certificate, your upstream application can trust the forwarded X-SSL-Client-* headers.

Here is a Python FastAPI example reading client credentials from headers:

# app.py
from fastapi import FastAPI, Header, HTTPException, status

app = FastAPI()

@app.get("/api/v1/resource")
async def get_protected_resource(
    x_ssl_client_verify: str = Header(None),
    x_ssl_client_dn: str = Header(None),
    x_ssl_client_serial: str = Header(None),
):
    # Verify Nginx confirmed client validity
    if x_ssl_client_verify != "SUCCESS":
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Valid client certificate required",
        )

    # Extract Common Name (CN) from Distinguished Name string
    # Example DN: CN=worker-service-01,OU=Workers,O=Engineering,ST=Jakarta,C=ID
    client_cn = "unknown"
    if x_ssl_client_dn:
        for part in x_ssl_client_dn.split(","):
            if part.strip().startswith("CN="):
                client_cn = part.strip().split("=")[1]

    return {
        "message": "Access granted",
        "authenticated_client": client_cn,
        "serial_number": x_ssl_client_serial,
    }

7. Revoking Compromised Certificates with CRL

If a laptop is stolen or a worker private key is exposed, you need a way to invalidate that specific certificate before its expiration date.

Step 1: Set Up OpenSSL CA Database

Create the minimal index and serial files required for OpenSSL revocation management:

cd ~/pki/ca
touch index.txt
echo 1000 > crlnumber

Create openssl_ca.cnf:

[ca]
default_ca = CA_default

[CA_default]
dir           = .
database      = $dir/index.txt
crlnumber     = $dir/crlnumber
certificate   = $dir/ca.crt
private_key   = $dir/ca.key
default_md    = sha256
default_crl_days = 30

Step 2: Revoke the Compromised Certificate

# Revoke certificate
openssl ca -config openssl_ca.cnf -revoke client_worker1.crt

Step 3: Generate the CRL File

openssl ca -config openssl_ca.cnf -gencrl -out crl.pem

Step 4: Configure Nginx to Check CRL

Copy crl.pem to /etc/ssl/myca/crl.pem on the Nginx server and add ssl_crl:

ssl_client_certificate /etc/ssl/myca/ca.crt;
ssl_crl /etc/ssl/myca/crl.pem;
ssl_verify_client on;

Reload Nginx:

nginx -t && systemctl reload nginx

Any subsequent connection using client_worker1.crt is rejected during the TLS handshake with certificate revoked.