TLS Handshake Failures: How to Diagnose and Fix Them

A failed TLS handshake is almost always one of seven things, and you can identify which one in roughly 20 minutes per endpoint using OpenSSL and the terminating server's error log. This playbook reads the TLS handshake failure from both ends at once — the alert code the peer sends on the wire, and the error string your client prints — then applies the certificate, protocol, or cipher fix that actually resolves it. Assumes an existing nginx origin, shell access, and one config reload. Most wasted debugging time comes from looking at one side only: the client says "handshake failure" while the server log already names the exact missing parameter.

What causes a TLS handshake failure: the alert-code map

TLS alerts are numbered in RFC 8446 and RFC 5246, and most stacks print the number or its name. Start every investigation here, because the alert tells you which side rejected what.

If you can read the alert number, you have already narrowed a TLS handshake failure from seven possible causes to one or two.

Prerequisites

• OpenSSL 3.x (1.1.1 works, but lacks post-quantum group support) on a host that can reach the endpoint.
• Shell access to the terminating server and permission to reload it.
• The leaf certificate and the intermediate bundle as separate files.
• A second vantage point outside your corporate network, to rule out inspection middleboxes.

Handshake troubleshooting, step by step

Step 1 — Reproduce the TLS handshake failure from the client side

openssl s_client -connect EXAMPLE_HOSTNAME:443 -servername EXAMPLE_HOSTNAME -showcerts -status
openssl s_client -connect EXAMPLE_HOSTNAME:443 -servername EXAMPLE_HOSTNAME -tls1_2
openssl s_client -connect EXAMPLE_HOSTNAME:443 -servername EXAMPLE_HOSTNAME -tls1_3
openssl s_client -connect EXAMPLE_HOSTNAME:443 -servername EXAMPLE_HOSTNAME -alpn h2
openssl s_client -connect ORIGIN_IP:443 -servername EXAMPLE_HOSTNAME

Substitute EXAMPLE_HOSTNAME with the public hostname and ORIGIN_IP with the origin address. Running the version-pinned variants tells you immediately whether the failure is version-scoped; the last line bypasses any proxy in front of the origin so you know which hop is failing.

Step 2 — Read the same handshake from the server side

nginx writes the decisive string to the error log: "no shared cipher", "unsupported protocol", "certificate verify failed", "peer closed connection in SSL handshake". Raise verbosity only for the failing listener, and only briefly, since debug logging on a busy terminator can add tens of megabytes per minute.

error_log ERROR_LOG_PATH debug;

log_format tlsdiag '$remote_addr $ssl_protocol $ssl_cipher '
                   '$ssl_server_name $ssl_session_reused $status';

ERROR_LOG_PATH is your existing error log destination. The tlsdiag format gives you a per-request record of negotiated version and suite, which is what you need to prove the fix later.

Step 3 — Verify the chain, not just the leaf

openssl x509 -noout -subject -issuer -dates -in LEAF_PEM_PATH
openssl verify -untrusted CHAIN_PEM_PATH LEAF_PEM_PATH

LEAF_PEM_PATH is the server certificate, CHAIN_PEM_PATH the intermediates. Browsers hide missing intermediates by fetching them through the Authority Information Access extension, so the classic signature of this bug is "works in Chrome, certificate error in the Go or Java client". Chain order matters: leaf first, then each issuer, no root required.

Publicly trusted TLS certificates issued on or after 1 September 2020 are capped at 398 days of validity, and the CA/Browser Forum schedule adopted in 2025 shortens that ceiling to 200 days in March 2026 and 100 days in March 2027. Expiry-driven handshake failures therefore shift from a yearly risk to a quarterly one, which makes automated renewal plus a 30-day expiry alert a hard requirement rather than hygiene.

Step 4 — Fix protocol and cipher overlap

server {
    listen 443 ssl;
    http2 on;
    server_name EXAMPLE_HOSTNAME;

    ssl_certificate     FULLCHAIN_PEM_PATH;
    ssl_certificate_key PRIVATE_KEY_PEM_PATH;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305;
    ssl_prefer_server_ciphers off;

    ssl_session_cache shared:TLS:10m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;

    ssl_stapling on;
    ssl_stapling_verify on;
}

FULLCHAIN_PEM_PATH must be leaf plus intermediates in one file; PRIVATE_KEY_PEM_PATH is the matching key. Keeping both ECDSA and RSA suites present is what prevents alert 40 from clients that support only one signature algorithm. If your estate includes payment terminals or embedded devices, expect a subset that still cannot do TLS 1.2 with AEAD suites; isolate those on a dedicated listener rather than weakening the main one.

Step 5 — Check SNI and virtual host selection

A client that omits SNI gets whatever certificate the default server presents, which produces a hostname-mismatch certificate error rather than a clean alert. Older Java runtimes and some IoT stacks still do this. Confirm by comparing the subject returned with and without the -servername flag in Step 1.

Step 6 — Rule out the oversized ClientHello

Since Chrome 124 in 2024, hybrid post-quantum key exchange has been on by default, moving to X25519MLKEM768 in Chrome 131 later that year. The resulting ClientHello exceeds 1,400 bytes and no longer fits in a single TCP segment. TLS terminators and inspection devices that assume a one-segment ClientHello then stall or reset, producing an ssl handshake failure that reproduces in Chrome but not in older tooling. Test by forcing a classical group, and if that resolves it, patch the terminator rather than permanently disabling the group.

Validation: what a healthy handshake looks like

A successful run of Step 1 prints "Verify return code: 0 (ok)", a chain with depth of at least 1, a negotiated line such as "New, TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256", and "OCSP Response Status: successful" when stapling is on. In the access log, $ssl_protocol should show TLSv1.3 for modern clients and $ssl_session_reused should climb above zero within a minute of steady traffic. Count occurrences of "SSL_do_handshake() failed" per minute before and after; the target is zero for clients you intend to support.

Rollback

Keep the previous certificate bundle and the previous server block. Validate syntax before reloading, and reload rather than restart so established connections survive. If handshake errors rise after a change, revert the ssl_protocols and ssl_ciphers lines first, reload, and confirm within 60 seconds using the tlsdiag log.

Symptom, cause, fix

If the same endpoint succeeds from one network and returns a certificate error from another, the certificate is fine and the network path is rewriting it.

Tuning once the handshake succeeds

Session resumption is the biggest remaining win: a resumed TLS 1.3 handshake costs zero extra round trips beyond TCP, versus one for a full handshake. Tickets scale better across a fleet than a shared cache, but rotating ticket keys badly weakens forward secrecy, so pick one and own the key rotation. Enabling 0-RTT trades replay safety for latency and belongs only on idempotent requests.

Where TLS terminates at a CDN, every check above runs twice: client-to-edge and edge-to-origin. The origin leg is the one that quietly ships an incomplete chain for years, because no browser ever touches it. Providers differ in how much of that leg you control, and BlazingCDN exposes edge and origin TLS configuration options separately, which is where you pin the full chain and the accepted protocol range for origin pulls.

FAQ: TLS handshake failure troubleshooting

Why does an SSL handshake failure affect only some clients?

Selective failures mean the negotiation parameters differ per client, not that the server is broken. Older runtimes offer narrower cipher and signature algorithm sets, skip SNI, or carry stale trust stores. Compare the negotiated $ssl_protocol and $ssl_cipher values for a working and a failing client; the difference names the parameter you need to add or upgrade.

How do I fix "unable to get local issuer certificate"?

Serve the intermediate certificates alongside the leaf in a single chain file, leaf first. That error means the verifying client could not build a path to a trusted root because the server presented only the end-entity certificate. Browsers mask the problem through Authority Information Access fetching, so validate with OpenSSL verify rather than trusting a green padlock.

Does disabling TLS 1.0 and TLS 1.1 cause tls handshake failed errors?

Yes, for any client that cannot negotiate TLS 1.2 or higher, and the failure surfaces as alert 70 protocol_version. Before disabling, log negotiated versions for a full week to size the affected population. If legacy devices remain, terminate them on a separate hostname and listener rather than lowering the minimum version fleet-wide.

Can a firewall or inspection proxy cause a TLS handshake failure?

Yes. Interception proxies re-sign traffic with a local certificate authority, and any client without that CA installed reports a certificate error. Devices that assume a single-segment ClientHello also break modern browsers using post-quantum key exchange. Testing from a network outside the inspection path isolates this in under a minute.

Run this against your top ten endpoints this week

Take the ten hostnames with the most traffic, run Step 1 against each from inside and outside your corporate network, and record the negotiated version, cipher, chain depth, and days-to-expiry in a single sheet. Anything with chain depth 0, a validity window under 30 days, or a differing issuer between vantage points is a future incident already scheduled. Then add the tlsdiag log format to one terminator and set an alert on handshake errors per minute. It takes an afternoon and it converts handshake troubleshooting from forensic work into a dashboard.