What a certificate chain is
A server's TLS certificate is almost never trusted on its own. It is vouched for by the certificate authority (CA) that issued it, and that CA's certificate is vouched for by another, until the chain reaches a root certificate that your system already trusts. The standard calls this a certification path[1]. Three kinds of certificate make it up:
- The leaf (also called the end-entity or server certificate) belongs to the site. Its subject lists the names it is valid for.
- Intermediates belong to CAs that sit between the leaf and the root. In public PKI a root usually signs only intermediates, and the intermediates do the day-to-day issuing.
- The root signs itself: its issuer and subject are the same name. It is trusted because it is in a trust store, not because it carries a signature anyone could check.
Every certificate names its issuer, and links to the issuer's key. The authority key identifier field is there for that: the standard says a conforming CA must include it in the certificates it issues, to make path construction easier[1]. A client matches the issuer name and the key identifier to find the next link up.
What a server sends
A server does not send the whole chain. In TLS 1.3 the sender's certificate comes first, and each following certificate should directly certify the one before it[2]. A certificate that names a trust anchor may be left out, provided the peers are known to have it[2]. In practice that means a server sends the leaf and its intermediates and leaves the root out, because the client has the root already.
The same rule explains the two most common failures. If the server omits an intermediate, the client has nothing to link the leaf to. If the client does not have the root, it has nowhere to end.
See the chain a server sends
OpenSSL prints every certificate the server sends, in order, when you ask for them:
openssl s_client -connect example.com:443 -servername example.com -showcerts
Each block between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- is one certificate, and the first one is the leaf. Copy the blocks into a file, or paste them together into the Certificate Decoder. To see how a chain would be verified, give OpenSSL the pieces yourself:
openssl verify -show_chain -CAfile root.pem -untrusted intermediate.pem leaf.pem
With a root, an intermediate and a leaf this prints the path, one line per level:
leaf.pem: OK Chain: depth=0: O=fjordtools test, CN=server.test (untrusted) depth=1: O=fjordtools test, CN=fjordtools test intermediate (untrusted) depth=2: O=fjordtools test, CN=fjordtools test root
Depth 0 is the leaf and a higher depth is an issuer. The same numbers appear in OpenSSL's error lines, for example error 20 at 1 depth lookup, so you know which link failed.
Read it in the Certificate Decoder
Paste the certificates, one block after another. The Decoder reads them in your browser, links them by name and key identifier, and gives each a role and a link:
- leaf · issued by 2 means the next certificate in the input is its issuer.
- intermediate · issuer not in the input is what a normal server chain looks like: the root was left out on purpose.
- root · self-signed means the issuer and subject are the same name.
- leaf · issuer not in the input on a leaf that should have an intermediate is the gap that makes clients fail.
The status line counts how many certificates are expired or not valid yet, and the order check tells you when the leaf is not first, which nginx, Apache and HAProxy all expect. The Decoder matches names and key identifiers. It does not check signatures or decide whether a root is trusted, so a chain that looks complete there can still fail on a client that lacks the root.
What goes wrong, and where to read about it
- An intermediate is missing. The server sent only the leaf. OpenSSL, curl and Python say unable to get local issuer certificate. Node says unable to verify the first certificate.
- The root is not trusted. The chain is complete and ends in a root the client does not have: self-signed certificate in certificate chain, typical for a private CA.
- The server's own certificate is self-signed. Nothing above it at all: self-signed certificate.
- A certificate in the chain is out of date. It may be the leaf or an intermediate; read the depth. Certificate has expired and certificate is not yet valid cover both.
- The chain is in the wrong order. Some clients cope and some do not. Put the leaf first and each issuer after it.
A certificate can also carry a CA issuers address, a place where its issuer can be downloaded. The standard says that information is meant to help a client pick a path that ends at a trusted point[1]. Whether a client uses it depends on the client. In our tests Node, OpenSSL, curl and Python each refused a server that sent only its own certificate, even when the certificate named a download address, so do not rely on it. Send the intermediates.
The short version of the fix
Configure the server with the leaf first, then each intermediate, root left out. Let's Encrypt clients write this file as fullchain.pem, and nginx takes it in ssl_certificate. Then run the s_client command above against the live site and paste the output into the Decoder: if it shows your leaf followed by your intermediates, a client has what it needs. Whether it trusts the root is a question about the client's trust store, and a private root has to be added there.
The Certificate Decoder runs in your browser. After the page has loaded it makes no network requests, and what you paste stays in the tab. You can check that yourself.