Step-CA (mutual TLS)

Mutual TLS (mTLS) is a form of transport security where both the client and server authenticate each other using digital certificates. Unlike traditional TLS (e.g., Let’s Encrypt), where only the server’s identity is verified, mTLS provides mutual verification of the server and client, making it ideal for protecting data exchanges in interconnected environments like microservices and zero-trust architectures. By using certificates issued by trusted Certificate Authorities (CA), mTLS prevents unauthorized access and guarantees that both parties using a communication channel are legitimate.

Step-CA is an open source Certificate Authority (CA) and an Automated Certificate Management Enviornment (ACME). You can self-host Step-CA and use it to add secure mTLS authentication in all of your applications and infrastructure.

Info

This chapter is focused soley on enhancing client authentication with mTLS, but it still uses the Let’s Encrypt CA for the server certificates. If you don’t want to rely upon Let’s Encrypt, and you want to run a fully self-hosted Public Key Infrastructure (PKI) for all of your services, please see the appendix chapter Private ACME.

Although primarily used for machine-to-machine communication, mTLS can also be used in the web browser to provide strong end-user client authentication, enhancing security for sensitive web applications. This is an advanced topic, and will be discussed in the appendix chapter Mutual TLS for Web and Mobile.

Configure Step-CA

Run this on your Raspberry Pi
pi make step-ca config

Set the hostname for Step-CA itself:

(stdout)
STEP_CA_TRAEFIK_HOST: Enter the step-ca domain name (eg. ca.example.com)

: ca.pi.example.com

Set the allowed list of domains to create certificates for:

(stdout)
STEP_CA_AUTHORITY_POLICY_X509_ALLOW_DNS: Enter the list of allowed domain wildcards (comma separated) (eg. *.example.com,*.example.org)

: *.clients.pi.example.com
Info

In an earlier chapter Traefik was setup with ACME. All of the server-side applications on your Raspberry Pi already have automatic TLS certificates issued via Let’s Encrypt, which is a free and public CA, therfore, in this example Step-CA will not be required for any server-side certificates. Lets Encrypt does not support requesting client certificates, so Step-CA will be required only to create and verify the client-side certificates (*.clients.pi.example.com).

In this mode, mTLS authentication works like this:

  • The server will verify the client’s certificate using the Step-CA authority, whose root certificate must be added to the server’s trust store during install.
  • The client will verify the server’s certificate using the Let’s Encrypt authority, whose root certificate is already widely trusted in most operating systems factory default settings, which simplifies installation.
---
title: Mutual TLS with two separate Certificate Authorities
---
sequenceDiagram
    title Mutual TLS with two separate Certificate Authorities

participant Client
participant StepCA as Step CA
participant Server
participant LetsEncrypt as Let's Encrypt CA

Client->>StepCA: Requests new client certificate
StepCA->>Client: Receives new client certificate
Server->>LetsEncrypt: Requests new server certificate
LetsEncrypt->>Server: Receives new server certificate
Client->>Server: Sends Step-CA signed request
Server->>Client: Receives Let's Encrypt signed response

Theoretically you could just use Step-CA for both sides, but this would require modifying the root certificate store on every client, which is a risky and complex procedure, and ultimately unnecessary because Let’s Encrypt is already commonly trusted by all operating systems and browsers.

Tip

Each client certificate must have a unique name (CN) written in domain name format (e.g., foo.clients.pi.example.com), but this does not need to resolve to any IP address. Each certificate can have an arbitrary name as long as it matches the Step-CA policy (STEP_CA_AUTHORITY_POLICY_X509_ALLOW_DNS). The certificate name is used as the transport authentication id (essentially used as a username), and it is forwarded to your backend HTTP services through a trusted header X-Client-CN for the purpose of secondary authorization by the app itself.

Define default TLS certificate expiration

By default, TLS certificates are valid for 7 days, and then must be re-issued. You can customize the certificate durations in the config. There are three limits defined:

  • STEP_CA_AUTHORITY_CLAIMS_MIN_TLS_CERT_DURATION=5m the minimum duration a certificate may be requested for, 5 minutes by default.
  • STEP_CA_AUTHORITY_CLAIMS_MAX_TLS_CERT_DURATION=2160h the maximum duration a certificate may be requested for, 90 days be default.
  • STEP_CA_AUTHORITY_CLAIMS_DEFAULT_TLS_CERT_DURATION=168h the default duration a certificate will be issued for if the client does not specify one, 7 days by default.
Run this on your Raspberry Pi
## Increase the minimum duration to 1 day:
pi make step-ca reconfigure var=STEP_CA_AUTHORITY_CLAIMS_MIN_TLS_CERT_DURATION=24h

## Increase the maximum duration to 1 year:
pi make step-ca reconfigure var=STEP_CA_AUTHORITY_CLAIMS_MAX_TLS_CERT_DURATION=8760h

## Increase the default duration to 90 days:
pi make step-ca reconfigure var=STEP_CA_AUTHORITY_CLAIMS_DEFAULT_TLS_CERT_DURATION=2160h

Install Step-CA

Run this on your Raspberry Pi
pi make step-ca install wait

Retrieve the admin password

The admin password is printed in the container logs only the first time it starts. Retrieve it by running:

Run this on your Raspberry Pi
pi make step-ca logs-out | grep password
Tip

Save this password to a safe place, you will need it everytime you sign a new (non-ACME) certificate.

Finish Step-CA configuration by restarting the service

Important

After installation, you must restart Step-CA one more time to finish the setup procedure. This will finalize the configuration in /home/step/config/ca.json.

Run this on your Raspberry Pi
## Re-install forces the service to restart:
pi make step-ca reinstall

This also has a side effect of clearing the admin password from the logs, hope you saved it!

Retrieve the server CA cert fingerprint:

Run this on your Raspberry Pi
pi make step-ca inspect-fingerprint

The fingerprint is the public identifier of the Step-CA root certificate. Copy it into your clipboard for use in the next section.

Reconfigure Traefik

Run this on your Raspberry Pi
pi make traefik config

Configure Traefik for mTLS

(stdout)
? Traefik:
> Config
  Install (make install)
  Admin
  Exit (ESC)

? Traefik Configuration:
  Traefik user
  Entrypoints (including dashboard)
> TLS certificates and authorities
  Middleware (including sentry auth)
  Advanced Routing (Layer 7 / Layer 4 / Wireguard)
  Error page template
v Logging level

? Traefik TLS config:
> Configure certificate authorities (CA)
  Configure ACME (Let's Encrypt or Step-CA)
  Configure TLS certificates (make certs)

? How do you want to configure the list of trusted Certificate Authorities (CA)?
  Use the stock list, (alpine: ca-certificates ca-certificates-bundle).
> Use the stock list, plus add my own root Step-CA certificate.
  Delete the entire list, and add my own root Step-CA certificate.
  Delete the entire list.
  Cancel / Go back.

Enter the Step-CA endpoint URL:

(stdout)
TRAEFIK_STEP_CA_ENDPOINT: Enter your Step-CA endpoint URL (eg. https://ca.example.com)

: https://ca.pi.example.com

You will need to paste the fingerprint you copied from the last section into the answer for the next question:

(stdout)
TRAEFIK_STEP_CA_FINGERPRINT: Enter your Step-CA root CA certificate
fingerprint (eg. xxxxxxxxxxxxxxxxxxxx)

: xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Press ESC two times to go back to the main menu, then re-install Traefik:

(stdout)
? Traefik:
  Config
> Install (make install)
  Admin
  Exit (ESC)

After installation, press ESC to quit the config tool.

Add a new route on the sentry

Run this on your Raspberry Pi
sentry route set pi ca.pi.example.com
Tip

You may also create the route interactively through the Traefik config menu.

Log in with the step-cli client

Run this on your Raspberry Pi
pi make step-ca client-bootstrap
(stdout)
The root certificate has been saved in /home/pi/.step/certs/root_ca.crt.
The authority configuration has been saved in /home/pi/.step/config/defaults.json.

Create certificates

Do this anytime you wish to create a new client certificate:

Run this on your Raspberry Pi
pi make step-ca cert

Enter the client certificate name (CN):

(stdout)
Enter the subject (CN) to be certified, a domain name, or a client name

: foo.clients.pi.example.com

When signing certificates, you need to enter the step-ca root admin passphrase, and then the certificates will be signed:

(stdout)
Please enter the password to decrypt the provisioner key: xxxxx

✔ Provisioner: admin (JWK)
✔ Certificate: certs/foo.clients.pi.example.com.crt
✔ Private Key: certs/foo.clients.pi.example.com.key
Warning

The next question asks you to set a temporary passphrase to encrypt the .p12 formatted client key.

(stdout)
Please enter a password to encrypt the .p12 file:

The p12 password should NOT be the same as the root CA password! It should be a separate password that you will share with the end user, so that they can unlock the certificate you are giving them.

Finally you will see some details printed about the issued certificate.

(stdout)
Certificate:
    Data:
       .....
        Validity
            Not Before: Oct 19 17:51:27 2024 UTC
            Not After : Oct 26 17:52:27 2024 UTC
        Subject: CN=foo.clients.pi.example.com

The client certificate foo has been written to the step-ca/certs directory (.e.g., ~/git/vendor/enigmacurry/d.rymcg.tech/step-ca/certs.)

  • foo.clients.pi.example.com.crt This is the public certificate for the foo client.
  • foo.clients.pi.example.com.key This is the private key for the foo certificate. This file is unencrypted.
  • foo.clients.pi.example.com.p12 This is also the private key for the foo certificate written in a different format that is used on some devices, this file is encrypted with the temporary passphrase you chose and needs to be provided to the client when installing the certificate.

Configure apps for mTLS

For demo purposes, enable mTLS for the whoami service:

Run this on your Raspberry Pi
pi make whoami config
(stdout)
WHOAMI_TRAEFIK_HOST: Enter the whoami domain name (eg. whoami.example.com)

: whoami.pi.example.com

? Do you want to enable sentry authorization in front of this app (effectively making the entire site private)?
  No
  Yes, with HTTP Basic Authentication
  Yes, with Oauth2
> Yes, with Mutual TLS (mTLS)

WHOAMI_MTLS_AUTHORIZED_CERTS: Enter comma separated list of allowed client certificate names (or blank to allow all) (eg. *.clients.whoami.example.com)

: *.clients.pi.example.com
Tip

Make sure to enter the same authorized cert wildcard that matches your client certificate, otherwise you will encounter the error No matching DNS names.

Re-install whoami:

Run this on your Raspberry Pi
pi make whoami install

Test the service without specifying a certificate:

Run this on your Raspberry Pi
curl https://whoami.pi.example.com

You should now receive one of the following errors, both indicating that a client certificate is required:

(stdout)
curl: (56) OpenSSL SSL_read: OpenSSL/3.0.14: error:0A00045C:SSL routines::tlsv13 alert certificate required, errno 0
curl: (55) getpeername() failed with errno 107: Transport endpoint is not connected

Now test it with your client certificate:

Run this on your Raspberry Pi
(
NAME=foo.clients.pi.example.com
CERTS=~/git/vendor/enigmacurry/d.rymcg.tech/step-ca/certs

curl https://whoami.pi.example.com \
--cert ${CERTS}/${NAME}.crt \
--key ${CERTS}/${NAME}.key
)

With the correctly supplied certificate and key, the server now resolves to the whoami response:

(stdout)
Name: default
Hostname: 427db0a29c30
IP: 127.0.0.1
IP: ::1
IP: 172.19.0.2
RemoteAddr: 172.19.0.1:45068
GET / HTTP/1.1
Host: whoami.pi.example.com
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Client-Cn: CN=foo.clients.pi.example.com
X-Forwarded-For: 127.0.0.1
X-Forwarded-Host: whoami.pi.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: pi5
X-Forwarded-User: CN=foo.clients.pi.example.com
X-Real-Ip: 127.0.0.1

Notice that there are two headers sent to the whoami backend:

  • X-Client-Cn
  • X-Forwarded-User

These two headers contain the same information, identifying the client cert id to the backend server. Either of these may be used for secondary authorization in your app.