A TLS Terminating Reverse Proxy with OpenBSD

by ian

In my last post, one of the things I glossed over was the TLS termination setup. While there are many ways of doing this, and half a dozen tutorials on most of them, you may remember that my setup on the cloud VPS is explicitly trying to stay within the base OpenBSD distribution as much as possible. And it delivers, even for this use case!

httpd

You may recall that I'm using httpd for serving plain HTTP traffic. In fact, pretty much all that httpd does is respond to ACME challenges. ACME is the protocol that Let's Encrypt and other CAs use to issue and renew certificates, without requiring human involvement and 20 mins of cursing while you search for the right openssl invocation.

The first step to setting this up is httpd.conf, which is dead simple, so I won't give much commentary. Just substitute your domain(s) and IPs.

# /etc/httpd.conf

server "blog.ianwwagner.com" {
    alias "ianwwagner.com"
    alias "www.ianwwagner.com"
    listen on 46.23.95.102 port 80
    listen on 2a03:6000:95f2:627::80 port 80

    location "/.well-known/acme-challenge/*" {
        # NB: httpd runs chrooted to /var/www by default,
        # so this is /var/www from root's perspective
        root "/acme"
        request strip 2
    }

    location * {
        block return 302 "https://$HTTP_HOST$REQUEST_URI"
    }
}

server "matrix.ianwwagner.com" {
    listen on 46.23.95.102 port 80
    listen on 2a03:6000:95f2:627::6167 port 80

    location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
    }

    location * {
        block return 302 "https://$HTTP_HOST$REQUEST_URI"
    }
}

Note that both of the domains in this setup are listening on port 80. httpd disambiguates via the Host header.

acme-client configuration

The above handles the plain HTTP side, but this is really just in service to the ACME client. OpenBSD also has this in the base system: acme-client(1)! Configuration is predictably simple:

# /etc/acme-client.conf

authority letsencrypt {
    api url "https://acme-v02.api.letsencrypt.org/directory"
    account key "/etc/secrets/letsencrypt.key"
}

domain blog.ianwwagner.com {
    alternative names { ianwwagner.com www.ianwwagner.com }
    domain key "/etc/secrets/blog.ianwwagner.com.key" # NB: acme-client supports ecdsa, but relayd is RSA-only
    domain full chain certificate "/etc/ssl/blog.ianwwagner.com.crt"
    sign with letsencrypt
}

domain matrix.ianwwagner.com {
    domain key "/etc/secrets/matrix.ianwwagner.com.key"
    domain full chain certificate "/etc/ssl/matrix.ianwwagner.com.crt"
    sign with letsencrypt
}

I found it interesting (neutral) that while acme-client supports ECDSA, relayd only supports RSA certs. A more typical config will have the keys in /etc/ssl/private or similar, but I'm using an encrypted volume that I'll explain in a future post so keep that in mind. Other than that, I think the config is pretty self-explanatory.

Generating the initial certificates

Now we're all set to generate the first certs. First, enable httpd:

# Enable and start the HTTP daemon
rcctl enable httpd
rcctl start httpd

Then, trigger the initial cert generation:

# Trigger acme-client
acme-client -v blog.ianwwagner.com
acme-client -v matrix.ianwwagner.com

I'd recommend adding this to your crontab. In the process of writing this, I learned just how nonstandard crontab is; specifically how OpenBSD includes a nonstandard ~ for randomizing components!

~ * * * * mount | grep -q /etc/secrets && acme-client blog.ianwwagner.com && rcctl reload relayda
~ * * * * mount | grep -q /etc/secrets && acme-client matrix.ianwwagner.com && rcctl reload relayd

Setting up relayd with multiple domains

And now for the last stage: setting up relayd with multiple domains. This part was by far the least intuitive, so I'll have a lot more commentary in the config.

# /etc/relayd.conf

# Tables; these are somewhat specific to my configuration, but you can adapt.
# I'm running varnish (vinyl) cache locally,
# and my homelab backend is on the class B private subnet
table <vinyl> { 127.0.0.1 }
table <bsdcube> { 172.16.42.2 }

# --- Protocols ---
# Three protocols because each relay can only forward to tables declared
# in its block, and any table referenced by a protocol "match ... forward to"
# MUST appear in the relay. Using one shared protocol everywhere would force
# blog-only relays to declare the bsdcube table (and vice versa).
# Opinions on how to improve this are VERY welcome!

# Shared HTTPS protocol for the IPv4 :443 listener.
#
# Blog and Matrix share the same IPv4 address:
# - TLS SNI selects the certificate
# - an HTTP Host match selects the backend
# - requests that do not match a more specific rule use the relay's main
#   forward-to table
http protocol "https" {
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header append "X-Real-IP" value "$REMOTE_ADDR"
    match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    match response header remove "Server"

    # Multiple keypairs enable SNI; relayd serves the right cert
    # based on the ClientHello server_name extension.
    tls keypair "blog.ianwwagner.com"
    tls keypair "matrix.ianwwagner.com"

    # Host match overrides the relay's default (first) forward-to table.
    # Unmatched hosts fall through to <vinyl>.
    match request header "Host" value "matrix.ianwwagner.com" forward to <bsdcube>
}

# Blog-only (dedicated blog IPv6 address)
http protocol "blog_https" {
    match request header set "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header set "X-Real-IP" value "$REMOTE_ADDR"
    match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    match response header remove "Server"

    tls keypair "blog.ianwwagner.com"
}

# Matrix-only (dedicated matrix IPv6 :443 and federation :8448)
http protocol "matrix_https" {
    match request header set "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header set "X-Real-IP" value "$REMOTE_ADDR"
    match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    match response header remove "Server"

    tls keypair "matrix.ianwwagner.com"
}

# --- Relays ---
# One block per listen address...

# Port 443: shared IPv4 (SNI selects the certificate; Host selects the backend).
# The relay's main table is <vinyl>; the protocol's
# Host match can route Matrix traffic to <bsdcube>.
relay "https4" {
    listen on 46.23.95.102 port 443 tls
    protocol "https"
    forward to <vinyl> port 8080
    forward to <bsdcube> port 6167
}

# Port 443: blog IPv6
relay "blog_tls6" {
    listen on 2a03:6000:95f2:627::80 port 443 tls
    protocol "blog_https"
    forward to <vinyl> port 8080
}

# Port 443: matrix IPv6
relay "matrix_tls6" {
    listen on 2a03:6000:95f2:627::6167 port 443 tls
    protocol "matrix_https"
    forward to <bsdcube> port 6167
}

# Port 8448: matrix federation (IPv4)
relay "matrix_fed4" {
    listen on 46.23.95.102 port 8448 tls
    protocol "matrix_https"
    forward to <bsdcube> port 6167
}

# Port 8448: matrix federation (IPv6)
relay "matrix_fed6" {
    listen on 2a03:6000:95f2:627::6167 port 8448 tls
    protocol "matrix_https"
    forward to <bsdcube> port 6167
}

It's a bit more that I wish were necessary, but on the other hand, it's definitely not rocket science! Hopefully someone finds this useful; no need for Caddy or other packages with the OpenBSD base!

Related content