These last weeks I'm finally settling in my garage network setup, the place where my nomad cluster lives. An old laptop is part of that setup too, docked, always on, running some long lived stuff. With full disk encryption, which means every reboot, planned or not, stops at a passphrase prompt until somebody walks over and types it.

What I wanted was quite simple; when the laptop boots inside my home network it unlocks itself, anywhere else it asks for the passphrase like it always did. That is exactly what network bound disk encryption does, with tang on the server side and clevis on the client side.

A TPM would have been the other option, but that laptop doesn't expose a TPM 2.0 to linux, clevis only speaks TPM2, and a TPM unlocks the disk wherever the laptop is which isn't what I was after anyway.

tang itself is a tiny stateless http server holding a signing and an exchange key. When you bind a LUKS device to it, clevis derives a secret from the server's advertisement and stores it, encrypted, in a new keyslot. At boot that secret can only be recovered by talking to the same tang server again. The server never sees the disk key and keeps no state about its clients, so when it's gone you're simply back at the passphrase prompt.

image

There is no official tang container image. The popular community one only has latest and <git hash> tags, so no way to pin it nor to get notified about updates. My first idea was an alpine image with the tang package, until it turned out alpine doesn't package tang at all. Debian does, but one release behind upstream.

So I build it myself from the upstream release tag, the same way I did for my cloudflared image back in the days. A multi stage alpine build, meson in the first stage, only tangd and its runtime libraries in the second, running as an unprivileged user with tini as PID 1. The full Dockerfile lives in the dockerfile-tang repository.

Since v16 tangd can listen on a port by itself (tangd -l), so no socat or systemd socket activation needed anymore. The entrypoint.sh only generates keys when the directory is still empty and then hands over to tangd;

entrypoint.sh

#!/bin/sh
set -e

if [ ! -d "${JWKDIR}" ] || [ ! -w "${JWKDIR}" ]; then
    echo "jwkdir ${JWKDIR} does not exist or is not writable" >&2
    exit 1
fi

# Generate a signing and an exchange key on first start only. Existing keys,
# including rotated ones (.<thumbprint>.jwk), are never touched.
if ! ls "${JWKDIR}"/*.jwk >/dev/null 2>&1; then
    echo "no keys in ${JWKDIR}, generating"
    /usr/libexec/tangd-keygen "${JWKDIR}"
fi

exec /usr/libexec/tangd -l -p "${PORT}" "${JWKDIR}"

tini wasn't in my first version. Turns out tangd running as PID 1 ignores a SIGTERM, so every stop of the container waited the full 10 seconds before getting killed.

The goss tests don't stop at an open port, they fetch a real advertisement from the running container and count the keys in it;

command:
  advertisement is a signed JWS with both keys:
    exec: sh -c 'wget -qO- http://127.0.0.1:8080/adv | jose fmt -j- -Og payload -SyOg keys -A -l -o-'
    exit-status: 0
    stdout:
    - "2"

And before anything went near a real disk I did a clevis round trip against the container from a throwaway debian one with clevis installed;

$ echo "secret-ok" | clevis encrypt tang '{"url":"http://127.0.0.1:8080"}' -y > /tmp/jwe && echo "encrypted $(wc -c < /tmp/jwe) bytes"
encrypted 1248 bytes
$ clevis decrypt < /tmp/jwe
secret-ok

The image is available on docker hub and the source on github.

server

The keys are the only state tang has, so they live on persistent storage. Losing them isn't data loss, but every bound client falls back to its passphrase until it's bound again. The container runs in my homelab behind my internal reverse proxy, so the url clevis uses is https and only reachable from inside the house;

$ curl -s https://tang.example.lan/adv | jose fmt -j- -Og payload -SyOg keys -AUo-

Before touching the boot of the laptop I did that same round trip once more, this time from the laptop itself against the real https url. Same path the boot would take, minus the boot.

initramfs

The laptop runs arch with mkinitcpio and the classic encrypt hook. I first went looking for a clevis hook for mkinitcpio, but the one in the AUR is TPM only: no network, no curl, no CA certificates. clevis upstream however ships dracut modules, arch packages them, and dracut has proper network support.

I didn't want to bet my only way of booting on that though. So dracut builds a second initramfs next to the mkinitcpio one, with its own boot menu entry. If it breaks I pick the old entry and type my passphrase like before;

/etc/dracut.conf.d/tang.conf

hostonly="yes"
hostonly_cmdline="no"
add_dracutmodules+=" crypt clevis clevis-pin-tang network url-lib "
add_drivers+=" e1000e "

That url-lib module is easy to miss. The clevis-pin-tang module pulls in curl but no CA bundle, so https towards tang fails in the initramfs without it. The e1000e driver is the onboard NIC, the network port of the dock is just passed through to it so no usb network driver needed.

Binding the disk adds a keyslot next to the one with the passphrase;

# clevis luks bind -d /dev/sda2 tang '{"url":"https://tang.example.lan"}'
# clevis luks list -d /dev/sda2
1: tang '{"url":"https://tang.example.lan"}'

The image gets its own file name and a syslinux entry to boot it;

# dracut --force --kver "$(uname -r)" /boot/initramfs-linux-dracut.img

LABEL archtang
    MENU LABEL Arch Linux (tang unlock)
    LINUX ../vmlinuz-linux
    INITRD ../intel-ucode.img,../initramfs-linux-dracut.img
    APPEND root=/dev/mapper/root rootflags=subvol=__current/root rd.luks.uuid=<uuid> rd.luks.name=<uuid>=root rd.neednet=1 ip=dhcp rw

Nothing rebuilds that image after a kernel update though, and an initramfs without the modules of the running kernel won't boot. The dracut-hook package from the AUR would have done it, but it overwrites initramfs-linux.img, which is exactly the image my fallback entry boots. Hence a small pacman hook of my own which only touches that one file;

/etc/pacman.d/hooks/90-dracut-tang.hook

[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Target = usr/lib/modules/*/vmlinuz

[Trigger]
Type = Package
Operation = Install
Operation = Upgrade
Target = dracut
Target = clevis
Target = cryptsetup
Target = systemd

[Action]
Description = Rebuilding tang initramfs (dracut)...
When = PostTransaction
Exec = /usr/local/bin/dracut-tang-rebuild

/usr/local/bin/dracut-tang-rebuild

#!/bin/sh
set -e
kver=""
for f in /usr/lib/modules/*/pkgbase; do
    [ "$(cat "$f")" = "linux" ] && kver=$(basename "$(dirname "$f")")
done
[ -n "$kver" ] || { echo "dracut-tang: no linux kernel found" >&2; exit 1; }
dracut --force --kver "$kver" /boot/initramfs-linux-dracut.img

wake on lan

With the passphrase out of the way there was one manual step left, the power button. The network profile of that laptop already enabled wake on lan on the onboard NIC (enp0s25) each time the link comes up;

/etc/netctl/wired

Description='Wired connection'
Interface=enp0s25
Connection=ethernet
IP=dhcp
ExecUpPost='/usr/bin/ethtool -s enp0s25 wol g'

A magic packet is a broadcast though, and broadcasts don't cross VLANs. My own laptop sits in another network than this one, so the packet has to come from something that lives in the same one. The router does, and RouterOS has a wake on lan tool built in, /tool wol;

/tool wol mac=AA:BB:CC:DD:EE:FF interface=bridge-LAN

From a laptop that is switched off to one that is up and running, without touching it once.

trade-off

tang moves the protection from something you know to somewhere you are. Somebody who steals the laptop and plugs it into my network gets an unlocked disk, that's just how it works. Outside of my network nothing changed. For an old laptop which never leaves its dock that's a trade I'm happy to make, for the laptop I carry around every day it wouldn't be.

When tang, the reverse proxy or DNS is down during a boot the laptop just asks for the passphrase, nothing gets stuck. Rotating the keys is a tangd-rotate-keys on the server followed by a clevis luks regen on every client, the old keys are kept as hidden files so clients which aren't regenerated yet keep on working in between. And the key directory of the tang server deserves the same backup treatment as any other secret.

Thanks to the latchset folks for tang and clevis, and for shipping those dracut modules which made the whole initramfs part a configuration file instead of a project.

references;