Seeking guidance on provisioning hosts to bootstrap SSH certificates in the presence of StrictHostKeyChecking.

[kaysond]

I have step-ca set up as an SSH CA, and I use Ansible to provision hosts. Currently, I have a provisioner password stored in an ansible vault, and this gets passed to step ssh certificate via bash process substitution. Once hosts have the certificates, everything is great. I set StrictHostKeyChecking yes in /etc/ssh/ssh_config, add the CA to /etc/ssh/ssh_known_hosts, and unless something gets overridden, users shouldn't ever see any host key checking dialogs. If they try to connect to something without a signed host cert, it will fail. This is how SSH should be!!

But there's a chicken-and-egg problem: Ansible also uses SSH to connect to and provision hosts. When it tries to connect to a brand new host to provision it, it crashes because the host will have an unsigned cert. The easy option is to disable StrictHostKeyChecking for new hosts, but that defeats the purpose. It's also dangerous to automate.

What are some good ways to deal with this?

Notably, your blog post on the subject uses a hardcoded provisioner password in a bootstrap script which isn't production secure.

Cloud-init is one obvious option as you can supply the host cert/key when the host is created, but in some environments (e.g. proxmox), it's not trivially automatable. Cloud-init also usually won't work for bare metal hosts where images just won't have those packages installed.

Storing a provisioner password in the image would be insecure, but an ephemeral token could work. The tokens are principal-specific, though, so they would have to be generated on the fly. This means images would have to be generated on the fly. Do-able, but also not trivial.

Any other ideas? I'd be grateful for any suggestions. Thanks!