SIA: reconcile sshd host certificate lines on key type change

[psasidhar]

Problem

When the configured ssh host key type changes (e.g. rsa → ecdsa), updateSSH appends the new HostCertificate line to the sshd config and restarts sshd — but the previous key type's line is left in place. sshd keeps presenting a certificate that SIA is no longer refreshing, and once that stale certificate expires, sshd fails to start on the next restart/reboot, locking operators out of the instance.

Separately, the restart itself had robustness gaps: the config was never validated before restarting (a bad config bricks sshd), systemctl restart stops the listener when a reload suffices, and the hardcoded sshd unit name does not exist on debian/ubuntu systems where the unit is ssh.

Change

Reconcile in one pass. When updateSSH adds its HostCertificate line (i.e. the key type changed or this is initial setup), it now also comments out the host certificate lines of the other SIA-supported key types in the same write. The stale path set is derived from the hostkey package (hostkey.CertFile for Rsa/Ecdsa/Ed25519 in the same directory), so:

• only files SIA itself manages are ever touched — operator-managed certificates at custom paths are provably out of scope
• the logic is symmetric (rsa→ecdsa, ecdsa→rsa, →ed25519) and automatically covers any key type added to the enum later

The new line is inserted before the first Match block if one exists, since HostCertificate is only valid in the global section (previously a trailing Match block would produce an invalid config).

Validate, then reload. The updated config is checked with sshd -t -f <config> before the daemon is asked to re-read it. If validation fails, the original config is restored and the running daemon is left untouched. On success, systemctl reload-or-restart is used — sshd re-execs in place without stopping the listener or affecting established sessions, falling back to a full start if the daemon is not running — trying the sshd unit then ssh (debian/ubuntu; the sshd alias link only exists while the unit is enabled). Command output is captured into returned errors so failures are actionable from the SIA log.

Parser fix. hostCertificateLinePresent previously used a prefix match, so HostCertificate /path/cert.pub.old counted as a match for /path/cert.pub, and lowercase keywords (valid for sshd) were missed. It now parses by field with a case-insensitive keyword match, as sshd does.

Steady state is unchanged: when the certificate line is already present, refreshes do not touch the config file and do not reload sshd. updateSSH's signature and both call sites are unchanged, and ssh update errors remain non-fatal to the agent.

Testing

• TestUpdateSSHConfigFile: 7 cases — the 2 original cases (behavior preserved byte-for-byte), rsa→ecdsa, ecdsa→rsa, multiple stale types, custom-path certificate untouched, insertion before a Match block
• TestHostCertificateLinePresent: extended with case-insensitive keyword, tab separator, and partial-path-match cases (the last fails against the previous prefix-match implementation)
• TestHostCertificateFile, TestSiblingHostCertFiles: new helper coverage
• TestCheckSshdConfig, TestReloadSshdService: exercised with fake shell-script executables recording their invocations — covers the reload happy path, the debian/ubuntu ssh-unit fallback, and error aggregation when both units fail
• go build, go vet, and the full libs/go/sia/agent suite pass

The validate-then-reload behavior was also verified end-to-end on a live AWS EC2 instance (AlmaLinux 8.10) via a downstream build: migrating rsa→ecdsa commented out the stale line, sshd -t gated the reload, and journald showed a true SIGHUP reload with established sessions surviving.

Notes

provider/azure/sia-vm has its own private copy of restartSshdService with the same unvalidated-restart pattern; updating it the same way is a candidate follow-up, left out to keep this change scoped to libs/go/sia/agent.

🤖 Generated with Claude Code