Repository navigation
test(proxy): fail the build when kamal-proxy grows a flag the gem cannot emit - #83
Merged
Merged
Conversation
…not emit kamal-proxy accepts 80 deploy flags and 31 run flags. The gem emits 34 and 3. Most of R3 and R5 — rate limiting, mTLS, the response cache, compression, header rules, redirects, scale-to-zero — plus every ACME/DNS-01 option ships in the image and cannot be turned on from deploy.yml. Nothing noticed it happening. 23 feat commits landed between v0.9.2.2 and v1.0.0.0 and the gem's config surface never moved, because no test relates the two. Fixing the backlog without fixing the detection just resets the clock. So: every flag kamal-proxy accepts must be emitted by the gem or waived with a reason. Waivers split into NEVER_EXPOSED (a decision) and R7_BACKLOG (a todo carrying its issue number) so the second cannot quietly become the first. Coverage is measured by generating the real commands from a maximal fixture, not by reading the keys of deploy_options — a config accessor that exists but is never wired does not pass. Deleting ssl_redirect from the fixture drops --tls-redirect into the uncovered list, which is the property that makes each R7 issue self-verifying: delete the waiver, and the test proves the wiring. The flag list is generated by bin/sync-proxy-flags from Cobra's own --help, never by parsing the Go source. That is not caution for its own sake: the counts in the R7 epic came from a regex that silently missed every flag registered via Int64Var, because the method name contains digits. Cobra found three the regex did not. Bumps MINIMUM_VERSION to v1.0.0.0 — the manifest has to be generated against a version, and the backlog is only accurate against this one. The docs example version moves with it, as its own guard test requires. Refs #82
54 tasks
setup.sh seeds the private registry with the proxy image so app_with_roles keeps exercising proxy.run.registry. The tag was hardcoded to v0.9.2.2 next to a comment saying it must match MINIMUM_VERSION — so bumping the constant to v1.0.0.0 left the two disagreeing, and every integration deploy died with "manifest for registry:4443/...:v1.0.0.0 not found: manifest unknown". The failure surfaces inside an unrelated deploy, several layers from the line that caused it, which is the expensive part. Read the constant from the source instead, and fail loudly at setup time if it cannot be read. Refs #82
`kamal proxy boot` failed intermittently on a random host with:
ERROR (RuntimeError): Exception while executing on host vm1:
Digest::Base cannot be directly inherited in Ruby
lib/kamal/configuration/proxy/run.rb:21:in `digest'
Kamal::Cli::Proxy#boot calls drift.expected_digest inside
`on(KAMAL.proxy_hosts)`, so Proxy::Run.digest first touches Digest::SHA256
from one SSHKit thread per host, simultaneously. Nothing in lib/ required
digest, so that first reference went through Digest.const_missing, which runs
`require "digest/sha2"` outside any mutex. Two threads entering it together
observe the extension's half-built class hierarchy and one raises.
Requiring "digest" would not have fixed it — that defines the module but still
leaves SHA256 to const_missing. It has to be the implementation file.
Pre-existing, latent since config_digest was introduced; surfaced here because
one of the two Ruby 3.3 CI legs happened to lose the race. Both legs run
identical code inside the deployer container, which is what identifies this as
non-deterministic rather than a gemfile difference.
The test runs in a subprocess on purpose: the suite already pulls in digest via
test/sshkit_patch_drift_test.rb, so an in-process check would pass whether or
not lib/kamal.rb requires it.
Refs #82
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
kamal-proxy v1.0.0.0 accepts 80 deploy flags and 31 run flags. The gem emits 34 and 3.
Most of R3 and R5 — rate limiting, IP allow lists, mTLS, the response cache, compression, header rules, redirects, canary splits, scale-to-zero — plus every ACME/DNS-01 option, ships in the image and cannot be turned on from
deploy.yml.Nothing noticed it happening. 23
featcommits landed betweenv0.9.2.2andv1.0.0.0and the gem's config surface never moved, because no test relates the two. Fixing the backlog without fixing the detection just resets the clock.This adds the detection.
Closes #82
How it works
bin/sync-proxy-flags--helptest/fixtures/kamal_proxy_flags.ymltest/fixtures/deploy_with_every_proxy_option.ymltest/proxy_flag_coverage_test.rbEvery flag must be emitted by the gem or waived with a reason. Waivers split in two, deliberately:
NEVER_EXPOSED— a decision (--forceis a CLI concern;--http-portis published by docker)R7_BACKLOG— a todo, keyed by its issue numberCollapsing those into one hash would let the backlog quietly become permanent, which is how this gap happened.
Two choices worth reviewing
Coverage is measured by generating the real commands, not by reading the keys of
deploy_options. Costlier, but a config accessor that exists and is never wired does not pass. This is what makes each R7 issue self-verifying: implement it, delete its waiver, and the test proves the flag is actually emitted rather than merely named.The flag list is generated from Cobra, never parsed out of the Go source. Not caution for its own sake — the flag counts I put in the R7 epic came from a regex over
internal/cmd/*.gothat silently missed every flag registered viaInt64Var/Uint16Var, because the method name contains digits. Cobra found three the regex did not (cache-max-variants,cache-lease-ttl,cache-lease-wait). Building a drift guard on the technique that caused the drift would have been a poor foundation.Corrected figures, epic #13's summary table is understated: 80/31 flags (not 79/29), 45 deploy + 25 run unexposed (not 44/26). No flag moves between R7 issues; I'll fix the table.
Test plan
brand-new-thinginto the manifest; fails naming it, with the expose-or-waive instructionscanonical-host; fails naming the now-dead waiverMINIMUM_VERSIONbumped without refreshing — set it tov1.0.1.1; fails telling you to runbin/sync-proxy-flagsssl_redirectfrom the fixture;--tls-redirectdrops into the uncovered listbundle exec rubocop --parallelclean (205 files)bin/testnot run: integration needsghcr.io/mhenrixon/kamal-proxy:v1.0.0.0, which was still building when this was pushed. Must be green before merge — see below.Deviations & judgment calls
Deviations
MINIMUM_VERSIONbumpv0.9.2.2→v1.0.0.0. Not asked for, but the manifest must be generated against some version and the backlog is only accurate against this one —cache-max-variants,cache-lease-ttlandcache-lease-waitexist only in v1.0.0.0. Generating against v0.9.2.2 would have encoded a stale backlog on day one. This is the merge risk: integration tests now pull the v1.0.0.0 image. If you'd rather decouple, say so and I'll regenerate against v0.9.2.2 and land the bump separately.docs/proxy.yml:341— not optional,test_docs_example_run_version_matches_the_pinned_minimum_versionenforces it.Judgment calls
kamal-proxy flags --jsonsubcommand, but that needs a proxy-repo change and another release before the gem can use it.bin/sync-proxy-flagsruns the proxy's own--helpinstead — same authoritativeness, no cross-repo dependency. Ifflags --jsonlands later, only the script changes; the manifest format and the test do not.host,tlsandtargetare emitted bydeploy_command_args. Waiving them would have been a lie that masked real coverage.deployandrun. The other subcommands (list,stop,pause,drain,rollout,cache,domains) carry no operator-tunable options, so they are not adeploy.ymlsurface. Documented in the script.--recheck-targets-on-restoreneeded no waiver — it is already emitted unconditionally.