[Kafka] Fail partition preflight when a hot-path topic is missing

[sirdeggen]

What Changed

• kafka.CheckPartitions now returns an error when minPartitions > 1 and the broker reports ErrTopicNotFound, instead of logging a warning and continuing.
• The minPartitions <= 1 early return is unchanged, so standalone and single-replica setups (including the in-memory broker, which creates topics on subscribe and never returns ErrTopicNotFound) are not affected.
• Startup in app.Bootstrap still passes an empty topic list. Propagation continues to use the existing fail-closed CheckMinPartitions. Wiring real fan-out topics into this check remains a separate change, as noted in Bootstrap.
• Doc comment, Kafka.MinPartitions field comment, and docs/production-kafka.md now describe the hard-fail contract.
• Unit tests cover a missing topic with minPartitions > 1 and with minPartitions <= 1, plus the in-memory broker at a floor of 0 or 1.

Why It Was Necessary

Finding F-013 (issue #71) from the 2026-05 review: when the partition floor is greater than 1, a missing hot-path topic used to pass preflight. Kafka auto-creation then used the broker default partition count, which can be lower than the pod count, so a scaled deployment started cleanly and then ran with idle consumers.

CheckMinPartitions already treats a missing topic as a hard error. CheckPartitions described the warning as intentional. That split let the horizontal-scaling guard pass a topic that did not exist.

Choice: fail, rather than create the topic. Arcade does not provision Kafka topics. docs/production-kafka.md and config.example.yaml tell operators to pre-create them, because production clusters usually disable auto-creation and because partition count, replication, and retention are operator decisions. The Broker interface has no create-topic method, and the in-memory backend has nothing to create. Failing closed matches CheckMinPartitions and leaves topic creation with the operator. Creating topics inside the preflight would also not help the current startup call, which deliberately passes an empty list so a leftover kafka.min_partitions cannot crash-loop an upgrade.

Fixes #71

Testing Performed

• go test -count=1 ./kafka/ -run 'TestCheckPartitions_'
  • TestCheckPartitions_TopicMissing_MinPartitionsGreaterThanOneFails — missing topic with floor 3 returns an error naming the topic and the required minimum; an earlier topic that already meets the floor does not hide it.
  • TestCheckPartitions_TopicMissing_MinPartitionsAtMostOneNoop — floors 1, 0, and -1 return nil and do not query the broker.
  • TestCheckPartitions_MemoryBrokerMinPartitionsAtMostOneUnchanged — in-memory broker with a not-yet-subscribed topic and floor 0 or 1 still returns nil.
  • TestCheckPartitions_ExistingTopicStillEnforcesFloor — an existing topic that is too narrow still fails, and one that meets the floor still passes.
• go build ./... — passed.
• go vet ./... — passed.
• magex lint (golangci-lint 2.14.0, repo .golangci.json, 58 linters including gosec) — 0 issues. go vet inside that run also passed.
• go test -count=1 ./... — passed (default build tags; e2e/smoke/integration packages stay excluded, same as the unit-test CI path).

Impact / Risk

• Breaking change: callers that passed a missing topic with minPartitions > 1 and relied on the warning will now get an error. The only production call site passes a nil topic list and only runs when kafka.min_partitions > 1, so current startups do not change.
• Standalone / in-memory: unchanged. minPartitions <= 1 returns before any broker call, which is the documented standalone setting. The in-memory broker never returns ErrTopicNotFound.
• Single-pod: unchanged for the same reason (min_partitions 0 or 1).
• Performance: none. The check still runs once at startup.
• Migration: none until a future change passes real fan-out topics into CheckPartitions. When that happens, those topics must already exist with at least kafka.min_partitions partitions.

Notifications

• @deggen