Skip to content

fix: raise health poll-age threshold from 300s to 480s - #138

Merged
jghoman merged 2 commits into
mainfrom
codex/health-poll-timeout-480s
Sep 9, 2026
Merged

fix: raise health poll-age threshold from 300s to 480s#138
jghoman merged 2 commits into
mainfrom
codex/health-poll-timeout-480s

Conversation

@bill-ph

@bill-ph bill-ph commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Synchronous DuckLake writes stop Kafka polling, so a write still making progress after five minutes currently causes /healthz to fail and can trigger a restart before the batch completes. Raise the default poll-age threshold from 300 to 480 seconds to give slow writes more time to finish and reduce avoidable batch replay after restarts.

/readyz uses the same health state and also moves to 480 seconds. Kafka's configured max.poll.interval.ms remains 600,000. The application threshold is not the total time until a pod is killed: Kubernetes probe failures and termination grace still apply. This mitigation delays detection of a genuinely stuck consumer by three minutes and does not resolve DuckLake commit contention.

Validation: the existing suite passed (804 tests, 1 expected failure; integration/e2e tests excluded), with five tests rerun outside the sandbox for network/local socket access. Ruff formatting and lint checks passed.

@bill-ph
bill-ph requested a review from a team September 9, 2026 19:58

@benben benben left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review generated on behalf of @benben.

Approve. Raises the default liveness/readiness poll-age threshold from 300s to 480s and updates two comments. The change is coherent with the 60s consume cap and the existing probe/Kafka configuration; the only gaps are that the pinning test was deleted in the second commit and the docs still describe 300s.

  • minor tests/unit/test_server.py:8 Second commit deleted the only test asserting the new 480s default. Reverting max_poll_age_s to 300 now passes the full suite unchanged.
  • nit AGENT.md:466 AGENT.md still documents 300s / 5-minute liveness threshold. Lines 383, 466 and 467 reference max_poll_age_s=300.
  • nit tests/unit/test_main.py:1000 TestConsumeTimeout docstring still says liveness fires at 300s. Docstring not updated alongside the main.py comments.

Model: fable.

@jghoman
jghoman merged commit 1d855aa into main Sep 9, 2026
17 checks passed
@jghoman
jghoman deleted the codex/health-poll-timeout-480s branch September 9, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants