feat(cli): Add GitHub Actions annotations for CLI errors and warnings#15683
Open
Swimburger wants to merge 3 commits intomainfrom
Open
feat(cli): Add GitHub Actions annotations for CLI errors and warnings#15683Swimburger wants to merge 3 commits intomainfrom
Swimburger wants to merge 3 commits intomainfrom
Conversation
- Introduced `githubAnnotations.ts` to handle rendering of GitHub Actions annotations. - Implemented structured error and warning annotations for CLI commands, particularly for `fern automations generate`. - Enhanced `GeneratorRunResult` to include workspace-relative paths and line numbers for better annotation accuracy. - Added tests for rendering annotations and handling workspace-relative paths. - Suppressed generic logger annotations during structured annotation emission to avoid duplication. - Emitted a trailing warning when the number of failures exceeds GitHub's annotation cap.
Contributor
🌱 Seed Test SelectorSelect languages to run seed tests for:
How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR. |
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
fern-support
approved these changes
May 4, 2026
…tions annotation tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Surface CLI errors and warnings as GitHub Actions annotations when running under GHA, so failures appear inline in the PR/run UI instead of being buried in the raw step log.
The screenshot motivating this:
fern automations generateproduced an "SDK generation failed (2/8 succeeded, 4 skipped)" summary table, but the GHA Annotations panel showed only the genericProcess completed with exit code 1— the user had to scroll the raw log to find which generator failed and why.How it works
Two layers, both gated on
GITHUB_ACTIONS=true:Generic logger hook (in
@fern-api/cli-logger) — everylogger.error/logger.warnacross the CLI also emits a::error::/::warning::workflow command on stdout. Coversfern check,fern generate, validation errors, etc. Status-only logs (omitOnTTY: true, e.g. the per-task "Failed." line) are filtered out so they don't burn through GitHub's per-step cap.Structured per-generator annotations for
fern automations generate— emitted from theGeneratorRunCollectorafter the run, anchored on the exact line ingenerators.yml(file=fern/apis/foo/generators.yml,line=42) so the annotation appears inline on the file in the PR's "Files changed" tab. Title format is<generator> failed (group=<group>, api=<api>)— explicitkey=valuequalifiers because generator names likefernapi/fern-python-sdkalready contain/.The structured emitter wraps its run in
withSuppressedLoggerAnnotations(...)(a scoped runner with try/finally restoration) so per-generator failures don't get annotated twice — once with raw text fromlogger.error, once with the richfile=/line=metadata.When the failure count exceeds GitHub's per-step cap (10), the structured emitter still emits one
::error::per failure (GHA drops the overflow itself) and appends a trailing::warning::like"3 additional generator failures were hidden by GitHub's per-step annotation cap. See the step summary table for the full list."so reviewers know what's missing.Related fix bundled in: auto-retry on 429 in automations mode
While verifying this PR with a real GHA run, every generator call hit
429 Too Many Requestsfrom Fiddle. The CLI already has a--retry-rate-limitedflag (bounded: 3 attempts, 2s→120s exponential backoff with jitter), butfern automations generatewas passingfalse. Automation runs are unattended — failing the run and asking a human to re-trigger with a flag isn't useful — so this PR flips that default totruefor automations only.fern generate(interactive) still requires the explicit flag. The retry warnings emitted during backoff are absorbed bywithSuppressedLoggerAnnotations, so the panel stays clean and only the final post-retry failure (if any) becomes a::error::annotation.Sanitization (per the GHA workflow command spec)
[31m...[39mnoise in the panel.%0Aso multi-line errors stay one workflow command (GHA renders them as a multi-line annotation).,:\r\n%per spec so they can't break out of the property list.Out of scope
fern checkvalidation errors annotate via the generic logger hook with nofile=/line=metadata. Anchoring those properly would need structured emission from a check-command emitter, not fromlogger.error— separate design pass.createJobfan-out, or revisiting Fiddle's rate limit) are follow-ups.Test plan
pnpm turbo run test --filter @fern-api/cli-logger --filter @fern-api/cli— 33 + 637 tests passpnpm check:fixcleanGITHUB_ACTIONS=trueand verified output matches expectations across env-var gate, suppression scope (incl. nested), level filter, omitOnTTY filter, prefix→title, and the cap-overflow warning5.9.0-rc.0, ranautomations generateagainst a multi-workspace fern config, and confirmed: rich annotations rendered inline ongenerators.ymllines,%2C/%3Adecoded correctly in titles, and no duplicate generic+structured annotations. Initial run also exposed the 429 issue, which motivated the retry default flip above.