Skip to content

ci(openapi-diff): Compare API schemas with the PR's merge-base - #130541

Open
sentry-junior[bot] wants to merge 2 commits into
masterfrom
ci/openapi-diff-merge-base
Open

sentry-junior[bot] wants to merge 2 commits into
masterfrom
ci/openapi-diff-merge-base

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

openapi-diff compared every backend PR with getsentry/sentry-api-schema main, which is the latest master and not where the PR branched off, so its diff also showed unrelated master changes. Its json-diff step exits 1 on any difference, so the job failed on every PR that changes the spec. That's why develop-docs calls the failure expected, and also why the RPC breaking-change step after it was skipped on exactly those PRs.

In the 100 runs I sampled, 69 hit the api_docs filter. 27 failed at json-diff, and the RPC check was skipped in all 27. The other 42 ran the RPC check and passed. #130472 moves @sentry/api into static/packages/api, which gives master a second baseline source besides sentry-api-schema: the api@ release assets.

What changes

  • Baselines come from the PR's merge-base. openapi.yml now uploads tests/apidocs/openapi-derefed.json and rpc_method_schema.json from each master build as an api-schemas artifact, kept for 30 days. scripts/openapi-diff.sh baseline gets the merge-base from the compare API and walks back up to 30 master commits to the closest one with that artifact. Every commit that can change the schemas uploads one. Fallbacks, in order:

    1. the newest api@ GitHub release, picked by listing releases, since releases/latest is sentry's calver release;
    2. sentry-api-schema main, for as long as it still receives the spec.

    The job summary says which baseline it used. Each source writes to a scratch directory, so a source that fails halfway leaves nothing behind.

  • The public spec diff is a changelog. The changelog job writes an oasdiff changelog --format markdown to the job summary, capped below GitHub's 1 MiB limit, and never fails. A missing baseline or an oasdiff error becomes a note in the summary.

  • The RPC check runs in its own job. The rpc-breaking job runs oasdiff breaking --fail-on WARN on the RPC schema, so nothing upstream can skip it. Control and cell silos deploy separately, and the new required api-client check (feat(api): Move the @sentry/api package into static/packages/api #130472) doesn't cover RPC.

  • One build. The build job still builds the spec and RPC schema once, with setup-sentry. The two compare jobs only download its artifact and run the oasdiff image, so CI cost stays about the same.

  • make diff-api-docs runs the same script: the merge-base baseline, then the changelog. It needs gh and docker. That replaces api-docs/openapi-diff.ts and its json-diff dependency, so the lockfile only loses the json-diff subtree.

  • A new make build-rpc-schema writes .artifacts/rpc_method_schema.json. It's the same target feat(api): Move the @sentry/api package into static/packages/api #130472 adds.

  • A shellcheck pre-commit hook (--shell=sh --enable=check-extra-masked-returns) covers the new script.

The develop-docs text that says this check is expected to fail is updated in getsentry/sentry-docs#20679, which already rewrites that section.

Interaction with #130472

This PR is off master and doesn't depend on #130472, but the two touch neighboring lines in openapi.yml and Makefile.

Verification

  • shellcheck (--shell=sh --enable=check-extra-masked-returns) is clean. actionlint reports nothing new; its only finding is the existing ubuntu-26.04-arm label. knip is clean.
  • Baseline lookup, run for real under dash with feat(api): Move the @sentry/api package into static/packages/api #130472's base and head SHAs: no api-schemas artifact exists yet and there's no api@ release, so it fell back to sentry-api-schema main (15.8 MB spec plus RPC schema) in 18 s.
  • Baseline lookup, run with a stubbed gh:
    • the artifact is on the second commit back: it's used;
    • the artifact is missing a file and an api@ release exists: it uses the release;
    • every source fails: it exits 1 and leaves no partial files behind;
    • gh fails: it exits 1.
  • Changelog and RPC check, run with the real oasdiff v1.33.0 binary:
    • a real spec diff gives an 8.8 KB Markdown changelog;
    • identical specs give "No changes detected";
    • a missing baseline or a bad JSON spec gives a note, and every one of these exits 0;
    • rpc-breaking on identical schemas exits 0, and on a schema with a removed method it prints ::error and exits 1.
  • No UI change.

Co-Authored-By: BYK 126780+BYK@users.noreply.github.com

via @BYK.

--

View Session in Sentry

The openapi-diff job compared each PR with getsentry/sentry-api-schema
main, so it showed unrelated master changes. json-diff exited 1 on any
spec change, so the job failed on every API PR and skipped the RPC
breaking-change check exactly then.

- The openapi workflow uploads the master spec and RPC schema as an
  api-schemas artifact. PRs take their baseline from the artifact for
  their merge-base, falling back to the newest api@ release and then to
  sentry-api-schema main.
- The public spec diff is an oasdiff changelog in the job summary and
  never fails.
- The RPC oasdiff breaking check runs in its own job, so it always runs.
- make diff-api-docs uses the same script, replacing the json-diff
  script and its dependency.

Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>
@sentry-junior
sentry-junior Bot requested review from a team as code owners November 9, 2026 22:53
@github-actions github-actions Bot added Scope: Frontend Automatically applied to PRs that change frontend components Scope: Backend Automatically applied to PRs that change backend components labels Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant