Skip to content

ci-fix.md

Latest commit

 

History

History
404 lines (291 loc) · 20.5 KB

ci-fix.md

File metadata and controls

404 lines (291 loc) · 20.5 KB
Raw

CI Fix Guide

Short link: aka.ms/ci-fix

This guide provides detailed troubleshooting instructions for the automated validation tooling checks running on Azure REST API specs PRs submitted to this repo.

If you need help with your specs PR, please first thoroughly read the aka.ms/azsdk/pr-getting-help document.

Table of Contents

Prerequisites

Most guides here require for you to have npm installed, which you can get by installing Node.js.

Checks troubleshooting guides

Format

This check covers repository tooling in .github and eng/tools. To reproduce and fix a formatting failure, run from the repository root:

pnpm install
pnpm format:check
pnpm format

Tooling formatting uses Oxfmt and excludes Swagger JSON.

CredScan

This check is owned by One Engineering System. See 1ES CredScan for help.

PoliCheck

This check is owned by One Engineering System. See 1ES PoliCheck for help.

Protected Files

Only allowlisted repository maintainers and trusted automation accounts may change protected tooling and configuration files. If you are not allowlisted, remove the changes listed in the check's errors from your PR, including any deletions or renames. If a tooling change is needed, open an issue for the repository maintainers.

This restriction also applies to maintenance-only PRs that do not change specification/. Approvals, labels or reruns by maintainers do not authorize a non-allowlisted PR author. The allowlist is maintained in ALLOWED_AUTHORS. Dependabot (dependabot[bot]) is a trusted automation author and may update protected files. Normal CODEOWNERS reviews and other merge requirements still apply, including to changes to the allowlist.

Files under eng/common/ and .github/skills/azsdk-common-* are synchronized from Azure/azure-sdk-tools. Make changes in that repository rather than editing the synchronized copies here.

.github/CODEOWNERS and non-synchronized .github/skills/ files are exempt from this check. Their normal code-owner review requirements still apply.

SDK Validation * checks, like SDK Validation - Go

Important

  • The SDK Validation Status check is a meta check that aggregates the results of all SDK Validation - {Language} checks and reports a unified status. Re-run any individual SDK Validation - {Language} checks will automatically trigger a re-run of this meta check.
  • The SDK Validation * checks are owned by the Shanghai division of the Azure SDK team, not the core Redmond Azure SDK team.
  • For more information, refer to SDK Validation FAQ.

If you have an issue or with any of checks listed in the first column of the table below:

Check name Owner GitHub login
SDK Validation - Go Chenjie Shi tadelesh
SDK Validation - Java Weidong Xu weidongxu-microsoft
SDK Validation - JS Qiaoqiao Zhang qiaozha
SDK Validation - .NET Wei Hu live1206
SDK Validation - Python Yuchao Yan msyyc

Do the following:

  1. Attempt to diagnose the issue yourself:
    1. Look at the affected PR's checks tab for the failing check.
    2. Click on the View more details on Azure Pipelines. link from that tab and inspect the devOps logs. For example, for SDK Validation - Go check look into the Azure Pipelines/SDK Validation - Go pipeline run logs.
  2. If your investigation denotes this is likely a bug in the check itself and not your PR, reach out to the owner of the check per the aforementioned table.

SDK Breaking Change Review

Important

  • If your PR is flagged with any label that matches the pattern BreakingChange-{Language}-Sdk, the SDK breaking changes will be reviewed by SDK reviewers around two business days after the completion of the first two review steps in PR review workflow, i.e. REST API breaking change review and ARM review.
  • If you need to suppress the SDK breaking changes, refer to SDK Suppressions.

If the SDK breaking changes haven't been reviewed after two additional business days, you may reach out to the reviewers:

Language Reviewer GitHub login
Go Chenjie Shi tadelesh
JS Qiaoqiao Zhang qiaozha
Python Yuchao Yan msyyc

Swagger APIView

Swagger and SDK language APIViews are generated as part of the Azure REST API specs PR build. When everything is successful you should see a comment box similar to the picture below showing the generated APIViews.

alt text

If an expected APIView was not generated, follow the step below to troubleshoot.

  • On the CI check click on details > View Azure DevOps build log for more details to view the devOps logs.
  • Investigate the CI job for the language with error. Swagger APIViews are generated in the SwaggerAPIView job, while APIViews for SDK languages are generated in their respective language jobs in the SDK Automation stage.
  • Ensure that all previous checks in the job are green before proceeding.

Diagnosing APIView failure for SDK Language (not Swagger)

  1. Check for an unexpected skip of the Publish SDK APIView Artifact to Pipeline Artifacts and Generate SDK APIView step.
  2. Look in SDK Automation step to verify that the API token generation completed successfully.
  3. Search logs for Read Temp File
  4. Below Read Temp File find the .json object and search within to locate the apiViewArtifact property.
  5. If not present, the APIView parser for the language failed to generate the APIView Token Artifacts.
  6. Please contact APIView Support Teams Channel for assistance.

Swagger ApiDocPreview

If you see Swagger ApiDocPreview check fail with a failure like this one:

Rule Message
❌ RestBuild error "logUrl":"https://apidrop.visualstudio.com/Content%20CI/_build/results?buildId=373646&view=logs&j=fd490c07-0b22-5182-fac9-6d67fe1e939b",
"detail":"Run.ps1 failed with exit code 1 "

Refer to troubleshooting REST API documentation.

Swagger Avocado

Moved to https://github.com/Azure/azure-rest-api-specs/wiki/Swagger-Avocado

Swagger BreakingChange and BreakingChange(Cross-Version)

See aka.ms/azsdk/pr-brch-deep. If you want a quick read, see only the summary section.

Run oad locally

To repro issues with "breaking changes" checks, you can locally run the tool that powers them: Azure/openapi-diff, aka oad:

npm install -g @azure/oad
oad compare <old-spec-path> <new-spec-path>

Please see readme for how to install or run tool in details. Refer to Oad Docs for detailed description of all oad rules.

Swagger LintDiff and Swagger Lint(RPaaS)

The LintDiff validation tool runs linting rules against specification difference. Two specifications are compared: the specification as it would be when proposed PR is merged, vs the specification as seen before the PR is merged.

Refer to openapi-authoring-automated-guidelines for detailed description of all lint rules and how-to-fix guidance. If that guidance is not enough, please also refer to the LintDiff rules.md doc. It links to .md files related to given error, containing instructions how to fix them.

To reproduce LintDiff failures locally, see CONTRIBUTING.md / How to locally reproduce a LintDiff failure occurring on a PR.

Swagger LintDiff for TypeSpec: troubleshooting guides

https://github.com/Azure/azure-rest-api-specs/wiki/Swagger-LintDiff-for-TypeSpec

Swagger ModelValidation

To repro issues with Swagger ModelValidation locally:

npm install -g oav
oav validate-example <openapi-spec-path>

Please see readme for how to install or run tool in details. Refer to Semantic and Model Violations Reference for detailed description of validations and how-to-fix guidance. Refer to Swagger-Example-Generation for example automatic generation.

Swagger PrettierCheck

First, ensure you have fulfilled Prerequisites as explained above.

To update all the spec files for a given service run the following:

# To fix all the files in the repo run from the root of the repo
cd <local_repo_clone_root>

# OPTIONAL STEP: To fix a particular service OpenAPI spec cd to that directory like
cd specification/contosowidgetmanager

# Install the dependencies to the local 'node_modules' folder.
pnpm install

# Run 'prettier --check' to verify the problems can be reproduced locally
pnpm prettier --check **/*.json

# Run 'prettier --write' to fix the problems.
pnpm prettier --write **/*.json

Then please commit and push changes made by prettier.

Prettier reference

Swagger SemanticValidation

To repro issues with Swagger SemanticValidation locally:

npm install -g oav
oav validate-spec <openapi-spec-path>

Please see readme for how to install or run tool in details. Refer to Semantic and Model Violations Reference for detailed description of validations and how-to-fix guidance.

Spell Check

If you receive a spelling failure, first try to fix the spelling in source.

If there are new words that need to be supported for your service, add the word to the override list in the words list in the specification/<service>/cspell.yaml file for your service. Specific files can also be overridden using the override field in your service's cspell.yaml file (see example below).

If the specification/<service>/cspell.yaml file does not exist, create it using the example below and set the words and overrides fields to the words that need to be suppressed. The import field is required.

For example (note the words list is sorted alphabetically):

import:
  - ../../cspell.yaml
words:
  - aardvark
  - zoology

# Optional overrides example for words in a specific file
overrides: 
  - filename: '**/specification/contosowidgetmanager/somefile.yaml'
    words:
      - aardwolf
      - zoo

Spell checking is case-insensitive so you only need to add a word once in lower-case. Try to kept the word list sorted for easier discovery.

For more information see cspell configuration.

TypeSpec Validation

For failed batch validation, run the pnpm install and per-project pnpm tsv commands from the final failure summary at the repository root. In GitHub Actions, each failed project's error annotation also includes these reproduction instructions.

https://github.com/Azure/azure-rest-api-specs/wiki/TypeSpec-Validation

TypeSpec Suppressions

This check detects new or changed TypeSpec lint suppressions introduced by your PR and requires them to be removed or approved before the PR can merge. It is distinct from the AutoRest/suppressions.yaml flow described in Suppression Process below. It analyzes two kinds of suppressions in the TypeSpec projects impacted by your changes:

  • Inline suppressions: #suppress directives in .tsp files.
  • Config suppressions: linter.disable entries in tspconfig.yaml.

Only suppressions for rules listed in check-rules.json count toward review/gating; suppressions for other rules are still detected and reported for visibility but do not block the check.

The check runs as the following workflows (mirroring the three-run structure used by other validations in this repo):

Check name Purpose
TypeSpec Suppressions - Analyze Code Computes the impacted TypeSpec folders, runs the analyzer, writes the markdown summary + a JSON report artifact (typespec-suppressions-report), and emits inline ::warning annotations anchored to each suppression on the PR diff.
TypeSpec Suppressions - Set Status Reports the check status; approval is granted by applying the typespec-suppressions-approved label.
TypeSpec Suppressions - Test Runs the analyzer tool's own test suite.

Where to see the results

  • A "TypeSpec suppressions requiring review" comment is surfaced on the PR, listing the new/changed suppressions with their rule, source location, and justification.
  • Inline ::warning annotations appear on the PR diff at each suppression's source location.
  • The full structured report is uploaded as the typespec-suppressions-report build artifact.

What to do

Suppressions should be a last resort. Before approving or justifying a suppression, prefer fixing the underlying issue:

  1. Address the underlying lint issue first. For each surfaced suppression, determine whether the flagged rule can instead be resolved in the spec. If so, remove the #suppress directive (or the linter.disable entry in tspconfig.yaml) and fix the underlying problem rather than suppressing it. For more information on TypeSpec linting rules, see: https://aka.ms/tsp-suppress/tsp-to-lintdiff
  2. Only if the suppression is genuinely necessary, ensure it has a clear, meaningful justification. Every suppression must include a justification string explaining why the rule cannot be satisfied; suppressions with no justification are flagged.
  3. If the suppression is legitimate and requires approval, ask the appropriate reviewer (ARM spec PRs: only the ARM reviewer can apply the label) to apply the typespec-suppressions-approved label. The TypeSpec Suppressions check remains blocking until the suppression is resolved or the label is applied.

Reproduce locally

The check is powered by the @azure-tools/typespec-suppressions CLI. See eng/tools/typespec-suppressions/README.md for usage. For example, to compare your branch against main:

npx typespec-suppressions --base origin/main --check-rules-file eng/tools/typespec-suppressions/check-rules.json --fail-on-approval <path-to-typespec-project-folder>

license/cla

This check is owned by One Engineering System. See 1ES GitHub inside Microsoft for help.

Suppression Process

In case there are validation errors reported against your service that you believe do not apply, we have a suppression process you can follow to permanently remove these reported errors for your specs. Refer to the suppression guide for detailed guidance.

This process (backed by suppressions.yaml files and the Approved-Suppression label) applies to most validation checks. It is separate from TypeSpec lint suppressions (#suppress directives and tspconfig.yaml linter.disable entries), which are surfaced by the TypeSpec Suppressions check and approved via the typespec-suppressions-approved label.

Checks not covered by this guide

If you have an issue with a check that is not covered by this guide and the help at aka.ms/azsdk/pr-getting-help, see aka.ms/azsdk/support.

Obsolete checks

Following checks have been removed from the validation toolchain as of August 2023.

  • API Readiness Check
  • Service API Readiness Test
  • Traffic validation