[Travis Prescott (tjprescott)]

This PR sets up necessary preconditions for release-gating through API Review Hub.

1. Adds steps to generate API hashes (as long as APIView token files were genereated) during extended_build and store the results in package info files.
2. Refactors the script that processed the raw api.md file from Powershell to Python.
3. Ensures that the create-apireview-hub-artifacts-python.yml and the build extended process share logic to generate API hashes

Release scripts link will rely on the presence of ApiHash in package info files for approval queries and marking packages released.

[azure-pipelines]

Azure Pipelines:
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds API hash generation to extended builds for API Review Hub release gating.

Changes:

• Replaces PowerShell metadata extraction with Python.
• Stores generated API hashes in PackageInfo artifacts.
• Shares metadata tooling with API Review Hub pipelines.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
eng/tools/azure-sdk-tools/tests/test_apistub.py	Updates extractor mocks.
eng/tools/azure-sdk-tools/azpysdk/apistub.py	Invokes the Python extractor.
eng/scripts/save_package_api_hash.py	Adds hashes to PackageInfo files.
eng/scripts/Extract-APIViewMetadata-Python.ps1	Removes the PowerShell extractor.
eng/scripts/extract_apiview_metadata.py	Implements Python metadata extraction.
eng/pipelines/templates/steps/create-apireview-hub-artifacts-python.yml	Installs the Python extractor.
eng/pipelines/templates/steps/build-extended-artifacts.yml	Adds extended-build hash generation.
eng/pipelines/templates/jobs/apireview-hub-job-python.yml	Copies the Python tooling into jobs.

Suppressed comments (1)

eng/scripts/extract_apiview_metadata.py:38

• The updated test_apistub.py mocks this script and only creates an empty metadata file, so none of the new hash/rewriting implementation is exercised. Add direct parity tests for metadata-header removal, fence-leading blanks, trailing-whitespace normalization, and LF/CRLF preservation; these values now feed the release gate and small normalization changes alter package approval identity.

    normalized_text = "\n".join(line.rstrip() for line in filtered)
    metadata["apiMdSha256"] = hashlib.sha256(normalized_text.encode("utf-8")).hexdigest()

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/scripts/extract_apiview_metadata.py:17

• The new hash-producing implementation is not exercised by the updated tests: test_apistub.py mocks this process and only creates an empty metadata file. A regression in header removal, LF/CRLF handling, trailing-whitespace normalization, or repeat execution would therefore silently change the ApiHash used by release gating. Add direct unit tests for extract_metadata that verify the rewritten markdown, metadata fields, and stable SHA-256 across those cases.

def extract_metadata(api_markdown_path: pathlib.Path) -> Dict[str, str]:
    with api_markdown_path.open(encoding="utf-8-sig", newline="") as api_markdown_file:
        file_text = api_markdown_file.read()
    line_ending = "\r\n" if "\r\n" in file_text else "\n"
    lines = re.split(r"\r?\n", file_text)

eng/scripts/save_package_api_hash.py:38

• This release-gating path has no test that verifies each PackageInfo file receives the hash from its matching package artifact. Please add a multi-package unit test that mocks markdown export, supplies distinct metadata hashes, checks the corresponding ApiHash values, and covers the intentional missing-token skip; otherwise package/hash correlation regressions can pass CI unnoticed.

    for package_info_path in package_info_dir.glob("*.json"):
        package_info = json.loads(package_info_path.read_text(encoding="utf-8-sig"))
        package_name = package_info["Name"]
        package_artifact_dir = artifact_dir / package_name
        token_file = package_artifact_dir / f"{package_name}_python.json"

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/scripts/save_package_api_hash.py:38

• This release-gating path has no tests: the current changes do not verify matching PackageInfo/token paths, the intentional missing-token skip, propagation of export failures, or that ApiHash is copied unchanged while preserving the remaining JSON properties. Please add focused tests for these cases before relying on this script in extended builds.

    for package_info_path in package_info_dir.glob("*.json"):
        package_info = json.loads(package_info_path.read_text(encoding="utf-8-sig"))
        package_name = package_info["Name"]
        package_artifact_dir = artifact_dir / package_name
        token_file = package_artifact_dir / f"{package_name}_python.json"

eng/scripts/extract_apiview_metadata.py:13

• The hash/rewriting implementation is not exercised by the updated test_apistub tests—they mock this script and only create empty output files. Please add unit tests for metadata extraction, header removal, leading-blank trimming, LF/CRLF preservation, and the exact normalized SHA-256 value so this PowerShell-to-Python migration cannot silently change the release-gating hash.

def extract_metadata(api_markdown_path: pathlib.Path) -> Dict[str, str]:

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/scripts/save_package_api_hash.py:34

• This new release-artifact enrichment path has no test coverage. Please add tests that build multiple temporary PackageInfo/artifact directories, verify each package receives its own hash, verify management packages without token files are skipped, and verify export or malformed-metadata failures stop publication; otherwise package-to-hash correlation regressions can reach the release gate unnoticed.

    for package_info_path in package_info_dir.glob("*.json"):

eng/scripts/extract_apiview_metadata.py:13

• The new metadata parser and SHA-256 calculation are not exercised by the existing apistub tests; those tests mock the subprocess and merely create an empty metadata file. Since this hash is now used for release gating, please add unit tests covering header extraction, header removal, trailing-whitespace normalization, and LF/CRLF input so a parser or hash-compatibility regression cannot publish an incorrect ApiHash.

def extract_metadata(api_markdown_path: pathlib.Path) -> Dict[str, str]:

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (3)

eng/scripts/save_package_api_hash.py:43

• A missing token is treated as if the package were management-plane, so the build succeeds without ApiHash. This also happens when API-stub generation is skipped or omits a data-plane package, while the linked Mark-PackageReleased.ps1 rejects every package-info file that lacks ApiHash. Distinguish intentionally unsupported packages from unexpected missing tokens and fail the latter, or ensure the downstream release flow excludes them explicitly.

        if not token_file.is_file():
            print(f"API token file was not found for {package_name}; skipping ApiHash update")
            continue

eng/scripts/extract_apiview_metadata.py:13

• The actual metadata extraction and hash calculation are not exercised: the updated apistub tests mock this script and only create an empty metadata file. Because this SHA becomes a release-gating identifier, add direct tests with known LF/CRLF inputs that verify header removal, leading-blank normalization, emitted metadata, rewritten markdown, and the expected SHA-256 value.

def extract_metadata(api_markdown_path: pathlib.Path) -> Dict[str, str]:

eng/scripts/save_package_api_hash.py:38

• The new PackageInfo enrichment path has no tests. Add coverage for matching each package to its token/artifact directory, preserving existing JSON fields while adding ApiHash, and the intended missing-token behavior; otherwise a mapping or serialization regression can publish unusable release metadata without being caught.

    for package_info_path in package_info_dir.glob("*.json"):
        package_info = json.loads(package_info_path.read_text(encoding="utf-8-sig"))
        package_name = package_info["Name"]
        package_artifact_dir = artifact_dir / package_name
        token_file = package_artifact_dir / f"{package_name}_python.json"

[Travis Prescott (tjprescott)]

/azp run python - pullrequest

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (3)

eng/scripts/save_package_api_hash.py:43

• run_apistub.yml excludes management-plane packages with --filter-type="Omit_management", and this branch therefore leaves every management package's PackageInfo without ApiHash. The linked release design requires ApiHash when marking a published package released, so management-package releases cannot complete that workflow. Generate hashes for management artifacts as well, or revise the release contract before silently skipping them here.

        # API stub generation intentionally omits management-plane packages, so they have no token file.
        # Revisit this behavior if management-plane packages are included in API stub generation.
        if not token_file.is_file():
            print(f"API token file was not found for {package_name}; skipping ApiHash update")
            continue

eng/scripts/save_package_api_hash.py:38

• The new package-to-hash correlation path has no test coverage. Add tests with multiple PackageInfo files to verify that each receives only its matching token's hash, that management/missing-token handling is explicit, and that malformed metadata or package JSON fails without updating the wrong file; the existing apistub tests do not invoke this script.

    for package_info_path in package_info_dir.glob("*.json"):
        package_info = json.loads(package_info_path.read_text(encoding="utf-8-sig"))
        package_name = package_info["Name"]
        package_artifact_dir = artifact_dir / package_name
        token_file = package_artifact_dir / f"{package_name}_python.json"

eng/scripts/extract_apiview_metadata.py:44

• No test executes this new metadata extractor: the updated test_apistub.py mocks the subprocess and merely creates an empty metadata file. Because this code defines the release-gating hash and replaces the PowerShell implementation, add focused tests that assert header removal, leading/trailing whitespace normalization, CRLF handling, the exact SHA-256 value, and the emitted metadata content.

    normalized_text = "\n".join(line.rstrip() for line in filtered)
    metadata["apiMdSha256"] = hashlib.sha256(normalized_text.encode("utf-8")).hexdigest()

    api_markdown_path.write_text(line_ending.join(filtered), encoding="utf-8", newline="")
    metadata_path = api_markdown_path.parent / "api.metadata.yml"
    metadata_text = line_ending.join(f"{key}: {metadata[key]}" for key in sorted(metadata)) + line_ending
    metadata_path.write_text(metadata_text, encoding="utf-8", newline="")

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (2)

eng/scripts/save_package_api_hash.py:38

• This package-to-hash correlation is release-critical but has no automated coverage. Add a multi-package pytest case that creates PackageInfo/token artifacts, verifies each package receives its own hash, and covers the intentional missing-token skip; otherwise a path or naming regression can silently publish PackageInfo files without the ApiHash required by mark-released.

    for package_info_path in package_info_dir.glob("*.json"):
        package_info = json.loads(package_info_path.read_text(encoding="utf-8-sig"))
        package_name = package_info["Name"]
        package_artifact_dir = artifact_dir / package_name
        token_file = package_artifact_dir / f"{package_name}_python.json"

eng/scripts/extract_apiview_metadata.py:39

• The new release-gating hash implementation is not exercised by the updated tests: test_apistub.py replaces this script with a mock that only creates an empty metadata file. Please add direct pytest cases that verify metadata-header removal, leading/trailing whitespace normalization, LF/CRLF handling, and the exact SHA-256 output so a normalization regression cannot associate a release with the wrong API revision.

    normalized_text = "\n".join(line.rstrip() for line in filtered)
    metadata["apiMdSha256"] = hashlib.sha256(normalized_text.encode("utf-8")).hexdigest()