[Cosmos] Skip resource provisioning for fixed-account live tests

[Tomas Varon (tvaron3)]

Description

Cosmos key-auth live-test jobs selected fixed accounts but still provisioned unused temporary accounts before restoring the fixed credentials. Separate these jobs from provisioned jobs so fixed-account testing no longer depends on resource deployment or its Azure authentication.

• Add opt-in FixedAccountMatrixConfigs and FixedAccountPreSteps to the shared Python test archetype, preserving existing defaults for other pipelines.
• Compile subscription setup, resource deployment, and cleanup out of fixed-account jobs. Run their tests with plain PowerShell, without a service-connection login or CI_HAS_DEPLOYED_RESOURCES requirement.
• Move the 12 fixed-account lanes into their own matrix and export normal ACCOUNT_* credentials directly with existing mask-before-environment secret registration. Remove the obsolete post-deployment rebinding hook.
• Preserve the 7 AAD lanes and 1 GSI lane, including provisioning, role assignments, federated test authentication, and cleanup. Preserve all 20 generated job names and settings; explicitly carry forward the two fixed circuit-breaker flags previously supplied by ARM outputs.
• Retain sdk/cosmos/pipeline/resolve-test-account-steps.yml and resolve-cosmos-test-account.ps1, which CI requires. Per the requested cleanup, remove the manual-only resolve-cosmos-test-account.tests.ps1 and test_fixed_account_pipeline.py scripts and their README commands. These scripts are not invoked by repository-defined CI or package-scoped test discovery; no equivalent toolkit replacement is claimed.
• Update the Cosmos live-test documentation. No changes to eng/common, Bicep, or the public SDK API.

Follow-up fixes in ad1824473421c90eb81d8edb02ddaf1c26bd9b87:

• Compare sync/async post-split change-feed IDs as multisets, without assuming cross-partition ordering. Add safe missing/unexpected/duplicate counts.
• Correct synthetic retry-region topology, isolate background refresh, and prevent network requests to synthetic hosts while preserving retry-count/routing assertions.
• Use a full-history Analyze checkout for APIView target/source comparison instead of the observed depth-1 merge-only checkout.
• Add regression coverage to existing CI-consumed package and engineering tests. The shared retry helper lives under package tests, not the removed manual pipeline folder.

Validation: python3 -m pytest --noconftest -q sdk/cosmos/azure-cosmos/tests/test_split_test_utils.py sdk/cosmos/azure-cosmos/tests/test_service_retry_policies.py::TestServiceRetryPolicyHelpers eng/tools/azure-sdk-tools/tests/test_native_build_pipeline.py::test_analysis_checkout_has_history_for_api_comparison passed 23 tests. Combined with a session-only memory-transport harness exercising the six original affected retry flows, 29 tests passed. Inherited credentials were removed, dummy values supplied, and socket/DNS access blocked. This is not live/emulator validation. Python 3.10 grammar and git diff --check passed; independent review found no significant issues.

The previous full live run 6969964 failed before these fixes. GSI subscription status may have changed externally and must be verified by the new run; unexplained emulator HTTP 500s were not patched speculatively. Local MCP validation was blocked by an unrelated sample mypy error and an unavailable inherited test endpoint; direct offline validation was explicitly authorized. No emulator was available locally. New full live validation against the latest commit is pending.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

Changelog: not applicable to these pipeline/test-only changes.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🔵 Needs a closer look

Shared pipeline authentication and resource-lifecycle changes require human review and validation of expanded Azure Pipelines jobs.

No open issues 🎉

What changed in this PR

Separates Cosmos fixed-account live tests from provisioned tests, removing unnecessary deployment and Azure login dependencies.

Changes:

• Moves 12 fixed-account lanes into a dedicated matrix while retaining 8 provisioned lanes.
• Adds opt-in shared-template controls to skip resource setup and cleanup.
• Exports selected credentials directly, removes obsolete hooks and manual tests, and updates documentation.

File	Description
sdk/​cosmos/​tests.yml	Connects the fixed-account matrix and resolver steps.
sdk/​cosmos/​test-resources-post.ps1	Removes obsolete credential rebinding.
sdk/​cosmos/​README.md	Documents the separate live-test paths.
sdk/​cosmos/​pipeline/​test-resources-post.tests.ps1	Removes tests for the deleted hook.
sdk/​cosmos/​pipeline/​resolve-test-account-steps.yml	Exports normal account variables directly.
sdk/​cosmos/​pipeline/​resolve-cosmos-test-account.tests.ps1	Removes manual resolver tests.
sdk/​cosmos/​pipeline/​resolve-cosmos-test-account.ps1	Updates parameter documentation only.
sdk/​cosmos/​live-platform-matrix.json	Retains AAD and GSI provisioning lanes.
sdk/​cosmos/​fixed-account-platform-matrix.json	Defines fixed-account lanes and circuit-breaker flags.
eng/​pipelines/​templates/​stages/​archetype-sdk-tests.yml	Adds optional fixed-account matrix generation.
eng/​pipelines/​templates/​jobs/​live.tests.yml	Conditionally omits resource setup and cleanup.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Shared provisioning and authentication changes need human review, with latest-commit live validation still pending.

No open issues 🎉

_{Review effort: Balanced}

[github-actions]

[Pilot] PR Pipeline Failure Analysis

What failed

Build 6973968 (python - cosmos - ci) failed across all 5 Python versions (3.10–3.14) × whl/sdist on windows2022___EmulatorTestsPython*Standard (10 artifact files), all with the same 6 failing tests:

• tests.test_service_retry_policies.TestServiceRetryPolicies.test_service_request_retry_policy
• tests.test_service_retry_policies.TestServiceRetryPolicies.test_service_response_retry_policy
• tests.test_timeout_and_failover_retry_policy.TestTimeoutRetryPolicy.test_cross_region_retry[408]
• tests.test_timeout_and_failover_retry_policy.TestTimeoutRetryPolicy.test_cross_region_retry[500]
• tests.test_timeout_and_failover_retry_policy.TestTimeoutRetryPolicy.test_cross_region_retry[502]
• tests.test_timeout_and_failover_retry_policy.TestTimeoutRetryPolicy.test_cross_region_retry[503]

Root cause (shared across all 6): Commit ad1824473421c90eb81d8edb02ddaf1c26bd9b87 added a new synthetic-transport test harness in sdk/cosmos/azure-cosmos/tests/_retry_test_utils.py (retry_test_client / send_to_test_host) that rewrites requests sent to synthetic hostnames like (westus.retrytest.invalid/redacted) back to the real emulator/account host`, so tests can exercise multi-region retry routing without a live network call. Two defects in that new harness break the tests using it:

1. send_to_test_host rewrites the URL to the real emulator host but never disables or configures TLS certificate verification for the rewritten request. Against the emulator's self-signed HTTPS certificate this now fails real TLS verification — explaining the SSL failures in test_service_response_retry_policy and all 4 test_cross_region_retry cases.
2. In test_service_request_retry_policy, the transport-level send patch (send_to_test_host) causes requests to succeed through the real rewritten path instead of being intercepted by the test's MockExecuteServiceRequestExceptionIgnoreQuery patched onto _retry_utility.ExecuteFunction, so the mock's counter is never incremented, producing assert 0 == 3.

Both defects are localized to the new _retry_test_utils.py harness added by this PR, not to the production SDK retry logic.

Relevant pipeline output

tests/test_service_retry_policies.py:77: AssertionError
E  assert 0 == 3
E   +  where 0 = <...MockExecuteServiceRequestExceptionIgnoreQuery object>.counter

...azure/core/pipeline/transport/_requests_basic.py:428: ServiceRequestError
E  azure.core.exceptions.ServiceRequestError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1007)

(Identical pattern repeats across windows2022 Python 3.10/3.11/3.12/3.13/3.14, whl and sdist — 10/10 artifact files, single root cause, not platform-specific.)

Recommended next steps

• In send_to_test_host / retry_test_client, disable TLS verification (or supply the emulator's cert) for requests rewritten to the local emulator host, matching how other emulator-targeting test fixtures in this package configure connection_verify=False.
• Ensure the exception-raising mock (ExecuteFunction patch) still intercepts calls made through the rewritten/synthetic transport path — verify the ordering/interaction between patch.object(transport, "send", ...) (via send_to_test_host) and patch.object(_retry_utility, 'ExecuteFunction', mf).
• Re-run the emulator test matrix locally against the emulator to confirm both fixes before pushing.
• See https://aka.ms/ci-fix

Automated fix: Fix found, view and apply fix

Generated by Pipeline Analysis Next Steps · copilot · sonnet50 · 141.1 AIC · ⌖ 6.83 AIC · ⊞ 8.9K · ◷

[Copilot]

🔵 Needs a closer look

Shared pipeline authentication and resource-lifecycle changes warrant human review, and latest-commit live validation remains pending.

No open issues 🎉

_{Review effort: Balanced}