Simplify test pipeline flag plumbing with a generic properties bag#16056
Open
Simplify test pipeline flag plumbing with a generic properties bag#16056
Conversation
13 tasks
Replace individual boolean flag inputs (requiresNugets, requiresTestSdk, requiresCliArchive, enablePlaywrightInstall) with a single 'properties' bag that passes through the entire CI pipeline: - MSBuild targets: nest boolean flags inside a 'properties' JSON object - build-test-matrix.ps1: read/write properties sub-object, handle both new and legacy metadata formats - split-test-matrix-by-deps.ps1: read flags from properties sub-object - run-tests.yml: replace 4 boolean inputs with single properties string, use fromJson(inputs.properties) for all flag checks - tests.yml: pass properties bag via toJson(matrix.properties) - specialized-test-runner.yml: inject enablePlaywrightInstall into properties bag after runsheet generation - TestDataBuilder.cs/CanonicalMatrixEntry: use Dictionary<string,bool> Properties instead of individual bool properties - Update docs and test assertions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: radical <1472+radical@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add support for arbitrary property pass-through in GitHub workflows
Replace per-flag boolean inputs with generic properties bag in test pipeline
Apr 11, 2026
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 16056Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 16056" |
Introduce eng/testing/CITestsProperties.props as the single source of truth for CI test property names, MSBuild property mappings, and default values. MSBuild targets now dynamically resolve properties via ItemGroup iteration using $(%(CITestsProperty.MSBuildProp)) instead of hardcoding each property. PowerShell scripts read the .props XML for defaults and use a generic Copy-CITestProperties helper instead of per-property copy blocks. Adding a new boolean test property now requires: 1. One line in CITestsProperties.props 2. Behavioral logic in the GitHub Actions workflow YAML Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Document the property registry in docs/ci/TestingOnCI.md:
- New 'CI Test Property Registry' section explaining CITestsProperties.props
- How-to for adding new boolean test properties (1 line + YAML)
- Updated file artifacts and scripts reference tables
- Cross-reference from metadata format to the registry
- Add 4 new tests in BuildTestMatrixTests.cs:
- AllCITestsPropertiesAppearInOutputWithDefaults: verifies all
registry properties flow through PS1 to canonical matrix output
- DefaultsAreAppliedWhenPropertiesAreMissingFromMetadata: verifies
partial input metadata gets defaults filled from the registry
- CITestsPropertiesPropsFileIsValidAndComplete: validates XML
structure, required attributes, and naming conventions
- SplitTestEntriesInheritAllCITestsProperties: verifies split/
partition entries also get all registry properties
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rBase.targets Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
radical
changed the title
Contributor
|
🎬 CLI E2E Test Recordings — 58 recordings uploaded (commit View recordings
📹 Recordings uploaded automatically from CI run #24276454765 |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors CI test-flag propagation to use a single properties dictionary (sourced from eng/testing/CITestsProperties.props) instead of multiple dedicated boolean fields, reducing cross-file plumbing when adding new flags.
Changes:
- Introduces
eng/testing/CITestsProperties.propsas a registry for CI test boolean flags (JSON key, MSBuild property name, default). - Updates MSBuild targets, PowerShell scripts, GitHub workflows, and Infrastructure.Tests to emit/consume a
propertiesobject end-to-end. - Expands Infra tests to validate property defaults and inheritance behavior in the generated canonical matrix.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Infrastructure.Tests/Shared/TestDataBuilder.cs | Updates test metadata and canonical matrix test helpers to emit/consume properties. |
| tests/Infrastructure.Tests/PowerShellScripts/SplitTestMatrixByDepsTests.cs | Adapts split-matrix tests to the nested properties format. |
| tests/Infrastructure.Tests/PowerShellScripts/ExpandTestMatrixGitHubTests.cs | Updates assertions to read flags via Properties[...]. |
| tests/Infrastructure.Tests/PowerShellScripts/BuildTestMatrixTests.cs | Adds coverage for default property population from CITestsProperties.props. |
| eng/testing/CITestsProperties.props | Adds centralized CI test property registry (keys, MSBuild prop mapping, defaults). |
| eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuilder.targets | Switches metadata emission to a properties JSON object driven by CITestsProperties.props. |
| eng/SpecializedTestRunsheetBuilderBase.targets | Switches specialized runsheet emission to include a properties JSON object. |
| eng/scripts/split-test-matrix-by-deps.ps1 | Updates dependency bucketing logic to read flags from entry.properties. |
| eng/scripts/build-test-matrix.ps1 | Updates canonical matrix generation to normalize and default-fill properties. |
| docs/ci/TestingOnCI.md | Documents the new properties object and the property registry file/flow. |
| .github/workflows/tests.yml | Passes properties through to run-tests.yml instead of individual booleans. |
| .github/workflows/specialized-test-runner.yml | Injects enablePlaywrightInstall into .properties and updates jq checks to .properties.*. |
| .github/workflows/run-tests.yml | Replaces boolean workflow inputs with a single JSON-string properties input and consumes via fromJson. |
eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuilder.targets
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adding a new boolean flag (e.g.
requiresFoo) to the test pipeline currently requires coordinated changes across ~7 files — MSBuild targets, PowerShell scripts, and workflow YAML — just to pass a single value through. This makes the pipeline brittle and discourages adding new per-project test flags.This replaces the four individual boolean inputs (
requiresNugets,requiresTestSdk,requiresCliArchive,enablePlaywrightInstall) with a singlepropertiesdictionary that flows generically through the entire pipeline. Adding a new flag now only requires: (1) define the MSBuild property, (2) add it to thepropertiesbag in targets, (3) check it where needed in workflows. No transport-layer changes.Validation
propertiesbag format (BuildTestMatrixTests,ExpandTestMatrixGitHubTests,SplitTestMatrixByDepsTests,TestDataBuilder)tests.yml→run-tests.ymlflow end-to-endChecklist