Add dotnet breaking changes skill

.github/CODEOWNERS

@@ -24,6 +24,8 @@
 
 /plugins/dotnet/skills/csharp-refactoring/ @dotnet/roslyn-ide @dotnet/skills-csharp-language-reviewers
 /tests/dotnet/csharp-refactoring/ @dotnet/roslyn-ide @dotnet/skills-csharp-language-reviewers
+/plugins/dotnet/skills/dotnet-breaking-changes/ @dotnet/roslyn-ide @dotnet/skills-csharp-language-reviewers
+/tests/dotnet/dotnet-breaking-changes/ @dotnet/roslyn-ide @dotnet/skills-csharp-language-reviewers
 /plugins/dotnet/lsp.json @dotnet/roslyn-ide @dotnet/skills-csharp-language-reviewers
 
 # dotnet-advanced (advanced .NET development skills)

.github/workflows/evaluation-run.yml

@@ -698,6 +698,46 @@ jobs:
           tar -xzf "$RUNNER_TEMP/trusted-validator-archive/skill-validator-dist.tar.gz" \
             -C "$RUNNER_TEMP/trusted-validator"
 
+      # Infrastructure fix, not a skill-content fix: `vally experiment run
+      # --workers 5` (and the native agent lane's parallel scenarios) launch
+      # several concurrent *first* `dotnet` invocations. Each one runs the CLI
+      # first-use configurer, whose NuGet MigrationRunner takes the named mutex
+      # `NuGet-Migrations`; concurrent creation races on /tmp/.dotnet/shm and
+      # crashes fixture setup (`mkdir(...) == -1; errno == EEXIST`), leaving an
+      # unmatched trial. Run first-use serially here for every SDK on PATH (no
+      # setup-dotnet runs in this job, so these are the runner-image SDKs that
+      # fixture global.json files roll forward onto). The sentinel is per SDK
+      # version: dotnet/sdk src/Cli/Microsoft.DotNet.Configurer/
+      # FirstTimeUseNoticeSentinel.cs (`{Product.Version}.dotnetFirstUseSentinel`).
+      # `--info`/`--version` skip first-use (FirstRunExperience.cs `Terminating`
+      # option early-return; 8.0 Program.cs InfoOption/VersionOption branches),
+      # so use a cheap real subcommand; DotnetFirstTimeUseConfigurer.Configure()
+      # then runs MigrationRunner.Run() and creates the sentinel.
+      - name: Warm up .NET SDK first-use state
+        if: steps.find-evals.outputs.has_evals == 'true'
+        env:
+          DOTNET_NOLOGO: "1"
+        run: |
+          set -euo pipefail
+          mapfile -t sdk_versions < <(dotnet --list-sdks | awk '{print $1}')
+          if [ ${#sdk_versions[@]} -eq 0 ]; then
+            echo "::error::dotnet --list-sdks reported no installed SDKs"
+            exit 1
+          fi
+          dotnet_user_dir="${DOTNET_CLI_HOME:-$HOME}/.dotnet"
+          for sdk_version in "${sdk_versions[@]}"; do
+            warmup_dir=$(mktemp -d "$RUNNER_TEMP/dotnet-first-use.XXXXXX")
+            printf '{ "sdk": { "version": "%s", "rollForward": "disable" } }\n' "$sdk_version" \
+              > "$warmup_dir/global.json"
+            (cd "$warmup_dir" && dotnet nuget locals global-packages --list > /dev/null)
+            rm -rf "$warmup_dir"
+            if [ ! -f "$dotnet_user_dir/$sdk_version.dotnetFirstUseSentinel" ]; then
+              echo "::error::First-use sentinel missing for .NET SDK $sdk_version after warm-up"
+              exit 1
+            fi
+            echo "Initialized .NET SDK $sdk_version first-use state"
+          done
+
       - name: Select available Copilot token from pool
         id: select-token
         if: steps.find-evals.outputs.has_evals == 'true'

.github/workflows/evaluation-workflow-tests.yml

@@ -29,6 +29,8 @@ on:
       - "eng/vally-adapter/**"
       - "eng/skill-validator/src/**"
       - "tests/**/eval.yaml"
+      - "tests/**/fixtures/grading/**"
+      - "tests/dotnet/dotnet-breaking-changes/**"
   push:
     branches: [main]
     paths:
@@ -58,6 +60,7 @@ on:
       - "eng/vally-adapter/**"
       - "eng/skill-validator/src/**"
       - "tests/**/eval.yaml"
+      - "tests/**/fixtures/grading/**"
   workflow_dispatch:
 
 permissions:
@@ -81,6 +84,9 @@ jobs:
       - name: Run adapter fault-injection and report tests
         run: node --test eng/vally-adapter/*.test.mjs eng/dashboard/*.test.js
 
+      - name: Run eval grader-script regression tests
+        run: node --test tests/*/*/fixtures/grading/*.test.mjs
+
   evaluation-tools:
     runs-on: ubuntu-latest
     timeout-minutes: 10

eng/eval-quality/README.md

@@ -405,11 +405,17 @@ symlink, both the link and its contained target must be tracked.
 ### 16. Golden patch does not apply to declared fixture inputs
 
 A patch can remain present and tracked after its fixture changes, but its
-preimage no longer exists. The gate materializes each stimulus's declared
-`environment.files` mappings in a scratch workspace and runs
-`git apply --check`. A stale patch therefore fails before Vally tries to use a
-broken oracle. Stimuli whose workspace is created only by commands are not
-checked because their preimage cannot be reconstructed statically.
+preimage no longer exists. The gate materializes the suite-level
+`environment.files` mappings, then each stimulus's own mappings, in a scratch
+workspace (the same merge Vally performs) and runs `git apply --check`. A stale
+patch therefore fails before Vally tries to use a broken oracle. A stimulus
+whose files all come from the suite-level environment is checked the same way,
+and the same rule decides whether a state-changing stimulus needs a golden
+patch. Stimuli whose
+workspace is created only by commands are not checked because their preimage
+cannot be reconstructed statically. A stimulus must not redeclare a suite-level
+destination: Vally rejects the duplicate entry and loads none of the eval's
+stimuli, so the gate fails it too.
 
 Materialization is fail-closed. Each mapping-style `environment.files` entry
 must declare a non-empty string `dest`; otherwise Vally rejects the specification

eng/eval-quality/check_eval_quality.py

@@ -779,7 +779,25 @@ def check_fixtures(spec: str, doc: dict, tracked: set[str]) -> None:
 
 def check_references(spec: str, doc: dict, tracked: set[str]) -> None:
     base = os.path.dirname(spec)
+    suite_environment = doc.get("environment")
+    suite_destinations = {
+        entry["dest"]
+        for entry in environment_file_entries(suite_environment)
+        if isinstance(entry, dict) and isinstance(entry.get("dest"), str)
+    }
     for stim in doc.get("stimuli") or []:
+        duplicates = sorted(
+            entry["dest"]
+            for entry in environment_file_entries(stim.get("environment"))
+            if isinstance(entry, dict)
+            and isinstance(entry.get("dest"), str)
+            and entry["dest"] in suite_destinations
+        )
+        if duplicates and not os.path.basename(base).startswith("agent."):
+            errors.append(
+                f"{spec}: '{stim.get('name')}' redeclares suite environment file "
+                f"destination(s) {duplicates}; Vally rejects duplicate file entries "
+                "and loads none of the eval's stimuli")
         grader_types = {g.get("type") for g in (stim.get("graders") or [])
                         if isinstance(g, dict)}
         if (stim.get("golden_patch")
@@ -790,8 +808,9 @@ def check_references(spec: str, doc: dict, tracked: set[str]) -> None:
                 "but no golden_trajectory to provide the reference response")
         if (stim.get("golden_trajectory")
                 and not stim.get("golden_patch")
-                and (stim.get("environment") or {}).get("files")
-                and stimulus_requires_patch(spec, stim)):
+                and (environment_file_entries(suite_environment)
+                     or environment_file_entries(stim.get("environment")))
+                and stimulus_requires_patch(spec, stim, suite_environment)):
             missing_workspace_change_references.append(
                 f"{spec}: {stim.get('name')!r}")
 
@@ -807,7 +826,9 @@ def check_references(spec: str, doc: dict, tracked: set[str]) -> None:
                         oversized_inline_references.append(
                             f"{spec}: {stim.get('name')!r}")
                 elif isinstance(reference["inline"], str):
-                    check_patch_applies(spec, stim, patch_text=reference["inline"])
+                    check_patch_applies(
+                        spec, stim, patch_text=reference["inline"],
+                        suite_environment=suite_environment)
                 continue
             if not reference.get("path"):
                 continue
@@ -851,7 +872,8 @@ def check_references(spec: str, doc: dict, tracked: set[str]) -> None:
                     nested_response_references.append(
                         f"{spec}: {stim.get('name')!r} -> {source}")
             elif key == "golden_patch":
-                check_patch_applies(spec, stim, resolved)
+                check_patch_applies(
+                    spec, stim, resolved, suite_environment=suite_environment)
 
 
 def sanitize_image_ref(reference: str) -> str:
@@ -1350,9 +1372,24 @@ def check_required_vally_inputs(spec: str, doc: dict) -> None:
                 "lowercase kebab-case")
 
 
-def materialize_declared_files(spec: str, stim: dict, workspace: str) -> None:
-    """Copy fixture inputs to the same destination layout used by Vally."""
-    entries = (stim.get("environment") or {}).get("files") or []
+def environment_file_entries(environment) -> list:
+    if not isinstance(environment, dict):
+        return []
+    entries = environment.get("files") or []
+    return entries if isinstance(entries, list) else []
+
+
+def materialize_declared_files(
+        spec: str, stim: dict, workspace: str, suite_environment=None) -> None:
+    """Copy fixture inputs to the same destination layout used by Vally.
+
+    Vally merges suite-level `environment.files` into every stimulus workspace,
+    so suite entries are materialized before the stimulus's own entries.
+    """
+    entries = (
+        environment_file_entries(suite_environment)
+        + environment_file_entries(stim.get("environment"))
+    )
     base = os.path.dirname(spec)
     for entry in entries:
         source = repository_source_path(base, entry["src"])
@@ -1435,7 +1472,8 @@ def grader_passes_on_fixture(grader: dict, workspace: str) -> bool | None:
     return negate
 
 
-def stimulus_requires_patch(spec: str, stim: dict) -> bool:
+def stimulus_requires_patch(
+        spec: str, stim: dict, suite_environment=None) -> bool:
     """Return whether the expected workspace differs from the declared fixture."""
     graders = stim.get("graders") or []
     for grader in graders:
@@ -1452,7 +1490,7 @@ def stimulus_requires_patch(spec: str, stim: dict) -> bool:
 
     try:
         with tempfile.TemporaryDirectory(prefix="eval-fixture-baseline-") as workspace:
-            materialize_declared_files(spec, stim, workspace)
+            materialize_declared_files(spec, stim, workspace, suite_environment)
             return any(
                 grader_passes_on_fixture(grader, workspace) is not True
                 for grader in file_graders
@@ -1463,15 +1501,18 @@ def stimulus_requires_patch(spec: str, stim: dict) -> bool:
 
 def check_patch_applies(
         spec: str, stim: dict, patch: str | None = None,
-        *, patch_text: str | None = None) -> None:
+        *, patch_text: str | None = None, suite_environment=None) -> None:
     """Materialize declared fixture inputs and prove the golden patch applies."""
-    entries = (stim.get("environment") or {}).get("files") or []
+    entries = (
+        environment_file_entries(suite_environment)
+        + environment_file_entries(stim.get("environment"))
+    )
     if not entries:
         return  # Command-generated workspaces cannot be reconstructed statically.
 
     try:
         with tempfile.TemporaryDirectory(prefix="eval-golden-patch-") as workspace:
-            materialize_declared_files(spec, stim, workspace)
+            materialize_declared_files(spec, stim, workspace, suite_environment)
             subprocess.run(
                 ["git", "init", "-q"], cwd=workspace, capture_output=True,
                 text=True, check=True)

eng/eval-quality/selftest_eval_quality.py

@@ -1032,6 +1032,61 @@ def stale_golden_patch(d):
     golden_patch(d, "class MissingThing {}")
 
 
+def add_suite_environment(d, src, dest):
+    eval_path = EV(d)
+    with open(eval_path) as f:
+        raw = f.read()
+    with open(eval_path, "w") as f:
+        f.write(raw.replace(
+            "stimuli:\n",
+            "environment:\n"
+            "  files:\n"
+            f"    - src: {src}\n"
+            f"      dest: {dest}\n"
+            "stimuli:\n",
+            1))
+
+
+def golden_patch_on_suite_file(d):
+    add_suite_environment(d, "fixtures/sample/Thing.cs", "shared/Thing.cs")
+    golden_patch(d, "class Thing {}")
+    patch = os.path.join(d, "tests", "demo", "widget", "references", "fix.patch")
+    with open(patch) as f:
+        text = f.read()
+    with open(patch, "w") as f:
+        f.write(text.replace("sample/Thing.cs", "shared/Thing.cs"))
+
+
+def stimulus_redeclares_suite_file(d):
+    add_suite_environment(d, "fixtures/sample", "sample")
+
+
+def move_fixture_to_suite_environment(d):
+    """Declare the sample fixture only at suite level, as many real evals do."""
+    path = EV(d)
+    with open(path) as f:
+        raw = f.read()
+    stimulus_environment = (
+        "    environment:\n"
+        "      files:\n"
+        "        - src: fixtures/sample\n"
+        "          dest: sample\n")
+    assert stimulus_environment in raw
+    with open(path, "w") as f:
+        f.write(raw.replace(stimulus_environment, "", 1))
+    add_suite_environment(d, "fixtures/sample", "sample")
+
+
+def stale_golden_patch_on_suite_only_fixture(d):
+    stale_golden_patch(d)
+    move_fixture_to_suite_environment(d)
+
+
+def suite_only_state_grader_without_reference(d):
+    state_grader_without_materialized_reference(d)
+    move_fixture_to_suite_environment(d)
+
+
 def patch_with_output_grader_without_trajectory(d):
     applicable_golden_patch(d)
     path = EV(d)
@@ -2078,6 +2133,15 @@ def unresolvable_base_ref(d):
     case("golden patch EOL normalization preserves context", mixed_eol_golden_patch,
          expect_fail=False),
     case("stale golden patch does not apply", stale_golden_patch, expect_fail=True),
+    case("golden patch applies to suite environment file", golden_patch_on_suite_file,
+         expect_fail=False),
+    case("stimulus redeclares a suite environment file", stimulus_redeclares_suite_file,
+         expect_fail=True),
+    case("stale golden patch on suite-only fixture", stale_golden_patch_on_suite_only_fixture,
+         expect_fail=True),
+    output_case("suite-only state grader without reference",
+                suite_only_state_grader_without_reference,
+                "workspace state that differs from the starting fixture"),
     case("patch with output grader requires response trajectory",
          patch_with_output_grader_without_trajectory, expect_fail=True),
     case("patch and response trajectory cover output grader",

eng/skill-validator/src/docs/InvestigatingResults.md

@@ -48,6 +48,18 @@ behavior against the corrected executor.
 > the Copilot CLI's no-authentication setup block. Unrelated service and
 > configuration failures remain terminal.
 
+> Before token selection or evaluation starts, the workflow serially warms up
+> first-use state for every .NET SDK installed on the runner. This prevents
+> parallel fixture setup from racing in the .NET CLI's first-use
+> `NuGet-Migrations` initialization. `dotnet --list-sdks` returning no versions
+> fails the job with `dotnet --list-sdks reported no installed SDKs`. A warm-up
+> that does not create the SDK-specific sentinel fails with
+> `First-use sentinel missing for .NET SDK <version> after warm-up`. Both are
+> evaluation-infrastructure failures that occur before model sessions and
+> result artifacts exist; do not classify them as skill regressions. Diagnose
+> the runner's SDK installation or .NET CLI first-use state, then rerun the
+> evaluation without changing the skill.
+
 > PR session replay publishing is auxiliary. A missing or invalid
 > `SKILLS_DATA_TOKEN`, or one that cannot authenticate for a non-mutating
 > `git push --dry-run` to `dotnet/skills-data`, is detected before replay

plugins/dotnet/README.md

@@ -22,6 +22,7 @@ Prerequisites:
 - [csharp-expert](skills/csharp-expert/SKILL.md)
 - [csharp-refactoring](skills/csharp-refactoring/SKILL.md)
 - [msbuild](skills/msbuild/SKILL.md) — routed MSBuild diagnosis, performance, and authoring guidance
+- [dotnet-breaking-changes](skills/dotnet-breaking-changes/SKILL.md)
 - [setup-local-sdk](skills/setup-local-sdk/SKILL.md)
 
 ### MSBuild entry and specialist skills