fix(audit): validate skill subsets against prepared lock-pinned replay

docs/src/content/docs/enterprise/enforce-in-ci.md

@@ -120,7 +120,7 @@ jobs:
 
 `setup-only: true` leaves every deployed file exactly as checked out.
 `apm audit --ci` now self-hydrates its scratch replay from `apm.lock.yaml`,
-so drift and `config-consistency` still run even when the checkout has no
+so drift, `config-consistency`, and `skill-subset-consistency` still run even when the checkout has no
 live `apm_modules/` tree. If the scratch replay itself cannot be materialized,
 the audit fails closed instead of reporting a green skip. The
 `content-integrity` check still verifies that every deployed file's SHA-256

docs/src/content/docs/integrations/ci-cd.md

@@ -92,7 +92,11 @@ provides the CLI, then run the full CI gate:
 
 In setup-only CI, `apm audit --ci` now self-hydrates a lock-pinned scratch
 install when `apm_modules/` is absent, so drift and `config-consistency`
-still run without mutating the checkout. Repos that gitignore deployed
+still run without mutating the checkout. `skill-subset-consistency` also
+checks selected skills against this lock-pinned tree, not the absent checkout
+dependencies. Invalid selections and manifest/lock mismatches still fail;
+deployed-file integrity and drift checks still inspect the checkout.
+Repos that gitignore deployed
 outputs can still use the audit-only pattern: `deployed-files-present`
 skips gitignored paths automatically, so a fresh checkout of a repo that
 gitignores a deploy directory (e.g. `.agents/`) passes the check without

docs/src/content/docs/reference/baseline-checks.md

@@ -118,8 +118,8 @@ the [policy schema](../policy-schema/).
 
 ### `skill-subset-consistency`
 
-- **What it verifies.** That each `skills:` selection in `apm.yml` matches the `skill_subset` recorded in the lockfile, and that every recorded skill path exists in the resolved package tree.
-- **Fails when.** The sorted manifest skill list differs from the sorted lockfile `skill_subset`, or a recorded subset path no longer maps to a deployable skill in the installed package.
+- **What it verifies.** That each `skills:` selection in `apm.yml` matches the `skill_subset` recorded in the lockfile, and that every recorded skill path exists in the resolved package tree. When CI audit has prepared a lock-pinned scratch replay, this check uses its dependency tree instead of checkout-local `apm_modules/`; no checkout install is required.
+- **Fails when.** The sorted manifest skill list differs from the sorted lockfile `skill_subset`, or a recorded subset path does not map to a deployable skill in the dependency tree being checked.
 - **Remediation.** Run `apm install` to regenerate the lockfile against the current selection.
 
 ### `config-consistency`
@@ -153,7 +153,7 @@ the [policy schema](../policy-schema/).
 
 ## Run order and fail-fast
 
-The aggregate runner in `run_baseline_checks` evaluates checks in this order: `manifest-parse` (only when `apm.yml` is unparseable), `lockfile-exists`, `ref-consistency`, `deployment-ledger-owners`, `deployed-files-present`, `no-orphaned-packages`, `skill-subset-consistency`, `config-consistency`, `content-integrity`, `includes-consent`. Drift is invoked separately by the audit command after the baseline batch, but in `--ci` mode it shares the same cold-cache scratch materialization with `config-consistency`.
+The aggregate runner in `run_baseline_checks` evaluates checks in this order: `manifest-parse` (only when `apm.yml` is unparseable), `lockfile-exists`, `ref-consistency`, `deployment-ledger-owners`, `deployed-files-present`, `no-orphaned-packages`, `skill-subset-consistency`, `config-consistency`, `content-integrity`, `includes-consent`. Drift is invoked separately by the audit command after the baseline batch, but in `--ci` mode it shares the same cold-cache scratch materialization with `skill-subset-consistency` and `config-consistency`.
 
 With fail-fast on (the default), the runner stops at the first failing check. `apm audit --ci --no-fail-fast` evaluates every check so the report lists every problem at once.
 

docs/src/content/docs/reference/cli/audit.md

@@ -16,7 +16,7 @@ apm audit [PACKAGE] [OPTIONS]
 `apm audit` is the explicit security and integrity tool. It runs in two modes:
 
 - **Content scan mode** (default). Discovers recognized deployed primitives and checks applicable prompt content for hidden Unicode, including untracked primitives and recorded files outside currently selected target directories. It replays the install pipeline into a scratch tree to detect drift (hand-edits to deployed files, missing integrations, orphaned files vs the lockfile). Can also remediate regular prompt documents with `--strip` or scan an arbitrary file with `--file`.
-- **CI gate mode** (`--ci`). Runs lockfile consistency checks plus drift in machine-readable form (text, JSON, or SARIF) suitable for branch-protection gates. When `apm_modules/` is absent but `apm.lock.yaml` is present, CI mode self-hydrates a lock-pinned scratch install for `config-consistency` and drift without mutating the checkout. Auto-discovers org policy from your project's git remote unless `--no-policy` is set.
+- **CI gate mode** (`--ci`). Runs lockfile consistency checks plus drift in machine-readable form (text, JSON, or SARIF) suitable for branch-protection gates. When `apm_modules/` is absent but `apm.lock.yaml` is present, CI mode self-hydrates a lock-pinned scratch install for `skill-subset-consistency`, `config-consistency`, and drift without mutating the checkout. Auto-discovers org policy from your project's git remote unless `--no-policy` is set.
 
 Global audit also checks resolved external deployment roots such as
 `HERMES_HOME` and `CLAUDE_CONFIG_DIR`. Default audit compares tracked files in

packages/apm-guide/.apm/skills/apm-usage/commands.md

@@ -309,6 +309,8 @@ rewriting files. Explicit `--file` remains user-directed.
 |---------|---------|-----------|
 | `apm audit [PKG]` | Scan installed primitives for hidden Unicode, drift, and lockfile/policy violations | `--file PATH`, `--strip`, `--dry-run`, `-v`, `-f [text\|json\|sarif\|md]`, `-o PATH`, `--ci`, `--policy SOURCE`, `--no-cache`, `--no-fail-fast`, `--no-drift`, `--external NAME` (experimental; ingest a third-party SARIF scanner, e.g. `skillspector`), `--external-sarif PATH`, `--external-llm/--no-external-llm`, `--external-args TEXT` |
 
+For `apm audit --ci` in a checkout without `apm_modules/`, skill-subset validation uses the prepared lock-pinned scratch dependency tree. No checkout install is required. Invalid selections, manifest/lock mismatches, deployed-file integrity failures, and drift still fail the audit.
+
 `apm audit` runs **drift detection by default** (issue #1071). It replays `apm install` into a temporary scratch tree and diffs the result against your working tree. Catches three failure modes: (1) `.apm/` source added without re-running `apm install`, (2) hand-edits to deployed files that diverge from canonical source, (3) orphan files left after their source was removed. The scan is read-only -- never writes to your project, lockfile, or live `apm_modules/`. Build IDs, CRLF line endings, and BOMs are normalized away so they cannot trigger false positives. Bare `apm audit` still uses the warmed local cache and skips with an informational message when the cache is absent. `apm audit --ci` is stricter: when `apm_modules/` is missing but `apm.lock.yaml` is present, it self-hydrates a lock-pinned scratch install for `config-consistency` and drift without touching the checkout. That closes the setup-only CI gap for repos that commit deployed files. Repos that gitignore deployed outputs still need those files present on disk for `deployed-files-present`, so keep the full-install CI pattern there. Use `--no-drift` to opt out (e.g. fast inner loops); the flag is mutually exclusive with `--strip`/`--file`. Ordinary drift remains advisory in bare audit and fails only in `--ci` mode or when policy promotes it. A stale canonical deployment owner is different: `deployment-ledger-owners` is a hard integrity failure in both modes, exits 1, names the owner and path in text/JSON/SARIF, and blocks `--strip`. Remediate it with `apm prune`, then rerun `apm audit`. Drift output is integrated into JSON (top-level `drift` key) and SARIF (rule IDs `apm/drift/<kind>` where kind is `modified`/`unintegrated`/`orphaned`).
 `apm audit` runs **drift detection by default** (issue #1071). It replays `apm install` cache-only into a temporary scratch tree and diffs the result against your working tree. It catches four failure modes: (1) `.apm/` source added without re-running `apm install`, (2) hand-edits to deployed files that diverge from canonical source, (3) orphan files left after their source was removed, and (4) `unrecorded` files that install deploys but no lockfile entry claims. The scan is read-only -- never writes to your project, lockfile, or `apm_modules/`. Build IDs, CRLF line endings, and BOMs are normalized away so they cannot trigger false positives. If the install cache has not been warmed (e.g. a fresh checkout before the first `apm install`), the drift check is skipped with an informational message and can still exit 0; run `apm install` before relying on drift until cold-cache replay lands. Use `--no-drift` to opt out with reduced coverage; the flag is mutually exclusive with `--strip`/`--file`. Ordinary drift remains advisory in bare audit and fails in `--ci` mode. Remediate `unrecorded` with `apm install`, then commit the regenerated `apm.lock.yaml`. A stale canonical deployment owner is different: `deployment-ledger-owners` is a hard integrity failure in both modes, exits 1, names the owner and path in text/JSON/SARIF, and blocks `--strip`. Remediate it with `apm prune`, then rerun `apm audit`. Drift output is integrated into JSON (top-level `drift` key) and SARIF (rule IDs `apm/drift/<kind>` where kind is `modified`/`unintegrated`/`orphaned`/`unrecorded`).
 `apm audit` runs **drift detection by default** (issue #1071). It replays `apm install` cache-only into a temporary scratch tree and diffs the result against your working tree. It catches four failure modes: (1) `.apm/` source added without re-running `apm install`, (2) hand-edits to deployed files that diverge from canonical source, (3) orphan files left after their source was removed, and (4) `unrecorded` files that install deploys but no lockfile entry claims. The scan is read-only -- never writes to your project, lockfile, or `apm_modules/`. Build IDs, CRLF line endings, and BOMs are normalized away so they cannot trigger false positives. If the install cache has not been warmed (e.g. a fresh checkout before the first `apm install`), the drift check is skipped with an informational message and can still exit 0; run `apm install` before relying on drift until cold-cache replay lands. Use `--no-drift` to opt out with reduced coverage; the flag is mutually exclusive with `--strip`/`--file`. Drift is advisory in bare audit by default unless policy enables `security.audit.fail_on_drift`; `--ci` always gates on drift. Remediate `unrecorded` with `apm install`, then commit the regenerated `apm.lock.yaml`. A stale canonical deployment owner is different: `deployment-ledger-owners` is a hard integrity failure in both modes, exits 1, names the owner and path in text/JSON/SARIF, and blocks `--strip`. Remediate it with `apm prune`, then rerun `apm audit`. Drift output is integrated into JSON (top-level `drift` key) and SARIF (rule IDs `apm/drift/<kind>` where kind is `modified`/`unintegrated`/`orphaned`/`unrecorded`).

scripts/architecture_linter/checks/install_frozen_and_audit.py

@@ -283,11 +283,15 @@ def check_audit_replay(provider: FactsProvider) -> tuple[Violation, ...]:
     config_body = _awk_body(
         ci_checks, re.compile(r"^def _check_config_consistency\("), re.compile(r"^def ")
     )
+    subset_body = _awk_body(
+        ci_checks, re.compile(r"^def _check_skill_subset_consistency\("), re.compile(r"^def ")
+    )
     if (
         not _present_re(owner, re.compile(r"^def prepare_ci_audit_replay\("))
         or "prepare_ci_audit_replay" not in audit_gate_calls
         or "run_replay" in audit_gate_calls
         or not _body_has(config_body, "prepared_replay.modules_root")
+        or not _body_has(subset_body, "prepared_replay.modules_root")
     ):
         findings.append(
             _summary(

src/apm_cli/install/audit_replay.py

@@ -2,8 +2,9 @@
 
 This module owns the one-shot orchestration that turns a checkout with a
 lockfile but no live ``apm_modules/`` tree into a prepared, lock-pinned scratch
-replay. ``commands/audit.py`` creates the replay once, then both
-``config-consistency`` and ``drift`` consume the same materialized state.
+replay. ``commands/audit.py`` creates the replay once, then
+``skill-subset-consistency``, ``config-consistency`` and ``drift`` consume
+the same materialized state.
 """
 
 from __future__ import annotations

src/apm_cli/policy/ci_checks.py

@@ -27,6 +27,7 @@
     from ..install.drift import DriftFinding
     from ..integration.targets import TargetProfile
     from ..models.apm_package import APMPackage
+    from ..models.dependency.reference import DependencyReference
 
 _logger = logging.getLogger(__name__)
 
@@ -342,8 +343,17 @@ def _check_skill_subset_consistency(
     manifest: APMPackage,
     lock: LockFile,
     project_root: Path,
+    *,
+    prepared_replay: PreparedCiAuditReplay | None = None,
 ) -> CheckResult:
     """Verify skill subsets match the lockfile and real package tree."""
+    from ..constants import APM_MODULES_DIR
+
+    modules_root = (
+        prepared_replay.modules_root
+        if prepared_replay is not None
+        else project_root / APM_MODULES_DIR
+    )
     mismatches: list[str] = []
     for dep_ref in manifest.get_all_apm_dependencies():
         key = dep_ref.get_unique_key()
@@ -360,7 +370,7 @@ def _check_skill_subset_consistency(
             )
             continue
         missing = _missing_recorded_skill_subset_paths(
-            project_root,
+            modules_root,
             dep_ref,
             locked_dep.package_type,
             lock_subset,
@@ -388,8 +398,8 @@ def _check_skill_subset_consistency(
 
 
 def _missing_recorded_skill_subset_paths(
-    project_root: Path,
-    dep_ref,
+    modules_root: Path,
+    dep_ref: DependencyReference,
     package_type: str | None,
     subset: list[str],
 ) -> tuple[str, ...]:
@@ -399,15 +409,14 @@ def _missing_recorded_skill_subset_paths(
 
     from types import SimpleNamespace
 
-    from ..constants import APM_MODULES_DIR
     from ..install.outcome import missing_requested_components
     from ..integration.skill_integrator import SkillIntegrator
     from ..models.validation import PackageType
 
     try:
         resolved_package_type = PackageType(package_type) if package_type else None
         package_info = SimpleNamespace(
-            install_path=dep_ref.get_install_path(project_root / APM_MODULES_DIR),
+            install_path=dep_ref.get_install_path(modules_root),
             package_type=resolved_package_type,
         )
         available = SkillIntegrator.available_skill_names(package_info)
@@ -1005,7 +1014,11 @@ def _run(check: CheckResult) -> bool:
         return result
 
     # Check 6: Skill subset consistency (manifest vs lockfile)
-    if _run(_check_skill_subset_consistency(manifest, lock, project_root)):
+    if _run(
+        _check_skill_subset_consistency(
+            manifest, lock, project_root, prepared_replay=prepared_replay
+        )
+    ):
         return result
 
     # Check 7: Config consistency (MCP)

tests/integration/test_architecture_install_compound_mutations.py

@@ -482,6 +482,15 @@ def _replace(old: str, new: str) -> tuple[tuple[str, str], ...]:
             "                    prepared_replay = prepare_ci_audit_replay(",
         ),
     ),
+    CompoundMutation(
+        "audit-replay-subset-checkout-root",
+        AUDIT_RULE,
+        "src/apm_cli/policy/ci_checks.py",
+        _replace(
+            "    modules_root = (\n        prepared_replay.modules_root\n",
+            "    modules_root = (\n        project_root / APM_MODULES_DIR\n",
+        ),
+    ),
     CompoundMutation(
         "audit-replay-config-root",
         AUDIT_RULE,