refactor(agents): extract OpenClaw from BenchFlow core

.github/integration/scope_map.yml

@@ -106,8 +106,6 @@ roster:
 affected_agent_map:
   - glob: "src/benchflow/agents/codex_config.py"
     agent: codex-acp
-  - glob: "src/benchflow/agents/openclaw_acp_shim.py"
-    agent: openclaw
   - glob: "src/benchflow/agents/pi_acp_launcher.py"
     agent: pi-acp
   # Generic per-agent source files: src/benchflow/agents/<name>*.py where <name>

docs/external-agents.md

@@ -3,7 +3,7 @@
 BenchFlow's built-in registry covers a handful of agents (`bench agent list`).
 Everything else — goose, qwen-code, prime-agent, the omnigent harnesses, … —
 lives in the public **[benchflow-ai/agents](https://github.com/benchflow-ai/agents)**
-repo and loads into BenchFlow through one of four paths. For most users the
+repo and loads into BenchFlow through one of three paths. For most users the
 first one is all there is to know.
 
 ## 1. Zero-config remote autoload (the default)
@@ -65,6 +65,7 @@ directory, or off entirely:
 
 ```bash
 export BENCHFLOW_AGENTS_SOURCE="benchflow-ai/agents@my-branch"   # owner/repo[@ref]
+export BENCHFLOW_AGENTS_SOURCE="benchflow-ai/agents@0123456789abcdef0123456789abcdef01234567" # reproducible
 export BENCHFLOW_AGENTS_SOURCE="/path/to/agents-checkout"        # local dir
 export BENCHFLOW_AGENTS_SOURCE="off"                             # disable autoload
 ```
@@ -74,21 +75,27 @@ the standard way to try an agent from an open PR — e.g. verified live on
 BenchFlow 0.6.6: `BENCHFLOW_AGENTS_SOURCE="benchflow-ai/agents@add-prime-agent"`
 resolved and ran the `prime-agent` manifest with zero local setup.
 
-## 3. Local checkout at import: `BENCHFLOW_AGENTS_DIR`
+`bench agent list` reads this catalog best-effort and reports one consolidated
+warning when it is incomplete. `bench agent show NAME` uses runtime resolution.
+Unknown bare IDs fail closed. Raw commands require explicit syntax: whitespace
+(`agent --flag`) or a path prefix (`/`, `./`, `../`, or `~/`). Remote refs are
+fetched/refreshed by the normal source resolver; there is no offline fallback.
 
-For agents-repo development: point at a checkout and every
-`<dir>/manifest.toml` under it merges into the registry when `benchflow`
-imports (not lazily on miss):
+## 3. Local checkout override: `BENCHFLOW_AGENTS_DIR`
+
+For agents-repo development, point at a checkout. This selects the same lazy
+catalog path used by `BENCHFLOW_AGENTS_SOURCE`:
 
 ```bash
 export BENCHFLOW_AGENTS_DIR=/path/to/agents-checkout
 ```
 
-Unlike the miss-driven autoload, this path loads even for names that would
-never miss, and it is the loop used while editing a manifest. It is additive
-and compatible-merge only: colliding with an existing agent's aliases is a
-hard error rather than a silent shadow. Unset, the import is byte-for-byte
-identical to core — the mechanism is strictly opt-in.
+When both variables are set, a nonblank `BENCHFLOW_AGENTS_DIR` wins. Its local
+override mode may update an unchanged built-in's manifest-owned fields, but
+never replaces a plugin/runtime-modified entry. Loading remains one-shot after
+first runtime resolution. `bench agent list` may preview the selected catalog
+without activating or caching it; after activation, listing reuses the applied
+result.
 
 ## 4. Plugin packages (entry points)
 
@@ -109,17 +116,11 @@ error message if its name is later requested.
 ## Precedence
 
 1. Built-in registry (core `AGENTS`).
-2. `BENCHFLOW_AGENTS_DIR` manifests — merged at import; a collision with an
-   existing agent name or alias is a hard error, so manifests never shadow
-   built-ins.
-3. Entry-point plugin packages — loaded at import, after the manifest merge.
-   These register through plain `register_agent`, which overwrites by name:
-   a plugin **can** replace a built-in (or manifest-registered) agent that
-   shares its name. Well-behaved plugins skip names the registry already owns
-   (as the acp-registry package does).
-4. Remote autoload (`BENCHFLOW_AGENTS_SOURCE`, default `benchflow-ai/agents@main`)
-   — consulted last, once, only for names still unknown at resolution time;
-   it fills gaps and never overwrites.
+2. Entry-point plugin packages, loaded at import. Plugins may replace built-ins.
+3. One lazily selected manifest catalog. Nonblank `BENCHFLOW_AGENTS_DIR` wins
+   over `BENCHFLOW_AGENTS_SOURCE` and uses local-override policy; otherwise
+   remote/default loading uses gap-fill policy and never replaces existing
+   entries.
 
 Manifest capabilities are deliberately bounded: a `manifest.toml` is data-only
 (install/launch commands, env mapping, model-routing hints — the

docs/reference/python-api.md

@@ -263,7 +263,7 @@ config = rollout_config_from_yaml("rollout.yaml")
 result = await bf.run(config)
 ```
 
-## Registered Agents
+## Built-in Agents
 
 | Agent | Protocol | Auth | Aliases |
 |-------|----------|------|---------|
@@ -273,7 +273,6 @@ result = await bf.run(config)
 | `opencode` | ACP | inferred from model/provider | — |
 | `openhands` | ACP | LLM_API_KEY | `oh` |
 | `pi-acp` | ACP | ANTHROPIC_API_KEY | `pi` |
-| `openclaw` | ACP | inferred from model | — |
 
 The Auth column shows each agent's native/default credentials. Provider-prefixed
 models can use provider-specific credentials instead; for example, Azure
@@ -282,6 +281,9 @@ as `azure-foundry-openai/gpt-5.5` or
 `azure-foundry-anthropic/claude-opus-4-5`. BenchFlow routes these providers
 through LiteLLM on both Docker and Daytona.
 
+Additional agents load lazily from the external agents catalog. See
+[External agents](../external-agents.md).
+
 Any agent can be prefixed with `acpx/` to run via [ACPX](https://acpx.sh/) (e.g. `acpx/gemini`, `acpx/claude`). ACPX is a headless ACP client with persistent sessions and crash recovery. The underlying agent's install, env, credentials, and skill paths are preserved.
 
 ## Retry and Error Handling

src/benchflow/_utils/config.py

@@ -13,7 +13,7 @@ def normalize_agent_name(agent: str) -> str:
     stable runtime key is returned, so the Rollout/Evaluation path resolves
     acpx install/launch commands instead of the literal spec string.
 
-    Unknown specs are returned unchanged.
+    Unknown bare IDs fail closed. Explicit raw commands pass through unchanged.
     """
     return resolve_agent_key(agent)
 

src/benchflow/acp/client.py

@@ -330,7 +330,7 @@ async def session_load(
         cwd: str = "/app",
         mcp_servers: list[McpServerSpec] | None = None,
     ) -> ACPSession:  # ACP spec; unused until session resume is wired
-        """Load an existing session (used by agents like openclaw that need pre-created sessions).
+        """Load an existing pre-created session.
 
         ``mcp_servers`` mirrors :meth:`session_new` — the same task-configured
         servers are attached to the resumed session.

src/benchflow/acp/session.py

@@ -469,14 +469,14 @@ def handle_update(self, update: dict) -> None:
                 self._pending_text.append({"type": "agent_message", "text": text})
 
         elif update_type == "text_update":
-            # Used by openclaw shim — full text (not chunked)
+            # Some ACP agents send full text rather than chunks.
             text = update.get("text", "")
             if text:
                 self.message_chunks.append(text)
                 self._pending_text.append({"type": "agent_message", "text": text})
 
         elif update_type == "agent_thought":
-            # Used by openclaw shim — full thought (not chunked)
+            # Some ACP agents send full thoughts rather than chunks.
             text = update.get("text", "")
             if text:
                 self.thought_chunks.append(text)

src/benchflow/agents/__init__.py

@@ -16,11 +16,6 @@
                      native LLM providers, ``base_url`` / ``url_params``
                      resolution, ADC handling. The "add a new provider"
                      recipe lives in the module docstring.
-- ``openclaw_acp_shim.py``  Standalone script (read at import time by
-                     ``registry.py``) that wraps ``openclaw agent
-                     --local`` as an ACP server over stdio. Needed
-                     because openclaw's native ACP bridge requires a
-                     gateway.
 Nothing is re-exported from this ``__init__``: importers go through
 ``benchflow.agents.registry`` / ``benchflow.agents.providers`` directly,
 which is what the registry-only-change rule depends on.

src/benchflow/agents/env.py

@@ -484,7 +484,7 @@ def resolve_provider_env(
     # Resolve bare family ids (e.g. "deepseek-v4-pro") too, not just explicit
     # "provider/" prefixes — otherwise the provider env (NAME/BASE_URL/API_KEY)
     # is never emitted and harnesses that rely on it misroute (openhands' litellm
-    # saw no provider; openclaw defaulted to anthropic/). Mirrors acp/runtime.py.
+    # saw no provider and defaulted incorrectly). Mirrors acp/runtime.py.
     _prov = find_provider(model) or find_provider_for_bare_model(model)
     if _prov:
         _prov_name, _prov_cfg = _prov