Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 41 additions & 38 deletions packages/opencode/specs/effect-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,9 @@ That is fine for leaf files like `schema.ts`. Keep the service surface in the ow

## Migration checklist

Fully migrated (single namespace, InstanceState where needed, flattened facade):
Service-shape migrated (single namespace, traced methods, `InstanceState` where needed).

This checklist is only about the service shape migration. Many of these services still keep `makeRuntime(...)` plus async facade exports; that facade-removal phase is tracked separately in [Destroying the facades](#destroying-the-facades).

- [x] `Account` — `account/index.ts`
- [x] `Agent` — `agent/agent.ts`
Expand Down Expand Up @@ -221,20 +223,22 @@ Fully migrated (single namespace, InstanceState where needed, flattened facade):
- [x] `Provider` — `provider/provider.ts`
- [x] `Storage` — `storage/storage.ts`
- [x] `ShareNext` — `share/share-next.ts`
- [x] `SessionTodo` — `session/todo.ts`

Still open:
Still open at the service-shape level:

- [x] `SessionTodo` — `session/todo.ts`
- [ ] `SyncEvent` — `sync/index.ts`
- [ ] `Workspace` — `control-plane/workspace.ts`
- [ ] `SyncEvent` — `sync/index.ts` (deferred pending sync with James)
- [ ] `Workspace` — `control-plane/workspace.ts` (deferred pending sync with James)

## Tool interface → Effect

`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch. Tool definitions should now stay Effect-native all the way through initialization instead of using Promise-returning init callbacks. Tools can still use lazy init callbacks when they need instance-bound state at init time, but those callbacks should return `Effect`, not `Promise`. Remaining work is:
`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch, and the current tools in `src/tool/*.ts` have been migrated to the Effect-native `Tool.define(...)` shape.

1. Migrate each tool body to return Effects
2. Keep `Tool.define()` inputs Effect-native
3. Update remaining callers to `yield*` tool initialization instead of `await`ing
The remaining work here is follow-on cleanup rather than the top-level tool interface migration:

1. Remove internal `Effect.promise(...)` bridges where practical
2. Keep replacing raw platform helpers with Effect services inside tool bodies
3. Update remaining callers and tests to prefer `yield* info.init()` / `Tool.init(...)` over older Promise-oriented patterns

### Tool migration details

Expand All @@ -254,52 +258,49 @@ This keeps migrated tool tests aligned with the production service graph today,

Individual tools, ordered by value:

- [ ] `apply_patch.ts` — HIGH: multi-step orchestration, error accumulation, Bus events
- [ ] `bash.ts` — HIGH: shell orchestration, quoting, timeout handling, output capture
- [x] `read.ts` — HIGH: streaming I/O, readline, binary detection → FileSystem + Stream
- [ ] `edit.ts` — HIGH: multi-step diff/format/publish pipeline, FileWatcher lock
- [ ] `grep.ts` — MEDIUM: spawns ripgrep → ChildProcessSpawner, timeout handling
- [ ] `write.ts` — MEDIUM: permission checks, diagnostics polling, Bus events
- [ ] `codesearch.ts` — MEDIUM: HTTP + SSE + manual timeout → HttpClient + Effect.timeout
- [ ] `webfetch.ts` — MEDIUM: fetch with UA retry, size limits → HttpClient
- [ ] `websearch.ts` — MEDIUM: MCP over HTTP → HttpClient
- [ ] `batch.ts` — MEDIUM: parallel execution, per-call error recovery → Effect.all
- [ ] `task.ts` — MEDIUM: task state management
- [ ] `ls.ts` — MEDIUM: bounded directory listing over ripgrep-backed traversal
- [ ] `multiedit.ts` — MEDIUM: sequential edit orchestration over `edit.ts`
- [ ] `glob.ts` — LOW: simple async generator
- [ ] `lsp.ts` — LOW: dispatch switch over LSP operations
- [ ] `question.ts` — LOW: prompt wrapper
- [ ] `skill.ts` — LOW: skill tool adapter
- [ ] `todo.ts` — LOW: todo persistence wrapper
- [ ] `invalid.ts` — LOW: invalid-tool fallback
- [ ] `plan.ts` — LOW: plan file operations
- [x] `apply_patch.ts` — HIGH: multi-step orchestration, error accumulation, Bus events
- [x] `bash.ts` — HIGH: shell orchestration, quoting, timeout handling, output capture
- [x] `read.ts` — HIGH: effectful interface migrated; still has raw fs/readline internals tracked below
- [x] `edit.ts` — HIGH: multi-step diff/format/publish pipeline, FileWatcher lock
- [x] `grep.ts` — MEDIUM: spawns ripgrep → ChildProcessSpawner, timeout handling
- [x] `write.ts` — MEDIUM: permission checks, diagnostics polling, Bus events
- [x] `codesearch.ts` — MEDIUM: HTTP + SSE + manual timeout → HttpClient + Effect.timeout
- [x] `webfetch.ts` — MEDIUM: fetch with UA retry, size limits → HttpClient
- [x] `websearch.ts` — MEDIUM: MCP over HTTP → HttpClient
- [x] `task.ts` — MEDIUM: task state management
- [x] `ls.ts` — MEDIUM: bounded directory listing over ripgrep-backed traversal
- [x] `multiedit.ts` — MEDIUM: sequential edit orchestration over `edit.ts`
- [x] `glob.ts` — LOW: simple async generator
- [x] `lsp.ts` — LOW: dispatch switch over LSP operations
- [x] `question.ts` — LOW: prompt wrapper
- [x] `skill.ts` — LOW: skill tool adapter
- [x] `todo.ts` — LOW: todo persistence wrapper
- [x] `invalid.ts` — LOW: invalid-tool fallback
- [x] `plan.ts` — LOW: plan file operations

`batch.ts` was removed from `src/tool/` and is no longer tracked here.

## Effect service adoption in already-migrated code

Some already-effectified areas still use raw `Filesystem.*` or `Process.spawn` in their implementation or helper modules. These are low-hanging fruit — the layers already exist, they just need the dependency swap.

### `Filesystem.*` → `AppFileSystem.Service` (yield in layer)

- [ ] `file/index.ts` — 1 remaining `Filesystem.readText()` call in untracked diff handling
- [ ] `config/config.ts` — 5 remaining `Filesystem.*` calls in `installDependencies()`
- [ ] `provider/provider.ts` — 1 remaining `Filesystem.readJson()` call for recent model state
- [x] `config/config.ts` — `installDependencies()` now uses `AppFileSystem`
- [x] `provider/provider.ts` — recent model state now reads via `AppFileSystem.Service`

### `Process.spawn` → `ChildProcessSpawner` (yield in layer)

- [ ] `format/formatter.ts` — 2 remaining `Process.spawn()` checks (`air`, `uv`)
- [x] `format/formatter.ts` — direct `Process.spawn()` checks removed (`air`, `uv`)
- [ ] `lsp/server.ts` — multiple `Process.spawn()` installs/download helpers

## Filesystem consolidation

`util/filesystem.ts` (raw fs wrapper) is currently imported by **34 files**. The effectified `AppFileSystem` service (`filesystem/index.ts`) is currently imported by **15 files**. As services and tools are effectified, they should switch from `Filesystem.*` to yielding `AppFileSystem.Service` — this happens naturally during each migration, not as a separate effort.

Similarly, **21 files** still import raw `fs` or `fs/promises` directly. These should migrate to `AppFileSystem` or `Filesystem.*` as they're touched.
`util/filesystem.ts` is still used widely across `src/`, and raw `fs` / `fs/promises` imports still exist in multiple tooling and infrastructure files. As services and tools are effectified, they should switch from `Filesystem.*` to yielding `AppFileSystem.Service` where possible — this should happen naturally during each migration, not as a separate sweep.

Current raw fs users that will convert during tool migration:

- `tool/read.ts` — fs.createReadStream, readline
- `tool/apply_patch.ts` — fs/promises
- `file/ripgrep.ts` — fs/promises
- `patch/index.ts` — fs, fs/promises

Expand All @@ -312,7 +313,9 @@ Current raw fs users that will convert during tool migration:

## Destroying the facades

Every service currently exports async facade functions at the bottom of its namespace — `export async function read(...) { return runPromise(...) }` — backed by a per-service `makeRuntime`. These exist because cyclic imports used to force each service to build its own independent runtime. Now that the layer DAG is acyclic and `AppRuntime` (`src/effect/app-runtime.ts`) composes everything into one `ManagedRuntime`, we're removing them.
This phase is still broadly open. As of 2026-04-11 there are still 31 `makeRuntime(...)` call sites under `src/`, and many service namespaces still export async facade helpers like `export async function read(...) { return runPromise(...) }`.

These facades exist because cyclic imports used to force each service to build its own independent runtime. Now that the layer DAG is acyclic and `AppRuntime` (`src/effect/app-runtime.ts`) composes everything into one `ManagedRuntime`, we're removing them.

### Process

Expand Down
152 changes: 95 additions & 57 deletions packages/opencode/src/config/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,19 @@ import { Instance, type InstanceContext } from "../project/instance"
import { LSPServer } from "../lsp/server"
import { Installation } from "@/installation"
import { ConfigMarkdown } from "./markdown"
import { constants, existsSync } from "fs"
import { existsSync } from "fs"
import { Bus } from "@/bus"
import { GlobalBus } from "@/bus/global"
import { Event } from "../server/event"
import { Glob } from "../util/glob"
import { iife } from "@/util/iife"
import { Account } from "@/account"
import { isRecord } from "@/util/record"
import { ConfigPaths } from "./paths"
import { Filesystem } from "@/util/filesystem"
import type { ConsoleState } from "./console-state"
import { AppFileSystem } from "@/filesystem"
import { InstanceState } from "@/effect/instance-state"
import { makeRuntime } from "@/effect/run-service"
import { Duration, Effect, Layer, Option, Context } from "effect"
import { Context, Duration, Effect, Exit, Fiber, Layer, Option } from "effect"
import { Flock } from "@/util/flock"
import { isPathPluginSpec, parsePluginSpecifier, resolvePathPluginTarget } from "@/plugin/shared"
import { Npm } from "@/npm"
Expand Down Expand Up @@ -140,53 +138,11 @@ export namespace Config {
}

export type InstallInput = {
signal?: AbortSignal
waitTick?: (input: { dir: string; attempt: number; delay: number; waited: number }) => void | Promise<void>
}

export async function installDependencies(dir: string, input?: InstallInput) {
if (!(await isWritable(dir))) return
await using _ = await Flock.acquire(`config-install:${Filesystem.resolve(dir)}`, {
signal: input?.signal,
onWait: (tick) =>
input?.waitTick?.({
dir,
attempt: tick.attempt,
delay: tick.delay,
waited: tick.waited,
}),
})
input?.signal?.throwIfAborted()

const pkg = path.join(dir, "package.json")
const target = Installation.isLocal() ? "*" : Installation.VERSION
const json = await Filesystem.readJson<{ dependencies?: Record<string, string> }>(pkg).catch(() => ({
dependencies: {},
}))
json.dependencies = {
...json.dependencies,
"@opencode-ai/plugin": target,
}
await Filesystem.writeJson(pkg, json)

const gitignore = path.join(dir, ".gitignore")
const ignore = await Filesystem.exists(gitignore)
if (!ignore) {
await Filesystem.write(
gitignore,
["node_modules", "package.json", "package-lock.json", "bun.lock", ".gitignore"].join("\n"),
)
}
await Npm.install(dir)
}

async function isWritable(dir: string) {
try {
await fsNode.access(dir, constants.W_OK)
return true
} catch {
return false
}
type Package = {
dependencies?: Record<string, string>
}

function rel(item: string, patterns: string[]) {
Expand Down Expand Up @@ -1111,14 +1067,15 @@ export namespace Config {
type State = {
config: Info
directories: string[]
deps: Promise<void>[]
deps: Fiber.Fiber<void, never>[]
consoleState: ConsoleState
}

export interface Interface {
readonly get: () => Effect.Effect<Info>
readonly getGlobal: () => Effect.Effect<Info>
readonly getConsoleState: () => Effect.Effect<ConsoleState>
readonly installDependencies: (dir: string, input?: InstallInput) => Effect.Effect<void, AppFileSystem.Error>
readonly update: (config: Info) => Effect.Effect<void>
readonly updateGlobal: (config: Info) => Effect.Effect<Info>
readonly invalidate: (wait?: boolean) => Effect.Effect<void>
Expand Down Expand Up @@ -1320,6 +1277,74 @@ export namespace Config {
return yield* cachedGlobal
})

const install = Effect.fnUntraced(function* (dir: string) {
const pkg = path.join(dir, "package.json")
const gitignore = path.join(dir, ".gitignore")
const plugin = path.join(dir, "node_modules", "@opencode-ai", "plugin", "package.json")
const target = Installation.isLocal() ? "*" : Installation.VERSION
const json = yield* fs.readJson(pkg).pipe(
Effect.catch(() => Effect.succeed({} satisfies Package)),
Effect.map((x): Package => (isRecord(x) ? (x as Package) : {})),
)
const hasDep = json.dependencies?.["@opencode-ai/plugin"] === target
const hasIgnore = yield* fs.existsSafe(gitignore)
const hasPkg = yield* fs.existsSafe(plugin)

if (!hasDep) {
yield* fs.writeJson(pkg, {
...json,
dependencies: {
...json.dependencies,
"@opencode-ai/plugin": target,
},
})
}

if (!hasIgnore) {
yield* fs.writeFileString(
gitignore,
["node_modules", "package.json", "package-lock.json", "bun.lock", ".gitignore"].join("\n"),
)
}

if (hasDep && hasIgnore && hasPkg) return

yield* Effect.promise(() => Npm.install(dir))
})

const installDependencies = Effect.fn("Config.installDependencies")(function* (
dir: string,
input?: InstallInput,
) {
if (
!(yield* fs.access(dir, { writable: true }).pipe(
Effect.as(true),
Effect.orElseSucceed(() => false),
))
)
return

const key =
process.platform === "win32" ? "config-install:win32" : `config-install:${AppFileSystem.resolve(dir)}`

yield* Effect.acquireUseRelease(
Effect.promise((signal) =>
Flock.acquire(key, {
signal,
onWait: (tick) =>
input?.waitTick?.({
dir,
attempt: tick.attempt,
delay: tick.delay,
waited: tick.waited,
}),
}),
),
() => install(dir),
(lease) => Effect.promise(() => lease.release()),
)
})

const loadInstanceState = Effect.fnUntraced(function* (ctx: InstanceContext) {
const auth = yield* authSvc.all().pipe(Effect.orDie)

Expand Down Expand Up @@ -1402,7 +1427,7 @@ export namespace Config {
log.debug("loading config from OPENCODE_CONFIG_DIR", { path: Flag.OPENCODE_CONFIG_DIR })
}

const deps: Promise<void>[] = []
const deps: Fiber.Fiber<void, never>[] = []

for (const dir of unique(directories)) {
if (dir.endsWith(".opencode") || dir === Flag.OPENCODE_CONFIG_DIR) {
Expand All @@ -1416,12 +1441,18 @@ export namespace Config {
}
}

const dep = iife(async () => {
await installDependencies(dir)
})
void dep.catch((err) => {
log.warn("background dependency install failed", { dir, error: err })
})
const dep = yield* installDependencies(dir).pipe(
Effect.exit,
Effect.tap((exit) =>
Exit.isFailure(exit)
? Effect.sync(() => {
log.warn("background dependency install failed", { dir, error: String(exit.cause) })
})
: Effect.void,
),
Effect.asVoid,
Effect.forkScoped,
)
deps.push(dep)

result.command = mergeDeep(result.command ?? {}, yield* Effect.promise(() => loadCommand(dir)))
Expand Down Expand Up @@ -1558,7 +1589,9 @@ export namespace Config {
})

const waitForDependencies = Effect.fn("Config.waitForDependencies")(function* () {
yield* InstanceState.useEffect(state, (s) => Effect.promise(() => Promise.all(s.deps).then(() => undefined)))
yield* InstanceState.useEffect(state, (s) =>
Effect.forEach(s.deps, Fiber.join, { concurrency: "unbounded" }).pipe(Effect.asVoid),
)
})

const update = Effect.fn("Config.update")(function* (config: Info) {
Expand Down Expand Up @@ -1613,6 +1646,7 @@ export namespace Config {
get,
getGlobal,
getConsoleState,
installDependencies,
update,
updateGlobal,
invalidate,
Expand Down Expand Up @@ -1642,6 +1676,10 @@ export namespace Config {
return runPromise((svc) => svc.getConsoleState())
}

export async function installDependencies(dir: string, input?: InstallInput) {
return runPromise((svc) => svc.installDependencies(dir, input))
}

export async function update(config: Info) {
return runPromise((svc) => svc.update(config))
}
Expand Down
Loading
Loading