feat(parser-pipeline): create @endo/parser-pipeline

[boneskull]

Babel AST parsing is expensive. When multiple consumers need to analyze or transform the same module source (@endo/evasive-transform, @endo/module-source, LavaMoat policy inspection), parsing each one independently wastes significant wall-clock time.

This package composes Babel visitor passes from multiple packages into a single parse-traverse-generate cycle. It provides two parserForLanguage-compatible parsers:

• createComposedParser -- synchronous, in-process; for the execution path where modules load on-demand
• createWorkerParser -- async, worker-pool-backed; for policy generation where hundreds of modules can parse in parallel

[boneskull]

📚 Pull Request Stack

• branch: master
  • feat(parser-pipeline): create @endo/parser-pipeline #3158 - branch: boneskull/ast-service (this PR)
    • feat(module-source): expose createModuleSourcePasses() #3159 - branch: boneskull/module-source-pipeline
      • feat(evasive-transform): expose createEvasiveTransformPass for use with @endo/parser-pipeline #3161 - branch: boneskull/evasive-pipeline
        • branch: boneskull/cjs-ast-parser

---

Managed by gh-stack

[changeset-bot]

⚠️ No Changeset found

Latest commit: 460895c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[boneskull]

TODO: figure out why the docs build is failing

[Copilot]

Pull request overview

This PR introduces a new @endo/parser-pipeline workspace package to avoid redundant Babel AST parsing by composing multiple analyzer/transform visitor passes into a single parse→traverse→generate cycle, and adds an async worker-pool-backed parser option for parallel policy-generation workflows. It also updates @endo/compartment-mapper’s parser typing and mapping logic to support asynchronous parsers alongside synchronous ones.

Changes:

• Add @endo/parser-pipeline with createComposedParser (sync) and createWorkerParser (async worker pool), plus worker-side runPipelineInWorker.
• Refactor @endo/compartment-mapper parser types and makeMapParsers to support async parsers and async module transforms.
• Add tests for composed parsing and worker-pool behavior.

Reviewed changes

Copilot reviewed 23 out of 25 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
yarn.lock	Adds new workspace package and Babel-related deps/types.
typedoc.json	Formatting tweaks and excludes the new package from typedoc output.
packages/parser-pipeline/package.json	Defines the new package, dependencies, and exports.
packages/parser-pipeline/index.js	Public entrypoint exporting types + parser constructors.
packages/parser-pipeline/src/composed-parser.js	Sync composed parser implementation.
packages/parser-pipeline/src/worker-parser.js	Async parser wrapper backed by worker pool.
packages/parser-pipeline/src/worker-pool.js	Worker thread pool and dispatch/queue/terminate logic.
packages/parser-pipeline/src/worker-runner.js	Worker-side message listener that runs the pipeline per task.
packages/parser-pipeline/src/types/external.ts	Public types for pipeline configuration and worker protocol.
packages/parser-pipeline/src/external.types.*	Type-export shim for JS consumers.
packages/parser-pipeline/test/*	AVA tests for composed parser, worker runner, and worker pool.
packages/parser-pipeline/README.md	New package documentation and usage examples.
packages/parser-pipeline/SECURITY.md	Package security policy doc.
packages/parser-pipeline/LICENSE	Apache-2.0 license file.
packages/parser-pipeline/tsconfig*.json	Typechecking/build config for the new package.
packages/compartment-mapper/src/types/external.ts	Splits sync vs async parser types and updates ParserForLanguage typing.
packages/compartment-mapper/src/types/internal.ts	Adjusts internal operator typings to accept ParseFn or AsyncParseFn.
packages/compartment-mapper/src/map-parser.js	Refactors parser selection/trampolines for sync+async support.
packages/compartment-mapper/src/import-hook.js	Adds a sync-parse type guard for importNow/dynamic-require constraints.
packages/compartment-mapper/src/link.js	Simplifies heuristicImports access with updated parser types.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.