feat: add engine benchmark demo and fix Flare package name

[sauravpanda]

Summary

• Engine benchmark demo (examples/benchmark/): Standalone HTML page that compares MLC (WebGPU), Transformers.js (ONNX/WASM), and Flare (GGUF/WASM) engines side-by-side. Measures model load time, TTFT, and tokens/sec with configurable runs, warmup, and prompt. Includes comparison table and bar charts.
• Flare package rename: @aspect/flare → @sauravpanda/flare throughout the codebase. The package is published on npm as @sauravpanda/flare@0.2.0.

Test plan

• All 62 tests pass (including updated Flare test assertions)
• npm run build succeeds
• Benchmark page loads and runs at http://localhost:3456 via npx serve
• MLC, Transformers.js, and Flare engines all functional in benchmark

Closes #299

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added a browser-based benchmark tool allowing users to test and compare multiple local inference engines (MLC via WebGPU, Transformers.js via ONNX/WASM, Flare via WASM/GGUF) with configurable generation parameters, real-time progress tracking, and exportable results in JSON format.

• Chores

  • Updated Flare engine package integration and added serving configuration for benchmark deployment.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 827ef234-2d42-4bab-b409-4e2116553513

📥 Commits

Reviewing files that changed from the base of the PR and between 716b4cd and e24b981.

📒 Files selected for processing (4)

• examples/benchmark/index.html
• examples/benchmark/serve.json
• src/engines/flare-engine-wrapper.test.ts
• src/engines/flare-engine-wrapper.ts

---

📝 Walkthrough

Walkthrough

This PR adds a comprehensive browser-based benchmark UI at examples/benchmark/index.html for comparing three ML inference engines (MLC, Transformers.js, and Flare), plus supporting configuration. It also updates the Flare engine wrapper to reference the package under its new maintainer.

Changes

Cohort / File(s)	Summary
Benchmark UI & Config examples/benchmark/index.html, examples/benchmark/serve.json	New benchmark page with model/engine selection, real-time progress tracking, configurable generation parameters (prompt, max tokens, temperature, run counts), and results visualization with sortable tables and comparison charts. Includes CORS header configuration for cross-origin resource access.
Flare Engine Package Update src/engines/flare-engine-wrapper.ts, src/engines/flare-engine-wrapper.test.ts	Updated package references from @aspect/flare to @sauravpanda/flare in dynamic imports, error messages, and test assertions.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant UI as Benchmark UI
    participant EngMgr as Engine Manager
    participant MLC as MLC Engine
    participant TransJS as Transformers.js
    participant Flare as Flare Engine
    participant ModelCDN as Model CDN
    participant Results as Results Display

    User->>UI: Select model & engines
    User->>UI: Configure parameters
    User->>UI: Run benchmark

    UI->>EngMgr: Load selected engines
    par Engine Loading
        EngMgr->>MLC: Initialize WebGPU
        EngMgr->>TransJS: Initialize ONNX/WASM
        EngMgr->>Flare: Initialize WASM/GGUF
    end

    par Model Loading
        MLC->>ModelCDN: Fetch TVM artifact
        TransJS->>ModelCDN: Fetch ONNX model
        Flare->>ModelCDN: Fetch GGUF model
    end

    UI->>EngMgr: Execute warmup runs
    EngMgr->>MLC: Warmup inference
    EngMgr->>TransJS: Warmup inference
    EngMgr->>Flare: Warmup inference

    UI->>EngMgr: Run benchmark iterations
    loop Per Engine Per Run
        EngMgr->>MLC: Streaming inference (measure TTFT, tokens/sec)
        EngMgr->>TransJS: Streaming inference (measure TTFT, tokens/sec)
        EngMgr->>Flare: Streaming inference (measure TTFT, tokens/sec)
    end

    EngMgr->>UI: Aggregate metrics
    UI->>Results: Display comparison charts & tables
    User->>UI: Export results as JSON

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• feat: add Flare WASM inference engine integration #301: Directly modifies the Flare engine wrapper that is updated in this PR to reference the new package maintainer.
• chore: upgrade jest to v30, document Demucs and Flare in README #306: Updates Flare-related tests and package references alongside the changes in this PR.

Suggested labels

size/L

Poem

🐰 A benchmark born from code so bright,
Three engines racing through the night,
MLC, Transformers, Flare take flight—
Compare and measure with delight,
Let users choose what's truly right! ✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/benchmark-and-flare-package-rename

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}