diff --git a/docs/design/embed-discoverability.md b/docs/design/embed-discoverability.md new file mode 100644 index 0000000..0b0c4e8 --- /dev/null +++ b/docs/design/embed-discoverability.md @@ -0,0 +1,352 @@ +# Design note: embed discoverability, `{{query:...}}`, mermaid, and `rivet query` + +**Status:** Design only. No code changes in this note. +**Author:** dogfooding feedback (UX bug report, 2026-04) +**Scope:** Four small, independent gaps that showed up when a real user tried +to author documents and artifacts against a live project. + +The four gaps share one root cause: **the things that make the dashboard +valuable (embeds, s-expr queries, diagrams) are not surfaced through the CLI +or `rivet --help`.** You have to grep the source to find them. This note +proposes the minimum set of changes that makes them first-class without +re-architecting anything. + +Recommendations are listed in **priority order** at the end. + +--- + +## 1. Embed token discoverability — `rivet docs embeds` + +### Problem + +Today the canonical list of valid `{{...}}` tokens lives in two places: + +- Rust match arms in `rivet-core/src/embed.rs:162-181` (the `resolve_embed` + dispatcher) and the inline legacy embeds in + `rivet-core/src/document.rs:793-900` (for `{{artifact:...}}`, `{{links:...}}`, + `{{table:...}}`). +- A static markdown constant `EMBED_SYNTAX_DOC` in + `rivet-cli/src/docs.rs:1620-1700`, exposed only as + `rivet docs embed-syntax` — a slug you have to know. + +Neither shows up in `rivet --help`, `rivet docs --list`, or the dashboard's +Help view without knowing what to look for. The `rivet embed QUERY` subcommand +(`rivet-cli/src/main.rs:671-679`) takes a query but offers no way to list +the valid token names. + +### Proposal + +Add a new subcommand **`rivet docs embeds`** (or `rivet embeds list`) that +prints every registered embed token with its signature, a one-line +description, and a runnable example. The list must be **generated from the +same source of truth the resolver uses**, not a hand-maintained string. + +Concretely: + +- Introduce a small registry in `rivet-core/src/embed.rs` — a `const` + slice of `EmbedSpec { name, args, summary, example }` that + `resolve_embed` matches against instead of the current hard-coded + `match request.name.as_str()` (lines 162-181). Legacy inline embeds + (`artifact`, `links`, `table`) get registered as `is_legacy: true` so + they still dispatch via `document.rs` but appear in the listing. +- Expose `embed::registry()` as a public function returning + `&'static [EmbedSpec]`. +- In `rivet-cli/src/main.rs`, extend the `Docs` subcommand (line 410) or + add a sibling variant `Embeds { action: EmbedsAction }`. The latter is + cleaner because `rivet embed QUERY` already exists (line 672) — keeping + `rivet embeds list` parallel to `rivet schema list` (line 709) matches + the existing convention. +- In `rivet-cli/src/render/help.rs`, add a dashboard panel listing the + same registry, rendered into the existing Help view (`render_help` + around line 1-60) so the discoverability fix hits both CLI and web. + +### Output sketch + +``` +$ rivet embeds list +NAME ARGS SUMMARY +stats [section] Project statistics (types, status, validation) +coverage [rule] Traceability coverage bars +diagnostics [severity] Validation findings (error|warning|info) +matrix [from:to] Requirement↔feature (or any pair) matrix +artifact ID[:modifier[:depth]] Inline card for a single artifact +links ID Incoming+outgoing link table for an artifact +table TYPE:FIELDS Filtered artifact table +query SEXPR (proposed) Results of an s-expression filter + +Run `rivet docs embed-syntax` for full syntax reference. +Run `rivet embed [:args]` to render any of these from the CLI. +``` + +### Files touched (future PR, not this note) + +- `rivet-core/src/embed.rs:14-181` — add `EmbedSpec`, registry, make + `resolve_embed` table-driven. +- `rivet-cli/src/main.rs:410-430` — new subcommand wiring. +- `rivet-cli/src/docs.rs` — thin wrapper that prints the registry. +- `rivet-cli/src/render/help.rs:240-300` — dashboard listing. + +--- + +## 2. `{{query:}}` embed — MVP + +### Why this is the highest-value gap for dogfooding + +`{{table:TYPE:FIELDS}}` already exists but is **type-scoped only** — you +can't say "all requirements tagged `stpa` whose status is `approved`." That +query is trivial with the existing s-expression evaluator +(`rivet-core/src/sexpr_eval.rs`, already powering `rivet list --filter` +and MCP's `rivet_query`), but there is no way to embed the result in a +document today. + +Users are writing these queries in their heads and transcribing results +by hand. A `{{query:...}}` embed closes the loop. + +### MVP scope (and explicit non-scope) + +**In scope:** + +- **Read-only evaluation.** Reuses + `rivet_core::sexpr_eval::parse_filter` (already called from + `rivet-cli/src/mcp.rs:936`) and + `matches_filter_with_store` (line 945). No new evaluator. +- **Cacheable result.** Output depends only on the filter string plus + the current `Store` + `LinkGraph` hashes — the same inputs that drive + incremental validation. Hash those as the cache key; invalidate on any + store mutation. Piggy-back on the salsa/incremental layer + (`rivet-core/src/incremental.rs`, if present) or do a simple + `BTreeMap<(String, StoreHash), String>` cache keyed inside `EmbedContext`. +- **No side effects.** The embed cannot trigger validation, fetch + externals, or mutate anything. It is a pure function of + `(filter, store, graph) → HTML`. +- **Rendered as a compact table** with columns `id | type | title | + status`. Same look as `{{table:...}}` so users don't learn a new style. +- **Hard `limit` arg** with a safe default (50) and a max (500) to keep + render time bounded. Over-limit renders a "showing N of M — narrow your + filter" footer, never silently truncates without a hint. + +**Out of scope for MVP (deliberately):** + +- Custom column selection. Start with the fixed column set; add + `fields=...` later if needed. +- Sort order. MVP renders in store order. +- Cross-repo queries against externals. MVP is project-local. +- Reactive updates in the dashboard. MVP renders once per page request. +- Write/aggregate expressions (sum, count, group-by). Read-only only. + +### Syntax + +```markdown +{{query:(and (= type "requirement") (has-tag "stpa"))}} +{{query:(and (= type "hazard") (= status "approved")) limit=25}} +``` + +The parser in `rivet-core/src/embed.rs:108-147` already accepts +`key=val` options after a space, so `limit=25` drops in for free. The +s-expression itself is the first (and only) positional arg; it will +contain colons and parens, so the parser's `split(':')` at line 128 +needs a small adjustment: treat the whole tail after `query:` as the +s-expression, not as colon-separated args. + +### Files touched (future PR) + +- `rivet-core/src/embed.rs:162` — add `"query" => Ok(render_query(...))` + to the match. +- `rivet-core/src/embed.rs:108-147` — special-case `name == "query"` so + args aren't colon-split. +- `rivet-core/src/embed.rs` (new function `render_query`) — calls + `sexpr_eval::parse_filter`, iterates `ctx.store`, emits the same + table markup as `render_matrix` (lines 558+). +- `rivet-cli/src/docs.rs:1620` — add the new embed to + `EMBED_SYNTAX_DOC` (or, if we land recommendation #1 first, just add + it to the registry). + +### Security + +Same surface as `{{table:...}}` today: results go through `html_escape` +before insertion. The s-expression evaluator is pure and has no I/O — +confirmed by reading `rivet-core/src/sexpr_eval.rs`. No new attack +surface. + +--- + +## 3. Mermaid in artifact bodies — one-line fix in `markdown.rs` + +### Where the escape fails today + +There are **two markdown renderers** in the codebase, and they disagree: + +- `rivet-core/src/document.rs:371-401` — a hand-rolled line-by-line + renderer used for **document bodies** (`.md` files with + frontmatter). This one **does** handle fenced ` ```mermaid ` + blocks (line 388): it emits `
content
`, which + the dashboard's mermaid.js picks up via the selector + `mermaid.run({querySelector:'.mermaid'})` (see + `rivet-cli/src/serve/layout.rs:176,270,305`). +- `rivet-core/src/markdown.rs::render_markdown` (line 56-76) — a + pulldown-cmark-based renderer used for **artifact descriptions and + custom fields** (called from + `rivet-cli/src/render/artifacts.rs:283,397`). This one emits + pulldown-cmark's default output: `
...
`. + The `.mermaid` selector misses it, so the diagram renders as a + literal code block with `graph TD` text. + +Result: users who drop a mermaid diagram into a document body are +happy; users who drop the same snippet into `description:` see raw source. + +### Proposal + +Intercept fenced code blocks in `render_markdown` using +pulldown-cmark's event stream, and for `language-mermaid` emit the +same `
...
` tag the document renderer already +produces. This preserves the single-source dashboard JS glue — nothing +on the client side changes. + +### Concrete change (future PR) + +In `rivet-core/src/markdown.rs:56-76`, replace the current +`Parser::new_ext(...).filter(...)` pipeline with a small event-mapping +pass: + +```rust +// pseudocode — not to be taken as final +use pulldown_cmark::{CodeBlockKind, Event, Tag, TagEnd}; + +let mut in_mermaid = false; +let parser = Parser::new_ext(input, options) + .filter(|e| !matches!(e, Event::Html(_) | Event::InlineHtml(_))) + .map(|e| match e { + Event::Start(Tag::CodeBlock(CodeBlockKind::Fenced(ref lang))) + if lang.as_ref() == "mermaid" => + { + in_mermaid = true; + Event::Html(r#"
"#.into())
+        }
+        Event::End(TagEnd::CodeBlock) if in_mermaid => {
+            in_mermaid = false;
+            Event::Html("
".into()) + } + Event::Text(t) if in_mermaid => Event::Text(t), // passthrough + other => other, + }); +``` + +One subtlety: the current filter strips `Event::Html` events for XSS +defense. The mermaid branch needs to emit its wrapper *before* that +filter runs (or we special-case the two wrapper strings by bypassing +the filter only for them — but synthesising the events upstream of the +filter is cleaner). The `sanitize_html` post-pass on line 75 will +happily leave `
` alone because it targets
+`
 "#;
diff --git a/tests/playwright/filter-sort.spec.ts b/tests/playwright/filter-sort.spec.ts
index e2dcb49..e8cee50 100644
--- a/tests/playwright/filter-sort.spec.ts
+++ b/tests/playwright/filter-sort.spec.ts
@@ -212,4 +212,69 @@ test.describe("Artifacts Filter/Sort/Pagination", () => {
     );
     expect(realErrors).toEqual([]);
   });
+
+  // ── Search URL persistence (Fixes: REQ-007) ─────────────────────────────
+  //
+  // Regression tests for the bug where typing in the search input did not
+  // update the URL, so Cmd+R reload discarded the search term.
+  //
+  // Two surfaces to cover:
+  //   1. /artifacts filter-bar search input (HTMX-driven, hx-push-url="true")
+  //   2. Cmd+K global search overlay (JS-driven, history.replaceState)
+
+  test("typing in /artifacts search input updates the URL (hx-push-url)", async ({
+    page,
+  }) => {
+    await page.goto("/artifacts");
+    await waitForHtmx(page);
+
+    const searchInput = page.locator(
+      ".filter-bar input[name='q'], .filter-bar input[type='search']",
+    );
+    await expect(searchInput).toBeVisible();
+
+    // Verify the input is wired for URL push-on-type.
+    await expect(searchInput).toHaveAttribute("hx-push-url", "true");
+
+    // Type a query — HTMX fires a debounced hx-get and pushes the URL.
+    await searchInput.fill("OSLC");
+    await waitForHtmx(page);
+
+    // URL must now carry ?q=OSLC.
+    await expect(page).toHaveURL(/[?&]q=OSLC/);
+
+    // Reload the page: the search term survives.
+    await page.reload();
+    await waitForHtmx(page);
+    await expect(page).toHaveURL(/[?&]q=OSLC/);
+    await expect(searchInput).toHaveValue("OSLC");
+  });
+
+  test("typing in Cmd+K updates the URL with ?cmdk= (replaceState)", async ({
+    page,
+  }) => {
+    await page.goto("/artifacts");
+    await waitForHtmx(page);
+
+    // Open Cmd+K overlay.
+    const isMac = process.platform === "darwin";
+    await page.keyboard.press(isMac ? "Meta+k" : "Control+k");
+
+    const cmdKInput = page.locator("#cmd-k-input");
+    await expect(cmdKInput).toBeVisible();
+
+    await cmdKInput.fill("OSLC");
+    // Let the 200ms debounce fire.
+    await page.waitForTimeout(300);
+
+    // URL should have been updated in-place (no navigation) with ?cmdk=OSLC.
+    await expect(page).toHaveURL(/[?&]cmdk=OSLC/);
+
+    // Reload: the overlay re-opens pre-filled with the saved query.
+    await page.reload();
+    await waitForHtmx(page);
+    await expect(page).toHaveURL(/[?&]cmdk=OSLC/);
+    await expect(page.locator("#cmd-k-input")).toHaveValue("OSLC");
+    await expect(page.locator("#cmd-k-overlay")).toHaveClass(/open/);
+  });
 });