diff --git a/.agents/skills/klicker-data-model/SKILL.md b/.agents/skills/klicker-data-model/SKILL.md index 689b6822f5..20d7838dc9 100644 --- a/.agents/skills/klicker-data-model/SKILL.md +++ b/.agents/skills/klicker-data-model/SKILL.md @@ -31,6 +31,14 @@ Provenance: steps 2 requires a database; on a machine without one running, write - **Decimal fields**: Python client needs `enable_experimental_decimal = true` in `apps/analytics/prisma/schema/py.prisma` (already set — don't remove); TS side never truthy-checks Decimals. - **Don't touch synced Analytics model files by hand** — `prisma:sync` overwrites them while preserving Analytics-owned `py.prisma` and `datasource.prisma`. - **Participant email uniqueness is per auth mode** (`@@unique([email, isSSOAccount])`) — cross-mode duplicate prevention lives in service logic, not the schema. +- **KB ingestion has two state axes** — `KBResource` holds the latest operation plus active serving identity; `KBIngestionRun` is append-only. Lecturer runs use the ingestion attempt/idempotency UUID; a signed `resource.content_refreshed` webhook uses its event UUID and persists the platform operation ID. A platform refresh may advance only serving identity and must not overwrite a current lecturer operation; resource-list operation status and filtering must resolve through the stored attempt, not the newest ledger row; a failed replacement must not erase the active version. +- **KB graph pointers are untrusted state** — `KB.publishedGraphBuildId` is a plain UUID rather than a relation. A reader must verify that the resolved `KBGraphBuild` belongs to the requested KB and is `SUCCEEDED` before using its graph name; retain old per-build graphs through a grace period and never sweep the active or published graph. +- **KB graph cost state is a locked ledger** — `KBGraphQuota` is unique per owner and semester, `KBGraphBuild.costStatus` fences reservation settlement by build id, and `dispatchClaimedAt` durably distinguishes an unattempted dispatch from an accepted-but-uncorrelated provider run. All monetary and persisted usage counters stay within PostgreSQL `INTEGER` bounds. Use the `ON CONFLICT DO NOTHING` insert followed by a row lock for concurrent first reservations; valid metered non-success terminal results settle actual usage without publishing, while invalid, mismatched, overflowed, or cleanup-fenced results become `NEEDS_HUMAN_REVIEW` and retain the reservation. A timed-out success may publish only after locked no-newer-build and current-digest reconciliation; stale or superseded late results settle without publication. +- **KB/chatbot activation is a database invariant** — `KBChatbot` may keep disabled history, but a SQL partial unique index permits only one enabled KB per chatbot. Prisma cannot express that index; preserve and verify it in the migration SQL. +- **KB deletion retains correlation state** — `KB`/`KBResource` tombstones hide owner reads immediately, while `KBIngestionOperation.DELETE` runs remain until external serving is empty and storage cleanup succeeds. Preserve the restrictive `KBUploadTicket` relation so pending uploads block parent hard deletion. +- **KB quotas count retained allocations** — include every resource row (including tombstones) plus every upload ticket for the 100-item limit, and sum resource plus ticket `sizeBytes` for the 500 MiB limit. Allocate under the parent-KB row lock; confirmation consumes a reservation rather than claiming quota again. An unknown-size URL row reserves the full 25 MiB source limit until the worker records its measured size. +- **KB scale operations preserve stable order** — resource cursors use immutable `createdAt` plus UUID, and bulk mutations lock the live parent before sorted resource ids. Do not make `createdAt` mutable or introduce a competing child-first lock order. +- **KB owner cascade is not a cleanup mechanism** — `KB.owner` currently uses `onDelete: Cascade`; deleting a User would remove resource/run correlation before external and Blob cleanup. Any future account-deletion or GDPR flow must complete the KB tombstone lifecycle before deleting the User. ## Seeds — two independent paths diff --git a/.agents/skills/klicker-environment-doctor/SKILL.md b/.agents/skills/klicker-environment-doctor/SKILL.md index d150701467..1de1b8b707 100644 --- a/.agents/skills/klicker-environment-doctor/SKILL.md +++ b/.agents/skills/klicker-environment-doctor/SKILL.md @@ -53,7 +53,9 @@ If `apps/analytics` complains about schema drift or a schema edit isn't visible: lsof -nP -iTCP:5432 -sTCP:LISTEN # repeat for 6379 6380 6381 7077 8888 80 443 ``` -`Bind for :::5432 failed: port is already allocated` means another stack holds the port — stop it or don't start the colliding service. Plain localhost and legacy host-based paths publish fixed ports, so only one such stack runs per machine. Parallel devcontainer worktrees use the port-free base compose file plus `.devcontainer/docker-compose.devrouter.yml`; the one-at-a-time fallback uses `.devcontainer/docker-compose.localhost.yml`. If manage media uploads fail with an Azure Blob CORS error while GraphQL auth still works, check the storage account before changing app CORS. The media library uploads directly from the browser to Azure Blob Storage via SAS, so its CORS rule must allow the actual local origin. Use exact origins for production/staging accounts and dev-only localhost rules for a dedicated dev storage account. +`Bind for :::5432 failed: port is already allocated` means another stack holds the port — stop it or don't start the colliding service. Plain localhost and legacy host-based paths publish fixed ports, so only one such stack runs per machine. Parallel devcontainer worktrees use the port-free base compose file plus `.devcontainer/docker-compose.devrouter.yml`; the one-at-a-time fallback uses `.devcontainer/docker-compose.localhost.yml`. + +The managed DevPod needs no Azure credentials for media or KB uploads: it starts a Blob-only Azurite service, routes `blob.klicker[.].localhost`, and configures the exact Manage origin as local Blob CORS during `post-start.sh`. `Blob storage is not configured` means the app process predates that environment; run `devrouter ensure .` and retry against the printed Blob route. A browser CORS failure with working GraphQL should first be reproduced as an `OPTIONS` request to that route using the exact Manage origin. Do not print the SAS query. Production and staging Azure accounts still require exact deployed origins; never add localhost CORS to them. ## Check 6 — infra bring-up / server status (headless-safe) @@ -69,7 +71,7 @@ devrouter exec . -- cat /tmp/devrouter-process-klicker-dev.state devrouter exec . -- tail -n 50 /tmp/dev.log ``` -`devrouter ensure` delivers its matching process helper to the exact validated container. Released `0.0.35` fingerprints the workspace, command, adapter bytes, and declared non-secret origin allowlist. The helper replaces a stale owned process group and leaves unknown processes untouched. Host-side ensure checks all routes and can recreate one stale or unhealthy exact-path DevPod once. +`devrouter ensure` delivers its matching process helper to the exact validated container. Released `0.0.35` fingerprints the workspace, command, adapter bytes, and declared non-secret origin allowlist. The helper replaces a stale owned process group and leaves unknown processes untouched. Host-side ensure checks all eleven routes, including Blob Storage, and can recreate one stale or unhealthy exact-path DevPod once. `devrouter doctor --repo .` provides static diagnostics. `devrouter ensure .` resolves the checkout-specific overlay and is the authoritative runtime proof. diff --git a/.agents/skills/klicker-frontend-ui/SKILL.md b/.agents/skills/klicker-frontend-ui/SKILL.md index 3bd488ec48..0b1e6af0f3 100644 --- a/.agents/skills/klicker-frontend-ui/SKILL.md +++ b/.agents/skills/klicker-frontend-ui/SKILL.md @@ -25,19 +25,33 @@ Conventions (design system, Tailwind v4, Apollo, i18n, CSP): [docs/frontend-conv services. - Forms: Formik + Yup. Conditional classes: `twMerge`. Feature flags gate alone — never `flag && count > 0`. - No Next.js middleware for CSP/headers — that belongs at the proxy layer. + - KB graph panel: show the per-KB opt-in and localized billing/quota states before the rebuild control. Format current estimates, maxima, and quota values with the current persisted quota currency, and historical settled build cost with its recorded currency; treat persisted quota currency/limit drift as unavailable. Keep rebuild disabled when opt-in, cost configuration, or active-build conditions fail; display actual cost and usage only after settlement; keep provider credentials out of the browser; and map billing enums to localized text. - Assessment comparison charts use equal-width categorical bars and a labelled 0–100 percentile ruler; keep the exact range/count table and highlight the student's containing range. 3. **Verify in the browser — mandatory, not optional.** Depending on your environment path: + - **Inside Devcontainer:** Dev servers auto-start in the background. No need to start/stop them. View logs via `tail -f /tmp/dev.log`. - **Host-based Setup:** You are authorized to start the dev servers needed for this verification, and must clean up after with `./_down.sh`. Bring-up per [docs/getting-started.md](../../../docs/getting-started.md) (localhost `dev:raw` path works without secrets). - On bring-up / server failure → `klicker-environment-doctor`. - Open the changed pages with `npx agent-browser` (never bare `agent-browser`), log in via **delegated** access with the AGENTS.md test credentials (not Edu-ID). - Capture before/after screenshots of every changed state (including error/empty states you touched); check both locales if strings changed. - Iterate on issues you see yourself; hand to the user for manual verification only after your own pass succeeds. + 4. **Pre-PR** — `klicker-testing-verification` checklist; attach the screenshots to the PR description. ## App boundaries - `frontend-manage` (lecturer), `frontend-pwa` (student; also has a localforage offline side-channel for live-quiz answers — don't bypass `storageHelpers.ts`), `frontend-control` (mobile controller), `auth` (login flows — auth changes also need [docs/auth-model.md](../../../docs/auth-model.md)). +- Knowledge-base management is a reusable package mounted by `frontend-manage`: edit `packages/kb-management`, not duplicate app-local components. Verify `/resources/knowledgeBases` plus the detail route at desktop and mobile widths, both locales, and every changed empty/active/success/failure state. +- The KB navigation item is an interim `user.privatePreview` discovery gate. Direct catalog/detail URLs must render the localized `KB_PREVIEW_ACCESS_REQUIRED` service error for a non-preview lecturer; never rely on hidden navigation as authorization. +- The knowledge-resource Ingest action accepts only the resource identifier. Do not expose transport tuning in the UI unless the GraphQL and ingestion-platform contracts add a real user-controlled setting. +- Keep full KB attempt history out of the two-second detail poll. Load the bounded, owner-checked history query only when a lecturer expands a resource, while the parent query carries only the latest run needed for operation status. +- Localize KB failure detail from stable status/error codes. Do not render raw ingestion-platform status text into the EN/DE lecturer UI. +- Replacing a chatbot's enabled KB requires an explicit warning state; verify attach, replace, detach, linked-KB, and no-KB states in both locales and at desktop/mobile widths. +- KB delete copy must distinguish immediate removal from background external/blob cleanup; never claim that asynchronous cleanup completed in the mutation success toast. +- KB uploads expose only PDF, TXT, and MD up to 25 MiB while the ingestion bridge supports PDF/plain text; map Markdown to `text/plain` and do not advertise DOCX/PPTX prematurely. Localize stable quota codes rather than raw service messages. +- KB catalog/detail scale uses server-backed search/filter connections, design-system `SelectField` filters, and explicit load-more controls. While one row is active, poll page zero plus known active pages every two seconds and run a full loaded-window walk every tenth tick; fall back immediately on cursor/page-length drift. Preserve the latest loaded window across action refreshes, use `no-cache` promise queries for background polls, and invalidate in-flight refreshes when filters change. Use indeterminate operation progress, not fabricated percentages. Keep selection bounded to 50 and remove rows from selection when they become active. Keep bulk deletion behind a named confirmation, and expose source, operation-versus-serving state, contextual actions, and lazy history in the keyboard-accessible inspector. +- Treat a KB mutation and its follow-up query refresh as separate outcomes. After mutation success, close/reset and show success even when a best-effort refresh fails; log the refresh failure without surfacing a mutation error or encouraging a duplicate retry. +- KB metrics must distinguish visible data from quota usage, reservations, pending cleanup, unknown-size conservative claims, and linked consumers. Verify these states in EN/DE at desktop and 390 px widths. - **`apps/chat` is out of scope here** — app router, zustand, assistant-ui; read [docs/chat-platform.md](../../../docs/chat-platform.md) and follow its local conventions instead. diff --git a/.agents/skills/klicker-graphql-api/SKILL.md b/.agents/skills/klicker-graphql-api/SKILL.md index 7d6b67147e..e158fe50df 100644 --- a/.agents/skills/klicker-graphql-api/SKILL.md +++ b/.agents/skills/klicker-graphql-api/SKILL.md @@ -11,7 +11,7 @@ Facts (auth ladder, layering, error conventions): [docs/graphql-api-layer.md](.. 1. **Service function** — `packages/graphql/src/services/.ts`. All logic, Prisma, Redis, pubSub here. Signature `(args, ctx: ContextWithUser) => …`. Errors: `GraphQLError` with `extensions.code` (grep `LIVE_QUIZ_PIN_INVALID` for the pattern) — not bare `Error`. 2. **Schema field** — `packages/graphql/src/schema/query.ts` / `mutation.ts` / `subscription.ts` (+ new object types in the area file). The resolver is a **one-liner** delegating to the service. -3. **Auth on the field** — copy the existing composition exactly (real shape from `deleteCourse` in `mutation.ts`; `withPermission` WRAPS the resolver): +3. **Auth on the field** — always declare the role with `t.withAuth(...)`. For shareable aggregates represented by `PermissionCheck`, copy the existing composition exactly (real shape from `deleteCourse` in `mutation.ts`; `withPermission` WRAPS the resolver): ```ts deleteCourse: t.withAuth(asUser).field({ @@ -30,7 +30,7 @@ Facts (auth ladder, layering, error conventions): [docs/graphql-api-layer.md](.. }) ``` - Participant-facing fields usually need only `t.withAuth(asParticipant)`. Note `withPermission` returns `null` on failure (client sees a null field, not an error) — don't "fix" that. + Participant-facing fields usually need only `t.withAuth(asParticipant)`. Note `withPermission` returns `null` on failure (client sees a null field, not an error) — don't "fix" that. Owner-only aggregates that have no `PermissionCheck` key, including `KB`, keep the role gate on the schema field and must resolve the persisted owner relation inside every service entry point. Do not invent a permission key or make an owner-only aggregate shareable only to reuse this wrapper. Multi-object batch fields are the deliberate exception: `withPermission` accepts one object selector and can only return one nullable field. Protect @@ -57,6 +57,24 @@ Facts (auth ladder, layering, error conventions): [docs/graphql-api-layer.md](.. 7. **Frontend wiring** — `import { Document } from '@klicker-uzh/graphql/dist/ops'`; `useQuery`/`useMutation` (+ `refetchQueries`) per [docs/frontend-conventions.md](../../../docs/frontend-conventions.md). 8. **Tests** — graphql vitest for service logic (`pnpm --filter @klicker-uzh/graphql test:local`; see the heavy pattern in `38c92d035`); route further via `klicker-testing-verification`. +Do not nest full history under a frequently polled parent list. The KB detail query loads only each resource's latest run; the separate owner-checked history query returns at most the five newest runs and is called on expansion. + +Every KB service query and mutation must start with `assertKbPreviewAccess(ctx)`, which reads the current `User.privatePreview` value instead of trusting a JWT claim. Apply the separate `assertKbIngestionEnabled()` kill switch only to upload-ticket issue, URL-resource creation, and Ingest/Retry/Re-ingest; reads, confirmation, deletion, and chatbot binding must stay available while ingestion is disabled. + +Knowledge-graph mutations add a distinct `KB_GRAPH_DISABLED` generation switch and require the persisted per-KB `knowledgeGraphEnabled` opt-in. Before dispatch, reserve the configured estimate in the owner-semester `KBGraphQuota`; recheck the complete reservation and linked quota identity at the worker effect boundary; claim `dispatchClaimedAt` before the provider call; and hold an accepted-but-uncorrelated run instead of retrying an ambiguous external effect. Keep its reservation and active KB build slot fenced until recovery, cancellation, settlement, or manual resolution, and refuse a rebuild mutation that would start a second external run. Expose cost and quota state without credentials. Provider status is not a GraphQL success proof: wire a versioned terminal result through `settleKbKnowledgeGraphResult`, validate build/KB/owner/run/digest/artifact/currency/bounded-counter/metering identity, settle valid metered non-success results without publication, and let `KBGraphBuild.costStatus` make settlement idempotent. The production backend and general worker explicitly pass `getKBGraphTerminalResult` and `settleKbKnowledgeGraphResult` into `prepareHatchetTasks`; both adapters are required for the supported runtime composition. A timed-out success requires locked no-newer-build and current-digest reconciliation before publication; stale or superseded late results settle without publication. The config query selects the newest graph attempt for lifecycle and cost fields, while it resolves `isStale` only from a verified successful published build, so a held or charged rebuild remains visible without changing the served pointer. Report persisted quota currency/limit drift as unavailable and keep historical build-cost currency separate from quota display. + +KB/chatbot attach and detach must lock both owner rows, replace the enabled link atomically, and reconcile only the `tutor` and `explainer` KB MCP configurations. Do not expose a configuration without the matching scoped-retrieval runtime support. + +KB child creation and deletion share a KB-first lock order. Upload-ticket issue, confirmation, URL creation, resource deletion, and whole-KB deletion must require a live parent under that lock. Deletion keeps hidden tombstones and queues the external operation after commit; queue failure must remain hidden and retryable. + +KB resource-count and byte quotas use that same parent lock. Reserve unknown-size URL resources at the full 25 MiB source limit until the worker records their measured size. Return stable GraphQL codes for quota and ticket mismatches, and derive every ingestion `kb_id` from the persisted owner-checked KB/resource relationship rather than client-supplied scope text. + +The source gateway is system-to-system, not caller-owner authorization: one `KB_SOURCE_GATEWAY_KEY` may fetch any owner's exact eligible BLOB resource. Keep the non-tombstoned BLOB/digest/QUEUED-or-PROCESSING database predicate ahead of Blob Storage access, derive the container from the persisted owner relation, and never describe the shared key as tenant-scoped. + +For scalable KB lists, use the existing owner/filter-bound opaque keyset connections: `(updatedAt, id)` for KBs and immutable `(createdAt, id)` for resources, both descending, bounded to 50. Do not reintroduce the former unbounded `getUserKbs` or nested `KB.resources` fields. Keep exact metrics derived with grouped queries, keep full run history in its separate five-row query, filter resources by their latest ingestion-run status, and reset cursors when normalized search/type/status filters change. + +Bulk resource deletion accepts at most 50 unique ids from one owned KB. Lock the parent then sorted children, reject the selection atomically for missing/foreign/active rows, create one fenced delete run per row before commit, and treat post-commit task dispatches independently. + For pagination changes, test both finite `take`/`skip` values and omitted values in the service, and verify that the generated operation variables and public schema make the arguments optional. Do not emulate an unbounded query diff --git a/.agents/skills/klicker-testing-verification/SKILL.md b/.agents/skills/klicker-testing-verification/SKILL.md index 5cbae7b144..26a89fbfed 100644 --- a/.agents/skills/klicker-testing-verification/SKILL.md +++ b/.agents/skills/klicker-testing-verification/SKILL.md @@ -19,6 +19,7 @@ Facts about the test landscape: [docs/testing.md](../../../docs/testing.md). Thi | UI or user flows | e2e — use `klicker-playwright-e2e` | | React component appearance/behavior only | there is **no component-test layer** — verify in the browser (below) and rely on e2e if a flow covers it | | Office Add-in source, build, or manifest | Run its `check`, `lint`, `test`, `build:docs`, `verify:docs`, and `validate` scripts; use a stubbed Office API for browser UI checks and sideload the manifest in PowerPoint before release | +| Prisma seed reconciliation | `pnpm --filter @klicker-uzh/prisma-data test` — Node test runner through the package's existing `tsx` toolchain | For the manage-list `All` page size, the focused browser evidence must cover the finite-to-All-to-50 state transition and explicit selection. A 200-record @@ -28,6 +29,20 @@ atomicity from it. Never run root `pnpm run test:run` blind — the graphql vitest config forces `pool: forks, singleFork: true` (serialized specs sharing DB state). +The focused `knowledge.test.ts`, `knowledgeIngestion.test.ts`, and `knowledgeWebhooks.test.ts` suites use real PostgreSQL but deliberately stub or avoid Hatchet, so they can verify owner-scoped binding replacement, MCP configuration, attempt-ledger, platform-refresh idempotency, current-attempt list projection, and serving-state transitions without a client token. Keep the full GraphQL suite on `test:local`. + +For KB deletion changes, add real-PostgreSQL coverage for owner-hidden tombstones and KB-first create/delete races, plus Hatchet unit coverage for the exact external delete request, operation fencing, empty-serving cutover, ticket expiry, blob-before-row ordering, and idempotent maintenance retry. + +For KB quota changes, use real PostgreSQL for exact 100-resource/500-MiB boundaries, concurrent reservations, ticket conversion, tombstone retention, and cleanup release. Use Hatchet tests for persisted KB-scope mismatch and locked URL replacement accounting (`usage - old size + observed size`) before dispatch. + +For KB pagination and bulk operations, use real PostgreSQL for tied keyset order, malformed/owner/filter-mismatched cursors, status changes between resource pages, exact grouped metrics, tombstone hiding, deterministic lock order, all-or-nothing active/foreign guards, input bounds, and independent post-commit dispatch failure. The KB UI has no component-test layer; use the real delegated-login browser for EN/DE desktop/390 px catalog and detail flows. + +For the interim KB gate, exercise every KB service entry point with a non-preview real-PostgreSQL user and verify `KB_PREVIEW_ACCESS_REQUIRED`. Toggle `KB_INGESTION_DISABLED` at call time and prove it blocks upload-ticket issue, URL creation, and ingestion while leaving deletion available. Browser proof covers preview/non-preview navigation plus direct-route denial in both locales. + +For KB file-upload browser proof, use the managed DevPod's routed Azurite service. Upload a synthetic PDF/TXT/MD fixture through the real hidden `[data-cy="kb-file-input"]`, then verify the resource row, exact size, and cleared upload reservation. The browser flow must cover ticket issue, Blob PUT, and confirmation; a CORS preflight may supplement it, but never print the SAS query. This proves local upload and confirmation only, not external ingestion acceptance. + +For maintenance recovery, prove a stale `QUEUED` UPSERT with no external operation id re-dispatches the same stored attempt id, young/in-flight/tombstoned/DELETE rows are excluded, and a repeated sweep creates no replacement run. Source-gateway coverage uses real PostgreSQL to prove the exact version, non-tombstoned BLOB, digest, and QUEUED/PROCESSING predicate before Blob access; direct resolver tests cover loopback/private/link-local/IPv6 rejection without external network. + For OpenAI-compatible chat stream changes, run `apps/chat/test/openai-chat-streaming.test.ts` before the full chat suite. The fixture uses injected OpenAI-compatible SSE with a sparse first tool-call index @@ -146,6 +161,8 @@ For a Prisma major or driver-adapter change, also run a frozen install; Prisma g The Office Add-in is a browser application bundled by Rollup. Its package check must use the workspace TypeScript version, `moduleResolution: Bundler`, `noEmit`, and explicit `types: ["office-js"]`. `build:docs` regenerates and replaces the deployable directory; `verify:docs` must then prove exact parity. Browser checks with an Office API stub verify the UI state machine only. Persistence, multi-instance behavior, and embedded evaluation rendering still require a real PowerPoint sideload. +For KB graph lifecycle changes, use real PostgreSQL for quota-lock and settlement tests, including a valid metered non-success result that settles without publishing, the dispatch-claim ambiguity hold, and matching/stale/newer-build late-success reconciliation; apply the current migration to a disposable database before the seam tests. Use pure tests for the W1 result/cost contracts, PostgreSQL integer bounds, and quota-configuration drift; and use Hatchet unit tests for provider-status reconciliation and complete reservation identity before dispatch. A provider `COMPLETED` response without a versioned result must be asserted as fail-closed; it is not evidence of publication or cost settlement. + ## Reporting State what you ran, what passed, what you did NOT run and why (e.g. "no Infisical access — e2e left to CI"). An honest gap beats a fabricated green. diff --git a/.devcontainer/README.md b/.devcontainer/README.md index 0a32820511..86315b08fd 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -3,7 +3,7 @@ Self-contained local environment for `klicker-uzh`. No Infisical/Doppler, no external EduID, no `/etc/hosts` edits — clone, route through devrouter, and run. The devcontainer owns the whole stack (toolchain, Postgres, 3× Redis, MailHog, -Hatchet, install + build + seed, `turbo dev`); +Azurite Blob Storage, Hatchet, install + build + seed, `turbo dev`); [devrouter](https://github.com/rschlaefli/devrouter) fronts it on a shared `:443` / `:5432`. Linked worktrees publish no host ports and can coexist; the primary checkout intentionally keeps fixed localhost ports and is @@ -38,6 +38,7 @@ The primary checkout keeps fixed localhost ports and receives stable unnamespace - Auth Service: `http://localhost:3010` - MailHog UI: `http://localhost:8025` - Hatchet Dashboard: `http://localhost:8888` + - Azurite Blob Storage: `http://localhost:10000/klickerdev` - Postgres DB: `localhost:5432` ### Mode 2: Linked checkout @@ -78,23 +79,38 @@ internal port. The linked-worktree overlay publishes no host ports and exposes stable unnamespaced aliases plus fixed localhost ports. `.devrouter.yml` uses the selected checkout identity in every proxy upstream. -| What | Host | Upstream (devnet) | -| ----------------- | ---------------------------------------------------- | ----------------------- | -| API (GraphQL) | `https://api.klicker..localhost` | `${WORKSPACE}-app:3000` | -| Auth | `https://auth.klicker..localhost` | `${WORKSPACE}-app:3010` | -| PWA (student) | `https://pwa.klicker..localhost` | `${WORKSPACE}-app:3001` | -| Manage (lecturer) | `https://manage.klicker..localhost` | `${WORKSPACE}-app:3002` | -| Control | `https://control.klicker..localhost` | `${WORKSPACE}-app:3003` | -| OLAT API | `https://olat-api.klicker..localhost` | `${WORKSPACE}-app:3030` | -| Response API | `https://response-api.klicker..localhost` | `${WORKSPACE}-app:7078` | -| LTI Service | `https://lti.klicker..localhost` | `${WORKSPACE}-app:4000` | -| Chat App | `https://chat.klicker..localhost` | `${WORKSPACE}-app:3004` | -| Postgres | `db.klicker..localhost:5432` | `${WORKSPACE}-db:5432` | +| What | Host | Upstream (devnet) | +| ----------------- | ---------------------------------------------------- | ---------------------------- | +| API (GraphQL) | `https://api.klicker..localhost` | `${WORKSPACE}-app:3000` | +| Auth | `https://auth.klicker..localhost` | `${WORKSPACE}-app:3010` | +| PWA (student) | `https://pwa.klicker..localhost` | `${WORKSPACE}-app:3001` | +| Manage (lecturer) | `https://manage.klicker..localhost` | `${WORKSPACE}-app:3002` | +| Control | `https://control.klicker..localhost` | `${WORKSPACE}-app:3003` | +| OLAT API | `https://olat-api.klicker..localhost` | `${WORKSPACE}-app:3030` | +| Response API | `https://response-api.klicker..localhost` | `${WORKSPACE}-app:7078` | +| Blob Storage | `https://blob.klicker..localhost` | `${WORKSPACE}-azurite:10000` | +| Graph Blob source | `http://127.0.0.1:10003` (host only) | `${WORKSPACE}-azurite:10000` | +| LTI Service | `https://lti.klicker..localhost` | `${WORKSPACE}-app:4000` | +| Chat App | `https://chat.klicker..localhost` | `${WORKSPACE}-app:3004` | +| Postgres | `db.klicker..localhost:5432` | `${WORKSPACE}-db:5432` | The two Hatchet workers (`hatchet-worker-general`, `hatchet-worker-response-processor`) also run in the `app` container but have **no port/route** — they consume the Hatchet event queue (responses pushed by `response-api`). +Azurite's browser-facing account URL is +`https://blob.klicker..localhost/klickerdev`. Server-side SDK calls +use the workspace-specific Azurite alias over HTTP; `post-start.sh` configures +the exact Manage origin as Blob CORS on that local emulator. This split keeps +browser uploads production-like without making Node trust the host's routed +certificate or requiring Azure credentials. + +The linked DevRouter overlay also maps the same Azurite container to loopback +port `10003` by default. `KB_GRAPH_BLOB_HOST_PORT` can override that host port; +the generated graph source URLs use `KB_GRAPH_BLOB_ACCOUNT_URL` and never +change the browser-facing upload URL. Cleartext graph source URLs are accepted +only for loopback and `.localhost` development hosts. + The lecturer and student MCP servers also run in the `app` container with **no route** — they listen on `localhost:7081` and `localhost:7080`, respectively, and `apps/chat` reaches them directly (in-container; see @@ -129,14 +145,15 @@ the hardcoded defaults only know `klicker.com`. ## What's inside -| Service | Image | Purpose | -| ----------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------- | -| `app` | local `Dockerfile` (Node 24 + pnpm 11.5.0) | runs every routed app plus the two Hatchet workers and both MCP servers | -| `postgres` | `postgres:15` | DB (klicker-prod + shadow/lti/qa/hatchet via init.sql) | -| `redis_exec`/`_assessment`/`_cache` | `redis:7` | live-quiz exec / assessment / cache + pub/sub | -| `mailhog` | `mailhog/mailhog` | dev SMTP sink | -| `hatchet` | `hatchet-lite-dev:v0.101.0` | workflow engine (gRPC :7077, no UI auth) | -| `litellm` | `ghcr.io/berriai/litellm-database:v1.88.1` | LLM proxy + complexity router for chat (port 4000 intra-net) | +| Service | Image | Purpose | +| ----------------------------------- | ------------------------------------------------ | ----------------------------------------------------------------------- | +| `app` | local `Dockerfile` (Node 24 + pnpm 11.5.0) | runs every routed app plus the two Hatchet workers and both MCP servers | +| `postgres` | `postgres:15` | DB (klicker-prod + shadow/lti/qa/hatchet via init.sql) | +| `redis_exec`/`_assessment`/`_cache` | `redis:7` | live-quiz exec / assessment / cache + pub/sub | +| `mailhog` | `mailhog/mailhog` | dev SMTP sink | +| `azurite` | `mcr.microsoft.com/azure-storage/azurite:3.36.0` | local Blob service for browser uploads | +| `hatchet` | `hatchet-lite-dev:v0.101.0` | workflow engine (gRPC :7077, no UI auth) | +| `litellm` | `ghcr.io/berriai/litellm-database:v1.96.2` | LLM proxy + Auto V2 complexity router for chat (port 4000 intra-net) | Environment lives in `devcontainer.env` (committed, dev-only). Lifecycle: `post-create.sh` (install + build packages + prisma reset/push/seed + token) then @@ -157,6 +174,72 @@ container-recreate budget. The image also carries uv `0.11.12` and selects Python 3.12, matching the analytics image and lint CI so the root quality gate runs inside the container. +## Local KB ingestion and graph builder + +The Klicker worker uses the producer-neutral `data-ingestion` resource API for +KB resource acceptance. It is not part of this DevPod compose project. Start +the sibling service on the host before clicking **Ingest**: + +```bash +DATA_INGESTION_REPO=/path/to/data-ingestion \ +KLICKER_KB_APP_ORIGIN=https://api.klicker..localhost \ + ./util/start-local-kb-ingestion.sh +``` + +This starts the real `modules/ingestion-api` service on +`http://127.0.0.1:18081` with an ignored SQLite state file and a local-only +Klicker producer registry. `KB_INGESTION_LOCAL_PORT` can override the +loopback-only host port. The DevPod reaches the default through +`http://host.docker.internal:18081`; the app's source-gateway URL is rewritten +to the namespaced API route so blob sources remain addressable by a host-side +worker. The API service accepts operations durably; running the separate +data-ingestion dispatcher/worker fleet is still required for downstream +fetching, embeddings, and vector-store activation. + +For the full local dispatcher/fetch-worker path, explicitly point the API and +worker fleet at the same PostgreSQL state store. SQLite remains the default for +API-only development: + +```bash +KB_INGESTION_STATE_BACKEND=postgres \ +KB_INGESTION_STATE_DSN=postgresql://@127.0.0.1:/hatchet \ +KB_INGESTION_STATE_SCHEMA=ingestion_state_local \ +DATA_INGESTION_REPO=/path/to/data-ingestion \ +KLICKER_KB_APP_ORIGIN=https://api.klicker..localhost \ + ./util/start-local-kb-ingestion.sh --foreground +``` + +The worker-side `INGESTION_STATE_DSN`, `INGESTION_STATE_SCHEMA`, and +`INGESTION_STATE_ENSURE_SCHEMA` must use the same values. Keep the source +gateway credential in the worker environment rather than a committed file. + +For the graph-builder boundary, start the canonical +`kg-content-generation/lightrag_research` local Hatchet/FalkorDB stack, then +write its local token into the ignored DevPod env file: + +```bash +KG_CONTENT_GENERATION_REPO=/path/to/kg-content-generation +( + cd "$KG_CONTENT_GENERATION_REPO" + FALKORDB_HOST_PORT=16379 \ + ./lightrag_research/scripts/hatchet/start_local_stack.sh +) + +KB_GRAPH_FALKORDB_HOST_PORT=16379 \ +KG_CONTENT_GENERATION_REPO="$KG_CONTENT_GENERATION_REPO" \ + ./util/configure-local-kb-graph-builder.sh +``` + +The alternate host port leaves DevRouter's Redis port untouched while the +host-side graph worker and DevPod use the same FalkorDB instance. Do not copy +model IDs from an older local branch; this script keeps the branch's current +model configuration. + +Restart or re-run `devrouter ensure ` after generating that file so +the app worker loads the local Hatchet and FalkorDB connection. No graph token +is committed. Without this optional file, the graph integration remains +disabled and the rest of the DevPod still starts normally. + ## Notes - `node_modules` is a named volume (pnpm hoists natives into the root diff --git a/.devcontainer/devcontainer.env b/.devcontainer/devcontainer.env index 955ed31ff8..a7e48c672d 100644 --- a/.devcontainer/devcontainer.env +++ b/.devcontainer/devcontainer.env @@ -18,6 +18,18 @@ # queries in dev — so they must run as development. (next dev forces this for the # Next apps anyway; this also covers the backend node process.) --- NODE_ENV=development +KB_SOURCE_GATEWAY_URL=http://localhost:3000 +KB_SOURCE_GATEWAY_KEY=dev-kb-source-gateway-key +KB_WEBHOOK_SECRET=dev-kb-webhook-secret +# Host-side graph worker source URL. Browser uploads continue to use the +# DevRouter HTTPS Blob URL; this endpoint reaches the same Azurite container +# through the loopback-only mapping in docker-compose.devrouter.yml. +KB_GRAPH_BLOB_ACCOUNT_URL=http://127.0.0.1:10003/klickerdev/ +# Local-only producer connection. Start the sibling data-ingestion resource API +# with util/start-local-kb-ingestion.sh before triggering a resource ingest. +KB_INGESTION_API_URL=http://host.docker.internal:18081 +KB_INGESTION_API_KEY=dev-local-kb-ingestion-api-key +KB_INGESTION_PROJECT_ID=klicker-course-materials # --- Database (postgres service) --- DATABASE_URL=postgres://klicker-prod:klicker@postgres:5432/klicker-prod @@ -118,6 +130,9 @@ LTI_DEV_MODE=true NEXT_PUBLIC_CHAT_URL=https://chat.klicker.localhost APP_ORIGIN_CHAT=https://chat.klicker.localhost OPENAI_BASE_URL=http://litellm:4000 +DOC_QUERY_SCOPE_ISSUER=https://chat.klicker.localhost +DOC_QUERY_SCOPE_AUDIENCE=klicker-doc-query-dev +DOC_QUERY_SCOPE_KID=dev-unconfigured # Lecturer assistant feature flags (the floating widget in Manage and its # lecturer MCP tools). No GrowthBook runs locally, so both are forced on here. diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index a60129fabf..8cd2a68ddd 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -18,6 +18,7 @@ "redis_assessment", "redis_cache", "mailhog", + "azurite", "hatchet", "litellm" ], diff --git a/.devcontainer/docker-compose.devrouter.yml b/.devcontainer/docker-compose.devrouter.yml index 31f4d2ccf6..d1009c77a1 100644 --- a/.devcontainer/docker-compose.devrouter.yml +++ b/.devcontainer/docker-compose.devrouter.yml @@ -7,9 +7,25 @@ services: aliases: - '${WORKSPACE:-klicker-uzh}-db' + azurite: + # Keep the browser-facing Blob route on DevRouter while exposing the same + # container to host-side graph workers through loopback only. + ports: + - '127.0.0.1:${KB_GRAPH_BLOB_HOST_PORT:-10003}:10000' + networks: + devnet: + aliases: + - '${WORKSPACE:-klicker-uzh}-azurite' + app: environment: DEVROUTER_WORKSPACE: '${DEVROUTER_WORKSPACE:-}' + BLOB_STORAGE_ACCOUNT_URL: 'https://blob.klicker.${WORKSPACE:-klicker-uzh}.localhost/klickerdev' + BLOB_STORAGE_INTERNAL_ACCOUNT_URL: 'http://${WORKSPACE:-klicker-uzh}-azurite:10000/klickerdev' + KB_GRAPH_BLOB_HOST_PORT: '${KB_GRAPH_BLOB_HOST_PORT:-10003}' + KB_GRAPH_BLOB_ACCOUNT_URL: 'http://127.0.0.1:${KB_GRAPH_BLOB_HOST_PORT:-10003}/klickerdev/' + KB_INGESTION_LOCAL_PORT: '${KB_INGESTION_LOCAL_PORT:-18081}' + KB_INGESTION_API_URL: 'http://host.docker.internal:${KB_INGESTION_LOCAL_PORT:-18081}' extra_hosts: - 'api.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' - 'auth.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' @@ -20,6 +36,7 @@ services: - 'response-api.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' - 'lti.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' - 'chat.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' + - 'blob.klicker.${WORKSPACE:-klicker-uzh}.localhost:host-gateway' networks: devnet: aliases: diff --git a/.devcontainer/docker-compose.localhost.yml b/.devcontainer/docker-compose.localhost.yml index 22fbf41ee0..4bf4322b0a 100644 --- a/.devcontainer/docker-compose.localhost.yml +++ b/.devcontainer/docker-compose.localhost.yml @@ -21,7 +21,20 @@ services: - '127.0.0.1:8888:8888' - '127.0.0.1:7077:7077' + azurite: + ports: + - '127.0.0.1:10000:10000' + networks: + devnet: + aliases: + - klicker-uzh-azurite + app: + environment: + BLOB_STORAGE_ACCOUNT_URL: 'https://blob.klicker.localhost/klickerdev' + BLOB_STORAGE_INTERNAL_ACCOUNT_URL: 'http://klicker-uzh-azurite:10000/klickerdev' + extra_hosts: + - 'blob.klicker.localhost:host-gateway' networks: devnet: aliases: diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index e45296c735..c36c679907 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -50,6 +50,35 @@ services: image: mailhog/mailhog:latest networks: [default] + # --- local Azure Blob Storage emulator for browser uploads --- + # The custom key decodes to the literal "test". It is deterministic, + # low-entropy, and local-only, not a cloud credential. Path-style URLs let + # one workspace-safe devrouter host serve the account without per-account DNS. + azurite: + image: mcr.microsoft.com/azure-storage/azurite:3.36.0 + command: + - azurite-blob + - --blobHost + - 0.0.0.0 + - --blobPort + - '10000' + - --location + - /data + - --disableProductStyleUrl + - --disableTelemetry + environment: + AZURITE_ACCOUNTS: 'klickerdev:dGVzdA==' + healthcheck: + test: ['CMD-SHELL', 'nc -z 127.0.0.1 10000'] + interval: 2s + timeout: 2s + retries: 30 + networks: + default: {} + devnet: {} + volumes: + - azurite_data:/data + # --- Hatchet workflow engine (gRPC :7077, http :8888). Reached at hatchet:7077 # by the app via compose DNS. SERVER_GRPC_BROADCAST_ADDRESS is set to the compose # name so the issued client token advertises a container-reachable endpoint. --- @@ -82,6 +111,10 @@ services: - devcontainer.env environment: - WORKSPACE=${WORKSPACE:-} + - BLOB_STORAGE_ACCOUNT_NAME=klickerdev + - BLOB_STORAGE_ACCESS_KEY=dGVzdA== + - BLOB_STORAGE_ACCOUNT_URL=http://localhost:10000/klickerdev + - BLOB_STORAGE_INTERNAL_ACCOUNT_URL=http://azurite:10000/klickerdev # Beta opt-in against a GrowthBook instance the developer supplies from # their own shell. The management key is write-capable, so it is passed # through from the host and never written into devcontainer.env, which is @@ -118,6 +151,8 @@ services: depends_on: postgres: condition: service_healthy + azurite: + condition: service_healthy # --- LiteLLM for Chat (Tier 3) --- # Pin an image tag that still exists on ghcr — BerriAI prunes old main-vX.Y.Z @@ -153,6 +188,7 @@ services: volumes: pgdata: + azurite_data: node_modules_root: hatchet_lite_config: diff --git a/.devcontainer/post-start.sh b/.devcontainer/post-start.sh index 31b6aa54c7..4831dad2c2 100755 --- a/.devcontainer/post-start.sh +++ b/.devcontainer/post-start.sh @@ -12,6 +12,11 @@ set -a . /workspaces/klicker-uzh/.devcontainer/devcontainer.env # shellcheck source=/dev/null [ -f /workspaces/klicker-uzh/.devcontainer/.hatchet.env ] && . /workspaces/klicker-uzh/.devcontainer/.hatchet.env +# Optional host-side graph-builder settings. The file is generated by +# util/configure-local-kb-graph-builder.sh and is intentionally gitignored +# because it contains the local Hatchet token. +# shellcheck source=/dev/null +[ -f /workspaces/klicker-uzh/.devcontainer/.local-kb-services.env ] && . /workspaces/klicker-uzh/.devcontainer/.local-kb-services.env set +a # Detect if devrouter routing is active (via mkcert CA mount) or fallback to plain localhost ports @@ -37,6 +42,11 @@ if [ ! -s /etc/devrouter/mkcert-rootCA.pem ]; then export NEXT_PUBLIC_ADD_RESPONSE_URL=http://localhost:7078 export NEXT_PUBLIC_CHAT_URL=http://localhost:3004 export CORS_ALLOWED_ORIGINS=http://localhost:3001 + export BLOB_STORAGE_ACCOUNT_URL=http://localhost:10000/klickerdev + export BLOB_STORAGE_INTERNAL_ACCOUNT_URL=http://azurite:10000/klickerdev + export KB_GRAPH_BLOB_ACCOUNT_URL=http://127.0.0.1:10000/klickerdev/ + export KB_INGESTION_API_URL=http://host.docker.internal:${KB_INGESTION_LOCAL_PORT:-18081} + export KB_SOURCE_GATEWAY_URL=http://localhost:3000 export NODE_EXTRA_CA_CERTS="" elif [ -n "${WORKSPACE:-}" ]; then echo "[post-start] Namespacing URLs for workspace: $WORKSPACE" @@ -65,15 +75,28 @@ elif [ -n "${WORKSPACE:-}" ]; then export APP_ORIGIN_LTI=https://lti.klicker.${WORKSPACE}.localhost export NEXT_PUBLIC_CHAT_URL=https://chat.klicker.${WORKSPACE}.localhost export APP_ORIGIN_CHAT=https://chat.klicker.${WORKSPACE}.localhost + export BLOB_STORAGE_ACCOUNT_URL=https://blob.klicker.${WORKSPACE}.localhost/klickerdev + export BLOB_STORAGE_INTERNAL_ACCOUNT_URL=http://${WORKSPACE}-azurite:10000/klickerdev + export KB_GRAPH_BLOB_ACCOUNT_URL=http://127.0.0.1:${KB_GRAPH_BLOB_HOST_PORT:-10003}/klickerdev/ + export KB_INGESTION_API_URL=http://host.docker.internal:${KB_INGESTION_LOCAL_PORT:-18081} + export KB_SOURCE_GATEWAY_URL=https://api.klicker.${WORKSPACE}.localhost +else + export KB_SOURCE_GATEWAY_URL=https://api.klicker.localhost + export BLOB_STORAGE_ACCOUNT_URL=https://blob.klicker.localhost/klickerdev + export BLOB_STORAGE_INTERNAL_ACCOUNT_URL=http://klicker-uzh-azurite:10000/klickerdev + export KB_GRAPH_BLOB_ACCOUNT_URL=http://127.0.0.1:10000/klickerdev/ + export KB_INGESTION_API_URL=http://host.docker.internal:${KB_INGESTION_LOCAL_PORT:-18081} fi # No-TTY pnpm hardening (see post-create.sh). (GOTCHAS #18) export CI=true export npm_config_verify_deps_before_run=false +pnpm --filter @klicker-uzh/graphql exec tsx src/scripts/setupLocalBlobStorage.ts + : "${DEVROUTER_PROCESS_HELPER:?Run devrouter ensure to start this managed application process.}" -export DEVROUTER_PROCESS_FINGERPRINT_ENV='APP_ORIGIN_API,APP_ORIGIN_AUTH,APP_ORIGIN_PWA,APP_ORIGIN_MANAGE,APP_ORIGIN_CONTROL,APP_ORIGIN_ASSESSMENT_API,APP_ORIGIN_ASSESSMENT_PWA,APP_ORIGIN_LTI,APP_ORIGIN_CHAT,APP_MANAGE_SUBDOMAIN,APP_STUDENT_SUBDOMAIN,APP_CONTROL_SUBDOMAIN,NEXTAUTH_URL,COOKIE_DOMAIN,NEXT_PUBLIC_API_URL,NEXT_PUBLIC_AUTH_URL,NEXT_PUBLIC_MANAGE_URL,NEXT_PUBLIC_PWA_URL,NEXT_PUBLIC_ASSESSMENT_URL,NEXT_PUBLIC_CONTROL_URL,NEXT_PUBLIC_ADD_RESPONSE_URL,NEXT_PUBLIC_CHAT_URL,CORS_ALLOWED_ORIGINS,AUTH_LECTURER_ALLOWED_HOSTS,AUTH_STUDENT_ALLOWED_HOSTS,NODE_EXTRA_CA_CERTS' +export DEVROUTER_PROCESS_FINGERPRINT_ENV='APP_ORIGIN_API,APP_ORIGIN_AUTH,APP_ORIGIN_PWA,APP_ORIGIN_MANAGE,APP_ORIGIN_CONTROL,APP_ORIGIN_ASSESSMENT_API,APP_ORIGIN_ASSESSMENT_PWA,APP_ORIGIN_LTI,APP_ORIGIN_CHAT,APP_MANAGE_SUBDOMAIN,APP_STUDENT_SUBDOMAIN,APP_CONTROL_SUBDOMAIN,NEXTAUTH_URL,COOKIE_DOMAIN,NEXT_PUBLIC_API_URL,NEXT_PUBLIC_AUTH_URL,NEXT_PUBLIC_MANAGE_URL,NEXT_PUBLIC_PWA_URL,NEXT_PUBLIC_ASSESSMENT_URL,NEXT_PUBLIC_CONTROL_URL,NEXT_PUBLIC_ADD_RESPONSE_URL,NEXT_PUBLIC_CHAT_URL,CORS_ALLOWED_ORIGINS,AUTH_LECTURER_ALLOWED_HOSTS,AUTH_STUDENT_ALLOWED_HOSTS,BLOB_STORAGE_ACCOUNT_URL,BLOB_STORAGE_INTERNAL_ACCOUNT_URL,KB_GRAPH_BLOB_ACCOUNT_URL,KB_SOURCE_GATEWAY_URL,KB_INGESTION_API_URL,KB_INGESTION_PROJECT_ID,KB_GRAPH_HATCHET_CLIENT_HOST_PORT,KB_GRAPH_HATCHET_API_URL,KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY,KB_GRAPH_HATCHET_WORKFLOW_NAME,KB_FALKORDB_HOST,KB_FALKORDB_PORT,KB_FALKORDB_TLS,NODE_EXTRA_CA_CERTS' # The test seed connects Benibot's Tutor and Explainer modes to this local, # read-only MCP fixture. Keep it in the app container so the seeded @@ -120,6 +143,7 @@ if [ -s /etc/devrouter/mkcert-rootCA.pem ]; then [post-start] Response API -> ${NEXT_PUBLIC_ADD_RESPONSE_URL} [post-start] LTI Service -> ${APP_ORIGIN_LTI} [post-start] Chat -> ${NEXT_PUBLIC_CHAT_URL} (requires UPSTREAM_OPENAI_API_KEY) +[post-start] Blob Storage -> ${BLOB_STORAGE_ACCOUNT_URL} (Azurite) [post-start] MCP fixture -> http://localhost:1417/mcp (Benibot Tutor/Explainer) [post-start] Workers -> hatchet-worker-general + -response-processor (no URL; consume hatchet queue) [post-start] Lecturer MCP -> http://localhost:7081/mcp (no route; chat reaches it in-container) @@ -139,6 +163,7 @@ else [post-start] Response API -> http://localhost:7078 [post-start] LTI Service -> http://localhost:4000 [post-start] Chat -> http://localhost:3004 (requires UPSTREAM_OPENAI_API_KEY) +[post-start] Blob Storage -> http://localhost:10000/klickerdev (Azurite) [post-start] MCP fixture -> http://localhost:1417/mcp (Benibot Tutor/Explainer) [post-start] Workers -> hatchet-worker-general + -response-processor (no URL; consume hatchet queue) [post-start] Lecturer MCP -> http://localhost:7081/mcp (no route; chat reaches it in-container) diff --git a/.devrouter.yml b/.devrouter.yml index f6387718ca..9a1a10ba93 100644 --- a/.devrouter.yml +++ b/.devrouter.yml @@ -68,6 +68,14 @@ apps: runtime: proxy upstream: ${WORKSPACE}-app:7078 + # Azurite Blob endpoint. The account name stays in the path so one + # workspace-namespaced host supports browser SAS uploads. + - name: blob + host: blob.klicker.localhost + protocol: http + runtime: proxy + upstream: ${WORKSPACE}-azurite:10000 + # Postgres: db.klicker.localhost:5432 -> `klicker-db` (SNI). Connect with # direct-SSL so the ClientHello carries the SNI (libpq 17+): psql # "host=db.klicker.localhost port=5432 user=klicker-prod password=klicker diff --git a/.github/workflows/test-chat.yml b/.github/workflows/test-chat.yml index f5f870ba04..c6ba0cd007 100644 --- a/.github/workflows/test-chat.yml +++ b/.github/workflows/test-chat.yml @@ -24,7 +24,7 @@ jobs: id: filter uses: ./.github/actions/changed-paths with: - pattern: '^(apps/chat/|packages/(i18n|prisma|graphql|types|grading|util|shared-components|markdown|next-config)/|package\.json|pnpm-lock\.yaml|pnpm-workspace\.yaml|tsconfig\.json|\.github/workflows/test-chat\.yml|\.github/actions/changed-paths/)' + pattern: '^(apps/chat/|packages/(i18n|prisma|graphql|knowledge-graph|types|grading|util|shared-components|markdown|next-config)/|package\.json|pnpm-lock\.yaml|pnpm-workspace\.yaml|tsconfig\.json|\.github/workflows/test-chat\.yml|\.github/actions/changed-paths/)' - name: Define node version if: steps.filter.outputs.should_run == 'true' diff --git a/.github/workflows/test-graphql.yml b/.github/workflows/test-graphql.yml index e4470fcffe..ac4e3abe98 100644 --- a/.github/workflows/test-graphql.yml +++ b/.github/workflows/test-graphql.yml @@ -22,7 +22,7 @@ jobs: id: check uses: ./.github/actions/changed-paths with: - pattern: '^(packages/graphql/|packages/prisma/|packages/grading/|packages/types/|packages/util/|packages/hatchet/|apps/hatchet-worker-general/|package\.json|pnpm-lock\.yaml|tsconfig\.json|\.github/workflows/test-graphql\.yml|\.github/actions/changed-paths/)' + pattern: '^(packages/graphql/|packages/prisma/|packages/grading/|packages/types/|packages/util/|packages/hatchet/|apps/(backend-docker|hatchet-worker-general)/|package\.json|pnpm-lock\.yaml|tsconfig\.json|\.github/workflows/test-graphql\.yml|\.github/actions/changed-paths/)' test-graphql: needs: filter @@ -141,12 +141,16 @@ jobs: pnpm run build cd ../util pnpm run build + cd ../knowledge-graph + pnpm run build cd ../graphql pnpm run build cd ../hatchet pnpm run build cd ../../apps/hatchet-worker-general pnpm run build + cd ../backend-docker + pnpm run build - name: Setup database run: | @@ -159,6 +163,8 @@ jobs: run: | set -euo pipefail + pnpm --filter @klicker-uzh/backend-docker test + . ./util/_create_hatchet_token_graphql.sh HATCHET_CLIENT_TOKEN=$(grep -m1 '^HATCHET_CLIENT_TOKEN=' packages/graphql/.env | cut -d'=' -f2-) diff --git a/.github/workflows/test-playwright.yml b/.github/workflows/test-playwright.yml index ac466dfefa..8dc2ad9705 100644 --- a/.github/workflows/test-playwright.yml +++ b/.github/workflows/test-playwright.yml @@ -140,6 +140,7 @@ jobs: packages/grading/dist packages/graphql/dist packages/hatchet/dist + packages/knowledge-graph/dist packages/markdown/dist packages/prisma/dist packages/types/dist diff --git a/.gitignore b/.gitignore index 422ddd866e..0e5e9c1ec3 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,7 @@ secrets*.yaml **/.DS_Store *.pem +.devcontainer/certs/ /apps/functions/local.settings.json @@ -67,6 +68,7 @@ output/ # Devcontainer: runtime-minted Hatchet client token (post-create writes it) .devcontainer/.hatchet.env +.devcontainer/.local-kb-services.env playwright/playwright-report/ playwright/test-results/ diff --git a/.gitleaks.toml b/.gitleaks.toml index 299ff9eaf6..42f82014fc 100644 --- a/.gitleaks.toml +++ b/.gitleaks.toml @@ -18,6 +18,9 @@ paths = [ '''apps/docs/docusaurus\.config\.ts''', # Self-host docker-compose example env — placeholder/template values only. '''deploy/compose-traefik-proxy/.*\.env$''', + # Devcontainer compose holds the Azurite emulator key (base64 of a dev-only + # placeholder, documented inline as non-cloud) — verified false positive. + '''\.devcontainer/docker-compose\.yml''', # Documentation-only fixture snippets with deliberately illustrative values. '''^\.agents/skills/playwright-best-practices/advanced/authentication-flows\.md$''', '''^project/plans_future/PLAN-k6-load-testing\.md$''', diff --git a/apps/analytics/prisma/schema/chat.prisma b/apps/analytics/prisma/schema/chat.prisma index 509aeafb17..1c4ffa76db 100644 --- a/apps/analytics/prisma/schema/chat.prisma +++ b/apps/analytics/prisma/schema/chat.prisma @@ -87,7 +87,7 @@ model ChatAttachment { type ChatAttachmentType position Int - imageBase64 String? @db.Text // base64 data URL for display + imageBase64 String? @db.Text // base64 data URL for display imagePreviewBase64 String? @db.Text // compact preview data URL for history rendering imageDescription String? @db.Text // AI-generated description for context injection @@ -132,6 +132,7 @@ model Chatbot { // Relations mcpConfigurations ChatbotMCPConfig[] + knowledgeBases KBChatbot[] threads ChatThread[] usageCredits ChatUsageCredits[] diff --git a/apps/analytics/prisma/schema/knowledge.prisma b/apps/analytics/prisma/schema/knowledge.prisma new file mode 100644 index 0000000000..f95f98dd20 --- /dev/null +++ b/apps/analytics/prisma/schema/knowledge.prisma @@ -0,0 +1,296 @@ +// ----- KNOWLEDGE BASES ----- +// #region + +enum KBResourceType { + BLOB // uploaded file stored in Azure Blob Storage + URL // external web resource (e.g., website, Kaltura MediaSpace video) fetched during ingestion +} + +enum KBResourceStatus { + ADDED // resource registered (blob uploaded or URL added), not yet queued for ingestion + QUEUED + PROCESSING + READY + FAILED +} + +enum KBIngestionStatus { + QUEUED + PROCESSING + SUCCEEDED + FAILED + SUPERSEDED +} + +enum KBIngestionOperation { + UPSERT + DELETE +} + +enum KBGraphBuildStatus { + QUEUED + PROCESSING + SUCCEEDED + FAILED // includes builds failed by the reconciliation timeout + SUPERSEDED +} + +enum KBGraphQualityTier { + STANDARD + HIGH +} + +enum KBGraphCostStatus { + RESERVED + SETTLED + RELEASED + NEEDS_HUMAN_REVIEW +} + +model KB { + id String @id @default(uuid()) @db.Uuid + name String + description String? + + owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade, onUpdate: Cascade) + ownerId String @db.Uuid + + deletedAt DateTime? + deletedBy User? @relation("KBDeletedBy", fields: [deletedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + deletedById String? @db.Uuid + + // at most one build may hold the slot at a time; claimed with a conditional update + activeGraphBuildId String? @db.Uuid + // the build FalkorDB currently serves, kept in place even once its digest goes stale + publishedGraphBuildId String? @db.Uuid + // graph generation is opt-in per KB; the lecturer-level preview flag is a separate gate + knowledgeGraphEnabled Boolean @default(false) + + resources KBResource[] + uploadTickets KBUploadTicket[] + chatbots KBChatbot[] + graphBuilds KBGraphBuild[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([ownerId]) + @@index([deletedAt]) +} + +model KBResource { + id String @id @default(uuid()) @db.Uuid + + type KBResourceType + title String + + // URL resources: location of the external web resource + sourceUrl String? + + // BLOB resources: upload metadata and blob location (resolver-enforced as required for type BLOB) + originalFilename String? + mimeType String? + sizeBytes Int? + blobName String? + blobHref String? + + status KBResourceStatus @default(ADDED) + statusMessage String? + ingestedAt DateTime? + ingestionAttemptId String? @db.Uuid + resourceVersion Int @default(0) + contentSha256 String? + externalOperationId String? + externalOperationStartedAt DateTime? + activeResourceVersion Int? + activeContentSha256 String? + errorCode String? + ingestionOperation KBIngestionOperation @default(UPSERT) + + deletedAt DateTime? + deletedBy User? @relation("KBResourceDeletedBy", fields: [deletedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + deletedById String? @db.Uuid + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + ingestionRuns KBIngestionRun[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([kbId, status]) + @@index([status]) + @@index([deletedAt]) +} + +model KBIngestionRun { + id String @id @db.Uuid + + operation KBIngestionOperation @default(UPSERT) + status KBIngestionStatus @default(QUEUED) + resourceVersion Int + contentSha256 String? + externalOperationId String? + statusMessage String? + errorCode String? + startedAt DateTime? + finishedAt DateTime? + + resource KBResource @relation(fields: [resourceId], references: [id], onDelete: Cascade, onUpdate: Cascade) + resourceId String @db.Uuid + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([resourceId, createdAt]) + @@index([status]) +} + +// Append-only ledger of knowledge graph build attempts, mirroring KBIngestionRun. +// The id is supplied by KlickerUZH and doubles as the idempotency key handed to the +// external generation service, which answers with its own externalOperationId. +model KBGraphBuild { + id String @id @db.Uuid + + status KBGraphBuildStatus @default(QUEUED) + qualityTier KBGraphQualityTier @default(STANDARD) + + // digest over the KB's active serving set at request time; pins what this build represents + sourceContentDigest String + // FalkorDB graph the completed build is written to, in one step at the end + graphName String + // GraphML export retained on Blob for versioning, once the build succeeds + graphmlBlobName String? + + // Cost reservation is part of the build ledger so settlement is idempotent by build id. + estimatedCostMinorUnits Int? + actualCostMinorUnits Int? + actualInputTokens Int? + actualOutputTokens Int? + actualEmbeddingTokens Int? + actualRequestCount Int? + costCurrency String? + costPricingVersion String? + costStatus KBGraphCostStatus? + /// [PrismaKBGraphMeteredCost] + meteredCost Json? + semesterKey String? + quota KBGraphQuota? @relation(fields: [quotaId], references: [id], onDelete: SetNull, onUpdate: Cascade) + quotaId String? @db.Uuid + + externalOperationId String? + externalStartedAt DateTime? + // Durable claim written before the provider call. If the call is accepted + // but its run id cannot be correlated, this prevents releasing the reserve. + dispatchClaimedAt DateTime? + statusMessage String? + errorCode String? + startedAt DateTime? + finishedAt DateTime? + // terminal builds keep their ledger row; maintenance records once their + // external FalkorDB graph has been retired. + // Set before external deletion so late external completions cannot publish + // while retention cleanup is in flight. Stale claims are reclaimable after + // the retention grace window. + cleanupStartedAt DateTime? + cleanedAt DateTime? + // The GraphML archive outlives the serving graph: it is purged on a separate, + // longer clock (the KB deletion recovery grace) so an earlier successful + // version stays restorable while the knowledge base exists. Tracked apart from + // `cleanedAt` because whole-KB hard deletion keys off graph retirement. + graphmlPurgedAt DateTime? + + // builds spend the requesting lecturer's AI budget, so the requester is recorded + requestedBy User? @relation(fields: [requestedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + requestedById String? @db.Uuid + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + // The graph must remain explainable even if a source resource is later + // deleted or replaced, so this is a build-local snapshot rather than a FK. + sources KBGraphBuildSource[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([kbId, createdAt]) + @@index([status]) + @@index([quotaId]) +} + +model KBGraphQuota { + id String @id @default(uuid()) @db.Uuid + + owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade, onUpdate: Cascade) + ownerId String @db.Uuid + + semesterKey String + currency String + limitMinorUnits Int + reservedMinorUnits Int @default(0) + settledMinorUnits Int @default(0) + graphBuilds KBGraphBuild[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([ownerId, semesterKey]) + @@index([ownerId]) +} + +model KBGraphBuildSource { + id String @id @default(uuid()) @db.Uuid + + resourceId String @db.Uuid + title String + type KBResourceType + contentSha256 String + sourceUrl String? + blobName String? + + build KBGraphBuild @relation(fields: [buildId], references: [id], onDelete: Cascade, onUpdate: Cascade) + buildId String @db.Uuid + + createdAt DateTime @default(now()) + + @@unique([buildId, resourceId]) + @@index([resourceId]) +} + +model KBUploadTicket { + id String @id @db.Uuid + + blobName String + sizeBytes Int @default(0) + expiresAt DateTime + + kb KB @relation(fields: [kbId], references: [id], onDelete: Restrict, onUpdate: Cascade) + kbId String @db.Uuid + + createdAt DateTime @default(now()) + + @@unique([kbId, blobName]) + @@index([expiresAt]) +} + +model KBChatbot { + id String @id @default(uuid()) @db.Uuid + + isEnabled Boolean @default(true) + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + chatbot Chatbot @relation(fields: [chatbotId], references: [id], onDelete: Cascade, onUpdate: Cascade) + chatbotId String @db.Uuid + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([kbId, chatbotId]) + @@index([chatbotId]) +} + +// #endregion diff --git a/apps/analytics/prisma/schema/user.prisma b/apps/analytics/prisma/schema/user.prisma index 12eb2ce605..1a84243f67 100644 --- a/apps/analytics/prisma/schema/user.prisma +++ b/apps/analytics/prisma/schema/user.prisma @@ -132,6 +132,11 @@ model User { answerCollections AnswerCollection[] chatbots Chatbot[] chatbotDisclaimers ChatbotDisclaimer[] + kbs KB[] + deletedKbs KB[] @relation("KBDeletedBy") + deletedKbResources KBResource[] @relation("KBResourceDeletedBy") + requestedKbGraphBuilds KBGraphBuild[] + kbGraphQuotas KBGraphQuota[] revokedVerificationRecords VerifiableCredential[] @relation("RevokedVerifiableCredentials") userGroups UserGroup[] @relation("UserGroupMembers") diff --git a/apps/backend-docker/.env.example b/apps/backend-docker/.env.example index 53452b40fa..d6a6c6d5eb 100644 --- a/apps/backend-docker/.env.example +++ b/apps/backend-docker/.env.example @@ -16,6 +16,9 @@ HATCHET_CLIENT_TOKEN=__HATCHET_CLIENT_TOKEN__ HATCHET_CLIENT_HOST_PORT=localhost:7077 HATCHET_CLIENT_TLS_STRATEGY=none HATCHET_LOG_LEVEL=DEBUG +KB_WEBHOOK_SECRET="dev-kb-webhook-secret" +KB_WEBHOOK_PREVIOUS_SECRET= +KB_SOURCE_GATEWAY_KEY="dev-kb-source-gateway-key" # Canonical app origins (prefer these; issuers fall back to them) APP_ORIGIN_AUTH="http://127.0.0.1:3010" @@ -26,3 +29,11 @@ APP_ORIGIN_MANAGE="http://127.0.0.1:3002" APP_ORIGIN_CONTROL="http://127.0.0.1:3003" APP_ORIGIN_ASSESSMENT_API="http://127.0.0.1:3000" APP_ORIGIN_ASSESSMENT_PWA="http://127.0.0.1:3001" + +# Example-only local FalkorDB settings. Replace them through your secret manager. +KB_FALKORDB_HOST=localhost +KB_FALKORDB_PORT=6379 +KB_FALKORDB_USERNAME=example-user +KB_FALKORDB_PASSWORD=example-password +KB_FALKORDB_TLS=false +KB_FALKORDB_QUERY_TIMEOUT_MS=5000 diff --git a/apps/backend-docker/.env.test b/apps/backend-docker/.env.test index 918e0d572d..4db909468f 100644 --- a/apps/backend-docker/.env.test +++ b/apps/backend-docker/.env.test @@ -13,6 +13,7 @@ HATCHET_CLIENT_TOKEN=__HATCHET_CLIENT_TOKEN__ HATCHET_CLIENT_HOST_PORT=localhost:7077 HATCHET_CLIENT_TLS_STRATEGY=none HATCHET_LOG_LEVEL=DEBUG +KB_WEBHOOK_SECRET="dev-kb-webhook-secret" # Canonical app origins (prefer these; issuers fall back to them) APP_ORIGIN_AUTH="http://127.0.0.1:3010" diff --git a/apps/backend-docker/package.json b/apps/backend-docker/package.json index d59ff43ab2..ba07a6453c 100644 --- a/apps/backend-docker/package.json +++ b/apps/backend-docker/package.json @@ -80,7 +80,8 @@ "script": "../../util/_run_with_infisical.sh --env dev tsx", "script:prod": "../../util/_run_with_infisical.sh --env prd tsx", "start": "node -r dotenv/config dist/index.js", - "start:test": "nyc --silent node -r dotenv/config dist/index.js" + "start:test": "nyc --silent node -r dotenv/config dist/index.js", + "test": "tsx --test test/*.test.ts" }, "engines": { "node": "=24" diff --git a/apps/backend-docker/src/app.ts b/apps/backend-docker/src/app.ts index a473dcffaf..cb01559cb2 100644 --- a/apps/backend-docker/src/app.ts +++ b/apps/backend-docker/src/app.ts @@ -10,6 +10,7 @@ import cors from 'cors' import express from 'express' import { createYoga } from 'graphql-yoga' import { createRequire } from 'node:module' +import { registerKBHttpRoutes } from './kbHttpRoutes.js' const require = createRequire(import.meta.url) const persistedOperations = require('@klicker-uzh/graphql/dist/server.json') @@ -106,6 +107,11 @@ function prepareApp({ next() } + // The ingestion bridge authenticates with its own gateway key and webhook + // signature. Register these routes before the end-user JWT middleware so a + // system bearer key is never interpreted as a Klicker session token. + registerKBHttpRoutes(app, { prisma }) + app.use(cookieParser()) app.use(jwtMiddleware) diff --git a/apps/backend-docker/src/index.ts b/apps/backend-docker/src/index.ts index 6b28076942..c3983a2d4b 100644 --- a/apps/backend-docker/src/index.ts +++ b/apps/backend-docker/src/index.ts @@ -1,11 +1,20 @@ import { createRedisEventTarget } from '@graphql-yoga/redis-event-target' -import { enhanceContext, handlers, schema } from '@klicker-uzh/graphql' +import { + enhanceContext, + handlers, + schema, + settleKbKnowledgeGraphResult, +} from '@klicker-uzh/graphql' import { prisma as prismaBase } from '@klicker-uzh/prisma' // import * as Sentry from '@sentry/node' // import '@sentry/tracing' import { createInMemoryCache, type Cache } from '@envelop/response-cache' import { createRedisCache } from '@envelop/response-cache-redis' -import { hatchetClient, prepareHatchetTasks } from '@klicker-uzh/hatchet' +import { + getKBGraphTerminalResult, + hatchetClient, + prepareHatchetTasks, +} from '@klicker-uzh/hatchet' import { useServer } from 'graphql-ws/lib/use/ws' import { createPubSub } from 'graphql-yoga' import { Redis } from 'ioredis' @@ -111,6 +120,18 @@ migrate(prisma).then(() => { redisExec, redisAssessmentExec, handlers, + getKBGraphTerminalResult, + settleKBGraphTerminalResult: ({ + buildId, + result, + finishedAt, + allowLateSuccess, + }) => + settleKbKnowledgeGraphResult( + prisma, + { buildId, result, allowLateSuccess }, + finishedAt + ), }) console.log('Hatchet tasks initialized.', Object.keys(tasks)) diff --git a/apps/backend-docker/src/kbHttpRoutes.ts b/apps/backend-docker/src/kbHttpRoutes.ts new file mode 100644 index 0000000000..1a1be87428 --- /dev/null +++ b/apps/backend-docker/src/kbHttpRoutes.ts @@ -0,0 +1,87 @@ +import { + handleKBIngestionWebhook, + handleKBSourceGateway, +} from '@klicker-uzh/graphql' +import express, { type Express } from 'express' + +const UUID_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + +type KBHttpRouteDependencies = { + prisma: Parameters[0]['prisma'] + sourceGateway?: typeof handleKBSourceGateway + ingestionWebhook?: typeof handleKBIngestionWebhook +} + +export function registerKBHttpRoutes( + app: Express, + { + prisma, + sourceGateway = handleKBSourceGateway, + ingestionWebhook = handleKBIngestionWebhook, + }: KBHttpRouteDependencies +) { + app.get( + '/api/ingestion/resources/:resourceId/versions/:resourceVersion', + async (req, res) => { + const resourceVersion = Number(req.params.resourceVersion) + if ( + !UUID_PATTERN.test(req.params.resourceId) || + !/^[1-9]\d*$/.test(req.params.resourceVersion) || + !Number.isSafeInteger(resourceVersion) + ) { + res.status(404).json({ error: 'Resource not found' }) + return + } + + try { + const result = await sourceGateway({ + prisma, + resourceId: req.params.resourceId, + resourceVersion, + authorization: req.headers.authorization, + }) + if (result.statusCode !== 200) { + res.status(result.statusCode).json(result.body) + return + } + + res.status(200) + res.set({ + 'Cache-Control': 'private, no-store', + 'Content-Length': String(result.contentLength), + 'X-Content-Type-Options': 'nosniff', + }) + res.setHeader('Content-Type', result.contentType) + result.stream.on('error', () => res.destroy()) + result.stream.pipe(res) + } catch { + console.error('KB source gateway failed') + res.status(500).json({ error: 'Internal server error' }) + } + } + ) + + app.post( + '/api/webhooks/kb-ingestion', + express.raw({ type: 'application/json', limit: '1mb' }), + async (req, res) => { + try { + if (!Buffer.isBuffer(req.body)) { + res.status(400).json({ error: 'Invalid request' }) + return + } + + const result = await ingestionWebhook({ + prisma, + rawBody: req.body, + headers: req.headers, + }) + res.status(result.statusCode).json(result.body) + } catch { + console.error('KB ingestion webhook failed') + res.status(500).json({ error: 'Internal server error' }) + } + } + ) +} diff --git a/apps/backend-docker/test/kbHttpRoutes.test.ts b/apps/backend-docker/test/kbHttpRoutes.test.ts new file mode 100644 index 0000000000..58ad2bb83d --- /dev/null +++ b/apps/backend-docker/test/kbHttpRoutes.test.ts @@ -0,0 +1,175 @@ +import { + handleKBIngestionWebhook, + handleKBSourceGateway, +} from '@klicker-uzh/graphql' +import assert from 'node:assert/strict' +import { once } from 'node:events' +import type { AddressInfo } from 'node:net' +import { Readable } from 'node:stream' +import test from 'node:test' +import express from 'express' +import { registerKBHttpRoutes } from '../src/kbHttpRoutes.js' + +const RESOURCE_ID = '4b3ff764-e876-4c57-952c-f26d70309714' + +async function withRoutes( + dependencies: Parameters[1], + run: (origin: string) => Promise +) { + const app = express() + registerKBHttpRoutes(app, dependencies) + const server = app.listen(0, '127.0.0.1') + await once(server, 'listening') + const address = server.address() as AddressInfo + + try { + await run(`http://127.0.0.1:${address.port}`) + } finally { + await new Promise((resolve, reject) => { + server.close((error) => (error ? reject(error) : resolve())) + }) + } +} + +function routeDependencies({ + sourceGateway, + ingestionWebhook, +}: { + sourceGateway?: typeof handleKBSourceGateway + ingestionWebhook?: typeof handleKBIngestionWebhook +} = {}): Parameters[1] { + return { + prisma: {} as never, + sourceGateway, + ingestionWebhook, + } +} + +test('rejects an invalid source route before calling the gateway', async () => { + let called = false + const sourceGateway = (async () => { + called = true + throw new Error('unexpected') + }) as typeof handleKBSourceGateway + + await withRoutes(routeDependencies({ sourceGateway }), async (origin) => { + const response = await fetch( + `${origin}/api/ingestion/resources/not-a-uuid/versions/0` + ) + + assert.equal(response.status, 404) + assert.deepEqual(await response.json(), { error: 'Resource not found' }) + assert.equal(called, false) + }) +}) + +test('streams an eligible source with defensive response headers', async () => { + let observedAuthorization: string | undefined + const sourceGateway = (async ({ authorization }) => { + observedAuthorization = authorization + return { + statusCode: 200, + contentLength: 7, + contentType: 'text/plain', + stream: Readable.from('content'), + } + }) as typeof handleKBSourceGateway + + await withRoutes(routeDependencies({ sourceGateway }), async (origin) => { + const response = await fetch( + `${origin}/api/ingestion/resources/${RESOURCE_ID}/versions/2`, + { headers: { Authorization: 'Bearer gateway-key' } } + ) + + assert.equal(response.status, 200) + assert.equal(await response.text(), 'content') + assert.equal(response.headers.get('cache-control'), 'private, no-store') + assert.equal(response.headers.get('content-length'), '7') + assert.equal(response.headers.get('content-type'), 'text/plain') + assert.equal(response.headers.get('x-content-type-options'), 'nosniff') + assert.equal(observedAuthorization, 'Bearer gateway-key') + }) +}) + +test('returns a generic error when the source gateway throws', async () => { + const sourceGateway = (async () => { + throw new Error('sensitive upstream detail') + }) as typeof handleKBSourceGateway + + await withRoutes(routeDependencies({ sourceGateway }), async (origin) => { + const response = await fetch( + `${origin}/api/ingestion/resources/${RESOURCE_ID}/versions/1` + ) + + assert.equal(response.status, 500) + assert.deepEqual(await response.json(), { + error: 'Internal server error', + }) + }) +}) + +test('forwards the exact raw webhook body and headers', async () => { + const rawBody = Buffer.from('{"event":"accepted"}') + let observedBody: Buffer | undefined + let observedHeader: string | string[] | undefined + const ingestionWebhook = (async ({ rawBody, headers }) => { + observedBody = rawBody + observedHeader = headers['x-ingestion-event-id'] + return { statusCode: 202, body: { accepted: true } } + }) as typeof handleKBIngestionWebhook + + await withRoutes(routeDependencies({ ingestionWebhook }), async (origin) => { + const response = await fetch(`${origin}/api/webhooks/kb-ingestion`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'X-Ingestion-Event-Id': RESOURCE_ID, + }, + body: rawBody, + }) + + assert.equal(response.status, 202) + assert.deepEqual(await response.json(), { accepted: true }) + assert.deepEqual(observedBody, rawBody) + assert.equal(observedHeader, RESOURCE_ID) + }) +}) + +test('rejects a webhook without an application/json raw body', async () => { + let called = false + const ingestionWebhook = (async () => { + called = true + throw new Error('unexpected') + }) as typeof handleKBIngestionWebhook + + await withRoutes(routeDependencies({ ingestionWebhook }), async (origin) => { + const response = await fetch(`${origin}/api/webhooks/kb-ingestion`, { + method: 'POST', + headers: { 'Content-Type': 'text/plain' }, + body: '{}', + }) + + assert.equal(response.status, 400) + assert.deepEqual(await response.json(), { error: 'Invalid request' }) + assert.equal(called, false) + }) +}) + +test('returns a generic error when webhook handling throws', async () => { + const ingestionWebhook = (async () => { + throw new Error('sensitive webhook detail') + }) as typeof handleKBIngestionWebhook + + await withRoutes(routeDependencies({ ingestionWebhook }), async (origin) => { + const response = await fetch(`${origin}/api/webhooks/kb-ingestion`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + + assert.equal(response.status, 500) + assert.deepEqual(await response.json(), { + error: 'Internal server error', + }) + }) +}) diff --git a/apps/chat/.env.local.example b/apps/chat/.env.local.example index 60db2f9370..1c57dcb7af 100644 --- a/apps/chat/.env.local.example +++ b/apps/chat/.env.local.example @@ -6,3 +6,17 @@ CHAT_OPENAI_STORE_RESPONSES=false # MCP Key MCP_KEY=sk-XXX + +# Dynamic knowledge-base retrieval. Keep the PKCS8 private key uncommitted. +DOC_QUERY_SCOPE_PRIVATE_KEY= +DOC_QUERY_SCOPE_KID= +DOC_QUERY_SCOPE_ISSUER= +DOC_QUERY_SCOPE_AUDIENCE= + +# Example-only local FalkorDB settings. Replace them through your secret manager. +KB_FALKORDB_HOST=localhost +KB_FALKORDB_PORT=6379 +KB_FALKORDB_USERNAME=example-user +KB_FALKORDB_PASSWORD=example-password +KB_FALKORDB_TLS=false +KB_FALKORDB_QUERY_TIMEOUT_MS=5000 diff --git a/apps/chat/next.config.ts b/apps/chat/next.config.ts index d81cc2f414..523d29c717 100644 --- a/apps/chat/next.config.ts +++ b/apps/chat/next.config.ts @@ -1,14 +1,37 @@ import { getNextBaseConfig } from '@klicker-uzh/next-config' import type { NextConfig } from 'next' import createNextIntlPlugin from 'next-intl/plugin' +import { fileURLToPath } from 'node:url' const withNextIntl = createNextIntlPlugin('./src/types/i18n.ts') - -const nextConfig = getNextBaseConfig({ +const repositoryRoot = fileURLToPath(new URL('../..', import.meta.url)) +const baseConfig = getNextBaseConfig({ BLOB_STORAGE_ACCOUNT_URL: process.env.BLOB_STORAGE_ACCOUNT_URL ?? '', includeI18n: false, - NODE_ENV: process.env.NODE_ENV, - NEXT_PUBLIC_ENV: process.env.NEXT_PUBLIC_ENV, -}) as NextConfig + NODE_ENV: process.env.NODE_ENV as string, + NEXT_PUBLIC_ENV: process.env.NEXT_PUBLIC_ENV as string, +}) + +// @ts-expect-error Next's config type does not expose all bundler options used here. +const nextConfig: NextConfig = { + ...baseConfig, + outputFileTracingRoot: repositoryRoot, + turbopack: { + root: repositoryRoot, + }, + serverExternalPackages: [ + '@klicker-uzh/knowledge-graph', + '@js-temporal/polyfill', + 'falkordb', + 'jsbi', + ], + webpack: (config, { isServer }) => { + if (baseConfig.webpack) { + config = baseConfig.webpack(config, { isServer } as any) + } + + return config + }, +} export default withNextIntl(nextConfig) diff --git a/apps/chat/package.json b/apps/chat/package.json index c6483192cc..25c5bf161f 100644 --- a/apps/chat/package.json +++ b/apps/chat/package.json @@ -14,6 +14,7 @@ "@fortawesome/react-fontawesome": "0.2.2", "@klicker-uzh/feature-flags": "workspace:*", "@klicker-uzh/i18n": "workspace:*", + "@klicker-uzh/knowledge-graph": "workspace:*", "@klicker-uzh/markdown": "workspace:*", "@klicker-uzh/next-config": "workspace:*", "@klicker-uzh/prisma": "workspace:*", diff --git a/apps/chat/src/app/[chatbotId]/graph/page.tsx b/apps/chat/src/app/[chatbotId]/graph/page.tsx new file mode 100644 index 0000000000..b2be165180 --- /dev/null +++ b/apps/chat/src/app/[chatbotId]/graph/page.tsx @@ -0,0 +1,3 @@ +export default function KnowledgeGraphPage() { + return null +} diff --git a/apps/chat/src/app/api/chatbots/[chatbotId]/chat/route.ts b/apps/chat/src/app/api/chatbots/[chatbotId]/chat/route.ts index 7b10b65f02..d6b3f4afa0 100644 --- a/apps/chat/src/app/api/chatbots/[chatbotId]/chat/route.ts +++ b/apps/chat/src/app/api/chatbots/[chatbotId]/chat/route.ts @@ -27,6 +27,7 @@ import { STUDENT_PRACTICE_QUIZ_TOOL_NAME, toPracticeCandidateId, } from '@/src/services/studentPracticeMcp' +import { resolveMcpScopeSessionId } from '@/src/services/mcpScope' import { ThreadService } from '@/src/services/threads' import { z } from 'zod' import { withChatbotAuth } from '@/src/lib/server/apiGuards' @@ -722,6 +723,7 @@ export async function POST( let systemPrompt = '' let mcpServersWithConfigs: MCPServerWithConfig[] = [] let chatbot = null + let enabledKnowledgeBaseId: string | undefined try { chatbot = await prisma.chatbot.findUnique({ @@ -736,10 +738,17 @@ export async function POST( }, orderBy: { priority: 'asc' }, }, + knowledgeBases: { + where: { isEnabled: true }, + select: { kbId: true }, + take: 1, + }, }, }) if (chatbot) { + enabledKnowledgeBaseId = chatbot.knowledgeBases[0]?.kbId + // Extract system prompt const systemPrompts = chatbot.systemPrompts as Record< string, @@ -811,16 +820,38 @@ export async function POST( }, })) + // The doc-query scope token carries this session id as its JWT subject. An + // existing id must belong to this participant and chatbot. For a new thread, + // allocate the eventual database id before loading MCP tools so every request + // for that thread uses one stable scope subject without persisting anything + // when a required MCP server is unavailable. + const pendingThreadId = currentThreadId ? undefined : randomUUID() + const scopeOwningThread = currentThreadId + ? await prisma.chatThread.findFirst({ + where: { id: currentThreadId, participantId, chatbotId }, + select: { id: true }, + }) + : null + const mcpScopeSessionId = resolveMcpScopeSessionId({ + requestedThreadId: currentThreadId, + owningThreadId: scopeOwningThread?.id, + fallbackId: pendingThreadId ?? requestId, + }) + if (mcpScopeSessionId === null) { + return NextResponse.json({ error: 'Thread not found' }, { status: 404 }) + } + // MCP availability is checked before creating a thread or doing any model, // credit, image-generation, or message-persistence work. let mcpTools: ToolSet try { - mcpTools = await getAggregatedMCPTools( - mcpServersWithConfigs, + mcpTools = await getAggregatedMCPTools(mcpServersWithConfigs, { chatbotId, participantId, - authMode - ) + authMode, + kbId: enabledKnowledgeBaseId, + sessionId: mcpScopeSessionId, + }) } catch (error) { if (error instanceof RequiredMCPUnavailableError) { return NextResponse.json( @@ -840,7 +871,8 @@ export async function POST( const newThread = await ThreadService.createThread( participantId, chatbotId, - null + null, + pendingThreadId ) currentThreadId = newThread.id } catch (error) { diff --git a/apps/chat/src/app/api/chatbots/[chatbotId]/knowledge-graph/route.ts b/apps/chat/src/app/api/chatbots/[chatbotId]/knowledge-graph/route.ts new file mode 100644 index 0000000000..fccc0fa64c --- /dev/null +++ b/apps/chat/src/app/api/chatbots/[chatbotId]/knowledge-graph/route.ts @@ -0,0 +1,98 @@ +import { withChatbotAuth } from '@/src/lib/server/apiGuards' +import { + type ChatbotKnowledgeGraphReadRequest, + isKnowledgeGraphNotPublishedError, + readPublishedChatbotKnowledgeGraph, +} from '@/src/lib/server/knowledgeGraph' +import { NextRequest, NextResponse } from 'next/server' +import { z } from 'zod' + +export const runtime = 'nodejs' + +const operationSchema = z.enum(['overview', 'search', 'neighbors']) +const searchQuerySchema = z.string().trim().min(1).max(100) +const nodeIdSchema = z.string().regex(/^\d+$/) + +function invalidRequestResponse() { + return NextResponse.json( + { + code: 'INVALID_KNOWLEDGE_GRAPH_REQUEST', + error: 'Invalid knowledge graph request', + }, + { status: 400 } + ) +} + +function parseReadRequest( + req: NextRequest +): ChatbotKnowledgeGraphReadRequest | null { + const operation = operationSchema.safeParse( + req.nextUrl.searchParams.get('operation') + ) + if (!operation.success) { + return null + } + + if (operation.data === 'search') { + const query = searchQuerySchema.safeParse(req.nextUrl.searchParams.get('q')) + return query.success ? { operation: 'search', query: query.data } : null + } + + if (operation.data === 'neighbors') { + const nodeId = nodeIdSchema.safeParse( + req.nextUrl.searchParams.get('nodeId') + ) + return nodeId.success + ? { operation: 'neighbors', nodeId: nodeId.data } + : null + } + + return { operation: 'overview' } +} + +export async function GET( + req: NextRequest, + { params }: { params: Promise<{ chatbotId: string }> } +) { + const { chatbotId } = await params + const authResult = await withChatbotAuth(req, chatbotId) + if ('response' in authResult) { + return authResult.response + } + + const readRequest = parseReadRequest(req) + if (readRequest === null) { + return invalidRequestResponse() + } + + try { + const response = await readPublishedChatbotKnowledgeGraph( + chatbotId, + readRequest + ) + return NextResponse.json(response) + } catch (error) { + if (isKnowledgeGraphNotPublishedError(error)) { + return NextResponse.json( + { + code: 'KNOWLEDGE_GRAPH_NOT_PUBLISHED', + error: 'Knowledge graph is not published', + publicationStatus: error.code, + }, + { status: 409 } + ) + } + + console.error('Participant knowledge graph read failed', { + chatbotId, + operation: readRequest.operation, + }) + return NextResponse.json( + { + code: 'KNOWLEDGE_GRAPH_TEMPORARILY_UNAVAILABLE', + error: 'Knowledge graph is temporarily unavailable', + }, + { status: 503 } + ) + } +} diff --git a/apps/chat/src/components/app-sidebar.tsx b/apps/chat/src/components/app-sidebar.tsx index 2d172bf265..f8b4545d2e 100644 --- a/apps/chat/src/components/app-sidebar.tsx +++ b/apps/chat/src/components/app-sidebar.tsx @@ -18,6 +18,7 @@ import Image from 'next/image' import Link from 'next/link' import { useParams, useRouter } from 'next/navigation' import * as React from 'react' +import { ChatGraphModeSwitch } from './knowledge-graph/ChatGraphModeSwitch' import { CreditsFooter } from './credits-footer' import { SettingsPanel } from './settings-panel' import { ThreadList } from './thread-list' @@ -96,6 +97,9 @@ export function AppSidebar({ ...props }: React.ComponentProps) { +
+ +

{t('chat.sidebar.conversationsLabel')}

diff --git a/apps/chat/src/components/assistant.tsx b/apps/chat/src/components/assistant.tsx index bf91661a98..ab30953030 100644 --- a/apps/chat/src/components/assistant.tsx +++ b/apps/chat/src/components/assistant.tsx @@ -9,7 +9,7 @@ import { import { Plus } from 'lucide-react' import Image from 'next/image' import Link from 'next/link' -import { useParams, useRouter } from 'next/navigation' +import { useParams, usePathname, useRouter } from 'next/navigation' import { useTranslations } from 'next-intl' import { useEffect, useState } from 'react' import { twMerge } from 'tailwind-merge' @@ -27,6 +27,8 @@ import { ChatUiProvider, useChatUi } from './chat-ui-context' import { DisclaimerModal } from './disclaimer-modal' import { MobileCreditsBar } from './credits-footer' import { EmbeddedCreditsBar, EmbeddedSettings } from './embedded-settings' +import { ChatGraphModeSwitch } from './knowledge-graph/ChatGraphModeSwitch' +import { ChatKnowledgeGraphWorkspace } from './knowledge-graph/ChatKnowledgeGraphWorkspace' import { ModeSwitcher } from './mode-switcher' import { Thread } from './thread' import { Tooltip, TooltipContent, TooltipTrigger } from './ui/tooltip' @@ -74,9 +76,11 @@ export function Assistant({ initialModeOptions, initialModeOptionsAreFallback, }: AssistantProps) { - const t = useTranslations() - // Stuff CHIPS fallback tokens into sessionStorage and strip them from the URL. + // Stuff `?_t=` (CHIPS-unsupported-browser fallback) into + // sessionStorage and strip it from the URL on first render. useChatGuestTokenBootstrap() + + const t = useTranslations() usePwaEmbedTokenBootstrap() const embedded = useEmbedded() const participationRequired = useChatStore( @@ -224,6 +228,7 @@ function useDisclaimerGate(chatbotId: string, participationRequired: boolean) { }), } ) + if (response.ok) { setDisclaimerStatus((prev) => ({ ...(prev ?? {}), @@ -260,6 +265,7 @@ function useDisclaimerGate(chatbotId: string, participationRequired: boolean) { }), } ) + if (response.ok) { setDisclaimerStatus((prev) => ({ ...(prev ?? {}), @@ -487,10 +493,12 @@ function ThreadSkeleton() { function SidebarMain({ chatbot, + graphMode, initialModeOptions, initialModeOptionsAreFallback, }: { chatbot: { id: string; name: string; avatar?: string } + graphMode: boolean initialModeOptions: Record initialModeOptionsAreFallback: boolean }) { @@ -566,21 +574,32 @@ function SidebarMain({ -
+
+ +
+
{isLoading && (
)} - + {graphMode ? ( + + ) : ( + + )}
-
+ ) } @@ -596,6 +615,8 @@ function AssistantLayout({ }) { const { showSidebar } = useChatUi() const isLoading = useChatStore((state) => state.isLoading) + const pathname = usePathname() + const graphMode = pathname === `/${chatbot.id}/graph` useEmbeddedChatContext() const context = useChatContextStore((state) => state.context) const contextLabel = getKlickerChatContextLabel(context) @@ -607,6 +628,7 @@ function AssistantLayout({ @@ -627,20 +649,27 @@ function AssistantLayout({ tabIndex={-1} className="flex min-h-0 flex-1 flex-col" > +
+ +
{isLoading && (
)} - + {graphMode ? ( + + ) : ( + + )}
diff --git a/apps/chat/src/components/knowledge-graph/ChatGraphModeSwitch.tsx b/apps/chat/src/components/knowledge-graph/ChatGraphModeSwitch.tsx new file mode 100644 index 0000000000..2274b71775 --- /dev/null +++ b/apps/chat/src/components/knowledge-graph/ChatGraphModeSwitch.tsx @@ -0,0 +1,70 @@ +'use client' + +import { MessagesSquare, Network } from 'lucide-react' +import Link from 'next/link' +import { usePathname, useSearchParams } from 'next/navigation' +import { twMerge } from 'tailwind-merge' + +export function ChatGraphModeSwitch({ + chatbotId, + className, + compact = false, +}: { + chatbotId: string + className?: string + compact?: boolean +}) { + const pathname = usePathname() + const searchParams = useSearchParams() + const embedValue = searchParams.get('embed') + const embedSuffix = + embedValue === 'true' || embedValue === '1' ? '?embed=true' : '' + const chatPath = `/${chatbotId}` + const graphPath = `/${chatbotId}/graph` + const isGraphMode = pathname === graphPath + + return ( + + ) +} diff --git a/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphViewer.tsx b/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphViewer.tsx new file mode 100644 index 0000000000..e1460dd551 --- /dev/null +++ b/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphViewer.tsx @@ -0,0 +1,37 @@ +'use client' + +import type { KnowledgeGraphDataSource } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import dynamic from 'next/dynamic' + +const DynamicKnowledgeGraphViewer = dynamic( + () => + import( + '@klicker-uzh/shared-components/src/knowledgeGraph/KnowledgeGraphViewer' + ).then((module) => module.KnowledgeGraphViewer), + { + ssr: false, + loading: () => ( +
+ Loading knowledge graph… +
+ ), + } +) + +export function ChatKnowledgeGraphViewer({ + dataSource, +}: { + dataSource: KnowledgeGraphDataSource +}) { + return ( + + ) +} diff --git a/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphWorkspace.tsx b/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphWorkspace.tsx new file mode 100644 index 0000000000..c687e175b7 --- /dev/null +++ b/apps/chat/src/components/knowledge-graph/ChatKnowledgeGraphWorkspace.tsx @@ -0,0 +1,230 @@ +'use client' + +import { authedFetch } from '@/src/lib/client/authedFetch' +import { useChatStore } from '@/src/stores/chatStore' +import type { KnowledgeGraphDataSource } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import { KnowledgeGraphUnavailableError } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import type { KnowledgeGraphResponse } from '@klicker-uzh/types' +import { useMemo } from 'react' +import { ChatKnowledgeGraphViewer } from './ChatKnowledgeGraphViewer' + +type KnowledgeGraphFetch = ( + input: RequestInfo | URL, + init?: RequestInit +) => Promise + +type PublicationStatus = 'EMPTY' | 'QUEUED' | 'PROCESSING' | 'FAILED' + +const PUBLICATION_STATUSES = new Set([ + 'EMPTY', + 'QUEUED', + 'PROCESSING', + 'FAILED', +]) + +type UnknownRecord = Record + +export class ChatKnowledgeGraphUnavailableError extends KnowledgeGraphUnavailableError { + readonly status = 409 + readonly publicationStatus?: PublicationStatus + + constructor(publicationStatus?: PublicationStatus) { + super('Knowledge graph is not published') + this.name = 'ChatKnowledgeGraphUnavailableError' + this.publicationStatus = publicationStatus + } +} + +export class ChatKnowledgeGraphRequestError extends Error { + readonly retryable: boolean + readonly status: number + + constructor(status: number, retryable: boolean) { + super( + retryable + ? 'Knowledge graph is temporarily unavailable' + : 'Knowledge graph request failed' + ) + this.name = 'ChatKnowledgeGraphRequestError' + this.retryable = retryable + this.status = status + } +} + +function knowledgeGraphUrl( + chatbotId: string, + operation: 'overview' | 'search' | 'neighbors', + input?: { key: 'q' | 'nodeId'; value: string } +): string { + const searchParams = new URLSearchParams({ operation }) + if (input !== undefined) { + searchParams.set(input.key, input.value) + } + return `/api/chatbots/${encodeURIComponent(chatbotId)}/knowledge-graph?${searchParams.toString()}` +} + +async function publicationStatus( + response: Response +): Promise { + try { + const body = (await response.json()) as { publicationStatus?: unknown } + return typeof body.publicationStatus === 'string' && + PUBLICATION_STATUSES.has(body.publicationStatus as PublicationStatus) + ? (body.publicationStatus as PublicationStatus) + : undefined + } catch { + return undefined + } +} + +async function readKnowledgeGraphResponse( + url: string, + fetcher: KnowledgeGraphFetch +): Promise { + const response = await fetcher(url) + if (response.status === 409) { + throw new ChatKnowledgeGraphUnavailableError( + await publicationStatus(response) + ) + } + if (response.status === 403) { + useChatStore.getState().setParticipationRequired(true) + throw new ChatKnowledgeGraphRequestError(403, false) + } + if (!response.ok) { + throw new ChatKnowledgeGraphRequestError( + response.status, + response.status === 503 + ) + } + + let body: unknown + try { + body = await response.json() + } catch { + throw new ChatKnowledgeGraphRequestError(502, false) + } + if (!isKnowledgeGraphResponse(body)) { + throw new ChatKnowledgeGraphRequestError(502, false) + } + return body +} + +function isRecord(value: unknown): value is UnknownRecord { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +function isOptionalString(value: unknown): boolean { + return value === undefined || typeof value === 'string' +} + +function isSourceReference(value: unknown): boolean { + return ( + isRecord(value) && + typeof value.resourceId === 'string' && + typeof value.title === 'string' && + isOptionalString(value.reference) + ) +} + +function isKnowledgeGraphNode(value: unknown): boolean { + return ( + isRecord(value) && + typeof value.id === 'string' && + Array.isArray(value.labels) && + value.labels.every((label) => typeof label === 'string') && + typeof value.kind === 'string' && + typeof value.displayLabel === 'string' && + isOptionalString(value.summary) && + isOptionalString(value.content) && + typeof value.degree === 'number' && + Number.isSafeInteger(value.degree) && + value.degree >= 0 && + Array.isArray(value.sourceReferences) && + value.sourceReferences.every(isSourceReference) + ) +} + +function isEdgeProperty(value: unknown): boolean { + return ( + typeof value === 'string' || + typeof value === 'boolean' || + (typeof value === 'number' && Number.isFinite(value)) + ) +} + +function isKnowledgeGraphEdge(value: unknown): boolean { + return ( + isRecord(value) && + typeof value.id === 'string' && + typeof value.source === 'string' && + typeof value.target === 'string' && + typeof value.type === 'string' && + typeof value.label === 'string' && + isRecord(value.properties) && + Object.values(value.properties).every(isEdgeProperty) + ) +} + +function isKnowledgeGraphResponse( + value: unknown +): value is KnowledgeGraphResponse { + return ( + isRecord(value) && + typeof value.kbId === 'string' && + typeof value.buildId === 'string' && + typeof value.isStale === 'boolean' && + Array.isArray(value.nodes) && + value.nodes.every(isKnowledgeGraphNode) && + Array.isArray(value.edges) && + value.edges.every(isKnowledgeGraphEdge) && + typeof value.truncated === 'boolean' + ) +} + +export function createChatKnowledgeGraphDataSource( + chatbotId: string, + fetcher: KnowledgeGraphFetch = authedFetch +): KnowledgeGraphDataSource { + return { + overview: () => + readKnowledgeGraphResponse( + knowledgeGraphUrl(chatbotId, 'overview'), + fetcher + ), + search: (query) => + readKnowledgeGraphResponse( + knowledgeGraphUrl(chatbotId, 'search', { key: 'q', value: query }), + fetcher + ), + neighbors: (nodeId) => + readKnowledgeGraphResponse( + knowledgeGraphUrl(chatbotId, 'neighbors', { + key: 'nodeId', + value: nodeId, + }), + fetcher + ), + } +} + +export function ChatKnowledgeGraphWorkspace({ + chatbotId, +}: { + chatbotId: string +}) { + const dataSource = useMemo( + () => createChatKnowledgeGraphDataSource(chatbotId), + [chatbotId] + ) + + return ( +
+ +
+ ) +} diff --git a/apps/chat/src/components/student-practice-quiz-card.tsx b/apps/chat/src/components/student-practice-quiz-card.tsx index 7b55eb4e63..4f475289f7 100644 --- a/apps/chat/src/components/student-practice-quiz-card.tsx +++ b/apps/chat/src/components/student-practice-quiz-card.tsx @@ -309,6 +309,14 @@ export function StudentPracticeQuizCard({ return } + const submittedAtMs = Date.now() + const stackAnswerTimeSeconds = Math.max( + 0, + Math.round( + (submittedAtMs - (startedAtMs.current ?? submittedAtMs)) / 1000 + ) + ) + setIsSubmitting(true) try { const response = await fetch( @@ -317,12 +325,7 @@ export function StudentPracticeQuizCard({ body: JSON.stringify({ questionRef: activeQuiz.questionRef, responses: built.responses, - stackAnswerTimeSeconds: Math.max( - 0, - Math.round( - (Date.now() - (startedAtMs.current ?? Date.now())) / 1000 - ) - ), + stackAnswerTimeSeconds, }), headers: { 'Content-Type': 'application/json' }, method: 'POST', diff --git a/apps/chat/src/lib/server/docQueryScopeToken.ts b/apps/chat/src/lib/server/docQueryScopeToken.ts new file mode 100644 index 0000000000..a083378c7c --- /dev/null +++ b/apps/chat/src/lib/server/docQueryScopeToken.ts @@ -0,0 +1,64 @@ +import { importPKCS8, SignJWT } from 'jose' + +const SCOPE_TOKEN_ALGORITHM = 'ES256' +const SCOPE_TOKEN_TTL_SECONDS = 5 * 60 + +export class DocQueryScopeTokenError extends Error { + constructor(message: string) { + super(message) + this.name = 'DocQueryScopeTokenError' + } +} + +function requireScopeTokenEnv(name: string): string { + const value = process.env[name]?.trim() + if (!value) { + throw new DocQueryScopeTokenError(`${name} is not configured`) + } + return value +} + +export async function signDocQueryScopeToken({ + kbId, + chatbotId, + sessionId, + jti, +}: { + kbId: string + chatbotId: string + sessionId: string + jti: string +}): Promise { + const privateKeyPem = requireScopeTokenEnv( + 'DOC_QUERY_SCOPE_PRIVATE_KEY' + ).replaceAll('\\n', '\n') + const kid = requireScopeTokenEnv('DOC_QUERY_SCOPE_KID') + const issuer = requireScopeTokenEnv('DOC_QUERY_SCOPE_ISSUER') + const audience = requireScopeTokenEnv('DOC_QUERY_SCOPE_AUDIENCE') + + try { + const privateKey = await importPKCS8(privateKeyPem, SCOPE_TOKEN_ALGORITHM) + + return await new SignJWT({ + kb_id: kbId, + chatbot_id: chatbotId, + }) + .setProtectedHeader({ + alg: SCOPE_TOKEN_ALGORITHM, + typ: 'JWT', + kid, + }) + .setIssuer(issuer) + .setAudience(audience) + .setSubject(sessionId) + .setJti(jti) + .setIssuedAt() + .setExpirationTime(`${SCOPE_TOKEN_TTL_SECONDS}s`) + .sign(privateKey) + } catch (error) { + if (error instanceof DocQueryScopeTokenError) { + throw error + } + throw new DocQueryScopeTokenError('Scope token signing failed') + } +} diff --git a/apps/chat/src/lib/server/knowledgeGraph.ts b/apps/chat/src/lib/server/knowledgeGraph.ts new file mode 100644 index 0000000000..e71717484d --- /dev/null +++ b/apps/chat/src/lib/server/knowledgeGraph.ts @@ -0,0 +1,86 @@ +import { prisma } from '@klicker-uzh/prisma' +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphResponse, + KnowledgeGraphSourceReference, +} from '@klicker-uzh/types' +import { + getPublishedKnowledgeGraphForChatbot, + readKnowledgeGraphNeighbors, + readKnowledgeGraphOverview, + searchKnowledgeGraph, +} from './knowledgeGraphRuntime' + +export { isKnowledgeGraphNotPublishedError } from './knowledgeGraphRuntime' + +export type ChatbotKnowledgeGraphReadRequest = + | { operation: 'overview' } + | { operation: 'search'; query: string } + | { operation: 'neighbors'; nodeId: string } + +function browserSafeSourceReference( + source: KnowledgeGraphSourceReference +): KnowledgeGraphSourceReference { + return { + resourceId: source.resourceId, + title: source.title, + ...(source.reference === undefined ? {} : { reference: source.reference }), + } +} + +function browserSafeNode(node: KnowledgeGraphNode): KnowledgeGraphNode { + return { + id: node.id, + labels: node.labels, + kind: node.kind, + displayLabel: node.displayLabel, + ...(node.summary === undefined ? {} : { summary: node.summary }), + ...(node.content === undefined ? {} : { content: node.content }), + degree: node.degree, + sourceReferences: node.sourceReferences.map(browserSafeSourceReference), + } +} + +function browserSafeEdge(edge: KnowledgeGraphEdge): KnowledgeGraphEdge { + return { + id: edge.id, + source: edge.source, + target: edge.target, + type: edge.type, + label: edge.label, + properties: edge.properties, + } +} + +function browserSafeResponse( + response: KnowledgeGraphResponse +): KnowledgeGraphResponse { + return { + kbId: response.kbId, + buildId: response.buildId, + isStale: response.isStale, + nodes: response.nodes.map(browserSafeNode), + edges: response.edges.map(browserSafeEdge), + truncated: response.truncated, + } +} + +export async function readPublishedChatbotKnowledgeGraph( + chatbotId: string, + request: ChatbotKnowledgeGraphReadRequest +): Promise { + const publication = await getPublishedKnowledgeGraphForChatbot( + prisma, + chatbotId + ) + + const response = + request.operation === 'overview' + ? await readKnowledgeGraphOverview(publication) + : request.operation === 'search' + ? await searchKnowledgeGraph(publication, request.query) + : await readKnowledgeGraphNeighbors(publication, request.nodeId) + + return browserSafeResponse(response) +} diff --git a/apps/chat/src/lib/server/knowledgeGraphRuntime.ts b/apps/chat/src/lib/server/knowledgeGraphRuntime.ts new file mode 100644 index 0000000000..e3a32d3b3e --- /dev/null +++ b/apps/chat/src/lib/server/knowledgeGraphRuntime.ts @@ -0,0 +1,121 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' +import type { KnowledgeGraphResponse } from '@klicker-uzh/types' +import { createRequire } from 'node:module' + +export type PublishedKnowledgeGraph = { + kbId: string + buildId: string + graphName: string + isStale: boolean + sources: { resourceId: string; title: string }[] +} + +type KnowledgeGraphPublicationCode = + | 'EMPTY' + | 'QUEUED' + | 'PROCESSING' + | 'FAILED' + +type KnowledgeGraphModule = { + KnowledgeGraphNotPublishedError: new ( + code: KnowledgeGraphPublicationCode + ) => Error & { readonly code: KnowledgeGraphPublicationCode } + getPublishedKnowledgeGraph: ( + client: PrismaClient, + kbId: string + ) => Promise + readKnowledgeGraphOverview: ( + context: PublishedKnowledgeGraph + ) => Promise + searchKnowledgeGraph: ( + context: PublishedKnowledgeGraph, + query: string + ) => Promise + readKnowledgeGraphNeighbors: ( + context: PublishedKnowledgeGraph, + nodeId: string + ) => Promise +} + +const nodeRequire = createRequire(import.meta.url) +let knowledgeGraph: KnowledgeGraphModule | undefined +let knowledgeGraphPromise: Promise | undefined + +function loadKnowledgeGraph(): Promise { + knowledgeGraphPromise ??= (async () => { + // Turbopack cannot evaluate FalkorDB's CommonJS Temporal/JSBI dependency + // chain. Use Node in development and let the production bundler resolve it. + const loaded = + process.env.NODE_ENV === 'development' + ? (nodeRequire('@klicker-uzh/knowledge-graph') as KnowledgeGraphModule) + : ((await import( + '@klicker-uzh/knowledge-graph' + )) as KnowledgeGraphModule) + + knowledgeGraph = loaded + return loaded + })() + + return knowledgeGraphPromise +} + +export async function getPublishedKnowledgeGraph( + client: PrismaClient, + kbId: string +): Promise { + return (await loadKnowledgeGraph()).getPublishedKnowledgeGraph(client, kbId) +} + +export async function getPublishedKnowledgeGraphForChatbot( + client: PrismaClient, + chatbotId: string +): Promise { + const binding = await client.kBChatbot.findFirst({ + where: { + chatbotId, + isEnabled: true, + kb: { deletedAt: null, knowledgeGraphEnabled: true }, + }, + select: { kbId: true }, + orderBy: { updatedAt: 'desc' }, + }) + + const knowledgeGraphModule = await loadKnowledgeGraph() + if (binding === null) { + throw new knowledgeGraphModule.KnowledgeGraphNotPublishedError('EMPTY') + } + + return knowledgeGraphModule.getPublishedKnowledgeGraph(client, binding.kbId) +} + +export async function readKnowledgeGraphOverview( + context: PublishedKnowledgeGraph +): Promise { + return (await loadKnowledgeGraph()).readKnowledgeGraphOverview(context) +} + +export async function searchKnowledgeGraph( + context: PublishedKnowledgeGraph, + query: string +): Promise { + return (await loadKnowledgeGraph()).searchKnowledgeGraph(context, query) +} + +export async function readKnowledgeGraphNeighbors( + context: PublishedKnowledgeGraph, + nodeId: string +): Promise { + return (await loadKnowledgeGraph()).readKnowledgeGraphNeighbors( + context, + nodeId + ) +} + +export function isKnowledgeGraphNotPublishedError( + error: unknown +): error is Error & { readonly code: KnowledgeGraphPublicationCode } { + return ( + knowledgeGraph !== undefined && + error instanceof knowledgeGraph.KnowledgeGraphNotPublishedError + ) +} diff --git a/apps/chat/src/services/mcpClients.ts b/apps/chat/src/services/mcpClients.ts index f99d1b27c7..ae287387a1 100644 --- a/apps/chat/src/services/mcpClients.ts +++ b/apps/chat/src/services/mcpClients.ts @@ -1,19 +1,21 @@ 'use server' +import { createHash, randomUUID } from 'node:crypto' import { experimental_createMCPClient as createSDKMCPClient } from '@ai-sdk/mcp' import { safeDecrypt } from '@klicker-uzh/util' import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js' -import { createHash } from 'node:crypto' import { MAX_TOOL_NAME_LENGTH, TOOL_NAME_SUFFIX_LENGTH, } from '@/src/lib/config/toolNames' +import { signDocQueryScopeToken } from '@/src/lib/server/docQueryScopeToken' +import type { AuthMode } from '@/src/lib/server/ltiGuest' +import { mintParticipantMcpJwt } from '@/src/lib/server/mcpAuthMint' import { parseMCPRuntimePolicy, RequiredMCPUnavailableError, } from '@/src/lib/server/mcpRuntimePolicy' -import type { AuthMode } from '@/src/lib/server/ltiGuest' -import { mintParticipantMcpJwt } from '@/src/lib/server/mcpAuthMint' +import { DOC_QUERY_MCP_SERVER_NAME } from './mcpScope' // Type definitions for MCP server configuration export interface MCPServerConfig { @@ -39,6 +41,14 @@ export interface MCPServerWithConfig { config: MCPConfigSettings } +export interface MCPRequestContext { + chatbotId: string + participantId?: string + authMode: AuthMode + kbId?: string + sessionId?: string +} + export interface MCPRequestOptions { requestTimeoutMs?: number } @@ -110,33 +120,54 @@ function toSafeToolName( /** * Creates authentication headers based on server auth type */ -async function createAuthHeaders( +export async function createAuthHeaders( server: MCPServerConfig, - chatbotId: string, - participantId = '', - authMode: AuthMode + context: MCPRequestContext ): Promise> { const baseHeaders: Record = { 'Content-Type': 'application/json', } + const authType = server.authType.toLowerCase() + + if (server.name === DOC_QUERY_MCP_SERVER_NAME) { + if (!context.kbId || !context.sessionId) { + throw new Error('Scoped knowledge retrieval is not available') + } + + const token = await signDocQueryScopeToken({ + kbId: context.kbId, + chatbotId: context.chatbotId, + sessionId: context.sessionId, + jti: randomUUID(), + }) + baseHeaders.Authorization = `Bearer ${token}` + return baseHeaders + } + + if (authType === 'scope_token') { + throw new Error('Scoped knowledge retrieval is not available') + } // Add chatbot ID if configured (new behavior - defaults to false for backward compatibility) if (server.passChatbotId) { const raw = server.chatbotIdHeader || 'Chatbot-ID' const headerName = raw.replace(/[^A-Za-z0-9-]/g, '') || 'Chatbot-ID' - baseHeaders[headerName] = chatbotId + baseHeaders[headerName] = context.chatbotId } // Per-participant JWT mint for the Klicker MCP server. Identity // comes from the caller's verified participant cookie, not a static // shared secret, so the MCP server can apply row-level auth. - if (server.authType.toLowerCase() === 'klicker-participant-jwt') { - if (!participantId) { + if (authType === 'klicker-participant-jwt') { + if (!context.participantId) { throw new Error( 'Participant identity is required for participant MCP auth' ) } - const token = await mintParticipantMcpJwt(participantId, authMode) + const token = await mintParticipantMcpJwt( + context.participantId, + context.authMode + ) baseHeaders.Authorization = `Bearer ${token}` return baseHeaders } @@ -147,7 +178,7 @@ async function createAuthHeaders( const decryptedSecret = safeDecrypt(server.authSecret) - switch (server.authType.toLowerCase()) { + switch (authType) { case 'custom': // Parse and apply custom headers from JSON { @@ -196,12 +227,41 @@ async function createAuthHeaders( return baseHeaders } +function normalizeMCPRequest( + contextOrChatbotId: MCPRequestContext | string, + participantIdOrOptions: string | MCPRequestOptions = '', + authMode: AuthMode = 'account' +): { context: MCPRequestContext; options: MCPRequestOptions } { + if (typeof contextOrChatbotId !== 'string') { + return { + context: contextOrChatbotId, + options: + typeof participantIdOrOptions === 'string' + ? {} + : participantIdOrOptions, + } + } + + return { + context: { + chatbotId: contextOrChatbotId, + participantId: + typeof participantIdOrOptions === 'string' + ? participantIdOrOptions + : undefined, + authMode, + }, + options: + typeof participantIdOrOptions === 'string' ? {} : participantIdOrOptions, + } +} + /** * Creates and initializes a single MCP client for a specific server configuration */ export async function createMCPClient( server: MCPServerConfig, - chatbotId: string, + contextOrChatbotId: MCPRequestContext | string, participantIdOrOptions: string | MCPRequestOptions = '', authMode: AuthMode = 'account' ) { @@ -209,18 +269,14 @@ export async function createMCPClient( throw new Error(`MCP server ${server.name} has no URL defined`) } - const participantId = - typeof participantIdOrOptions === 'string' ? participantIdOrOptions : '' - const options = - typeof participantIdOrOptions === 'string' ? {} : participantIdOrOptions + const { context, options } = normalizeMCPRequest( + contextOrChatbotId, + participantIdOrOptions, + authMode + ) try { - const headers = await createAuthHeaders( - server, - chatbotId, - participantId, - authMode - ) + const headers = await createAuthHeaders(server, context) const httpTransport = new StreamableHTTPClientTransport( new URL(server.url), @@ -228,7 +284,7 @@ export async function createMCPClient( requestInit: { headers, redirect: 'error', - ...(options.requestTimeoutMs + ...(options.requestTimeoutMs !== undefined ? { signal: AbortSignal.timeout(options.requestTimeoutMs) } : {}), }, @@ -275,9 +331,8 @@ function isToolAllowed(toolName: string, allowedTools: string[]): boolean { */ async function loadServerTools( serverWithConfig: MCPServerWithConfig, - chatbotId: string, - participantIdOrOptions: string | MCPRequestOptions = '', - authMode: AuthMode = 'account' + context: MCPRequestContext, + options: MCPRequestOptions ): Promise> { const { server, config } = serverWithConfig const runtimePolicy = parseMCPRuntimePolicy(config.parameters) @@ -305,12 +360,7 @@ async function loadServerTools( } try { - const client = await createMCPClient( - server, - chatbotId, - participantIdOrOptions, - authMode - ) + const client = await createMCPClient(server, context, options) const rawTools = await client.tools() if (runtimePolicy.required && requiredRawToolName) { @@ -372,12 +422,18 @@ async function loadServerTools( */ export async function getAggregatedMCPTools( serversWithConfigs: MCPServerWithConfig[], - chatbotId: string, + contextOrChatbotId: MCPRequestContext | string, participantIdOrOptions: string | MCPRequestOptions = '', authMode: AuthMode = 'account' ): Promise> { console.log(`Loading MCP Tools from ${serversWithConfigs.length} servers...`) + const { context, options } = normalizeMCPRequest( + contextOrChatbotId, + participantIdOrOptions, + authMode + ) + if (serversWithConfigs.length === 0) { console.log('No MCP servers configured') return {} @@ -396,9 +452,8 @@ export async function getAggregatedMCPTools( try { const serverTools = await loadServerTools( serverWithConfig, - chatbotId, - participantIdOrOptions, - authMode + context, + options ) const runtimePolicy = parseMCPRuntimePolicy( serverWithConfig.config.parameters @@ -462,9 +517,8 @@ export async function getMCPTools( try { const serverTools = await loadServerTools( { server: legacyServer, config: legacyConfig }, - chatbotId, - participantId, - authMode + { chatbotId, participantId, authMode }, + {} ) return serverTools } catch (error) { diff --git a/apps/chat/src/services/mcpScope.ts b/apps/chat/src/services/mcpScope.ts new file mode 100644 index 0000000000..1300b0aebe --- /dev/null +++ b/apps/chat/src/services/mcpScope.ts @@ -0,0 +1,40 @@ +export const DOC_QUERY_MCP_SERVER_NAME = 'KB' +export const DOC_QUERY_TOOL_NAME = `${DOC_QUERY_MCP_SERVER_NAME}_doc_query` + +export function resolveMcpScopeSessionId({ + requestedThreadId, + owningThreadId, + fallbackId, +}: { + requestedThreadId?: string | null + owningThreadId?: string + fallbackId: string +}): string | null { + if (requestedThreadId && requestedThreadId !== owningThreadId) { + return null + } + + return owningThreadId ?? fallbackId +} + +export function canLoadMCPServer( + server: { name: string; authType: string }, + context: { + chatbotId?: string + participantId?: string + kbId?: string + sessionId?: string + } +): boolean { + const authType = server.authType.toLowerCase() + + if (server.name === DOC_QUERY_MCP_SERVER_NAME) { + return Boolean(context.kbId) && Boolean(context.sessionId) + } + + if (authType !== 'scope_token') { + return true + } + + return false +} diff --git a/apps/chat/src/services/studentPracticeMcp.ts b/apps/chat/src/services/studentPracticeMcp.ts index 45c530c63e..6ac08f7988 100644 --- a/apps/chat/src/services/studentPracticeMcp.ts +++ b/apps/chat/src/services/studentPracticeMcp.ts @@ -243,12 +243,11 @@ async function withStudentPracticeMcp({ url, } - const client = await createMCPClient( - server, + const client = await createMCPClient(server, { + authMode, chatbotId, participantId, - authMode - ) + }) try { const tools = (await client.tools()) as unknown as Record< diff --git a/apps/chat/src/services/threads.ts b/apps/chat/src/services/threads.ts index 0a0e7c1775..0e7388e3ab 100644 --- a/apps/chat/src/services/threads.ts +++ b/apps/chat/src/services/threads.ts @@ -37,10 +37,12 @@ export class ThreadService { static async createThread( participantId: string, chatbotId: string, - title?: string | null + title?: string | null, + id?: string ): Promise { const thread = await prisma.chatThread.create({ data: { + ...(id ? { id } : {}), title, participant: { connect: { id: participantId }, diff --git a/apps/chat/src/stores/chatStore.ts b/apps/chat/src/stores/chatStore.ts index ae023e6080..fcb97abae0 100644 --- a/apps/chat/src/stores/chatStore.ts +++ b/apps/chat/src/stores/chatStore.ts @@ -84,6 +84,12 @@ interface ChatState { isLoading: boolean participationRequired: boolean participationMessage: string | null + /** + * KG workspace entry point: a 403 on the knowledge-graph read means the + * guest/participant has no course participation. Delegates to the shared + * participation notice (or clears it when the read succeeds again). + */ + setParticipationRequired: (required: boolean, message?: string) => void /** * Set when `loadThreads` fails for a reason other than the 403 * participation case (which `handleApiError` already surfaces via @@ -170,6 +176,15 @@ export const useChatStore = create((set, get) => { }) } + const setParticipationRequired = (required: boolean, message?: string) => { + if (required) { + markParticipationRequired(message) + return + } + + clearParticipationNotice() + } + const clearParticipationNotice = () => { set({ participationRequired: false, participationMessage: null }) } @@ -266,6 +281,7 @@ export const useChatStore = create((set, get) => { isLoading: false, participationRequired: false, participationMessage: null, + setParticipationRequired, threadsLoadError: false, ratingErrors: {}, diff --git a/apps/chat/test/doc-query-scope-token.test.ts b/apps/chat/test/doc-query-scope-token.test.ts new file mode 100644 index 0000000000..2029715022 --- /dev/null +++ b/apps/chat/test/doc-query-scope-token.test.ts @@ -0,0 +1,98 @@ +import { exportPKCS8, generateKeyPair, jwtVerify, type KeyLike } from 'jose' +import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest' +import { + DocQueryScopeTokenError, + signDocQueryScopeToken, +} from '../src/lib/server/docQueryScopeToken' + +const TEST_ISSUER = 'https://chat.klicker.test' +const TEST_AUDIENCE = 'klicker-doc-query-test' +const TEST_KID = 'test-key-2026-07' +const TEST_KB_ID = '7016810d-31e9-4b39-9529-cd46feb2fb63' +const TEST_CHATBOT_ID = '8f9c2e1d-4b7a-4c3e-9f5d-1a2b3c4d5e6f' +const TEST_SESSION_ID = 'opaque-session-4ca8d6a4' +const TEST_JTI = '9b3cc7c6-3a11-4f6b-93d0-4b3678cf89fc' + +let publicKey: KeyLike + +describe('signDocQueryScopeToken', () => { + beforeEach(async () => { + const keyPair = await generateKeyPair('ES256') + publicKey = keyPair.publicKey + vi.stubEnv( + 'DOC_QUERY_SCOPE_PRIVATE_KEY', + await exportPKCS8(keyPair.privateKey) + ) + vi.stubEnv('DOC_QUERY_SCOPE_KID', TEST_KID) + vi.stubEnv('DOC_QUERY_SCOPE_ISSUER', TEST_ISSUER) + vi.stubEnv('DOC_QUERY_SCOPE_AUDIENCE', TEST_AUDIENCE) + }) + + afterEach(() => { + vi.useRealTimers() + vi.unstubAllEnvs() + }) + + test('mints the five-minute ES256 scope contract', async () => { + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-07-27T12:00:00.000Z')) + + const token = await signDocQueryScopeToken({ + kbId: TEST_KB_ID, + chatbotId: TEST_CHATBOT_ID, + sessionId: TEST_SESSION_ID, + jti: TEST_JTI, + }) + const { payload, protectedHeader } = await jwtVerify(token, publicKey, { + algorithms: ['ES256'], + issuer: TEST_ISSUER, + audience: TEST_AUDIENCE, + }) + + expect(protectedHeader).toMatchObject({ + alg: 'ES256', + typ: 'JWT', + kid: TEST_KID, + }) + expect(payload).toMatchObject({ + iss: TEST_ISSUER, + aud: TEST_AUDIENCE, + sub: TEST_SESSION_ID, + jti: TEST_JTI, + kb_id: TEST_KB_ID, + chatbot_id: TEST_CHATBOT_ID, + }) + expect(payload.exp! - payload.iat!).toBe(300) + }) + + test.each([ + 'DOC_QUERY_SCOPE_PRIVATE_KEY', + 'DOC_QUERY_SCOPE_KID', + 'DOC_QUERY_SCOPE_ISSUER', + 'DOC_QUERY_SCOPE_AUDIENCE', + ])('fails closed when %s is missing', async (name) => { + delete process.env[name] + + await expect( + signDocQueryScopeToken({ + kbId: TEST_KB_ID, + chatbotId: TEST_CHATBOT_ID, + sessionId: TEST_SESSION_ID, + jti: TEST_JTI, + }) + ).rejects.toBeInstanceOf(DocQueryScopeTokenError) + }) + + test('fails closed without exposing invalid private-key material', async () => { + vi.stubEnv('DOC_QUERY_SCOPE_PRIVATE_KEY', 'not-a-private-key') + + await expect( + signDocQueryScopeToken({ + kbId: TEST_KB_ID, + chatbotId: TEST_CHATBOT_ID, + sessionId: TEST_SESSION_ID, + jti: TEST_JTI, + }) + ).rejects.toThrow('Scope token signing failed') + }) +}) diff --git a/apps/chat/test/knowledge-graph-api-client.test.ts b/apps/chat/test/knowledge-graph-api-client.test.ts new file mode 100644 index 0000000000..bbbdbfba14 --- /dev/null +++ b/apps/chat/test/knowledge-graph-api-client.test.ts @@ -0,0 +1,248 @@ +import { + ChatKnowledgeGraphRequestError, + createChatKnowledgeGraphDataSource, +} from '@/src/components/knowledge-graph/ChatKnowledgeGraphWorkspace' +import { CHAT_GUEST_SESSION_STORAGE_KEY } from '@/src/hooks/useChatGuestTokenBootstrap' +import { useChatStore } from '@/src/stores/chatStore' +import { KnowledgeGraphUnavailableError } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import type { KnowledgeGraphResponse } from '@klicker-uzh/types' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' + +const chatbotId = '11111111-1111-4111-8111-111111111111' +const graphResponse: KnowledgeGraphResponse = { + kbId: '22222222-2222-4222-8222-222222222222', + buildId: '33333333-3333-4333-8333-333333333333', + isStale: false, + nodes: [ + { + id: '12', + labels: ['Concept'], + kind: 'Concept', + displayLabel: 'Android security', + degree: 2, + sourceReferences: [], + }, + ], + edges: [], + truncated: false, +} + +function jsonResponse(body: unknown, status = 200): Response { + return new Response(JSON.stringify(body), { + status, + headers: { 'Content-Type': 'application/json' }, + }) +} + +describe('chat knowledge graph API client', () => { + let originalFetch: typeof globalThis.fetch + const originalSessionStorage = globalThis.sessionStorage + + beforeEach(() => { + originalFetch = globalThis.fetch + const store = new Map() + Object.defineProperty(globalThis, 'sessionStorage', { + configurable: true, + value: { + getItem: (key: string) => store.get(key) ?? null, + setItem: (key: string, value: string) => store.set(key, value), + removeItem: (key: string) => store.delete(key), + clear: () => store.clear(), + key: (index: number) => Array.from(store.keys())[index] ?? null, + length: 0, + }, + }) + useChatStore.getState().setParticipationRequired(false) + }) + + afterEach(() => { + globalThis.fetch = originalFetch + Object.defineProperty(globalThis, 'sessionStorage', { + configurable: true, + value: originalSessionStorage, + }) + vi.restoreAllMocks() + }) + + it('encodes search text with URLSearchParams', async () => { + const fetcher = vi.fn().mockResolvedValue(jsonResponse(graphResponse)) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + await dataSource.search('Android security & privacy') + + expect(fetcher).toHaveBeenCalledWith( + `/api/chatbots/${chatbotId}/knowledge-graph?operation=search&q=Android+security+%26+privacy` + ) + }) + + it('passes decimal FalkorDB node IDs through the neighbors operation', async () => { + const fetcher = vi.fn().mockResolvedValue(jsonResponse(graphResponse)) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + await dataSource.neighbors('12004') + + expect(fetcher).toHaveBeenCalledWith( + `/api/chatbots/${chatbotId}/knowledge-graph?operation=neighbors&nodeId=12004` + ) + }) + + it('maps unpublished responses to an unavailable graph error with status', async () => { + const fetcher = vi.fn().mockResolvedValue( + jsonResponse( + { + code: 'KNOWLEDGE_GRAPH_NOT_PUBLISHED', + error: 'Knowledge graph is not published', + publicationStatus: 'PROCESSING', + }, + 409 + ) + ) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + const error = await dataSource.overview().catch((caught) => caught) + + expect(error).toBeInstanceOf(KnowledgeGraphUnavailableError) + expect(error).toMatchObject({ + status: 409, + publicationStatus: 'PROCESSING', + }) + }) + + it('maps a temporary read failure to a safe retryable error', async () => { + const fetcher = vi.fn().mockResolvedValue( + jsonResponse( + { + code: 'KNOWLEDGE_GRAPH_TEMPORARILY_UNAVAILABLE', + error: 'redis://reader:secret@falkordb.internal', + }, + 503 + ) + ) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + const error = await dataSource.overview().catch((caught) => caught) + + expect(error).toBeInstanceOf(ChatKnowledgeGraphRequestError) + expect(error).toMatchObject({ + message: 'Knowledge graph is temporarily unavailable', + retryable: true, + status: 503, + }) + expect(JSON.stringify(error)).not.toContain('secret') + }) + + it('opens the existing participation gate when an embedded graph gets a 403 on its first request', async () => { + const fetcher = vi.fn().mockResolvedValue( + jsonResponse( + { + error: + 'No valid participation found for this chatbot; redis://secret', + }, + 403 + ) + ) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + await expect(dataSource.overview()).rejects.toMatchObject({ + message: 'Knowledge graph request failed', + retryable: false, + status: 403, + }) + expect(useChatStore.getState()).toMatchObject({ + participationRequired: true, + participationMessage: null, + }) + expect(JSON.stringify(useChatStore.getState())).not.toContain('secret') + }) + + it('uses authedFetch so the guest bearer token reaches the API', async () => { + sessionStorage.setItem(CHAT_GUEST_SESSION_STORAGE_KEY, 'guest-token') + const fetchMock = vi.fn().mockResolvedValue(jsonResponse(graphResponse)) + globalThis.fetch = fetchMock as unknown as typeof globalThis.fetch + const dataSource = createChatKnowledgeGraphDataSource(chatbotId) + + await dataSource.overview() + + expect(fetchMock).toHaveBeenCalledTimes(1) + const [url, init] = fetchMock.mock.calls[0] as unknown as [ + string, + RequestInit, + ] + expect(url).toBe( + `/api/chatbots/${chatbotId}/knowledge-graph?operation=overview` + ) + expect(new Headers(init.headers).get('authorization')).toBe( + 'Bearer guest-token' + ) + }) + + it('decodes a successful browser-safe DTO', async () => { + const fetcher = vi.fn().mockResolvedValue(jsonResponse(graphResponse)) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + await expect(dataSource.overview()).resolves.toEqual(graphResponse) + }) + + it.each([ + { + name: 'invalid top-level nodes', + value: { ...graphResponse, nodes: 'not-an-array' }, + }, + { + name: 'invalid nested node degree', + value: { + ...graphResponse, + nodes: [{ ...graphResponse.nodes[0]!, degree: 'secret-degree' }], + }, + }, + { + name: 'invalid nested edge properties', + value: { + ...graphResponse, + edges: [ + { + id: '41', + source: '12', + target: '13', + type: 'RELATED_TO', + label: 'related to', + properties: { embedding: ['secret'] }, + }, + ], + }, + }, + ])('rejects a malformed 2xx DTO safely: $name', async ({ value }) => { + const fetcher = vi.fn().mockResolvedValue(jsonResponse(value)) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + const error = await dataSource.overview().catch((caught) => caught) + + expect(error).toBeInstanceOf(ChatKnowledgeGraphRequestError) + expect(error).toMatchObject({ + message: 'Knowledge graph request failed', + retryable: false, + status: 502, + }) + expect(JSON.stringify(error)).not.toContain('secret') + }) + + it('rejects malformed 2xx JSON without leaking parser input', async () => { + const fetcher = vi.fn().mockResolvedValue( + new Response('{"password":"secret"', { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) + const dataSource = createChatKnowledgeGraphDataSource(chatbotId, fetcher) + + const error = await dataSource.overview().catch((caught) => caught) + + expect(error).toBeInstanceOf(ChatKnowledgeGraphRequestError) + expect(error).toMatchObject({ + message: 'Knowledge graph request failed', + retryable: false, + status: 502, + }) + expect(JSON.stringify(error)).not.toContain('secret') + }) +}) diff --git a/apps/chat/test/knowledge-graph-route.test.ts b/apps/chat/test/knowledge-graph-route.test.ts new file mode 100644 index 0000000000..7dbcb3295d --- /dev/null +++ b/apps/chat/test/knowledge-graph-route.test.ts @@ -0,0 +1,309 @@ +import type { + KnowledgeGraphResponse, + KnowledgeGraphSourceReference, +} from '@klicker-uzh/types' +import { NextRequest, NextResponse } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const boundaries = vi.hoisted(() => ({ + getPublishedKnowledgeGraphForChatbot: vi.fn(), + isKnowledgeGraphNotPublishedError: vi.fn(), + readKnowledgeGraphNeighbors: vi.fn(), + readKnowledgeGraphOverview: vi.fn(), + searchKnowledgeGraph: vi.fn(), + withChatbotAuth: vi.fn(), +})) + +vi.mock('@/src/lib/server/apiGuards', () => ({ + withChatbotAuth: boundaries.withChatbotAuth, +})) + +vi.mock('@klicker-uzh/prisma', () => ({ + prisma: { kBChatbot: {} }, +})) + +vi.mock('@/src/lib/server/knowledgeGraphRuntime', () => { + return { + getPublishedKnowledgeGraphForChatbot: + boundaries.getPublishedKnowledgeGraphForChatbot, + isKnowledgeGraphNotPublishedError: + boundaries.isKnowledgeGraphNotPublishedError, + readKnowledgeGraphNeighbors: boundaries.readKnowledgeGraphNeighbors, + readKnowledgeGraphOverview: boundaries.readKnowledgeGraphOverview, + searchKnowledgeGraph: boundaries.searchKnowledgeGraph, + } +}) + +import { GET } from '../src/app/api/chatbots/[chatbotId]/knowledge-graph/route' + +const chatbotId = '11111111-1111-4111-8111-111111111111' +const kbId = '22222222-2222-4222-8222-222222222222' +const publication = { + kbId, + buildId: '33333333-3333-4333-8333-333333333333', + graphName: `klickeruzh:kb:${kbId}:33333333-3333-4333-8333-333333333333`, + isStale: false, + sources: [ + { + resourceId: '22222222-2222-4222-8222-222222222222', + title: 'Lecture notes', + }, + ], +} +const response: KnowledgeGraphResponse = { + kbId, + buildId: publication.buildId, + isStale: false, + nodes: [ + { + id: '12', + labels: ['Concept'], + kind: 'Concept', + displayLabel: 'Access control', + summary: 'Authorization follows authentication.', + content: 'A bounded piece of source content.', + degree: 2, + sourceReferences: [ + { + resourceId: publication.sources[0]!.resourceId, + title: publication.sources[0]!.title, + reference: 'p. 4', + }, + ], + }, + ], + edges: [ + { + id: '41', + source: '12', + target: '13', + type: 'RELATED_TO', + label: 'related to', + properties: { confidence: 0.8 }, + }, + ], + truncated: false, +} + +function graphRequest(search: string): NextRequest { + return new NextRequest( + `http://localhost/api/chatbots/${chatbotId}/knowledge-graph?${search}` + ) +} + +async function callRoute(search: string) { + return GET(graphRequest(search), { + params: Promise.resolve({ chatbotId }), + }) +} + +beforeEach(() => { + vi.resetAllMocks() + boundaries.withChatbotAuth.mockResolvedValue({ + participantId: 'participant-id', + authMode: 'account', + chatbot: { courseId: 'course-id' }, + }) + boundaries.getPublishedKnowledgeGraphForChatbot.mockResolvedValue(publication) + boundaries.isKnowledgeGraphNotPublishedError.mockImplementation( + (error) => + error instanceof Error && error.name === 'KnowledgeGraphNotPublishedError' + ) + boundaries.readKnowledgeGraphOverview.mockResolvedValue(response) + boundaries.searchKnowledgeGraph.mockResolvedValue(response) + boundaries.readKnowledgeGraphNeighbors.mockResolvedValue(response) +}) + +describe('participant knowledge graph route', () => { + it('returns the authentication response before validation or graph access', async () => { + boundaries.withChatbotAuth.mockResolvedValue({ + response: NextResponse.json( + { error: 'No authentication token found' }, + { status: 401 } + ), + }) + + const result = await callRoute('operation=arbitrary-cypher') + + expect(result.status).toBe(401) + await expect(result.json()).resolves.toEqual({ + error: 'No authentication token found', + }) + expect( + boundaries.getPublishedKnowledgeGraphForChatbot + ).not.toHaveBeenCalled() + expect(boundaries.readKnowledgeGraphOverview).not.toHaveBeenCalled() + }) + + it('does not expose graph publication or data to a non-participant', async () => { + boundaries.withChatbotAuth.mockResolvedValue({ + response: NextResponse.json( + { error: 'No valid participation found for this chatbot' }, + { status: 403 } + ), + }) + + const result = await callRoute('operation=overview') + + expect(result.status).toBe(403) + expect( + boundaries.getPublishedKnowledgeGraphForChatbot + ).not.toHaveBeenCalled() + expect(boundaries.readKnowledgeGraphOverview).not.toHaveBeenCalled() + }) + + it.each([ + 'EMPTY', + 'QUEUED', + 'PROCESSING', + 'FAILED', + ] as const)('returns a safe 409 for an unpublished %s graph', async (publicationStatus) => { + const error = Object.assign(new Error('Knowledge graph is not published'), { + code: publicationStatus, + name: 'KnowledgeGraphNotPublishedError', + }) + boundaries.getPublishedKnowledgeGraphForChatbot.mockRejectedValue(error) + + const result = await callRoute('operation=overview') + + expect(result.status).toBe(409) + await expect(result.json()).resolves.toEqual({ + code: 'KNOWLEDGE_GRAPH_NOT_PUBLISHED', + error: 'Knowledge graph is not published', + publicationStatus, + }) + expect(boundaries.readKnowledgeGraphOverview).not.toHaveBeenCalled() + }) + + it('returns the normalized overview DTO', async () => { + const result = await callRoute('operation=overview') + + expect(result.status).toBe(200) + await expect(result.json()).resolves.toEqual(response) + expect( + boundaries.getPublishedKnowledgeGraphForChatbot + ).toHaveBeenCalledWith(expect.anything(), chatbotId) + expect(boundaries.readKnowledgeGraphOverview).toHaveBeenCalledWith( + publication + ) + expect(boundaries.searchKnowledgeGraph).not.toHaveBeenCalled() + expect(boundaries.readKnowledgeGraphNeighbors).not.toHaveBeenCalled() + }) + + it('trims and passes a valid bounded search only to the fixed reader', async () => { + const result = await callRoute( + `operation=search&q=${encodeURIComponent(' Android security ')}` + ) + + expect(result.status).toBe(200) + expect(boundaries.searchKnowledgeGraph).toHaveBeenCalledWith( + publication, + 'Android security' + ) + expect(boundaries.readKnowledgeGraphOverview).not.toHaveBeenCalled() + }) + + it.each([ + 'operation=search', + 'operation=search&q=%20%20', + `operation=search&q=${'a'.repeat(101)}`, + 'operation=delete', + ])('rejects an invalid operation or search input: %s', async (search) => { + const result = await callRoute(search) + + expect(result.status).toBe(400) + await expect(result.json()).resolves.toEqual({ + code: 'INVALID_KNOWLEDGE_GRAPH_REQUEST', + error: 'Invalid knowledge graph request', + }) + expect( + boundaries.getPublishedKnowledgeGraphForChatbot + ).not.toHaveBeenCalled() + expect(boundaries.searchKnowledgeGraph).not.toHaveBeenCalled() + }) + + it('passes a numeric node ID only to the fixed neighborhood reader', async () => { + const result = await callRoute('operation=neighbors&nodeId=12004') + + expect(result.status).toBe(200) + expect(boundaries.readKnowledgeGraphNeighbors).toHaveBeenCalledWith( + publication, + '12004' + ) + expect(boundaries.readKnowledgeGraphOverview).not.toHaveBeenCalled() + }) + + it.each([ + 'operation=neighbors', + 'operation=neighbors&nodeId=', + 'operation=neighbors&nodeId=-1', + 'operation=neighbors&nodeId=12.4', + 'operation=neighbors&nodeId=node-12', + ])('rejects an invalid neighborhood node ID: %s', async (search) => { + const result = await callRoute(search) + + expect(result.status).toBe(400) + expect( + boundaries.getPublishedKnowledgeGraphForChatbot + ).not.toHaveBeenCalled() + expect(boundaries.readKnowledgeGraphNeighbors).not.toHaveBeenCalled() + }) + + it('sanitizes temporary FalkorDB failures and operational logs', async () => { + boundaries.readKnowledgeGraphOverview.mockRejectedValue( + new Error( + 'redis://reader:secret@falkordb.internal/graph?source=https://private.example' + ) + ) + const consoleError = vi + .spyOn(console, 'error') + .mockImplementation(() => undefined) + + try { + const result = await callRoute('operation=overview') + + expect(result.status).toBe(503) + await expect(result.json()).resolves.toEqual({ + code: 'KNOWLEDGE_GRAPH_TEMPORARILY_UNAVAILABLE', + error: 'Knowledge graph is temporarily unavailable', + }) + expect(consoleError).toHaveBeenCalledWith( + 'Participant knowledge graph read failed', + { chatbotId, operation: 'overview' } + ) + expect(JSON.stringify(consoleError.mock.calls)).not.toMatch( + /secret|private\.example|redis:\/\// + ) + } finally { + consoleError.mockRestore() + } + }) + + it('returns only the public DTO fields supplied by the fixed reader', async () => { + const unsafeSource = { + ...response.nodes[0]!.sourceReferences[0]!, + sourceUrl: 'https://blob.example/private.pdf?sig=secret', + } as KnowledgeGraphSourceReference + boundaries.readKnowledgeGraphOverview.mockResolvedValue({ + ...response, + graphName: publication.graphName, + connectionString: 'redis://reader:secret@falkordb.internal', + nodes: [ + { + ...response.nodes[0]!, + embedding: [1, 2, 3], + sourceReferences: [unsafeSource], + }, + ], + edges: [{ ...response.edges[0]!, cypher: 'MATCH (n) RETURN n' }], + }) + + const result = await callRoute('operation=overview') + const body = await result.json() + + expect(body).toEqual(response) + expect(JSON.stringify(body)).not.toMatch( + /connectionString|embedding|graphName|sourceUrl|cypher|secret/ + ) + }) +}) diff --git a/apps/chat/test/knowledge-graph-state.test.ts b/apps/chat/test/knowledge-graph-state.test.ts new file mode 100644 index 0000000000..6f1d6ff0b2 --- /dev/null +++ b/apps/chat/test/knowledge-graph-state.test.ts @@ -0,0 +1,358 @@ +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphResponse, +} from '@klicker-uzh/types' +import { describe, expect, it } from 'vitest' + +import { + type KnowledgeGraphState, + initialKnowledgeGraphState, + knowledgeGraphReducer, + mergeKnowledgeGraphResponse, +} from '../../../packages/shared-components/src/knowledgeGraph/knowledgeGraphState.js' +import { + nextKnowledgeGraphZoom, + relationshipLabels, +} from '../../../packages/shared-components/src/knowledgeGraph/knowledgeGraphView.js' + +function node(id: string, overrides: Partial = {}) { + return { + id, + labels: ['Konzept'], + kind: 'Concept', + displayLabel: `Node ${id}`, + degree: 1, + sourceReferences: [], + ...overrides, + } +} + +function edge(id: string, source: string, target: string): KnowledgeGraphEdge { + return { + id, + source, + target, + type: 'RELATED', + label: 'RELATED', + properties: {}, + } +} + +function response( + buildId: string | number, + nodes: KnowledgeGraphNode[], + edges: KnowledgeGraphEdge[] = [], + overrides: Partial = {} +): KnowledgeGraphResponse { + return { + kbId: 'kb-a', + buildId: String(buildId), + isStale: false, + nodes, + edges, + truncated: false, + ...overrides, + } +} + +describe('knowledge graph state', () => { + it('deduplicates nodes and edges by ID while preferring incoming values', () => { + const current = { + ...initialKnowledgeGraphState, + kbId: 'kb-a', + buildId: '1', + nodes: [node('1', { displayLabel: 'Old' }), node('1')], + edges: [edge('10', '1', '2'), edge('10', '1', '2')], + } + + const merged = mergeKnowledgeGraphResponse( + current, + response( + 1, + [node('1', { displayLabel: 'Updated' }), node('2')], + [edge('10', '1', '2'), edge('11', '2', '1')] + ) + ) + + expect(merged.nodes.map((entry) => entry.id)).toEqual(['1', '2']) + expect(merged.nodes[0]?.displayLabel).toBe('Updated') + expect(merged.edges.map((entry) => entry.id)).toEqual(['10', '11']) + }) + + it('replaces the whole graph when the published revision changes', () => { + const current = { + ...initialKnowledgeGraphState, + kbId: 'kb-a', + buildId: '1', + nodes: [node('old')], + edges: [edge('old-edge', 'old', 'other')], + selectedNodeId: 'old', + } + + const replaced = mergeKnowledgeGraphResponse( + current, + response(2, [node('new')]) + ) + + expect(replaced).toMatchObject({ + buildId: '2', + nodes: [node('new')], + edges: [], + selectedNodeId: null, + selectedEdgeId: null, + }) + }) + + it('merges a same-revision neighborhood without losing the overview', () => { + let state = knowledgeGraphReducer(initialKnowledgeGraphState, { + type: 'request-started', + operation: 'overview', + requestId: 1, + }) + state = knowledgeGraphReducer(state, { + type: 'request-succeeded', + operation: 'overview', + requestId: 1, + response: response(3, [node('1')]), + }) + state = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'neighbors', + requestId: 2, + }) + state = knowledgeGraphReducer(state, { + type: 'request-succeeded', + operation: 'neighbors', + requestId: 2, + response: response(3, [node('2')], [edge('10', '1', '2')]), + }) + + expect(state.nodes.map((entry) => entry.id)).toEqual(['1', '2']) + expect(state.edges.map((entry) => entry.id)).toEqual(['10']) + }) + + it('keeps node and edge selection mutually exclusive and closes to deselect', () => { + const loaded = { + ...initialKnowledgeGraphState, + nodes: [node('1'), node('2')], + edges: [edge('10', '1', '2')], + } + + const selectedNode = knowledgeGraphReducer(loaded, { + type: 'select-node', + nodeId: '1', + }) + expect(selectedNode).toMatchObject({ + selectedNodeId: '1', + selectedEdgeId: null, + focusedNodeId: '1', + }) + + const selectedEdge = knowledgeGraphReducer(selectedNode, { + type: 'select-edge', + edgeId: '10', + }) + expect(selectedEdge).toMatchObject({ + selectedNodeId: null, + selectedEdgeId: '10', + }) + + expect( + knowledgeGraphReducer(selectedEdge, { type: 'close-details' }) + ).toMatchObject({ selectedNodeId: null, selectedEdgeId: null }) + }) + + it('focuses the first successful search result', () => { + const loaded = { + ...initialKnowledgeGraphState, + kbId: 'kb-a', + buildId: '4', + nodes: [node('1')], + } + let state = knowledgeGraphReducer(loaded, { + type: 'request-started', + operation: 'search', + requestId: 7, + }) + state = knowledgeGraphReducer(state, { + type: 'request-succeeded', + operation: 'search', + requestId: 7, + response: response(4, [node('2'), node('3')]), + }) + + expect(state.searchResults.map((entry) => entry.id)).toEqual(['2', '3']) + expect(state).toMatchObject({ + selectedNodeId: '2', + focusedNodeId: '2', + }) + }) + + it('models initial loading, temporary error, retry, and unavailable states', () => { + const loading = knowledgeGraphReducer(initialKnowledgeGraphState, { + type: 'request-started', + operation: 'overview', + requestId: 1, + }) + expect(loading.status).toBe('loading') + + const error = knowledgeGraphReducer(loading, { + type: 'request-failed', + operation: 'overview', + requestId: 1, + message: 'Knowledge graph is temporarily unavailable.', + }) + expect(error).toMatchObject({ + status: 'error', + errorMessage: 'Knowledge graph is temporarily unavailable.', + failedRequest: { operation: 'overview', input: null }, + }) + + const retrying = knowledgeGraphReducer(error, { + type: 'request-started', + operation: 'overview', + requestId: 2, + }) + expect(retrying).toMatchObject({ status: 'loading', errorMessage: null }) + + const unavailable = knowledgeGraphReducer(retrying, { + type: 'request-unavailable', + operation: 'overview', + requestId: 2, + message: 'Build the current selection before opening the graph.', + }) + expect(unavailable).toMatchObject({ + status: 'unavailable', + nodes: [], + edges: [], + unavailableMessage: + 'Build the current selection before opening the graph.', + }) + }) + + it('suppresses stale success, failure, and unavailable responses', () => { + let state = knowledgeGraphReducer(initialKnowledgeGraphState, { + type: 'request-started', + operation: 'overview', + requestId: 1, + }) + state = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'overview', + requestId: 2, + }) + + const staleSuccess = knowledgeGraphReducer(state, { + type: 'request-succeeded', + operation: 'overview', + requestId: 1, + response: response(1, [node('stale')]), + }) + const staleFailure = knowledgeGraphReducer(staleSuccess, { + type: 'request-failed', + operation: 'overview', + requestId: 1, + message: 'stale error', + }) + const staleUnavailable = knowledgeGraphReducer(staleFailure, { + type: 'request-unavailable', + operation: 'overview', + requestId: 1, + message: 'stale unavailable', + }) + + expect(staleUnavailable).toEqual(state) + + const current = knowledgeGraphReducer(staleUnavailable, { + type: 'request-succeeded', + operation: 'overview', + requestId: 2, + response: response(2, [node('current')]), + }) + expect(current.nodes.map((entry) => entry.id)).toEqual(['current']) + }) + + it('invalidates requests and graph data when the data source resets', () => { + let state: KnowledgeGraphState = { + ...initialKnowledgeGraphState, + kbId: 'kb-old', + buildId: '1', + nodes: [node('old')], + } + state = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'overview', + requestId: 20, + }) + state = knowledgeGraphReducer(state, { type: 'reset' }) + state = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'overview', + requestId: 21, + }) + + const stale = knowledgeGraphReducer(state, { + type: 'request-succeeded', + operation: 'overview', + requestId: 20, + response: response(1, [node('old')], [], { kbId: 'kb-old' }), + }) + + expect(stale).toEqual(state) + expect(stale.nodes).toEqual([]) + expect(stale.status).toBe('loading') + }) + + it('retains the failed operation input for an exact retry', () => { + let state: KnowledgeGraphState = { + ...initialKnowledgeGraphState, + kbId: 'kb-a', + buildId: '1', + nodes: [node('1')], + status: 'ready' as const, + } + state = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'neighbors', + requestId: 30, + input: '1', + }) + state = knowledgeGraphReducer(state, { + type: 'request-failed', + operation: 'neighbors', + requestId: 30, + input: '1', + message: 'Connections are temporarily unavailable. Try again.', + }) + + expect(state.failedRequest).toEqual({ + operation: 'neighbors', + input: '1', + }) + + const retrying = knowledgeGraphReducer(state, { + type: 'request-started', + operation: 'neighbors', + requestId: 31, + input: '1', + }) + expect(retrying.failedRequest).toBeNull() + expect(retrying.errorMessage).toBeNull() + }) + + it('clamps zoom and resolves relationship endpoints to concept labels', () => { + expect(nextKnowledgeGraphZoom(2.9, 1.25)).toBe(3) + expect(nextKnowledgeGraphZoom(0.16, 0.8)).toBe(0.15) + expect(nextKnowledgeGraphZoom(1, 1.25)).toBe(1.25) + + const nodes = new Map([ + ['1', node('1', { displayLabel: 'Alpha' })], + ['2', node('2', { displayLabel: 'Beta' })], + ]) + expect(relationshipLabels(edge('10', '1', '2'), nodes)).toEqual({ + source: 'Alpha', + target: 'Beta', + }) + }) +}) diff --git a/apps/chat/test/mcp-clients-scope-token.test.ts b/apps/chat/test/mcp-clients-scope-token.test.ts new file mode 100644 index 0000000000..5374e51ec3 --- /dev/null +++ b/apps/chat/test/mcp-clients-scope-token.test.ts @@ -0,0 +1,142 @@ +import { exportPKCS8, generateKeyPair, jwtVerify, type KeyLike } from 'jose' +import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest' +import { + createAuthHeaders, + type MCPRequestContext, + type MCPServerConfig, +} from '../src/services/mcpClients' +import { + canLoadMCPServer, + DOC_QUERY_TOOL_NAME, + resolveMcpScopeSessionId, +} from '../src/services/mcpScope' + +const TEST_ISSUER = 'https://chat.klicker.test' +const TEST_AUDIENCE = 'klicker-doc-query-test' +const TEST_KB_ID = '7016810d-31e9-4b39-9529-cd46feb2fb63' +const TEST_CHATBOT_ID = '8f9c2e1d-4b7a-4c3e-9f5d-1a2b3c4d5e6f' +const TEST_CONTEXT: MCPRequestContext = { + chatbotId: TEST_CHATBOT_ID, + participantId: 'participant-must-not-leave-klicker', + authMode: 'account', + kbId: TEST_KB_ID, + sessionId: 'opaque-chat-session', +} +const SCOPE_SERVER: MCPServerConfig = { + id: 'kb-server', + name: 'KB', + url: 'http://doc-query.test/mcp', + authType: 'scope_token', + passChatbotId: true, +} + +let publicKey: KeyLike + +describe('doc-query MCP scope authentication', () => { + beforeEach(async () => { + const keyPair = await generateKeyPair('ES256') + publicKey = keyPair.publicKey + vi.stubEnv( + 'DOC_QUERY_SCOPE_PRIVATE_KEY', + await exportPKCS8(keyPair.privateKey) + ) + vi.stubEnv('DOC_QUERY_SCOPE_KID', 'test-key') + vi.stubEnv('DOC_QUERY_SCOPE_ISSUER', TEST_ISSUER) + vi.stubEnv('DOC_QUERY_SCOPE_AUDIENCE', TEST_AUDIENCE) + }) + + afterEach(() => { + vi.unstubAllEnvs() + }) + + test('sends only a scoped bearer token to the KB server', async () => { + const headers = await createAuthHeaders(SCOPE_SERVER, TEST_CONTEXT) + const token = headers.Authorization?.replace(/^Bearer /, '') + + expect(token).toBeTruthy() + expect(headers).not.toHaveProperty('Chatbot-ID') + + const { payload } = await jwtVerify(token!, publicKey, { + algorithms: ['ES256'], + issuer: TEST_ISSUER, + audience: TEST_AUDIENCE, + }) + expect(payload).toMatchObject({ + sub: TEST_CONTEXT.sessionId, + kb_id: TEST_KB_ID, + chatbot_id: TEST_CHATBOT_ID, + }) + expect(payload).not.toHaveProperty('participantId') + expect(payload).not.toHaveProperty('participant_id') + }) + + test('skips scoped servers when no enabled KB was resolved', () => { + expect( + canLoadMCPServer(SCOPE_SERVER, { + chatbotId: TEST_CHATBOT_ID, + participantId: TEST_CONTEXT.participantId, + sessionId: TEST_CONTEXT.sessionId, + }) + ).toBe(false) + }) + + test('never sends a KB scope token to another server', async () => { + const otherServer = { ...SCOPE_SERVER, name: 'Other' } + + expect(canLoadMCPServer(otherServer, TEST_CONTEXT)).toBe(false) + await expect(createAuthHeaders(otherServer, TEST_CONTEXT)).rejects.toThrow( + 'Scoped knowledge retrieval is not available' + ) + }) + + test('replaces legacy KB authentication with a scoped token', async () => { + const legacyServer = { + ...SCOPE_SERVER, + authType: 'bearer', + authSecret: 'legacy-secret-must-not-leave-klicker', + } + + expect(canLoadMCPServer(legacyServer, TEST_CONTEXT)).toBe(true) + const headers = await createAuthHeaders(legacyServer, TEST_CONTEXT) + const token = headers.Authorization?.replace(/^Bearer /, '') + + expect(token).toBeTruthy() + expect(headers.Authorization).not.toContain(legacyServer.authSecret) + expect(headers).not.toHaveProperty('Chatbot-ID') + await expect( + jwtVerify(token!, publicKey, { + algorithms: ['ES256'], + issuer: TEST_ISSUER, + audience: TEST_AUDIENCE, + }) + ).resolves.toBeTruthy() + }) + + test('keeps the citation card aligned with the runtime tool name', () => { + expect(DOC_QUERY_TOOL_NAME).toBe('KB_doc_query') + }) + + test('never signs a client-supplied foreign thread as the session subject', () => { + expect( + resolveMcpScopeSessionId({ + requestedThreadId: 'foreign-thread', + owningThreadId: undefined, + fallbackId: 'server-request', + }) + ).toBeNull() + expect( + resolveMcpScopeSessionId({ + requestedThreadId: 'owned-thread', + owningThreadId: 'owned-thread', + fallbackId: 'server-request', + }) + ).toBe('owned-thread') + expect( + resolveMcpScopeSessionId({ + requestedThreadId: null, + owningThreadId: undefined, + fallbackId: 'server-request', + }) + ).toBe('server-request') + }) +}) diff --git a/apps/chat/test/mcp-clients.test.ts b/apps/chat/test/mcp-clients.test.ts index 27cff8d9be..85dbe38389 100644 --- a/apps/chat/test/mcp-clients.test.ts +++ b/apps/chat/test/mcp-clients.test.ts @@ -81,9 +81,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) expect(Object.keys(tools)).toEqual(['IW_doc_query']) @@ -103,9 +101,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) @@ -120,9 +116,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) expect(createSDKMCPClientMock).toHaveBeenCalledTimes(1) @@ -139,20 +133,16 @@ describe('MCP runtime policy', () => { { allowedTools: ['search*'] } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).resolves.toEqual({}) setTools({ search_docs: { description: 'search' }, unrelated: {} }) await expect( - getAggregatedMCPTools( - [createServer({}, { allowedTools: ['search*'] })], - 'chatbot-1', - '', - 'account' - ) + getAggregatedMCPTools([createServer({}, { allowedTools: ['search*'] })], { + chatbotId: 'chatbot-1', + authMode: 'account', + }) ).resolves.toEqual({ IW_search_docs: { description: 'search' } }) }) @@ -182,9 +172,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) }) @@ -201,9 +189,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) expect(createSDKMCPClientMock).not.toHaveBeenCalled() @@ -221,9 +207,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) expect(createSDKMCPClientMock).not.toHaveBeenCalled() @@ -242,9 +226,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) const [toolName] = Object.keys(tools) @@ -274,7 +256,10 @@ describe('MCP runtime policy', () => { tools: vi.fn().mockResolvedValue({ video_expert: {} }), }) await expect( - getAggregatedMCPTools([optional, required], 'chatbot-1', '', 'account') + getAggregatedMCPTools([optional, required], { + chatbotId: 'chatbot-1', + authMode: 'account', + }) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) vi.clearAllMocks() @@ -288,7 +273,10 @@ describe('MCP runtime policy', () => { tools: vi.fn().mockResolvedValue({ doc_query: {} }), }) await expect( - getAggregatedMCPTools([optional, required], 'chatbot-1', '', 'account') + getAggregatedMCPTools([optional, required], { + chatbotId: 'chatbot-1', + authMode: 'account', + }) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) }) @@ -309,9 +297,7 @@ describe('MCP runtime policy', () => { } ), ], - 'chatbot-1', - '', - 'account' + { chatbotId: 'chatbot-1', authMode: 'account' } ) ).rejects.toMatchObject({ code: REQUIRED_MCP_UNAVAILABLE_CODE }) expect(createSDKMCPClientMock).not.toHaveBeenCalled() diff --git a/apps/chat/test/required-mcp-route.test.ts b/apps/chat/test/required-mcp-route.test.ts index 53fa6f9c2b..f610f06ac7 100644 --- a/apps/chat/test/required-mcp-route.test.ts +++ b/apps/chat/test/required-mcp-route.test.ts @@ -5,6 +5,7 @@ const mocks = vi.hoisted(() => ({ withChatbotAuth: vi.fn(), checkDisclaimerStatus: vi.fn(), findUnique: vi.fn(), + findFirstThread: vi.fn(), getAggregatedMCPTools: vi.fn(), createThread: vi.fn(), })) @@ -24,6 +25,9 @@ vi.mock('@klicker-uzh/prisma', () => ({ chatbot: { findUnique: mocks.findUnique, }, + chatThread: { + findFirst: mocks.findFirstThread, + }, }, })) @@ -43,7 +47,7 @@ import { RequiredMCPUnavailableError, } from '../src/lib/server/mcpRuntimePolicy' -function createRequest(selectedMode?: string) { +function createRequest(selectedMode?: string, threadId?: string) { return new NextRequest('http://localhost/api/chatbots/chatbot-1/chat', { method: 'POST', headers: { 'content-type': 'application/json' }, @@ -53,6 +57,7 @@ function createRequest(selectedMode?: string) { ], selectedModel: 'gpt-4.1', ...(selectedMode ? { selectedMode } : {}), + ...(threadId ? { threadId } : {}), assistantMessageId: 'assistant-1', }), }) @@ -91,12 +96,50 @@ describe('required MCP chat preflight', () => { }, }, ], + knowledgeBases: [], }) + mocks.findFirstThread.mockResolvedValue(null) mocks.getAggregatedMCPTools.mockRejectedValue( new RequiredMCPUnavailableError() ) }) + test('refuses a thread id the caller does not own before any MCP work', async () => { + const response = await POST(createRequest(undefined, 'thread-foreign'), { + params: Promise.resolve({ chatbotId: 'chatbot-1' }), + }) + + expect(response.status).toBe(404) + await expect(response.json()).resolves.toEqual({ + error: 'Thread not found', + }) + expect(mocks.findFirstThread).toHaveBeenCalledWith( + expect.objectContaining({ + where: { + id: 'thread-foreign', + participantId: 'participant-1', + chatbotId: 'chatbot-1', + }, + }) + ) + expect(mocks.getAggregatedMCPTools).not.toHaveBeenCalled() + expect(mocks.createThread).not.toHaveBeenCalled() + }) + + test('scopes the MCP session to an owned thread id', async () => { + mocks.findFirstThread.mockResolvedValueOnce({ id: 'thread-owned' }) + + const response = await POST(createRequest(undefined, 'thread-owned'), { + params: Promise.resolve({ chatbotId: 'chatbot-1' }), + }) + + expect(response.status).toBe(503) + expect(mocks.getAggregatedMCPTools).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ sessionId: 'thread-owned' }) + ) + }) + test('returns before thread creation and forwards inactive required configs', async () => { const response = await POST(createRequest(), { params: Promise.resolve({ chatbotId: 'chatbot-1' }), @@ -113,9 +156,14 @@ describe('required MCP chat preflight', () => { server: expect.objectContaining({ isActive: false }), }), ], - 'chatbot-1', - 'participant-1', - 'account' + expect.objectContaining({ + chatbotId: 'chatbot-1', + participantId: 'participant-1', + authMode: 'account', + sessionId: expect.stringMatching( + /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i + ), + }) ) expect(mocks.createThread).not.toHaveBeenCalled() }) @@ -146,6 +194,7 @@ describe('required MCP chat preflight', () => { parameters: { required: true, toolAlias: 'doc_query' }, }, ], + knowledgeBases: [], }) const response = await POST(createRequest(), { diff --git a/apps/chat/test/threads.test.ts b/apps/chat/test/threads.test.ts new file mode 100644 index 0000000000..be4e3268f5 --- /dev/null +++ b/apps/chat/test/threads.test.ts @@ -0,0 +1,47 @@ +import { beforeEach, describe, expect, test, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + create: vi.fn(), +})) + +vi.mock('@klicker-uzh/prisma', () => ({ + prisma: { + chatThread: { + create: mocks.create, + }, + }, +})) + +import { ThreadService } from '../src/services/threads' + +describe('ThreadService.createThread', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.create.mockResolvedValue({ + id: 'thread-preallocated', + participantId: 'participant-1', + chatbotId: 'chatbot-1', + title: null, + createdAt: new Date('2026-08-23T12:00:00.000Z'), + updatedAt: new Date('2026-08-23T12:00:00.000Z'), + }) + }) + + test('persists a preallocated scope subject as the thread id', async () => { + await ThreadService.createThread( + 'participant-1', + 'chatbot-1', + null, + 'thread-preallocated' + ) + + expect(mocks.create).toHaveBeenCalledWith({ + data: { + id: 'thread-preallocated', + title: null, + participant: { connect: { id: 'participant-1' } }, + chatbot: { connect: { id: 'chatbot-1' } }, + }, + }) + }) +}) diff --git a/apps/frontend-manage/next.config.mjs b/apps/frontend-manage/next.config.mjs index d490c146d4..4b34f02869 100644 --- a/apps/frontend-manage/next.config.mjs +++ b/apps/frontend-manage/next.config.mjs @@ -23,7 +23,11 @@ let nextConfig = { } nextConfig.transpilePackages = Array.from( - new Set([...(nextConfig.transpilePackages ?? []), 'formik']) + new Set([ + ...(nextConfig.transpilePackages ?? []), + '@klicker-uzh/kb-management', + 'formik', + ]) ) if (process.env.NODE_ENV !== 'test') { diff --git a/apps/frontend-manage/package.json b/apps/frontend-manage/package.json index bfd1f28e1b..9577cf65bf 100644 --- a/apps/frontend-manage/package.json +++ b/apps/frontend-manage/package.json @@ -19,6 +19,7 @@ "@klicker-uzh/feature-flags": "workspace:*", "@klicker-uzh/graphql": "workspace:*", "@klicker-uzh/i18n": "workspace:*", + "@klicker-uzh/kb-management": "workspace:*", "@klicker-uzh/markdown": "workspace:*", "@klicker-uzh/next-config": "workspace:*", "@klicker-uzh/prisma": "workspace:*", diff --git a/apps/frontend-manage/src/components/common/Header.tsx b/apps/frontend-manage/src/components/common/Header.tsx index 0d271161a0..ed474f2ca6 100644 --- a/apps/frontend-manage/src/components/common/Header.tsx +++ b/apps/frontend-manage/src/components/common/Header.tsx @@ -42,6 +42,24 @@ function Header({ user }: { user?: User | null }): React.ReactElement { const courses = courseData?.userCourses const resourceElements: NavigationMenuItemProps[] = [ + ...(user?.privatePreview + ? [ + { + key: 'knowledge-bases-item', + type: 'link' as const, + label: t('kb.title'), + onClick: () => router.push('/resources/knowledgeBases'), + data: { cy: 'knowledge-bases' }, + }, + ] + : []), + { + key: 'knowledge-bases-item', + type: 'link' as const, + label: t('kb.title'), + onClick: () => router.push('/resources/knowledgeBases'), + data: { cy: 'knowledge-bases' }, + }, { key: 'answer-collections-item', type: 'link' as const, @@ -139,6 +157,7 @@ function Header({ user }: { user?: User | null }): React.ReactElement { label: t('manage.general.resources'), icon: faBolt, active: + router.pathname.startsWith('/resources/knowledgeBases') || router.pathname == '/resources/answerCollections' || router.pathname === '/resources/chatbots' || router.pathname === '/resources/catalog' || diff --git a/apps/frontend-manage/src/components/resources/chatbots/ChatbotDetails.tsx b/apps/frontend-manage/src/components/resources/chatbots/ChatbotDetails.tsx index 061534daaa..e3ef7488c0 100644 --- a/apps/frontend-manage/src/components/resources/chatbots/ChatbotDetails.tsx +++ b/apps/frontend-manage/src/components/resources/chatbots/ChatbotDetails.tsx @@ -463,6 +463,27 @@ function ChatbotDetails({ )} +
+
+ {t('manage.resources.knowledgeBase')} +
+ {chatbot.enabledKnowledgeBase ? ( + + {chatbot.enabledKnowledgeBase.name} + + ) : ( + + )} +
+ {chatbot.mcpConfigurations && chatbot.mcpConfigurations.length > 0 && (
diff --git a/apps/frontend-manage/src/globals.css b/apps/frontend-manage/src/globals.css index 1e933dfdd1..1a6826a7f6 100644 --- a/apps/frontend-manage/src/globals.css +++ b/apps/frontend-manage/src/globals.css @@ -9,6 +9,7 @@ @plugin "@tailwindcss/container-queries"; @source "../node_modules/@uzh-bf/design-system/src"; +@source "../../../packages/kb-management/src"; @source "../../../packages/shared-components/src"; @source "../../../packages/markdown/src"; diff --git a/apps/frontend-manage/src/pages/resources/knowledgeBases.tsx b/apps/frontend-manage/src/pages/resources/knowledgeBases.tsx new file mode 100644 index 0000000000..78e912ad47 --- /dev/null +++ b/apps/frontend-manage/src/pages/resources/knowledgeBases.tsx @@ -0,0 +1,24 @@ +import { KnowledgeBaseManager } from '@klicker-uzh/kb-management' +import { GetStaticPropsContext } from 'next' +import { useTranslations } from 'next-intl' +import Layout from '../../components/Layout' + +function KnowledgeBasesPage() { + const t = useTranslations() + + return ( + + + + ) +} + +export async function getStaticProps({ locale }: GetStaticPropsContext) { + return { + props: { + messages: (await import(`@klicker-uzh/i18n/messages/${locale}`)).default, + }, + } +} + +export default KnowledgeBasesPage diff --git a/apps/frontend-manage/src/pages/resources/knowledgeBases/[id].tsx b/apps/frontend-manage/src/pages/resources/knowledgeBases/[id].tsx new file mode 100644 index 0000000000..94c003430a --- /dev/null +++ b/apps/frontend-manage/src/pages/resources/knowledgeBases/[id].tsx @@ -0,0 +1,35 @@ +import { KnowledgeBaseDetail } from '@klicker-uzh/kb-management' +import type { GetStaticPropsContext } from 'next' +import { useTranslations } from 'next-intl' +import Layout from '../../../components/Layout' + +function KnowledgeBasePage({ kbId }: { kbId: string }) { + const t = useTranslations() + + return ( + + + + ) +} + +export async function getStaticProps({ + locale, + params, +}: GetStaticPropsContext) { + return { + props: { + messages: (await import(`@klicker-uzh/i18n/messages/${locale}`)).default, + kbId: params?.id, + }, + } +} + +export function getStaticPaths() { + return { + paths: [], + fallback: 'blocking', + } +} + +export default KnowledgeBasePage diff --git a/apps/hatchet-worker-general/.env.example b/apps/hatchet-worker-general/.env.example index 2e7947f1c9..dbeebe75d5 100644 --- a/apps/hatchet-worker-general/.env.example +++ b/apps/hatchet-worker-general/.env.example @@ -4,6 +4,30 @@ HATCHET_CLIENT_TLS_STRATEGY=none HATCHET_LOG_LEVEL=WARN HATCHET_WORKFLOWS= LOG_LEVEL=info +KB_INGESTION_API_URL= +KB_INGESTION_API_KEY= +KB_INGESTION_PROJECT_ID=klicker-course-materials +KB_SOURCE_GATEWAY_URL=__KB_SOURCE_GATEWAY_URL__ +BLOB_STORAGE_ACCOUNT_NAME=__BLOB_STORAGE_ACCOUNT_NAME__ +BLOB_STORAGE_ACCESS_KEY=__BLOB_STORAGE_ACCESS_KEY__ +# The external KB graph builder is all-or-nothing: setting any KB_GRAPH_* value +# other than the client token makes worker startup demand the complete set (plus +# KB_FALKORDB_*). The token is excluded on purpose so that deploying it alone, +# ahead of the rest of the configuration, cannot stop the whole general worker; +# it is still required once any other value is set. Keep the block commented out +# to run without graph builds; uncomment it as a whole and fill in every value to +# enable them — util/configure-local-kb-graph-builder.sh writes a working local +# set. +# KB_GRAPH_HATCHET_CLIENT_TOKEN=__KB_GRAPH_HATCHET_CLIENT_TOKEN__ +# KB_GRAPH_HATCHET_CLIENT_HOST_PORT= +# KB_GRAPH_HATCHET_API_URL= +# KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY=none +# KB_GRAPH_HATCHET_WORKFLOW_NAME= +# KB_GRAPH_TIMEOUT_SECONDS=21600 +# KB_GRAPH_STANDARD_GENERATION_MODEL= +# KB_GRAPH_STANDARD_CLEANING_MODEL= +# KB_GRAPH_HIGH_GENERATION_MODEL= +# KB_GRAPH_HIGH_CLEANING_MODEL= DATABASE_URL="postgres://klicker-prod:klicker@localhost:5432/klicker-prod" APP_SECRET="abcd" diff --git a/apps/hatchet-worker-general/package.json b/apps/hatchet-worker-general/package.json index 8f8a496257..fe3914ffb7 100644 --- a/apps/hatchet-worker-general/package.json +++ b/apps/hatchet-worker-general/package.json @@ -37,7 +37,7 @@ "dev:infisical": "../../util/_run_with_infisical.sh --env dev pnpm run dev", "dev:offline": "pnpm run dev", "dev:test": "cross-env NODE_ENV=test tsx --env-file .env src/index.ts", - "dev:ts": "cross-env NODE_ENV=development tsx --watch --env-file .env src/index.ts", + "dev:ts": "cross-env NODE_ENV=development tsx --env-file .env src/index.ts", "start:test": "node --env-file .env dist/index.js" }, "engines": { diff --git a/apps/hatchet-worker-general/src/index.ts b/apps/hatchet-worker-general/src/index.ts index 27238d27ba..7153e47e92 100644 --- a/apps/hatchet-worker-general/src/index.ts +++ b/apps/hatchet-worker-general/src/index.ts @@ -1,9 +1,16 @@ // basic structure according to https://github.com/hatchet-dev/hatchet-typescript-quickstart/tree/main/monorepo import { createRedisEventTarget } from '@graphql-yoga/redis-event-target' -import { handlers } from '@klicker-uzh/graphql' +import { handlers, settleKbKnowledgeGraphResult } from '@klicker-uzh/graphql' import type { PreparedHatchetTasks } from '@klicker-uzh/hatchet' -import { hatchetClient, prepareHatchetTasks } from '@klicker-uzh/hatchet' +import { + getKBGraphTerminalResult, + hatchetClient, + prepareHatchetTasks, + validateKBGraphWorkerConfig, + validateKBIngestionWorkerConfig, +} from '@klicker-uzh/hatchet' +import { prisma } from '@klicker-uzh/prisma' import EventEmitter from 'events' import { createPubSub } from 'graphql-yoga' import { Redis } from 'ioredis' @@ -58,6 +65,8 @@ function selectWorkflows(workflows: PreparedHatchetTasks) { } async function main() { + validateKBIngestionWorkerConfig() + validateKBGraphWorkerConfig() logger.info({ workerName: HATCHET_WORKER_NAME }, 'Starting Hatchet worker') const redisExec = new Redis({ @@ -119,6 +128,18 @@ async function main() { redisAssessmentExec, redisCache, handlers, + getKBGraphTerminalResult, + settleKBGraphTerminalResult: ({ + buildId, + result, + finishedAt, + allowLateSuccess, + }) => + settleKbKnowledgeGraphResult( + prisma, + { buildId, result, allowLateSuccess }, + finishedAt + ), }) const workflows = selectWorkflows(preparedWorkflows) diff --git a/apps/hatchet-worker-general/src/logger.ts b/apps/hatchet-worker-general/src/logger.ts index ca394f3357..515df4492b 100644 --- a/apps/hatchet-worker-general/src/logger.ts +++ b/apps/hatchet-worker-general/src/logger.ts @@ -1,4 +1,5 @@ import pino from 'pino' +import pretty from 'pino-pretty' // Service name for log base context const SERVICE_NAME = process.env.HATCHET_WORKER_NAME ?? 'hatchet-worker-general' @@ -7,32 +8,28 @@ const SERVICE_NAME = process.env.HATCHET_WORKER_NAME ?? 'hatchet-worker-general' const level = (process.env.LOG_LEVEL ?? 'info').toLowerCase() const isPretty = - (process.env.NODE_ENV !== 'production' && - process.env.PINO_PRETTY !== 'false') ?? - false + process.env.NODE_ENV !== 'production' && process.env.PINO_PRETTY !== 'false' -// Configure transport only in pretty/dev mode to avoid extra deps in prod -const transport = isPretty - ? pino.transport({ - target: 'pino-pretty', - options: { +const options = { + level, + base: { + service: SERVICE_NAME, + }, + timestamp: pino.stdTimeFunctions.isoTime, + messageKey: 'message', +} + +// Keep development formatting in-process so logging does not need a +// background transport thread. +export const logger = isPretty + ? pino( + options, + pretty({ colorize: true, singleLine: true, translateTime: 'SYS:standard', - }, - }) - : undefined - -export const logger = pino( - { - level, - base: { - service: SERVICE_NAME, - }, - timestamp: pino.stdTimeFunctions.isoTime, - messageKey: 'message', - }, - transport as any -) + }) + ) + : pino(options) export default logger diff --git a/apps/hatchet-worker-response-processor/package.json b/apps/hatchet-worker-response-processor/package.json index 887aedb06d..04a51f209d 100644 --- a/apps/hatchet-worker-response-processor/package.json +++ b/apps/hatchet-worker-response-processor/package.json @@ -34,7 +34,7 @@ "dev:infisical": "../../util/_run_with_infisical.sh --env dev pnpm run dev", "dev:offline": "pnpm run dev", "dev:test": "cross-env NODE_ENV=test ASSESSMENT_MODE=false tsx --env-file .env src/index.ts", - "dev:ts": "cross-env NODE_ENV=development tsx --watch --env-file .env src/index.ts", + "dev:ts": "cross-env NODE_ENV=development tsx --env-file .env src/index.ts", "start:test": "node --env-file .env dist/index.js" }, "engines": { diff --git a/apps/mcp-lecturer/Dockerfile b/apps/mcp-lecturer/Dockerfile index fd76ae3392..2a80e90e38 100644 --- a/apps/mcp-lecturer/Dockerfile +++ b/apps/mcp-lecturer/Dockerfile @@ -10,6 +10,7 @@ RUN npm i -g --ignore-scripts pnpm@11.5.0 turbo@2.5.6 COPY package.json pnpm-lock.yaml pnpm-workspace.yaml turbo.json ./ COPY apps/mcp-lecturer ./apps/mcp-lecturer COPY packages/prisma ./packages/prisma +COPY packages/util ./packages/util RUN turbo prune --scope=@klicker-uzh/mcp-lecturer --docker @@ -44,6 +45,7 @@ RUN adduser --system --uid 1001 nodejs COPY --from=builder /app/apps/mcp-lecturer/dist/ /app/apps/mcp-lecturer/dist/ COPY --from=builder /app/packages/prisma/dist/ /app/packages/prisma/dist/ +COPY --from=builder /app/packages/util/dist/ /app/packages/util/dist/ USER nodejs diff --git a/apps/mcp-lecturer/src/auth.ts b/apps/mcp-lecturer/src/auth.ts index ecfcbcd2f9..7b337641bf 100644 --- a/apps/mcp-lecturer/src/auth.ts +++ b/apps/mcp-lecturer/src/auth.ts @@ -1,4 +1,4 @@ -import { extractBearerToken } from '@klicker-uzh/util' +import { extractBearerToken } from '@klicker-uzh/util/auth' import type { IncomingHttpHeaders } from 'node:http' import type { RuntimeSettings } from './config.js' import { verifyLecturerJwt } from './jwt.js' diff --git a/deploy/charts/klicker-uzh-v3/templates/cm-backend-graphql.yaml b/deploy/charts/klicker-uzh-v3/templates/cm-backend-graphql.yaml index a71714c7f2..c34cfeaa58 100644 --- a/deploy/charts/klicker-uzh-v3/templates/cm-backend-graphql.yaml +++ b/deploy/charts/klicker-uzh-v3/templates/cm-backend-graphql.yaml @@ -17,6 +17,10 @@ data: {{- if .Values.backendGraphql.debug }} DEBUG: {{ .Values.backendGraphql.debug | quote }} {{ end }} + {{- if .Values.backendGraphql.kbIngestionDisabled }} + # interim kill switch: refuses new KB content ingestion while reads/deletes/serving stay live + KB_INGESTION_DISABLED: {{ .Values.backendGraphql.kbIngestionDisabled | quote }} + {{ end }} {{- if .Values.heartbeats.dailyGroupScores }} HEARTBEAT_DAILY_GROUP_SCORES: {{ .Values.heartbeats.dailyGroupScores | quote }} {{ end }} @@ -34,3 +38,9 @@ data: HATCHET_CLIENT_TLS_STRATEGY: {{ .Values.hatchet.client.tlsStrategy | quote }} # Deprecated - kept for backward compatibility during transition HATCHET_API_URL: {{ .Values.hatchet.client.apiUrl | quote }} + {{- if .Values.backendGraphql.knowledgeGraph.host }} + KB_FALKORDB_HOST: {{ .Values.backendGraphql.knowledgeGraph.host | quote }} + KB_FALKORDB_PORT: {{ .Values.backendGraphql.knowledgeGraph.port | quote }} + KB_FALKORDB_TLS: {{ .Values.backendGraphql.knowledgeGraph.tls | quote }} + KB_FALKORDB_QUERY_TIMEOUT_MS: {{ .Values.backendGraphql.knowledgeGraph.queryTimeoutMs | quote }} + {{- end }} diff --git a/deploy/charts/klicker-uzh-v3/templates/cm-chat.yaml b/deploy/charts/klicker-uzh-v3/templates/cm-chat.yaml index 10c20b231b..48c26f5bab 100644 --- a/deploy/charts/klicker-uzh-v3/templates/cm-chat.yaml +++ b/deploy/charts/klicker-uzh-v3/templates/cm-chat.yaml @@ -13,6 +13,9 @@ data: OPENAI_BASE_URL: {{ .Values.chat.openai.baseUrl | quote }} CHAT_OPENAI_STORE_RESPONSES: {{ .Values.chat.openai.storeResponses | default false | quote }} CHAT_ENABLE_AI_TELEMETRY: {{ $chatTelemetryEnabled | quote }} + DOC_QUERY_SCOPE_ISSUER: {{ .Values.chat.docQueryScope.issuer | quote }} + DOC_QUERY_SCOPE_AUDIENCE: {{ .Values.chat.docQueryScope.audience | quote }} + DOC_QUERY_SCOPE_KID: {{ .Values.chat.docQueryScope.keyId | quote }} {{- if .Values.chat.modelRegistry }} CHAT_MODEL_REGISTRY_JSON: {{ .Values.chat.modelRegistry | toJson | quote }} {{- end }} diff --git a/deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml b/deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml index 919c2de924..ec92c7887a 100644 --- a/deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml +++ b/deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml @@ -1,4 +1,7 @@ {{- $fullName := include "chart.fullname" . -}} +{{- if and .Values.hatchet.kbGraph.workflowName (empty .Values.backendGraphql.knowledgeGraph.host) }} +{{- fail "backendGraphql.knowledgeGraph.host must be configured when hatchet.kbGraph.workflowName is enabled" }} +{{- end }} apiVersion: v1 kind: ConfigMap metadata: @@ -8,6 +11,27 @@ metadata: data: HATCHET_CLIENT_TLS_STRATEGY: {{ .Values.hatchet.client.tlsStrategy | quote }} HATCHET_API_URL: {{ .Values.hatchet.client.apiUrl | quote }} + KB_INGESTION_API_URL: {{ .Values.hatchet.kbIngestion.apiUrl | quote }} + KB_INGESTION_PROJECT_ID: {{ .Values.hatchet.kbIngestion.projectId | quote }} + KB_SOURCE_GATEWAY_URL: {{ .Values.hatchet.kbIngestion.sourceGatewayUrl | quote }} + {{- if .Values.hatchet.kbGraph.workflowName }} + KB_GRAPH_HATCHET_CLIENT_HOST_PORT: {{ .Values.hatchet.kbGraph.hostPort | quote }} + KB_GRAPH_HATCHET_API_URL: {{ .Values.hatchet.kbGraph.apiUrl | quote }} + KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY: {{ .Values.hatchet.kbGraph.tlsStrategy | quote }} + KB_GRAPH_HATCHET_WORKFLOW_NAME: {{ .Values.hatchet.kbGraph.workflowName | quote }} + KB_GRAPH_TIMEOUT_SECONDS: {{ .Values.hatchet.kbGraph.timeoutSeconds | quote }} + KB_GRAPH_STANDARD_GENERATION_MODEL: {{ .Values.hatchet.kbGraph.standardGenerationModel | quote }} + KB_GRAPH_STANDARD_CLEANING_MODEL: {{ .Values.hatchet.kbGraph.standardCleaningModel | quote }} + KB_GRAPH_HIGH_GENERATION_MODEL: {{ .Values.hatchet.kbGraph.highGenerationModel | quote }} + KB_GRAPH_HIGH_CLEANING_MODEL: {{ .Values.hatchet.kbGraph.highCleaningModel | quote }} + {{- end }} + {{- if .Values.backendGraphql.knowledgeGraph.host }} + KB_FALKORDB_HOST: {{ .Values.backendGraphql.knowledgeGraph.host | quote }} + KB_FALKORDB_PORT: {{ .Values.backendGraphql.knowledgeGraph.port | quote }} + KB_FALKORDB_TLS: {{ .Values.backendGraphql.knowledgeGraph.tls | quote }} + KB_FALKORDB_QUERY_TIMEOUT_MS: {{ .Values.backendGraphql.knowledgeGraph.queryTimeoutMs | quote }} + {{- end }} + BLOB_STORAGE_ACCOUNT_NAME: {{ .Values.blobStorage.accountName | quote }} --- apiVersion: v1 kind: ConfigMap diff --git a/deploy/charts/klicker-uzh-v3/values.yaml b/deploy/charts/klicker-uzh-v3/values.yaml index ad8ec09f51..5ae49cfdf0 100644 --- a/deploy/charts/klicker-uzh-v3/values.yaml +++ b/deploy/charts/klicker-uzh-v3/values.yaml @@ -86,6 +86,20 @@ hatchet: apiUrl: "" tenantId: "" hostPort: "" + kbIngestion: + apiUrl: "" + projectId: "klicker-course-materials" + sourceGatewayUrl: "" + kbGraph: + hostPort: "" + apiUrl: "" + tlsStrategy: "" + workflowName: "" + timeoutSeconds: "21600" + standardGenerationModel: "" + standardCleaningModel: "" + highGenerationModel: "" + highCleaningModel: "" workers: general: replicaCount: 1 @@ -207,6 +221,11 @@ chat: mcp: key: '' + docQueryScope: + issuer: '' + audience: '' + keyId: '' + image: repository: ghcr.io/uzh-bf/klicker-uzh/chat pullPolicy: Always @@ -468,6 +487,12 @@ frontendControl: backendGraphql: priorityClassName: production-workload + knowledgeGraph: + host: '' + port: '6379' + tls: 'false' + queryTimeoutMs: '5000' + # Id of the GrowthBook saved group backing the lecturer beta opt-in. Leave # empty to disable the opt-in switch entirely: the setting hides itself when # the backend cannot resolve a group. diff --git a/deploy/env-uzh-prd/values.yaml b/deploy/env-uzh-prd/values.yaml index a610c5cb23..b32f91ea86 100644 --- a/deploy/env-uzh-prd/values.yaml +++ b/deploy/env-uzh-prd/values.yaml @@ -655,6 +655,7 @@ backendGraphql: appStudentDomain: pwa.klicker.uzh.ch appControlDomain: control.klicker.uzh.ch cookieDomain: .klicker.uzh.ch + # kbIngestionDisabled: true # interim kill switch — set to true to refuse new KB ingestion during an incident ingress: className: haproxy diff --git a/deploy/env-uzh-stg/values.yaml b/deploy/env-uzh-stg/values.yaml index ef35c98cba..42811dc14f 100644 --- a/deploy/env-uzh-stg/values.yaml +++ b/deploy/env-uzh-stg/values.yaml @@ -581,6 +581,7 @@ backendGraphql: appStudentDomain: pwa.klicker.stg.df-app.ch appControlDomain: control.klicker.stg.df-app.ch cookieDomain: .klicker.stg.df-app.ch + # kbIngestionDisabled: true # interim kill switch — set to true to refuse new KB ingestion during an incident ingress: className: haproxy diff --git a/docs/adr/0009-kb-owns-two-derived-projections.md b/docs/adr/0009-kb-owns-two-derived-projections.md new file mode 100644 index 0000000000..fc517563c1 --- /dev/null +++ b/docs/adr/0009-kb-owns-two-derived-projections.md @@ -0,0 +1,61 @@ +--- +type: Decision Record +title: Knowledge base owns two derived projections +description: The knowledge base owns RAG and graph projections with independent lifecycles. +timestamp: '2026-08-01' +tags: + - backend + - knowledge-base +--- + +# 9. The knowledge base owns two derived projections with independent lifecycles + +Status: Accepted (2026-07-31) + +## Context + +A knowledge base feeds two different AI capabilities: semantic RAG over a Milvus +index, and graph-backed features — GraphRAG, question generation, visualization — +over a FalkorDB knowledge graph. The obvious design, and the one actually built in +[PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206), gives each chatbot its +own graph: `ChatbotKnowledgeGraph` owns a graph-specific resource selection, its own +`selectionRevision`, and a graph named `klickeruzh:`. + +That design has two costs. Several chatbots bound to the same knowledge base each +pay for their own build of substantially the same content, and the knowledge base +stops being the single source of truth for what an AI feature knows — a chatbot's +graph can be built from a different resource set than the chatbot's own RAG. + +The two projections also behave nothing alike. Milvus ingestion is per-resource, +cheap, and continuous. Graph generation is KB-wide, expensive, performed by an +external system outside this repository, and billed to the lecturer. + +## Decision + +The knowledge base owns both projections over one resource set. Chatbots consume +them through their enabled KB binding and select nothing of their own. Multiple +chatbots bound to one KB share a single graph and a single build. + +The projections keep independent lifecycles. Milvus ingestion stays resource-scoped +and governs ordinary RAG readiness. Graph builds are KB-wide, explicitly requested, +and never scheduled. A graph build failure never blocks or regresses Milvus. + +Consistency between them is a content digest over the active serving set, not a +timestamp. A published graph that no longer matches the current digest keeps +serving and is labelled stale on lecturer-facing views. + +## Consequences + +Per-chatbot graph tailoring is given up. A lecturer who wants two chatbots to reason +over different graphs must give them different knowledge bases. + +Graph identity, build state, authorization, and build controls all move from the +chatbot to the KB, and `ChatbotKnowledgeGraph` does not survive. + +Because generation lives outside this repository, KlickerUZH owns only the +integration contract, trigger and status state, authorization, FalkorDB reads, and +the user-facing features. Graph generation must not be reimplemented here. + +[PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206) stays open as the +preserved record of the rejected chatbot-owned alternative until the replacement is +validated against it. diff --git a/docs/adr/0010-graphml-archive-recovers-falkordb.md b/docs/adr/0010-graphml-archive-recovers-falkordb.md new file mode 100644 index 0000000000..acdc136ac4 --- /dev/null +++ b/docs/adr/0010-graphml-archive-recovers-falkordb.md @@ -0,0 +1,24 @@ +--- +type: Decision Record +title: GraphML archive recovers the FalkorDB serving projection +description: Completed GraphML artifacts are durable; FalkorDB is reconstructible. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 10. GraphML archive recovers the FalkorDB serving projection + +Status: Accepted (2026-08-10) + +FalkorDB is a reconstructible serving projection rather than the durable graph +record. Every completed graph build is archived as GraphML from the first +release, and operational recovery imports and validates that artifact before +Klicker repoints publication. The lecturer beta therefore does not require +FalkorDB high availability or database backup as its source of recovery. + +This accepts a recovery interval after FalkorDB loss in exchange for simpler +beta operations. The GraphML archive, its retention policy, and the tested +restore path become production-critical; a non-empty restored graph is not +enough without build identity, source digest, provenance, and count checks. diff --git a/docs/adr/0011-catalyst-owns-knowledge-graph-runtime.md b/docs/adr/0011-catalyst-owns-knowledge-graph-runtime.md new file mode 100644 index 0000000000..44dd40392f --- /dev/null +++ b/docs/adr/0011-catalyst-owns-knowledge-graph-runtime.md @@ -0,0 +1,27 @@ +--- +type: Decision Record +title: Catalyst owns the knowledge-graph runtime, not AI ingestion +description: Klicker owns the product, Catalyst owns the KG system, and AI infrastructure owns ingestion. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 11. Catalyst owns the knowledge-graph runtime, not AI ingestion + +Status: Accepted (2026-08-10) + +Klicker owns KB product state, authorization, graph lifecycle, quota +enforcement, and the lecturer/student experience. Catalyst owns graph +generation, FalkorDB operation, the GraphML archive, graph-quality evaluation, +and KG-system end-to-end testing. AI infrastructure continues to own +data-ingestion, doc-processing, and pgvector; Catalyst consumes those services +through explicit contracts and does not import their code or operational +lifecycle. + +This split keeps the knowledge-graph product under the Klicker team without +turning AI infrastructure into the product owner or duplicating its ingestion +platform. Cross-system KG tests belong to Catalyst, while each AI +infrastructure service remains responsible for its own provider-contract and +internal tests. diff --git a/docs/adr/0012-catalyst-imports-complete-graph-history.md b/docs/adr/0012-catalyst-imports-complete-graph-history.md new file mode 100644 index 0000000000..7d1904a2a9 --- /dev/null +++ b/docs/adr/0012-catalyst-imports-complete-graph-history.md @@ -0,0 +1,25 @@ +--- +type: Decision Record +title: Catalyst imports the complete graph-runtime history +description: Preserve Patrick's authorship and visualizations before refactoring the graph runtime. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 12. Catalyst imports the complete graph-runtime history + +Status: Superseded by ADR 0016 (2026-08-10) + +The empty Catalyst repository starts by importing the complete +`kg-content-generation` Git history after that history passes a secrets and +private-data audit. The migration preserves commit authors, dates, ancestry, +and Patrick's visualizations. Production-oriented refactoring happens only in +new Catalyst commits after the import; the migration is not a clean snapshot +or a squashed rewrite. + +This keeps attribution and design context inspectable and makes a later GitHub +to GitLab move portable. It accepts inherited research history and the need for +later cleanup. The source repository and its current branch remain intact until +Catalyst proves the expected history and files are reachable. diff --git a/docs/adr/0013-klicker-reserves-and-settles-graph-cost.md b/docs/adr/0013-klicker-reserves-and-settles-graph-cost.md new file mode 100644 index 0000000000..3919da4183 --- /dev/null +++ b/docs/adr/0013-klicker-reserves-and-settles-graph-cost.md @@ -0,0 +1,29 @@ +--- +type: Decision Record +title: Klicker reserves and settles graph cost +description: Enforce semester and per-build monetary limits without storing billing details. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 13. Klicker reserves and settles graph cost + +Status: Accepted (2026-08-10) + +Klicker owns a non-sensitive monetary quota for each lecturer and semester, +stored as a currency plus integer minor units. Before graph dispatch, Klicker +atomically reserves a conservative estimated maximum and rejects work that +would exceed either the remaining semester quota or the per-build maximum. +Catalyst reports actual metered cost against the graph build id; Klicker uses +that id as the settlement key, records the actual amount idempotently, and +releases the unused reservation. + +Reservation prevents concurrent builds from overspending a shared quota while +settlement avoids charging every build at its worst-case estimate. The cost +calculation version and provider usage evidence must remain auditable because +model pricing can change. Billing-account details are not part of this ledger; +the beta maintains that sensitive association externally for UZH-issued keys, +while BYOK lecturers are billed by their own provider. Quota controls apply to +both paths. diff --git a/docs/adr/0014-beta-learns-before-quality-thresholds.md b/docs/adr/0014-beta-learns-before-quality-thresholds.md new file mode 100644 index 0000000000..0c1b24ccfc --- /dev/null +++ b/docs/adr/0014-beta-learns-before-quality-thresholds.md @@ -0,0 +1,25 @@ +--- +type: Decision Record +title: The graph beta learns before quality thresholds gate it +description: Existing tests and the canary open beta; curated evaluation gates later widening. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 14. The graph beta learns before quality thresholds gate it + +Status: Accepted (2026-08-10) + +Existing system testing and the internal production canary are sufficient to +open the feature as an explicitly labeled beta. Catalyst then builds a versioned +quality dataset of 30–50 reviewed, non-personal goldens from approved or +synthetic source documents and records local DeepEval and CI evidence. A new +quality threshold does not retroactively block the initial beta or ordinary +change requests. + +This favors real lecturer feedback and representative production evidence over +delaying beta for a speculative threshold. In return, quality evidence gates +beta widening, general availability, and explicit graph-quality claims. Hosted +reporting remains separately gated on its data boundary. diff --git a/docs/adr/0015-graphml-follows-kb-lifecycle.md b/docs/adr/0015-graphml-follows-kb-lifecycle.md new file mode 100644 index 0000000000..db33b81ef0 --- /dev/null +++ b/docs/adr/0015-graphml-follows-kb-lifecycle.md @@ -0,0 +1,22 @@ +--- +type: Decision Record +title: GraphML archive follows the knowledge-base lifecycle +description: Retain successful graph versions with the KB and purge after deletion grace. +timestamp: '2026-08-10' +tags: + - backend + - knowledge-base +--- + +# 15. GraphML archive follows the knowledge-base lifecycle + +Status: Accepted (2026-08-10) + +Every successful GraphML version remains in the recovery archive while its +knowledge base exists. Deleting the KB starts a 30-day recovery grace period; +after that deadline, maintenance purges every archived graph for that KB. +Failed or incomplete builds do not become durable archive versions. + +This retains simple, complete beta recovery history without keeping lecturer +content indefinitely after deletion. Long-term institutional archiving or a +different retention period requires a new decision before general availability. diff --git a/docs/adr/0016-catalyst-integrates-complete-graph-history.md b/docs/adr/0016-catalyst-integrates-complete-graph-history.md new file mode 100644 index 0000000000..2d599dfac5 --- /dev/null +++ b/docs/adr/0016-catalyst-integrates-complete-graph-history.md @@ -0,0 +1,29 @@ +--- +type: Decision Record +title: Catalyst integrates the complete graph-runtime history +description: Preserve the existing Catalyst stack and Patrick's graph history before refactoring. +timestamp: '2026-08-15' +tags: + - backend + - knowledge-base +--- + +# 16. Catalyst integrates the complete graph-runtime history + +Status: Accepted (2026-08-10) + +Catalyst pull requests 2 and 3 have been merged, and `main` contains later +merged work as well. The graph-runtime migration therefore starts from the +latest fetched Catalyst `main` and integrates the complete +`kg-content-generation` Git history without squashing or rewriting either +ancestry. It is delivered as an ordinary pull request; no native stack +relationship is required. Whether that pull request needs an internal split is +decided only after the W1 history inventory and size review. Patrick's +authorship, dates, and graph assets remain reachable. Production-oriented +refactoring happens only in later Catalyst commits. + +This preserves provenance on both sides and avoids replacing the active +Catalyst `main` with a clean graph snapshot. It requires an explicit integration +commit and may carry research history that later refactoring removes from the +working tree. The source repository and selected Catalyst base remain intact +until history and file coverage are verified. diff --git a/docs/adr/0017-graph-build-ledger-is-canonical.md b/docs/adr/0017-graph-build-ledger-is-canonical.md new file mode 100644 index 0000000000..683a1c147a --- /dev/null +++ b/docs/adr/0017-graph-build-ledger-is-canonical.md @@ -0,0 +1,38 @@ +--- +type: Decision Record +title: The graph-build ledger is the canonical version record +description: One append-only KBGraphBuild table answers which graph is live; derived pipelines key off its build id. +timestamp: '2026-08-18' +tags: + - backend + - knowledge-base +--- + +# 17. The graph-build ledger is the canonical version record + +Status: Accepted (2026-08-18) + +Every `KBGraphBuild` row is one build attempt and, when it succeeds, the +immutable version record of the graph it produced. The Klicker-minted build id +is the external idempotency and correlation key toward the graph runtime, and +the build history is the ledger ordered by `createdAt`. Which graph is active +or published is expressed only through the conditional-update pointers +`activeGraphBuildId` and `publishedGraphBuildId` on the knowledge base. There +is no separate graph-version table and no second version-number sequence. + +A second version entity would answer "which graph is live" independently of the +pointers and the ledger, and keeping the three consistent would become a +permanent obligation. With the ledger alone, rollback repoints +`publishedGraphBuildId` to an earlier succeeded build whose GraphML the +archive still retains (ADRs 0010 and 0015), while cost, quality tier, and +provenance stay on the same row the quota settles against (ADR 0013). +Superseding a build marks its status; it never rewrites the row. + +Derived pipelines follow the same key. The question-generation stack drops its +`KBGraphVersion` entities and its own `KBGraphBuild` shape, keys +`QuestionGenerationBuild` on the ledger's build id so it inherits the source +content digest and provenance, points its cost block at the same +`KBGraphQuota` row with a spend-class discriminator, and models review and +generated drafts as children of the generation build. A generated set that must +outlive its graph build is settled through artifact retention, not through a +second version identity. diff --git a/docs/adr/0018-providers-ship-launchers-consumers-run-e2e.md b/docs/adr/0018-providers-ship-launchers-consumers-run-e2e.md new file mode 100644 index 0000000000..a8f80d54db --- /dev/null +++ b/docs/adr/0018-providers-ship-launchers-consumers-run-e2e.md @@ -0,0 +1,43 @@ +--- +type: Decision Record +title: Providers ship a launcher, consumers run the end-to-end test +description: An AI-infrastructure service exposes a supported local launcher for its own path; the consuming system's E2E runner invokes it instead of reproducing it. +timestamp: '2026-08-19' +tags: + - backend + - knowledge-base +--- + +# 18. Providers ship a launcher, consumers run the end-to-end test + +Status: Accepted (2026-08-19) + +ADR 0011 assigns cross-system knowledge-graph tests to Catalyst and leaves each +AI-infrastructure service responsible for its own provider contract. This record +fixes the seam those two halves meet at, because a cross-system test needs the +provider actually running and there are two ways to get there. + +A provider exposes a supported, documented launcher that starts its own path +locally, with the flags and configuration knobs a caller needs and its own tests +and documentation. The consuming system's end-to-end runner invokes that +launcher. It does not assemble the provider's process set, construct the +provider's configuration, or carry a copy of either. A provider-side launcher +change lands and merges before the consumer package that depends on it, so the +consumer is written against a contract that already exists. + +The alternative was tried and withdrawn. MR !119 in data-ingestion proposed +adding the Klicker graph harness to the provider repository, which put +consumer-owned behavior behind the provider's release boundary — the thing +ADR 0011 exists to prevent. The pressure that produced it is structural rather +than a one-time lapse: whoever writes the cross-system test is the party feeling +the missing setup, and the shortest path from there is always to add it where +they are standing. Naming the launcher as the provider's deliverable gives that +pressure somewhere legitimate to go. + +Two consequences follow. A provider's shipped fixtures stay minimal and carry no +credential, so a consumer that needs capability a fixture deliberately withholds +supplies its own configuration directory rather than editing the provider's +fixture. And a capability the provider genuinely cannot offer locally stays +absent rather than being simulated: the consumer's runner records what its local +run does and does not prove, instead of relaxing a provider policy to make a +test pass. diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md index fa45e93762..783c0003b3 100644 --- a/docs/architecture-overview.md +++ b/docs/architecture-overview.md @@ -2,7 +2,7 @@ type: Architecture Overview title: Architecture Overview description: System map of apps and packages, the request path from browser to resolver, the async response pipeline, and where business logic lives. -timestamp: '2026-08-06' +timestamp: '2026-08-24' tags: - architecture --- @@ -33,7 +33,10 @@ Apps (dev ports in [Getting Started](./getting-started.md)): | `apps/olat-api`, `apps/lti`, `apps/office-addin` | LMS/Office integrations | | `apps/docs` | User-facing Docusaurus site (not this wiki) | -Packages: `graphql` (schema + services + ops — the heart), `prisma` (schema + migrations), `prisma-data` (seeds), `grading` (pure scoring math), `hatchet` (task definitions), `feature-flags` (typed GrowthBook contracts and browser/Node adapters), `types`, `util` (JWT/cookie helpers), `i18n`, `shared-components`, `markdown`, `export`, `word-cloud`, `next-config`, `transactional` (react-email). +Packages: `graphql` (schema + services + ops — the heart), `prisma` (schema + migrations), `prisma-data` (seeds), `grading` (pure scoring math), `hatchet` (task definitions), `types`, `util` (JWT/cookie helpers), `i18n`, `shared-components`, `markdown`, `export`, `word-cloud`, `next-config`, `transactional` (react-email), `knowledge-graph` (FalkorDB client, graph naming, content digests), `kb-management` (lecturer knowledge-base UI). + +FalkorDB is the serving projection for published knowledge graphs, not their durable record: each build writes its own named graph and the archived GraphML export is the recovery source ([ADR 0009](./adr/0009-kb-owns-two-derived-projections.md), [ADR 0011](./adr/0011-catalyst-owns-knowledge-graph-runtime.md)). +The `feature-flags` package provides typed GrowthBook contracts and browser/Node adapters for behavior that has migrated to that platform; existing KB preview access remains persisted per user until its consumers migrate. ## Request flow (query/mutation) diff --git a/docs/async-and-workers.md b/docs/async-and-workers.md index 2137dce885..7f53f74300 100644 --- a/docs/async-and-workers.md +++ b/docs/async-and-workers.md @@ -2,7 +2,7 @@ type: Async Architecture title: Async & Workers description: The Hatchet-based response pipeline, worker task catalog, scheduled jobs, and what silently breaks without workers. -timestamp: '2026-08-20' +timestamp: '2026-08-24' tags: - backend - hatchet @@ -46,6 +46,33 @@ Bare `http.createServer`, two routes: `GET /healthz` and `POST /AddResponse`. No - `aggregate-block-closure-*` — live-quiz block aggregation - Daily crons (`0 0 * * *`): `updateGroupAverageScores`, `runningRandomGroupAssignments`, `finalRandomGroupAssignments`, `updateWeeklyTimelineEntries` +## Knowledge-base ingestion + +`packages/hatchet/src/index.ts:prepareHatchetTasks` registers six local workflows: + +- `ingest-kb-resource` accepts the selected resource, version, and attempt identifiers, prepares the exact source bytes, then calls `packages/hatchet/src/kbIngestion.ts:dispatchKBIngestion`. Dispatch awaits `POST /v1/resources`, stores the returned operation identifier, and reuses the same version, digest, source URL, and idempotency key when an attempt is retried. Source identity and accepted-operation correlation update `KBResource` and its `KBIngestionRun` in one transaction. +- `monitor-kb-ingestions` runs every minute and calls `packages/hatchet/src/kbIngestion.ts:monitorActiveKBIngestions`. It rotates through at most 32 active operations per tick, polls `GET /v1/operations/{operation_id}` eight at a time, and applies only responses matching the local operation, resource version, and content digest. Operation state, safe error details, and active serving identity update atomically. A succeeded replacement remains `PROCESSING` with a `SUCCEEDED` run while an older version is serving; it becomes `READY` only when the observed digest and actively serving version/digest match. +- `delete-kb-resource` sends the exact canonical `DELETE /v1/resources/{external_resource_id}` request with the delete-run UUID as its stable idempotency key. Polling and webhooks fence that attempt as `DELETE`, require `expected_sha256=null`, and consider the tombstone served only when both active serving fields are null. +- `maintain-kb-resources` runs every 15 minutes with one active run. Each pass handles at most 32 items per class and at most eight concurrently: re-dispatching a live `QUEUED` UPSERT that is at least one maintenance interval old and still has no external operation id; re-enqueuing a `QUEUED` graph build that is equally stale and carries neither an external operation id nor a dispatch claim; retrying undispatched tombstones with their stable attempt; starting a freshly fenced attempt after a terminal external delete failure; removing expired unconfirmed uploads after the 24-hour grace; retiring the FalkorDB graph of a build that is neither active nor published after the 24-hour graph grace, keeping the GraphML export of any build that produced one; purging every remaining GraphML export and graph of a knowledge base 30 days after it was deleted; deleting confirmed blob storage only after the current external delete succeeded; hard-deleting those resource rows; and finally removing empty pending KBs that never built a graph. UPSERT recovery reuses the stored `ingestionAttemptId`, so the external Idempotency-Key stays stable whether the earlier process crashed before or after acceptance. The bounded windows rotate on each schedule slot so retained failures cannot starve later rows, and dispatch setup failures do not stop independent storage or row cleanup. Storage or API failures retain the exact ticket or tombstone for another pass (`packages/hatchet/src/kbMaintenance.ts:maintainKBResources`). +- `build-kb-knowledge-graph` rechecks the global graph kill switch, the persisted per-KB opt-in, and a complete cost reservation at the worker effect boundary before dispatching the active KB's immutable source manifest to the external graph workflow. It first records a conditional durable dispatch claim. A queued build that fails those gates is failed closed, releases an ordinary reservation, or holds an incomplete legacy reservation for human review; if the provider accepts a run but its id cannot be correlated and persisted, the claim keeps the reservation and active KB build slot in `NEEDS_HUMAN_REVIEW` and prevents a duplicate external start. A later retry of the same build asks the provider before parking it again, but only once the dispatch claim is older than a 15-minute in-flight grace: inside that window a duplicate task run leaves the build untouched, because a sibling attempt may still be inside the provider call and "no run yet" is not evidence that no run will start. Past the grace, a recovered run is correlated, a definitive "no run for this build id" is released as an ordinary `KB_GRAPH_DISPATCH_FAILED` that frees the quota and the slot, and only an unanswered lookup keeps the hold. The worker correlates the returned run id and never publishes an unverified graph or artifact path. +- `monitor-kb-graph-builds` runs every minute, rotates through at most 32 active builds per tick, and polls at most eight concurrently. Every provider status, result, cancellation, and ambiguity-recovery call has a ten-second deadline, so one stalled provider call cannot overrun the sweep or block independent builds; a call timeout aborts its underlying request before the concurrency slot is reused and leaves the correlated build fenced for the next tick. The monitor cancels build-timeout runs and requires a versioned terminal-result callback before settlement or publication. Provider `COMPLETED` is not sufficient. A missing callback or malformed result clears the active slot without moving the published pointer and holds the reservation as `NEEDS_HUMAN_REVIEW`; a valid non-success result with metering settles actual usage without publishing, while a non-success result without metering releases only an ordinary `RESERVED` build. A late success after a timeout is reconciled under KB and serving-resource locks: it can reclaim the slot and publish only when no newer build exists and the pinned source digest still matches; stale or superseded late results settle metered usage without publication. A malformed or late failure result remains held and every callback still passes the same identity, artifact, currency, counter, and metering checks. + +Single and bulk lecturer deletion both create their fenced runs inside the database transaction and enqueue only after commit. Bulk dispatch is bounded to eight concurrent tasks; each rejection records retry state independently so one unavailable Hatchet call cannot prevent sibling tombstones or later W5 maintenance. + +Operation events also return through the raw-body `/api/webhooks/kb-ingestion` route registered before end-user JWT middleware by `apps/backend-docker/src/kbHttpRoutes.ts:registerKBHttpRoutes`. `packages/graphql/src/services/knowledgeWebhooks.ts:handleKBIngestionWebhook` accepts the strict canonical event body and the four `X-Ingestion-*` headers, verifies an HMAC-SHA256 signature within the five-minute replay window against the current or previous webhook secret, then applies the same operation/version/digest correlation guards and atomic resource/run updates as polling. Client-initiated lifecycle events remain attempt-scoped. The distinct platform `resource.content_refreshed` event requires a non-null serving version/hash matching `resource_version`, locks the live resource, writes a terminal UPSERT ledger row correlated to `operation_id`, and advances only the active serving fields and `ingestedAt`; repeated delivery is deduplicated by operation ID and an older refresh is retained as `SUPERSEDED`. The owner resource list resolves its operation status through the resource's stored attempt rather than ledger timestamp order, so a refresh cannot display success over a concurrent lecturer operation. Later serving events can complete a successful replacement cutover; terminal run guards prevent delayed processing or failure events from regressing it. + +URL resources are registered only with public HTTP(S) destinations using ports 80 or 443 and without credentials, fragments, or secret-like query parameters. Before dispatch, every redirect hop is resolved to a public IPv4 address and fetched through that pinned address while the original public URL remains the ingestion source identity. Private blobs are exposed to the ingestion platform through the authenticated backend source gateway; no Azure storage credential or SAS URL crosses the API contract. + +Source preparation also verifies that the task's KB id matches the resource's persisted live parent. It records the exact fetched byte size. Under a parent-KB row lock, URL replacement accounting applies `current usage - previous resource size + observed size`; an over-limit candidate becomes `FAILED` with `KB_STORAGE_LIMIT_REACHED` before any external API call. Production-v1 source preparation accepts PDF and plain text; lecturer Markdown uploads are deliberately stored as `text/plain`. + +The interim backend kill switch `KB_INGESTION_DISABLED=true` is checked by `packages/graphql/src/services/knowledge.ts:assertKbIngestionEnabled`. It refuses new upload tickets, URL resources, and ingestion attempts while leaving reads, deletion, cleanup, and already-queued worker reconciliation live. The separate `KB_GRAPH_DISABLED=true` switch refuses graph opt-in and rebuild mutations; the worker checks it again before an external start, fails an unstarted queued build closed, and continues reconciling a run already accepted externally. It does not revoke an already published graph. The graph switch and cost settings must be injected into the GraphQL backend's environment; the chart currently maps only the non-secret external graph-worker connection settings. + +The general worker requires `KB_INGESTION_API_URL`, `KB_INGESTION_API_KEY`, and `KB_SOURCE_GATEWAY_URL`; `KB_INGESTION_PROJECT_ID` defaults to `klicker-course-materials`. The backend requires `KB_SOURCE_GATEWAY_KEY` and `KB_WEBHOOK_SECRET`, with optional `KB_WEBHOOK_PREVIOUS_SECRET` during webhook-key rotation. The API key, gateway key, and webhook keys are secrets and must stay outside chart ConfigMaps. + +KB graph builds use the separate `KB_GRAPH_HATCHET_*` connection and workflow settings plus `KB_GRAPH_TIMEOUT_SECONDS` and the named standard/high model pairs. The worker validates a partially configured graph integration at startup, then dispatches only a pinned build manifest and reconciles its external run. The GraphQL backend owns quota reservation and exposes `settleKbKnowledgeGraphResult` for the W1 terminal-result handoff; `prepareHatchetTasks` accepts the result-fetch and settlement callbacks so the worker never treats provider status as a publication contract. The production backend and general worker explicitly pass `getKBGraphTerminalResult` (the external Hatchet run output) and `settleKbKnowledgeGraphResult` into `prepareHatchetTasks`; omitting either adapter is not a supported runtime composition. `KB_GRAPH_HATCHET_CLIENT_TOKEN` remains in the general-worker secret; the non-secret settings belong under `hatchet.kbGraph` in the chart values. The startup gate is armed only by the ConfigMap-owned `KB_GRAPH_*` names, deliberately excluding that token: a secret rollout on its own must never fail the general worker's startup and stop every unrelated job. Once the gate is armed the token is still required, so the secret must carry it before `hatchet.kbGraph.workflowName` is set. Graph build input URLs and generated Blob SAS values must never be logged or placed in ConfigMaps. + +Both Hatchet workers intentionally run `tsx` without `--watch`; watch restarts unregister workflows during development. + ## Running locally (config-derived — verify on your machine) The Hatchet engine runs as the `hatchet` compose service using `hatchet-lite-dev` (gRPC 7077, UI 8888, no UI authentication required); workers pick up the client token automatically minted to `/config/authdisabled-token` or populated by `./util/_create_hatchet_token.sh`. Workers must see the **same `DATABASE_URL`, `APP_SECRET`, and Redis settings** as the app stack — a worker pointed at the wrong database happily processes events into nowhere. The `packages/graphql` vitest suite also requires a live Hatchet + `HATCHET_CLIENT_TOKEN` (see [Testing](./testing.md)). diff --git a/docs/chat-platform.md b/docs/chat-platform.md index 7651bc01b1..648160e74c 100644 --- a/docs/chat-platform.md +++ b/docs/chat-platform.md @@ -588,6 +588,14 @@ PostgreSQL is the only rating store. Do not mirror votes to Langfuse while the t - **Streaming failures need both client and server evidence**: a client-side generic error bubble does not distinguish a provider failure from a response-pipe failure. For staging smoke tests, correlate the browser request time with the chat pod logs and check for `failed to pipe response`, `stream.error`, and `stream.finish` before changing ingress timeouts or model routing. - **Message edits must go through the edit composer's own send** — `messageRuntime.composer.send({ startRun: true })` in `thread.tsx:EditComposer`. The public `threadRuntime.append()` normalizes a `null` parentId to "last message in the current path" (vendor `toAppendMessage`), so submitting an edit through it turns a root-message edit into a brand-new turn instead of a sibling branch and the branch pager (`branch-picker.tsx`) never shows. `startRun: true` is required because the vendor's own change gate compares only composer text/attachments and cannot see the kept-original-attachment state this app tracks outside the composer; the app-side `canSubmit` is the real change gate. +## Scoped KB retrieval + +The chat route derives the enabled knowledge-base id from the authenticated chatbot in PostgreSQL; it never accepts a client-supplied KB id. `src/services/mcpClients.ts` passes that id with the chatbot and session context only to the configured `KB` MCP server. For rollout and rollback compatibility, the new Chat runtime recognizes that server by its reserved `KB` name even when the persisted row still has its legacy auth type. It ignores all persisted credentials and always sends the scoped token. Without an enabled binding, complete signer configuration, or the exact KB server, KB tools stay unavailable while other MCP servers continue to load. + +`src/lib/server/docQueryScopeToken.ts:signDocQueryScopeToken` signs a five-minute ES256 token with `DOC_QUERY_SCOPE_PRIVATE_KEY`, `DOC_QUERY_SCOPE_KID`, `DOC_QUERY_SCOPE_ISSUER`, and `DOC_QUERY_SCOPE_AUDIENCE`. Claims bind `kb_id`, `chatbot_id`, session subject, and a unique `jti`; participant identity is intentionally absent. Scope-token requests carry only the bearer token and content type, never the legacy `Chatbot-ID` header. Existing participant-JWT MCP authentication is unchanged. + +The assistant UI registers the retrieval card through `src/components/tools-ui/rag-tool-ui.tsx:RAGToolUI`. Its registration uses `src/services/mcpScope.ts:DOC_QUERY_TOOL_NAME` (`KB_doc_query`), matching the namespaced runtime tool name. The card is localized through `pwa.chatbot.retrieval` and renders only a generic failure state; raw retrieval-service errors must never reach participants. + ## Testing The self-contained devcontainer starts the seeded local MCP fixture through diff --git a/docs/ci-and-deployment.md b/docs/ci-and-deployment.md index 25d2b35d68..1e661df217 100644 --- a/docs/ci-and-deployment.md +++ b/docs/ci-and-deployment.md @@ -2,7 +2,7 @@ type: Operations title: CI & Deployment description: PR gates, image builds, the standard-version release flow, Helm deployment reality, and what is NOT in this repo. -timestamp: '2026-08-22' +timestamp: '2026-08-24' tags: - ci - deployment @@ -61,6 +61,8 @@ Version bumps are **local and manual** via standard-version: `pnpm run release[: - **Hatchet endpoint pair**: `hatchet.client.apiUrl` in the environment values renders `HATCHET_API_URL`, while the external secret supplies `HATCHET_CLIENT_HOST_PORT`. They must resolve to the same Hatchet installation; worker health alone does not validate programmatic schedule creation over the HTTP API. Staging uses `app-hatchet-svc-api.stg-hatchet-svc.svc.cluster.local:8080`, and production uses `app-hatchet-svc-api.prd-hatchet-svc.svc.cluster.local:8080` (see [Async & Workers](./async-and-workers.md)). - **Rollout strategy**: use `RollingUpdate` in prd values; `Recreate` can leave a service with zero endpoints during slow image pulls (PDBs don't protect against Deployment-driven scale-downs). `maxUnavailable: 0` only for singletons. - `deploy/compose*` are v2-era self-hoster examples; `deploy/scripts/rollout.sh` is a legacy manual `kubectl rollout restart`. +- **KB graph builds couple two values**: `hatchet.kbGraph.workflowName` and `backendGraphql.knowledgeGraph.host` must be set together, or the chart stops at render time with an explicit `fail`. +- **KB graph token ordering**: the general worker's external secret must already carry `KB_GRAPH_HATCHET_CLIENT_TOKEN` before `hatchet.kbGraph.workflowName` is set. The token alone does not arm the worker's startup gate (so a secret rollout cannot stop unrelated jobs), but once any chart-owned `KB_GRAPH_*` value is present the token is required and startup fails without it. ## Deployment migrations diff --git a/docs/data-and-migrations.md b/docs/data-and-migrations.md index cc226e641f..e46a490558 100644 --- a/docs/data-and-migrations.md +++ b/docs/data-and-migrations.md @@ -2,7 +2,7 @@ type: Data Layer title: Data & Migrations description: Split Prisma schema, the migrate→sync→build ritual, seeding paths, typed Json fields, and schema-level gotchas. -timestamp: '2026-08-04' +timestamp: '2026-08-16' tags: - backend - prisma @@ -129,6 +129,13 @@ Json columns are typed via `prisma-json-types-generator`: a `/// [TypeName]` doc - **Prisma `Decimal` is an object, never truthy-check it** — `Decimal(0)` is truthy. Convert with a `toNumber()` helper and compare with `!= null` (pattern in `packages/graphql/src/services/chatbots.ts`). - **`Participant` email is unique per auth mode**: `@@unique([email, isSSOAccount])` means the same normalized email can exist once as manual and once as SSO. Queries by email alone can return the wrong account; blocking new cross-mode duplicates must happen in service logic (`packages/graphql/src/services/accounts.ts`). +- **One enabled KB per chatbot is a SQL invariant**: Prisma cannot express the partial unique index `KBChatbot_one_enabled_per_chatbot_key`. Preserve it in `packages/prisma/src/prisma/schema/migrations/20260825190000_kb_management_foundation/migration.sql` and any replacement migration. The migration deliberately leaves an existing KB MCP server row unchanged so the previous Chat runtime remains usable during rollout and rollback. The new runtime identifies the reserved `KB` server by name, ignores its persisted credentials, and sends only a scoped token. `packages/prisma-data/src/data/seedMCPServers.ts:seedMCPServers` reconciles new or explicitly reseeded environments to `scope_token` auth and leaves KB MCP configs disabled unless an enabled binding exists. +- **KB upload tickets are quota reservations**: `KBUploadTicket.sizeBytes` is the declared byte reservation. New tickets always persist the exact positive upload size; the database default exists only so pre-W6 ephemeral tickets migrate safely. Quota aggregates include every retained resource and ticket until W5 cleanup removes it. +- **Unknown-size KB URLs reserve the maximum**: `packages/graphql/src/services/knowledge.ts:createKbUrlResource` charges one resource plus 25 MiB under the parent lock. When the worker observes the exact body size, replacement accounting swaps that conservative claim for the measured size; never create an unmeasured URL row with a zero-byte claim. +- **KB list order and bulk locks are deterministic**: resource pagination uses immutable `(createdAt DESC, id DESC)` keys, while bulk deletion locks the live parent KB and then the selected resource UUIDs in sorted order. Preserve `createdAt` as an immutable cursor key and the KB-first lock order when extending list operations. +- **User deletion cannot rely on the KB cascade**: `packages/prisma/src/prisma/schema/knowledge.prisma:KB.owner` has `onDelete: Cascade`, which would remove KB resources and ingestion runs before external and Blob cleanup completes. There is no current user hard-delete path. Any future account-deletion/GDPR implementation must first drive each KB through its tombstone lifecycle and verify cleanup before deleting the User. +- **The source-gateway key is deliberately tenant-wide**: `packages/graphql/src/services/knowledgeSourceGateway.ts:handleKBSourceGateway` authenticates the ingestion bridge with one shared `KB_SOURCE_GATEWAY_KEY`, then resolves the Blob container from the resource's persisted KB owner. It is not a per-owner credential; a valid key plus exact eligible resource id/version crosses owner containers by design. Preserve the live BLOB/digest/status/tombstone predicate before Blob access, and treat key exposure as all-tenant blast radius. +- **KB graph cost accounting is integer-only and transactional**: `KBGraphQuota` is unique per owner and semester, and `reserveKBGraphCost` inserts the row with `ON CONFLICT DO NOTHING`, locks it, and increments `reservedMinorUnits` only after checking the configured limit. `KBGraphBuild.costStatus` is the idempotency fence, while `dispatchClaimedAt` is the durable claim that distinguishes an unattempted dispatch from a provider-accepted run whose correlation is ambiguous. A valid W1 success result settles and publishes; a valid non-success result with metering settles actual usage without publishing; a non-success result without metering releases only an ordinary reservation. Malformed, mismatched, overflowed, or cleanup-fenced results move to `NEEDS_HUMAN_REVIEW` without publishing. A timed-out late success is eligible for publication only after an atomic no-newer-build and current-digest check under the KB/resource lock order; stale or superseded results still settle usage without publication. The worker also refuses active builds unless the reservation fields and linked quota identity are complete, which is the compatibility guard for pre-accounting rows; deploy the schema with old graph dispatch drained or behind a two-phase rollout so an old writer cannot create an unreserved run during migration. Keep `meteredCost` aligned with the typed `PrismaKBGraphMeteredCost` declaration and run `prisma:sync` after editing the shared schema. ## Adjacent: export package (`packages/export`) diff --git a/docs/domain-model.md b/docs/domain-model.md index 7cf82444dc..92a5e10a64 100644 --- a/docs/domain-model.md +++ b/docs/domain-model.md @@ -2,7 +2,7 @@ type: Domain Model title: Domain Model description: Core entities (User vs Participant, Course, Element, activities), status lifecycles, and the two-track gamification system. -timestamp: '2026-08-20' +timestamp: '2026-08-24' tags: - backend - prisma @@ -57,6 +57,48 @@ Lifecycle enums: Scheduled publication/ending is executed by the Hatchet general worker — without it, SCHEDULED activities never go live (see [Async & Workers](./async-and-workers.md)). +## Knowledge bases + +Lecturer-owned knowledge bases use `KB` with child `KBResource` records (`packages/prisma/src/prisma/schema/knowledge.prisma:KB`, `packages/prisma/src/prisma/schema/knowledge.prisma:KBResource`). A resource is either a private uploaded blob or a public HTTP(S) URL. URL registration rejects credentials, fragments, secret-like query parameters, non-standard ports, and literal local, private, reserved, or IPv6 destinations through `packages/util/src/publicUrl.ts:normalizePublicHttpUrl`. Dispatch preparation resolves and pins every redirect hop to a public IPv4 address; the ingestion platform still enforces its own independent egress policy. + +The KB is the source of truth for two derived representations: a Milvus index for semantic RAG and a FalkorDB knowledge graph for GraphRAG, question generation, visualization, and other graph-based AI features. Both representations use the same KB resource set; there is no graph-specific resource selection. Chatbots consume these KB-owned representations through their enabled KB binding rather than owning either representation. + +Ownership follows three explicit system boundaries. Klicker owns KB product state, authorization, graph lifecycle, graph quota enforcement, and the lecturer/student experience. Catalyst owns graph generation, FalkorDB operation, the GraphML archive, graph-quality evaluation, and KG-system end-to-end tests. AI infrastructure owns data-ingestion, doc-processing, and pgvector. Catalyst consumes those provider contracts but does not import their code or control their operational lifecycle. This boundary is recorded in [ADR 0011](./adr/0011-catalyst-owns-knowledge-graph-runtime.md). + +Knowledge graphs are optional per KB. A lecturer-level public-beta feature flag grants permission to enable the capability, but it changes no KB by itself. A lecturer with that permission explicitly opts individual KBs into graph generation; student graph access requires both that opt-in and a successfully published graph. The backend kill switch `KB_GRAPH_DISABLED=true` blocks new opt-ins and dispatches while leaving existing read and cleanup paths available. + +The two representations have independent lifecycles. Milvus ingestion remains resource-scoped and determines ordinary RAG readiness. FalkorDB generation is an expensive, KB-wide operation performed by an external graph-generation system outside this repository, and it consumes the processed documents ingestion already produced rather than the original blobs and URLs. Because each build spends the lecturer's own AI budget, builds are never scheduled: only an explicit request from a user with KB-edit permission starts one. The worker rechecks the global switch, KB opt-in, and reservation before the external effect; an already accepted external run is still reconciled if a gate changes afterward. A graph build failure does not block or regress the Milvus representation. + +The KB's content revision is a digest over the active serving set — the `resourceId` and `activeContentSha256` of every non-deleted resource with an active hash — computed on demand rather than materialized on `KB`, so concurrent ingestion never serializes behind a single row. A resource stays in that set while a newer lecturer operation is `QUEUED` or `PROCESSING`; the pending operation state never suppresses its still-serving revision. A build request pins a KlickerUZH-generated build id, the KB id, that digest, and the per-resource hashes; the external system resolves them to processed documents and must fail the build when what it reads does not match. `KBGraphBuild` is the append-only build ledger, mirroring `KBIngestionRun`: its UUID is the build/idempotency key handed to the external system, which answers with its own operation id. `KB.activeGraphBuildId` is the single build slot and `KB.publishedGraphBuildId` names the build FalkorDB currently serves. At most one build is active per KB, claimed by conditional update, and a repeat request for the revision already building returns that build. When the KB advances mid-build the running build still finishes and publishes for its own revision; nothing is cancelled, no follow-up build is queued automatically, and KB edits are never blocked. Reconciliation uses the versioned terminal-result handoff with cron polling as backstop; provider status alone is never a publication proof: a W1-versioned terminal result must match the build, KB, owner, run, source digest, graph name, artifact, and metering identity before settlement. + +Nothing lands in FalkorDB until a build is complete: each build writes to its own recorded graph name and the published pointer moves only to a successful build, so no reader ever observes a partial graph and a failed build leaves the serving graph untouched. An operational FalkorDB graph that is no longer active or published stays through a bounded retirement grace period before KB maintenance sweeps it. A successful build's GraphML export follows the longer knowledge-base archive lifecycle: it remains while the knowledge base exists and for 30 days after deletion, and it is excluded from the resource quota. Failed or incomplete builds do not create a durable GraphML archive. + +FalkorDB is a reconstructible serving projection rather than the durable graph record. Completed GraphML artifacts form the graph archive and recovery source; a restore validates build identity, source digest, provenance, and graph counts before publication moves. The operational trade-off is recorded in [ADR 0010](./adr/0010-graphml-archive-recovers-falkordb.md). + +The restore half of that decision is not built yet: no code in this repository imports a GraphML export back into FalkorDB (roadmap W4 step 6). Until it lands, losing FalkorDB means rebuilding every affected graph from its sources at the lecturer's cost, even though the archive itself is now retained for the full knowledge-base lifetime per [ADR 0015](./adr/0015-graphml-follows-kb-lifecycle.md). Operationally: the archive protects the record, not the recovery time. + +Every successful GraphML version remains archived while its KB exists. Deleting the KB starts a 30-day recovery grace period, after which maintenance purges its archived graphs. Long-term archival beyond that beta policy requires a new general-availability decision. The lifecycle is recorded in [ADR 0015](./adr/0015-graphml-follows-kb-lifecycle.md). + +Version identity is the build ledger, not a separate version table. `KBGraphBuild` rows are append-only, each row is one attempt and the immutable record of the graph it produced, and the knowledge base's `activeGraphBuildId`/`publishedGraphBuildId` pointers are the only liveness state. Derived pipelines such as question generation key off the build id instead of minting a second version identity. This is recorded in [ADR 0017](./adr/0017-graph-build-ledger-is-canonical.md). + +Graph quota, AI credentials, and billing are separate concerns. Klicker enforces a non-sensitive per-lecturer, per-semester monetary quota and a per-build maximum in integer minor units, with persisted usage counters bounded to the database integer range. `KBGraphQuota` is locked while a reservation is created or settled; each build records `RESERVED`, `SETTLED`, `RELEASED`, or `NEEDS_HUMAN_REVIEW` so duplicate terminal delivery cannot double-charge. Before dispatch it reserves the estimated maximum cost and durably claims the provider-dispatch phase; an accepted run whose id cannot be correlated retains its reservation for human review and is not externally retried. Catalyst later reports actual metered cost against the build id and Klicker settles the reservation idempotently. A valid non-success result with metering settles actual usage without publishing, while an unmetered non-success releases only an ordinary reservation. A malformed, mismatched, over-reserved, overflowed, or unmetered success result holds the reservation for human review and never publishes. After a timeout, a matching late success can reclaim and publish only when no newer build exists and the current active-resource digest still matches; stale or superseded late results settle usage without publication. Cleanup claims also fence successful late results, so an artifact that is already being deleted cannot be resurrected as published. Automatic release only closes an ordinary `RESERVED` build; a later valid callback may reconcile a held build once before cleanup, while another malformed or failed result remains held. The lecturer config keeps persisted quota currency separate from historical build-cost currency and reports quota currency/limit drift as unavailable. This accounting contract is recorded in [ADR 0013](./adr/0013-klicker-reserves-and-settles-graph-cost.md). For UZH-issued credentials, sensitive lecturer-to-cost-account information stays outside the Klicker database and is maintained manually in a spreadsheet for the beta. BYOK lecturers are billed by their own provider, while Klicker quota controls still apply. AI-provider credentials are a reusable platform concern shared by every AI feature, not part of the knowledge-graph model. Consumer applications retain only opaque handles and safe status; the generic custody and runtime-resolution design remains a separate work item. + +The lecturer sees the cost boundary before spending: estimated maximum cost, remaining semester quota, and worst-case resulting balance. After Catalyst settles the build, the lecturer sees actual usage and cost. BYOK is identified as provider-billed; UZH-issued usage is identified as semester-billed. + +The initial release is explicitly a beta and may open after the existing system tests and internal production canary pass. Curated real-model evaluation is a beta learning loop rather than an entry gate: Catalyst versions 30–50 reviewed, non-personal goldens from approved or synthetic sources and starts with local reports. That evidence gates widening, general availability, and graph-quality claims, as recorded in [ADR 0014](./adr/0014-beta-learns-before-quality-thresholds.md). + +The last successfully published graph may remain available after the KB's active content revision advances, including after a resource is deleted or withdrawn. Staleness is the mismatch between the graph's pinned source digest and the current KB digest; timestamps alone are not the consistency contract. Klicker does not disable the graph automatically, and it does not surface staleness to students: the label appears only on the lecturer-facing KB and graph views, where the people who can spend a rebuild are the ones who see it. A provider `COMPLETED`, `FAILED`, `CANCELLED`, or timeout observation without the versioned result handoff clears the active slot, does not move the published pointer, and holds the reserved cost for human review; a later valid result may settle the ledger but still cannot publish without passing the same identity checks. + +Resources move through `ADDED → QUEUED → PROCESSING → READY | FAILED`. `KBResource` stores the latest operation identity (`resourceVersion`, exact-byte `contentSha256`, attempt, and external operation), the independently active serving identity (`activeResourceVersion` and `activeContentSha256`), and the latest safe error code. `KBIngestionRun` is the append-only, resource-scoped ledger: lecturer dispatch uses the local attempt UUID, while a signed platform `resource.content_refreshed` event uses its event UUID and records the platform operation ID. A refresh advances only the serving identity, so it cannot overwrite a concurrent lecturer operation; the resource list and its status filter resolve through the stored lecturer attempt rather than the newest ledger row. A failed replacement therefore remains visible without erasing the previously active serving version. Ingestion transport and atomic status reconciliation are described in [Async & Workers](./async-and-workers.md). + +Deletion is asynchronous and fenced by `deletedAt`/`deletedById` on both `KB` and `KBResource`. Owner queries hide tombstones immediately, while a `DELETE` ingestion run advances the resource version and retains local correlation state until the external serving version and digest are both empty. `KBUploadTicket` persists every blob-scoped upload grant with the same 15-minute expiry; confirmation atomically consumes it after creating the resource. The restrictive KB relation keeps pending tickets discoverable while abandoned blobs wait through the 24-hour retention grace. + +`KBChatbot` is the typed ownership link between a knowledge base and a chatbot. A chatbot may retain historical disabled links, but the partial unique index `KBChatbot_one_enabled_per_chatbot_key` permits at most one enabled knowledge base per chatbot. The corresponding KB MCP configurations are derived runtime state, not the ownership relation itself. + +Each KB retains at most 100 resource allocations and 500 MiB. Quota accounting includes hidden resource tombstones and unconsumed upload tickets, so asynchronous cleanup and concurrent upload requests cannot free or oversubscribe capacity early. A ticket reserves its declared bytes; confirmation converts that reservation into a resource without double counting. URL bytes become known during source preparation and atomically replace that resource's previous measured size under the parent-KB lock. + +Lecturer-facing metrics are derived from these rows rather than stored counters. They distinguish visible resources and known bytes from retained quota usage, conservative 25 MiB reservations for legacy unknown-size rows, upload reservations, pending cleanup, and enabled chatbot consumers. The catalog computes the same measures with bounded grouped queries for one page of owned KBs. + ## Course deletion **Deleting a non-assessment course does not normally delete its live quizzes.** diff --git a/docs/frontend-conventions.md b/docs/frontend-conventions.md index 989b12abe8..0a424d7725 100644 --- a/docs/frontend-conventions.md +++ b/docs/frontend-conventions.md @@ -2,7 +2,7 @@ type: Frontend Conventions title: Frontend Conventions description: Shared conventions for manage, pwa, control, and auth — design system, Apollo with generated ops, i18n, Formik, data-cy, and CSP rules. -timestamp: '2026-08-20' +timestamp: '2026-08-24' tags: - frontend --- @@ -76,6 +76,29 @@ contract is unchanged. Apollo Client with **generated documents only** — `import { UserProfileDocument } from '@klicker-uzh/graphql/dist/ops'`; never inline `gql`. Standard query guard: `if (!data?.field) return `. Mutations declare `refetchQueries`. New/changed ops require the codegen ritual ([API layer](./graphql-api-layer.md)). Server state lives in Apollo cache; local state in React hooks. The PWA additionally uses **localforage** as an offline side-channel for live-quiz answers (`apps/frontend-pwa/src/components/liveQuiz/storageHelpers.ts`). +## Knowledge-base management + +The lecturer routes `apps/frontend-manage/src/pages/resources/knowledgeBases.tsx:KnowledgeBasesPage` and `apps/frontend-manage/src/pages/resources/knowledgeBases/[id].tsx:KnowledgeBasePage` mount the buildless `@klicker-uzh/kb-management` package inside the authenticated manage layout. The dynamic detail route uses `getServerSideProps`; its arbitrary database ids are resolved per request rather than through empty build-time paths. Keep reusable KB UI in that package rather than duplicating it in the host app. + +`apps/frontend-manage/src/components/common/Header.tsx:Header` shows the KB navigation item only for `user.privatePreview`. That client gate is discoverability only: direct catalog/detail URLs rely on the service's fresh database guard and render the localized `KB_PREVIEW_ACCESS_REQUIRED` message. GrowthBook course-cohort gating remains deferred; do not treat this interim per-account flag as the final rollout model. + +The catalog uses server search and cursor-driven “load more” rather than loading all owned KBs. The detail page keeps metadata/metrics separate from `packages/kb-management/src/components/KnowledgeBaseResourceList.tsx:KnowledgeBaseResourceList`, which owns server search plus design-system type/status filters, selection, confirmed bulk deletion, the source inspector, and contextual Ingest/Retry/Re-ingest/Delete actions. While any loaded row is `QUEUED`/`PROCESSING`, the two-second interval fetches page zero plus pages known to contain active rows and runs a full loaded-window walk every tenth tick. Cursor or page-length drift triggers an immediate full walk. Promise-only polls use `ApolloClient.query` with `no-cache`; generation fencing, the latest loaded-count ref, and shared cache merge preserve the loaded window and remove rows from selection when they become active. Show indeterminate real-operation progress and safe-to-leave messaging rather than invented percentages. + +The inspector loads the owner-checked five-attempt history lazily. Full attempt history must stay outside the two-second list poll. Lecturer-facing failure detail is localized from stable status/error codes; raw platform messages are not rendered. Transport tuning is not user-controlled. Changes must preserve EN/DE messages, `data-cy` hooks, keyboard/focus behavior, and browser evidence for desktop plus 390 px mobile states, including search/filter, selection/confirmation, empty, active, ready, failed, and replacement-cutover feedback where affected. + +KB and resource deletion dialogs explain the two observable phases: the item disappears immediately, while stored files and the external index are removed in the background. Success toasts confirm removal from the lecturer view without claiming that external cleanup has already completed. + +KB mutations and their follow-up refreshes are separate outcomes (`packages/kb-management/src/refreshAfterMutation.ts:refreshAfterMutation`). Once a mutation succeeds, show its success state and close/reset the form even if a best-effort list or metrics refresh fails; log that refresh error without converting the successful mutation into an error toast or a retryable mutation. + +The KB file picker exposes only the production ingestion contract: PDF, TXT, and MD up to 25 MiB. Markdown is uploaded as `text/plain`; do not re-add DOCX/PPTX until the external ingestion platform supports them. Stable quota error codes are localized rather than exposing worker messages. + +`packages/kb-management/src/components/KnowledgeBaseChatbotBindings.tsx:KnowledgeBaseChatbotBindings` owns the single-enabled-KB binding UI. Replacing an existing chatbot binding requires an explicit warning step; detach is available from the current KB. `apps/frontend-manage/src/components/resources/chatbots/ChatbotDetails.tsx:ChatbotDetails` shows the reciprocal linked-KB state or an actionable no-KB warning. + +The detail metrics distinguish visible data, quota usage, upload reservations, pending asynchronous cleanup, unknown-size conservative reservations, and linked consumers. Do not present tombstoned storage as already released or treat derived values as mutable counters. + +`packages/kb-management/src/components/KnowledgeGraphPanel.tsx:KnowledgeGraphPanel` is the lecturer-facing graph lifecycle boundary. It exposes the per-KB opt-in, standard/high estimate, maximum reservation, billing mode, reservation status, remaining quota, worst-case balance, settled cost, actual token/request usage, and the localized safe status state. Quota amounts use the persisted quota currency, while historical build cost uses its recorded build currency; a persisted quota currency/limit mismatch makes the cost configuration unavailable until reconciled. The rebuild action stays disabled while the KB is opted out, the global graph switch leaves cost configuration incomplete, or a build is active. Display billing and reservation statuses through localized labels rather than raw enum values or backend status prose, keep provider credentials out of the client, and preserve the `data-cy` hooks for the switch, cost block, status, and rebuild action. + +The shared graph viewer keeps fixed hooks for search input/submit, search results, loaded nodes, loaded relationships, details close, expand, zoom, fit, reset, and retry actions. Node and relationship identifiers remain content, not selector names; combine the fixed hook with accessible name or visible text when a test must distinguish one item. The manage Elements and Activities lists use the shared `Pagination` control with finite `10`, `20`, and `50` page sizes plus an opt-in `All` value. `All` keeps the active filters and sort, resets to page 1, omits `numEntries` and @@ -106,10 +129,11 @@ Namespaces are per-app plus `shared` (`shared`, `auth`, `pwa`, `manage`, `contro ## Gotchas absorbed from experience - **Feature flags gate alone.** Don't combine a flag with data-dependent counts (`flag && count > 0`) — that creates chicken-and-egg visibility problems. - - **Legacy active preview fields:** - - `privatePreview` (User-profile level): Gates advanced beta features such as element/activity sharing, microlearning, and administrator panels. Managed via the admin page (`apps/frontend-manage/src/pages/admin.tsx`). + - **Active Feature Flags:** + - `privatePreview` (User-profile level): Gates advanced beta features such as knowledge-base management, element/activity sharing, microlearning, and administrator panels. Managed via the admin page (`apps/frontend-manage/src/pages/admin.tsx`). The KB service reads the database value per request, so disabling it does not require re-login. - `publicPreview` (User-profile level): Gates general preview features like microlearning analytics and new evaluation navigation interfaces. - The behavior-free `@klicker-uzh/feature-flags` GrowthBook foundation is available for incremental migration, but an existing preview field remains authoritative until all consumers for that behavior move. See [Feature Flags](./feature-flags.md) and [ADR 0008](./adr/0008-use-growthbook-for-feature-flags.md). + - _Interim KB rollout_: `privatePreview` remains the authoritative per-account gate for KB management until its consumers migrate to the GrowthBook foundation. - **CSP `frame-ancestors` is set at the proxy, never in Next.js middleware.** Middleware CSP breaks `_next/data` routes in production builds (known Next.js bug). Production: HAProxy ingress annotations (`haproxy.org/response-set-header` in `deploy/charts/klicker-uzh-v3/templates/ingress-*.yaml`); local: Traefik `customResponseHeaders` (`util/traefik/rules_docker.yaml`). - **Embedded PWA messaging**: use a parent-initiated `postMessage` handshake to capture `event.origin`; no `'*'` target origins and no second per-platform allowlist in page code — embedding permission is enforced by ingress `frame-ancestors`. - **Local embed testing**: `util/embed-harness/` must target the branch-local PWA (`http://127.0.0.1:3101/...`), not the production PWA — production CSP blocks localhost embedding. diff --git a/docs/graphql-api-layer.md b/docs/graphql-api-layer.md index 61913466de..afed1175b5 100644 --- a/docs/graphql-api-layer.md +++ b/docs/graphql-api-layer.md @@ -2,7 +2,7 @@ type: API Layer title: GraphQL API Layer description: Pothos code-first schema, the three-layer authorization pattern, service contract, operation naming, and the codegen ritual. -timestamp: '2026-08-21' +timestamp: '2026-08-24' tags: - backend - graphql @@ -13,6 +13,7 @@ tags: > **Migration in flight (2026-07):** a dual GraphQL→tRPC migration is open as PR #5132 (not yet merged) — a tRPC API in `packages/api` mounted at `/api/trpc` beside `/api/graphql`, with frontends moving to React Query app by app. This page describes current reality and stays authoritative until that PR merges; before extending the API surface, check the PR's status and which surface your target app uses. Staged doc/skill changes: `project/plans_future/2026-07-07-wiki-skills-migration-roadmap.md`. **The pattern to copy exactly: resolvers are one-liners; authorization is three explicit, named layers.** Protected single-object fields in `packages/graphql/src/schema/` compose the same three pieces — declare the role with `t.withAuth(...)`, check object-level permission with `withPermission(...)`, and let the service do the work. Multi-object batch fields are the explicit exception described below. Deviating from these shapes (inline logic in resolvers, unbounded service checks) is the number-one review flag. +Owner-only aggregates such as `KB` enforce their persisted owner relation at the service boundary because they have no sharing primitive. Resolvers still delegate immediately rather than implementing authorization or business logic inline. ## Three-layer authorization @@ -20,6 +21,8 @@ tags: 2. **Object-level permission — `withPermission(argsToCheck, PermissionLevel, resolver)`** (`packages/graphql/src/services/sharing.ts:withPermission`). Maps resolver args to a `PermissionCheck` (one of `courseId | liveQuizId | practiceQuizId | microLearningId | groupActivityId | elementId | answerCollectionId | catalogCollectionId`) and a required `PermissionLevel`. **On failure it returns `null` instead of throwing** — clients see a null field, not an error. A multi-object batch field cannot use this single-selector wrapper: gate the field with `t.withAuth(...)`, then perform a bounded service query and an explicit permission check for every unique object before mutation. Return per-object outcomes instead of collapsing the batch to one nullable field. 3. **Derived-permission lookup — `checkAccess`** (same file): resolves ownership and sharing grants (`DerivedPermission`) for the target object. +`PermissionCheck` has no KB key because knowledge bases are not shareable aggregates. KB schema fields use the appropriate `t.withAuth(...)` scope, then every service query or mutation resolves the KB through `ownerId: ctx.user.sub` or an equivalent persisted owner relation before reading or mutating it. Do not add a fake permission mapping or widen KB sharing to make its resolver look like a course resolver. + Worked examples: `deleteCourse` in `mutation.ts` (asUser + ADMIN permission on courseId, plus a nullable boolean that preserves the existing behavior when omitted), `controlCourse` in `query.ts` (EXECUTE), `getLiveQuizSummary` (READ). @@ -49,6 +52,26 @@ pnpm --filter @klicker-uzh/graphql generate and **commit the regenerated outputs** (`src/ops.ts`, `src/ops.schema.json`, `src/public/schema.graphql`, `src/public/client.json`, `src/public/server.json`) in the same change. They are git-tracked and load-bearing: frontends import typed documents from `@klicker-uzh/graphql/dist/ops`, and outside dev/test the backend only executes hashes present in `server.json` (see [Architecture Overview](./architecture-overview.md)). Stale artifacts fail in two distinct ways: typecheck errors (missing document) or runtime persisted-query rejection (unknown hash). +`getUserKbsConnection` and `getKbResources` are owner-scoped cursor connections with a maximum page size of 50 and exact `totalCount`. KBs use `(updatedAt DESC, id DESC)`; resources use immutable `(createdAt DESC, id DESC)` so operation polling cannot move rows between pages. Opaque cursors are bound to the owner and normalized search/filter set and reject malformed, foreign, or mismatched reuse. Resource page and count predicates reassert the live, non-deleted owned parent relation after the initial authorization check. Search runs server-side across KB name/description or resource title/filename/URL, with resource type and current-operation-status filters. `getKb` returns metadata and exact derived usage/consumer metrics rather than an unbounded child list. The former unbounded `getUserKbs` field and the misleading nested `KB.resources` field are not exposed; callers use the bounded connections. + +The resource connection includes only the run identified by each row's stored `ingestionAttemptId`, which is the lecturer's current operation. A signed platform refresh appends its own historic ledger row without changing that projection or its status filter. Full attempt history remains the separate owner-checked `getKbResourceIngestionRuns` query: it returns at most the five newest runs and is requested only from the inspector. Do not nest full history under the polled connection or expose the unbounded ledger. + +Knowledge-base/chatbot binding uses `getKbChatbotBindings`, `attachKbToChatbot`, and `detachKbFromChatbot`. The query and mutations are owner-scoped, attach/detach require full-access scope, and `packages/graphql/src/services/knowledge.ts` locks both owner rows before replacing a binding. Attach atomically enables the one selected link and reconciles exactly the `tutor` and `explainer` KB MCP configurations; detach disables those configurations when no enabled link remains. + +Every knowledge-base service entry point starts with `packages/graphql/src/services/knowledge.ts:assertKbPreviewAccess`, which reads the current `User.privatePreview` value by primary key on each request and returns `KB_PREVIEW_ACCESS_REQUIRED` when disabled. This is an interim per-account gate, independent of the login token. The separate `assertKbIngestionEnabled` kill switch reads `KB_INGESTION_DISABLED` at call time and blocks only upload-ticket issue, URL-resource creation, and Ingest/Retry/Re-ingest with `KB_INGESTION_DISABLED`; reads, upload confirmation, deletion, and chatbot binding remain available. + +The graph lifecycle has two additional gates. `KB_GRAPH_DISABLED=true` blocks graph opt-in and rebuild mutations, while `KB.knowledgeGraphEnabled` is required before a build can reserve quota or be served to chatbot students. The worker rechecks both gates and a complete `RESERVED` cost ledger, including its linked quota identity, immediately before starting the external run; it claims `dispatchClaimedAt` before the provider call, and an accepted-but-uncorrelated run is held for review rather than externally retried. An unstarted gated build fails closed and releases its ordinary reservation, while an incomplete pre-accounting row is held for review. An accepted-but-uncorrelated build keeps the active KB build slot fenced, and a rebuild mutation refuses to start a second external run; that hold is a waiting state rather than an operator task, because `packages/hatchet/src/kbGraphIngestion.ts:resolveAmbiguousKBGraphDispatch` asks the provider again on every graph-monitor tick and either correlates the recovered run or, once the provider definitively reports no run for that build id, releases the reservation and frees the slot as an ordinary `KB_GRAPH_DISPATCH_FAILED`. `setKbKnowledgeGraphEnabled` validates the cost configuration before enabling a KB. Rebuild reserves the configured estimate in minor currency units under the owner-semester quota lock, and the external monitor never publishes from provider status alone: `settleKbKnowledgeGraphResult` accepts only a W1-versioned terminal result whose build, KB, owner, run, source digest, graph name, artifact, currency, bounded counters, and metering match the reservation. A valid success settles and publishes; a valid non-success result with metering settles actual usage without publishing; an unmetered non-success releases only an ordinary `RESERVED` build. A timed-out success may publish only after settlement atomically confirms no newer build and a matching current KB digest; stale or superseded late results settle usage without publication. Settlement is fenced by `KBGraphBuild.costStatus` and cleanup claims; invalid results become `NEEDS_HUMAN_REVIEW` and retain the reservation. The config query selects the newest graph attempt for lifecycle and cost fields, while it resolves `isStale` only from a verified successful published build, so a held or charged rebuild remains visible without changing the served pointer. The lecturer config reports persisted quota-currency/limit drift as unavailable and keeps historical build-cost currency separate from quota display. + +Knowledge-base deletion is an immediate visibility change, not synchronous storage removal. Resource and whole-KB delete mutations lock the parent KB first, retain owner-attributed tombstones, create explicit `DELETE` runs, and queue external deletion after commit. Whole-KB deletion also disables its chatbot links and KB MCP configurations. Upload-ticket issue, confirmation, URL creation, and deletion use the same parent lock so no live child can appear beneath a tombstoned KB; queue failure records only an opaque retry state and never restores visibility. + +`deleteKbResources` accepts 1–50 unique resource UUIDs from one owned KB. It locks the parent and sorted child ids, rejects the whole selection when any row is missing, foreign, or active, creates one independently retryable delete run per row in a single transaction, and dispatches each operation only after commit. A post-commit dispatch failure does not roll back sibling tombstones; W5 maintenance retries the correlated failed dispatch. + +The same parent lock serializes quota allocation. A KB permits 100 retained-or-reserved resources and 500 MiB of retained-or-reserved bytes. Upload requests reserve their exact count and bytes, confirmation consumes the matching reservation, and URL creation reserves one resource plus the conservative 25 MiB unknown-size claim before the worker measures its exact size. Mutation failures use stable `KB_RESOURCE_LIMIT_REACHED`, `KB_STORAGE_LIMIT_REACHED`, and `KB_UPLOAD_TICKET_MISMATCH` codes. Klicker derives ingestion `kb_id` only from owner-checked persisted relations; platform-side validation against a registered per-project set remains a separate deployment gate. + +The authenticated source gateway in `packages/graphql/src/services/knowledgeSourceGateway.ts:handleKBSourceGateway` intentionally uses one system-to-system `KB_SOURCE_GATEWAY_KEY`, not an end-user or per-owner credential. A caller holding that key and an exact resource id/version can read any tenant's eligible live BLOB source; the gateway derives the owner container from the persisted KB relation and requires a non-tombstoned resource with a digest in `QUEUED` or `PROCESSING` before Blob Storage access. Treat key exposure as an all-tenant source-read incident and do not describe this boundary as caller-owner authorization. + +`packages/prisma/src/prisma/schema/knowledge.prisma:KB.owner` still uses `onDelete: Cascade`. There is no current user hard-delete path, but a future account-deletion or GDPR workflow must drain every KB through the tombstone/external/blob cleanup lifecycle before deleting the User; relying on the database cascade would remove the reconciliation rows and orphan external or Blob state. + ### Assessment invitation API The lecturer invitation surface is intentionally course-scoped: `assessmentParticipantInvitations`, `createAssessmentParticipantInvitations`, and `deletePendingAssessmentParticipantInvitation` all combine the USER role with course `ADMIN` permission; mutations additionally require `FULL_ACCESS` login scope (`packages/graphql/src/schema/query.ts:assessmentParticipantInvitations`, `packages/graphql/src/schema/mutation.ts:createAssessmentParticipantInvitations`). The service rejects non-assessment courses and scopes deletion by both invitation id and course id. Bulk creation returns per-row statuses plus aggregate counts so one malformed email does not discard valid rows, while unexpected database failures propagate as GraphQL errors (`packages/graphql/src/schema/participantInvitation.ts:CreateAssessmentParticipantInvitationsPayload`). Auto-acceptance requires exactly one active participant behind verified eligible accounts, preserves `Participation.isActive`, and accepted invitation metadata is immutable. diff --git a/docs/solutions/best-practice/repeat-production-seeds-use-prior-state.md b/docs/solutions/best-practice/repeat-production-seeds-use-prior-state.md index d9c36a0c2d..1193b3cea2 100644 --- a/docs/solutions/best-practice/repeat-production-seeds-use-prior-state.md +++ b/docs/solutions/best-practice/repeat-production-seeds-use-prior-state.md @@ -1,4 +1,7 @@ --- +type: Best Practice +title: Repeat Production Seeds Must Validate Prior Mutable State +description: Validate the exact mutable state that a follow-up production seed extends and bind dry-run evidence to the payload. module: prisma-data date: 2026-07-16 problem_type: best_practice diff --git a/docs/superpowers/plans/2026-07-20-external-kb-hatchet-bridge.md b/docs/superpowers/plans/2026-07-20-external-kb-hatchet-bridge.md new file mode 100644 index 0000000000..f0caa9f098 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-external-kb-hatchet-bridge.md @@ -0,0 +1,1085 @@ +--- +type: Implementation Plan +title: External KB Hatchet Bridge Implementation Plan +description: Implement the POC bridge from Klicker resource ingestion to an external Hatchet workflow. +timestamp: '2026-07-20' +tags: + - kb + - hatchet + - ingestion +--- + +# External KB Hatchet Bridge Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Extend the existing KB POC so one resource and a user-selected speed mode are dispatched through Klicker's existing local Hatchet task to a workflow on a separately configured Hatchet instance, then monitored by one local minute cron until the existing signed webhook advances the UI to `READY` or `FAILED`. + +**Architecture:** GraphQL atomically creates a latest-attempt UUID and queues the unchanged local `ingest-kb-resource` task. That task keeps its current privacy-safe log, creates a temporary blob read SAS only when required, and uses a lazy second Hatchet SDK client to start or recover the external run. PostgreSQL stores only the latest attempt/run metadata. A non-overlapping local `monitor-kb-ingestions` cron discovers all active runs from PostgreSQL, checks them sequentially through the external SDK, and calls Klicker's existing signed webhook with the attempt UUID as a stale-write guard. + +**Tech Stack:** TypeScript 5.6, Node.js 24, Prisma 6/PostgreSQL, Pothos GraphQL, React/Next.js/Apollo, `@hatchet-dev/typescript-sdk` 1.9.4, `@azure/storage-blob` 12.25.0, Vitest 3.2, Helm/Kubernetes, pnpm/Turborepo. + +## Global Constraints + +- Preserve the implementation record in `project/2026-07-15-pr-5174-kb-poc-plan.md`. This is a follow-on bridge, not a rewrite of that plan. +- Do not remove or replace the S5 `KB ingestion dispatch stub` log in `packages/hatchet/src/index.ts`. The external dispatch happens after that awaited log. +- Trigger exactly one external source: the resource whose row-level Ingest button was clicked. +- Use the stable KB UUID as `course_id`, the stable resource UUID as `source_id`, and `klickeruzh:` as `falkordb_graph_name`. +- Keep `upload_markdown` and `export_to_falkordb` fixed to `true`. +- Keep KB containers private. Do not change media-library visibility or move KB blobs into media containers. +- Mint blob-scoped read-only, HTTPS-only SAS URLs for one hour with five minutes of clock-skew allowance. Include the approved comment that the duration may need adjustment for larger files or slower workflows. +- Do not persist or log SAS URLs, access keys, Hatchet tokens, webhook secrets, or raw external SDK errors. +- Normalize URL resources through the shared public-HTTP URL guard at registration and immediately before dispatch. Reject credentials and local/private/reserved literal destinations; deployment testing must also prove DNS and redirect egress controls in the external namespace. +- Do not change the external Python workflow and do not require it to call Klicker's webhook. +- Store only the latest attempt/run metadata; do not add an ingestion-history table, outbox, or webhook inbox. +- Default speed mode to `balanced` in the UI and do not persist it as a KB resource preference. +- Use one cron sweep per minute, not one sleeping or scheduled task per resource. +- Default `KB_INGESTION_TIMEOUT_SECONDS` to `3600`; reject a present non-positive or non-integer value at general-worker startup. +- Keep external addresses, workflow name, and credentials environment-configured. Do not hard-code Kubernetes namespaces, service names, tenant IDs, or tokens. +- Use `apply_patch` for hand-authored edits, Prettier for formatting, and conventional commits after each independently green task. +- Do not modify dependencies outside the exact additions listed in Task 4. + +--- + +## File Structure + +### New files + +- `packages/prisma/src/prisma/schema/migrations/20260720120000_kb_external_ingestion_bridge/migration.sql` — nullable latest-attempt columns on `KBResource`. +- `packages/util/src/kbWebhook.ts` — server-safe, byte-exact webhook signing helper shared by GraphQL and the worker. +- `packages/util/test/kbWebhook.test.ts` — signing contract tests independent of GraphQL/database setup. +- `packages/hatchet/src/kbIngestion.ts` — external-client configuration, SAS generation, exact payload creation, retry recovery, webhook sending, and singleton sweep logic. +- `packages/hatchet/test/kbIngestion.test.ts` — deterministic bridge and sweeper tests with mocked Azure/Hatchet/HTTP boundaries. +- `packages/hatchet/vitest.config.ts` — node Vitest configuration matching other server packages. + +### Modified files + +- `packages/prisma/src/prisma/schema/knowledge.prisma` — `ingestionAttemptId`, `externalWorkflowRunId`, and `externalWorkflowStartedAt`. +- `apps/analytics/prisma/schema/knowledge.prisma` — ignored local mirror refreshed by `pnpm run prisma:sync`; verify it but do not stage it. +- `packages/util/src/index.ts` — export the shared signing helper. +- `packages/graphql/src/services/knowledgeWebhooks.ts` — import/re-export the helper and require matching attempt IDs. +- `packages/graphql/test/knowledgeWebhooks.test.ts` — current/stale attempt transition coverage. +- `packages/types/src/hatchet.ts` — speed-mode type and attempt-correlated local task input. +- `packages/graphql/src/schema/knowledge.ts` — `KBSpeedMode` enum mapped to lowercase internal values. +- `packages/graphql/src/schema/mutation.ts` — required `speedMode` argument on `ingestKbResource`. +- `packages/graphql/src/graphql/ops/MIngestKbResource.graphql` — generated-client mutation variable. +- `packages/graphql/src/services/knowledge.ts` — new attempt claim, metadata clearing/rollback, and enriched local task payload. +- `packages/graphql/test/knowledge.test.ts` — mode mapping, attempt metadata, rollback, and race assertions. +- `packages/hatchet/src/index.ts` — retain the S5 log, invoke the bridge, add final-failure handling, and register the singleton sweep. +- `packages/hatchet/package.json` and `pnpm-lock.yaml` — exact direct dependencies/test script required by the worker package. +- `apps/hatchet-worker-general/src/index.ts` — worker-only timeout validation before connections start. +- `packages/kb-management/src/components/KnowledgeBaseResourceList.tsx` — per-resource speed selector and selected mutation value. +- `packages/i18n/messages/en.ts` and `packages/i18n/messages/de.ts` — selector labels/options. +- `apps/hatchet-worker-general/.env.example` — local placeholders for the external Hatchet, timeout, webhook, and storage configuration. +- `turbo.json` — global environment allow-list. +- `deploy/charts/klicker-uzh-v3/values.yaml` — non-secret external bridge values and the 3600-second default. +- `deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml` — general-worker non-secret environment mapping. +- `project/2026-07-15-pr-5174-kb-poc-plan.md` — append follow-on implementation evidence only after verification; do not rewrite S5. +- Draft PR #5182 body/comment — whole-branch summary, configuration checklist, test evidence, and screenshots. + +--- + +## Task 1: Add Latest-Attempt Persistence + +**Files:** + +- Modify: `packages/prisma/src/prisma/schema/knowledge.prisma` +- Create: `packages/prisma/src/prisma/schema/migrations/20260720120000_kb_external_ingestion_bridge/migration.sql` +- Generate locally (ignored): `apps/analytics/prisma/schema/knowledge.prisma` + +- [x] **Step 1: Confirm the new fields are absent** + +Run: + +```bash +rg -n "ingestionAttemptId|externalWorkflowRunId|externalWorkflowStartedAt" packages/prisma/src/prisma/schema/knowledge.prisma +``` + +Expected: exit code 1 and no matches. + +- [x] **Step 2: Extend `KBResource` with the approved nullable fields** + +Add after `ingestedAt`: + +```prisma + ingestionAttemptId String? @db.Uuid + externalWorkflowRunId String? + externalWorkflowStartedAt DateTime? +``` + +Keep them nullable so existing rows and the first deployment migrate without a backfill. + +- [x] **Step 3: Add the additive SQL migration** + +Create the exact migration: + +```sql +ALTER TABLE "public"."KBResource" +ADD COLUMN "ingestionAttemptId" UUID, +ADD COLUMN "externalWorkflowRunId" TEXT, +ADD COLUMN "externalWorkflowStartedAt" TIMESTAMP(3); +``` + +Do not edit the already-applied `20260715213657_kb_poc_schema` migration. + +- [x] **Step 4: Regenerate and mirror the schema** + +Run: + +```bash +pnpm --filter @klicker-uzh/prisma generate +pnpm run prisma:sync +pnpm --filter @klicker-uzh/prisma check +``` + +Expected: Prisma generation and TypeScript checks pass; `apps/analytics/prisma/schema/knowledge.prisma` contains the same three fields. + +- [x] **Step 5: Verify migration application against the disposable/local database** + +Run: + +```bash +pnpm --filter @klicker-uzh/prisma prisma:deploy:raw +``` + +Expected: migration `20260720120000_kb_external_ingestion_bridge` applies successfully. Use a disposable database if the current local database must not be mutated. + +- [x] **Step 6: Commit Task 1** + +```bash +git add packages/prisma/src/prisma/schema/knowledge.prisma packages/prisma/src/prisma/schema/migrations/20260720120000_kb_external_ingestion_bridge/migration.sql +git commit -m "feat(kb): track latest external ingestion attempt" +``` + +--- + +## Task 2: Share the Webhook Signer and Correlate Transitions + +**Files:** + +- Create: `packages/util/src/kbWebhook.ts` +- Create: `packages/util/test/kbWebhook.test.ts` +- Modify: `packages/util/src/index.ts` +- Modify: `packages/graphql/src/services/knowledgeWebhooks.ts` +- Modify: `packages/graphql/test/knowledgeWebhooks.test.ts` + +- [x] **Step 1: Write the failing shared signing tests** + +Cover: + +```ts +const rawBody = Buffer.from('{"resourceId":"abc"}') + +expect( + signKBIngestionWebhook({ + rawBody, + secret: 'secret', + timestamp: 1_721_488_400, + }) +).toEqual({ + 'x-kb-timestamp': '1721488400', + 'x-kb-signature': createHmac('sha256', 'secret') + .update(Buffer.concat([Buffer.from('1721488400.'), rawBody])) + .digest('hex'), +}) +``` + +Also prove that two bodies with the same decoded JSON but different raw bytes produce different signatures. + +Run: + +```bash +pnpm --filter @klicker-uzh/util exec vitest run test/kbWebhook.test.ts +``` + +Expected: FAIL because `kbWebhook.ts` does not exist. + +- [x] **Step 2: Move the byte-exact helper without changing its API** + +Implement `packages/util/src/kbWebhook.ts` with the current `Buffer.concat`, timestamp header, and HMAC-SHA256 hex behavior. Export it from `packages/util/src/index.ts`. + +In `knowledgeWebhooks.ts`, replace the local implementation with: + +```ts +import { signKBIngestionWebhook } from '@klicker-uzh/util' + +export { signKBIngestionWebhook } from '@klicker-uzh/util' +``` + +Keep `timingSafeEqual` local to verification. + +- [x] **Step 3: Make the webhook payload attempt-correlated** + +Extend the parsed payload: + +```ts +type KBIngestionWebhookPayload = { + resourceId: string + ingestionAttemptId: string + status: 'PROCESSING' | 'READY' | 'FAILED' + statusMessage?: string +} +``` + +Validate both UUIDs with the existing UUID pattern. Add the attempt guard to the single atomic update: + +```ts +where: { + id: payload.resourceId, + ingestionAttemptId: payload.ingestionAttemptId, + status: { in: allowedSources }, +} +``` + +Continue returning `{ statusCode: 200, body: { ok: true } }` when a correctly signed stale attempt updates zero rows. + +- [x] **Step 4: Update the integration fixture and add the stale-attempt regression** + +Use fixed valid UUIDs for `ingestionAttemptId` and `staleAttemptId`. Store the current ID in `beforeEach`, include it in every valid payload, and add: + +```ts +it('does not let a stale attempt mutate the latest ingestion', async () => { + const request = createRequest({ + resourceId, + ingestionAttemptId: staleAttemptId, + status: 'READY', + }) + + await expect( + handleKBIngestionWebhook({ prisma, ...request }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + ingestionAttemptId, + status: KBResourceStatus.QUEUED, + ingestedAt: null, + }) +}) +``` + +Add a malformed-attempt UUID test returning 400 before database mutation. + +- [x] **Step 5: Run focused tests and type checks** + +```bash +pnpm --filter @klicker-uzh/util exec vitest run test/kbWebhook.test.ts +pnpm --filter @klicker-uzh/util build +pnpm --filter @klicker-uzh/graphql exec vitest run test/knowledgeWebhooks.test.ts +pnpm --filter @klicker-uzh/util check +pnpm --filter @klicker-uzh/graphql check +``` + +Expected: all tests and checks pass; the old GraphQL export path remains usable. + +- [x] **Step 6: Commit Task 2** + +```bash +git add packages/util/src/kbWebhook.ts packages/util/test/kbWebhook.test.ts packages/util/src/index.ts packages/graphql/src/services/knowledgeWebhooks.ts packages/graphql/test/knowledgeWebhooks.test.ts +git commit -m "feat(kb): correlate signed ingestion callbacks" +``` + +--- + +## Task 3: Carry Speed Mode and a Fresh Attempt Through GraphQL + +**Files:** + +- Modify: `packages/types/src/hatchet.ts` +- Modify: `packages/graphql/src/schema/knowledge.ts` +- Modify: `packages/graphql/src/schema/mutation.ts` +- Modify: `packages/graphql/src/graphql/ops/MIngestKbResource.graphql` +- Modify: `packages/graphql/src/services/knowledge.ts` +- Modify: `packages/graphql/test/knowledge.test.ts` +- Generate: `packages/graphql/src/ops.ts` +- Generate: `packages/graphql/src/ops.schema.json` +- Generate: `packages/graphql/src/public/schema.graphql` +- Generate: `packages/graphql/src/public/client.json` +- Generate: `packages/graphql/src/public/server.json` + +- [x] **Step 1: Write failing service tests for mode, metadata, rollback, and races** + +Update the owned URL and BLOB expectations so the local payload contains: + +```ts +{ + resourceId: resource.id, + kbId: created.id, + ingestionAttemptId: expect.stringMatching(UUID_PATTERN), + speedMode: 'balanced', + // existing discriminated resource fields remain unchanged +} +``` + +Add parameterized cases for `balanced`, `quality`, and `fast`. Add a case starting from a resource with old attempt/run metadata and assert that an accepted click stores a new attempt ID and clears both external fields. Extend the concurrent-click test to assert exactly one new attempt and exactly one local dispatch. + +Extend failed-local-dispatch coverage so the conditional rollback restores the pre-click status, message, `ingestedAt`, attempt ID, run ID, and run start only when the new attempt is still current. + +Run: + +```bash +pnpm --filter @klicker-uzh/graphql exec vitest run test/knowledge.test.ts +``` + +Expected: FAIL because the API and payload do not yet accept the new fields. + +- [x] **Step 2: Define one shared lowercase speed-mode contract** + +In `packages/types/src/hatchet.ts` add: + +```ts +export const kbIngestionSpeedModes = ['balanced', 'quality', 'fast'] as const +export type KBIngestionSpeedMode = (typeof kbIngestionSpeedModes)[number] +``` + +Add these required fields to `IngestKBResourceInputBase`: + +```ts +ingestionAttemptId: string +speedMode: KBIngestionSpeedMode +``` + +- [x] **Step 3: Add the GraphQL enum and required mutation argument** + +In `schema/knowledge.ts`, map GraphQL enum names to the lowercase internal contract: + +```ts +export const KBSpeedMode = builder.enumType('KBSpeedMode', { + values: { + BALANCED: { value: 'balanced' }, + QUALITY: { value: 'quality' }, + FAST: { value: 'fast' }, + } as const, +}) +``` + +In `schema/mutation.ts` require `speedMode` beside `id`. In `MIngestKbResource.graphql` use: + +```graphql +mutation IngestKbResource($id: ID!, $speedMode: KBSpeedMode!) { + ingestKbResource(id: $id, speedMode: $speedMode) { + id + status + } +} +``` + +GraphQL itself now rejects values outside the three enum members. + +- [x] **Step 4: Generate and conditionally claim the latest attempt** + +Change the service signature to receive `speedMode: KBIngestionSpeedMode`. Generate one `randomUUID()` before the conditional claim. The successful claim must set: + +```ts +data: { + status: DB.KBResourceStatus.QUEUED, + statusMessage: null, + ingestedAt: null, + ingestionAttemptId, + externalWorkflowRunId: null, + externalWorkflowStartedAt: null, +} +``` + +Pass `ingestionAttemptId` and `speedMode` in the local task payload. On local `runNoWait` failure, restore the complete pre-click snapshot only with: + +```ts +where: { + id: resource.id, + status: DB.KBResourceStatus.QUEUED, + ingestionAttemptId, +} +``` + +This retains the existing S5 claim/rollback behavior while preventing a stale failure from reverting a newer attempt. + +- [x] **Step 5: Regenerate GraphQL and run the focused checks** + +```bash +pnpm --filter @klicker-uzh/graphql generate +pnpm --filter @klicker-uzh/graphql exec vitest run test/knowledge.test.ts test/knowledgeWebhooks.test.ts +pnpm --filter @klicker-uzh/types check +pnpm --filter @klicker-uzh/graphql check +``` + +Expected: all mode/attempt/race tests pass and generated operation types expose `KbSpeedMode`. + +- [x] **Step 6: Commit Task 3** + +Stage the hand-written and generated GraphQL artifacts reported by `git status`, then: + +```bash +git commit -m "feat(kb): add correlated speed-aware ingestion attempts" +``` + +--- + +## Task 4: Implement the External Hatchet Dispatch Bridge + +**Files:** + +- Create: `packages/hatchet/src/kbIngestion.ts` +- Create: `packages/hatchet/test/kbIngestion.test.ts` +- Create: `packages/hatchet/vitest.config.ts` +- Modify: `packages/hatchet/package.json` +- Modify: `pnpm-lock.yaml` + +- [x] **Step 1: Add only the worker package's direct runtime/test dependencies** + +Add exact versions already used by the monorepo: + +```json +{ + "dependencies": { + "@azure/storage-blob": "12.25.0", + "@klicker-uzh/util": "workspace:*" + }, + "devDependencies": { + "vitest": "~3.2.4" + }, + "scripts": { + "test": "vitest run", + "test:watch": "vitest" + } +} +``` + +Run: + +```bash +pnpm install +``` + +Expected: only the `@klicker-uzh/hatchet` importer changes in `pnpm-lock.yaml`; no package version upgrades. + +- [x] **Step 2: Add the node Vitest configuration and failing contract tests** + +Mirror `packages/util/vitest.config.ts`. In `kbIngestion.test.ts`, mock the SDK boundary, Prisma calls, clock, and webhook fetch. Add failing tests for: + +- absent timeout returns 3600; +- `0`, negative, decimal, non-numeric, and whitespace-only present timeouts throw; +- the dedicated external Hatchet env names populate `token`, `host_port`, `api_url`, nested `tls_config.tls_strategy`, and workflow name; +- URL input is unchanged; +- BLOB SAS is scoped to the exact container/blob, has `sp=r`, `spr=https`, starts five minutes before the fixed clock, and expires one hour after it; +- payload equality for all agreed Python fields and exactly one `sources` entry; +- `additionalMetadata` contains only the attempt-correlation key/value added by Klicker; +- an existing persisted run ID prevents lookup/trigger; +- a run found by attempt metadata is persisted without triggering a duplicate; +- a new `runNoWait` result is persisted with its ID/start time; +- losing the attempt-guard update causes best-effort cancellation; +- neither logger calls nor thrown user-facing errors contain a SAS URL or raw SDK error. + +Run: + +```bash +pnpm --filter @klicker-uzh/hatchet test +``` + +Expected: FAIL because `kbIngestion.ts` does not exist. + +- [x] **Step 3: Implement strict environment parsing and the lazy client** + +Export `getKBIngestionTimeoutSeconds(env = process.env)` and `validateKBIngestionWorkerConfig(env = process.env)`. The latter calls the parser but does not instantiate a client. + +Export a lazy `getExternalHatchetClient()` that initializes `HatchetClient` only on first dispatch/sweep with: + +```ts +HatchetClient.init({ + token, + host_port: hostPort, + api_url: apiUrl, + tls_config: { tls_strategy: tlsStrategy }, +}) +``` + +Require the five dedicated external Hatchet variables. Accept only `tls`, `mtls`, or `none`. Do not read the local `HATCHET_*` variables as fallback. + +- [x] **Step 4: Implement exact source URL and payload construction** + +Use these named constants: + +```ts +const KB_BLOB_SAS_CLOCK_SKEW_MS = 5 * 60 * 1000 +const KB_BLOB_SAS_VALIDITY_MS = 60 * 60 * 1000 +// This duration may need adjustment for larger files or slower ingestion workflows in future modifications. +``` + +For BLOB input, build a `StorageSharedKeyCredential`, obtain the selected blob client's URL, and append `generateBlobSASQueryParameters` with `BlobSASPermissions.parse('r')`, `SASProtocol.Https`, `startsOn`, and `expiresOn`. For URL input, return `sourceUrl` unchanged. + +Build exactly: + +```ts +{ + course_id: input.kbId, + sources: [{ source_id: input.resourceId, source_url: sourceUrl }], + upload_markdown: true, + export_to_falkordb: true, + falkordb_graph_name: `klickeruzh:${input.kbId}`, + speed_mode: input.speedMode, +} +``` + +Never log or persist `sourceUrl`. + +- [x] **Step 5: Implement idempotent external dispatch** + +Export `dispatchKBIngestion(input, dependencies)` and use this order: + +1. Read the resource by ID and exit if its attempt ID differs or its status is not `QUEUED`/`PROCESSING`. +2. Return the stored `externalWorkflowRunId` if present. +3. Call `client.runs.list` with the configured workflow name, `additionalMetadata: { klickerKBIngestionAttemptId: input.ingestionAttemptId }`, `onlyTasks: false`, `includePayloads: false`, `limit: 1`, and `since: new Date(resource.updatedAt.getTime() - KB_BLOB_SAS_CLOCK_SKEW_MS)`. +4. If a matching row exists, use `workflowRunExternalId` and `new Date(row.createdAt)` without calling `runNoWait`. +5. Otherwise create the source URL and call: + +```ts +const run = await client.runNoWait(workflowName, payload, { + additionalMetadata: { + klickerKBIngestionAttemptId: input.ingestionAttemptId, + }, +}) +const runId = await run.getWorkflowRunId() +``` + +6. Persist the run ID and start time with `updateMany` guarded by resource ID, current attempt ID, active local status, and `externalWorkflowRunId: null`. For a new run, capture the start time immediately before `runNoWait`; for a recovered run, use its returned `createdAt`. +7. If that update affects zero rows, call `client.runs.cancel({ ids: [runId] })` best-effort and return without mutating the resource. + +The attempt-metadata lookup is the retry recovery for an accepted run whose first response/persistence was ambiguous. Do not add an outbox in this POC. + +- [x] **Step 6: Make errors retryable but privacy-safe** + +Let configuration, Azure, and SDK failures throw so local Hatchet applies its configured retries. Log only a stable category plus resource/KB/attempt identifiers. Do not interpolate `error.message` into the UI status or any log that could contain a URL/token. + +- [x] **Step 7: Run bridge tests and package checks** + +```bash +pnpm --filter @klicker-uzh/hatchet test +pnpm --filter @klicker-uzh/hatchet check +pnpm --filter @klicker-uzh/hatchet build +``` + +Expected: all bridge tests pass and Rollup includes `kbIngestion.js` through the `index.ts` export/import graph. + +- [x] **Step 8: Commit Task 4** + +```bash +git add packages/hatchet/src/kbIngestion.ts packages/hatchet/test/kbIngestion.test.ts packages/hatchet/vitest.config.ts packages/hatchet/package.json pnpm-lock.yaml +git commit -m "feat(kb): dispatch selected resources to external Hatchet" +``` + +--- + +## Task 5: Register Final-Failure Handling and the Singleton Sweeper + +**Files:** + +- Modify: `packages/hatchet/src/kbIngestion.ts` +- Modify: `packages/hatchet/test/kbIngestion.test.ts` +- Modify: `packages/hatchet/src/index.ts` +- Modify: `apps/hatchet-worker-general/src/index.ts` + +- [x] **Step 1: Add failing signed-webhook and sweep tests** + +Test `sendKBIngestionStatus` with a fixed clock and secret. Assert the POST body contains: + +```json +{ + "resourceId": "", + "ingestionAttemptId": "", + "status": "PROCESSING" +} +``` + +Assert `content-type: application/json`, the exact shared signature headers, and rejection on non-2xx responses. + +Add sweep cases for every SDK status: + +| External | Expected local webhook action | +| ----------- | ------------------------------- | +| `QUEUED` | none | +| `RUNNING` | `PROCESSING` | +| `COMPLETED` | `READY` | +| `FAILED` | `FAILED` with sanitized message | +| `CANCELLED` | `FAILED` with sanitized message | + +Add timeout coverage using a configured value different from 3600: a non-terminal run older than the limit gets `runs.cancel({ ids: [runId] })` and then a `FAILED` webhook even if cancellation rejects. Add multiple resources and prove one status-query or webhook failure does not stop later rows. + +Run: + +```bash +pnpm --filter @klicker-uzh/hatchet test +``` + +Expected: FAIL because status posting/sweeping is not implemented. + +- [x] **Step 2: Implement the shared signed webhook sender** + +Serialize the payload once to `Buffer`, sign those exact bytes with `signKBIngestionWebhook`, and POST the same bytes to `KB_WEBHOOK_URL`. Require `KB_WEBHOOK_URL` and `KB_WEBHOOK_SECRET` only when sending. Return no response body to callers and throw a generic error for non-2xx responses. + +The final local dispatch failure reports: + +```ts +{ + status: 'FAILED', + statusMessage: 'The external ingestion workflow could not be started.', +} +``` + +The webhook attempt guard makes a delayed local failure a no-op after a newer click. + +- [x] **Step 3: Implement one sequential database-driven sweep** + +Query only rows matching: + +```ts +where: { + status: { in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING] }, + ingestionAttemptId: { not: null }, + externalWorkflowRunId: { not: null }, + externalWorkflowStartedAt: { not: null }, +} +``` + +Process with `for ... of`, not `Promise.all`. For each row: + +1. Fetch `client.runs.get_status(runId)`. +2. If `COMPLETED`, `FAILED`, or `CANCELLED`, report the mapped terminal status. +3. If `QUEUED`/`RUNNING` exceeds the configured timeout, cancel best-effort and report `FAILED` with `External ingestion timed out.` +4. Otherwise report `PROCESSING` only for `RUNNING`; leave `QUEUED` unchanged. +5. Catch/log a sanitized per-resource category and continue. + +Checking terminal status before timeout prevents a completed run from being mislabeled when a sweep occurs just after the duration boundary. + +- [x] **Step 4: Retain S5 and register both local tasks** + +In `packages/hatchet/src/index.ts`, keep this log unchanged in place: + +```ts +await ctx.logger.info('KB ingestion dispatch stub', { + resourceId: input.resourceId, + kbId: input.kbId, + type: input.type, +}) +``` + +Immediately after it, call `dispatchKBIngestion`. Keep `retries: 3`. Add an `onFailure` handler with its own retries that calls the signed failure reporter for the same input/attempt. + +Register: + +```ts +const monitorKBIngestions = hatchet.task({ + name: 'monitor-kb-ingestions', + onCrons: ['* * * * *'], + concurrency: { + expression: '"monitor-kb-ingestions"', + maxRuns: 1, + limitStrategy: ConcurrencyLimitStrategy.CANCEL_NEWEST, + }, + fn: async () => monitorActiveKBIngestions(), +}) +``` + +Return `monitorKBIngestions` from `prepareHatchetTasks`; the general worker's existing dynamic selection then registers it automatically. Do not add it to the GraphQL-context `PreparedHatchetTasks` interface because GraphQL never triggers the cron directly. + +- [x] **Step 5: Validate timeout only in the actual general worker process** + +At the beginning of `main()` in `apps/hatchet-worker-general/src/index.ts`, call `validateKBIngestionWorkerConfig()` before Redis clients are created. This intentionally validates the timeout at worker startup while preserving lazy external-client configuration. + +Do not validate in `prepareHatchetTasks()`: the GraphQL backend also calls that function to obtain task declarations, and external worker configuration must not prevent the API from starting. + +- [x] **Step 6: Test the declaration and worker startup behavior** + +Add a lightweight mocked `hatchet.task` assertion or exported declaration constants proving: + +- cron is exactly `* * * * *`; +- concurrency expression is constant; +- `maxRuns` is 1; +- strategy is `CANCEL_NEWEST`; +- local dispatch logging occurs before bridge dispatch; +- final-failure webhook carries the same attempt ID. + +Run: + +```bash +pnpm --filter @klicker-uzh/hatchet test +pnpm --filter @klicker-uzh/hatchet check +pnpm --filter @klicker-uzh/hatchet-worker-general check +``` + +Expected: all tests/checks pass. A process started with `KB_INGESTION_TIMEOUT_SECONDS=abc` exits during startup; an absent value resolves to 3600. + +- [x] **Step 7: Commit Task 5** + +```bash +git add packages/hatchet/src/kbIngestion.ts packages/hatchet/test/kbIngestion.test.ts packages/hatchet/src/index.ts apps/hatchet-worker-general/src/index.ts +git commit -m "feat(kb): monitor external ingestion runs with one cron" +``` + +--- + +## Task 6: Add the Per-Click Speed Selector + +**Files:** + +- Modify: `packages/kb-management/src/components/KnowledgeBaseResourceList.tsx` +- Modify: `packages/i18n/messages/en.ts` +- Modify: `packages/i18n/messages/de.ts` + +- [x] **Step 1: Establish the browser validation path before editing UI** + +Read `.agents/skills/agent-browser/SKILL.md` completely. Confirm that the real manage app can be reached after: + +```bash +./_run_app_dependencies.sh +pnpm run dev +``` + +Use delegated local credentials `lecturer` / `abcd`; do not use Edu-ID. Record the working route, normally `http://manage.klicker.com/resources/knowledgeBases` or the direct local manage port when routing is unavailable. + +- [x] **Step 2: Add localized labels with exact EN/DE parity** + +Add keys: + +```ts +speedModeLabel +speedModeBalanced +speedModeQuality +speedModeFast +``` + +English: `Speed`, `Balanced`, `Quality`, `Fast`. + +German: `Geschwindigkeit`, `Ausgewogen`, `Qualität`, `Schnell`. + +- [x] **Step 3: Render one controlled selector per resource row** + +Import generated `KbSpeedMode` and the design-system `Select`. Keep a per-resource state map whose missing value resolves to `KbSpeedMode.Balanced`. Render three items and stable selectors. Compose `Select` with a sibling native label because design-system 4.1.6 `SelectField` does not forward its ID or ARIA labeling to the Radix combobox trigger: + +```ts +data={{ cy: `kb-speed-mode-${resource.id}` }} +``` + +Use item selectors `kb-speed-mode-balanced`, `kb-speed-mode-quality`, and `kb-speed-mode-fast`. Disable the selector while that resource is active or while any row mutation is in flight, matching the current Ingest button behavior. + +Update `handleIngest` to send: + +```ts +variables: { + id: resource.id, + speedMode: speedModeByResource[resource.id] ?? KbSpeedMode.Balanced, +} +``` + +The state may remain for the current mounted page but is never persisted to the server or resource model. + +- [x] **Step 4: Verify types, formatting, and translations** + +```bash +pnpm --filter @klicker-uzh/kb-management check +pnpm --filter @klicker-uzh/frontend-manage check +pnpm exec prettier --check packages/kb-management/src/components/KnowledgeBaseResourceList.tsx packages/i18n/messages/en.ts packages/i18n/messages/de.ts +``` + +Expected: checks pass and EN/DE `kb.*` keys remain identical. + +- [x] **Step 5: Run the mandatory real browser walkthrough** + +With the local dependencies, backend, local Hatchet worker, and manage app running: + +1. Log in through delegated access. +2. Open an existing KB with at least two resources. +3. Confirm both selectors initially show Balanced. +4. Select Quality for one row and Fast for the other. +5. Click Ingest on only the first row. +6. Confirm the GraphQL variables contain `QUALITY` and only that resource becomes `QUEUED`. +7. Confirm the second resource is unchanged and its Fast choice is not sent. +8. Recheck at 375 px and in German. + +Capture committed screenshots under: + +- `project/screenshots/kb-external-ingestion-speed-en-desktop.png` +- `project/screenshots/kb-external-ingestion-speed-de-mobile.png` + +- [x] **Step 6: Commit Task 6** + +```bash +git add packages/kb-management/src/components/KnowledgeBaseResourceList.tsx packages/i18n/messages/en.ts packages/i18n/messages/de.ts project/screenshots/kb-external-ingestion-speed-en-desktop.png project/screenshots/kb-external-ingestion-speed-de-mobile.png +git commit -m "feat(kb): select ingestion speed per resource" +``` + +--- + +## Task 7: Wire Local and Kubernetes Configuration + +**Files:** + +- Modify: `apps/hatchet-worker-general/.env.example` +- Modify: `turbo.json` +- Modify: `deploy/charts/klicker-uzh-v3/values.yaml` +- Modify: `deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml` + +- [x] **Step 1: Add documented local placeholders without credentials** + +Add to `apps/hatchet-worker-general/.env.example`: + +```dotenv +KB_INGESTION_HATCHET_CLIENT_TOKEN=__KB_INGESTION_HATCHET_CLIENT_TOKEN__ +KB_INGESTION_HATCHET_CLIENT_HOST_PORT=__KB_INGESTION_HATCHET_CLIENT_HOST_PORT__ +KB_INGESTION_HATCHET_API_URL=__KB_INGESTION_HATCHET_API_URL__ +KB_INGESTION_HATCHET_CLIENT_TLS_STRATEGY=none +KB_INGESTION_HATCHET_WORKFLOW_NAME=__KB_INGESTION_HATCHET_WORKFLOW_NAME__ +KB_INGESTION_TIMEOUT_SECONDS=3600 +BLOB_STORAGE_ACCOUNT_NAME=__BLOB_STORAGE_ACCOUNT_NAME__ +BLOB_STORAGE_ACCESS_KEY=__BLOB_STORAGE_ACCESS_KEY__ +``` + +Keep the existing `KB_WEBHOOK_URL` and `KB_WEBHOOK_SECRET`. Do not add real namespace/service addresses or secrets. + +- [x] **Step 2: Add every new variable to Turbo's allow-list** + +Add the six `KB_INGESTION_*` variables. `BLOB_STORAGE_*` and `KB_WEBHOOK_*` are already present; do not duplicate them. + +- [x] **Step 3: Define non-secret Helm values** + +Under `hatchet`, add: + +```yaml +kbIngestion: + clientHostPort: '' + apiUrl: '' + tlsStrategy: 'none' + workflowName: '' + timeoutSeconds: 3600 + webhookUrl: '' +``` + +Reuse the existing top-level `blobStorage.accountName`. Do not add token, webhook-secret, or access-key values to the ConfigMap section. + +- [x] **Step 4: Map only non-secrets into the general-worker ConfigMap** + +Add: + +```yaml +KB_INGESTION_HATCHET_CLIENT_HOST_PORT: + { { .Values.hatchet.kbIngestion.clientHostPort | quote } } +KB_INGESTION_HATCHET_API_URL: { { .Values.hatchet.kbIngestion.apiUrl | quote } } +KB_INGESTION_HATCHET_CLIENT_TLS_STRATEGY: + { { .Values.hatchet.kbIngestion.tlsStrategy | quote } } +KB_INGESTION_HATCHET_WORKFLOW_NAME: + { { .Values.hatchet.kbIngestion.workflowName | quote } } +KB_INGESTION_TIMEOUT_SECONDS: + { { .Values.hatchet.kbIngestion.timeoutSeconds | quote } } +KB_WEBHOOK_URL: { { .Values.hatchet.kbIngestion.webhookUrl | quote } } +BLOB_STORAGE_ACCOUNT_NAME: { { .Values.blobStorage.accountName | quote } } +``` + +The existing `*-secret-hatchet-worker-general` reference must receive these keys out-of-repo through the deployment secret-management process: + +- `KB_INGESTION_HATCHET_CLIENT_TOKEN` +- `KB_WEBHOOK_SECRET` +- `BLOB_STORAGE_ACCESS_KEY` + +Do not create a chart-managed Secret containing those values. + +- [x] **Step 5: Render and inspect the chart** + +Run: + +```bash +helm template klicker deploy/charts/klicker-uzh-v3 --set hatchet.kbIngestion.timeoutSeconds=3600 --set blobStorage.accountName=testaccount | rg "KB_INGESTION_|KB_WEBHOOK_URL|BLOB_STORAGE_ACCOUNT_NAME" +``` + +Expected: seven non-secret ConfigMap entries appear with timeout `3600`; token, secret, and access key do not appear in rendered ConfigMap data. + +- [x] **Step 6: Run config checks and commit** + +```bash +pnpm exec prettier --check turbo.json apps/hatchet-worker-general/.env.example deploy/charts/klicker-uzh-v3/values.yaml deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml +git diff --check +``` + +Then: + +```bash +git add apps/hatchet-worker-general/.env.example turbo.json deploy/charts/klicker-uzh-v3/values.yaml deploy/charts/klicker-uzh-v3/templates/cm-hatchet-workers.yaml +git commit -m "chore(kb): configure external Hatchet ingestion bridge" +``` + +--- + +## Task 8: Full Verification and Cluster Smoke Test + +**Files:** + +- Modify after evidence exists: `project/2026-07-15-pr-5174-kb-poc-plan.md` +- Modify after review: draft PR #5182 body or comment + +- [x] **Step 1: Run all targeted deterministic tests** + +```bash +pnpm --filter @klicker-uzh/util exec vitest run test/kbWebhook.test.ts +pnpm --filter @klicker-uzh/graphql exec vitest run test/knowledge.test.ts test/knowledgeWebhooks.test.ts +pnpm --filter @klicker-uzh/hatchet test +``` + +Expected: shared signer, GraphQL/database, bridge/retry/SAS, and sweeper suites are all green. + +- [x] **Step 2: Run package and application checks** + +```bash +pnpm --filter @klicker-uzh/prisma check +pnpm --filter @klicker-uzh/types check +pnpm --filter @klicker-uzh/util check +pnpm --filter @klicker-uzh/hatchet check +pnpm --filter @klicker-uzh/graphql check +pnpm --filter @klicker-uzh/hatchet-worker-general check +pnpm --filter @klicker-uzh/kb-management check +pnpm --filter @klicker-uzh/frontend-manage check +``` + +Expected: all focused type checks pass. + +- [x] **Step 3: Run whole-repository quality gates** + +```bash +pnpm run check:all +opengrep scan --config auto packages/hatchet/src/kbIngestion.ts packages/graphql/src/services/knowledge.ts packages/graphql/src/services/knowledgeWebhooks.ts packages/util/src/kbWebhook.ts +git diff --check +``` + +Expected: repository checks pass. Classify any repository-wide pre-existing Opengrep findings separately; no new finding may be ignored without explicit rationale. + +- [x] **Step 4: Run a local end-to-end status loop** + +Using mocked/local external Hatchet configuration where appropriate: + +1. Trigger one resource from the real manage UI. +2. Confirm PostgreSQL stores a new attempt ID and later the external run ID/start time. +3. Confirm the local worker log still contains `KB ingestion dispatch stub` and never prints the source URL. +4. Confirm one minute sweep reports `PROCESSING` and then `READY`/`FAILED` through the HTTP webhook. +5. Confirm Apollo polling renders those states without a manual reload. +6. Confirm a stale signed callback using the previous attempt ID returns 200 but does not mutate the resource. + +Capture terminal-state screenshots if they differ materially from the existing S7 screenshots. + +- [ ] **Step 5: Run the real cluster smoke test after configuration is installed** + +Manual deployment prerequisite (2026-07-20): the external Hatchet +host/API/TLS/workflow values and the three out-of-repo secrets have not yet +been installed in the general-worker deployment, so the real cluster smoke +test cannot run from this branch workspace. Local deterministic and real-UI +status-loop verification is complete; keep this checkbox open until deployment +configuration is available. + +Configure the external Hatchet host/API/TLS/workflow values and the three secrets in the general-worker deployment. Then: + +1. Upload a real PDF to its existing private `kb-` container. +2. Select a non-default speed once and click only that resource's Ingest button. +3. Confirm the external run input has one source, the exact KB/resource UUID mapping, fixed booleans, graph name, selected lowercase speed, and a working blob-scoped SAS. +4. Confirm the external run metadata contains the attempt UUID and the run ID/start time are stored on `KBResource`. +5. Confirm `monitor-kb-ingestions` runs once per minute without overlap and advances the UI. +6. Confirm FalkorDB contains `klickeruzh:` and the resource becomes `READY`. +7. Exercise an external failure or cancellation and confirm the resource becomes `FAILED` with a sanitized message. +8. Confirm no external workflow call to Klicker's webhook is configured or required. + +If cluster credentials/configuration are not yet available, mark only this step as a manual deployment prerequisite; deterministic local tests must still be complete. + +- [x] **Step 6: Perform the required independent reviews** + +Before publishing the final branch update: + +1. Run a security-focused review of SAS scope/expiry, secret handling, webhook correlation, ownership, and logged data. +2. Run `$thermo-nuclear-code-quality-review` and resolve or explicitly defer each maintainability finding. +3. Have a separate review agent inspect the complete branch diff against `kb-poc`, with particular attention to task registration in both the backend and worker processes. +4. Re-run affected tests after accepted fixes. + +- [x] **Step 7: Update the original implementation record without rewriting S5** + +Append a dated follow-on progress entry to `project/2026-07-15-pr-5174-kb-poc-plan.md` that states: + +- S5's existing log was retained; +- external dispatch is a separately approved follow-on bridge; +- selected-resource/speed behavior; +- one-hour private blob SAS behavior and its future-size comment; +- latest-attempt/run fields; +- singleton one-minute sweep and 3600-second timeout; +- tests, browser evidence, cluster result or explicit cluster prerequisite. + +Do not edit the original S5 evidence into claiming it originally contained the external call. + +- [x] **Step 8: Commit final evidence if needed** + +```bash +git add project/2026-07-15-pr-5174-kb-poc-plan.md project/screenshots +git commit -m "docs(kb): record external ingestion bridge verification" +``` + +Skip this commit if no files changed. + +--- + +## Task 9: Push and Update Draft PR #5182 + +**Files/Systems:** + +- Git branch: `feat/kb-poc-management-ui` +- Target branch: `kb-poc` +- GitHub draft PR: `https://github.com/uzh-bf/klicker-uzh/pull/5182` + +- [ ] **Step 1: Audit the whole branch against the target** + +```bash +git status --short +git log --oneline kb-poc..HEAD +git diff --stat kb-poc...HEAD +git diff --check kb-poc...HEAD +``` + +Expected: worktree clean, intended commits only, and no whitespace errors. + +- [ ] **Step 2: Push the complete branch** + +```bash +git push origin feat/kb-poc-management-ui +``` + +- [ ] **Step 3: Update the draft PR as a whole-branch description** + +Follow the repository's PR-writing/publishing workflow. The body or a structured comment must include: + +- the full KB management POC plus this follow-on bridge, not only the latest commit; +- explicit statement that the S5 log remains; +- architecture and attempt-correlation summary; +- exact non-secret variables and the three out-of-repo secret keys; +- private SAS scope/expiry and URL-resource reachability requirement; +- deterministic test/check results; +- desktop/mobile screenshots; +- real cluster smoke result or a clearly unchecked deployment prerequisite; +- residual cross-system ambiguity and out-of-scope outbox/history work. + +Keep PR #5182 in draft state. Do not merge or mark ready without explicit user approval. + +- [ ] **Step 4: Check CI and address branch-caused failures** + +```bash +gh pr checks 5182 --watch +``` + +Expected: all required checks pass. Diagnose exact logs before changing code; do not alter unrelated pre-existing failures. + +--- + +## Final Self-Review Checklist + +- [ ] No `TODO`, placeholder branch, fake success, sleep, or per-resource monitor task was introduced. +- [ ] The literal `KB ingestion dispatch stub` log still exists and precedes the external dispatch call. +- [ ] The external payload has exactly one selected source and no Klicker-only attempt field. +- [ ] The attempt UUID appears in external `additionalMetadata`, local persistence guards, and signed webhook payloads. +- [ ] Blob SAS is read-only, HTTPS-only, blob-scoped, one hour, clock-skew tolerant, and contains the approved future-file-size comment. +- [ ] No SAS URL or secret appears in application logs, database fields, screenshots, commits, or PR text. +- [ ] URL resources pass both public-destination guards; external DNS and redirect egress controls are verified before lecturer exposure. +- [ ] Only the latest attempt/run metadata is stored. +- [ ] External terminal statuses and timeout map through the signed webhook; the external workflow itself does not call Klicker. +- [ ] The sweep is database-driven, once per minute, non-overlapping, sequential, and failure-isolated. +- [ ] `KB_INGESTION_TIMEOUT_SECONDS` defaults to 3600 and invalid present values fail only the general worker's initialization. +- [ ] GraphQL backend startup does not require external Hatchet worker configuration. +- [ ] All environment variables are present in `.env.example`, Turbo, and Helm as applicable; secrets remain outside ConfigMaps. +- [ ] GraphQL codegen, Prisma generation/sync, focused tests, package checks, root checks, browser validation, and review gates are complete. +- [ ] PR #5182 still targets `kb-poc` and remains draft. diff --git a/docs/superpowers/plans/2026-07-21-hatchet-worker-dev-ordering.md b/docs/superpowers/plans/2026-07-21-hatchet-worker-dev-ordering.md new file mode 100644 index 0000000000..c442123986 --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-hatchet-worker-dev-ordering.md @@ -0,0 +1,151 @@ +--- +type: Implementation Plan +title: Hatchet Worker Development Startup Ordering Implementation Plan +description: Build the Hatchet package before persistent local workers and remove watch-mode interference. +timestamp: '2026-07-21' +tags: + - hatchet + - development + - turborepo +--- + +# Hatchet Worker Development Startup Ordering Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Ensure every local application development variant builds `@klicker-uzh/hatchet` before persistent workers and applications start. + +**Architecture:** Extend the existing explicit Turborepo build prerequisites for the four application development variants. Run both Hatchet workers as persistent `tsx` processes without watch mode, remove the unsupported Hatchet SDK internal-logger patch, and keep the general worker's Pino development formatter in-process. Verify both the static task graph and real local resource-ingestion worker registration. + +**Tech Stack:** Turborepo 2.5.6, pnpm 11.5.0, Hatchet TypeScript SDK 1.9.4, JSON configuration + +## Global Constraints + +- Modify `turbo.json` for startup ordering, remove `tsx --watch` from both Hatchet workers, remove the SDK-internal logger workaround, and keep the general worker logger in-process. +- Apply the same prerequisite to `dev`, `dev:lti`, `dev:offline`, and `dev:assessment`. +- Do not change dependencies, environment variables, or external service configuration. +- Keep the existing dependency order and add `@klicker-uzh/hatchet#build` immediately after `@klicker-uzh/graphql#build` in each task. +- Do not restart `run_app_dependencies` or the external port forwards. + +--- + +### Task 1: Order Hatchet Builds Before Persistent Development Tasks + +**Files:** + +- Modify: `turbo.json:142-189` +- Verify: `turbo.json` + +**Interfaces:** + +- Consumes: Turborepo's package-specific `package#task` dependency syntax and the existing `@klicker-uzh/hatchet` package `build` script. +- Produces: Four development task graphs in which `@klicker-uzh/hatchet#build` completes before persistent development tasks start. + +- [x] **Step 1: Record the failing task graph** + +Run: + +```bash +pnpm exec turbo run dev --dry=json > /tmp/klicker-turbo-dev-before.json +jq -e '.tasks[] | select(.package == "@klicker-uzh/hatchet-worker-general" and .task == "dev") | .dependencies | index("@klicker-uzh/hatchet#build")' /tmp/klicker-turbo-dev-before.json +``` + +Expected: `jq` exits non-zero because the general worker development task does not currently depend on the Hatchet build. + +- [x] **Step 2: Add the explicit prerequisite** + +In each of `tasks.dev.dependsOn`, `tasks["dev:lti"].dependsOn`, `tasks["dev:offline"].dependsOn`, and `tasks["dev:assessment"].dependsOn`, change: + +```json +"@klicker-uzh/graphql#build", +"@klicker-uzh/markdown#build" +``` + +to: + +```json +"@klicker-uzh/graphql#build", +"@klicker-uzh/hatchet#build", +"@klicker-uzh/markdown#build" +``` + +- [x] **Step 3: Validate configuration and formatting** + +Run: + +```bash +jq empty turbo.json +pnpm exec prettier --check turbo.json +git diff --check +``` + +Expected: all commands exit zero. + +- [x] **Step 4: Verify all four resolved task graphs** + +Run a Turbo dry-run for each task: + +```bash +for task in dev dev:lti dev:offline dev:assessment; do + pnpm exec turbo run "$task" --dry=json > "/tmp/klicker-turbo-${task//:/-}-after.json" + jq -e --arg task "$task" '[.tasks[] | select(.task == $task)] | length > 0 and all(.dependencies | index("@klicker-uzh/hatchet#build"))' "/tmp/klicker-turbo-${task//:/-}-after.json" +done +``` + +Expected: every `jq` invocation exits zero and prints `true`. + +- [x] **Step 5: Run a real startup verification** + +Stop only the current `pnpm run dev` process, leave Docker dependencies and external port forwards running, and start: + +```bash +pnpm run dev +``` + +Expected worker output includes: + +```text +Starting Hatchet worker +Selected workflows +Starting worker to process jobs... +``` + +Verify local Hatchet contains both core KB workflows: + +```sql +SELECT name +FROM "Workflow" +WHERE name IN ( + 'ingest-kb-resource', + 'monitor-kb-ingestions' +) +ORDER BY name; +``` + +Expected: two rows. + +- [x] **Step 6: Verify resource Ingest dispatch** + +Trigger Ingest for a resource in the manage UI. + +Expected: + +- local `ingest-kb-resource` is accepted by a current general worker; +- resource status progresses from `QUEUED` to `PROCESSING`; +- `externalWorkflowRunId` is persisted on the resource; +- external Hatchet contains a new ingestion run. + +- [x] **Step 7: Record the startup-ordering invariant** + +Add this entry under `project/CODEBASE_NOTES.md`'s infrastructure section: + +```markdown +- **General Hatchet worker dev ordering**: Persistent application development tasks must depend on `@klicker-uzh/hatchet#build`; the worker imports the package's generated `dist` output and otherwise can fail before registering workflows on a clean startup. (`turbo.json`, `apps/hatchet-worker-general`) +``` + +- [x] **Step 8: Commit the implementation** + +```bash +git add apps/hatchet-worker-general/package.json apps/hatchet-worker-general/src/logger.ts apps/hatchet-worker-response-processor/package.json packages/hatchet/src/client.ts turbo.json docs/superpowers/specs/2026-07-21-hatchet-worker-dev-ordering-design.md docs/superpowers/plans/2026-07-21-hatchet-worker-dev-ordering.md project/CODEBASE_NOTES.md +git commit -m "fix(hatchet): stabilize worker startup in development" +``` diff --git a/docs/superpowers/specs/2026-07-20-external-kb-hatchet-bridge-design.md b/docs/superpowers/specs/2026-07-20-external-kb-hatchet-bridge-design.md new file mode 100644 index 0000000000..1a2cf4a846 --- /dev/null +++ b/docs/superpowers/specs/2026-07-20-external-kb-hatchet-bridge-design.md @@ -0,0 +1,281 @@ +--- +type: Design +title: External KB Hatchet Bridge Design +description: Design the POC bridge from one selected KB resource to an external Hatchet workflow. +timestamp: '2026-07-20' +tags: + - kb + - hatchet + - ingestion +--- + +# External KB Hatchet Bridge Design + +Date: 2026-07-20 + +Status: approved in conversation on 2026-07-20 + +Related implementation record: `project/2026-07-15-pr-5174-kb-poc-plan.md` + +## Goal + +Extend the existing KB POC so that clicking **Ingest** keeps the local `ingest-kb-resource` Hatchet log and also triggers a workflow on a separate Hatchet instance. The external workflow processes only the selected resource and exports it to a FalkorDB graph for the knowledge base. + +The external workflow remains unchanged and does not call Klicker's webhook. Klicker monitors the external run and updates its own resource status through the existing signed webhook. + +## Existing Behavior Retained + +- KB files remain in the existing Azure storage account configured by `BLOB_STORAGE_ACCOUNT_NAME` and `BLOB_STORAGE_ACCESS_KEY`. +- KB files remain separate from media-library files. They are stored in private per-user containers named `kb-` and are represented by `KBResource`, not `MediaFile`. +- The local Hatchet task remains named `ingest-kb-resource` and retains its existing privacy-safe log. +- GraphQL continues to atomically claim a resource before dispatch. +- Resource status changes continue to use the signed webhook transition rules. +- Only one ingestion can be active for a resource at a time. + +## User Experience + +Each resource row receives a speed-mode selector beside **Ingest** with these values: + +- `balanced` (default) +- `quality` +- `fast` + +The selection applies only to that ingestion click. It is sent through the GraphQL mutation and local Hatchet task but is not persisted as a resource preference. + +GraphQL exposes the choice as a `KBSpeedMode` input enum with `BALANCED`, `QUALITY`, and `FAST`. The bridge maps those API values to the lowercase strings expected by the Python workflow. + +The existing live UI behavior remains: + +- `QUEUED` and `PROCESSING` resources are polled by the browser. +- `READY` and `FAILED` are terminal UI states. +- Re-ingestion remains available from the existing allowed terminal states. + +## External Hatchet Client + +The general Hatchet worker creates a second, lazily initialized Hatchet client for the separate Hatchet server and tenant. It uses the installed `@hatchet-dev/typescript-sdk` client rather than a custom HTTP wrapper. + +The workflow is triggered by its environment-configured name with `runNoWait(workflowName, payload, options)`. The returned workflow run ID is persisted before the local dispatch task completes. + +Required configuration: + +| Variable | Purpose | Secret | +| ------------------------------------------ | --------------------------------------------------------------------- | ------ | +| `KB_INGESTION_HATCHET_CLIENT_TOKEN` | External Hatchet client token and tenant identity | yes | +| `KB_INGESTION_HATCHET_CLIENT_HOST_PORT` | External Hatchet engine address, normally cluster service DNS | no | +| `KB_INGESTION_HATCHET_API_URL` | External Hatchet API address used for run inspection and cancellation | no | +| `KB_INGESTION_HATCHET_CLIENT_TLS_STRATEGY` | External Hatchet TLS behavior | no | +| `KB_INGESTION_HATCHET_WORKFLOW_NAME` | External Python workflow name | no | +| `KB_INGESTION_TIMEOUT_SECONDS` | Maximum external runtime; defaults to `3600` | no | + +The connection values are Kubernetes/Infisical configuration. No namespace, service name, tenant ID, URL, or token is hard-coded. + +An absent timeout uses `3600`. A present timeout must parse as a positive integer; invalid configuration fails worker initialization rather than silently using an unintended duration. + +## External Workflow Payload + +One click sends one source: + +```json +{ + "course_id": "", + "sources": [ + { + "source_id": "", + "source_url": "" + } + ], + "upload_markdown": true, + "export_to_falkordb": true, + "falkordb_graph_name": "klickeruzh:", + "speed_mode": "balanced" +} +``` + +Mapping decisions: + +- `course_id` is the stable KB UUID, not the editable KB name. +- `source_id` is the stable resource UUID, not a title or filename. +- `sources` contains only the selected resource. +- `speed_mode` is the value selected for that click. +- The other processing options are fixed POC behavior. +- URL resources must use HTTP(S) without embedded credentials and a public hostname or IPv4 address. Klicker rejects local, private, reserved, IPv6-literal, and non-public test/internal destinations both when the resource is created and immediately before dispatch. +- The external ingestion service remains responsible for DNS-resolution and redirect egress controls; do not expose this bridge to lecturer traffic until that deployment boundary is verified. + +The local ingestion-attempt UUID is attached as Hatchet `additionalMetadata`; it is not added to the Python workflow input. + +## Private Blob Access + +For a blob resource, the local Hatchet worker generates a blob-scoped Azure SAS URL immediately before triggering the external run: + +- permission: read only (`r`) +- protocol: HTTPS only +- scope: the selected blob only +- clock-skew allowance: start time five minutes in the past +- expiry: one hour + +The one-hour duration is represented by a clearly named constant. A nearby comment must state that it may need adjustment for larger files or slower ingestion workflows in future modifications. + +The SAS URL is not stored in Klicker's database or local Hatchet input and is never logged. It is necessarily stored in the external Hatchet run input/history until that system's retention removes it, but becomes unusable after expiry. + +The general worker must receive the same blob storage account name and access key already used by the backend. KB containers remain private; media-library container visibility is not changed. + +## Latest-Attempt Data Model + +Add these nullable fields to `KBResource`: + +- `ingestionAttemptId` (`UUID`): newly generated for each accepted ingestion click +- `externalWorkflowRunId` (`String`): latest external Hatchet workflow run ID +- `externalWorkflowStartedAt` (`DateTime`): local acceptance time used for timeout evaluation + +No ingestion history table is added. Starting a new attempt replaces the attempt ID and clears the previous external run ID/start time. + +The attempt ID is the correlation guard across GraphQL, local Hatchet, the external Hatchet run metadata, the sweeper, and the signed webhook. Every conditional update verifies that the resource still belongs to the same attempt. An old dispatch retry or sweep result therefore cannot overwrite a newer attempt. + +## Dispatch and Retry Flow + +1. GraphQL validates ownership, current status, and speed mode. +2. GraphQL generates an attempt UUID and atomically changes the resource to `QUEUED`, storing the attempt ID and clearing previous external-run metadata. +3. GraphQL dispatches the existing local `ingest-kb-resource` task with the resource location, speed mode, and attempt ID. +4. The local task retains the existing identifier/type-only log. +5. On task execution or retry, the worker first confirms that the attempt is still current. +6. If an external run ID is already stored, the task returns without starting another run. +7. Otherwise, it looks for an external run carrying the same attempt ID in `additionalMetadata`. This recovers an accepted run after an ambiguous client/network failure. +8. If no matching run exists, it generates the source URL and triggers the configured external workflow. +9. The task conditionally stores the external run ID and start time only if the attempt remains current. +10. If the conditional store loses to a newer attempt, the just-created external run is cancelled on a best-effort basis. + +If dispatch permanently fails, the final local failure path reports `FAILED` through the signed webhook using the attempt ID. Detailed connection and SDK errors remain in Hatchet logs; the user-facing message is sanitized. + +A very small residual ambiguity remains if the external Hatchet server accepts a run but neither returns its ID nor makes the attempt metadata searchable before every local retry. This POC mitigates that window with metadata lookup but does not add a cross-system transactional outbox. + +## Singleton Status Sweeper + +Define one local task named `monitor-kb-ingestions` with the cron expression `* * * * *`. It is configured with a constant concurrency expression, `maxRuns: 1`, and `CANCEL_NEWEST` so overlapping sweeps cannot run simultaneously. + +Each sweep queries `KBResource` rows with: + +- local status `QUEUED` or `PROCESSING` +- non-null `ingestionAttemptId` +- non-null `externalWorkflowRunId` +- non-null `externalWorkflowStartedAt` + +Resources are handled independently and sequentially for the POC. One external API or webhook failure is logged and does not prevent later resources from being checked. The next one-minute sweep retries any resource that remains active. + +External Hatchet status mapping: + +| External status | Klicker action | +| --------------- | -------------------------------- | +| `QUEUED` | keep `QUEUED` | +| `RUNNING` | signed webhook sets `PROCESSING` | +| `COMPLETED` | signed webhook sets `READY` | +| `FAILED` | signed webhook sets `FAILED` | +| `CANCELLED` | signed webhook sets `FAILED` | + +If elapsed time exceeds `KB_INGESTION_TIMEOUT_SECONDS`, the sweeper attempts to cancel the external run and then reports `FAILED`. Cancellation failure is logged but does not prevent the local timeout transition. + +Because the sweeper discovers active work from PostgreSQL, monitoring resumes automatically after local worker or pod restarts. No per-resource sleeping monitor tasks or recursive schedules are created. + +## Signed Webhook Correlation + +The local worker, not the external workflow, calls `KB_WEBHOOK_URL`. Requests continue to use HMAC-SHA256 over the exact raw body and timestamp with `KB_WEBHOOK_SECRET`. + +The webhook payload is extended with `ingestionAttemptId`. Its database transition adds an equality condition for that attempt ID. Validly signed callbacks for stale attempts return the existing successful no-op response and cannot mutate the new attempt. + +The worker and backend must share the same webhook secret through secret management. Move the byte-exact `signKBIngestionWebhook` implementation into a server-safe module under `packages/util`; `packages/graphql/src/services/knowledgeWebhooks.ts` imports and re-exports it to preserve the existing public API, and the general worker imports that same function. Generation and verification therefore use one implementation and one set of contract tests. + +## Deployment Wiring + +The general Hatchet worker ConfigMap/Helm values expose: + +- external Hatchet host/API/TLS/workflow configuration +- `KB_INGESTION_TIMEOUT_SECONDS`, default `3600` +- `KB_WEBHOOK_URL` +- `BLOB_STORAGE_ACCOUNT_NAME` + +The existing general-worker secret receives through the deployment secret-management process: + +- `KB_INGESTION_HATCHET_CLIENT_TOKEN` +- `KB_WEBHOOK_SECRET` +- `BLOB_STORAGE_ACCESS_KEY` + +The new variable names are also added to Turbo's global environment allow-list and local `.env.example` files. Real tokens, storage keys, namespace names, and environment-specific service addresses are never committed. + +## Failure Semantics + +- Missing external configuration: dispatch retries, then the current attempt becomes `FAILED` with a generic configuration message. +- Azure SAS generation failure: dispatch retries, then the current attempt becomes `FAILED`. +- External trigger failure: attempt-metadata lookup is used before retrying creation; final failure becomes `FAILED`. +- External status-query failure: only that resource is skipped until the next sweep. +- Webhook failure: only that resource is retried during the next sweep. +- External `FAILED`/`CANCELLED`: local `FAILED` with a sanitized message. +- Timeout: best-effort external cancellation followed by local `FAILED`. +- Stale attempt or terminal resource: dispatch/sweeper exits without mutation. + +No external SDK error, token, SAS URL, storage key, or webhook secret is returned to the browser or written to application logs. + +## Verification Strategy + +### GraphQL and database integration tests + +- Accept `balanced`, `quality`, and `fast`; reject invalid values. +- Generate a fresh attempt UUID and clear previous external metadata. +- Preserve the existing ownership and allowed-status checks. +- Keep concurrent ingest clicks to one active attempt and one local dispatch. +- Guard webhook transitions by attempt ID. +- Prove stale callbacks cannot mutate a newer attempt. + +### Bridge tests with mocked Azure and external Hatchet clients + +- Generate a one-hour, read-only, HTTPS-only SAS for the exact blob. +- Pass a URL resource through unchanged. +- Build the exact agreed Python workflow payload. +- Read workflow and connection settings from the dedicated environment variables. +- Attach the attempt ID as external run metadata. +- Reuse a previously accepted external run found by attempt metadata. +- Persist only the latest run ID and cancel a run that loses the attempt guard. +- Sanitize final dispatch failures. + +### Sweeper tests + +- Select only active resources with complete external-run metadata. +- Map all five external Hatchet statuses correctly. +- Send signed webhook requests containing the attempt ID. +- Skip stale and terminal resources. +- Apply the environment-configured timeout, defaulting to `3600`. +- Attempt external cancellation on timeout. +- Continue processing after one resource's lookup or webhook fails. +- Verify the non-overlapping one-minute cron declaration. + +### Browser verification + +- Selector defaults to `balanced`. +- All three modes can be selected. +- Ingest sends the chosen value and changes only the selected resource to `QUEUED`. +- Sweeper-driven `PROCESSING`, `READY`, and `FAILED` states appear through existing UI polling. +- Recheck desktop/mobile and English/German states affected by the new selector. + +### Cluster smoke test + +After environment-specific credentials and service addresses are configured: + +1. Upload a real PDF into the private KB container. +2. Select a speed mode and trigger ingestion. +3. Confirm the external Hatchet run receives the agreed payload and a working read SAS URL. +4. Confirm the external run ID is stored on the KB resource. +5. Confirm the singleton sweep advances the UI status. +6. Confirm FalkorDB contains `klickeruzh:`. +7. Confirm the resource reaches `READY`. +8. Exercise one external failure or cancellation and confirm `FAILED`. + +The shared external cluster is not used for deterministic race, timeout, or failure-path tests; those remain mocked and locally repeatable. + +## Out of Scope + +- Changes to the external Python workflow. +- An ingestion-attempt history table or webhook inbox/outbox. +- Course-to-KB relationships; the KB UUID remains `course_id`. +- Multiple selected sources in one external run. +- Making KB containers public. +- Persisting speed-mode preferences. +- Consuming the FalkorDB graph from chat/runtime features. +- Eliminating the final cross-system trigger ambiguity with a transactional outbox. diff --git a/docs/superpowers/specs/2026-07-21-hatchet-worker-dev-ordering-design.md b/docs/superpowers/specs/2026-07-21-hatchet-worker-dev-ordering-design.md new file mode 100644 index 0000000000..4e94c4cfee --- /dev/null +++ b/docs/superpowers/specs/2026-07-21-hatchet-worker-dev-ordering-design.md @@ -0,0 +1,85 @@ +--- +type: Design +title: Hatchet Worker Development Startup Ordering +description: Define deterministic local Hatchet package build and worker startup ordering. +timestamp: '2026-07-21' +tags: + - hatchet + - development + - turborepo +--- + +# Hatchet Worker Development Startup Ordering + +## Problem + +The general Hatchet worker imports `@klicker-uzh/hatchet` through the +package's generated `dist` output. The root Turborepo development tasks do +not currently require that package's build to finish before persistent +development processes start. On a clean startup, the worker can therefore +fail its initial import and remain idle, leaving workflows such as +`ingest-kb-resource` unregistered in local Hatchet. + +## Design + +Add `@klicker-uzh/hatchet#build` as an explicit prerequisite in `turbo.json` +for the four development task variants that share the local application +stack: + +- `dev` +- `dev:lti` +- `dev:offline` +- `dev:assessment` + +This follows the repository's existing explicit package-build dependency +pattern. It also follows Turborepo's package-specific `package#task` +ordering mechanism: each persistent development task starts only after the +Hatchet package build succeeds. + +Clean-start verification exposed a second, independent failure in the +existing development runtime. Hatchet SDK 1.9.4 assumes every heartbeat +worker-thread message contains a log-level `type`, while `tsx --watch` sends +unrelated control messages over the same worker-thread channel. Run the +Hatchet workers without `tsx --watch` instead of importing and modifying the +SDK's internal logger implementation. They remain persistent Turbo tasks and +can be restarted when their source changes. The general worker's Pino +development formatter stays in-process; production JSON logging remains +unchanged. + +## Runtime Flow + +1. A developer starts one of the four supported development variants. +2. Turborepo builds the Hatchet package before starting persistent `dev` + processes. +3. Both Hatchet workers run without a `tsx --watch` control thread; the general + worker uses in-process pretty formatting. +4. The general worker imports a complete Hatchet package and registers its + workflows with local Hatchet. +5. A resource Ingest request can enqueue `ingest-kb-resource`, which can then + dispatch the configured external ingestion workflow. + +## Error Handling + +Build failures remain visible through Turbo and prevent the persistent +development processes from starting with incomplete dependencies. Normal +Hatchet log levels and existing worker/GraphQL error handling are unchanged. + +## Verification + +1. Validate `turbo.json` as JSON and run formatting checks for the file. +2. Inspect Turbo's dry-run task graph to confirm the Hatchet build precedes + the general worker's development task. +3. Start the normal development stack from built dependencies and confirm a + current `hatchet-worker-general` listener exists. +4. Confirm local Hatchet registers `ingest-kb-resource` and + `monitor-kb-ingestions`. +5. Trigger one resource Ingest action and correlate it with the external run + ID persisted on the resource. + +## Scope + +The change fixes local development startup and watch compatibility only. It +does not alter production deployment ordering, external Hatchet configuration, +FalkorDB connectivity, or the separate chat i18n module-resolution issue. +Knowledge-graph startup and dispatch verification belongs to the parked W9 +scope. diff --git a/docs/testing.md b/docs/testing.md index 0f1002ce24..f794297e5c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,8 +1,8 @@ --- type: Testing Guide title: Testing -description: Which test level to use when, what runs safely without services, the Playwright e2e stack and its seeds, and the CI test matrix. -timestamp: '2026-08-20' +description: Which test level to use when, what runs safely without services, the two e2e stacks and their seeds, and the CI test matrix. +timestamp: '2026-08-24' tags: - testing - ci @@ -36,6 +36,14 @@ offers only finite `10`, `20`, and `50` sizes, rejects CSV files above 1 MiB or 200 data rows before submission, and must verify page totals and page-one reset after import or deletion. +The focused KB CRUD, ingestion, and signed-webhook suites deliberately avoid a real Hatchet client: CRUD and ingestion use test-only task stubs, and webhook tests use Prisma directly. They still run against real PostgreSQL and cover owner-scoped bounded history, atomic resource/run transitions, retry races, serving cutover, and terminal-event ordering. + +KB quota coverage must use real PostgreSQL for parent-row lock serialization, exact count/byte boundaries, pending tickets, tombstones, confirmation conversion, and cleanup release. Hatchet unit coverage owns persisted KB-scope rejection plus URL-size replacement arithmetic and the no-dispatch `KB_STORAGE_LIMIT_REACHED` transition. + +KB graph accounting coverage also uses real PostgreSQL: `packages/graphql/test/knowledgeGraphAccounting.test.ts` proves same-owner semester-quota lock serialization, one-time success settlement, metered non-success settlement without publication, dispatch-failure release, cleanup-fenced late success, matching/stale/newer-build late-success reconciliation, bounded actual token/request aggregation, publication only after contract validation, and reservation hold on an invalid result. The disposable migration applies the durable dispatch-claim column before these tests. Pure W1 terminal-result, database-integer-bound, cost-configuration, and quota-drift validation remains in `kbGraphContract.test.ts`, `knowledgeGraphCost.test.ts`, and `knowledgeGraphConfig.test.ts`; `packages/hatchet/test/kbGraphIngestion.test.ts` proves worker-side kill-switch/opt-in/complete-reservation gates, pre-accounting fencing, accepted-but-uncorrelated dispatch holds, provider-status-only reconciliation failure, abort-before-slot-reuse for eight concurrent provider calls, and versioned-result handoff to the settlement callback. + +KB scale coverage uses real PostgreSQL for tied keyset traversal, cursor/filter binding, owner isolation, tombstone hiding, immutable resource-page order during status changes, exact derived metrics, and all-or-nothing bounded bulk deletion. UI appearance and interaction have no component-test layer in this repository: verify the generated-operation typechecks, then exercise catalog/detail search, filters, inspector, selection/confirmation, active polling, EN/DE, and desktop/390 px layouts through the real delegated-login browser path. + **Never run root `pnpm run test:run` blind.** The graphql vitest config forces `pool: forks, singleFork: true` (serialized specs sharing DB state) — don't parallelize it. For OpenAI-compatible chat stream changes, run diff --git a/packages/graphql/package.json b/packages/graphql/package.json index e4d14c1f53..8ca21c5049 100644 --- a/packages/graphql/package.json +++ b/packages/graphql/package.json @@ -13,6 +13,7 @@ "@graphql-yoga/redis-event-target": "1.0.0", "@hatchet-dev/typescript-sdk": "1.9.4", "@klicker-uzh/grading": "workspace:*", + "@klicker-uzh/knowledge-graph": "workspace:*", "@klicker-uzh/prisma": "workspace:*", "@klicker-uzh/types": "workspace:*", "@klicker-uzh/util": "workspace:*", diff --git a/packages/graphql/src/graphql/ops/MAttachKbToChatbot.graphql b/packages/graphql/src/graphql/ops/MAttachKbToChatbot.graphql new file mode 100644 index 0000000000..2d3ed7bf6b --- /dev/null +++ b/packages/graphql/src/graphql/ops/MAttachKbToChatbot.graphql @@ -0,0 +1,8 @@ +mutation AttachKbToChatbot($kbId: ID!, $chatbotId: ID!) { + attachKbToChatbot(kbId: $kbId, chatbotId: $chatbotId) { + chatbotId + chatbotName + enabledKbId + enabledKbName + } +} diff --git a/packages/graphql/src/graphql/ops/MConfirmKbFileUpload.graphql b/packages/graphql/src/graphql/ops/MConfirmKbFileUpload.graphql new file mode 100644 index 0000000000..7e1d3ecba5 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MConfirmKbFileUpload.graphql @@ -0,0 +1,19 @@ +mutation ConfirmKbFileUpload( + $kbId: ID! + $blobName: String! + $title: String! + $originalFilename: String! + $mimeType: String! + $sizeBytes: Int! +) { + confirmKbFileUpload( + kbId: $kbId + blobName: $blobName + title: $title + originalFilename: $originalFilename + mimeType: $mimeType + sizeBytes: $sizeBytes + ) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MCreateKb.graphql b/packages/graphql/src/graphql/ops/MCreateKb.graphql new file mode 100644 index 0000000000..1162bc5bc3 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MCreateKb.graphql @@ -0,0 +1,5 @@ +mutation CreateKb($name: String!, $description: String) { + createKb(name: $name, description: $description) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MCreateKbUrlResource.graphql b/packages/graphql/src/graphql/ops/MCreateKbUrlResource.graphql new file mode 100644 index 0000000000..9cd3377203 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MCreateKbUrlResource.graphql @@ -0,0 +1,5 @@ +mutation CreateKbUrlResource($kbId: ID!, $url: String!, $title: String!) { + createKbUrlResource(kbId: $kbId, url: $url, title: $title) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MDeleteKb.graphql b/packages/graphql/src/graphql/ops/MDeleteKb.graphql new file mode 100644 index 0000000000..7b8b6cb6c2 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MDeleteKb.graphql @@ -0,0 +1,5 @@ +mutation DeleteKb($id: ID!) { + deleteKb(id: $id) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MDeleteKbResource.graphql b/packages/graphql/src/graphql/ops/MDeleteKbResource.graphql new file mode 100644 index 0000000000..a8da067fcb --- /dev/null +++ b/packages/graphql/src/graphql/ops/MDeleteKbResource.graphql @@ -0,0 +1,5 @@ +mutation DeleteKbResource($id: ID!) { + deleteKbResource(id: $id) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MDeleteKbResources.graphql b/packages/graphql/src/graphql/ops/MDeleteKbResources.graphql new file mode 100644 index 0000000000..bf8a73033c --- /dev/null +++ b/packages/graphql/src/graphql/ops/MDeleteKbResources.graphql @@ -0,0 +1,5 @@ +mutation DeleteKbResources($kbId: ID!, $ids: [String!]!) { + deleteKbResources(kbId: $kbId, ids: $ids) { + id + } +} diff --git a/packages/graphql/src/graphql/ops/MDetachKbFromChatbot.graphql b/packages/graphql/src/graphql/ops/MDetachKbFromChatbot.graphql new file mode 100644 index 0000000000..6853f48039 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MDetachKbFromChatbot.graphql @@ -0,0 +1,3 @@ +mutation DetachKbFromChatbot($kbId: ID!, $chatbotId: ID!) { + detachKbFromChatbot(kbId: $kbId, chatbotId: $chatbotId) +} diff --git a/packages/graphql/src/graphql/ops/MIngestKbResource.graphql b/packages/graphql/src/graphql/ops/MIngestKbResource.graphql new file mode 100644 index 0000000000..1d5ce530f8 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MIngestKbResource.graphql @@ -0,0 +1,6 @@ +mutation IngestKbResource($id: ID!) { + ingestKbResource(id: $id) { + id + status + } +} diff --git a/packages/graphql/src/graphql/ops/MRebuildKbKnowledgeGraph.graphql b/packages/graphql/src/graphql/ops/MRebuildKbKnowledgeGraph.graphql new file mode 100644 index 0000000000..e5fcf96507 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MRebuildKbKnowledgeGraph.graphql @@ -0,0 +1,38 @@ +mutation RebuildKbKnowledgeGraph($kbId: ID!, $qualityTier: KBGraphQualityTier) { + rebuildKbKnowledgeGraph(kbId: $kbId, qualityTier: $qualityTier) { + kbId + isEnabled + buildId + status + statusMessage + qualityTier + sourceContentDigest + activeBuildId + publishedBuildId + isStale + startedAt + finishedAt + createdAt + updatedAt + costConfigurationReady + costCurrency + quotaCurrency + billingLabel + standardEstimateMinorUnits + highEstimateMinorUnits + estimatedCostMinorUnits + actualCostMinorUnits + actualInputTokens + actualOutputTokens + actualEmbeddingTokens + actualRequestCount + maxCostMinorUnits + costStatus + semesterKey + semesterQuotaMinorUnits + semesterReservedMinorUnits + semesterSettledMinorUnits + remainingSemesterQuotaMinorUnits + worstCaseRemainingMinorUnits + } +} diff --git a/packages/graphql/src/graphql/ops/MRequestKbFileUpload.graphql b/packages/graphql/src/graphql/ops/MRequestKbFileUpload.graphql new file mode 100644 index 0000000000..1770c98580 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MRequestKbFileUpload.graphql @@ -0,0 +1,17 @@ +mutation RequestKbFileUpload( + $kbId: ID! + $fileName: String! + $contentType: String! + $sizeBytes: Int! +) { + requestKbFileUpload( + kbId: $kbId + fileName: $fileName + contentType: $contentType + sizeBytes: $sizeBytes + ) { + uploadSasURL + containerName + blobName + } +} diff --git a/packages/graphql/src/graphql/ops/MSetKbKnowledgeGraphEnabled.graphql b/packages/graphql/src/graphql/ops/MSetKbKnowledgeGraphEnabled.graphql new file mode 100644 index 0000000000..74e4f47012 --- /dev/null +++ b/packages/graphql/src/graphql/ops/MSetKbKnowledgeGraphEnabled.graphql @@ -0,0 +1,38 @@ +mutation SetKbKnowledgeGraphEnabled($kbId: ID!, $enabled: Boolean!) { + setKbKnowledgeGraphEnabled(kbId: $kbId, enabled: $enabled) { + kbId + isEnabled + buildId + status + statusMessage + qualityTier + sourceContentDigest + activeBuildId + publishedBuildId + isStale + startedAt + finishedAt + createdAt + updatedAt + costConfigurationReady + costCurrency + quotaCurrency + billingLabel + standardEstimateMinorUnits + highEstimateMinorUnits + estimatedCostMinorUnits + actualCostMinorUnits + actualInputTokens + actualOutputTokens + actualEmbeddingTokens + actualRequestCount + maxCostMinorUnits + costStatus + semesterKey + semesterQuotaMinorUnits + semesterReservedMinorUnits + semesterSettledMinorUnits + remainingSemesterQuotaMinorUnits + worstCaseRemainingMinorUnits + } +} diff --git a/packages/graphql/src/graphql/ops/QGetChatbotsInfo.graphql b/packages/graphql/src/graphql/ops/QGetChatbotsInfo.graphql index 86f1278160..ac5abc6fe4 100644 --- a/packages/graphql/src/graphql/ops/QGetChatbotsInfo.graphql +++ b/packages/graphql/src/graphql/ops/QGetChatbotsInfo.graphql @@ -46,6 +46,10 @@ query GetChatbotsInfo { priority allowedToolsCount } + enabledKnowledgeBase { + id + name + } createdAt updatedAt } diff --git a/packages/graphql/src/graphql/ops/QGetKb.graphql b/packages/graphql/src/graphql/ops/QGetKb.graphql new file mode 100644 index 0000000000..26f480f6fb --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKb.graphql @@ -0,0 +1,21 @@ +query GetKb($id: ID!) { + getKb(id: $id) { + id + name + description + metrics { + visibleResourceCount + visibleSizeBytes + unknownSizeResourceCount + quotaResourceCount + quotaSizeBytes + resourceLimit + storageLimitBytes + pendingCleanupCount + pendingCleanupSizeBytes + reservedResourceCount + reservedSizeBytes + linkedConsumerCount + } + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbChatbotBindings.graphql b/packages/graphql/src/graphql/ops/QGetKbChatbotBindings.graphql new file mode 100644 index 0000000000..0842174405 --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbChatbotBindings.graphql @@ -0,0 +1,8 @@ +query GetKbChatbotBindings($kbId: ID!) { + getKbChatbotBindings(kbId: $kbId) { + chatbotId + chatbotName + enabledKbId + enabledKbName + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphConfig.graphql b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphConfig.graphql new file mode 100644 index 0000000000..28098ee285 --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphConfig.graphql @@ -0,0 +1,38 @@ +query GetKbKnowledgeGraphConfig($kbId: ID!) { + getKbKnowledgeGraphConfig(kbId: $kbId) { + kbId + isEnabled + buildId + status + statusMessage + qualityTier + sourceContentDigest + activeBuildId + publishedBuildId + isStale + startedAt + finishedAt + createdAt + updatedAt + costConfigurationReady + costCurrency + quotaCurrency + billingLabel + standardEstimateMinorUnits + highEstimateMinorUnits + estimatedCostMinorUnits + actualCostMinorUnits + actualInputTokens + actualOutputTokens + actualEmbeddingTokens + actualRequestCount + maxCostMinorUnits + costStatus + semesterKey + semesterQuotaMinorUnits + semesterReservedMinorUnits + semesterSettledMinorUnits + remainingSemesterQuotaMinorUnits + worstCaseRemainingMinorUnits + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphNeighbors.graphql b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphNeighbors.graphql new file mode 100644 index 0000000000..da862a318f --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphNeighbors.graphql @@ -0,0 +1,30 @@ +query GetKbKnowledgeGraphNeighbors($kbId: ID!, $nodeId: ID!) { + getKbKnowledgeGraphNeighbors(kbId: $kbId, nodeId: $nodeId) { + kbId + buildId + isStale + nodes { + id + labels + kind + displayLabel + summary + content + degree + sourceReferences { + resourceId + title + reference + } + } + edges { + id + source + target + type + label + properties + } + truncated + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphOverview.graphql b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphOverview.graphql new file mode 100644 index 0000000000..0321c6f7ee --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbKnowledgeGraphOverview.graphql @@ -0,0 +1,30 @@ +query GetKbKnowledgeGraphOverview($kbId: ID!) { + getKbKnowledgeGraphOverview(kbId: $kbId) { + kbId + buildId + isStale + nodes { + id + labels + kind + displayLabel + summary + content + degree + sourceReferences { + resourceId + title + reference + } + } + edges { + id + source + target + type + label + properties + } + truncated + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbResourceIngestionRuns.graphql b/packages/graphql/src/graphql/ops/QGetKbResourceIngestionRuns.graphql new file mode 100644 index 0000000000..f0de94d918 --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbResourceIngestionRuns.graphql @@ -0,0 +1,9 @@ +query GetKbResourceIngestionRuns($resourceId: ID!) { + getKbResourceIngestionRuns(resourceId: $resourceId) { + id + status + resourceVersion + errorCode + createdAt + } +} diff --git a/packages/graphql/src/graphql/ops/QGetKbResources.graphql b/packages/graphql/src/graphql/ops/QGetKbResources.graphql new file mode 100644 index 0000000000..12a7002d38 --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetKbResources.graphql @@ -0,0 +1,43 @@ +query GetKbResources( + $kbId: ID! + $first: Int + $after: String + $search: String + $type: KBResourceType + $status: KBIngestionStatus +) { + getKbResources( + kbId: $kbId + first: $first + after: $after + search: $search + type: $type + status: $status + ) { + items { + id + type + title + sourceUrl + originalFilename + mimeType + sizeBytes + status + ingestedAt + resourceVersion + activeResourceVersion + latestIngestionRun { + id + status + errorCode + } + createdAt + updatedAt + } + pageInfo { + hasNextPage + endCursor + } + totalCount + } +} diff --git a/packages/graphql/src/graphql/ops/QGetUserKbs.graphql b/packages/graphql/src/graphql/ops/QGetUserKbs.graphql new file mode 100644 index 0000000000..0df43c2228 --- /dev/null +++ b/packages/graphql/src/graphql/ops/QGetUserKbs.graphql @@ -0,0 +1,19 @@ +query GetUserKbs($first: Int, $after: String, $search: String) { + getUserKbsConnection(first: $first, after: $after, search: $search) { + items { + id + name + description + metrics { + visibleResourceCount + visibleSizeBytes + linkedConsumerCount + } + } + pageInfo { + hasNextPage + endCursor + } + totalCount + } +} diff --git a/packages/graphql/src/graphql/ops/QSearchKbKnowledgeGraph.graphql b/packages/graphql/src/graphql/ops/QSearchKbKnowledgeGraph.graphql new file mode 100644 index 0000000000..04fbcd390d --- /dev/null +++ b/packages/graphql/src/graphql/ops/QSearchKbKnowledgeGraph.graphql @@ -0,0 +1,30 @@ +query SearchKbKnowledgeGraph($kbId: ID!, $query: String!) { + searchKbKnowledgeGraph(kbId: $kbId, query: $query) { + kbId + buildId + isStale + nodes { + id + labels + kind + displayLabel + summary + content + degree + sourceReferences { + resourceId + title + reference + } + } + edges { + id + source + target + type + label + properties + } + truncated + } +} diff --git a/packages/graphql/src/index.ts b/packages/graphql/src/index.ts index 4f359378d7..3e95339892 100644 --- a/packages/graphql/src/index.ts +++ b/packages/graphql/src/index.ts @@ -1,6 +1,15 @@ import type { HatchetHandlers } from '@klicker-uzh/types' export { default as enhanceContext } from './lib/context.js' +export { settleKbKnowledgeGraphResult } from './services/knowledge.js' +export { + handleKBSourceGateway, + type KBSourceGatewayResult, +} from './services/knowledgeSourceGateway.js' +export { + handleKBIngestionWebhook, + signKBIngestionWebhook, +} from './services/knowledgeWebhooks.js' import builder from './builder.js' @@ -12,6 +21,8 @@ import './schema/element.js' import './schema/elementData.js' import './schema/evaluation.js' import './schema/groupActivity.js' +import './schema/kbKnowledgeGraph.js' +import './schema/knowledge.js' import './schema/microLearning.js' import './schema/participant.js' import './schema/participantInvitation.js' diff --git a/packages/graphql/src/ops.schema.json b/packages/graphql/src/ops.schema.json index 2315202351..9f92e7c2bc 100644 --- a/packages/graphql/src/ops.schema.json +++ b/packages/graphql/src/ops.schema.json @@ -8218,6 +8218,18 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "enabledKnowledgeBase", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "ChatbotKnowledgeBaseSummary", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "id", "description": null, @@ -8428,6 +8440,50 @@ "enumValues": null, "possibleTypes": null }, + { + "kind": "OBJECT", + "name": "ChatbotKnowledgeBaseSummary", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "name", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, { "kind": "OBJECT", "name": "ChatbotMcpConfigurationSummary", @@ -19999,24 +20055,28 @@ }, { "kind": "OBJECT", - "name": "LeaderboardEntry", + "name": "KB", "description": null, "isOneOf": null, "fields": [ { - "name": "avatar", + "name": "createdAt", "description": null, "args": [], "type": { - "kind": "SCALAR", - "name": "String", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "email", + "name": "description", "description": null, "args": [], "type": { @@ -20036,7 +20096,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "Int", + "name": "ID", "ofType": null } }, @@ -20044,67 +20104,79 @@ "deprecationReason": null }, { - "name": "isSelf", + "name": "metrics", "description": null, "args": [], "type": { - "kind": "SCALAR", - "name": "Boolean", - "ofType": null - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "isTemporary", - "description": null, - "args": [], - "type": { - "kind": "SCALAR", - "name": "Boolean", + "kind": "OBJECT", + "name": "KBMetrics", "ofType": null }, "isDeprecated": false, "deprecationReason": null }, { - "name": "lastBlockOrder", + "name": "name", "description": null, "args": [], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "level", + "name": "updatedAt", "description": null, "args": [], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null - }, + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBChatbotBinding", + "description": null, + "isOneOf": null, + "fields": [ { - "name": "participant", + "name": "chatbotId", "description": null, "args": [], "type": { - "kind": "OBJECT", - "name": "Participant", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "participantId", + "name": "chatbotName", "description": null, "args": [], "type": { @@ -20120,43 +20192,75 @@ "deprecationReason": null }, { - "name": "participation", + "name": "enabledKbId", "description": null, "args": [], "type": { - "kind": "OBJECT", - "name": "Participation", + "kind": "SCALAR", + "name": "ID", "ofType": null }, "isDeprecated": false, "deprecationReason": null }, { - "name": "rank", + "name": "enabledKbName", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBConnection", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "items", "description": null, "args": [], "type": { "kind": "NON_NULL", "name": null, "ofType": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KB", + "ofType": null + } + } } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "score", + "name": "pageInfo", "description": null, "args": [], "type": { "kind": "NON_NULL", "name": null, "ofType": { - "kind": "SCALAR", - "name": "Float", + "kind": "OBJECT", + "name": "KBPageInfo", "ofType": null } }, @@ -20164,7 +20268,7 @@ "deprecationReason": null }, { - "name": "username", + "name": "totalCount", "description": null, "args": [], "type": { @@ -20172,7 +20276,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Int", "ofType": null } }, @@ -20187,12 +20291,12 @@ }, { "kind": "OBJECT", - "name": "LeaderboardStatistics", + "name": "KBFileUpload", "description": null, "isOneOf": null, "fields": [ { - "name": "averageScore", + "name": "blobName", "description": null, "args": [], "type": { @@ -20200,7 +20304,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "Float", + "name": "String", "ofType": null } }, @@ -20208,35 +20312,7 @@ "deprecationReason": null }, { - "name": "participantCount", - "description": null, - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "Int", - "ofType": null - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "LeaveCourseParticipation", - "description": null, - "isOneOf": null, - "fields": [ - { - "name": "id", + "name": "containerName", "description": null, "args": [], "type": { @@ -20252,15 +20328,15 @@ "deprecationReason": null }, { - "name": "participation", + "name": "uploadSasURL", "description": null, "args": [], "type": { "kind": "NON_NULL", "name": null, "ofType": { - "kind": "OBJECT", - "name": "Participation", + "kind": "SCALAR", + "name": "String", "ofType": null } }, @@ -20274,206 +20350,2272 @@ "possibleTypes": null }, { - "kind": "OBJECT", - "name": "Level", + "kind": "ENUM", + "name": "KBGraphBuildStatus", "description": null, "isOneOf": null, - "fields": [ + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ { - "name": "avatar", + "name": "FAILED", "description": null, - "args": [], - "type": { - "kind": "SCALAR", - "name": "String", - "ofType": null - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "id", + "name": "PROCESSING", "description": null, - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "Int", - "ofType": null - } - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "index", + "name": "QUEUED", "description": null, - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "Int", - "ofType": null - } - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "name", + "name": "SUCCEEDED", "description": null, - "args": [], - "type": { - "kind": "SCALAR", - "name": "String", - "ofType": null - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "nextLevel", - "description": null, - "args": [], - "type": { - "kind": "OBJECT", - "name": "Level", - "ofType": null - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "requiredXp", + "name": "SUPERSEDED", "description": null, - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "Int", - "ofType": null - } - }, "isDeprecated": false, "deprecationReason": null } ], - "inputFields": null, - "interfaces": [], - "enumValues": null, "possibleTypes": null }, { - "kind": "OBJECT", - "name": "LiveQuiz", + "kind": "ENUM", + "name": "KBGraphCostStatus", "description": null, "isOneOf": null, - "fields": [ + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ { - "name": "accessMode", + "name": "NEEDS_HUMAN_REVIEW", "description": null, - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "ENUM", - "name": "LiveQuizAccessMode", - "ofType": null - } - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "activeBlock", + "name": "RELEASED", "description": null, - "args": [], - "type": { - "kind": "OBJECT", - "name": "ElementBlock", - "ofType": null - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "beforeFirstBlock", + "name": "RESERVED", "description": null, - "args": [], - "type": { - "kind": "SCALAR", - "name": "Boolean", - "ofType": null - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "blocks", + "name": "SETTLED", "description": null, - "args": [], - "type": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "OBJECT", - "name": "ElementBlock", - "ofType": null - } - } - }, "isDeprecated": false, "deprecationReason": null - }, + } + ], + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "KBGraphQualityTier", + "description": null, + "isOneOf": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ { - "name": "confusionFeedbacks", + "name": "HIGH", "description": null, - "args": [], - "type": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "OBJECT", - "name": "ConfusionTimestep", - "ofType": null - } - } - }, "isDeprecated": false, "deprecationReason": null }, { - "name": "confusionSummary", + "name": "STANDARD", "description": null, - "args": [], - "type": { - "kind": "OBJECT", - "name": "ConfusionSummary", - "ofType": null - }, "isDeprecated": false, "deprecationReason": null - }, + } + ], + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBIngestionRun", + "description": null, + "isOneOf": null, + "fields": [ { - "name": "course", + "name": "contentSha256", "description": null, "args": [], "type": { - "kind": "OBJECT", - "name": "Course", + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "createdAt", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errorCode", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "finishedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "resourceVersion", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "startedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "status", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "KBIngestionStatus", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "statusMessage", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "updatedAt", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "KBIngestionStatus", + "description": null, + "isOneOf": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "FAILED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "PROCESSING", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "QUEUED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "SUCCEEDED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "SUPERSEDED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBKnowledgeGraphConfig", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "activeBuildId", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "actualCostMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "actualEmbeddingTokens", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "actualInputTokens", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "actualOutputTokens", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "actualRequestCount", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "billingLabel", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "buildId", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "costConfigurationReady", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "costCurrency", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "costStatus", + "description": null, + "args": [], + "type": { + "kind": "ENUM", + "name": "KBGraphCostStatus", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "createdAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "estimatedCostMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "finishedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "highEstimateMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "isEnabled", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "isStale", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "maxCostMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "publishedBuildId", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "qualityTier", + "description": null, + "args": [], + "type": { + "kind": "ENUM", + "name": "KBGraphQualityTier", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "quotaCurrency", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "remainingSemesterQuotaMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "semesterKey", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "semesterQuotaMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "semesterReservedMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "semesterSettledMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sourceContentDigest", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "standardEstimateMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "startedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "status", + "description": null, + "args": [], + "type": { + "kind": "ENUM", + "name": "KBGraphBuildStatus", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "statusMessage", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "updatedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "worstCaseRemainingMinorUnits", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBMetrics", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "linkedConsumerCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pendingCleanupCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pendingCleanupSizeBytes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "quotaResourceCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "quotaSizeBytes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "reservedResourceCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "reservedSizeBytes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "resourceLimit", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "storageLimitBytes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "unknownSizeResourceCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "visibleResourceCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "visibleSizeBytes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBPageInfo", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "endCursor", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "hasNextPage", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBResource", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "activeContentSha256", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "activeResourceVersion", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "createdAt", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errorCode", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "ingestedAt", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "latestIngestionRun", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "KBIngestionRun", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "mimeType", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "originalFilename", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "resourceVersion", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sizeBytes", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sourceUrl", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "status", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "KBResourceStatus", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "statusMessage", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "title", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "type", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "KBResourceType", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "updatedAt", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Date", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KBResourceConnection", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "items", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pageInfo", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBPageInfo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "totalCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "KBResourceStatus", + "description": null, + "isOneOf": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "ADDED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "FAILED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "PROCESSING", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "QUEUED", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "READY", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "KBResourceType", + "description": null, + "isOneOf": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "BLOB", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "URL", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KnowledgeGraphEdge", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "label", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "properties", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Json", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "source", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "target", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "type", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KnowledgeGraphNode", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "content", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "degree", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "displayLabel", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kind", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "labels", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sourceReferences", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphSourceReference", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "summary", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KnowledgeGraphResponse", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "buildId", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "edges", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphEdge", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "isStale", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nodes", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphNode", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "truncated", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "KnowledgeGraphSourceReference", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "reference", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "resourceId", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "title", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "LeaderboardEntry", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "avatar", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "email", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "isSelf", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "isTemporary", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "lastBlockOrder", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "level", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "participant", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "Participant", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "participantId", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "participation", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "Participation", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "rank", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "score", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Float", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "username", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "LeaderboardStatistics", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "averageScore", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Float", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "participantCount", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "LeaveCourseParticipation", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "participation", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "Participation", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "Level", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "avatar", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "index", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "name", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nextLevel", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "Level", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "requiredXp", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "LiveQuiz", + "description": null, + "isOneOf": null, + "fields": [ + { + "name": "accessMode", + "description": null, + "args": [], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "LiveQuizAccessMode", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "activeBlock", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "ElementBlock", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "beforeFirstBlock", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "blocks", + "description": null, + "args": [], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "ElementBlock", + "ofType": null + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "confusionFeedbacks", + "description": null, + "args": [], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "ConfusionTimestep", + "ofType": null + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "confusionSummary", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "ConfusionSummary", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "course", + "description": null, + "args": [], + "type": { + "kind": "OBJECT", + "name": "Course", "ofType": null }, "isDeprecated": false, @@ -22422,6 +24564,55 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "attachKbToChatbot", + "description": null, + "args": [ + { + "name": "chatbotId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBChatbotBinding", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "bookmarkElementStack", "description": null, @@ -23234,6 +25425,119 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "confirmKbFileUpload", + "description": null, + "args": [ + { + "name": "blobName", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "mimeType", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "originalFilename", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sizeBytes", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "title", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "copyCatalogObjectToAccount", "description": null, @@ -24408,6 +26712,116 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "createKb", + "description": null, + "args": [ + { + "name": "description", + "description": null, + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "name", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KB", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "createKbUrlResource", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "title", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "url", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "createLiveQuiz", "description": null, @@ -25784,6 +28198,137 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "deleteKb", + "description": null, + "args": [ + { + "name": "id", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KB", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "deleteKbResource", + "description": null, + "args": [ + { + "name": "id", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "deleteKbResources", + "description": null, + "args": [ + { + "name": "ids", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "deleteLiveQuiz", "description": null, @@ -25836,46 +28381,104 @@ ], "type": { "kind": "OBJECT", - "name": "MicroLearning", - "ofType": null - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "deleteParticipantAccount", - "description": null, - "args": [], - "type": { - "kind": "SCALAR", - "name": "Boolean", + "name": "MicroLearning", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "deleteParticipantAccount", + "description": null, + "args": [], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "deletePendingAssessmentParticipantInvitation", + "description": null, + "args": [ + { + "name": "courseId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "invitationId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "OBJECT", + "name": "AssessmentParticipantInvitation", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "deletePracticeQuiz", + "description": null, + "args": [ + { + "name": "id", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "OBJECT", + "name": "PracticeQuiz", "ofType": null }, "isDeprecated": false, "deprecationReason": null }, { - "name": "deletePendingAssessmentParticipantInvitation", + "name": "deleteTag", "description": null, "args": [ { - "name": "courseId", - "description": null, - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "String", - "ofType": null - } - }, - "defaultValue": null, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "invitationId", + "name": "id", "description": null, "type": { "kind": "NON_NULL", @@ -25893,25 +28496,25 @@ ], "type": { "kind": "OBJECT", - "name": "AssessmentParticipantInvitation", + "name": "Tag", "ofType": null }, "isDeprecated": false, "deprecationReason": null }, { - "name": "deletePracticeQuiz", + "name": "deleteUserGroup", "description": null, "args": [ { - "name": "id", + "name": "groupId", "description": null, "type": { "kind": "NON_NULL", "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Int", "ofType": null } }, @@ -25921,15 +28524,19 @@ } ], "type": { - "kind": "OBJECT", - "name": "PracticeQuiz", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "deleteTag", + "name": "deleteUserLogin", "description": null, "args": [ { @@ -25940,7 +28547,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "Int", + "name": "String", "ofType": null } }, @@ -25951,58 +28558,41 @@ ], "type": { "kind": "OBJECT", - "name": "Tag", + "name": "UserLogin", "ofType": null }, "isDeprecated": false, "deprecationReason": null }, { - "name": "deleteUserGroup", + "name": "demoteGroupAdminToMember", "description": null, "args": [ { - "name": "groupId", + "name": "adminId", "description": null, "type": { "kind": "NON_NULL", "name": null, "ofType": { "kind": "SCALAR", - "name": "Int", + "name": "String", "ofType": null } }, "defaultValue": null, "isDeprecated": false, "deprecationReason": null - } - ], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "SCALAR", - "name": "Boolean", - "ofType": null - } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "deleteUserLogin", - "description": null, - "args": [ + }, { - "name": "id", + "name": "groupId", "description": null, "type": { "kind": "NON_NULL", "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Int", "ofType": null } }, @@ -26012,26 +28602,30 @@ } ], "type": { - "kind": "OBJECT", - "name": "UserLogin", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "demoteGroupAdminToMember", + "name": "detachKbFromChatbot", "description": null, "args": [ { - "name": "adminId", + "name": "chatbotId", "description": null, "type": { "kind": "NON_NULL", "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "ID", "ofType": null } }, @@ -26040,14 +28634,14 @@ "deprecationReason": null }, { - "name": "groupId", + "name": "kbId", "description": null, "type": { "kind": "NON_NULL", "name": null, "ofType": { "kind": "SCALAR", - "name": "Int", + "name": "ID", "ofType": null } }, @@ -27622,6 +30216,39 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "ingestKbResource", + "description": null, + "args": [ + { + "name": "id", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResource", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "issueAssessmentReport", "description": null, @@ -29534,6 +32161,51 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "rebuildKbKnowledgeGraph", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "qualityTier", + "description": null, + "type": { + "kind": "ENUM", + "name": "KBGraphQualityTier", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBKnowledgeGraphConfig", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "removeCatalogObjectAssignment", "description": null, @@ -29869,6 +32541,87 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "requestKbFileUpload", + "description": null, + "args": [ + { + "name": "contentType", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "fileName", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sizeBytes", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBFileUpload", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "resetAssessmentLiveQuiz", "description": null, @@ -30449,6 +33202,55 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "setKbKnowledgeGraphEnabled", + "description": null, + "args": [ + { + "name": "enabled", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBKnowledgeGraphConfig", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "setLiveQuizPin", "description": null, @@ -38489,6 +41291,329 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "getKb", + "description": null, + "args": [ + { + "name": "id", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KB", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbChatbotBindings", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBChatbotBinding", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbKnowledgeGraphConfig", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBKnowledgeGraphConfig", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbKnowledgeGraphNeighbors", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nodeId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphResponse", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbKnowledgeGraphOverview", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphResponse", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbResourceIngestionRuns", + "description": null, + "args": [ + { + "name": "resourceId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBIngestionRun", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "getKbResources", + "description": null, + "args": [ + { + "name": "after", + "description": null, + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "first", + "description": null, + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "search", + "description": null, + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "status", + "description": null, + "type": { + "kind": "ENUM", + "name": "KBIngestionStatus", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "type", + "description": null, + "type": { + "kind": "ENUM", + "name": "KBResourceType", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBResourceConnection", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "getLecturerViewLiveQuiz", "description": null, @@ -39338,6 +42463,59 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "getUserKbsConnection", + "description": null, + "args": [ + { + "name": "after", + "description": null, + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "first", + "description": null, + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "search", + "description": null, + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KBConnection", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "getUsersAiFeatures", "description": null, @@ -39960,6 +43138,55 @@ "isDeprecated": false, "deprecationReason": null }, + { + "name": "searchKbKnowledgeGraph", + "description": null, + "args": [ + { + "name": "kbId", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "query", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "KnowledgeGraphResponse", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, { "name": "self", "description": null, diff --git a/packages/graphql/src/ops.ts b/packages/graphql/src/ops.ts index 6521bfb957..6cbdf06c86 100644 --- a/packages/graphql/src/ops.ts +++ b/packages/graphql/src/ops.ts @@ -806,6 +806,7 @@ export type Chatbot = { creditResetPeriod: CreditResetPeriod; description?: Maybe; disclaimerSummary?: Maybe; + enabledKnowledgeBase?: Maybe; id: Scalars['ID']['output']; mcpConfigurations: Array; modelSelection: Scalars['Boolean']['output']; @@ -824,6 +825,12 @@ export type ChatbotDisclaimerSummary = { title: Scalars['String']['output']; }; +export type ChatbotKnowledgeBaseSummary = { + __typename?: 'ChatbotKnowledgeBaseSummary'; + id: Scalars['ID']['output']; + name: Scalars['String']['output']; +}; + export type ChatbotMcpConfigurationSummary = { __typename?: 'ChatbotMcpConfigurationSummary'; allowedToolsCount?: Maybe; @@ -1927,6 +1934,220 @@ export type IssuedAssessmentReport = { token: Scalars['String']['output']; }; +export type Kb = { + __typename?: 'KB'; + createdAt: Scalars['Date']['output']; + description?: Maybe; + id: Scalars['ID']['output']; + metrics?: Maybe; + name: Scalars['String']['output']; + updatedAt: Scalars['Date']['output']; +}; + +export type KbChatbotBinding = { + __typename?: 'KBChatbotBinding'; + chatbotId: Scalars['ID']['output']; + chatbotName: Scalars['String']['output']; + enabledKbId?: Maybe; + enabledKbName?: Maybe; +}; + +export type KbConnection = { + __typename?: 'KBConnection'; + items: Array; + pageInfo: KbPageInfo; + totalCount: Scalars['Int']['output']; +}; + +export type KbFileUpload = { + __typename?: 'KBFileUpload'; + blobName: Scalars['String']['output']; + containerName: Scalars['String']['output']; + uploadSasURL: Scalars['String']['output']; +}; + +export enum KbGraphBuildStatus { + Failed = 'FAILED', + Processing = 'PROCESSING', + Queued = 'QUEUED', + Succeeded = 'SUCCEEDED', + Superseded = 'SUPERSEDED' +} + +export enum KbGraphCostStatus { + NeedsHumanReview = 'NEEDS_HUMAN_REVIEW', + Released = 'RELEASED', + Reserved = 'RESERVED', + Settled = 'SETTLED' +} + +export enum KbGraphQualityTier { + High = 'HIGH', + Standard = 'STANDARD' +} + +export type KbIngestionRun = { + __typename?: 'KBIngestionRun'; + contentSha256?: Maybe; + createdAt: Scalars['Date']['output']; + errorCode?: Maybe; + finishedAt?: Maybe; + id: Scalars['ID']['output']; + resourceVersion: Scalars['Int']['output']; + startedAt?: Maybe; + status: KbIngestionStatus; + statusMessage?: Maybe; + updatedAt: Scalars['Date']['output']; +}; + +export enum KbIngestionStatus { + Failed = 'FAILED', + Processing = 'PROCESSING', + Queued = 'QUEUED', + Succeeded = 'SUCCEEDED', + Superseded = 'SUPERSEDED' +} + +export type KbKnowledgeGraphConfig = { + __typename?: 'KBKnowledgeGraphConfig'; + activeBuildId?: Maybe; + actualCostMinorUnits?: Maybe; + actualEmbeddingTokens?: Maybe; + actualInputTokens?: Maybe; + actualOutputTokens?: Maybe; + actualRequestCount?: Maybe; + billingLabel?: Maybe; + buildId?: Maybe; + costConfigurationReady: Scalars['Boolean']['output']; + costCurrency?: Maybe; + costStatus?: Maybe; + createdAt?: Maybe; + estimatedCostMinorUnits?: Maybe; + finishedAt?: Maybe; + highEstimateMinorUnits?: Maybe; + isEnabled: Scalars['Boolean']['output']; + isStale: Scalars['Boolean']['output']; + kbId: Scalars['ID']['output']; + maxCostMinorUnits?: Maybe; + publishedBuildId?: Maybe; + qualityTier?: Maybe; + quotaCurrency?: Maybe; + remainingSemesterQuotaMinorUnits?: Maybe; + semesterKey?: Maybe; + semesterQuotaMinorUnits?: Maybe; + semesterReservedMinorUnits?: Maybe; + semesterSettledMinorUnits?: Maybe; + sourceContentDigest?: Maybe; + standardEstimateMinorUnits?: Maybe; + startedAt?: Maybe; + status?: Maybe; + statusMessage?: Maybe; + updatedAt?: Maybe; + worstCaseRemainingMinorUnits?: Maybe; +}; + +export type KbMetrics = { + __typename?: 'KBMetrics'; + linkedConsumerCount: Scalars['Int']['output']; + pendingCleanupCount: Scalars['Int']['output']; + pendingCleanupSizeBytes: Scalars['Int']['output']; + quotaResourceCount: Scalars['Int']['output']; + quotaSizeBytes: Scalars['Int']['output']; + reservedResourceCount: Scalars['Int']['output']; + reservedSizeBytes: Scalars['Int']['output']; + resourceLimit: Scalars['Int']['output']; + storageLimitBytes: Scalars['Int']['output']; + unknownSizeResourceCount: Scalars['Int']['output']; + visibleResourceCount: Scalars['Int']['output']; + visibleSizeBytes: Scalars['Int']['output']; +}; + +export type KbPageInfo = { + __typename?: 'KBPageInfo'; + endCursor?: Maybe; + hasNextPage: Scalars['Boolean']['output']; +}; + +export type KbResource = { + __typename?: 'KBResource'; + activeContentSha256?: Maybe; + activeResourceVersion?: Maybe; + createdAt: Scalars['Date']['output']; + errorCode?: Maybe; + id: Scalars['ID']['output']; + ingestedAt?: Maybe; + latestIngestionRun?: Maybe; + mimeType?: Maybe; + originalFilename?: Maybe; + resourceVersion: Scalars['Int']['output']; + sizeBytes?: Maybe; + sourceUrl?: Maybe; + status: KbResourceStatus; + statusMessage?: Maybe; + title: Scalars['String']['output']; + type: KbResourceType; + updatedAt: Scalars['Date']['output']; +}; + +export type KbResourceConnection = { + __typename?: 'KBResourceConnection'; + items: Array; + pageInfo: KbPageInfo; + totalCount: Scalars['Int']['output']; +}; + +export enum KbResourceStatus { + Added = 'ADDED', + Failed = 'FAILED', + Processing = 'PROCESSING', + Queued = 'QUEUED', + Ready = 'READY' +} + +export enum KbResourceType { + Blob = 'BLOB', + Url = 'URL' +} + +export type KnowledgeGraphEdge = { + __typename?: 'KnowledgeGraphEdge'; + id: Scalars['ID']['output']; + label: Scalars['String']['output']; + properties: Scalars['Json']['output']; + source: Scalars['ID']['output']; + target: Scalars['ID']['output']; + type: Scalars['String']['output']; +}; + +export type KnowledgeGraphNode = { + __typename?: 'KnowledgeGraphNode'; + content?: Maybe; + degree: Scalars['Int']['output']; + displayLabel: Scalars['String']['output']; + id: Scalars['ID']['output']; + kind: Scalars['String']['output']; + labels: Array; + sourceReferences: Array; + summary?: Maybe; +}; + +export type KnowledgeGraphResponse = { + __typename?: 'KnowledgeGraphResponse'; + buildId: Scalars['ID']['output']; + edges: Array; + isStale: Scalars['Boolean']['output']; + kbId: Scalars['ID']['output']; + nodes: Array; + truncated: Scalars['Boolean']['output']; +}; + +export type KnowledgeGraphSourceReference = { + __typename?: 'KnowledgeGraphSourceReference'; + reference?: Maybe; + resourceId: Scalars['ID']['output']; + title: Scalars['String']['output']; +}; + export type LeaderboardEntry = { __typename?: 'LeaderboardEntry'; avatar?: Maybe; @@ -2104,6 +2325,7 @@ export type Mutation = { applyActivityBatchOperations: Scalars['Int']['output']; applyElementBatchOperations: Scalars['Int']['output']; approveObjectSharingRequest: Scalars['Boolean']['output']; + attachKbToChatbot: KbChatbotBinding; bookmarkElementStack?: Maybe>; cancelLiveQuiz?: Maybe; cancelObjectSharingRequest: Scalars['Boolean']['output']; @@ -2120,6 +2342,7 @@ export type Mutation = { changeShortname?: Maybe; changeUserGroupName: Scalars['Boolean']['output']; changeUserLocale?: Maybe; + confirmKbFileUpload: KbResource; copyCatalogObjectToAccount: Scalars['Boolean']['output']; correctAssessmentPointsInstance?: Maybe; correctAssessmentPointsLiveQuiz?: Maybe; @@ -2130,6 +2353,8 @@ export type Mutation = { createCourse?: Maybe; createFeedback?: Maybe; createGroupActivity?: Maybe; + createKb: Kb; + createKbUrlResource: KbResource; createLiveQuiz?: Maybe; createLiveQuizFromTemplate?: Maybe; createMicroLearning?: Maybe; @@ -2150,6 +2375,9 @@ export type Mutation = { deleteFeedback?: Maybe; deleteFeedbackResponse?: Maybe; deleteGroupActivity?: Maybe; + deleteKb: Kb; + deleteKbResource: KbResource; + deleteKbResources: Array; deleteLiveQuiz?: Maybe; deleteMicroLearning?: Maybe; deleteParticipantAccount?: Maybe; @@ -2159,6 +2387,7 @@ export type Mutation = { deleteUserGroup: Scalars['Boolean']['output']; deleteUserLogin?: Maybe; demoteGroupAdminToMember: Scalars['Boolean']['output']; + detachKbFromChatbot: Scalars['Boolean']['output']; duplicateAnswerCollection?: Maybe; editActivityTemplate: Scalars['Boolean']['output']; editAnswerCollectionEntry?: Maybe; @@ -2181,6 +2410,7 @@ export type Mutation = { gradeGroupActivitySubmission?: Maybe; grantPrivatePreviewAccess?: Maybe; importCatalogObject: Scalars['Boolean']['output']; + ingestKbResource: KbResource; issueAssessmentReport: IssuedAssessmentReport; joinCourseLeaderboard?: Maybe; joinCourseWithPin?: Maybe; @@ -2215,6 +2445,7 @@ export type Mutation = { publishMicroLearning?: Maybe; publishPracticeQuiz?: Maybe; rateElement?: Maybe; + rebuildKbKnowledgeGraph: KbKnowledgeGraphConfig; removeCatalogObjectAssignment: Scalars['Boolean']['output']; removeObject?: Maybe; removeUserFromGroup: Scalars['Boolean']['output']; @@ -2222,6 +2453,7 @@ export type Mutation = { requestCatalogCollection?: Maybe; requestCatalogObject: Scalars['Boolean']['output']; requestCatalystAccess: Scalars['Boolean']['output']; + requestKbFileUpload: KbFileUpload; resetAssessmentLiveQuiz?: Maybe; resolveActivityLogEntry?: Maybe; resolveFeedback?: Maybe; @@ -2234,6 +2466,7 @@ export type Mutation = { setActivityReviewStatus?: Maybe; setAiFeatures?: Maybe; setBetaFeatures?: Maybe; + setKbKnowledgeGraphEnabled: KbKnowledgeGraphConfig; setLiveQuizPin: Scalars['Boolean']['output']; shareElementsBatch: ElementBatchSharingResult; shareObject?: Maybe; @@ -2345,6 +2578,12 @@ export type MutationApproveObjectSharingRequestArgs = { }; +export type MutationAttachKbToChatbotArgs = { + chatbotId: Scalars['ID']['input']; + kbId: Scalars['ID']['input']; +}; + + export type MutationBookmarkElementStackArgs = { bookmarked: Scalars['Boolean']['input']; courseId: Scalars['String']['input']; @@ -2446,6 +2685,16 @@ export type MutationChangeUserLocaleArgs = { }; +export type MutationConfirmKbFileUploadArgs = { + blobName: Scalars['String']['input']; + kbId: Scalars['ID']['input']; + mimeType: Scalars['String']['input']; + originalFilename: Scalars['String']['input']; + sizeBytes: Scalars['Int']['input']; + title: Scalars['String']['input']; +}; + + export type MutationCopyCatalogObjectToAccountArgs = { catalogCollectionId?: InputMaybe; objectId: Scalars['String']['input']; @@ -2555,6 +2804,19 @@ export type MutationCreateGroupActivityArgs = { }; +export type MutationCreateKbArgs = { + description?: InputMaybe; + name: Scalars['String']['input']; +}; + + +export type MutationCreateKbUrlResourceArgs = { + kbId: Scalars['ID']['input']; + title: Scalars['String']['input']; + url: Scalars['String']['input']; +}; + + export type MutationCreateLiveQuizArgs = { blocks: Array; courseId?: InputMaybe; @@ -2705,6 +2967,22 @@ export type MutationDeleteGroupActivityArgs = { }; +export type MutationDeleteKbArgs = { + id: Scalars['ID']['input']; +}; + + +export type MutationDeleteKbResourceArgs = { + id: Scalars['ID']['input']; +}; + + +export type MutationDeleteKbResourcesArgs = { + ids: Array; + kbId: Scalars['ID']['input']; +}; + + export type MutationDeleteLiveQuizArgs = { id: Scalars['String']['input']; }; @@ -2747,6 +3025,12 @@ export type MutationDemoteGroupAdminToMemberArgs = { }; +export type MutationDetachKbFromChatbotArgs = { + chatbotId: Scalars['ID']['input']; + kbId: Scalars['ID']['input']; +}; + + export type MutationDuplicateAnswerCollectionArgs = { id: Scalars['Int']['input']; }; @@ -2914,6 +3198,11 @@ export type MutationImportCatalogObjectArgs = { }; +export type MutationIngestKbResourceArgs = { + id: Scalars['ID']['input']; +}; + + export type MutationIssueAssessmentReportArgs = { courseId: Scalars['String']['input']; }; @@ -3145,6 +3434,12 @@ export type MutationRateElementArgs = { }; +export type MutationRebuildKbKnowledgeGraphArgs = { + kbId: Scalars['ID']['input']; + qualityTier?: InputMaybe; +}; + + export type MutationRemoveCatalogObjectAssignmentArgs = { assignmentId: Scalars['Int']['input']; }; @@ -3188,6 +3483,14 @@ export type MutationRequestCatalystAccessArgs = { }; +export type MutationRequestKbFileUploadArgs = { + contentType: Scalars['String']['input']; + fileName: Scalars['String']['input']; + kbId: Scalars['ID']['input']; + sizeBytes: Scalars['Int']['input']; +}; + + export type MutationResetAssessmentLiveQuizArgs = { id: Scalars['String']['input']; }; @@ -3262,6 +3565,12 @@ export type MutationSetBetaFeaturesArgs = { }; +export type MutationSetKbKnowledgeGraphEnabledArgs = { + enabled: Scalars['Boolean']['input']; + kbId: Scalars['ID']['input']; +}; + + export type MutationSetLiveQuizPinArgs = { liveQuizId: Scalars['String']['input']; pin: Scalars['String']['input']; @@ -3952,6 +4261,13 @@ export type Query = { getGradingGroupActivity?: Maybe; getGroupActivitySummary?: Maybe; getInstanceUpdateActivities?: Maybe>; + getKb: Kb; + getKbChatbotBindings: Array; + getKbKnowledgeGraphConfig: KbKnowledgeGraphConfig; + getKbKnowledgeGraphNeighbors: KnowledgeGraphResponse; + getKbKnowledgeGraphOverview: KnowledgeGraphResponse; + getKbResourceIngestionRuns: Array; + getKbResources: KbResourceConnection; getLecturerViewLiveQuiz?: Maybe; getLiveQuizEmbeddingInfo?: Maybe; getLiveQuizSummary?: Maybe; @@ -3977,6 +4293,7 @@ export type Query = { getTemplatePreviewAnswerCollectionEntries?: Maybe>; getUserActivitiesCourses?: Maybe>; getUserGroupsUser?: Maybe>; + getUserKbsConnection: KbConnection; getUsersAiFeatures?: Maybe>; getUsersPrivatePreview?: Maybe>; groupActivities?: Maybe>; @@ -3994,6 +4311,7 @@ export type Query = { practiceQuiz?: Maybe; previousPointCorrections?: Maybe>; publicParticipantProfile?: Maybe; + searchKbKnowledgeGraph: KnowledgeGraphResponse; self?: Maybe; selfWithAchievements?: Maybe; shortnameQuizzes?: Maybe>; @@ -4291,6 +4609,47 @@ export type QueryGetInstanceUpdateActivitiesArgs = { }; +export type QueryGetKbArgs = { + id: Scalars['ID']['input']; +}; + + +export type QueryGetKbChatbotBindingsArgs = { + kbId: Scalars['ID']['input']; +}; + + +export type QueryGetKbKnowledgeGraphConfigArgs = { + kbId: Scalars['ID']['input']; +}; + + +export type QueryGetKbKnowledgeGraphNeighborsArgs = { + kbId: Scalars['ID']['input']; + nodeId: Scalars['ID']['input']; +}; + + +export type QueryGetKbKnowledgeGraphOverviewArgs = { + kbId: Scalars['ID']['input']; +}; + + +export type QueryGetKbResourceIngestionRunsArgs = { + resourceId: Scalars['ID']['input']; +}; + + +export type QueryGetKbResourcesArgs = { + after?: InputMaybe; + first?: InputMaybe; + kbId: Scalars['ID']['input']; + search?: InputMaybe; + status?: InputMaybe; + type?: InputMaybe; +}; + + export type QueryGetLecturerViewLiveQuizArgs = { id: Scalars['String']['input']; }; @@ -4403,6 +4762,13 @@ export type QueryGetTemplatePreviewAnswerCollectionEntriesArgs = { }; +export type QueryGetUserKbsConnectionArgs = { + after?: InputMaybe; + first?: InputMaybe; + search?: InputMaybe; +}; + + export type QueryGroupActivitiesArgs = { courseId: Scalars['String']['input']; }; @@ -4481,6 +4847,12 @@ export type QueryPublicParticipantProfileArgs = { }; +export type QuerySearchKbKnowledgeGraphArgs = { + kbId: Scalars['ID']['input']; + query: Scalars['String']['input']; +}; + + export type QuerySelfArgs = { liveQuizId?: InputMaybe; }; @@ -5305,6 +5677,14 @@ export type ApproveObjectSharingRequestMutationVariables = Exact<{ export type ApproveObjectSharingRequestMutation = { __typename?: 'Mutation', approveObjectSharingRequest: boolean }; +export type AttachKbToChatbotMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + chatbotId: Scalars['ID']['input']; +}>; + + +export type AttachKbToChatbotMutation = { __typename?: 'Mutation', attachKbToChatbot: { __typename?: 'KBChatbotBinding', chatbotId: string, chatbotName: string, enabledKbId?: string | null, enabledKbName?: string | null } }; + export type BookmarkElementStackMutationVariables = Exact<{ stackId: Scalars['Int']['input']; courseId: Scalars['String']['input']; @@ -5427,6 +5807,18 @@ export type ChangeUserLocaleMutationVariables = Exact<{ export type ChangeUserLocaleMutation = { __typename?: 'Mutation', changeUserLocale?: { __typename?: 'User', id: string, locale: LocaleType } | null }; +export type ConfirmKbFileUploadMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + blobName: Scalars['String']['input']; + title: Scalars['String']['input']; + originalFilename: Scalars['String']['input']; + mimeType: Scalars['String']['input']; + sizeBytes: Scalars['Int']['input']; +}>; + + +export type ConfirmKbFileUploadMutation = { __typename?: 'Mutation', confirmKbFileUpload: { __typename?: 'KBResource', id: string } }; + export type CopyCatalogObjectToAccountMutationVariables = Exact<{ objectId: Scalars['String']['input']; objectType: ObjectType; @@ -5544,6 +5936,23 @@ export type CreateGroupActivityMutationVariables = Exact<{ export type CreateGroupActivityMutation = { __typename?: 'Mutation', createGroupActivity?: { __typename?: 'ActivityInfo', id: string, templateId?: string | null, type: ActivityType, status: PublicationStatus, courseId?: string | null, courseName?: string | null, courseStartDate?: any | null, courseLanguage?: LocaleType | null, numOfStacks: number, numOfElements: number, reviewStatus: ReviewStatus, automaticPublicationAt?: any | null, scheduledStartAt?: any | null, scheduledEndAt?: any | null, groupDeadlineDate?: any | null, numOfParticipantGroups?: number | null, name: string, displayName: string, permissionLevel: PermissionLevel, derivedAccess: boolean, areInstancesOutdated: boolean, isGamificationEnabled?: boolean | null, isAssessmentEnabled?: boolean | null, pinCode?: string | null, numSharedUsers?: number | null, isOwner: boolean, isManager: boolean, isEditor: boolean, isExecutor: boolean, isShared: boolean, isRemovable: boolean, isActivityReviewer: boolean, sharingType: SharingType, updatedAt: any } | null }; +export type CreateKbMutationVariables = Exact<{ + name: Scalars['String']['input']; + description?: InputMaybe; +}>; + + +export type CreateKbMutation = { __typename?: 'Mutation', createKb: { __typename?: 'KB', id: string } }; + +export type CreateKbUrlResourceMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + url: Scalars['String']['input']; + title: Scalars['String']['input']; +}>; + + +export type CreateKbUrlResourceMutation = { __typename?: 'Mutation', createKbUrlResource: { __typename?: 'KBResource', id: string } }; + export type CreateLiveQuizMutationVariables = Exact<{ name: Scalars['String']['input']; displayName: Scalars['String']['input']; @@ -5741,6 +6150,28 @@ export type DeleteGroupActivityMutationVariables = Exact<{ export type DeleteGroupActivityMutation = { __typename?: 'Mutation', deleteGroupActivity?: { __typename?: 'GroupActivity', id: string } | null }; +export type DeleteKbMutationVariables = Exact<{ + id: Scalars['ID']['input']; +}>; + + +export type DeleteKbMutation = { __typename?: 'Mutation', deleteKb: { __typename?: 'KB', id: string } }; + +export type DeleteKbResourceMutationVariables = Exact<{ + id: Scalars['ID']['input']; +}>; + + +export type DeleteKbResourceMutation = { __typename?: 'Mutation', deleteKbResource: { __typename?: 'KBResource', id: string } }; + +export type DeleteKbResourcesMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + ids: Array | Scalars['String']['input']; +}>; + + +export type DeleteKbResourcesMutation = { __typename?: 'Mutation', deleteKbResources: Array<{ __typename?: 'KBResource', id: string }> }; + export type DeleteLiveQuizMutationVariables = Exact<{ id: Scalars['String']['input']; }>; @@ -5804,6 +6235,14 @@ export type DemoteGroupAdminToMemberMutationVariables = Exact<{ export type DemoteGroupAdminToMemberMutation = { __typename?: 'Mutation', demoteGroupAdminToMember: boolean }; +export type DetachKbFromChatbotMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + chatbotId: Scalars['ID']['input']; +}>; + + +export type DetachKbFromChatbotMutation = { __typename?: 'Mutation', detachKbFromChatbot: boolean }; + export type DuplicateAnswerCollectionMutationVariables = Exact<{ id: Scalars['Int']['input']; }>; @@ -6015,6 +6454,13 @@ export type ImportCatalogObjectMutationVariables = Exact<{ export type ImportCatalogObjectMutation = { __typename?: 'Mutation', importCatalogObject: boolean }; +export type IngestKbResourceMutationVariables = Exact<{ + id: Scalars['ID']['input']; +}>; + + +export type IngestKbResourceMutation = { __typename?: 'Mutation', ingestKbResource: { __typename?: 'KBResource', id: string, status: KbResourceStatus } }; + export type MIssueCredentialMutationVariables = Exact<{ courseId: Scalars['String']['input']; }>; @@ -6320,6 +6766,14 @@ export type RateElementMutationVariables = Exact<{ export type RateElementMutation = { __typename?: 'Mutation', rateElement?: { __typename?: 'ElementFeedback', id: number, elementInstanceId: number, upvote: boolean, downvote: boolean, feedback?: string | null } | null }; +export type RebuildKbKnowledgeGraphMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + qualityTier?: InputMaybe; +}>; + + +export type RebuildKbKnowledgeGraphMutation = { __typename?: 'Mutation', rebuildKbKnowledgeGraph: { __typename?: 'KBKnowledgeGraphConfig', kbId: string, isEnabled: boolean, buildId?: string | null, status?: KbGraphBuildStatus | null, statusMessage?: string | null, qualityTier?: KbGraphQualityTier | null, sourceContentDigest?: string | null, activeBuildId?: string | null, publishedBuildId?: string | null, isStale: boolean, startedAt?: any | null, finishedAt?: any | null, createdAt?: any | null, updatedAt?: any | null, costConfigurationReady: boolean, costCurrency?: string | null, quotaCurrency?: string | null, billingLabel?: string | null, standardEstimateMinorUnits?: number | null, highEstimateMinorUnits?: number | null, estimatedCostMinorUnits?: number | null, actualCostMinorUnits?: number | null, actualInputTokens?: number | null, actualOutputTokens?: number | null, actualEmbeddingTokens?: number | null, actualRequestCount?: number | null, maxCostMinorUnits?: number | null, costStatus?: KbGraphCostStatus | null, semesterKey?: string | null, semesterQuotaMinorUnits?: number | null, semesterReservedMinorUnits?: number | null, semesterSettledMinorUnits?: number | null, remainingSemesterQuotaMinorUnits?: number | null, worstCaseRemainingMinorUnits?: number | null } }; + export type RemoveCatalogObjectAssignmentMutationVariables = Exact<{ assignmentId: Scalars['Int']['input']; }>; @@ -6377,6 +6831,16 @@ export type MRequestCatalystAccessMutationVariables = Exact<{ export type MRequestCatalystAccessMutation = { __typename?: 'Mutation', requestCatalystAccess: boolean }; +export type RequestKbFileUploadMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + fileName: Scalars['String']['input']; + contentType: Scalars['String']['input']; + sizeBytes: Scalars['Int']['input']; +}>; + + +export type RequestKbFileUploadMutation = { __typename?: 'Mutation', requestKbFileUpload: { __typename?: 'KBFileUpload', uploadSasURL: string, containerName: string, blobName: string } }; + export type ResetAssessmentLiveQuizMutationVariables = Exact<{ id: Scalars['String']['input']; }>; @@ -6475,6 +6939,14 @@ export type SetBetaFeaturesMutationVariables = Exact<{ export type SetBetaFeaturesMutation = { __typename?: 'Mutation', setBetaFeatures?: boolean | null }; +export type SetKbKnowledgeGraphEnabledMutationVariables = Exact<{ + kbId: Scalars['ID']['input']; + enabled: Scalars['Boolean']['input']; +}>; + + +export type SetKbKnowledgeGraphEnabledMutation = { __typename?: 'Mutation', setKbKnowledgeGraphEnabled: { __typename?: 'KBKnowledgeGraphConfig', kbId: string, isEnabled: boolean, buildId?: string | null, status?: KbGraphBuildStatus | null, statusMessage?: string | null, qualityTier?: KbGraphQualityTier | null, sourceContentDigest?: string | null, activeBuildId?: string | null, publishedBuildId?: string | null, isStale: boolean, startedAt?: any | null, finishedAt?: any | null, createdAt?: any | null, updatedAt?: any | null, costConfigurationReady: boolean, costCurrency?: string | null, quotaCurrency?: string | null, billingLabel?: string | null, standardEstimateMinorUnits?: number | null, highEstimateMinorUnits?: number | null, estimatedCostMinorUnits?: number | null, actualCostMinorUnits?: number | null, actualInputTokens?: number | null, actualOutputTokens?: number | null, actualEmbeddingTokens?: number | null, actualRequestCount?: number | null, maxCostMinorUnits?: number | null, costStatus?: KbGraphCostStatus | null, semesterKey?: string | null, semesterQuotaMinorUnits?: number | null, semesterReservedMinorUnits?: number | null, semesterSettledMinorUnits?: number | null, remainingSemesterQuotaMinorUnits?: number | null, worstCaseRemainingMinorUnits?: number | null } }; + export type SetLiveQuizPinMutationVariables = Exact<{ liveQuizId: Scalars['String']['input']; pin: Scalars['String']['input']; @@ -6913,7 +7385,7 @@ export type GetChatModelRegistryQuery = { __typename?: 'Query', getChatModelRegi export type GetChatbotsInfoQueryVariables = Exact<{ [key: string]: never; }>; -export type GetChatbotsInfoQuery = { __typename?: 'Query', getChatbotsInfo?: Array<{ __typename?: 'Chatbot', id: string, name: string, description?: string | null, avatar?: string | null, modelSelection: boolean, allowedModelIds: Array, creditInitialCredits: number, creditResetPeriod: CreditResetPeriod, creditResetAmount: number, creditMaxCredits: number, createdAt?: any | null, updatedAt?: any | null, allowedReasoningEffortsByModel: Array<{ __typename?: 'ChatbotReasoningConfig', modelId: string, efforts: Array }>, courses: Array<{ __typename?: 'CourseListEntry', id: string, name: string }>, usageSummary?: { __typename?: 'ChatbotUsageSummary', threadCount: number, messageCount: number, participantCount: number, lastActivityAt?: any | null, totalCredits?: number | null, currentCredits?: number | null, totalResets?: number | null, lastResetAt?: any | null } | null, disclaimerSummary?: { __typename?: 'ChatbotDisclaimerSummary', id: string, name: string, title: string, acceptedCount: number, declinedCount: number, pendingCount: number } | null, mcpConfigurations: Array<{ __typename?: 'ChatbotMcpConfigurationSummary', serverId: string, serverName: string, serverDescription?: string | null, serverIsActive: boolean, chatMode: string, isEnabled: boolean, priority: number, allowedToolsCount?: number | null }> }> | null }; +export type GetChatbotsInfoQuery = { __typename?: 'Query', getChatbotsInfo?: Array<{ __typename?: 'Chatbot', id: string, name: string, description?: string | null, avatar?: string | null, modelSelection: boolean, allowedModelIds: Array, creditInitialCredits: number, creditResetPeriod: CreditResetPeriod, creditResetAmount: number, creditMaxCredits: number, createdAt?: any | null, updatedAt?: any | null, allowedReasoningEffortsByModel: Array<{ __typename?: 'ChatbotReasoningConfig', modelId: string, efforts: Array }>, courses: Array<{ __typename?: 'CourseListEntry', id: string, name: string }>, usageSummary?: { __typename?: 'ChatbotUsageSummary', threadCount: number, messageCount: number, participantCount: number, lastActivityAt?: any | null, totalCredits?: number | null, currentCredits?: number | null, totalResets?: number | null, lastResetAt?: any | null } | null, disclaimerSummary?: { __typename?: 'ChatbotDisclaimerSummary', id: string, name: string, title: string, acceptedCount: number, declinedCount: number, pendingCount: number } | null, mcpConfigurations: Array<{ __typename?: 'ChatbotMcpConfigurationSummary', serverId: string, serverName: string, serverDescription?: string | null, serverIsActive: boolean, chatMode: string, isEnabled: boolean, priority: number, allowedToolsCount?: number | null }>, enabledKnowledgeBase?: { __typename?: 'ChatbotKnowledgeBaseSummary', id: string, name: string } | null }> | null }; export type GetCockpitQuizQueryVariables = Exact<{ id: Scalars['String']['input']; @@ -7165,6 +7637,61 @@ export type GetInstanceUpdateActivitiesQueryVariables = Exact<{ export type GetInstanceUpdateActivitiesQuery = { __typename?: 'Query', getInstanceUpdateActivities?: Array<{ __typename?: 'InstanceUpdateActivityInfo', activityId: string, activityName: string, courseName?: string | null, activityType: ActivityType, status: PublicationStatus }> | null }; +export type GetKbQueryVariables = Exact<{ + id: Scalars['ID']['input']; +}>; + + +export type GetKbQuery = { __typename?: 'Query', getKb: { __typename?: 'KB', id: string, name: string, description?: string | null, metrics?: { __typename?: 'KBMetrics', visibleResourceCount: number, visibleSizeBytes: number, unknownSizeResourceCount: number, quotaResourceCount: number, quotaSizeBytes: number, resourceLimit: number, storageLimitBytes: number, pendingCleanupCount: number, pendingCleanupSizeBytes: number, reservedResourceCount: number, reservedSizeBytes: number, linkedConsumerCount: number } | null } }; + +export type GetKbChatbotBindingsQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; +}>; + + +export type GetKbChatbotBindingsQuery = { __typename?: 'Query', getKbChatbotBindings: Array<{ __typename?: 'KBChatbotBinding', chatbotId: string, chatbotName: string, enabledKbId?: string | null, enabledKbName?: string | null }> }; + +export type GetKbKnowledgeGraphConfigQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; +}>; + + +export type GetKbKnowledgeGraphConfigQuery = { __typename?: 'Query', getKbKnowledgeGraphConfig: { __typename?: 'KBKnowledgeGraphConfig', kbId: string, isEnabled: boolean, buildId?: string | null, status?: KbGraphBuildStatus | null, statusMessage?: string | null, qualityTier?: KbGraphQualityTier | null, sourceContentDigest?: string | null, activeBuildId?: string | null, publishedBuildId?: string | null, isStale: boolean, startedAt?: any | null, finishedAt?: any | null, createdAt?: any | null, updatedAt?: any | null, costConfigurationReady: boolean, costCurrency?: string | null, quotaCurrency?: string | null, billingLabel?: string | null, standardEstimateMinorUnits?: number | null, highEstimateMinorUnits?: number | null, estimatedCostMinorUnits?: number | null, actualCostMinorUnits?: number | null, actualInputTokens?: number | null, actualOutputTokens?: number | null, actualEmbeddingTokens?: number | null, actualRequestCount?: number | null, maxCostMinorUnits?: number | null, costStatus?: KbGraphCostStatus | null, semesterKey?: string | null, semesterQuotaMinorUnits?: number | null, semesterReservedMinorUnits?: number | null, semesterSettledMinorUnits?: number | null, remainingSemesterQuotaMinorUnits?: number | null, worstCaseRemainingMinorUnits?: number | null } }; + +export type GetKbKnowledgeGraphNeighborsQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; + nodeId: Scalars['ID']['input']; +}>; + + +export type GetKbKnowledgeGraphNeighborsQuery = { __typename?: 'Query', getKbKnowledgeGraphNeighbors: { __typename?: 'KnowledgeGraphResponse', kbId: string, buildId: string, isStale: boolean, truncated: boolean, nodes: Array<{ __typename?: 'KnowledgeGraphNode', id: string, labels: Array, kind: string, displayLabel: string, summary?: string | null, content?: string | null, degree: number, sourceReferences: Array<{ __typename?: 'KnowledgeGraphSourceReference', resourceId: string, title: string, reference?: string | null }> }>, edges: Array<{ __typename?: 'KnowledgeGraphEdge', id: string, source: string, target: string, type: string, label: string, properties: any }> } }; + +export type GetKbKnowledgeGraphOverviewQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; +}>; + + +export type GetKbKnowledgeGraphOverviewQuery = { __typename?: 'Query', getKbKnowledgeGraphOverview: { __typename?: 'KnowledgeGraphResponse', kbId: string, buildId: string, isStale: boolean, truncated: boolean, nodes: Array<{ __typename?: 'KnowledgeGraphNode', id: string, labels: Array, kind: string, displayLabel: string, summary?: string | null, content?: string | null, degree: number, sourceReferences: Array<{ __typename?: 'KnowledgeGraphSourceReference', resourceId: string, title: string, reference?: string | null }> }>, edges: Array<{ __typename?: 'KnowledgeGraphEdge', id: string, source: string, target: string, type: string, label: string, properties: any }> } }; + +export type GetKbResourceIngestionRunsQueryVariables = Exact<{ + resourceId: Scalars['ID']['input']; +}>; + + +export type GetKbResourceIngestionRunsQuery = { __typename?: 'Query', getKbResourceIngestionRuns: Array<{ __typename?: 'KBIngestionRun', id: string, status: KbIngestionStatus, resourceVersion: number, errorCode?: string | null, createdAt: any }> }; + +export type GetKbResourcesQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; + first?: InputMaybe; + after?: InputMaybe; + search?: InputMaybe; + type?: InputMaybe; + status?: InputMaybe; +}>; + + +export type GetKbResourcesQuery = { __typename?: 'Query', getKbResources: { __typename?: 'KBResourceConnection', totalCount: number, items: Array<{ __typename?: 'KBResource', id: string, type: KbResourceType, title: string, sourceUrl?: string | null, originalFilename?: string | null, mimeType?: string | null, sizeBytes?: number | null, status: KbResourceStatus, ingestedAt?: any | null, resourceVersion: number, activeResourceVersion?: number | null, createdAt: any, updatedAt: any, latestIngestionRun?: { __typename?: 'KBIngestionRun', id: string, status: KbIngestionStatus, errorCode?: string | null } | null }>, pageInfo: { __typename?: 'KBPageInfo', hasNextPage: boolean, endCursor?: string | null } } }; + export type GetLecturerViewLiveQuizQueryVariables = Exact<{ id: Scalars['String']['input']; }>; @@ -7510,6 +8037,15 @@ export type GetUserGroupsUserQueryVariables = Exact<{ [key: string]: never; }>; export type GetUserGroupsUserQuery = { __typename?: 'Query', getUserGroupsUser?: Array<{ __typename?: 'UserGroup', id: number, name: string, numOfMembers?: number | null, isMember?: boolean | null, isAdmin?: boolean | null, isOwner?: boolean | null, members?: Array<{ __typename?: 'UserInfo', id?: string | null, shortname: string, email: string, isSelf?: boolean | null }> | null, admins?: Array<{ __typename?: 'UserInfo', id?: string | null, shortname: string, email: string, isSelf?: boolean | null }> | null, owner?: { __typename?: 'UserInfo', id?: string | null, shortname: string, email: string, isSelf?: boolean | null } | null }> | null }; +export type GetUserKbsQueryVariables = Exact<{ + first?: InputMaybe; + after?: InputMaybe; + search?: InputMaybe; +}>; + + +export type GetUserKbsQuery = { __typename?: 'Query', getUserKbsConnection: { __typename?: 'KBConnection', totalCount: number, items: Array<{ __typename?: 'KB', id: string, name: string, description?: string | null, metrics?: { __typename?: 'KBMetrics', visibleResourceCount: number, visibleSizeBytes: number, linkedConsumerCount: number } | null }>, pageInfo: { __typename?: 'KBPageInfo', hasNextPage: boolean, endCursor?: string | null } } }; + export type GetUserLoginsQueryVariables = Exact<{ [key: string]: never; }>; @@ -7563,6 +8099,14 @@ export type ParticipationsQueryVariables = Exact<{ export type ParticipationsQuery = { __typename?: 'Query', participations?: Array<{ __typename?: 'Participation', id: number, completedMicroLearnings: Array, subscriptions?: Array<{ __typename?: 'PushSubscription', id: number, endpoint: string }> | null, course?: { __typename?: 'Course', id: string, displayName: string, startDate: any, endDate: any, description?: string | null, isGamificationEnabled: boolean, microLearnings?: Array<{ __typename?: 'MicroLearning', id: string, displayName: string, scheduledStartAt: any, scheduledEndAt: any }> | null, liveQuizzes?: Array<{ __typename?: 'LiveQuiz', id: string, displayName: string }> | null } | null }> | null }; +export type SearchKbKnowledgeGraphQueryVariables = Exact<{ + kbId: Scalars['ID']['input']; + query: Scalars['String']['input']; +}>; + + +export type SearchKbKnowledgeGraphQuery = { __typename?: 'Query', searchKbKnowledgeGraph: { __typename?: 'KnowledgeGraphResponse', kbId: string, buildId: string, isStale: boolean, truncated: boolean, nodes: Array<{ __typename?: 'KnowledgeGraphNode', id: string, labels: Array, kind: string, displayLabel: string, summary?: string | null, content?: string | null, degree: number, sourceReferences: Array<{ __typename?: 'KnowledgeGraphSourceReference', resourceId: string, title: string, reference?: string | null }> }>, edges: Array<{ __typename?: 'KnowledgeGraphEdge', id: string, source: string, target: string, type: string, label: string, properties: any }> } }; + export type SelfQueryVariables = Exact<{ liveQuizId?: InputMaybe; }>; @@ -7703,6 +8247,7 @@ export const AddUserToUserGroupDocument = {"kind":"Document","definitions":[{"ki export const ApplyActivityBatchOperationsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ApplyActivityBatchOperations"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"basePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"correctnessPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"bonusPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"timeToZeroBonus"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"applyActivityBatchOperations"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"activityIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityIds"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"basePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"basePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"correctnessPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"correctnessPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"bonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"bonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"timeToZeroBonus"},"value":{"kind":"Variable","name":{"kind":"Name","value":"timeToZeroBonus"}}}]}]}}]} as unknown as DocumentNode; export const ApplyElementBatchOperationsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ApplyElementBatchOperations"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"elementIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"archive"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"unarchive"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"status"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementStatus"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"basePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"updateInstances"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"updateTemplateInstances"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"applyElementBatchOperations"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"elementIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"elementIds"}}},{"kind":"Argument","name":{"kind":"Name","value":"archive"},"value":{"kind":"Variable","name":{"kind":"Name","value":"archive"}}},{"kind":"Argument","name":{"kind":"Name","value":"unarchive"},"value":{"kind":"Variable","name":{"kind":"Name","value":"unarchive"}}},{"kind":"Argument","name":{"kind":"Name","value":"status"},"value":{"kind":"Variable","name":{"kind":"Name","value":"status"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"basePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"basePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"updateInstances"},"value":{"kind":"Variable","name":{"kind":"Name","value":"updateInstances"}}},{"kind":"Argument","name":{"kind":"Name","value":"updateTemplateInstances"},"value":{"kind":"Variable","name":{"kind":"Name","value":"updateTemplateInstances"}}}]}]}}]} as unknown as DocumentNode; export const ApproveObjectSharingRequestDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ApproveObjectSharingRequest"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"requestId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"userId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionLevel"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"propagation"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"approveObjectSharingRequest"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"requestId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"requestId"}}},{"kind":"Argument","name":{"kind":"Name","value":"userId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"userId"}}},{"kind":"Argument","name":{"kind":"Name","value":"permissionLevel"},"value":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}}},{"kind":"Argument","name":{"kind":"Name","value":"propagation"},"value":{"kind":"Variable","name":{"kind":"Name","value":"propagation"}}}]}]}}]} as unknown as DocumentNode; +export const AttachKbToChatbotDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"AttachKbToChatbot"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"chatbotId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"attachKbToChatbot"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"chatbotId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"chatbotId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"chatbotId"}},{"kind":"Field","name":{"kind":"Name","value":"chatbotName"}},{"kind":"Field","name":{"kind":"Name","value":"enabledKbId"}},{"kind":"Field","name":{"kind":"Name","value":"enabledKbName"}}]}}]}}]} as unknown as DocumentNode; export const BookmarkElementStackDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"BookmarkElementStack"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"stackId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"bookmarked"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"bookmarkElementStack"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"stackId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"stackId"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"bookmarked"},"value":{"kind":"Variable","name":{"kind":"Name","value":"bookmarked"}}}]}]}}]} as unknown as DocumentNode; export const CancelLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CancelLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"cancelLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const CancelObjectSharingRequestDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CancelObjectSharingRequest"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"cancelObjectSharingRequest"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}}]}]}}]} as unknown as DocumentNode; @@ -7718,6 +8263,7 @@ export const ChangeParticipantLocaleDocument = {"kind":"Document","definitions": export const ChangeShortnameDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ChangeShortname"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"shortname"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"changeShortname"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"shortname"},"value":{"kind":"Variable","name":{"kind":"Name","value":"shortname"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}}]}}]}}]} as unknown as DocumentNode; export const ChangeUserGroupNameDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ChangeUserGroupName"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"changeUserGroupName"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}}]}]}}]} as unknown as DocumentNode; export const ChangeUserLocaleDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ChangeUserLocale"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"locale"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"LocaleType"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"changeUserLocale"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"locale"},"value":{"kind":"Variable","name":{"kind":"Name","value":"locale"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"locale"}}]}}]}}]} as unknown as DocumentNode; +export const ConfirmKbFileUploadDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ConfirmKbFileUpload"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"blobName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"title"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"originalFilename"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"mimeType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"sizeBytes"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"confirmKbFileUpload"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"blobName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"blobName"}}},{"kind":"Argument","name":{"kind":"Name","value":"title"},"value":{"kind":"Variable","name":{"kind":"Name","value":"title"}}},{"kind":"Argument","name":{"kind":"Name","value":"originalFilename"},"value":{"kind":"Variable","name":{"kind":"Name","value":"originalFilename"}}},{"kind":"Argument","name":{"kind":"Name","value":"mimeType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"mimeType"}}},{"kind":"Argument","name":{"kind":"Name","value":"sizeBytes"},"value":{"kind":"Variable","name":{"kind":"Name","value":"sizeBytes"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const CopyCatalogObjectToAccountDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CopyCatalogObjectToAccount"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"copyCatalogObjectToAccount"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}},{"kind":"Argument","name":{"kind":"Name","value":"catalogCollectionId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}}}]}]}}]} as unknown as DocumentNode; export const CorrectAssessmentPointsInstanceDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CorrectAssessmentPointsInstance"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"instanceId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardBasePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardCorrectnessPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardBonusPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductBasePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductCorrectnessPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductBonusPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"reason"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"studentReason"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"scope"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"PointCorrectionType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"participantId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"participantIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"correctAssessmentPointsInstance"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"instanceId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"instanceId"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardBasePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardBasePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardCorrectnessPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardCorrectnessPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardBonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardBonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductBasePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductBasePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductCorrectnessPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductCorrectnessPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductBonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductBonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"reason"},"value":{"kind":"Variable","name":{"kind":"Name","value":"reason"}}},{"kind":"Argument","name":{"kind":"Name","value":"studentReason"},"value":{"kind":"Variable","name":{"kind":"Name","value":"studentReason"}}},{"kind":"Argument","name":{"kind":"Name","value":"scope"},"value":{"kind":"Variable","name":{"kind":"Name","value":"scope"}}},{"kind":"Argument","name":{"kind":"Name","value":"participantId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"participantId"}}},{"kind":"Argument","name":{"kind":"Name","value":"participantIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"participantIds"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"PointCorrectionData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"PointCorrectionData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"PointCorrection"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"correctnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"bonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"reason"}},{"kind":"Field","name":{"kind":"Name","value":"studentReason"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"correctedBy"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}}]}},{"kind":"Field","name":{"kind":"Name","value":"participant"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"email"}}]}},{"kind":"Field","name":{"kind":"Name","value":"participants"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"email"}}]}},{"kind":"Field","name":{"kind":"Name","value":"liveQuiz"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"instance"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const CorrectAssessmentPointsLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CorrectAssessmentPointsLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardBasePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardCorrectnessPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"awardBonusPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductBasePoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductCorrectnessPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"deductBonusPoints"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"reason"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"studentReason"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"scope"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"PointCorrectionType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"participantId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"participantIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"correctAssessmentPointsLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardBasePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardBasePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardCorrectnessPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardCorrectnessPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"awardBonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"awardBonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductBasePoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductBasePoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductCorrectnessPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductCorrectnessPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"deductBonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"deductBonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"reason"},"value":{"kind":"Variable","name":{"kind":"Name","value":"reason"}}},{"kind":"Argument","name":{"kind":"Name","value":"studentReason"},"value":{"kind":"Variable","name":{"kind":"Name","value":"studentReason"}}},{"kind":"Argument","name":{"kind":"Name","value":"scope"},"value":{"kind":"Variable","name":{"kind":"Name","value":"scope"}}},{"kind":"Argument","name":{"kind":"Name","value":"participantId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"participantId"}}},{"kind":"Argument","name":{"kind":"Name","value":"participantIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"participantIds"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"PointCorrectionData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"PointCorrectionData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"PointCorrection"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"correctnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"bonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"reason"}},{"kind":"Field","name":{"kind":"Name","value":"studentReason"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"correctedBy"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}}]}},{"kind":"Field","name":{"kind":"Name","value":"participant"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"email"}}]}},{"kind":"Field","name":{"kind":"Name","value":"participants"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"email"}}]}},{"kind":"Field","name":{"kind":"Name","value":"liveQuiz"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"instance"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}}]}}]}}]} as unknown as DocumentNode; @@ -7727,6 +8273,8 @@ export const CreateCatalogCollectionDocument = {"kind":"Document","definitions": export const CreateCourseDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateCourse"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"color"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isGroupCreationEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupDeadlineDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"maxGroupSize"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"preferredGroupSize"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"language"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"LocaleType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"notificationEmail"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"sourceCourseId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"duplicateLiveQuizzes"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"duplicatePracticeQuizzes"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"duplicateMicrolearnings"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"duplicateGroupActivities"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createCourse"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"displayName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"color"},"value":{"kind":"Variable","name":{"kind":"Name","value":"color"}}},{"kind":"Argument","name":{"kind":"Name","value":"startDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"endDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"isGroupCreationEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isGroupCreationEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"groupDeadlineDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupDeadlineDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"maxGroupSize"},"value":{"kind":"Variable","name":{"kind":"Name","value":"maxGroupSize"}}},{"kind":"Argument","name":{"kind":"Name","value":"preferredGroupSize"},"value":{"kind":"Variable","name":{"kind":"Name","value":"preferredGroupSize"}}},{"kind":"Argument","name":{"kind":"Name","value":"language"},"value":{"kind":"Variable","name":{"kind":"Name","value":"language"}}},{"kind":"Argument","name":{"kind":"Name","value":"notificationEmail"},"value":{"kind":"Variable","name":{"kind":"Name","value":"notificationEmail"}}},{"kind":"Argument","name":{"kind":"Name","value":"isGamificationEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"sourceCourseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"sourceCourseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"duplicateLiveQuizzes"},"value":{"kind":"Variable","name":{"kind":"Name","value":"duplicateLiveQuizzes"}}},{"kind":"Argument","name":{"kind":"Name","value":"duplicatePracticeQuizzes"},"value":{"kind":"Variable","name":{"kind":"Name","value":"duplicatePracticeQuizzes"}}},{"kind":"Argument","name":{"kind":"Name","value":"duplicateMicrolearnings"},"value":{"kind":"Variable","name":{"kind":"Name","value":"duplicateMicrolearnings"}}},{"kind":"Argument","name":{"kind":"Name","value":"duplicateGroupActivities"},"value":{"kind":"Variable","name":{"kind":"Name","value":"duplicateGroupActivities"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"color"}},{"kind":"Field","name":{"kind":"Name","value":"startDate"}},{"kind":"Field","name":{"kind":"Name","value":"endDate"}},{"kind":"Field","name":{"kind":"Name","value":"groupDeadlineDate"}},{"kind":"Field","name":{"kind":"Name","value":"maxGroupSize"}},{"kind":"Field","name":{"kind":"Name","value":"preferredGroupSize"}},{"kind":"Field","name":{"kind":"Name","value":"language"}},{"kind":"Field","name":{"kind":"Name","value":"notificationEmail"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isGroupCreationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"randomAssignmentFinalized"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}}]}}]}}]} as unknown as DocumentNode; export const CreateFeedbackDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateFeedback"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"quizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"content"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createFeedback"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"quizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"quizId"}}},{"kind":"Argument","name":{"kind":"Name","value":"content"},"value":{"kind":"Variable","name":{"kind":"Name","value":"content"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isPublished"}},{"kind":"Field","name":{"kind":"Name","value":"isPinned"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"votes"}}]}}]}}]} as unknown as DocumentNode; export const CreateGroupActivityDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateGroupActivity"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"clues"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"GroupActivityClueInput"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"stack"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementStackInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createGroupActivity"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"displayName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"startDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"endDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"clues"},"value":{"kind":"Variable","name":{"kind":"Name","value":"clues"}}},{"kind":"Argument","name":{"kind":"Name","value":"stack"},"value":{"kind":"Variable","name":{"kind":"Name","value":"stack"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"ActivityInfoData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ActivityInfoData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityInfo"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"templateId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"courseId"}},{"kind":"Field","name":{"kind":"Name","value":"courseName"}},{"kind":"Field","name":{"kind":"Name","value":"courseStartDate"}},{"kind":"Field","name":{"kind":"Name","value":"courseLanguage"}},{"kind":"Field","name":{"kind":"Name","value":"numOfStacks"}},{"kind":"Field","name":{"kind":"Name","value":"numOfElements"}},{"kind":"Field","name":{"kind":"Name","value":"reviewStatus"}},{"kind":"Field","name":{"kind":"Name","value":"automaticPublicationAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}},{"kind":"Field","name":{"kind":"Name","value":"groupDeadlineDate"}},{"kind":"Field","name":{"kind":"Name","value":"numOfParticipantGroups"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"areInstancesOutdated"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isExecutor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"isActivityReviewer"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode; +export const CreateKbDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateKb"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createKb"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; +export const CreateKbUrlResourceDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateKbUrlResource"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"url"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"title"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createKbUrlResource"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"url"},"value":{"kind":"Variable","name":{"kind":"Name","value":"url"}}},{"kind":"Argument","name":{"kind":"Name","value":"title"},"value":{"kind":"Variable","name":{"kind":"Name","value":"title"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const CreateLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"blocks"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementBlockInput"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"defaultPoints"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"defaultCorrectPoints"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"maxBonusPoints"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"timeToZeroBonus"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isPinProtected"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isConfusionFeedbackEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isLiveQAEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isModerationEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"displayName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"blocks"},"value":{"kind":"Variable","name":{"kind":"Name","value":"blocks"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"defaultPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"defaultPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"defaultCorrectPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"defaultCorrectPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"maxBonusPoints"},"value":{"kind":"Variable","name":{"kind":"Name","value":"maxBonusPoints"}}},{"kind":"Argument","name":{"kind":"Name","value":"timeToZeroBonus"},"value":{"kind":"Variable","name":{"kind":"Name","value":"timeToZeroBonus"}}},{"kind":"Argument","name":{"kind":"Name","value":"isGamificationEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"isPinProtected"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isPinProtected"}}},{"kind":"Argument","name":{"kind":"Name","value":"isConfusionFeedbackEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isConfusionFeedbackEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"isLiveQAEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isLiveQAEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"isModerationEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isModerationEnabled"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"ActivityInfoData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ActivityInfoData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityInfo"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"templateId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"courseId"}},{"kind":"Field","name":{"kind":"Name","value":"courseName"}},{"kind":"Field","name":{"kind":"Name","value":"courseStartDate"}},{"kind":"Field","name":{"kind":"Name","value":"courseLanguage"}},{"kind":"Field","name":{"kind":"Name","value":"numOfStacks"}},{"kind":"Field","name":{"kind":"Name","value":"numOfElements"}},{"kind":"Field","name":{"kind":"Name","value":"reviewStatus"}},{"kind":"Field","name":{"kind":"Name","value":"automaticPublicationAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}},{"kind":"Field","name":{"kind":"Name","value":"groupDeadlineDate"}},{"kind":"Field","name":{"kind":"Name","value":"numOfParticipantGroups"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"areInstancesOutdated"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isExecutor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"isActivityReviewer"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode; export const CreateLiveQuizFromTemplateDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateLiveQuizFromTemplate"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"templateId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"blocks"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"TemplateBlockInput"}}}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createLiveQuizFromTemplate"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"templateId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"templateId"}}},{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"displayName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"isGamificationEnabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isGamificationEnabled"}}},{"kind":"Argument","name":{"kind":"Name","value":"blocks"},"value":{"kind":"Variable","name":{"kind":"Name","value":"blocks"}}}]}]}}]} as unknown as DocumentNode; export const CreateMicroLearningDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateMicroLearning"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"stacks"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementStackInput"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createMicroLearning"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"displayName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"displayName"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"stacks"},"value":{"kind":"Variable","name":{"kind":"Name","value":"stacks"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"startDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"startDate"}}},{"kind":"Argument","name":{"kind":"Name","value":"endDate"},"value":{"kind":"Variable","name":{"kind":"Name","value":"endDate"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"ActivityInfoData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ActivityInfoData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityInfo"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"templateId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"courseId"}},{"kind":"Field","name":{"kind":"Name","value":"courseName"}},{"kind":"Field","name":{"kind":"Name","value":"courseStartDate"}},{"kind":"Field","name":{"kind":"Name","value":"courseLanguage"}},{"kind":"Field","name":{"kind":"Name","value":"numOfStacks"}},{"kind":"Field","name":{"kind":"Name","value":"numOfElements"}},{"kind":"Field","name":{"kind":"Name","value":"reviewStatus"}},{"kind":"Field","name":{"kind":"Name","value":"automaticPublicationAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}},{"kind":"Field","name":{"kind":"Name","value":"groupDeadlineDate"}},{"kind":"Field","name":{"kind":"Name","value":"numOfParticipantGroups"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"areInstancesOutdated"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isExecutor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"isActivityReviewer"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode; @@ -7748,6 +8296,9 @@ export const DeleteElementDocument = {"kind":"Document","definitions":[{"kind":" export const DeleteFeedbackDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteFeedback"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteFeedback"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const DeleteFeedbackResponseDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteFeedbackResponse"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteFeedbackResponse"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isPublished"}},{"kind":"Field","name":{"kind":"Name","value":"isPinned"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"votes"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"resolvedAt"}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"positiveReactions"}},{"kind":"Field","name":{"kind":"Name","value":"negativeReactions"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}}]}}]} as unknown as DocumentNode; export const DeleteGroupActivityDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteGroupActivity"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteGroupActivity"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; +export const DeleteKbDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteKb"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteKb"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; +export const DeleteKbResourceDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteKbResource"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteKbResource"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; +export const DeleteKbResourcesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteKbResources"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"ids"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteKbResources"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"ids"},"value":{"kind":"Variable","name":{"kind":"Name","value":"ids"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const DeleteLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const DeleteMicroLearningDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteMicroLearning"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteMicroLearning"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const DeleteParticipantAccountDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteParticipantAccount"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteParticipantAccount"}}]}}]} as unknown as DocumentNode; @@ -7757,6 +8308,7 @@ export const DeleteTagDocument = {"kind":"Document","definitions":[{"kind":"Oper export const DeleteUserGroupDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteUserGroup"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteUserGroup"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"groupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}}}]}]}}]} as unknown as DocumentNode; export const DeleteUserLoginDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteUserLogin"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteUserLogin"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; export const DemoteGroupAdminToMemberDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DemoteGroupAdminToMember"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"adminId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"demoteGroupAdminToMember"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"groupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}}},{"kind":"Argument","name":{"kind":"Name","value":"adminId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"adminId"}}}]}]}}]} as unknown as DocumentNode; +export const DetachKbFromChatbotDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DetachKbFromChatbot"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"chatbotId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"detachKbFromChatbot"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"chatbotId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"chatbotId"}}}]}]}}]} as unknown as DocumentNode; export const DuplicateAnswerCollectionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DuplicateAnswerCollection"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"duplicateAnswerCollection"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"AnswerCollectionData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"AnswerCollectionData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"AnswerCollection"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"ownerShortname"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"numOfEntries"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isDeletable"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"entries"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}}]} as unknown as DocumentNode; export const EditActivityTemplateDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"EditActivityTemplate"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"templateId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"name"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"description"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"instructions"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"editActivityTemplate"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"activityId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}}},{"kind":"Argument","name":{"kind":"Name","value":"activityType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityType"}}},{"kind":"Argument","name":{"kind":"Name","value":"templateId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"templateId"}}},{"kind":"Argument","name":{"kind":"Name","value":"name"},"value":{"kind":"Variable","name":{"kind":"Name","value":"name"}}},{"kind":"Argument","name":{"kind":"Name","value":"description"},"value":{"kind":"Variable","name":{"kind":"Name","value":"description"}}},{"kind":"Argument","name":{"kind":"Name","value":"instructions"},"value":{"kind":"Variable","name":{"kind":"Name","value":"instructions"}}}]}]}}]} as unknown as DocumentNode; export const EditAnswerCollectionEntryDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"EditAnswerCollectionEntry"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"value"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"collectionId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"editAnswerCollectionEntry"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"value"},"value":{"kind":"Variable","name":{"kind":"Name","value":"value"}}},{"kind":"Argument","name":{"kind":"Name","value":"collectionId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"collectionId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}}]} as unknown as DocumentNode; @@ -7779,6 +8331,7 @@ export const GetFileUploadSasDocument = {"kind":"Document","definitions":[{"kind export const GradeGroupActivitySubmissionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"GradeGroupActivitySubmission"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupActivityId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"gradingDecisions"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"GroupActivityGradingInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"gradeGroupActivitySubmission"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"groupActivityId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupActivityId"}}},{"kind":"Argument","name":{"kind":"Name","value":"gradingDecisions"},"value":{"kind":"Variable","name":{"kind":"Name","value":"gradingDecisions"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"decisionsSubmittedAt"}},{"kind":"Field","name":{"kind":"Name","value":"decisions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"instanceId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"freeTextResponse"}},{"kind":"Field","name":{"kind":"Name","value":"choicesResponse"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"ix"}},{"kind":"Field","name":{"kind":"Name","value":"selected"}}]}},{"kind":"Field","name":{"kind":"Name","value":"numericalResponse"}},{"kind":"Field","name":{"kind":"Name","value":"contentResponse"}},{"kind":"Field","name":{"kind":"Name","value":"selectionResponse"}},{"kind":"Field","name":{"kind":"Name","value":"caseStudyResponse"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"caseId"}},{"kind":"Field","name":{"kind":"Name","value":"itemResponses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"itemId"}},{"kind":"Field","name":{"kind":"Name","value":"criterionResponses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"criterionId"}},{"kind":"Field","name":{"kind":"Name","value":"response"}}]}}]}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"resultsComputedAt"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"passed"}},{"kind":"Field","name":{"kind":"Name","value":"points"}},{"kind":"Field","name":{"kind":"Name","value":"comment"}},{"kind":"Field","name":{"kind":"Name","value":"grading"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"instanceId"}},{"kind":"Field","name":{"kind":"Name","value":"score"}},{"kind":"Field","name":{"kind":"Name","value":"maxPoints"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const GrantPrivatePreviewAccessDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"GrantPrivatePreviewAccess"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"email"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"grantPrivatePreviewAccess"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"email"},"value":{"kind":"Variable","name":{"kind":"Name","value":"email"}}}]}]}}]} as unknown as DocumentNode; export const ImportCatalogObjectDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ImportCatalogObject"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"importCatalogObject"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}},{"kind":"Argument","name":{"kind":"Name","value":"catalogCollectionId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}}}]}]}}]} as unknown as DocumentNode; +export const IngestKbResourceDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"IngestKbResource"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"ingestKbResource"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"status"}}]}}]}}]} as unknown as DocumentNode; export const MIssueCredentialDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"MIssueCredential"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"issueAssessmentReport"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"token"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"issuedAt"}},{"kind":"Field","name":{"kind":"Name","value":"snapshot"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"subject"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"givenName"}},{"kind":"Field","name":{"kind":"Name","value":"surname"}},{"kind":"Field","name":{"kind":"Name","value":"matriculationNumber"}},{"kind":"Field","name":{"kind":"Name","value":"source"}}]}},{"kind":"Field","name":{"kind":"Name","value":"course"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}}]}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableBasePoints"}},{"kind":"Field","name":{"kind":"Name","value":"correctnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableCorrectnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"bonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableBonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"totalPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableTotalPoints"}}]}},{"kind":"Field","name":{"kind":"Name","value":"comparison"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"cohortSize"}},{"kind":"Field","name":{"kind":"Name","value":"percentile"}},{"kind":"Field","name":{"kind":"Name","value":"histogram"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"binStart"}},{"kind":"Field","name":{"kind":"Name","value":"binEnd"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const JoinCourseLeaderboardDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"JoinCourseLeaderboard"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"joinCourseLeaderboard"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"participation"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isActive"}}]}}]}}]}}]} as unknown as DocumentNode; export const JoinCourseWithPinDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"JoinCourseWithPin"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"pin"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"joinCourseWithPin"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"pin"},"value":{"kind":"Variable","name":{"kind":"Name","value":"pin"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}}]}}]} as unknown as DocumentNode; @@ -7813,6 +8366,7 @@ export const PublishGroupActivityDocument = {"kind":"Document","definitions":[{" export const PublishMicroLearningDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"PublishMicroLearning"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"publishMicroLearning"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"status"}}]}}]}}]} as unknown as DocumentNode; export const PublishPracticeQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"PublishPracticeQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"availableFrom"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Date"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"publishPracticeQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"availableFrom"},"value":{"kind":"Variable","name":{"kind":"Name","value":"availableFrom"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"availableFrom"}}]}}]}}]} as unknown as DocumentNode; export const RateElementDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RateElement"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"elementInstanceId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"elementId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"rating"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"rateElement"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"elementInstanceId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"elementInstanceId"}}},{"kind":"Argument","name":{"kind":"Name","value":"elementId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"elementId"}}},{"kind":"Argument","name":{"kind":"Name","value":"rating"},"value":{"kind":"Variable","name":{"kind":"Name","value":"rating"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementInstanceId"}},{"kind":"Field","name":{"kind":"Name","value":"upvote"}},{"kind":"Field","name":{"kind":"Name","value":"downvote"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}}]}}]}}]} as unknown as DocumentNode; +export const RebuildKbKnowledgeGraphDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RebuildKbKnowledgeGraph"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"qualityTier"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"KBGraphQualityTier"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"rebuildKbKnowledgeGraph"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"qualityTier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"qualityTier"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"isEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"statusMessage"}},{"kind":"Field","name":{"kind":"Name","value":"qualityTier"}},{"kind":"Field","name":{"kind":"Name","value":"sourceContentDigest"}},{"kind":"Field","name":{"kind":"Name","value":"activeBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"publishedBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"startedAt"}},{"kind":"Field","name":{"kind":"Name","value":"finishedAt"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"costConfigurationReady"}},{"kind":"Field","name":{"kind":"Name","value":"costCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"quotaCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"billingLabel"}},{"kind":"Field","name":{"kind":"Name","value":"standardEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"highEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"estimatedCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualInputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualOutputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualEmbeddingTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualRequestCount"}},{"kind":"Field","name":{"kind":"Name","value":"maxCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"costStatus"}},{"kind":"Field","name":{"kind":"Name","value":"semesterKey"}},{"kind":"Field","name":{"kind":"Name","value":"semesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterReservedMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterSettledMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"remainingSemesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"worstCaseRemainingMinorUnits"}}]}}]}}]} as unknown as DocumentNode; export const RemoveCatalogObjectAssignmentDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RemoveCatalogObjectAssignment"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"assignmentId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"removeCatalogObjectAssignment"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"assignmentId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"assignmentId"}}}]}]}}]} as unknown as DocumentNode; export const RemoveObjectDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RemoveObject"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"removeObject"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}}]}]}}]} as unknown as DocumentNode; export const RemoveUserFromGroupDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RemoveUserFromGroup"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"userId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"removeUserFromGroup"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"groupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}}},{"kind":"Argument","name":{"kind":"Name","value":"userId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"userId"}}}]}]}}]} as unknown as DocumentNode; @@ -7820,6 +8374,7 @@ export const RenameParticipantGroupDocument = {"kind":"Document","definitions":[ export const RequestCatalogCollectionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RequestCatalogCollection"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"requestedPermissionLevel"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionLevel"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"requestCatalogCollection"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"catalogCollectionId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}}},{"kind":"Argument","name":{"kind":"Name","value":"requestedPermissionLevel"},"value":{"kind":"Variable","name":{"kind":"Name","value":"requestedPermissionLevel"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"access"}},{"kind":"Field","name":{"kind":"Name","value":"ownerShortname"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isRequested"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}}]}}]}}]} as unknown as DocumentNode; export const RequestCatalogObjectDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RequestCatalogObject"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"requestedPermissionLevel"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionLevel"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"requestCatalogObject"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}},{"kind":"Argument","name":{"kind":"Name","value":"catalogCollectionId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"catalogCollectionId"}}},{"kind":"Argument","name":{"kind":"Name","value":"requestedPermissionLevel"},"value":{"kind":"Variable","name":{"kind":"Name","value":"requestedPermissionLevel"}}}]}]}}]} as unknown as DocumentNode; export const MRequestCatalystAccessDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"MRequestCatalystAccess"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"institution"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"useCase"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"requestCatalystAccess"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"institution"},"value":{"kind":"Variable","name":{"kind":"Name","value":"institution"}}},{"kind":"Argument","name":{"kind":"Name","value":"useCase"},"value":{"kind":"Variable","name":{"kind":"Name","value":"useCase"}}}]}]}}]} as unknown as DocumentNode; +export const RequestKbFileUploadDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RequestKbFileUpload"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"fileName"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"contentType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"sizeBytes"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"requestKbFileUpload"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"fileName"},"value":{"kind":"Variable","name":{"kind":"Name","value":"fileName"}}},{"kind":"Argument","name":{"kind":"Name","value":"contentType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"contentType"}}},{"kind":"Argument","name":{"kind":"Name","value":"sizeBytes"},"value":{"kind":"Variable","name":{"kind":"Name","value":"sizeBytes"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"uploadSasURL"}},{"kind":"Field","name":{"kind":"Name","value":"containerName"}},{"kind":"Field","name":{"kind":"Name","value":"blobName"}}]}}]}}]} as unknown as DocumentNode; export const ResetAssessmentLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ResetAssessmentLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resetAssessmentLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"ActivityInfoData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ActivityInfoData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityInfo"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"templateId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"courseId"}},{"kind":"Field","name":{"kind":"Name","value":"courseName"}},{"kind":"Field","name":{"kind":"Name","value":"courseStartDate"}},{"kind":"Field","name":{"kind":"Name","value":"courseLanguage"}},{"kind":"Field","name":{"kind":"Name","value":"numOfStacks"}},{"kind":"Field","name":{"kind":"Name","value":"numOfElements"}},{"kind":"Field","name":{"kind":"Name","value":"reviewStatus"}},{"kind":"Field","name":{"kind":"Name","value":"automaticPublicationAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}},{"kind":"Field","name":{"kind":"Name","value":"groupDeadlineDate"}},{"kind":"Field","name":{"kind":"Name","value":"numOfParticipantGroups"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"areInstancesOutdated"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isExecutor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"isActivityReviewer"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode; export const ResolveActivityLogEntryDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ResolveActivityLogEntry"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resolveActivityLogEntry"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"resolved"}},{"kind":"Field","name":{"kind":"Name","value":"resolvedAt"}}]}}]}}]} as unknown as DocumentNode; export const ResolveFeedbackDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ResolveFeedback"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isResolved"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resolveFeedback"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"isResolved"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isResolved"}}},{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}}]}}]}}]} as unknown as DocumentNode; @@ -7832,6 +8387,7 @@ export const SendMagicLinkDocument = {"kind":"Document","definitions":[{"kind":" export const SetActivityReviewStatusDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SetActivityReviewStatus"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"isReviewed"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"setActivityReviewStatus"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"activityId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}}},{"kind":"Argument","name":{"kind":"Name","value":"activityType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityType"}}},{"kind":"Argument","name":{"kind":"Name","value":"isReviewed"},"value":{"kind":"Variable","name":{"kind":"Name","value":"isReviewed"}}}]}]}}]} as unknown as DocumentNode; export const SetAiFeaturesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SetAiFeatures"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"email"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"setAiFeatures"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"email"},"value":{"kind":"Variable","name":{"kind":"Name","value":"email"}}},{"kind":"Argument","name":{"kind":"Name","value":"enabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}}}]}]}}]} as unknown as DocumentNode; export const SetBetaFeaturesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SetBetaFeatures"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"setBetaFeatures"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"enabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}}}]}]}}]} as unknown as DocumentNode; +export const SetKbKnowledgeGraphEnabledDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SetKbKnowledgeGraphEnabled"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"setKbKnowledgeGraphEnabled"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"enabled"},"value":{"kind":"Variable","name":{"kind":"Name","value":"enabled"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"isEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"statusMessage"}},{"kind":"Field","name":{"kind":"Name","value":"qualityTier"}},{"kind":"Field","name":{"kind":"Name","value":"sourceContentDigest"}},{"kind":"Field","name":{"kind":"Name","value":"activeBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"publishedBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"startedAt"}},{"kind":"Field","name":{"kind":"Name","value":"finishedAt"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"costConfigurationReady"}},{"kind":"Field","name":{"kind":"Name","value":"costCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"quotaCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"billingLabel"}},{"kind":"Field","name":{"kind":"Name","value":"standardEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"highEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"estimatedCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualInputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualOutputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualEmbeddingTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualRequestCount"}},{"kind":"Field","name":{"kind":"Name","value":"maxCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"costStatus"}},{"kind":"Field","name":{"kind":"Name","value":"semesterKey"}},{"kind":"Field","name":{"kind":"Name","value":"semesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterReservedMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterSettledMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"remainingSemesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"worstCaseRemainingMinorUnits"}}]}}]}}]} as unknown as DocumentNode; export const SetLiveQuizPinDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SetLiveQuizPin"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"pin"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"setLiveQuizPin"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}},{"kind":"Argument","name":{"kind":"Name","value":"pin"},"value":{"kind":"Variable","name":{"kind":"Name","value":"pin"}}}]}]}}]} as unknown as DocumentNode; export const ShareElementsBatchDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ShareElementsBatch"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"elementIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionLevel"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"shortnameOrEmail"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"userGroupId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"shareElementsBatch"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"elementIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"elementIds"}}},{"kind":"Argument","name":{"kind":"Name","value":"permissionLevel"},"value":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}}},{"kind":"Argument","name":{"kind":"Name","value":"shortnameOrEmail"},"value":{"kind":"Variable","name":{"kind":"Name","value":"shortnameOrEmail"}}},{"kind":"Argument","name":{"kind":"Name","value":"userGroupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"userGroupId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"targetError"}},{"kind":"Field","name":{"kind":"Name","value":"outcomes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"reason"}}]}}]}}]}}]} as unknown as DocumentNode; export const ShareObjectDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ShareObject"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ObjectType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionLevel"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"shortnameOrEmail"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"userGroupId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"propagation"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"shareObject"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"objectId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectId"}}},{"kind":"Argument","name":{"kind":"Name","value":"objectType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"objectType"}}},{"kind":"Argument","name":{"kind":"Name","value":"permissionLevel"},"value":{"kind":"Variable","name":{"kind":"Name","value":"permissionLevel"}}},{"kind":"Argument","name":{"kind":"Name","value":"shortnameOrEmail"},"value":{"kind":"Variable","name":{"kind":"Name","value":"shortnameOrEmail"}}},{"kind":"Argument","name":{"kind":"Name","value":"userGroupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"userGroupId"}}},{"kind":"Argument","name":{"kind":"Name","value":"propagation"},"value":{"kind":"Variable","name":{"kind":"Name","value":"propagation"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"PermissionInfoData"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"PermissionInfoData"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"PermissionInfo"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"permissionId"}},{"kind":"Field","name":{"kind":"Name","value":"userId"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"userEmail"}},{"kind":"Field","name":{"kind":"Name","value":"userGroupName"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"propagation"}},{"kind":"Field","name":{"kind":"Name","value":"isOwn"}}]}}]} as unknown as DocumentNode; @@ -7889,7 +8445,7 @@ export const GetCatalogObjectsDocument = {"kind":"Document","definitions":[{"kin export const GetCatalogSharingRequestsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetCatalogSharingRequests"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getCatalogSharingRequests"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"requestId"}},{"kind":"Field","name":{"kind":"Name","value":"objectName"}},{"kind":"Field","name":{"kind":"Name","value":"objectType"}},{"kind":"Field","name":{"kind":"Name","value":"userId"}},{"kind":"Field","name":{"kind":"Name","value":"userShortname"}},{"kind":"Field","name":{"kind":"Name","value":"userEmail"}}]}}]}}]} as unknown as DocumentNode; export const QGetCatalystRequestAccessDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"QGetCatalystRequestAccess"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userScope"}}]}}]} as unknown as DocumentNode; export const GetChatModelRegistryDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetChatModelRegistry"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getChatModelRegistry"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"fallback"}},{"kind":"Field","name":{"kind":"Name","value":"supportsReasoning"}},{"kind":"Field","name":{"kind":"Name","value":"supportedReasoningEfforts"}}]}}]}}]} as unknown as DocumentNode; -export const GetChatbotsInfoDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetChatbotsInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getChatbotsInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"modelSelection"}},{"kind":"Field","name":{"kind":"Name","value":"allowedModelIds"}},{"kind":"Field","name":{"kind":"Name","value":"allowedReasoningEffortsByModel"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"modelId"}},{"kind":"Field","name":{"kind":"Name","value":"efforts"}}]}},{"kind":"Field","name":{"kind":"Name","value":"creditInitialCredits"}},{"kind":"Field","name":{"kind":"Name","value":"creditResetPeriod"}},{"kind":"Field","name":{"kind":"Name","value":"creditResetAmount"}},{"kind":"Field","name":{"kind":"Name","value":"creditMaxCredits"}},{"kind":"Field","name":{"kind":"Name","value":"courses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"usageSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"threadCount"}},{"kind":"Field","name":{"kind":"Name","value":"messageCount"}},{"kind":"Field","name":{"kind":"Name","value":"participantCount"}},{"kind":"Field","name":{"kind":"Name","value":"lastActivityAt"}},{"kind":"Field","name":{"kind":"Name","value":"totalCredits"}},{"kind":"Field","name":{"kind":"Name","value":"currentCredits"}},{"kind":"Field","name":{"kind":"Name","value":"totalResets"}},{"kind":"Field","name":{"kind":"Name","value":"lastResetAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"disclaimerSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"acceptedCount"}},{"kind":"Field","name":{"kind":"Name","value":"declinedCount"}},{"kind":"Field","name":{"kind":"Name","value":"pendingCount"}}]}},{"kind":"Field","name":{"kind":"Name","value":"mcpConfigurations"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"serverId"}},{"kind":"Field","name":{"kind":"Name","value":"serverName"}},{"kind":"Field","name":{"kind":"Name","value":"serverDescription"}},{"kind":"Field","name":{"kind":"Name","value":"serverIsActive"}},{"kind":"Field","name":{"kind":"Name","value":"chatMode"}},{"kind":"Field","name":{"kind":"Name","value":"isEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"priority"}},{"kind":"Field","name":{"kind":"Name","value":"allowedToolsCount"}}]}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]}}]} as unknown as DocumentNode; +export const GetChatbotsInfoDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetChatbotsInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getChatbotsInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"modelSelection"}},{"kind":"Field","name":{"kind":"Name","value":"allowedModelIds"}},{"kind":"Field","name":{"kind":"Name","value":"allowedReasoningEffortsByModel"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"modelId"}},{"kind":"Field","name":{"kind":"Name","value":"efforts"}}]}},{"kind":"Field","name":{"kind":"Name","value":"creditInitialCredits"}},{"kind":"Field","name":{"kind":"Name","value":"creditResetPeriod"}},{"kind":"Field","name":{"kind":"Name","value":"creditResetAmount"}},{"kind":"Field","name":{"kind":"Name","value":"creditMaxCredits"}},{"kind":"Field","name":{"kind":"Name","value":"courses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"usageSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"threadCount"}},{"kind":"Field","name":{"kind":"Name","value":"messageCount"}},{"kind":"Field","name":{"kind":"Name","value":"participantCount"}},{"kind":"Field","name":{"kind":"Name","value":"lastActivityAt"}},{"kind":"Field","name":{"kind":"Name","value":"totalCredits"}},{"kind":"Field","name":{"kind":"Name","value":"currentCredits"}},{"kind":"Field","name":{"kind":"Name","value":"totalResets"}},{"kind":"Field","name":{"kind":"Name","value":"lastResetAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"disclaimerSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"acceptedCount"}},{"kind":"Field","name":{"kind":"Name","value":"declinedCount"}},{"kind":"Field","name":{"kind":"Name","value":"pendingCount"}}]}},{"kind":"Field","name":{"kind":"Name","value":"mcpConfigurations"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"serverId"}},{"kind":"Field","name":{"kind":"Name","value":"serverName"}},{"kind":"Field","name":{"kind":"Name","value":"serverDescription"}},{"kind":"Field","name":{"kind":"Name","value":"serverIsActive"}},{"kind":"Field","name":{"kind":"Name","value":"chatMode"}},{"kind":"Field","name":{"kind":"Name","value":"isEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"priority"}},{"kind":"Field","name":{"kind":"Name","value":"allowedToolsCount"}}]}},{"kind":"Field","name":{"kind":"Name","value":"enabledKnowledgeBase"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]}}]} as unknown as DocumentNode; export const GetCockpitQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetCockpitQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"cockpitQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isLiveQAEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isConfusionFeedbackEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isModerationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"namespace"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"startedAt"}},{"kind":"Field","name":{"kind":"Name","value":"course"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"language"}}]}},{"kind":"Field","name":{"kind":"Name","value":"blocks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"numOfParticipants"}},{"kind":"Field","name":{"kind":"Name","value":"order"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}},{"kind":"Field","name":{"kind":"Name","value":"timeLimit"}},{"kind":"Field","name":{"kind":"Name","value":"randomSelection"}},{"kind":"Field","name":{"kind":"Name","value":"execution"}},{"kind":"Field","name":{"kind":"Name","value":"elements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"elementType"}},{"kind":"FragmentSpread","name":{"kind":"Name","value":"ElementDataInfo"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"activeBlock"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}}]}},{"kind":"Field","name":{"kind":"Name","value":"confusionSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"speed"}},{"kind":"Field","name":{"kind":"Name","value":"difficulty"}},{"kind":"Field","name":{"kind":"Name","value":"numberOfParticipants"}}]}},{"kind":"Field","name":{"kind":"Name","value":"feedbacks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isPublished"}},{"kind":"Field","name":{"kind":"Name","value":"isPinned"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"votes"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"resolvedAt"}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"positiveReactions"}},{"kind":"Field","name":{"kind":"Name","value":"negativeReactions"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ElementDataInfo"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ElementInstance"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"elementData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}}]}}]}}]} as unknown as DocumentNode; export const GetControlCourseDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetControlCourse"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"controlCourse"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"liveQuizzes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}}]}}]}}]}}]} as unknown as DocumentNode; export const GetControlCoursesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetControlCourses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"controlCourses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"description"}}]}}]}}]} as unknown as DocumentNode; @@ -7924,6 +8480,13 @@ export const GetGroupActivityDocument = {"kind":"Document","definitions":[{"kind export const GetGroupActivityInstancesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetGroupActivityInstances"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"groupActivityInstances"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"groupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}}},{"kind":"Argument","name":{"kind":"Name","value":"courseId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"courseId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"decisionsSubmittedAt"}},{"kind":"Field","name":{"kind":"Name","value":"resultsComputedAt"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"passed"}},{"kind":"Field","name":{"kind":"Name","value":"points"}},{"kind":"Field","name":{"kind":"Name","value":"comment"}},{"kind":"Field","name":{"kind":"Name","value":"grading"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"instanceId"}},{"kind":"Field","name":{"kind":"Name","value":"score"}},{"kind":"Field","name":{"kind":"Name","value":"maxPoints"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"groupActivityId"}}]}}]}}]} as unknown as DocumentNode; export const GetGroupActivitySummaryDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetGroupActivitySummary"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getGroupActivitySummary"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"numOfStartedInstances"}},{"kind":"Field","name":{"kind":"Name","value":"numOfSubmissions"}}]}}]}}]} as unknown as DocumentNode; export const GetInstanceUpdateActivitiesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetInstanceUpdateActivities"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"elementId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"hasSampleSolution"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"includeTemplateInstances"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getInstanceUpdateActivities"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"elementId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"elementId"}}},{"kind":"Argument","name":{"kind":"Name","value":"hasSampleSolution"},"value":{"kind":"Variable","name":{"kind":"Name","value":"hasSampleSolution"}}},{"kind":"Argument","name":{"kind":"Name","value":"includeTemplateInstances"},"value":{"kind":"Variable","name":{"kind":"Name","value":"includeTemplateInstances"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"activityId"}},{"kind":"Field","name":{"kind":"Name","value":"activityName"}},{"kind":"Field","name":{"kind":"Name","value":"courseName"}},{"kind":"Field","name":{"kind":"Name","value":"activityType"}},{"kind":"Field","name":{"kind":"Name","value":"status"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKb"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKb"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"metrics"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"visibleResourceCount"}},{"kind":"Field","name":{"kind":"Name","value":"visibleSizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"unknownSizeResourceCount"}},{"kind":"Field","name":{"kind":"Name","value":"quotaResourceCount"}},{"kind":"Field","name":{"kind":"Name","value":"quotaSizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"resourceLimit"}},{"kind":"Field","name":{"kind":"Name","value":"storageLimitBytes"}},{"kind":"Field","name":{"kind":"Name","value":"pendingCleanupCount"}},{"kind":"Field","name":{"kind":"Name","value":"pendingCleanupSizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"reservedResourceCount"}},{"kind":"Field","name":{"kind":"Name","value":"reservedSizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"linkedConsumerCount"}}]}}]}}]}}]} as unknown as DocumentNode; +export const GetKbChatbotBindingsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbChatbotBindings"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbChatbotBindings"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"chatbotId"}},{"kind":"Field","name":{"kind":"Name","value":"chatbotName"}},{"kind":"Field","name":{"kind":"Name","value":"enabledKbId"}},{"kind":"Field","name":{"kind":"Name","value":"enabledKbName"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbKnowledgeGraphConfigDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbKnowledgeGraphConfig"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbKnowledgeGraphConfig"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"isEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"statusMessage"}},{"kind":"Field","name":{"kind":"Name","value":"qualityTier"}},{"kind":"Field","name":{"kind":"Name","value":"sourceContentDigest"}},{"kind":"Field","name":{"kind":"Name","value":"activeBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"publishedBuildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"startedAt"}},{"kind":"Field","name":{"kind":"Name","value":"finishedAt"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"costConfigurationReady"}},{"kind":"Field","name":{"kind":"Name","value":"costCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"quotaCurrency"}},{"kind":"Field","name":{"kind":"Name","value":"billingLabel"}},{"kind":"Field","name":{"kind":"Name","value":"standardEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"highEstimateMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"estimatedCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"actualInputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualOutputTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualEmbeddingTokens"}},{"kind":"Field","name":{"kind":"Name","value":"actualRequestCount"}},{"kind":"Field","name":{"kind":"Name","value":"maxCostMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"costStatus"}},{"kind":"Field","name":{"kind":"Name","value":"semesterKey"}},{"kind":"Field","name":{"kind":"Name","value":"semesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterReservedMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"semesterSettledMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"remainingSemesterQuotaMinorUnits"}},{"kind":"Field","name":{"kind":"Name","value":"worstCaseRemainingMinorUnits"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbKnowledgeGraphNeighborsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbKnowledgeGraphNeighbors"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"nodeId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbKnowledgeGraphNeighbors"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"nodeId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"nodeId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"nodes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"labels"}},{"kind":"Field","name":{"kind":"Name","value":"kind"}},{"kind":"Field","name":{"kind":"Name","value":"displayLabel"}},{"kind":"Field","name":{"kind":"Name","value":"summary"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"degree"}},{"kind":"Field","name":{"kind":"Name","value":"sourceReferences"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resourceId"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"reference"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"edges"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"source"}},{"kind":"Field","name":{"kind":"Name","value":"target"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"label"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}}]}},{"kind":"Field","name":{"kind":"Name","value":"truncated"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbKnowledgeGraphOverviewDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbKnowledgeGraphOverview"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbKnowledgeGraphOverview"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"nodes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"labels"}},{"kind":"Field","name":{"kind":"Name","value":"kind"}},{"kind":"Field","name":{"kind":"Name","value":"displayLabel"}},{"kind":"Field","name":{"kind":"Name","value":"summary"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"degree"}},{"kind":"Field","name":{"kind":"Name","value":"sourceReferences"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resourceId"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"reference"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"edges"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"source"}},{"kind":"Field","name":{"kind":"Name","value":"target"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"label"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}}]}},{"kind":"Field","name":{"kind":"Name","value":"truncated"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbResourceIngestionRunsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbResourceIngestionRuns"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"resourceId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbResourceIngestionRuns"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"resourceId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"resourceId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"resourceVersion"}},{"kind":"Field","name":{"kind":"Name","value":"errorCode"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}}]} as unknown as DocumentNode; +export const GetKbResourcesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetKbResources"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"first"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"after"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"search"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"type"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"KBResourceType"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"status"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"KBIngestionStatus"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getKbResources"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"first"},"value":{"kind":"Variable","name":{"kind":"Name","value":"first"}}},{"kind":"Argument","name":{"kind":"Name","value":"after"},"value":{"kind":"Variable","name":{"kind":"Name","value":"after"}}},{"kind":"Argument","name":{"kind":"Name","value":"search"},"value":{"kind":"Variable","name":{"kind":"Name","value":"search"}}},{"kind":"Argument","name":{"kind":"Name","value":"type"},"value":{"kind":"Variable","name":{"kind":"Name","value":"type"}}},{"kind":"Argument","name":{"kind":"Name","value":"status"},"value":{"kind":"Variable","name":{"kind":"Name","value":"status"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"items"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"sourceUrl"}},{"kind":"Field","name":{"kind":"Name","value":"originalFilename"}},{"kind":"Field","name":{"kind":"Name","value":"mimeType"}},{"kind":"Field","name":{"kind":"Name","value":"sizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"ingestedAt"}},{"kind":"Field","name":{"kind":"Name","value":"resourceVersion"}},{"kind":"Field","name":{"kind":"Name","value":"activeResourceVersion"}},{"kind":"Field","name":{"kind":"Name","value":"latestIngestionRun"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"errorCode"}}]}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"pageInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasNextPage"}},{"kind":"Field","name":{"kind":"Name","value":"endCursor"}}]}},{"kind":"Field","name":{"kind":"Name","value":"totalCount"}}]}}]}}]} as unknown as DocumentNode; export const GetLecturerViewLiveQuizDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetLecturerViewLiveQuiz"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getLecturerViewLiveQuiz"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isLiveQAEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isConfusionFeedbackEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isModerationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"confusionSummary"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"speed"}},{"kind":"Field","name":{"kind":"Name","value":"difficulty"}},{"kind":"Field","name":{"kind":"Name","value":"numberOfParticipants"}}]}},{"kind":"Field","name":{"kind":"Name","value":"feedbacks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isPublished"}},{"kind":"Field","name":{"kind":"Name","value":"isPinned"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"votes"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"resolvedAt"}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"positiveReactions"}},{"kind":"Field","name":{"kind":"Name","value":"negativeReactions"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const GetLiveQuizEmbeddingInfoDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetLiveQuizEmbeddingInfo"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getLiveQuizEmbeddingInfo"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"hmac"}},{"kind":"Field","name":{"kind":"Name","value":"instances"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}}]}}]} as unknown as DocumentNode; export const GetLiveQuizEvaluationDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetLiveQuizEvaluation"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"hmac"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"liveQuizEvaluation"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"hmac"},"value":{"kind":"Variable","name":{"kind":"Name","value":"hmac"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"courseLanguage"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"pinCode"}},{"kind":"FragmentSpread","name":{"kind":"Name","value":"EvaluationResults"}},{"kind":"Field","name":{"kind":"Name","value":"feedbacks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"isPublished"}},{"kind":"Field","name":{"kind":"Name","value":"isPinned"}},{"kind":"Field","name":{"kind":"Name","value":"isResolved"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"votes"}},{"kind":"Field","name":{"kind":"Name","value":"resolvedAt"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"positiveReactions"}},{"kind":"Field","name":{"kind":"Name","value":"negativeReactions"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"confusionFeedbacks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"speed"}},{"kind":"Field","name":{"kind":"Name","value":"difficulty"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"liveQuizLeaderboard"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"quizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}},{"kind":"Argument","name":{"kind":"Name","value":"hmac"},"value":{"kind":"Variable","name":{"kind":"Name","value":"hmac"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"participantId"}},{"kind":"Field","name":{"kind":"Name","value":"rank"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"score"}},{"kind":"Field","name":{"kind":"Name","value":"isTemporary"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"EvaluationResults"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ActivityEvaluation"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"stackId"}},{"kind":"Field","name":{"kind":"Name","value":"stackName"}},{"kind":"Field","name":{"kind":"Name","value":"stackDescription"}},{"kind":"Field","name":{"kind":"Name","value":"stackOrder"}},{"kind":"Field","name":{"kind":"Name","value":"stackActive"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}},{"kind":"Field","name":{"kind":"Name","value":"timeLimit"}},{"kind":"Field","name":{"kind":"Name","value":"instances"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"choices"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"count"}},{"kind":"Field","name":{"kind":"Name","value":"correct"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"maxValue"}},{"kind":"Field","name":{"kind":"Name","value":"minValue"}},{"kind":"Field","name":{"kind":"Name","value":"solutionRanges"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}},{"kind":"Field","name":{"kind":"Name","value":"exactSolutions"}},{"kind":"Field","name":{"kind":"Name","value":"responseValues"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"correct"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"statistics"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"max"}},{"kind":"Field","name":{"kind":"Name","value":"mean"}},{"kind":"Field","name":{"kind":"Name","value":"median"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"q1"}},{"kind":"Field","name":{"kind":"Name","value":"q3"}},{"kind":"Field","name":{"kind":"Name","value":"sd"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"maxLength"}},{"kind":"Field","name":{"kind":"Name","value":"solutions"}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"correct"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"numberOfInputs"}},{"kind":"Field","name":{"kind":"Name","value":"answerSolutionIds"}},{"kind":"Field","name":{"kind":"Name","value":"selectionResponses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"answerId"}},{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"cases"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}}]}},{"kind":"Field","name":{"kind":"Name","value":"items"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}},{"kind":"Field","name":{"kind":"Name","value":"criteria"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"labels"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"mid"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"caseResults"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"caseId"}},{"kind":"Field","name":{"kind":"Name","value":"items"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"itemId"}},{"kind":"Field","name":{"kind":"Name","value":"criteria"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"criterionId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}},{"kind":"Field","name":{"kind":"Name","value":"step"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"solutionMin"}},{"kind":"Field","name":{"kind":"Name","value":"solutionMax"}},{"kind":"Field","name":{"kind":"Name","value":"statistics"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}},{"kind":"Field","name":{"kind":"Name","value":"mean"}},{"kind":"Field","name":{"kind":"Name","value":"median"}},{"kind":"Field","name":{"kind":"Name","value":"q1"}},{"kind":"Field","name":{"kind":"Name","value":"q3"}},{"kind":"Field","name":{"kind":"Name","value":"sd"}}]}},{"kind":"Field","name":{"kind":"Name","value":"responses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}}]}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"correctCount"}},{"kind":"Field","name":{"kind":"Name","value":"partialCount"}},{"kind":"Field","name":{"kind":"Name","value":"incorrectCount"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentActivityEvaluationData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"totalAnswers"}},{"kind":"Field","name":{"kind":"Name","value":"anonymousAnswers"}}]}}]}}]}}]}}]}}]} as unknown as DocumentNode; @@ -7969,6 +8532,7 @@ export const GetUserActivitiesCoursesDocument = {"kind":"Document","definitions" export const GetUserCoursesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserCourses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userCourses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"color"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isAssessmentEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"isGroupCreationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"startDate"}},{"kind":"Field","name":{"kind":"Name","value":"endDate"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}}]}}]}}]} as unknown as DocumentNode; export const GetUserElementsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserElements"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"status"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementStatus"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"type"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"ElementType"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"hasSampleSolution"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"searchString"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"showOwned"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"showShared"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"showDependencies"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"tagIds"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"showUntagged"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"sortByType"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"SortByType"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"sortByAsc"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"showArchived"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"numEntries"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"offset"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userElements"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"status"},"value":{"kind":"Variable","name":{"kind":"Name","value":"status"}}},{"kind":"Argument","name":{"kind":"Name","value":"type"},"value":{"kind":"Variable","name":{"kind":"Name","value":"type"}}},{"kind":"Argument","name":{"kind":"Name","value":"hasSampleSolution"},"value":{"kind":"Variable","name":{"kind":"Name","value":"hasSampleSolution"}}},{"kind":"Argument","name":{"kind":"Name","value":"hasAnswerFeedbacks"},"value":{"kind":"Variable","name":{"kind":"Name","value":"hasAnswerFeedbacks"}}},{"kind":"Argument","name":{"kind":"Name","value":"searchString"},"value":{"kind":"Variable","name":{"kind":"Name","value":"searchString"}}},{"kind":"Argument","name":{"kind":"Name","value":"showOwned"},"value":{"kind":"Variable","name":{"kind":"Name","value":"showOwned"}}},{"kind":"Argument","name":{"kind":"Name","value":"showShared"},"value":{"kind":"Variable","name":{"kind":"Name","value":"showShared"}}},{"kind":"Argument","name":{"kind":"Name","value":"showDependencies"},"value":{"kind":"Variable","name":{"kind":"Name","value":"showDependencies"}}},{"kind":"Argument","name":{"kind":"Name","value":"tagIds"},"value":{"kind":"Variable","name":{"kind":"Name","value":"tagIds"}}},{"kind":"Argument","name":{"kind":"Name","value":"activityId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}}},{"kind":"Argument","name":{"kind":"Name","value":"multiplier"},"value":{"kind":"Variable","name":{"kind":"Name","value":"multiplier"}}},{"kind":"Argument","name":{"kind":"Name","value":"showUntagged"},"value":{"kind":"Variable","name":{"kind":"Name","value":"showUntagged"}}},{"kind":"Argument","name":{"kind":"Name","value":"sortByType"},"value":{"kind":"Variable","name":{"kind":"Name","value":"sortByType"}}},{"kind":"Argument","name":{"kind":"Name","value":"sortByAsc"},"value":{"kind":"Variable","name":{"kind":"Name","value":"sortByAsc"}}},{"kind":"Argument","name":{"kind":"Name","value":"showArchived"},"value":{"kind":"Variable","name":{"kind":"Name","value":"showArchived"}}},{"kind":"Argument","name":{"kind":"Name","value":"numEntries"},"value":{"kind":"Variable","name":{"kind":"Name","value":"numEntries"}}},{"kind":"Argument","name":{"kind":"Name","value":"offset"},"value":{"kind":"Variable","name":{"kind":"Name","value":"offset"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"numOfElements"}},{"kind":"Field","name":{"kind":"Name","value":"elements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"displayMode"}},{"kind":"Field","name":{"kind":"Name","value":"choices"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"ix"}},{"kind":"Field","name":{"kind":"Name","value":"correct"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"placeholder"}},{"kind":"Field","name":{"kind":"Name","value":"accuracy"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"solutionRanges"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}},{"kind":"Field","name":{"kind":"Name","value":"exactSolutions"}},{"kind":"Field","name":{"kind":"Name","value":"restrictions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"hasAnswerFeedbacks"}},{"kind":"Field","name":{"kind":"Name","value":"solutions"}},{"kind":"Field","name":{"kind":"Name","value":"restrictions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"maxLength"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"numberOfInputs"}}]}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"criteria"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}},{"kind":"Field","name":{"kind":"Name","value":"step"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"labels"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"mid"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"cases"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentElement"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"isArchived"}},{"kind":"Field","name":{"kind":"Name","value":"isDeleted"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"derivedAccess"}},{"kind":"Field","name":{"kind":"Name","value":"numSharedUsers"}},{"kind":"Field","name":{"kind":"Name","value":"permissionLevel"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}},{"kind":"Field","name":{"kind":"Name","value":"isManager"}},{"kind":"Field","name":{"kind":"Name","value":"isEditor"}},{"kind":"Field","name":{"kind":"Name","value":"isImported"}},{"kind":"Field","name":{"kind":"Name","value":"isShared"}},{"kind":"Field","name":{"kind":"Name","value":"isRemovable"}},{"kind":"Field","name":{"kind":"Name","value":"sharingType"}},{"kind":"Field","name":{"kind":"Name","value":"tags"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"order"}}]}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const GetUserGroupsUserDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserGroupsUser"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getUserGroupsUser"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"members"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}},{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"isSelf"}}]}},{"kind":"Field","name":{"kind":"Name","value":"admins"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}},{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"isSelf"}}]}},{"kind":"Field","name":{"kind":"Name","value":"owner"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}},{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"isSelf"}}]}},{"kind":"Field","name":{"kind":"Name","value":"numOfMembers"}},{"kind":"Field","name":{"kind":"Name","value":"isMember"}},{"kind":"Field","name":{"kind":"Name","value":"isAdmin"}},{"kind":"Field","name":{"kind":"Name","value":"isOwner"}}]}}]}}]} as unknown as DocumentNode; +export const GetUserKbsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserKbs"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"first"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Int"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"after"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"search"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"getUserKbsConnection"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"first"},"value":{"kind":"Variable","name":{"kind":"Name","value":"first"}}},{"kind":"Argument","name":{"kind":"Name","value":"after"},"value":{"kind":"Variable","name":{"kind":"Name","value":"after"}}},{"kind":"Argument","name":{"kind":"Name","value":"search"},"value":{"kind":"Variable","name":{"kind":"Name","value":"search"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"items"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"metrics"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"visibleResourceCount"}},{"kind":"Field","name":{"kind":"Name","value":"visibleSizeBytes"}},{"kind":"Field","name":{"kind":"Name","value":"linkedConsumerCount"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"pageInfo"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasNextPage"}},{"kind":"Field","name":{"kind":"Name","value":"endCursor"}}]}},{"kind":"Field","name":{"kind":"Name","value":"totalCount"}}]}}]}}]} as unknown as DocumentNode; export const GetUserLoginsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserLogins"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userLogins"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"scope"}},{"kind":"Field","name":{"kind":"Name","value":"lastLoginAt"}},{"kind":"Field","name":{"kind":"Name","value":"user"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"userScope"}}]}}]} as unknown as DocumentNode; export const GetUserMediaFilesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserMediaFiles"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userMediaFiles"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"href"}}]}}]}}]} as unknown as DocumentNode; export const GetUserRunningLiveQuizzesDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetUserRunningLiveQuizzes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userRunningLiveQuizzes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}}]} as unknown as DocumentNode; @@ -7978,6 +8542,7 @@ export const GetUsersPrivatePreviewDocument = {"kind":"Document","definitions":[ export const QGetVerifiableCredentialDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"QGetVerifiableCredential"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"token"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"assessmentReportVerification"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"token"},"value":{"kind":"Variable","name":{"kind":"Name","value":"token"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"issuedAt"}},{"kind":"Field","name":{"kind":"Name","value":"snapshot"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"version"}},{"kind":"Field","name":{"kind":"Name","value":"subject"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"source"}}]}},{"kind":"Field","name":{"kind":"Name","value":"course"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}}]}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableBasePoints"}},{"kind":"Field","name":{"kind":"Name","value":"correctnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableCorrectnessPoints"}},{"kind":"Field","name":{"kind":"Name","value":"bonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableBonusPoints"}},{"kind":"Field","name":{"kind":"Name","value":"totalPoints"}},{"kind":"Field","name":{"kind":"Name","value":"availableTotalPoints"}}]}},{"kind":"Field","name":{"kind":"Name","value":"comparison"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"cohortSize"}},{"kind":"Field","name":{"kind":"Name","value":"percentile"}},{"kind":"Field","name":{"kind":"Name","value":"histogram"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"binStart"}},{"kind":"Field","name":{"kind":"Name","value":"binEnd"}},{"kind":"Field","name":{"kind":"Name","value":"count"}}]}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const GroupActivityDetailsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GroupActivityDetails"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"groupActivityDetails"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"activityId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"activityId"}}},{"kind":"Argument","name":{"kind":"Name","value":"groupId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"groupId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}},{"kind":"Field","name":{"kind":"Name","value":"clues"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}}]}},{"kind":"Field","name":{"kind":"Name","value":"stacks"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"order"}},{"kind":"Field","name":{"kind":"Name","value":"elements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"elementType"}},{"kind":"FragmentSpread","name":{"kind":"Name","value":"ElementDataWithoutSolutions"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"course"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"color"}}]}},{"kind":"Field","name":{"kind":"Name","value":"group"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"participants"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"isSelf"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"activityInstance"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"clues"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"value"}},{"kind":"Field","name":{"kind":"Name","value":"participant"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"isSelf"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"decisionsSubmittedAt"}},{"kind":"Field","name":{"kind":"Name","value":"decisions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"instanceId"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"freeTextResponse"}},{"kind":"Field","name":{"kind":"Name","value":"choicesResponse"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"ix"}},{"kind":"Field","name":{"kind":"Name","value":"selected"}}]}},{"kind":"Field","name":{"kind":"Name","value":"numericalResponse"}},{"kind":"Field","name":{"kind":"Name","value":"contentResponse"}},{"kind":"Field","name":{"kind":"Name","value":"selectionResponse"}},{"kind":"Field","name":{"kind":"Name","value":"caseStudyResponse"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"caseId"}},{"kind":"Field","name":{"kind":"Name","value":"itemResponses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"itemId"}},{"kind":"Field","name":{"kind":"Name","value":"criterionResponses"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"criterionId"}},{"kind":"Field","name":{"kind":"Name","value":"response"}}]}}]}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"resultsComputedAt"}},{"kind":"Field","name":{"kind":"Name","value":"results"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"passed"}},{"kind":"Field","name":{"kind":"Name","value":"points"}},{"kind":"Field","name":{"kind":"Name","value":"comment"}},{"kind":"Field","name":{"kind":"Name","value":"grading"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"instanceId"}},{"kind":"Field","name":{"kind":"Name","value":"score"}},{"kind":"Field","name":{"kind":"Name","value":"maxPoints"}},{"kind":"Field","name":{"kind":"Name","value":"feedback"}}]}}]}}]}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"ElementDataWithoutSolutions"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ElementInstance"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"elementData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ChoicesElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"displayMode"}},{"kind":"Field","name":{"kind":"Name","value":"choices"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"ix"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"NumericalElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"accuracy"}},{"kind":"Field","name":{"kind":"Name","value":"placeholder"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"restrictions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FreeTextElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"restrictions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"maxLength"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"SelectionElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"numberOfInputs"}},{"kind":"Field","name":{"kind":"Name","value":"answerCollection"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"entries"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"CaseStudyElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}},{"kind":"Field","name":{"kind":"Name","value":"options"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"hasSampleSolution"}},{"kind":"Field","name":{"kind":"Name","value":"items"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"value"}}]}},{"kind":"Field","name":{"kind":"Name","value":"criteria"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"max"}},{"kind":"Field","name":{"kind":"Name","value":"step"}},{"kind":"Field","name":{"kind":"Name","value":"unit"}},{"kind":"Field","name":{"kind":"Name","value":"labels"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"min"}},{"kind":"Field","name":{"kind":"Name","value":"mid"}},{"kind":"Field","name":{"kind":"Name","value":"max"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"cases"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"description"}}]}}]}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"FlashcardElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}},{"kind":"InlineFragment","typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"ContentElementData"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"__typename"}},{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"elementId"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"explanation"}},{"kind":"Field","name":{"kind":"Name","value":"basePoints"}},{"kind":"Field","name":{"kind":"Name","value":"pointsMultiplier"}}]}}]}}]}}]} as unknown as DocumentNode; export const ParticipationsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"Participations"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"endpoint"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"assessmentOnly"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"Boolean"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"participations"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"endpoint"},"value":{"kind":"Variable","name":{"kind":"Name","value":"endpoint"}}},{"kind":"Argument","name":{"kind":"Name","value":"assessmentOnly"},"value":{"kind":"Variable","name":{"kind":"Name","value":"assessmentOnly"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"completedMicroLearnings"}},{"kind":"Field","name":{"kind":"Name","value":"subscriptions"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"endpoint"}}]}},{"kind":"Field","name":{"kind":"Name","value":"course"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"startDate"}},{"kind":"Field","name":{"kind":"Name","value":"endDate"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"isGamificationEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"microLearnings"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledStartAt"}},{"kind":"Field","name":{"kind":"Name","value":"scheduledEndAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"liveQuizzes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"displayName"}}]}}]}}]}}]}}]} as unknown as DocumentNode; +export const SearchKbKnowledgeGraphDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"SearchKbKnowledgeGraph"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"query"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"searchKbKnowledgeGraph"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"kbId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"kbId"}}},{"kind":"Argument","name":{"kind":"Name","value":"query"},"value":{"kind":"Variable","name":{"kind":"Name","value":"query"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"kbId"}},{"kind":"Field","name":{"kind":"Name","value":"buildId"}},{"kind":"Field","name":{"kind":"Name","value":"isStale"}},{"kind":"Field","name":{"kind":"Name","value":"nodes"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"labels"}},{"kind":"Field","name":{"kind":"Name","value":"kind"}},{"kind":"Field","name":{"kind":"Name","value":"displayLabel"}},{"kind":"Field","name":{"kind":"Name","value":"summary"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"degree"}},{"kind":"Field","name":{"kind":"Name","value":"sourceReferences"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"resourceId"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"reference"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"edges"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"source"}},{"kind":"Field","name":{"kind":"Name","value":"target"}},{"kind":"Field","name":{"kind":"Name","value":"type"}},{"kind":"Field","name":{"kind":"Name","value":"label"}},{"kind":"Field","name":{"kind":"Name","value":"properties"}}]}},{"kind":"Field","name":{"kind":"Name","value":"truncated"}}]}}]}}]} as unknown as DocumentNode; export const SelfDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"Self"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"self"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"liveQuizId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"liveQuizId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"role"}},{"kind":"Field","name":{"kind":"Name","value":"scopeQuizId"}},{"kind":"Field","name":{"kind":"Name","value":"isCourseParticipant"}},{"kind":"Field","name":{"kind":"Name","value":"isCourseParticipationActive"}},{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"institutionalEmail"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"locale"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"avatarSettings"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"skinTone"}},{"kind":"Field","name":{"kind":"Name","value":"eyes"}},{"kind":"Field","name":{"kind":"Name","value":"mouth"}},{"kind":"Field","name":{"kind":"Name","value":"hair"}},{"kind":"Field","name":{"kind":"Name","value":"facialHair"}},{"kind":"Field","name":{"kind":"Name","value":"accessory"}},{"kind":"Field","name":{"kind":"Name","value":"hairColor"}},{"kind":"Field","name":{"kind":"Name","value":"clothing"}},{"kind":"Field","name":{"kind":"Name","value":"clothingColor"}}]}},{"kind":"Field","name":{"kind":"Name","value":"isActive"}},{"kind":"Field","name":{"kind":"Name","value":"isProfilePublic"}},{"kind":"Field","name":{"kind":"Name","value":"xp"}},{"kind":"Field","name":{"kind":"Name","value":"level"}},{"kind":"Field","name":{"kind":"Name","value":"levelData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"index"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"requiredXp"}},{"kind":"Field","name":{"kind":"Name","value":"nextLevel"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"index"}},{"kind":"Field","name":{"kind":"Name","value":"requiredXp"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}}]}}]}}]} as unknown as DocumentNode; export const SelfWithAchievementsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"SelfWithAchievements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"selfWithAchievements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"participant"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"username"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"avatarSettings"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"skinTone"}},{"kind":"Field","name":{"kind":"Name","value":"eyes"}},{"kind":"Field","name":{"kind":"Name","value":"mouth"}},{"kind":"Field","name":{"kind":"Name","value":"hair"}},{"kind":"Field","name":{"kind":"Name","value":"facialHair"}},{"kind":"Field","name":{"kind":"Name","value":"accessory"}},{"kind":"Field","name":{"kind":"Name","value":"hairColor"}},{"kind":"Field","name":{"kind":"Name","value":"clothing"}},{"kind":"Field","name":{"kind":"Name","value":"clothingColor"}}]}},{"kind":"Field","name":{"kind":"Name","value":"xp"}},{"kind":"Field","name":{"kind":"Name","value":"level"}},{"kind":"Field","name":{"kind":"Name","value":"levelData"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"index"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"requiredXp"}},{"kind":"Field","name":{"kind":"Name","value":"nextLevel"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"index"}},{"kind":"Field","name":{"kind":"Name","value":"requiredXp"}},{"kind":"Field","name":{"kind":"Name","value":"avatar"}},{"kind":"Field","name":{"kind":"Name","value":"name"}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"achievements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"achievedAt"}},{"kind":"Field","name":{"kind":"Name","value":"achievedCount"}},{"kind":"Field","name":{"kind":"Name","value":"achievement"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"nameDE"}},{"kind":"Field","name":{"kind":"Name","value":"nameEN"}},{"kind":"Field","name":{"kind":"Name","value":"descriptionDE"}},{"kind":"Field","name":{"kind":"Name","value":"descriptionEN"}},{"kind":"Field","name":{"kind":"Name","value":"icon"}},{"kind":"Field","name":{"kind":"Name","value":"iconColor"}}]}}]}}]}},{"kind":"Field","name":{"kind":"Name","value":"achievements"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"nameDE"}},{"kind":"Field","name":{"kind":"Name","value":"nameEN"}},{"kind":"Field","name":{"kind":"Name","value":"descriptionDE"}},{"kind":"Field","name":{"kind":"Name","value":"descriptionEN"}},{"kind":"Field","name":{"kind":"Name","value":"icon"}},{"kind":"Field","name":{"kind":"Name","value":"iconColor"}}]}}]}}]}}]} as unknown as DocumentNode; export const UserProfileDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"UserProfile"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"userProfile"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"email"}},{"kind":"Field","name":{"kind":"Name","value":"sendProjectUpdates"}},{"kind":"Field","name":{"kind":"Name","value":"shortname"}},{"kind":"Field","name":{"kind":"Name","value":"role"}},{"kind":"Field","name":{"kind":"Name","value":"locale"}},{"kind":"Field","name":{"kind":"Name","value":"firstLogin"}},{"kind":"Field","name":{"kind":"Name","value":"catalyst"}},{"kind":"Field","name":{"kind":"Name","value":"catalystTier"}},{"kind":"Field","name":{"kind":"Name","value":"publicPreview"}},{"kind":"Field","name":{"kind":"Name","value":"privatePreview"}},{"kind":"Field","name":{"kind":"Name","value":"aiFeaturesEnabled"}},{"kind":"Field","name":{"kind":"Name","value":"numChatbots"}}]}}]}}]} as unknown as DocumentNode; diff --git a/packages/graphql/src/public/client.json b/packages/graphql/src/public/client.json index 291b3ae7f9..e9a0b9d0fd 100644 --- a/packages/graphql/src/public/client.json +++ b/packages/graphql/src/public/client.json @@ -10,6 +10,7 @@ "ApplyActivityBatchOperations": "44fd2142f1dc40f055ab0195cb92ea30d6168aa84b97afade3b6d6522795c7bc", "ApplyElementBatchOperations": "b5069c86bd44976aee11e3b1972adc067d557a4639cbeba00fd8ca4b272dd42e", "ApproveObjectSharingRequest": "801c3fe541540847d366e631e3f8ebe0890c05d0bab2228f9a6ae7060fc71850", + "AttachKbToChatbot": "88a76e615654d4ab8d5b7b3b1038ba2903cad03c034240b94f03f5c8c08c1432", "BookmarkElementStack": "1c54224ef0af1eb882af6538f46b4ac40c1415062075092e9f85396c4646b136", "CancelLiveQuiz": "a270a5ab4a8e102679be3d43db8de35cc35d5650f42c0d02aa40548edadee6d4", "CancelObjectSharingRequest": "1e8aaef9bda5428b625946891e39dabe1a24ad11dcbbc9eb9ca020daf46dc9e7", @@ -25,6 +26,7 @@ "ChangeShortname": "07ed8db96ea01e17e68f1b598ba830a22199d44248cc473072fe2428491c4656", "ChangeUserGroupName": "a8efa0083d3cff34731ce21a28536512bccb280742dfa25a3fbde448c376168d", "ChangeUserLocale": "0a5f88b275c2319ba6c569426183657cdb7962b17651bf16a738f24766affd67", + "ConfirmKbFileUpload": "cb7ef08d5a8ddc0d653531bba5864a77a7702c6ad854305c4459c85592b22ac1", "CopyCatalogObjectToAccount": "9c048459c0e03342d23e900d7b380a12fcad17d99c6270455a519ee3e109d996", "CorrectAssessmentPointsInstance": "08518a9a14356bd169e5bafedb59e85b5472b59c67cc64cda68404e60339ad30", "CorrectAssessmentPointsLiveQuiz": "b5a0b0f4d4fbc5cbe135a1f264292156e0bf634ac08204bcb249dba7492525a3", @@ -34,6 +36,8 @@ "CreateCourse": "0284a39ca95a55fc13d865698a440007e4a95651a4940b062bd9d82689343f82", "CreateFeedback": "5a4a9e710048cb5b679e7d8217907d623c7c53638fec41183a0257931fc8b136", "CreateGroupActivity": "8a92f323833cb25abffce681c8f87bbd064caae245e8c965b5ebaab5657465d8", + "CreateKb": "c7aaf2c482ac2ea3f30c4fcef2a472cc4f81124c22ed04682db39d5757aad0fe", + "CreateKbUrlResource": "4a55e4277cc73a76446d786c67f55894d89db86b92daace19c24fe5a8454ed11", "CreateLiveQuiz": "10d70644ccd6847f631be478034645236180e403f08438bee48335a31ebb880d", "CreateLiveQuizFromTemplate": "0fc76ab44ebdcd415ec682fa8fcb091ba55a53f175dd1eb5fa91de1a42d04a3f", "CreateMicroLearning": "e0f7a48a676827a99e1e86755464b1b50fb605138443db3bbae558ba37c58432", @@ -55,6 +59,9 @@ "DeleteFeedback": "45869881647aae7917737a0e65a2ba4da8de86f7ec4a2efd8c40ba8b2549e861", "DeleteFeedbackResponse": "70bfe7302dd66e4ce19af00f56bfa1c5088c7cfa9d393853164aaeb105465f50", "DeleteGroupActivity": "175787b750442e9549cad8454fda5f7dd41d9009023d80784da7aa6a0595eb68", + "DeleteKb": "0c53b53e0d71110d9984e282b282ae336f2a1e277348d4b3f08600fecb7ec4d9", + "DeleteKbResource": "c5377a0eedfe30fc3572eb9216a2923831a6c06900d7d26c02b7f4f52577a776", + "DeleteKbResources": "6b905335790fe0d94fe7b7176f15226ea625d85efbb6d0cb8b9f9f4beff0b7ad", "DeleteLiveQuiz": "5e8815e6b407fbce16f3eef8ef7250712d8e63ef7d6c464b3eae67327276271f", "DeleteMicroLearning": "21de9b40a085a42b0d47e8d0d6fa3453826dcfe297fc4ad0bbb75970d63baa5c", "DeleteParticipantAccount": "ca30e981ed46d11a3f0ec826b4fa333c5621a5cf4bf512496dadbc1f39967377", @@ -64,6 +71,7 @@ "DeleteUserGroup": "299ab1a532934b2deeb4d981fd173d516ec4430f8de399382c6ac97489623d0d", "DeleteUserLogin": "0439c352372a7b374ab58927cc48b8af8af84c844883b6edd37200c744405ad8", "DemoteGroupAdminToMember": "3a17c4705340f7870eb20b89389195ecf54e1f9c3dd521fdc73e205479e906ac", + "DetachKbFromChatbot": "48f52a46d35505e7185d5ec316407bafed4580cb2d19a95893a6bb40ceed0895", "DuplicateAnswerCollection": "de62ef965f4a0fa1ef599857d01aadbb9a1f6e720a343ba78c71b6d2bb6e8658", "EditActivityTemplate": "2f072fcfa325b8708317f2569765f0d105f2ef7ad49dacb391536d968e14d694", "EditAnswerCollectionEntry": "4f390a652c7dd8d364f515f434c9b58041d42b044eb933f794491f8a67096783", @@ -86,6 +94,7 @@ "GradeGroupActivitySubmission": "cb88315cf071bceb0cbc7d273f9fb9729cc942b654975dea9a0e33e7de52ac74", "GrantPrivatePreviewAccess": "95ea9c30e3c216c79c6dfe1a1b3922dfb7b287163214de7151285c7e468b20c0", "ImportCatalogObject": "7d3fccf0d29fe332e9f6974813e3d9fef22d1256b1e738ce795472c59decd9cf", + "IngestKbResource": "8db924e84e99b1a9dbee6a43ee91d9cc094e6b152f0356554e8b412ea67601f5", "MIssueCredential": "338cc255849c2e7e41fa13143afe898c710f3d57ebe2ce0999c62286da8075c8", "JoinCourseLeaderboard": "b52232bc4e0d09fef7c90374cb8823ef83acaed6f6c4bb28fb26cd6529923da9", "JoinCourseWithPin": "2aa04f62eccae935f5c93441912c9d13f84e1ea4fe66e68ec9b92235f169e1a5", @@ -120,6 +129,7 @@ "PublishMicroLearning": "65d10feac0f07d748c97dcaf5fcef53d7ebfb7f135fb9e50fc8eed66008cc55d", "PublishPracticeQuiz": "5082375d4e630f761e826156388eb00a294894d20e49bad8982a123c6752e5e4", "RateElement": "14c3ff2b1b5d4691c220231c63eb2f22fdf937e12dd0f67faef86d0f0ce9231d", + "RebuildKbKnowledgeGraph": "34b05c4bc0dff43fd56968c949bd4491a86dee819e1c1afdeaec2fc2d3ebc1f7", "RemoveCatalogObjectAssignment": "7605d19bd677854dcdf99d81cdaa7aeb09d6535b68726495a1e891421162a542", "RemoveObject": "ec1420ad392f8a95d277868c8d9ea0e90fa7349073f068fbdbdfea35ce9c676d", "RemoveUserFromGroup": "dd396efd34357dfa2a6a8edaa3b020cd9aa752d411f4ff08e42a1c40d86b75d1", @@ -127,6 +137,7 @@ "RequestCatalogCollection": "776a25a044dd45e3d8d72644e69219261a364308cecdddf956b06c77fab6b088", "RequestCatalogObject": "6ae8106fe3812691ea6ea239b5fd07b4b3f23a57e77c5ff96245f94cec2ea939", "MRequestCatalystAccess": "2cf3899fca7c379bb6d6383fe9a7b27059da2f59d00b54362660900336735fc0", + "RequestKbFileUpload": "8d75941b78eaf3c3b8132a93702f64b2dd24375cfe3ad3d4b02588379eb0452a", "ResetAssessmentLiveQuiz": "6732cd3543debba9429eda45f35fde15da049927509061a0790f774b447bd744", "ResolveActivityLogEntry": "b0d21de9f8e363fccafa70e7536ee35762f95bb08e97f20a696c0c48d0726312", "ResolveFeedback": "530545f6a9138062b848c5bb3a86e5c081d5db54fb6e60d0e110be2e00b80246", @@ -139,6 +150,7 @@ "SetActivityReviewStatus": "83985b6a0eae86d4346adbcce03ceedaae4c493cca39fea66cd403c1a827586a", "SetAiFeatures": "0eb6653fe8019011196e53ecdcf33c583837be0f0ba18985d292c1b9d2b2ab2b", "SetBetaFeatures": "a4752cc543598f97a831d3c037eddb557458f76947e08d5080ea8e603697e746", + "SetKbKnowledgeGraphEnabled": "192ac7e5f6324747b017a8cc202780738b5f68158cb236968d00f885370038a8", "SetLiveQuizPin": "aa3336fbe64a06f64d6d664db179ddc22cb32dab78536cad2b3c5ad34cb2221a", "ShareElementsBatch": "cd9a8481fde02ab6f7e8d4b412f24fedcd25fdb7b05174bbdfb7d6b7cf0add40", "ShareObject": "4ed36d7d73dd1f400325c9d06b8e856e9755feeaa99f29f11dd31f4b34a75a6d", @@ -196,7 +208,7 @@ "GetCatalogSharingRequests": "e59ead934b55cdd40cb94eec86cb94f1ae12490c053041704c6313555c091873", "QGetCatalystRequestAccess": "184baf28176c758da182405c0991c68bdaea7f83d2a3845fc94a5b9bdca64857", "GetChatModelRegistry": "afd5a7321e3f64c722185da6d71e6f574c09be439d076f3db530628515833fc5", - "GetChatbotsInfo": "856e57099c46397537583d9276de8d3466efe118a34a9923daa58cf379f05272", + "GetChatbotsInfo": "d690b406938918d1d111f4cf0fc7035508999b01bc87da18df299015a322242c", "GetCockpitQuiz": "26315050abef8a4f7daf07d2c424b31fbc05e99672bfa2c8f612b07c740bc5f8", "GetControlCourse": "e75fbff1c5cb7ff5e8ba8dea58c1cf225065c420b007433ad34477dd8c75c42b", "GetControlCourses": "f62c7d2cb59eb42a077e6d527c90f4838a2888cb9a9f60259979114b6a3b759f", @@ -231,6 +243,13 @@ "GetGroupActivityInstances": "c720e1970bcd63a7bce7c80754a5872c8389b1ba35e693e9914f6dd014ae530f", "GetGroupActivitySummary": "7a3935db0e7cae8d5baf574a1c0506f568ffaaae1522a49363245946cff64d2d", "GetInstanceUpdateActivities": "605ab5b696bdbe5c9d72bf5bb8228f6e5a24c5e779988908ef1ca6bac6028ff8", + "GetKb": "4fee195e47274aee88fe26597a1c6540ddaedb2c59259f911187a80eb8ff1760", + "GetKbChatbotBindings": "6f0a601d3ae538a91a4278f49f20a0c60d8b6fcc4169df1da444eaad13ddfd10", + "GetKbKnowledgeGraphConfig": "05f7588b286c01a72323d43fd54270d2aa8a60ee9f141fc88161f0eebf2c634e", + "GetKbKnowledgeGraphNeighbors": "6f29eda874b93652adf027082ed8fc7e351672d110619b874aa14fc5a69a62df", + "GetKbKnowledgeGraphOverview": "9384ade70aa981400b86fb28e9c721ef8fa509f753079cbb012c65fbe9e67cbb", + "GetKbResourceIngestionRuns": "e3d7a97009a79db9c928a6d2ef9f31ec628680f3d065baf1b0928cf193fb8db1", + "GetKbResources": "73d12da4a5f3056ad076465238b9d45498fb8b1f7118d6fef01400aa6f90c3d6", "GetLecturerViewLiveQuiz": "d1ab35eb40e5193d9ef4a4563d1057e860db1b9774d82076ccc1c380350139ff", "GetLiveQuizEmbeddingInfo": "1e69b5994b1bc2af7504ba701ce4f26cc5a0e5aa8b784c3045cee75e306b4f92", "GetLiveQuizEvaluation": "fa751a965ef3afe5dcc106c185df4c0aacf3e7c9614512d2e0825d6aec8ecbca", @@ -276,6 +295,7 @@ "GetUserCourses": "0d9a8165dc0c4bb67d178b858d7d23bc0328362194dc978530e0effe13bb76bd", "GetUserElements": "cea84675a3e93cae52225b24d240c8eef9986c516dedc156f4aa0a5def0e95ff", "GetUserGroupsUser": "c720d3ff1338c6a63f19f848126b5c30662b08d087d0250113db7549d80725bc", + "GetUserKbs": "d3794d64c91ea212794d42c52ca329960f8e2a53c9bb707739e06b46b434acfb", "GetUserLogins": "7b289235e4ea8e3fb2073ae175b1b900a77cb6c1aae9b12580cc08be825ad1aa", "GetUserMediaFiles": "af5513e8c56c1805060f0e2854a4ada90c4484d09481f2ba2a59136cb731cd63", "GetUserRunningLiveQuizzes": "756ada2de676dfd25fd4f9dd38e1255725a557ad945b379ea7e0a191df9135d0", @@ -285,6 +305,7 @@ "QGetVerifiableCredential": "611adf4b460b125a33a9296f774e33a19bcd0299e176a1e0cc30216dfb83bc88", "GroupActivityDetails": "ba512e4efff648ed70522f11905d073e2c89411794adc80a43d65301f905c649", "Participations": "c6f065a54c438a4da98a268e962e6c54850a1d599c908c291b77c695db75b7ef", + "SearchKbKnowledgeGraph": "e0e7bdb56ca9a878a795208ddd9290c51338e1ecb50029e20e275ecaed55b09f", "Self": "9c4e2c9e5469cf452a1d5d4b814f22ba45ff56571946ab3ff59d403cb4a8aa9c", "SelfWithAchievements": "07ed90975e46d0b5a00efb45e844030ae6bf16653e39c53ecec5afddc72c62f2", "UserProfile": "cb87e067f0165f623e1d917daad40b7d3f19cc2254ac6bfde9ab75997dd59061", diff --git a/packages/graphql/src/public/schema.graphql b/packages/graphql/src/public/schema.graphql index 2818ef7d68..00e49b5327 100644 --- a/packages/graphql/src/public/schema.graphql +++ b/packages/graphql/src/public/schema.graphql @@ -727,6 +727,7 @@ type Chatbot { creditResetPeriod: CreditResetPeriod! description: String disclaimerSummary: ChatbotDisclaimerSummary + enabledKnowledgeBase: ChatbotKnowledgeBaseSummary id: ID! mcpConfigurations: [ChatbotMcpConfigurationSummary!]! modelSelection: Boolean! @@ -744,6 +745,11 @@ type ChatbotDisclaimerSummary { title: String! } +type ChatbotKnowledgeBaseSummary { + id: ID! + name: String! +} + type ChatbotMcpConfigurationSummary { allowedToolsCount: Int chatMode: String! @@ -1772,6 +1778,206 @@ The `JSON` scalar type represents JSON values as specified by [ECMA-404](http:// """ scalar Json +type KB { + createdAt: Date! + description: String + id: ID! + metrics: KBMetrics + name: String! + updatedAt: Date! +} + +type KBChatbotBinding { + chatbotId: ID! + chatbotName: String! + enabledKbId: ID + enabledKbName: String +} + +type KBConnection { + items: [KB!]! + pageInfo: KBPageInfo! + totalCount: Int! +} + +type KBFileUpload { + blobName: String! + containerName: String! + uploadSasURL: String! +} + +enum KBGraphBuildStatus { + FAILED + PROCESSING + QUEUED + SUCCEEDED + SUPERSEDED +} + +enum KBGraphCostStatus { + NEEDS_HUMAN_REVIEW + RELEASED + RESERVED + SETTLED +} + +enum KBGraphQualityTier { + HIGH + STANDARD +} + +type KBIngestionRun { + contentSha256: String + createdAt: Date! + errorCode: String + finishedAt: Date + id: ID! + resourceVersion: Int! + startedAt: Date + status: KBIngestionStatus! + statusMessage: String + updatedAt: Date! +} + +enum KBIngestionStatus { + FAILED + PROCESSING + QUEUED + SUCCEEDED + SUPERSEDED +} + +type KBKnowledgeGraphConfig { + activeBuildId: ID + actualCostMinorUnits: Int + actualEmbeddingTokens: Int + actualInputTokens: Int + actualOutputTokens: Int + actualRequestCount: Int + billingLabel: String + buildId: ID + costConfigurationReady: Boolean! + costCurrency: String + costStatus: KBGraphCostStatus + createdAt: Date + estimatedCostMinorUnits: Int + finishedAt: Date + highEstimateMinorUnits: Int + isEnabled: Boolean! + isStale: Boolean! + kbId: ID! + maxCostMinorUnits: Int + publishedBuildId: ID + qualityTier: KBGraphQualityTier + quotaCurrency: String + remainingSemesterQuotaMinorUnits: Int + semesterKey: String + semesterQuotaMinorUnits: Int + semesterReservedMinorUnits: Int + semesterSettledMinorUnits: Int + sourceContentDigest: String + standardEstimateMinorUnits: Int + startedAt: Date + status: KBGraphBuildStatus + statusMessage: String + updatedAt: Date + worstCaseRemainingMinorUnits: Int +} + +type KBMetrics { + linkedConsumerCount: Int! + pendingCleanupCount: Int! + pendingCleanupSizeBytes: Int! + quotaResourceCount: Int! + quotaSizeBytes: Int! + reservedResourceCount: Int! + reservedSizeBytes: Int! + resourceLimit: Int! + storageLimitBytes: Int! + unknownSizeResourceCount: Int! + visibleResourceCount: Int! + visibleSizeBytes: Int! +} + +type KBPageInfo { + endCursor: String + hasNextPage: Boolean! +} + +type KBResource { + activeContentSha256: String + activeResourceVersion: Int + createdAt: Date! + errorCode: String + id: ID! + ingestedAt: Date + latestIngestionRun: KBIngestionRun + mimeType: String + originalFilename: String + resourceVersion: Int! + sizeBytes: Int + sourceUrl: String + status: KBResourceStatus! + statusMessage: String + title: String! + type: KBResourceType! + updatedAt: Date! +} + +type KBResourceConnection { + items: [KBResource!]! + pageInfo: KBPageInfo! + totalCount: Int! +} + +enum KBResourceStatus { + ADDED + FAILED + PROCESSING + QUEUED + READY +} + +enum KBResourceType { + BLOB + URL +} + +type KnowledgeGraphEdge { + id: ID! + label: String! + properties: Json! + source: ID! + target: ID! + type: String! +} + +type KnowledgeGraphNode { + content: String + degree: Int! + displayLabel: String! + id: ID! + kind: String! + labels: [String!]! + sourceReferences: [KnowledgeGraphSourceReference!]! + summary: String +} + +type KnowledgeGraphResponse { + buildId: ID! + edges: [KnowledgeGraphEdge!]! + isStale: Boolean! + kbId: ID! + nodes: [KnowledgeGraphNode!]! + truncated: Boolean! +} + +type KnowledgeGraphSourceReference { + reference: String + resourceId: ID! + title: String! +} + type LeaderboardEntry { avatar: String email: String @@ -1934,6 +2140,7 @@ type Mutation { applyActivityBatchOperations(activityIds: [String!]!, basePoints: Int, bonusPoints: Int, correctnessPoints: Int, courseId: String, multiplier: Int, timeToZeroBonus: Int): Int! applyElementBatchOperations(archive: Boolean!, basePoints: Boolean, elementIds: [Int!]!, multiplier: Int, status: ElementStatus, unarchive: Boolean!, updateInstances: Boolean!, updateTemplateInstances: Boolean!): Int! approveObjectSharingRequest(permissionLevel: PermissionLevel!, propagation: Boolean!, requestId: Int!, userId: String!): Boolean! + attachKbToChatbot(chatbotId: ID!, kbId: ID!): KBChatbotBinding! bookmarkElementStack(bookmarked: Boolean!, courseId: String!, stackId: Int!): [Int!] cancelLiveQuiz(id: String!): LiveQuiz cancelObjectSharingRequest(objectId: String!, objectType: ObjectType!): Boolean! @@ -1950,6 +2157,7 @@ type Mutation { changeShortname(shortname: String!): User changeUserGroupName(id: Int!, name: String!): Boolean! changeUserLocale(locale: LocaleType!): User + confirmKbFileUpload(blobName: String!, kbId: ID!, mimeType: String!, originalFilename: String!, sizeBytes: Int!, title: String!): KBResource! copyCatalogObjectToAccount(catalogCollectionId: String, objectId: String!, objectType: ObjectType!): Boolean! correctAssessmentPointsInstance(awardBasePoints: Boolean, awardBonusPoints: Boolean, awardCorrectnessPoints: Boolean, deductBasePoints: Boolean, deductBonusPoints: Boolean, deductCorrectnessPoints: Boolean, instanceId: Int!, participantId: String, participantIds: [String!], reason: String!, scope: PointCorrectionType!, studentReason: String!): PointCorrection correctAssessmentPointsLiveQuiz(awardBasePoints: Boolean, awardBonusPoints: Boolean, awardCorrectnessPoints: Boolean, deductBasePoints: Boolean, deductBonusPoints: Boolean, deductCorrectnessPoints: Boolean, liveQuizId: String!, participantId: String, participantIds: [String!], reason: String!, scope: PointCorrectionType!, studentReason: String!): PointCorrection @@ -1960,6 +2168,8 @@ type Mutation { createCourse(color: String, description: String, displayName: String!, duplicateGroupActivities: Boolean, duplicateLiveQuizzes: Boolean, duplicateMicrolearnings: Boolean, duplicatePracticeQuizzes: Boolean, endDate: Date!, groupDeadlineDate: Date!, isGamificationEnabled: Boolean!, isGroupCreationEnabled: Boolean!, language: LocaleType!, maxGroupSize: Int!, name: String!, notificationEmail: String, preferredGroupSize: Int!, sourceCourseId: String, startDate: Date!): Course createFeedback(content: String!, quizId: String!): Feedback createGroupActivity(clues: [GroupActivityClueInput!]!, courseId: String!, description: String, displayName: String!, endDate: Date!, multiplier: Int!, name: String!, stack: ElementStackInput!, startDate: Date!): ActivityInfo + createKb(description: String, name: String!): KB! + createKbUrlResource(kbId: ID!, title: String!, url: String!): KBResource! createLiveQuiz(blocks: [ElementBlockInput!]!, courseId: String, defaultCorrectPoints: Int, defaultPoints: Int, description: String, displayName: String!, isConfusionFeedbackEnabled: Boolean!, isGamificationEnabled: Boolean!, isLiveQAEnabled: Boolean!, isModerationEnabled: Boolean!, isPinProtected: Boolean!, maxBonusPoints: Int, multiplier: Int!, name: String!, timeToZeroBonus: Int): ActivityInfo createLiveQuizFromTemplate(blocks: [TemplateBlockInput!]!, courseId: String, description: String, displayName: String!, isGamificationEnabled: Boolean!, name: String!, templateId: String!): String createMicroLearning(courseId: String!, description: String, displayName: String!, endDate: Date!, multiplier: Int!, name: String!, stacks: [ElementStackInput!]!, startDate: Date!): ActivityInfo @@ -1980,6 +2190,9 @@ type Mutation { deleteFeedback(id: Int!, liveQuizId: String!): Feedback deleteFeedbackResponse(id: Int!, liveQuizId: String!): Feedback deleteGroupActivity(id: String!): GroupActivity + deleteKb(id: ID!): KB! + deleteKbResource(id: ID!): KBResource! + deleteKbResources(ids: [String!]!, kbId: ID!): [KBResource!]! deleteLiveQuiz(id: String!): LiveQuiz deleteMicroLearning(id: String!): MicroLearning deleteParticipantAccount: Boolean @@ -1989,6 +2202,7 @@ type Mutation { deleteUserGroup(groupId: Int!): Boolean! deleteUserLogin(id: String!): UserLogin demoteGroupAdminToMember(adminId: String!, groupId: Int!): Boolean! + detachKbFromChatbot(chatbotId: ID!, kbId: ID!): Boolean! duplicateAnswerCollection(id: Int!): AnswerCollection editActivityTemplate(activityId: String!, activityType: ActivityType!, description: String!, instructions: String!, name: String!, templateId: String!): Boolean! editAnswerCollectionEntry(collectionId: Int!, id: Int!, value: String!): AnswerCollectionEntry @@ -2011,6 +2225,7 @@ type Mutation { gradeGroupActivitySubmission(gradingDecisions: GroupActivityGradingInput!, groupActivityId: String!, id: Int!): GroupActivityInstance grantPrivatePreviewAccess(email: String!): Int importCatalogObject(catalogCollectionId: String, objectId: String!, objectType: ObjectType!): Boolean! + ingestKbResource(id: ID!): KBResource! issueAssessmentReport(courseId: String!): IssuedAssessmentReport! joinCourseLeaderboard(courseId: String!): ParticipantLearningData joinCourseWithPin(pin: Int!): Participant @@ -2045,6 +2260,7 @@ type Mutation { publishMicroLearning(id: String!): MicroLearning publishPracticeQuiz(availableFrom: Date, id: String!): PracticeQuiz rateElement(elementId: Int!, elementInstanceId: Int!, rating: Int!): ElementFeedback + rebuildKbKnowledgeGraph(kbId: ID!, qualityTier: KBGraphQualityTier): KBKnowledgeGraphConfig! removeCatalogObjectAssignment(assignmentId: Int!): Boolean! removeObject(objectId: String!, objectType: ObjectType!): String removeUserFromGroup(groupId: Int!, userId: String!): Boolean! @@ -2052,6 +2268,7 @@ type Mutation { requestCatalogCollection(catalogCollectionId: String!, requestedPermissionLevel: PermissionLevel): CatalogCollection requestCatalogObject(catalogCollectionId: String, objectId: String!, objectType: ObjectType!, requestedPermissionLevel: PermissionLevel): Boolean! requestCatalystAccess(institution: String!, useCase: String!): Boolean! + requestKbFileUpload(contentType: String!, fileName: String!, kbId: ID!, sizeBytes: Int!): KBFileUpload! resetAssessmentLiveQuiz(id: String!): ActivityInfo resolveActivityLogEntry(id: Int!): ActivityLogEntry resolveFeedback(id: Int!, isResolved: Boolean!, liveQuizId: String!): Feedback @@ -2064,6 +2281,7 @@ type Mutation { setActivityReviewStatus(activityId: String!, activityType: ActivityType!, isReviewed: Boolean!): ReviewStatus setAiFeatures(email: String!, enabled: Boolean!): Int setBetaFeatures(enabled: Boolean!): Boolean + setKbKnowledgeGraphEnabled(enabled: Boolean!, kbId: ID!): KBKnowledgeGraphConfig! setLiveQuizPin(liveQuizId: String!, pin: String!): Boolean! shareElementsBatch(elementIds: [Int!]!, permissionLevel: PermissionLevel!, shortnameOrEmail: String, userGroupId: Int): ElementBatchSharingResult! shareObject(objectId: String!, objectType: ObjectType!, permissionLevel: PermissionLevel!, propagation: Boolean!, shortnameOrEmail: String, userGroupId: Int): PermissionInfo @@ -2582,6 +2800,13 @@ type Query { getGradingGroupActivity(id: String!): GroupActivity getGroupActivitySummary(id: String!): GroupActivitySummary getInstanceUpdateActivities(elementId: Int!, hasSampleSolution: Boolean, includeTemplateInstances: Boolean!): [InstanceUpdateActivityInfo!] + getKb(id: ID!): KB! + getKbChatbotBindings(kbId: ID!): [KBChatbotBinding!]! + getKbKnowledgeGraphConfig(kbId: ID!): KBKnowledgeGraphConfig! + getKbKnowledgeGraphNeighbors(kbId: ID!, nodeId: ID!): KnowledgeGraphResponse! + getKbKnowledgeGraphOverview(kbId: ID!): KnowledgeGraphResponse! + getKbResourceIngestionRuns(resourceId: ID!): [KBIngestionRun!]! + getKbResources(after: String, first: Int, kbId: ID!, search: String, status: KBIngestionStatus, type: KBResourceType): KBResourceConnection! getLecturerViewLiveQuiz(id: String!): LiveQuiz getLiveQuizEmbeddingInfo(id: String!): LiveQuizEmbeddingInfo getLiveQuizSummary(quizId: String!): LiveQuizSummary @@ -2607,6 +2832,7 @@ type Query { getTemplatePreviewAnswerCollectionEntries(answerCollectionId: Int!, templateId: String!): [AnswerCollectionPreviewEntry!] getUserActivitiesCourses: [CourseListEntry!] getUserGroupsUser: [UserGroup!] + getUserKbsConnection(after: String, first: Int, search: String): KBConnection! getUsersAiFeatures: [UserInfo!] getUsersPrivatePreview: [UserInfo!] groupActivities(courseId: String!): [GroupActivity!] @@ -2624,6 +2850,7 @@ type Query { practiceQuiz(id: String!): PracticeQuiz previousPointCorrections(courseId: String, instanceId: Int, liveQuizId: String): [PointCorrection!] publicParticipantProfile(participantId: String!): Participant + searchKbKnowledgeGraph(kbId: ID!, query: String!): KnowledgeGraphResponse! self(liveQuizId: String): Participant selfWithAchievements: ParticipantWithAchievements shortnameQuizzes(shortname: String!): [LiveQuiz!] diff --git a/packages/graphql/src/public/server.json b/packages/graphql/src/public/server.json index 78fdc07748..d6ad73d9c4 100644 --- a/packages/graphql/src/public/server.json +++ b/packages/graphql/src/public/server.json @@ -10,6 +10,7 @@ "44fd2142f1dc40f055ab0195cb92ea30d6168aa84b97afade3b6d6522795c7bc": "mutation ApplyActivityBatchOperations($activityIds: [String!]!, $multiplier: Int, $courseId: String, $basePoints: Int, $correctnessPoints: Int, $bonusPoints: Int, $timeToZeroBonus: Int) {\n applyActivityBatchOperations(\n activityIds: $activityIds\n multiplier: $multiplier\n courseId: $courseId\n basePoints: $basePoints\n correctnessPoints: $correctnessPoints\n bonusPoints: $bonusPoints\n timeToZeroBonus: $timeToZeroBonus\n )\n}", "b5069c86bd44976aee11e3b1972adc067d557a4639cbeba00fd8ca4b272dd42e": "mutation ApplyElementBatchOperations($elementIds: [Int!]!, $archive: Boolean!, $unarchive: Boolean!, $status: ElementStatus, $multiplier: Int, $basePoints: Boolean, $updateInstances: Boolean!, $updateTemplateInstances: Boolean!) {\n applyElementBatchOperations(\n elementIds: $elementIds\n archive: $archive\n unarchive: $unarchive\n status: $status\n multiplier: $multiplier\n basePoints: $basePoints\n updateInstances: $updateInstances\n updateTemplateInstances: $updateTemplateInstances\n )\n}", "801c3fe541540847d366e631e3f8ebe0890c05d0bab2228f9a6ae7060fc71850": "mutation ApproveObjectSharingRequest($requestId: Int!, $userId: String!, $permissionLevel: PermissionLevel!, $propagation: Boolean!) {\n approveObjectSharingRequest(\n requestId: $requestId\n userId: $userId\n permissionLevel: $permissionLevel\n propagation: $propagation\n )\n}", + "88a76e615654d4ab8d5b7b3b1038ba2903cad03c034240b94f03f5c8c08c1432": "mutation AttachKbToChatbot($kbId: ID!, $chatbotId: ID!) {\n attachKbToChatbot(kbId: $kbId, chatbotId: $chatbotId) {\n chatbotId\n chatbotName\n enabledKbId\n enabledKbName\n __typename\n }\n}", "1c54224ef0af1eb882af6538f46b4ac40c1415062075092e9f85396c4646b136": "mutation BookmarkElementStack($stackId: Int!, $courseId: String!, $bookmarked: Boolean!) {\n bookmarkElementStack(\n stackId: $stackId\n courseId: $courseId\n bookmarked: $bookmarked\n )\n}", "a270a5ab4a8e102679be3d43db8de35cc35d5650f42c0d02aa40548edadee6d4": "mutation CancelLiveQuiz($id: String!) {\n cancelLiveQuiz(id: $id) {\n id\n __typename\n }\n}", "1e8aaef9bda5428b625946891e39dabe1a24ad11dcbbc9eb9ca020daf46dc9e7": "mutation CancelObjectSharingRequest($objectId: String!, $objectType: ObjectType!) {\n cancelObjectSharingRequest(objectId: $objectId, objectType: $objectType)\n}", @@ -25,6 +26,7 @@ "07ed8db96ea01e17e68f1b598ba830a22199d44248cc473072fe2428491c4656": "mutation ChangeShortname($shortname: String!) {\n changeShortname(shortname: $shortname) {\n id\n shortname\n __typename\n }\n}", "a8efa0083d3cff34731ce21a28536512bccb280742dfa25a3fbde448c376168d": "mutation ChangeUserGroupName($id: Int!, $name: String!) {\n changeUserGroupName(id: $id, name: $name)\n}", "0a5f88b275c2319ba6c569426183657cdb7962b17651bf16a738f24766affd67": "mutation ChangeUserLocale($locale: LocaleType!) {\n changeUserLocale(locale: $locale) {\n id\n locale\n __typename\n }\n}", + "cb7ef08d5a8ddc0d653531bba5864a77a7702c6ad854305c4459c85592b22ac1": "mutation ConfirmKbFileUpload($kbId: ID!, $blobName: String!, $title: String!, $originalFilename: String!, $mimeType: String!, $sizeBytes: Int!) {\n confirmKbFileUpload(\n kbId: $kbId\n blobName: $blobName\n title: $title\n originalFilename: $originalFilename\n mimeType: $mimeType\n sizeBytes: $sizeBytes\n ) {\n id\n __typename\n }\n}", "9c048459c0e03342d23e900d7b380a12fcad17d99c6270455a519ee3e109d996": "mutation CopyCatalogObjectToAccount($objectId: String!, $objectType: ObjectType!, $catalogCollectionId: String) {\n copyCatalogObjectToAccount(\n objectId: $objectId\n objectType: $objectType\n catalogCollectionId: $catalogCollectionId\n )\n}", "08518a9a14356bd169e5bafedb59e85b5472b59c67cc64cda68404e60339ad30": "fragment PointCorrectionData on PointCorrection {\n id\n type\n basePoints\n correctnessPoints\n bonusPoints\n reason\n studentReason\n createdAt\n correctedBy {\n id\n shortname\n __typename\n }\n participant {\n id\n username\n email\n __typename\n }\n participants {\n id\n username\n email\n __typename\n }\n liveQuiz {\n id\n name\n __typename\n }\n instance {\n id\n elementData {\n ... on ChoicesElementData {\n id\n name\n __typename\n }\n ... on NumericalElementData {\n id\n name\n __typename\n }\n ... on FreeTextElementData {\n id\n name\n __typename\n }\n ... on SelectionElementData {\n id\n name\n __typename\n }\n ... on CaseStudyElementData {\n id\n name\n __typename\n }\n ... on FlashcardElementData {\n id\n name\n __typename\n }\n ... on ContentElementData {\n id\n name\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n}\nmutation CorrectAssessmentPointsInstance($instanceId: Int!, $awardBasePoints: Boolean, $awardCorrectnessPoints: Boolean, $awardBonusPoints: Boolean, $deductBasePoints: Boolean, $deductCorrectnessPoints: Boolean, $deductBonusPoints: Boolean, $reason: String!, $studentReason: String!, $scope: PointCorrectionType!, $participantId: String, $participantIds: [String!]!) {\n correctAssessmentPointsInstance(\n instanceId: $instanceId\n awardBasePoints: $awardBasePoints\n awardCorrectnessPoints: $awardCorrectnessPoints\n awardBonusPoints: $awardBonusPoints\n deductBasePoints: $deductBasePoints\n deductCorrectnessPoints: $deductCorrectnessPoints\n deductBonusPoints: $deductBonusPoints\n reason: $reason\n studentReason: $studentReason\n scope: $scope\n participantId: $participantId\n participantIds: $participantIds\n ) {\n ...PointCorrectionData\n __typename\n }\n}", "b5a0b0f4d4fbc5cbe135a1f264292156e0bf634ac08204bcb249dba7492525a3": "fragment PointCorrectionData on PointCorrection {\n id\n type\n basePoints\n correctnessPoints\n bonusPoints\n reason\n studentReason\n createdAt\n correctedBy {\n id\n shortname\n __typename\n }\n participant {\n id\n username\n email\n __typename\n }\n participants {\n id\n username\n email\n __typename\n }\n liveQuiz {\n id\n name\n __typename\n }\n instance {\n id\n elementData {\n ... on ChoicesElementData {\n id\n name\n __typename\n }\n ... on NumericalElementData {\n id\n name\n __typename\n }\n ... on FreeTextElementData {\n id\n name\n __typename\n }\n ... on SelectionElementData {\n id\n name\n __typename\n }\n ... on CaseStudyElementData {\n id\n name\n __typename\n }\n ... on FlashcardElementData {\n id\n name\n __typename\n }\n ... on ContentElementData {\n id\n name\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n}\nmutation CorrectAssessmentPointsLiveQuiz($liveQuizId: String!, $awardBasePoints: Boolean, $awardCorrectnessPoints: Boolean, $awardBonusPoints: Boolean, $deductBasePoints: Boolean, $deductCorrectnessPoints: Boolean, $deductBonusPoints: Boolean, $reason: String!, $studentReason: String!, $scope: PointCorrectionType!, $participantId: String, $participantIds: [String!]!) {\n correctAssessmentPointsLiveQuiz(\n liveQuizId: $liveQuizId\n awardBasePoints: $awardBasePoints\n awardCorrectnessPoints: $awardCorrectnessPoints\n awardBonusPoints: $awardBonusPoints\n deductBasePoints: $deductBasePoints\n deductCorrectnessPoints: $deductCorrectnessPoints\n deductBonusPoints: $deductBonusPoints\n reason: $reason\n studentReason: $studentReason\n scope: $scope\n participantId: $participantId\n participantIds: $participantIds\n ) {\n ...PointCorrectionData\n __typename\n }\n}", @@ -34,6 +36,8 @@ "0284a39ca95a55fc13d865698a440007e4a95651a4940b062bd9d82689343f82": "mutation CreateCourse($name: String!, $displayName: String!, $description: String, $color: String, $startDate: Date!, $endDate: Date!, $isGroupCreationEnabled: Boolean!, $groupDeadlineDate: Date!, $maxGroupSize: Int!, $preferredGroupSize: Int!, $language: LocaleType!, $notificationEmail: String, $isGamificationEnabled: Boolean!, $sourceCourseId: String, $duplicateLiveQuizzes: Boolean, $duplicatePracticeQuizzes: Boolean, $duplicateMicrolearnings: Boolean, $duplicateGroupActivities: Boolean) {\n createCourse(\n name: $name\n displayName: $displayName\n description: $description\n color: $color\n startDate: $startDate\n endDate: $endDate\n isGroupCreationEnabled: $isGroupCreationEnabled\n groupDeadlineDate: $groupDeadlineDate\n maxGroupSize: $maxGroupSize\n preferredGroupSize: $preferredGroupSize\n language: $language\n notificationEmail: $notificationEmail\n isGamificationEnabled: $isGamificationEnabled\n sourceCourseId: $sourceCourseId\n duplicateLiveQuizzes: $duplicateLiveQuizzes\n duplicatePracticeQuizzes: $duplicatePracticeQuizzes\n duplicateMicrolearnings: $duplicateMicrolearnings\n duplicateGroupActivities: $duplicateGroupActivities\n ) {\n id\n name\n displayName\n description\n color\n startDate\n endDate\n groupDeadlineDate\n maxGroupSize\n preferredGroupSize\n language\n notificationEmail\n isArchived\n isGamificationEnabled\n isAssessmentEnabled\n isGroupCreationEnabled\n randomAssignmentFinalized\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isShared\n isRemovable\n __typename\n }\n}", "5a4a9e710048cb5b679e7d8217907d623c7c53638fec41183a0257931fc8b136": "mutation CreateFeedback($quizId: String!, $content: String!) {\n createFeedback(quizId: $quizId, content: $content) {\n id\n isPublished\n isPinned\n isResolved\n content\n votes\n __typename\n }\n}", "8a92f323833cb25abffce681c8f87bbd064caae245e8c965b5ebaab5657465d8": "fragment ActivityInfoData on ActivityInfo {\n id\n templateId\n type\n status\n courseId\n courseName\n courseStartDate\n courseLanguage\n numOfStacks\n numOfElements\n reviewStatus\n automaticPublicationAt\n scheduledStartAt\n scheduledEndAt\n groupDeadlineDate\n numOfParticipantGroups\n name\n displayName\n permissionLevel\n derivedAccess\n areInstancesOutdated\n isGamificationEnabled\n isAssessmentEnabled\n pinCode\n numSharedUsers\n isOwner\n isManager\n isEditor\n isExecutor\n isShared\n isRemovable\n isActivityReviewer\n sharingType\n updatedAt\n __typename\n}\nmutation CreateGroupActivity($name: String!, $displayName: String!, $description: String, $courseId: String!, $multiplier: Int!, $startDate: Date!, $endDate: Date!, $clues: [GroupActivityClueInput!]!, $stack: ElementStackInput!) {\n createGroupActivity(\n name: $name\n displayName: $displayName\n description: $description\n courseId: $courseId\n multiplier: $multiplier\n startDate: $startDate\n endDate: $endDate\n clues: $clues\n stack: $stack\n ) {\n ...ActivityInfoData\n __typename\n }\n}", + "c7aaf2c482ac2ea3f30c4fcef2a472cc4f81124c22ed04682db39d5757aad0fe": "mutation CreateKb($name: String!, $description: String) {\n createKb(name: $name, description: $description) {\n id\n __typename\n }\n}", + "4a55e4277cc73a76446d786c67f55894d89db86b92daace19c24fe5a8454ed11": "mutation CreateKbUrlResource($kbId: ID!, $url: String!, $title: String!) {\n createKbUrlResource(kbId: $kbId, url: $url, title: $title) {\n id\n __typename\n }\n}", "10d70644ccd6847f631be478034645236180e403f08438bee48335a31ebb880d": "fragment ActivityInfoData on ActivityInfo {\n id\n templateId\n type\n status\n courseId\n courseName\n courseStartDate\n courseLanguage\n numOfStacks\n numOfElements\n reviewStatus\n automaticPublicationAt\n scheduledStartAt\n scheduledEndAt\n groupDeadlineDate\n numOfParticipantGroups\n name\n displayName\n permissionLevel\n derivedAccess\n areInstancesOutdated\n isGamificationEnabled\n isAssessmentEnabled\n pinCode\n numSharedUsers\n isOwner\n isManager\n isEditor\n isExecutor\n isShared\n isRemovable\n isActivityReviewer\n sharingType\n updatedAt\n __typename\n}\nmutation CreateLiveQuiz($name: String!, $displayName: String!, $description: String, $blocks: [ElementBlockInput!]!, $courseId: String, $multiplier: Int!, $defaultPoints: Int!, $defaultCorrectPoints: Int!, $maxBonusPoints: Int!, $timeToZeroBonus: Int!, $isGamificationEnabled: Boolean!, $isPinProtected: Boolean!, $isConfusionFeedbackEnabled: Boolean!, $isLiveQAEnabled: Boolean!, $isModerationEnabled: Boolean!) {\n createLiveQuiz(\n name: $name\n displayName: $displayName\n description: $description\n blocks: $blocks\n courseId: $courseId\n multiplier: $multiplier\n defaultPoints: $defaultPoints\n defaultCorrectPoints: $defaultCorrectPoints\n maxBonusPoints: $maxBonusPoints\n timeToZeroBonus: $timeToZeroBonus\n isGamificationEnabled: $isGamificationEnabled\n isPinProtected: $isPinProtected\n isConfusionFeedbackEnabled: $isConfusionFeedbackEnabled\n isLiveQAEnabled: $isLiveQAEnabled\n isModerationEnabled: $isModerationEnabled\n ) {\n ...ActivityInfoData\n __typename\n }\n}", "0fc76ab44ebdcd415ec682fa8fcb091ba55a53f175dd1eb5fa91de1a42d04a3f": "mutation CreateLiveQuizFromTemplate($templateId: String!, $name: String!, $displayName: String!, $description: String, $courseId: String, $isGamificationEnabled: Boolean!, $blocks: [TemplateBlockInput!]!) {\n createLiveQuizFromTemplate(\n templateId: $templateId\n name: $name\n displayName: $displayName\n description: $description\n courseId: $courseId\n isGamificationEnabled: $isGamificationEnabled\n blocks: $blocks\n )\n}", "e0f7a48a676827a99e1e86755464b1b50fb605138443db3bbae558ba37c58432": "fragment ActivityInfoData on ActivityInfo {\n id\n templateId\n type\n status\n courseId\n courseName\n courseStartDate\n courseLanguage\n numOfStacks\n numOfElements\n reviewStatus\n automaticPublicationAt\n scheduledStartAt\n scheduledEndAt\n groupDeadlineDate\n numOfParticipantGroups\n name\n displayName\n permissionLevel\n derivedAccess\n areInstancesOutdated\n isGamificationEnabled\n isAssessmentEnabled\n pinCode\n numSharedUsers\n isOwner\n isManager\n isEditor\n isExecutor\n isShared\n isRemovable\n isActivityReviewer\n sharingType\n updatedAt\n __typename\n}\nmutation CreateMicroLearning($name: String!, $displayName: String!, $description: String, $stacks: [ElementStackInput!]!, $courseId: String!, $multiplier: Int!, $startDate: Date!, $endDate: Date!) {\n createMicroLearning(\n name: $name\n displayName: $displayName\n description: $description\n stacks: $stacks\n courseId: $courseId\n multiplier: $multiplier\n startDate: $startDate\n endDate: $endDate\n ) {\n ...ActivityInfoData\n __typename\n }\n}", @@ -55,6 +59,9 @@ "45869881647aae7917737a0e65a2ba4da8de86f7ec4a2efd8c40ba8b2549e861": "mutation DeleteFeedback($id: Int!, $liveQuizId: String!) {\n deleteFeedback(id: $id, liveQuizId: $liveQuizId) {\n id\n __typename\n }\n}", "70bfe7302dd66e4ce19af00f56bfa1c5088c7cfa9d393853164aaeb105465f50": "mutation DeleteFeedbackResponse($id: Int!, $liveQuizId: String!) {\n deleteFeedbackResponse(id: $id, liveQuizId: $liveQuizId) {\n id\n isPublished\n isPinned\n isResolved\n content\n votes\n createdAt\n resolvedAt\n responses {\n id\n content\n positiveReactions\n negativeReactions\n createdAt\n __typename\n }\n __typename\n }\n}", "175787b750442e9549cad8454fda5f7dd41d9009023d80784da7aa6a0595eb68": "mutation DeleteGroupActivity($id: String!) {\n deleteGroupActivity(id: $id) {\n id\n __typename\n }\n}", + "0c53b53e0d71110d9984e282b282ae336f2a1e277348d4b3f08600fecb7ec4d9": "mutation DeleteKb($id: ID!) {\n deleteKb(id: $id) {\n id\n __typename\n }\n}", + "c5377a0eedfe30fc3572eb9216a2923831a6c06900d7d26c02b7f4f52577a776": "mutation DeleteKbResource($id: ID!) {\n deleteKbResource(id: $id) {\n id\n __typename\n }\n}", + "6b905335790fe0d94fe7b7176f15226ea625d85efbb6d0cb8b9f9f4beff0b7ad": "mutation DeleteKbResources($kbId: ID!, $ids: [String!]!) {\n deleteKbResources(kbId: $kbId, ids: $ids) {\n id\n __typename\n }\n}", "5e8815e6b407fbce16f3eef8ef7250712d8e63ef7d6c464b3eae67327276271f": "mutation DeleteLiveQuiz($id: String!) {\n deleteLiveQuiz(id: $id) {\n id\n __typename\n }\n}", "21de9b40a085a42b0d47e8d0d6fa3453826dcfe297fc4ad0bbb75970d63baa5c": "mutation DeleteMicroLearning($id: String!) {\n deleteMicroLearning(id: $id) {\n id\n __typename\n }\n}", "ca30e981ed46d11a3f0ec826b4fa333c5621a5cf4bf512496dadbc1f39967377": "mutation DeleteParticipantAccount {\n deleteParticipantAccount\n}", @@ -64,6 +71,7 @@ "299ab1a532934b2deeb4d981fd173d516ec4430f8de399382c6ac97489623d0d": "mutation DeleteUserGroup($groupId: Int!) {\n deleteUserGroup(groupId: $groupId)\n}", "0439c352372a7b374ab58927cc48b8af8af84c844883b6edd37200c744405ad8": "mutation DeleteUserLogin($id: String!) {\n deleteUserLogin(id: $id) {\n id\n __typename\n }\n}", "3a17c4705340f7870eb20b89389195ecf54e1f9c3dd521fdc73e205479e906ac": "mutation DemoteGroupAdminToMember($groupId: Int!, $adminId: String!) {\n demoteGroupAdminToMember(groupId: $groupId, adminId: $adminId)\n}", + "48f52a46d35505e7185d5ec316407bafed4580cb2d19a95893a6bb40ceed0895": "mutation DetachKbFromChatbot($kbId: ID!, $chatbotId: ID!) {\n detachKbFromChatbot(kbId: $kbId, chatbotId: $chatbotId)\n}", "de62ef965f4a0fa1ef599857d01aadbb9a1f6e720a343ba78c71b6d2bb6e8658": "fragment AnswerCollectionData on AnswerCollection {\n id\n name\n description\n ownerShortname\n numSharedUsers\n numOfEntries\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isDeletable\n isRemovable\n sharingType\n entries {\n id\n value\n __typename\n }\n __typename\n}\nmutation DuplicateAnswerCollection($id: Int!) {\n duplicateAnswerCollection(id: $id) {\n ...AnswerCollectionData\n __typename\n }\n}", "2f072fcfa325b8708317f2569765f0d105f2ef7ad49dacb391536d968e14d694": "mutation EditActivityTemplate($activityId: String!, $activityType: ActivityType!, $templateId: String!, $name: String!, $description: String!, $instructions: String!) {\n editActivityTemplate(\n activityId: $activityId\n activityType: $activityType\n templateId: $templateId\n name: $name\n description: $description\n instructions: $instructions\n )\n}", "4f390a652c7dd8d364f515f434c9b58041d42b044eb933f794491f8a67096783": "mutation EditAnswerCollectionEntry($id: Int!, $value: String!, $collectionId: Int!) {\n editAnswerCollectionEntry(id: $id, value: $value, collectionId: $collectionId) {\n id\n value\n __typename\n }\n}", @@ -86,6 +94,7 @@ "cb88315cf071bceb0cbc7d273f9fb9729cc942b654975dea9a0e33e7de52ac74": "mutation GradeGroupActivitySubmission($id: Int!, $groupActivityId: String!, $gradingDecisions: GroupActivityGradingInput!) {\n gradeGroupActivitySubmission(\n id: $id\n groupActivityId: $groupActivityId\n gradingDecisions: $gradingDecisions\n ) {\n id\n decisionsSubmittedAt\n decisions {\n instanceId\n type\n freeTextResponse\n choicesResponse {\n ix\n selected\n __typename\n }\n numericalResponse\n contentResponse\n selectionResponse\n caseStudyResponse {\n caseId\n itemResponses {\n itemId\n criterionResponses {\n criterionId\n response\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n resultsComputedAt\n results {\n passed\n points\n comment\n grading {\n instanceId\n score\n maxPoints\n feedback\n __typename\n }\n __typename\n }\n __typename\n }\n}", "95ea9c30e3c216c79c6dfe1a1b3922dfb7b287163214de7151285c7e468b20c0": "mutation GrantPrivatePreviewAccess($email: String!) {\n grantPrivatePreviewAccess(email: $email)\n}", "7d3fccf0d29fe332e9f6974813e3d9fef22d1256b1e738ce795472c59decd9cf": "mutation ImportCatalogObject($objectId: String!, $objectType: ObjectType!, $catalogCollectionId: String) {\n importCatalogObject(\n objectId: $objectId\n objectType: $objectType\n catalogCollectionId: $catalogCollectionId\n )\n}", + "8db924e84e99b1a9dbee6a43ee91d9cc094e6b152f0356554e8b412ea67601f5": "mutation IngestKbResource($id: ID!) {\n ingestKbResource(id: $id) {\n id\n status\n __typename\n }\n}", "338cc255849c2e7e41fa13143afe898c710f3d57ebe2ce0999c62286da8075c8": "mutation MIssueCredential($courseId: String!) {\n issueAssessmentReport(courseId: $courseId) {\n token\n status\n issuedAt\n snapshot {\n version\n subject {\n email\n givenName\n surname\n matriculationNumber\n source\n __typename\n }\n course {\n id\n name\n displayName\n __typename\n }\n results {\n basePoints\n availableBasePoints\n correctnessPoints\n availableCorrectnessPoints\n bonusPoints\n availableBonusPoints\n totalPoints\n availableTotalPoints\n __typename\n }\n comparison {\n cohortSize\n percentile\n histogram {\n binStart\n binEnd\n count\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}", "b52232bc4e0d09fef7c90374cb8823ef83acaed6f6c4bb28fb26cd6529923da9": "mutation JoinCourseLeaderboard($courseId: String!) {\n joinCourseLeaderboard(courseId: $courseId) {\n id\n participation {\n id\n isActive\n __typename\n }\n __typename\n }\n}", "2aa04f62eccae935f5c93441912c9d13f84e1ea4fe66e68ec9b92235f169e1a5": "mutation JoinCourseWithPin($pin: Int!) {\n joinCourseWithPin(pin: $pin) {\n id\n __typename\n }\n}", @@ -120,6 +129,7 @@ "65d10feac0f07d748c97dcaf5fcef53d7ebfb7f135fb9e50fc8eed66008cc55d": "mutation PublishMicroLearning($id: String!) {\n publishMicroLearning(id: $id) {\n id\n name\n displayName\n status\n __typename\n }\n}", "5082375d4e630f761e826156388eb00a294894d20e49bad8982a123c6752e5e4": "mutation PublishPracticeQuiz($id: String!, $availableFrom: Date) {\n publishPracticeQuiz(id: $id, availableFrom: $availableFrom) {\n id\n name\n displayName\n status\n availableFrom\n __typename\n }\n}", "14c3ff2b1b5d4691c220231c63eb2f22fdf937e12dd0f67faef86d0f0ce9231d": "mutation RateElement($elementInstanceId: Int!, $elementId: Int!, $rating: Int!) {\n rateElement(\n elementInstanceId: $elementInstanceId\n elementId: $elementId\n rating: $rating\n ) {\n id\n elementInstanceId\n upvote\n downvote\n feedback\n __typename\n }\n}", + "34b05c4bc0dff43fd56968c949bd4491a86dee819e1c1afdeaec2fc2d3ebc1f7": "mutation RebuildKbKnowledgeGraph($kbId: ID!, $qualityTier: KBGraphQualityTier) {\n rebuildKbKnowledgeGraph(kbId: $kbId, qualityTier: $qualityTier) {\n kbId\n isEnabled\n buildId\n status\n statusMessage\n qualityTier\n sourceContentDigest\n activeBuildId\n publishedBuildId\n isStale\n startedAt\n finishedAt\n createdAt\n updatedAt\n costConfigurationReady\n costCurrency\n quotaCurrency\n billingLabel\n standardEstimateMinorUnits\n highEstimateMinorUnits\n estimatedCostMinorUnits\n actualCostMinorUnits\n actualInputTokens\n actualOutputTokens\n actualEmbeddingTokens\n actualRequestCount\n maxCostMinorUnits\n costStatus\n semesterKey\n semesterQuotaMinorUnits\n semesterReservedMinorUnits\n semesterSettledMinorUnits\n remainingSemesterQuotaMinorUnits\n worstCaseRemainingMinorUnits\n __typename\n }\n}", "7605d19bd677854dcdf99d81cdaa7aeb09d6535b68726495a1e891421162a542": "mutation RemoveCatalogObjectAssignment($assignmentId: Int!) {\n removeCatalogObjectAssignment(assignmentId: $assignmentId)\n}", "ec1420ad392f8a95d277868c8d9ea0e90fa7349073f068fbdbdfea35ce9c676d": "mutation RemoveObject($objectId: String!, $objectType: ObjectType!) {\n removeObject(objectId: $objectId, objectType: $objectType)\n}", "dd396efd34357dfa2a6a8edaa3b020cd9aa752d411f4ff08e42a1c40d86b75d1": "mutation RemoveUserFromGroup($groupId: Int!, $userId: String!) {\n removeUserFromGroup(groupId: $groupId, userId: $userId)\n}", @@ -127,6 +137,7 @@ "776a25a044dd45e3d8d72644e69219261a364308cecdddf956b06c77fab6b088": "mutation RequestCatalogCollection($catalogCollectionId: String!, $requestedPermissionLevel: PermissionLevel) {\n requestCatalogCollection(\n catalogCollectionId: $catalogCollectionId\n requestedPermissionLevel: $requestedPermissionLevel\n ) {\n id\n name\n access\n ownerShortname\n isOwner\n isManager\n isRequested\n isShared\n __typename\n }\n}", "6ae8106fe3812691ea6ea239b5fd07b4b3f23a57e77c5ff96245f94cec2ea939": "mutation RequestCatalogObject($objectId: String!, $objectType: ObjectType!, $catalogCollectionId: String, $requestedPermissionLevel: PermissionLevel) {\n requestCatalogObject(\n objectId: $objectId\n objectType: $objectType\n catalogCollectionId: $catalogCollectionId\n requestedPermissionLevel: $requestedPermissionLevel\n )\n}", "2cf3899fca7c379bb6d6383fe9a7b27059da2f59d00b54362660900336735fc0": "mutation MRequestCatalystAccess($institution: String!, $useCase: String!) {\n requestCatalystAccess(institution: $institution, useCase: $useCase)\n}", + "8d75941b78eaf3c3b8132a93702f64b2dd24375cfe3ad3d4b02588379eb0452a": "mutation RequestKbFileUpload($kbId: ID!, $fileName: String!, $contentType: String!, $sizeBytes: Int!) {\n requestKbFileUpload(\n kbId: $kbId\n fileName: $fileName\n contentType: $contentType\n sizeBytes: $sizeBytes\n ) {\n uploadSasURL\n containerName\n blobName\n __typename\n }\n}", "6732cd3543debba9429eda45f35fde15da049927509061a0790f774b447bd744": "fragment ActivityInfoData on ActivityInfo {\n id\n templateId\n type\n status\n courseId\n courseName\n courseStartDate\n courseLanguage\n numOfStacks\n numOfElements\n reviewStatus\n automaticPublicationAt\n scheduledStartAt\n scheduledEndAt\n groupDeadlineDate\n numOfParticipantGroups\n name\n displayName\n permissionLevel\n derivedAccess\n areInstancesOutdated\n isGamificationEnabled\n isAssessmentEnabled\n pinCode\n numSharedUsers\n isOwner\n isManager\n isEditor\n isExecutor\n isShared\n isRemovable\n isActivityReviewer\n sharingType\n updatedAt\n __typename\n}\nmutation ResetAssessmentLiveQuiz($id: String!) {\n resetAssessmentLiveQuiz(id: $id) {\n ...ActivityInfoData\n __typename\n }\n}", "b0d21de9f8e363fccafa70e7536ee35762f95bb08e97f20a696c0c48d0726312": "mutation ResolveActivityLogEntry($id: Int!) {\n resolveActivityLogEntry(id: $id) {\n id\n resolved\n resolvedAt\n __typename\n }\n}", "530545f6a9138062b848c5bb3a86e5c081d5db54fb6e60d0e110be2e00b80246": "mutation ResolveFeedback($id: Int!, $isResolved: Boolean!, $liveQuizId: String!) {\n resolveFeedback(id: $id, isResolved: $isResolved, liveQuizId: $liveQuizId) {\n id\n isResolved\n __typename\n }\n}", @@ -139,6 +150,7 @@ "83985b6a0eae86d4346adbcce03ceedaae4c493cca39fea66cd403c1a827586a": "mutation SetActivityReviewStatus($activityId: String!, $activityType: ActivityType!, $isReviewed: Boolean!) {\n setActivityReviewStatus(\n activityId: $activityId\n activityType: $activityType\n isReviewed: $isReviewed\n )\n}", "0eb6653fe8019011196e53ecdcf33c583837be0f0ba18985d292c1b9d2b2ab2b": "mutation SetAiFeatures($email: String!, $enabled: Boolean!) {\n setAiFeatures(email: $email, enabled: $enabled)\n}", "a4752cc543598f97a831d3c037eddb557458f76947e08d5080ea8e603697e746": "mutation SetBetaFeatures($enabled: Boolean!) {\n setBetaFeatures(enabled: $enabled)\n}", + "192ac7e5f6324747b017a8cc202780738b5f68158cb236968d00f885370038a8": "mutation SetKbKnowledgeGraphEnabled($kbId: ID!, $enabled: Boolean!) {\n setKbKnowledgeGraphEnabled(kbId: $kbId, enabled: $enabled) {\n kbId\n isEnabled\n buildId\n status\n statusMessage\n qualityTier\n sourceContentDigest\n activeBuildId\n publishedBuildId\n isStale\n startedAt\n finishedAt\n createdAt\n updatedAt\n costConfigurationReady\n costCurrency\n quotaCurrency\n billingLabel\n standardEstimateMinorUnits\n highEstimateMinorUnits\n estimatedCostMinorUnits\n actualCostMinorUnits\n actualInputTokens\n actualOutputTokens\n actualEmbeddingTokens\n actualRequestCount\n maxCostMinorUnits\n costStatus\n semesterKey\n semesterQuotaMinorUnits\n semesterReservedMinorUnits\n semesterSettledMinorUnits\n remainingSemesterQuotaMinorUnits\n worstCaseRemainingMinorUnits\n __typename\n }\n}", "aa3336fbe64a06f64d6d664db179ddc22cb32dab78536cad2b3c5ad34cb2221a": "mutation SetLiveQuizPin($liveQuizId: String!, $pin: String!) {\n setLiveQuizPin(liveQuizId: $liveQuizId, pin: $pin)\n}", "cd9a8481fde02ab6f7e8d4b412f24fedcd25fdb7b05174bbdfb7d6b7cf0add40": "mutation ShareElementsBatch($elementIds: [Int!]!, $permissionLevel: PermissionLevel!, $shortnameOrEmail: String, $userGroupId: Int) {\n shareElementsBatch(\n elementIds: $elementIds\n permissionLevel: $permissionLevel\n shortnameOrEmail: $shortnameOrEmail\n userGroupId: $userGroupId\n ) {\n targetError\n outcomes {\n elementId\n status\n reason\n __typename\n }\n __typename\n }\n}", "4ed36d7d73dd1f400325c9d06b8e856e9755feeaa99f29f11dd31f4b34a75a6d": "fragment PermissionInfoData on PermissionInfo {\n permissionId\n userId\n username\n userEmail\n userGroupName\n permissionLevel\n propagation\n isOwn\n __typename\n}\nmutation ShareObject($objectId: String!, $objectType: ObjectType!, $permissionLevel: PermissionLevel!, $shortnameOrEmail: String, $userGroupId: Int, $propagation: Boolean!) {\n shareObject(\n objectId: $objectId\n objectType: $objectType\n permissionLevel: $permissionLevel\n shortnameOrEmail: $shortnameOrEmail\n userGroupId: $userGroupId\n propagation: $propagation\n ) {\n ...PermissionInfoData\n __typename\n }\n}", @@ -196,7 +208,7 @@ "e59ead934b55cdd40cb94eec86cb94f1ae12490c053041704c6313555c091873": "query GetCatalogSharingRequests {\n getCatalogSharingRequests {\n requestId\n objectName\n objectType\n userId\n userShortname\n userEmail\n __typename\n }\n}", "184baf28176c758da182405c0991c68bdaea7f83d2a3845fc94a5b9bdca64857": "query QGetCatalystRequestAccess {\n userScope\n}", "afd5a7321e3f64c722185da6d71e6f574c09be439d076f3db530628515833fc5": "query GetChatModelRegistry {\n getChatModelRegistry {\n id\n name\n description\n fallback\n supportsReasoning\n supportedReasoningEfforts\n __typename\n }\n}", - "856e57099c46397537583d9276de8d3466efe118a34a9923daa58cf379f05272": "query GetChatbotsInfo {\n getChatbotsInfo {\n id\n name\n description\n avatar\n modelSelection\n allowedModelIds\n allowedReasoningEffortsByModel {\n modelId\n efforts\n __typename\n }\n creditInitialCredits\n creditResetPeriod\n creditResetAmount\n creditMaxCredits\n courses {\n id\n name\n __typename\n }\n usageSummary {\n threadCount\n messageCount\n participantCount\n lastActivityAt\n totalCredits\n currentCredits\n totalResets\n lastResetAt\n __typename\n }\n disclaimerSummary {\n id\n name\n title\n acceptedCount\n declinedCount\n pendingCount\n __typename\n }\n mcpConfigurations {\n serverId\n serverName\n serverDescription\n serverIsActive\n chatMode\n isEnabled\n priority\n allowedToolsCount\n __typename\n }\n createdAt\n updatedAt\n __typename\n }\n}", + "d690b406938918d1d111f4cf0fc7035508999b01bc87da18df299015a322242c": "query GetChatbotsInfo {\n getChatbotsInfo {\n id\n name\n description\n avatar\n modelSelection\n allowedModelIds\n allowedReasoningEffortsByModel {\n modelId\n efforts\n __typename\n }\n creditInitialCredits\n creditResetPeriod\n creditResetAmount\n creditMaxCredits\n courses {\n id\n name\n __typename\n }\n usageSummary {\n threadCount\n messageCount\n participantCount\n lastActivityAt\n totalCredits\n currentCredits\n totalResets\n lastResetAt\n __typename\n }\n disclaimerSummary {\n id\n name\n title\n acceptedCount\n declinedCount\n pendingCount\n __typename\n }\n mcpConfigurations {\n serverId\n serverName\n serverDescription\n serverIsActive\n chatMode\n isEnabled\n priority\n allowedToolsCount\n __typename\n }\n enabledKnowledgeBase {\n id\n name\n __typename\n }\n createdAt\n updatedAt\n __typename\n }\n}", "26315050abef8a4f7daf07d2c424b31fbc05e99672bfa2c8f612b07c740bc5f8": "fragment ElementDataInfo on ElementInstance {\n elementData {\n ... on ChoicesElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on NumericalElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on FreeTextElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on SelectionElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on CaseStudyElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on FlashcardElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on ContentElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n __typename\n }\n __typename\n}\nquery GetCockpitQuiz($id: String!) {\n cockpitQuiz(id: $id) {\n id\n isLiveQAEnabled\n isConfusionFeedbackEnabled\n isModerationEnabled\n isGamificationEnabled\n isAssessmentEnabled\n namespace\n name\n displayName\n pinCode\n status\n startedAt\n course {\n id\n displayName\n language\n __typename\n }\n blocks {\n id\n numOfParticipants\n order\n status\n expiresAt\n timeLimit\n randomSelection\n execution\n elements {\n id\n type\n elementType\n ...ElementDataInfo\n __typename\n }\n __typename\n }\n activeBlock {\n id\n __typename\n }\n confusionSummary {\n speed\n difficulty\n numberOfParticipants\n __typename\n }\n feedbacks {\n id\n isPublished\n isPinned\n isResolved\n content\n votes\n createdAt\n resolvedAt\n responses {\n id\n content\n positiveReactions\n negativeReactions\n createdAt\n __typename\n }\n __typename\n }\n __typename\n }\n}", "e75fbff1c5cb7ff5e8ba8dea58c1cf225065c420b007433ad34477dd8c75c42b": "query GetControlCourse($courseId: String!) {\n controlCourse(id: $courseId) {\n id\n name\n liveQuizzes {\n id\n name\n status\n __typename\n }\n __typename\n }\n}", "f62c7d2cb59eb42a077e6d527c90f4838a2888cb9a9f60259979114b6a3b759f": "query GetControlCourses {\n controlCourses {\n id\n name\n isArchived\n displayName\n description\n __typename\n }\n}", @@ -231,6 +243,13 @@ "c720e1970bcd63a7bce7c80754a5872c8389b1ba35e693e9914f6dd014ae530f": "query GetGroupActivityInstances($groupId: String!, $courseId: String!) {\n groupActivityInstances(groupId: $groupId, courseId: $courseId) {\n id\n decisionsSubmittedAt\n resultsComputedAt\n results {\n passed\n points\n comment\n grading {\n instanceId\n score\n maxPoints\n feedback\n __typename\n }\n __typename\n }\n groupActivityId\n __typename\n }\n}", "7a3935db0e7cae8d5baf574a1c0506f568ffaaae1522a49363245946cff64d2d": "query GetGroupActivitySummary($id: String!) {\n getGroupActivitySummary(id: $id) {\n numOfStartedInstances\n numOfSubmissions\n __typename\n }\n}", "605ab5b696bdbe5c9d72bf5bb8228f6e5a24c5e779988908ef1ca6bac6028ff8": "query GetInstanceUpdateActivities($elementId: Int!, $hasSampleSolution: Boolean, $includeTemplateInstances: Boolean!) {\n getInstanceUpdateActivities(\n elementId: $elementId\n hasSampleSolution: $hasSampleSolution\n includeTemplateInstances: $includeTemplateInstances\n ) {\n activityId\n activityName\n courseName\n activityType\n status\n __typename\n }\n}", + "4fee195e47274aee88fe26597a1c6540ddaedb2c59259f911187a80eb8ff1760": "query GetKb($id: ID!) {\n getKb(id: $id) {\n id\n name\n description\n metrics {\n visibleResourceCount\n visibleSizeBytes\n unknownSizeResourceCount\n quotaResourceCount\n quotaSizeBytes\n resourceLimit\n storageLimitBytes\n pendingCleanupCount\n pendingCleanupSizeBytes\n reservedResourceCount\n reservedSizeBytes\n linkedConsumerCount\n __typename\n }\n __typename\n }\n}", + "6f0a601d3ae538a91a4278f49f20a0c60d8b6fcc4169df1da444eaad13ddfd10": "query GetKbChatbotBindings($kbId: ID!) {\n getKbChatbotBindings(kbId: $kbId) {\n chatbotId\n chatbotName\n enabledKbId\n enabledKbName\n __typename\n }\n}", + "05f7588b286c01a72323d43fd54270d2aa8a60ee9f141fc88161f0eebf2c634e": "query GetKbKnowledgeGraphConfig($kbId: ID!) {\n getKbKnowledgeGraphConfig(kbId: $kbId) {\n kbId\n isEnabled\n buildId\n status\n statusMessage\n qualityTier\n sourceContentDigest\n activeBuildId\n publishedBuildId\n isStale\n startedAt\n finishedAt\n createdAt\n updatedAt\n costConfigurationReady\n costCurrency\n quotaCurrency\n billingLabel\n standardEstimateMinorUnits\n highEstimateMinorUnits\n estimatedCostMinorUnits\n actualCostMinorUnits\n actualInputTokens\n actualOutputTokens\n actualEmbeddingTokens\n actualRequestCount\n maxCostMinorUnits\n costStatus\n semesterKey\n semesterQuotaMinorUnits\n semesterReservedMinorUnits\n semesterSettledMinorUnits\n remainingSemesterQuotaMinorUnits\n worstCaseRemainingMinorUnits\n __typename\n }\n}", + "6f29eda874b93652adf027082ed8fc7e351672d110619b874aa14fc5a69a62df": "query GetKbKnowledgeGraphNeighbors($kbId: ID!, $nodeId: ID!) {\n getKbKnowledgeGraphNeighbors(kbId: $kbId, nodeId: $nodeId) {\n kbId\n buildId\n isStale\n nodes {\n id\n labels\n kind\n displayLabel\n summary\n content\n degree\n sourceReferences {\n resourceId\n title\n reference\n __typename\n }\n __typename\n }\n edges {\n id\n source\n target\n type\n label\n properties\n __typename\n }\n truncated\n __typename\n }\n}", + "9384ade70aa981400b86fb28e9c721ef8fa509f753079cbb012c65fbe9e67cbb": "query GetKbKnowledgeGraphOverview($kbId: ID!) {\n getKbKnowledgeGraphOverview(kbId: $kbId) {\n kbId\n buildId\n isStale\n nodes {\n id\n labels\n kind\n displayLabel\n summary\n content\n degree\n sourceReferences {\n resourceId\n title\n reference\n __typename\n }\n __typename\n }\n edges {\n id\n source\n target\n type\n label\n properties\n __typename\n }\n truncated\n __typename\n }\n}", + "e3d7a97009a79db9c928a6d2ef9f31ec628680f3d065baf1b0928cf193fb8db1": "query GetKbResourceIngestionRuns($resourceId: ID!) {\n getKbResourceIngestionRuns(resourceId: $resourceId) {\n id\n status\n resourceVersion\n errorCode\n createdAt\n __typename\n }\n}", + "73d12da4a5f3056ad076465238b9d45498fb8b1f7118d6fef01400aa6f90c3d6": "query GetKbResources($kbId: ID!, $first: Int, $after: String, $search: String, $type: KBResourceType, $status: KBIngestionStatus) {\n getKbResources(\n kbId: $kbId\n first: $first\n after: $after\n search: $search\n type: $type\n status: $status\n ) {\n items {\n id\n type\n title\n sourceUrl\n originalFilename\n mimeType\n sizeBytes\n status\n ingestedAt\n resourceVersion\n activeResourceVersion\n latestIngestionRun {\n id\n status\n errorCode\n __typename\n }\n createdAt\n updatedAt\n __typename\n }\n pageInfo {\n hasNextPage\n endCursor\n __typename\n }\n totalCount\n __typename\n }\n}", "d1ab35eb40e5193d9ef4a4563d1057e860db1b9774d82076ccc1c380350139ff": "query GetLecturerViewLiveQuiz($id: String!) {\n getLecturerViewLiveQuiz(id: $id) {\n id\n isLiveQAEnabled\n isConfusionFeedbackEnabled\n isModerationEnabled\n isGamificationEnabled\n confusionSummary {\n speed\n difficulty\n numberOfParticipants\n __typename\n }\n feedbacks {\n id\n isPublished\n isPinned\n isResolved\n content\n votes\n createdAt\n resolvedAt\n responses {\n id\n content\n positiveReactions\n negativeReactions\n createdAt\n __typename\n }\n __typename\n }\n __typename\n }\n}", "1e69b5994b1bc2af7504ba701ce4f26cc5a0e5aa8b784c3045cee75e306b4f92": "query GetLiveQuizEmbeddingInfo($id: String!) {\n getLiveQuizEmbeddingInfo(id: $id) {\n id\n hmac\n instances {\n id\n name\n __typename\n }\n __typename\n }\n}", "fa751a965ef3afe5dcc106c185df4c0aacf3e7c9614512d2e0825d6aec8ecbca": "fragment EvaluationResults on ActivityEvaluation {\n results {\n stackId\n stackName\n stackDescription\n stackOrder\n stackActive\n status\n expiresAt\n timeLimit\n instances {\n ... on ChoicesActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n choices {\n value\n count\n correct\n feedback\n __typename\n }\n __typename\n }\n }\n ... on NumericalActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n maxValue\n minValue\n solutionRanges {\n min\n max\n __typename\n }\n exactSolutions\n responseValues {\n value\n correct\n count\n __typename\n }\n __typename\n }\n statistics {\n max\n mean\n median\n min\n q1\n q3\n sd\n __typename\n }\n }\n ... on FreeTextActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n maxLength\n solutions\n responses {\n value\n correct\n count\n __typename\n }\n __typename\n }\n }\n ... on SelectionActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n numberOfInputs\n answerSolutionIds\n selectionResponses {\n answerId\n value\n count\n __typename\n }\n __typename\n }\n }\n ... on CaseStudyActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n cases {\n id\n name\n description\n __typename\n }\n items {\n id\n name\n __typename\n }\n criteria {\n id\n name\n labels {\n min\n mid\n max\n __typename\n }\n __typename\n }\n results {\n totalAnswers\n anonymousAnswers\n caseResults {\n caseId\n items {\n itemId\n criteria {\n criterionId\n name\n min\n max\n step\n unit\n solutionMin\n solutionMax\n statistics {\n min\n max\n mean\n median\n q1\n q3\n sd\n __typename\n }\n responses {\n value\n count\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n }\n ... on FlashcardActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n correctCount\n partialCount\n incorrectCount\n __typename\n }\n }\n ... on ContentActivityEvaluationData {\n __typename\n id\n type\n name\n content\n explanation\n hasSampleSolution\n hasAnswerFeedbacks\n results {\n totalAnswers\n anonymousAnswers\n __typename\n }\n }\n __typename\n }\n __typename\n }\n __typename\n}\nquery GetLiveQuizEvaluation($id: String!, $hmac: String) {\n liveQuizEvaluation(id: $id, hmac: $hmac) {\n id\n name\n displayName\n description\n courseLanguage\n isAssessmentEnabled\n pinCode\n ...EvaluationResults\n feedbacks {\n id\n isPublished\n isPinned\n isResolved\n content\n votes\n resolvedAt\n createdAt\n responses {\n id\n content\n positiveReactions\n negativeReactions\n createdAt\n __typename\n }\n __typename\n }\n confusionFeedbacks {\n speed\n difficulty\n createdAt\n __typename\n }\n __typename\n }\n liveQuizLeaderboard(quizId: $id, hmac: $hmac) {\n id\n participantId\n rank\n username\n avatar\n score\n isTemporary\n __typename\n }\n}", @@ -276,6 +295,7 @@ "0d9a8165dc0c4bb67d178b858d7d23bc0328362194dc978530e0effe13bb76bd": "query GetUserCourses {\n userCourses {\n id\n name\n displayName\n color\n isArchived\n isGamificationEnabled\n isAssessmentEnabled\n isGroupCreationEnabled\n description\n startDate\n endDate\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isShared\n isRemovable\n __typename\n }\n}", "cea84675a3e93cae52225b24d240c8eef9986c516dedc156f4aa0a5def0e95ff": "query GetUserElements($status: ElementStatus, $type: ElementType, $hasSampleSolution: Boolean!, $hasAnswerFeedbacks: Boolean!, $searchString: String, $showOwned: Boolean, $showShared: Boolean, $showDependencies: Boolean, $tagIds: [Int!]!, $activityId: String, $multiplier: Int, $showUntagged: Boolean!, $sortByType: SortByType!, $sortByAsc: Boolean!, $showArchived: Boolean!, $numEntries: Int, $offset: Int) {\n userElements(\n status: $status\n type: $type\n hasSampleSolution: $hasSampleSolution\n hasAnswerFeedbacks: $hasAnswerFeedbacks\n searchString: $searchString\n showOwned: $showOwned\n showShared: $showShared\n showDependencies: $showDependencies\n tagIds: $tagIds\n activityId: $activityId\n multiplier: $multiplier\n showUntagged: $showUntagged\n sortByType: $sortByType\n sortByAsc: $sortByAsc\n showArchived: $showArchived\n numEntries: $numEntries\n offset: $offset\n ) {\n numOfElements\n elements {\n __typename\n ... on ChoicesElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n options {\n __typename\n hasSampleSolution\n hasAnswerFeedbacks\n displayMode\n choices {\n ix\n correct\n feedback\n value\n __typename\n }\n }\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on NumericalElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n options {\n __typename\n hasSampleSolution\n hasAnswerFeedbacks\n placeholder\n accuracy\n unit\n solutionRanges {\n __typename\n min\n max\n }\n exactSolutions\n restrictions {\n __typename\n min\n max\n }\n }\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on FreeTextElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n options {\n __typename\n hasSampleSolution\n hasAnswerFeedbacks\n solutions\n restrictions {\n __typename\n maxLength\n }\n }\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on SelectionElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n options {\n __typename\n hasSampleSolution\n numberOfInputs\n }\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on CaseStudyElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n options {\n __typename\n hasSampleSolution\n criteria {\n id\n name\n order\n min\n max\n step\n unit\n labels {\n min\n mid\n max\n __typename\n }\n __typename\n }\n cases {\n id\n title\n description\n order\n __typename\n }\n }\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on FlashcardElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n ... on ContentElement {\n id\n name\n status\n type\n content\n basePoints\n pointsMultiplier\n version\n isArchived\n isDeleted\n createdAt\n updatedAt\n derivedAccess\n numSharedUsers\n permissionLevel\n isOwner\n isManager\n isEditor\n isImported\n isShared\n isRemovable\n sharingType\n tags {\n id\n name\n order\n __typename\n }\n __typename\n }\n }\n __typename\n }\n}", "c720d3ff1338c6a63f19f848126b5c30662b08d087d0250113db7549d80725bc": "query GetUserGroupsUser {\n getUserGroupsUser {\n id\n name\n members {\n id\n shortname\n email\n isSelf\n __typename\n }\n admins {\n id\n shortname\n email\n isSelf\n __typename\n }\n owner {\n id\n shortname\n email\n isSelf\n __typename\n }\n numOfMembers\n isMember\n isAdmin\n isOwner\n __typename\n }\n}", + "d3794d64c91ea212794d42c52ca329960f8e2a53c9bb707739e06b46b434acfb": "query GetUserKbs($first: Int, $after: String, $search: String) {\n getUserKbsConnection(first: $first, after: $after, search: $search) {\n items {\n id\n name\n description\n metrics {\n visibleResourceCount\n visibleSizeBytes\n linkedConsumerCount\n __typename\n }\n __typename\n }\n pageInfo {\n hasNextPage\n endCursor\n __typename\n }\n totalCount\n __typename\n }\n}", "7b289235e4ea8e3fb2073ae175b1b900a77cb6c1aae9b12580cc08be825ad1aa": "query GetUserLogins {\n userLogins {\n id\n name\n scope\n lastLoginAt\n user {\n id\n shortname\n __typename\n }\n __typename\n }\n userScope\n}", "af5513e8c56c1805060f0e2854a4ada90c4484d09481f2ba2a59136cb731cd63": "query GetUserMediaFiles {\n userMediaFiles {\n id\n name\n type\n href\n __typename\n }\n}", "756ada2de676dfd25fd4f9dd38e1255725a557ad945b379ea7e0a191df9135d0": "query GetUserRunningLiveQuizzes {\n userRunningLiveQuizzes {\n id\n name\n __typename\n }\n}", @@ -285,6 +305,7 @@ "611adf4b460b125a33a9296f774e33a19bcd0299e176a1e0cc30216dfb83bc88": "query QGetVerifiableCredential($token: String!) {\n assessmentReportVerification(token: $token) {\n status\n issuedAt\n snapshot {\n version\n subject {\n name\n source\n __typename\n }\n course {\n name\n displayName\n __typename\n }\n results {\n basePoints\n availableBasePoints\n correctnessPoints\n availableCorrectnessPoints\n bonusPoints\n availableBonusPoints\n totalPoints\n availableTotalPoints\n __typename\n }\n comparison {\n cohortSize\n percentile\n histogram {\n binStart\n binEnd\n count\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}", "ba512e4efff648ed70522f11905d073e2c89411794adc80a43d65301f905c649": "fragment ElementDataWithoutSolutions on ElementInstance {\n elementData {\n ... on ChoicesElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n options {\n hasSampleSolution\n displayMode\n choices {\n ix\n value\n __typename\n }\n __typename\n }\n }\n ... on NumericalElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n options {\n hasSampleSolution\n accuracy\n placeholder\n unit\n restrictions {\n min\n max\n __typename\n }\n __typename\n }\n }\n ... on FreeTextElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n options {\n hasSampleSolution\n restrictions {\n maxLength\n __typename\n }\n __typename\n }\n }\n ... on SelectionElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n options {\n hasSampleSolution\n numberOfInputs\n answerCollection {\n id\n entries {\n id\n value\n __typename\n }\n __typename\n }\n __typename\n }\n }\n ... on CaseStudyElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n options {\n hasSampleSolution\n items {\n id\n value\n __typename\n }\n criteria {\n id\n name\n min\n max\n step\n unit\n labels {\n min\n mid\n max\n __typename\n }\n __typename\n }\n cases {\n id\n title\n description\n __typename\n }\n __typename\n }\n }\n ... on FlashcardElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n ... on ContentElementData {\n __typename\n id\n elementId\n name\n type\n content\n explanation\n basePoints\n pointsMultiplier\n }\n __typename\n }\n __typename\n}\nquery GroupActivityDetails($activityId: String!, $groupId: String!) {\n groupActivityDetails(activityId: $activityId, groupId: $groupId) {\n id\n displayName\n status\n description\n scheduledStartAt\n scheduledEndAt\n clues {\n id\n displayName\n __typename\n }\n stacks {\n id\n type\n displayName\n description\n order\n elements {\n id\n type\n elementType\n ...ElementDataWithoutSolutions\n __typename\n }\n __typename\n }\n course {\n id\n displayName\n color\n __typename\n }\n group {\n id\n name\n participants {\n id\n username\n avatar\n isSelf\n __typename\n }\n __typename\n }\n activityInstance {\n id\n clues {\n id\n displayName\n type\n unit\n value\n participant {\n id\n username\n avatar\n isSelf\n __typename\n }\n __typename\n }\n decisionsSubmittedAt\n decisions {\n instanceId\n type\n freeTextResponse\n choicesResponse {\n ix\n selected\n __typename\n }\n numericalResponse\n contentResponse\n selectionResponse\n caseStudyResponse {\n caseId\n itemResponses {\n itemId\n criterionResponses {\n criterionId\n response\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n resultsComputedAt\n results {\n passed\n points\n comment\n grading {\n instanceId\n score\n maxPoints\n feedback\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}", "c6f065a54c438a4da98a268e962e6c54850a1d599c908c291b77c695db75b7ef": "query Participations($endpoint: String, $assessmentOnly: Boolean) {\n participations(endpoint: $endpoint, assessmentOnly: $assessmentOnly) {\n id\n completedMicroLearnings\n subscriptions {\n id\n endpoint\n __typename\n }\n course {\n id\n displayName\n startDate\n endDate\n description\n isGamificationEnabled\n microLearnings {\n id\n displayName\n scheduledStartAt\n scheduledEndAt\n __typename\n }\n liveQuizzes {\n id\n displayName\n __typename\n }\n __typename\n }\n __typename\n }\n}", + "e0e7bdb56ca9a878a795208ddd9290c51338e1ecb50029e20e275ecaed55b09f": "query SearchKbKnowledgeGraph($kbId: ID!, $query: String!) {\n searchKbKnowledgeGraph(kbId: $kbId, query: $query) {\n kbId\n buildId\n isStale\n nodes {\n id\n labels\n kind\n displayLabel\n summary\n content\n degree\n sourceReferences {\n resourceId\n title\n reference\n __typename\n }\n __typename\n }\n edges {\n id\n source\n target\n type\n label\n properties\n __typename\n }\n truncated\n __typename\n }\n}", "9c4e2c9e5469cf452a1d5d4b814f22ba45ff56571946ab3ff59d403cb4a8aa9c": "query Self($liveQuizId: String) {\n self(liveQuizId: $liveQuizId) {\n id\n role\n scopeQuizId\n isCourseParticipant\n isCourseParticipationActive\n email\n institutionalEmail\n username\n locale\n avatar\n avatarSettings {\n skinTone\n eyes\n mouth\n hair\n facialHair\n accessory\n hairColor\n clothing\n clothingColor\n __typename\n }\n isActive\n isProfilePublic\n xp\n level\n levelData {\n id\n index\n name\n avatar\n requiredXp\n nextLevel {\n id\n index\n requiredXp\n avatar\n name\n __typename\n }\n __typename\n }\n __typename\n }\n}", "07ed90975e46d0b5a00efb45e844030ae6bf16653e39c53ecec5afddc72c62f2": "query SelfWithAchievements {\n selfWithAchievements {\n participant {\n id\n username\n avatar\n avatarSettings {\n skinTone\n eyes\n mouth\n hair\n facialHair\n accessory\n hairColor\n clothing\n clothingColor\n __typename\n }\n xp\n level\n levelData {\n id\n index\n name\n avatar\n requiredXp\n nextLevel {\n id\n index\n requiredXp\n avatar\n name\n __typename\n }\n __typename\n }\n achievements {\n id\n achievedAt\n achievedCount\n achievement {\n id\n nameDE\n nameEN\n descriptionDE\n descriptionEN\n icon\n iconColor\n __typename\n }\n __typename\n }\n __typename\n }\n achievements {\n id\n nameDE\n nameEN\n descriptionDE\n descriptionEN\n icon\n iconColor\n __typename\n }\n __typename\n }\n}", "cb87e067f0165f623e1d917daad40b7d3f19cc2254ac6bfde9ab75997dd59061": "query UserProfile {\n userProfile {\n id\n email\n sendProjectUpdates\n shortname\n role\n locale\n firstLogin\n catalyst\n catalystTier\n publicPreview\n privatePreview\n aiFeaturesEnabled\n numChatbots\n __typename\n }\n}", diff --git a/packages/graphql/src/schema/kbKnowledgeGraph.ts b/packages/graphql/src/schema/kbKnowledgeGraph.ts new file mode 100644 index 0000000000..9b38bfa3ea --- /dev/null +++ b/packages/graphql/src/schema/kbKnowledgeGraph.ts @@ -0,0 +1,149 @@ +import * as DB from '@klicker-uzh/prisma/client' +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphResponse, + KnowledgeGraphSourceReference, +} from '@klicker-uzh/types' +import builder from '../builder.js' +import type { KBKnowledgeGraphConfig } from '../services/knowledge.js' + +export const KBGraphBuildStatus = builder.enumType('KBGraphBuildStatus', { + values: Object.values(DB.KBGraphBuildStatus), +}) + +export const KBGraphQualityTier = builder.enumType('KBGraphQualityTier', { + values: Object.values(DB.KBGraphQualityTier), +}) + +export const KBGraphCostStatus = builder.enumType('KBGraphCostStatus', { + values: Object.values(DB.KBGraphCostStatus), +}) + +export const KBKnowledgeGraphConfigRef = + builder.objectRef('KBKnowledgeGraphConfig') +export const KBKnowledgeGraphConfigType = KBKnowledgeGraphConfigRef.implement({ + fields: (t) => ({ + kbId: t.exposeID('kbId'), + isEnabled: t.exposeBoolean('isEnabled'), + buildId: t.exposeID('buildId', { nullable: true }), + status: t.expose('status', { type: KBGraphBuildStatus, nullable: true }), + statusMessage: t.exposeString('statusMessage', { nullable: true }), + qualityTier: t.expose('qualityTier', { + type: KBGraphQualityTier, + nullable: true, + }), + sourceContentDigest: t.exposeString('sourceContentDigest', { + nullable: true, + }), + activeBuildId: t.exposeID('activeBuildId', { nullable: true }), + publishedBuildId: t.exposeID('publishedBuildId', { nullable: true }), + isStale: t.exposeBoolean('isStale'), + startedAt: t.expose('startedAt', { type: 'Date', nullable: true }), + finishedAt: t.expose('finishedAt', { type: 'Date', nullable: true }), + createdAt: t.expose('createdAt', { type: 'Date', nullable: true }), + updatedAt: t.expose('updatedAt', { type: 'Date', nullable: true }), + costConfigurationReady: t.exposeBoolean('costConfigurationReady'), + costCurrency: t.exposeString('costCurrency', { nullable: true }), + quotaCurrency: t.exposeString('quotaCurrency', { nullable: true }), + billingLabel: t.exposeString('billingLabel', { nullable: true }), + standardEstimateMinorUnits: t.exposeInt('standardEstimateMinorUnits', { + nullable: true, + }), + highEstimateMinorUnits: t.exposeInt('highEstimateMinorUnits', { + nullable: true, + }), + estimatedCostMinorUnits: t.exposeInt('estimatedCostMinorUnits', { + nullable: true, + }), + actualCostMinorUnits: t.exposeInt('actualCostMinorUnits', { + nullable: true, + }), + actualInputTokens: t.exposeInt('actualInputTokens', { nullable: true }), + actualOutputTokens: t.exposeInt('actualOutputTokens', { nullable: true }), + actualEmbeddingTokens: t.exposeInt('actualEmbeddingTokens', { + nullable: true, + }), + actualRequestCount: t.exposeInt('actualRequestCount', { + nullable: true, + }), + maxCostMinorUnits: t.exposeInt('maxCostMinorUnits', { nullable: true }), + costStatus: t.expose('costStatus', { + type: KBGraphCostStatus, + nullable: true, + }), + semesterKey: t.exposeString('semesterKey', { nullable: true }), + semesterQuotaMinorUnits: t.exposeInt('semesterQuotaMinorUnits', { + nullable: true, + }), + semesterReservedMinorUnits: t.exposeInt('semesterReservedMinorUnits', { + nullable: true, + }), + semesterSettledMinorUnits: t.exposeInt('semesterSettledMinorUnits', { + nullable: true, + }), + remainingSemesterQuotaMinorUnits: t.exposeInt( + 'remainingSemesterQuotaMinorUnits', + { nullable: true } + ), + worstCaseRemainingMinorUnits: t.exposeInt('worstCaseRemainingMinorUnits', { + nullable: true, + }), + }), +}) + +export const KnowledgeGraphSourceReferenceRef = + builder.objectRef( + 'KnowledgeGraphSourceReference' + ) +export const KnowledgeGraphSourceReferenceType = + KnowledgeGraphSourceReferenceRef.implement({ + fields: (t) => ({ + resourceId: t.exposeID('resourceId'), + title: t.exposeString('title'), + reference: t.exposeString('reference', { nullable: true }), + }), + }) + +export const KnowledgeGraphNodeRef = + builder.objectRef('KnowledgeGraphNode') +export const KnowledgeGraphNodeType = KnowledgeGraphNodeRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + labels: t.exposeStringList('labels'), + kind: t.exposeString('kind'), + displayLabel: t.exposeString('displayLabel'), + summary: t.exposeString('summary', { nullable: true }), + content: t.exposeString('content', { nullable: true }), + degree: t.exposeInt('degree'), + sourceReferences: t.expose('sourceReferences', { + type: [KnowledgeGraphSourceReferenceRef], + }), + }), +}) + +export const KnowledgeGraphEdgeRef = + builder.objectRef('KnowledgeGraphEdge') +export const KnowledgeGraphEdgeType = KnowledgeGraphEdgeRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + source: t.exposeID('source'), + target: t.exposeID('target'), + type: t.exposeString('type'), + label: t.exposeString('label'), + properties: t.expose('properties', { type: 'Json' }), + }), +}) + +export const KnowledgeGraphResponseRef = + builder.objectRef('KnowledgeGraphResponse') +export const KnowledgeGraphResponseType = KnowledgeGraphResponseRef.implement({ + fields: (t) => ({ + kbId: t.exposeID('kbId'), + buildId: t.exposeID('buildId'), + isStale: t.exposeBoolean('isStale'), + nodes: t.expose('nodes', { type: [KnowledgeGraphNodeRef] }), + edges: t.expose('edges', { type: [KnowledgeGraphEdgeRef] }), + truncated: t.exposeBoolean('truncated'), + }), +}) diff --git a/packages/graphql/src/schema/knowledge.ts b/packages/graphql/src/schema/knowledge.ts new file mode 100644 index 0000000000..b4ed94d31f --- /dev/null +++ b/packages/graphql/src/schema/knowledge.ts @@ -0,0 +1,197 @@ +import * as DB from '@klicker-uzh/prisma/client' +import builder from '../builder.js' + +interface IKBFileUpload { + uploadSasURL: string + containerName: string + blobName: string +} + +export const KBFileUploadRef = builder.objectRef('KBFileUpload') +export const KBFileUpload = KBFileUploadRef.implement({ + fields: (t) => ({ + uploadSasURL: t.exposeString('uploadSasURL'), + containerName: t.exposeString('containerName'), + blobName: t.exposeString('blobName'), + }), +}) + +export const KBResourceType = builder.enumType('KBResourceType', { + values: Object.values(DB.KBResourceType), +}) + +export const KBResourceStatus = builder.enumType('KBResourceStatus', { + values: Object.values(DB.KBResourceStatus), +}) + +export const KBIngestionStatus = builder.enumType('KBIngestionStatus', { + values: Object.values(DB.KBIngestionStatus), +}) + +export const KBIngestionRunRef = + builder.objectRef('KBIngestionRun') +export const KBIngestionRun = KBIngestionRunRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + status: t.expose('status', { type: KBIngestionStatus }), + resourceVersion: t.exposeInt('resourceVersion'), + contentSha256: t.exposeString('contentSha256', { nullable: true }), + statusMessage: t.exposeString('statusMessage', { nullable: true }), + errorCode: t.exposeString('errorCode', { nullable: true }), + startedAt: t.expose('startedAt', { type: 'Date', nullable: true }), + finishedAt: t.expose('finishedAt', { type: 'Date', nullable: true }), + createdAt: t.expose('createdAt', { type: 'Date' }), + updatedAt: t.expose('updatedAt', { type: 'Date' }), + }), +}) + +interface IKBResource extends DB.KBResource { + ingestionRuns?: DB.KBIngestionRun[] +} + +export const KBResourceRef = builder.objectRef('KBResource') +export const KBResource = KBResourceRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + type: t.expose('type', { type: KBResourceType }), + title: t.exposeString('title'), + sourceUrl: t.exposeString('sourceUrl', { nullable: true }), + originalFilename: t.exposeString('originalFilename', { nullable: true }), + mimeType: t.exposeString('mimeType', { nullable: true }), + sizeBytes: t.exposeInt('sizeBytes', { nullable: true }), + status: t.expose('status', { type: KBResourceStatus }), + statusMessage: t.exposeString('statusMessage', { nullable: true }), + ingestedAt: t.expose('ingestedAt', { type: 'Date', nullable: true }), + resourceVersion: t.exposeInt('resourceVersion'), + activeResourceVersion: t.exposeInt('activeResourceVersion', { + nullable: true, + }), + activeContentSha256: t.exposeString('activeContentSha256', { + nullable: true, + }), + errorCode: t.exposeString('errorCode', { nullable: true }), + latestIngestionRun: t.field({ + type: KBIngestionRunRef, + nullable: true, + resolve: (resource) => resource.ingestionRuns?.[0] ?? null, + }), + createdAt: t.expose('createdAt', { type: 'Date' }), + updatedAt: t.expose('updatedAt', { type: 'Date' }), + }), +}) + +interface IKBMetrics { + visibleResourceCount: number + visibleSizeBytes: number + unknownSizeResourceCount: number + quotaResourceCount: number + quotaSizeBytes: number + resourceLimit: number + storageLimitBytes: number + pendingCleanupCount: number + pendingCleanupSizeBytes: number + reservedResourceCount: number + reservedSizeBytes: number + linkedConsumerCount: number +} + +export const KBMetricsRef = builder.objectRef('KBMetrics') +export const KBMetrics = KBMetricsRef.implement({ + fields: (t) => ({ + visibleResourceCount: t.exposeInt('visibleResourceCount'), + visibleSizeBytes: t.exposeInt('visibleSizeBytes'), + unknownSizeResourceCount: t.exposeInt('unknownSizeResourceCount'), + quotaResourceCount: t.exposeInt('quotaResourceCount'), + quotaSizeBytes: t.exposeInt('quotaSizeBytes'), + resourceLimit: t.exposeInt('resourceLimit'), + storageLimitBytes: t.exposeInt('storageLimitBytes'), + pendingCleanupCount: t.exposeInt('pendingCleanupCount'), + pendingCleanupSizeBytes: t.exposeInt('pendingCleanupSizeBytes'), + reservedResourceCount: t.exposeInt('reservedResourceCount'), + reservedSizeBytes: t.exposeInt('reservedSizeBytes'), + linkedConsumerCount: t.exposeInt('linkedConsumerCount'), + }), +}) + +interface IKB extends DB.KB { + metrics?: IKBMetrics +} + +export const KBRef = builder.objectRef('KB') +export const KB = KBRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + name: t.exposeString('name'), + description: t.exposeString('description', { nullable: true }), + metrics: t.field({ + type: KBMetricsRef, + nullable: true, + resolve: (kb) => kb.metrics ?? null, + }), + createdAt: t.expose('createdAt', { type: 'Date' }), + updatedAt: t.expose('updatedAt', { type: 'Date' }), + }), +}) + +interface IKBPageInfo { + hasNextPage: boolean + endCursor: string | null +} + +export const KBPageInfoRef = builder.objectRef('KBPageInfo') +export const KBPageInfo = KBPageInfoRef.implement({ + fields: (t) => ({ + hasNextPage: t.exposeBoolean('hasNextPage'), + endCursor: t.exposeString('endCursor', { nullable: true }), + }), +}) + +interface IKBConnection { + items: IKB[] + pageInfo: IKBPageInfo + totalCount: number +} + +export const KBConnectionRef = builder.objectRef('KBConnection') +export const KBConnection = KBConnectionRef.implement({ + fields: (t) => ({ + items: t.expose('items', { type: [KBRef] }), + pageInfo: t.expose('pageInfo', { type: KBPageInfoRef }), + totalCount: t.exposeInt('totalCount'), + }), +}) + +interface IKBResourceConnection { + items: IKBResource[] + pageInfo: IKBPageInfo + totalCount: number +} + +export const KBResourceConnectionRef = builder.objectRef( + 'KBResourceConnection' +) +export const KBResourceConnection = KBResourceConnectionRef.implement({ + fields: (t) => ({ + items: t.expose('items', { type: [KBResourceRef] }), + pageInfo: t.expose('pageInfo', { type: KBPageInfoRef }), + totalCount: t.exposeInt('totalCount'), + }), +}) + +interface IKBChatbotBinding { + chatbotId: string + chatbotName: string + enabledKbId: string | null + enabledKbName: string | null +} + +export const KBChatbotBindingRef = + builder.objectRef('KBChatbotBinding') +export const KBChatbotBinding = KBChatbotBindingRef.implement({ + fields: (t) => ({ + chatbotId: t.exposeID('chatbotId'), + chatbotName: t.exposeString('chatbotName'), + enabledKbId: t.exposeID('enabledKbId', { nullable: true }), + enabledKbName: t.exposeString('enabledKbName', { nullable: true }), + }), +}) diff --git a/packages/graphql/src/schema/mutation.ts b/packages/graphql/src/schema/mutation.ts index 4df7eada53..60367d9f57 100644 --- a/packages/graphql/src/schema/mutation.ts +++ b/packages/graphql/src/schema/mutation.ts @@ -10,6 +10,7 @@ import * as CourseService from '../services/courses.js' import * as ElementService from '../services/elements.js' import * as FeedbackService from '../services/feedbacks.js' import * as GroupService from '../services/groups.js' +import * as KnowledgeService from '../services/knowledge.js' import * as LiveQuizService from '../services/liveQuizzes.js' import * as MicroLearningService from '../services/microLearning.js' import * as NotificationService from '../services/notifications.js' @@ -44,6 +45,11 @@ import { GroupActivityGradingInput, GroupActivityInstance, } from './groupActivity.js' +import { + KBGraphQualityTier, + KBKnowledgeGraphConfigType, +} from './kbKnowledgeGraph.js' +import { KB, KBChatbotBinding, KBFileUpload, KBResource } from './knowledge.js' import { ConfusionTimestep, Feedback, @@ -1719,6 +1725,147 @@ export const Mutation = builder.mutationType({ }, }), + createKb: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KB, + args: { + name: t.arg.string({ required: true }), + description: t.arg.string({ required: false }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.createKb(args, ctx) + }, + }), + + deleteKb: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KB, + args: { id: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.deleteKb(args, ctx) + }, + }), + + attachKbToChatbot: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBChatbotBinding, + args: { + kbId: t.arg.id({ required: true }), + chatbotId: t.arg.id({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.attachKbToChatbot(args, ctx) + }, + }), + + detachKbFromChatbot: t.withAuth(asUserFullAccess).boolean({ + nullable: false, + args: { + kbId: t.arg.id({ required: true }), + chatbotId: t.arg.id({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.detachKbFromChatbot(args, ctx) + }, + }), + + requestKbFileUpload: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBFileUpload, + args: { + kbId: t.arg.id({ required: true }), + fileName: t.arg.string({ required: true }), + contentType: t.arg.string({ required: true }), + sizeBytes: t.arg.int({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.requestKbFileUpload(args, ctx) + }, + }), + + confirmKbFileUpload: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBResource, + args: { + kbId: t.arg.id({ required: true }), + blobName: t.arg.string({ required: true }), + title: t.arg.string({ required: true }), + originalFilename: t.arg.string({ required: true }), + mimeType: t.arg.string({ required: true }), + sizeBytes: t.arg.int({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.confirmKbFileUpload(args, ctx) + }, + }), + + createKbUrlResource: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBResource, + args: { + kbId: t.arg.id({ required: true }), + url: t.arg.string({ required: true }), + title: t.arg.string({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.createKbUrlResource(args, ctx) + }, + }), + + deleteKbResource: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBResource, + args: { id: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.deleteKbResource(args, ctx) + }, + }), + + deleteKbResources: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: [KBResource], + args: { + kbId: t.arg.id({ required: true }), + ids: t.arg.stringList({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.deleteKbResources(args, ctx) + }, + }), + + ingestKbResource: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBResource, + args: { id: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.ingestKbResource(args, ctx) + }, + }), + + rebuildKbKnowledgeGraph: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBKnowledgeGraphConfigType, + args: { + kbId: t.arg.id({ required: true }), + qualityTier: t.arg({ type: KBGraphQualityTier, required: false }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.rebuildKbKnowledgeGraph(args, ctx) + }, + }), + + setKbKnowledgeGraphEnabled: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBKnowledgeGraphConfigType, + args: { + kbId: t.arg.id({ required: true }), + enabled: t.arg.boolean({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.setKbKnowledgeGraphEnabled(args, ctx) + }, + }), + setAiFeatures: t.withAuth(asAdmin).int({ nullable: true, args: { diff --git a/packages/graphql/src/schema/query.ts b/packages/graphql/src/schema/query.ts index 7cc7288de3..ac04962e71 100644 --- a/packages/graphql/src/schema/query.ts +++ b/packages/graphql/src/schema/query.ts @@ -11,6 +11,7 @@ import * as CourseService from '../services/courses.js' import * as ElementService from '../services/elements.js' import * as FeedbackService from '../services/feedbacks.js' import * as GroupService from '../services/groups.js' +import * as KnowledgeService from '../services/knowledge.js' import * as LiveQuizService from '../services/liveQuizzes.js' import * as MicroLearningService from '../services/microLearning.js' import * as ParticipantInvitationService from '../services/participantInvitations.js' @@ -71,6 +72,19 @@ import { GroupActivityInstance, GroupActivitySummary, } from './groupActivity.js' +import { + KBKnowledgeGraphConfigType, + KnowledgeGraphResponseType, +} from './kbKnowledgeGraph.js' +import { + KB, + KBChatbotBinding, + KBConnection, + KBIngestionRun, + KBIngestionStatus, + KBResourceConnection, + KBResourceType, +} from './knowledge.js' import { Feedback, LiveQuiz, @@ -131,6 +145,10 @@ export const Query = builder.queryType({ fields(t) { const asParticipant = { authenticated: true, role: DB.UserRole.PARTICIPANT } const asUser = { authenticated: true, role: DB.UserRole.USER } + const asUserFullAccess = { + ...asUser, + scope: DB.UserLoginScope.FULL_ACCESS, + } const asAdmin = { authenticated: true, role: DB.UserRole.ADMIN } const asUserWithCatalyst = { ...asUser, catalyst: true } @@ -1461,6 +1479,104 @@ export const Query = builder.queryType({ }, }), + getUserKbsConnection: t.withAuth(asUser).field({ + nullable: false, + type: KBConnection, + args: { + first: t.arg.int({ required: false }), + after: t.arg.string({ required: false }), + search: t.arg.string({ required: false }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getUserKbsConnection(args, ctx) + }, + }), + + getKb: t.withAuth(asUser).field({ + nullable: false, + type: KB, + args: { id: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKb(args, ctx) + }, + }), + + getKbResources: t.withAuth(asUser).field({ + nullable: false, + type: KBResourceConnection, + args: { + kbId: t.arg.id({ required: true }), + first: t.arg.int({ required: false }), + after: t.arg.string({ required: false }), + search: t.arg.string({ required: false }), + type: t.arg({ type: KBResourceType, required: false }), + status: t.arg({ type: KBIngestionStatus, required: false }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbResourcesConnection(args, ctx) + }, + }), + + getKbChatbotBindings: t.withAuth(asUser).field({ + nullable: false, + type: [KBChatbotBinding], + args: { kbId: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbChatbotBindings(args, ctx) + }, + }), + + getKbResourceIngestionRuns: t.withAuth(asUser).field({ + nullable: false, + type: [KBIngestionRun], + args: { resourceId: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbResourceIngestionRuns(args, ctx) + }, + }), + + getKbKnowledgeGraphConfig: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KBKnowledgeGraphConfigType, + args: { kbId: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbKnowledgeGraphConfig(args, ctx) + }, + }), + + getKbKnowledgeGraphOverview: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KnowledgeGraphResponseType, + args: { kbId: t.arg.id({ required: true }) }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbKnowledgeGraphOverview(args, ctx) + }, + }), + + searchKbKnowledgeGraph: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KnowledgeGraphResponseType, + args: { + kbId: t.arg.id({ required: true }), + query: t.arg.string({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.searchKbKnowledgeGraph(args, ctx) + }, + }), + + getKbKnowledgeGraphNeighbors: t.withAuth(asUserFullAccess).field({ + nullable: false, + type: KnowledgeGraphResponseType, + args: { + kbId: t.arg.id({ required: true }), + nodeId: t.arg.id({ required: true }), + }, + resolve: async (_, args, ctx) => { + return await KnowledgeService.getKbKnowledgeGraphNeighbors(args, ctx) + }, + }), + getAnswerCollectionsElements: t.withAuth(asUser).field({ nullable: true, type: [AnswerCollection], diff --git a/packages/graphql/src/schema/resource.ts b/packages/graphql/src/schema/resource.ts index 5d902971ec..b9b1472c28 100644 --- a/packages/graphql/src/schema/resource.ts +++ b/packages/graphql/src/schema/resource.ts @@ -166,8 +166,24 @@ export interface IChatbot { usageSummary?: IChatbotUsageSummary | null disclaimerSummary?: IChatbotDisclaimerSummary | null mcpConfigurations?: IChatbotMcpConfigurationSummary[] + enabledKnowledgeBase?: IChatbotKnowledgeBaseSummary | null } +export interface IChatbotKnowledgeBaseSummary { + id: string + name: string +} + +export const ChatbotKnowledgeBaseSummaryRef = + builder.objectRef('ChatbotKnowledgeBaseSummary') +export const ChatbotKnowledgeBaseSummary = + ChatbotKnowledgeBaseSummaryRef.implement({ + fields: (t) => ({ + id: t.exposeID('id'), + name: t.exposeString('name'), + }), + }) + export interface IChatbotPublic { id: string name: string @@ -305,6 +321,11 @@ export const Chatbot = ChatbotRef.implement({ type: [ChatbotMcpConfigurationSummaryRef], resolve: (chatbot) => chatbot.mcpConfigurations ?? [], }), + enabledKnowledgeBase: t.field({ + type: ChatbotKnowledgeBaseSummaryRef, + nullable: true, + resolve: (chatbot) => chatbot.enabledKnowledgeBase ?? null, + }), createdAt: t.expose('createdAt', { type: 'Date', nullable: true }), updatedAt: t.expose('updatedAt', { type: 'Date', nullable: true }), }), diff --git a/packages/graphql/src/scripts/2025-09-14_hatchet_tasks_activity_publication.ts b/packages/graphql/src/scripts/2025-09-14_hatchet_tasks_activity_publication.ts index 3516329805..32851e0797 100644 --- a/packages/graphql/src/scripts/2025-09-14_hatchet_tasks_activity_publication.ts +++ b/packages/graphql/src/scripts/2025-09-14_hatchet_tasks_activity_publication.ts @@ -1,4 +1,8 @@ -import { hatchetClient, prepareHatchetTasks } from '@klicker-uzh/hatchet' +import { + getKBGraphTerminalResult, + hatchetClient, + prepareHatchetTasks, +} from '@klicker-uzh/hatchet' import { prisma } from '@klicker-uzh/prisma' import { PublicationStatus } from '@klicker-uzh/prisma/client' import { EventEmitter } from 'events' @@ -10,6 +14,7 @@ import { handleRunningRandomGroupAssignments, handleUpdateGroupAverageScores, } from '../services/groups.js' +import { settleKbKnowledgeGraphResult } from '../services/knowledge.js' import { handleAssessmentLiveQuizBlockClosureAggregation, handlePublishScheduledLiveQuiz, @@ -74,6 +79,18 @@ async function run() { handleStandardLiveQuizBlockClosureAggregation, handleAssessmentLiveQuizBlockClosureAggregation, }, + getKBGraphTerminalResult, + settleKBGraphTerminalResult: ({ + buildId, + result, + finishedAt, + allowLateSuccess, + }) => + settleKbKnowledgeGraphResult( + prisma, + { buildId, result, allowLateSuccess }, + finishedAt + ), }) // get all live quizzes that are scheduled for publication and add a corresponding hatchet task instance diff --git a/packages/graphql/src/scripts/2025-09-17_remove_cache_data_past_assessment_quizzes.ts b/packages/graphql/src/scripts/2025-09-17_remove_cache_data_past_assessment_quizzes.ts index c36197323a..7481fcb30e 100644 --- a/packages/graphql/src/scripts/2025-09-17_remove_cache_data_past_assessment_quizzes.ts +++ b/packages/graphql/src/scripts/2025-09-17_remove_cache_data_past_assessment_quizzes.ts @@ -1,4 +1,8 @@ -import { hatchetClient, prepareHatchetTasks } from '@klicker-uzh/hatchet' +import { + getKBGraphTerminalResult, + hatchetClient, + prepareHatchetTasks, +} from '@klicker-uzh/hatchet' import { prisma } from '@klicker-uzh/prisma' import { PublicationStatus } from '@klicker-uzh/prisma/client' import dayjs from 'dayjs' @@ -11,6 +15,7 @@ import { handleRunningRandomGroupAssignments, handleUpdateGroupAverageScores, } from '../services/groups.js' +import { settleKbKnowledgeGraphResult } from '../services/knowledge.js' import { handleAssessmentLiveQuizBlockClosureAggregation, handlePublishScheduledLiveQuiz, @@ -68,6 +73,18 @@ async function run() { handleStandardLiveQuizBlockClosureAggregation, handleAssessmentLiveQuizBlockClosureAggregation, }, + getKBGraphTerminalResult, + settleKBGraphTerminalResult: ({ + buildId, + result, + finishedAt, + allowLateSuccess, + }) => + settleKbKnowledgeGraphResult( + prisma, + { buildId, result, allowLateSuccess }, + finishedAt + ), }) // find all ended assessment live quizzes diff --git a/packages/graphql/src/scripts/setupLocalBlobStorage.ts b/packages/graphql/src/scripts/setupLocalBlobStorage.ts new file mode 100644 index 0000000000..1a6f15e48a --- /dev/null +++ b/packages/graphql/src/scripts/setupLocalBlobStorage.ts @@ -0,0 +1,59 @@ +import { + BlobServiceClient, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' + +const accountName = process.env.BLOB_STORAGE_ACCOUNT_NAME?.trim() +const accessKey = process.env.BLOB_STORAGE_ACCESS_KEY?.trim() +const manageUrl = process.env.NEXT_PUBLIC_MANAGE_URL?.trim() + +if (!accountName || !accessKey || !manageUrl) { + throw new Error('Local Blob storage configuration is incomplete') +} + +const publicAccountUrl = getBlobStorageAccountUrl( + accountName, + process.env.BLOB_STORAGE_ACCOUNT_URL +) +const setupAccountUrl = getBlobStorageAccountUrl( + accountName, + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? publicAccountUrl +) +const publicStorageUrl = new URL(publicAccountUrl) +const setupStorageUrl = new URL(setupAccountUrl) +const manageOrigin = new URL(manageUrl).origin + +if ( + publicStorageUrl.hostname !== 'localhost' && + !publicStorageUrl.hostname.endsWith('.localhost') +) { + throw new Error('Refusing to configure CORS on non-local Blob storage') +} + +if ( + setupStorageUrl.hostname !== 'azurite' && + !setupStorageUrl.hostname.endsWith('-azurite') && + setupStorageUrl.hostname !== 'localhost' && + !setupStorageUrl.hostname.endsWith('.localhost') +) { + throw new Error('Refusing to configure a non-local Blob storage endpoint') +} + +const serviceClient = new BlobServiceClient( + setupAccountUrl, + new StorageSharedKeyCredential(accountName, accessKey) +) +await serviceClient.setProperties({ + cors: [ + { + allowedOrigins: manageOrigin, + allowedMethods: 'DELETE,GET,HEAD,OPTIONS,PUT', + allowedHeaders: '*', + exposedHeaders: 'x-ms-*', + maxAgeInSeconds: 3600, + }, + ], +}) + +console.info(`[blob-storage] Azurite CORS configured for ${manageOrigin}`) diff --git a/packages/graphql/src/services/chatbots.ts b/packages/graphql/src/services/chatbots.ts index 4a2e480a0c..6df4dc57f9 100644 --- a/packages/graphql/src/services/chatbots.ts +++ b/packages/graphql/src/services/chatbots.ts @@ -305,6 +305,13 @@ export async function getChatbotsInfo(ctx: ContextWithUser) { }, }, }, + knowledgeBases: { + where: { isEnabled: true }, + select: { + kb: { select: { id: true, name: true } }, + }, + take: 1, + }, }, orderBy: { updatedAt: 'desc' }, }) @@ -434,6 +441,7 @@ export async function getChatbotsInfo(ctx: ContextWithUser) { usageSummary, disclaimerSummary, mcpConfigurations, + enabledKnowledgeBase: chatbot.knowledgeBases[0]?.kb ?? null, } }) } diff --git a/packages/graphql/src/services/elements.ts b/packages/graphql/src/services/elements.ts index eeb98f18c6..f9ec939a95 100644 --- a/packages/graphql/src/services/elements.ts +++ b/packages/graphql/src/services/elements.ts @@ -14,6 +14,7 @@ import { SortByType, } from '@klicker-uzh/types' import { + getBlobStorageAccountUrl, getInitialInstanceResults, PrismaTransactionClient, processElementData, @@ -1115,12 +1116,20 @@ export async function getFileUploadSas( process.env.BLOB_STORAGE_ACCESS_KEY as string ) - const storageAccount = `https://${ - process.env.BLOB_STORAGE_ACCOUNT_NAME as string - }.blob.core.windows.net` + const storageAccount = getBlobStorageAccountUrl( + process.env.BLOB_STORAGE_ACCOUNT_NAME as string, + process.env.BLOB_STORAGE_ACCOUNT_URL + ) + const internalStorageAccount = getBlobStorageAccountUrl( + process.env.BLOB_STORAGE_ACCOUNT_NAME as string, + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? storageAccount + ) // if nonexistent, create a container for the user on blob storage - const client = new BlobServiceClient(storageAccount, sharedKeyCredential) + const client = new BlobServiceClient( + internalStorageAccount, + sharedKeyCredential + ) const containerClient = client.getContainerClient(ctx.user.sub) if (!(await containerClient.exists())) { client.createContainer(ctx.user.sub, { diff --git a/packages/graphql/src/services/kbGraphContract.ts b/packages/graphql/src/services/kbGraphContract.ts new file mode 100644 index 0000000000..fd70129429 --- /dev/null +++ b/packages/graphql/src/services/kbGraphContract.ts @@ -0,0 +1,267 @@ +import { z } from 'zod' + +export const KB_GRAPH_CONTRACT_VERSION = 'klicker-kb-graph/v1' as const +export const KB_GRAPH_DATABASE_INT_MAX = 2_147_483_647 as const + +const safeIdentifier = z + .string() + .trim() + .regex(/^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/) + +const safeModelIdentifier = z + .string() + .trim() + .regex(/^[A-Za-z0-9][A-Za-z0-9._:/-]{0,127}$/) + +const uuidString = z + .string() + .uuid() + .transform((value) => value.toLowerCase()) + +const sha256String = z + .string() + .regex(/^[a-f0-9]{64}$/i) + .transform((value) => value.toLowerCase()) + +const artifactName = z + .string() + .trim() + .transform((value) => value.replace(/^\/+|\/+$/g, '')) + .refine((value) => value.length > 0) + +const graphMlArtifactSchema = z + .object({ + container_name: artifactName, + blob_name: artifactName, + }) + .strict() + +const meteredCostComponentSchema = z + .object({ + provider: safeIdentifier, + model: safeModelIdentifier, + amount_minor_units: z.number().int().min(0).max(KB_GRAPH_DATABASE_INT_MAX), + pricing_version: safeIdentifier, + embedding_tokens: z + .number() + .int() + .min(0) + .max(KB_GRAPH_DATABASE_INT_MAX) + .default(0), + input_tokens: z + .number() + .int() + .min(0) + .max(KB_GRAPH_DATABASE_INT_MAX) + .default(0), + output_tokens: z + .number() + .int() + .min(0) + .max(KB_GRAPH_DATABASE_INT_MAX) + .default(0), + request_count: z + .number() + .int() + .min(0) + .max(KB_GRAPH_DATABASE_INT_MAX) + .default(0), + }) + .strict() + +const meteredCostSchema = z + .object({ + currency: z + .string() + .trim() + .regex(/^[A-Z]{3}$/i) + .transform((value) => value.toUpperCase()), + amount_minor_units: z.number().int().min(0).max(KB_GRAPH_DATABASE_INT_MAX), + components: z.array(meteredCostComponentSchema).min(1), + metering_source: z.enum(['provider_reported', 'configured_pricing']), + }) + .strict() + .superRefine((value, context) => { + let componentTotal = 0 + for (const component of value.components) { + if ( + componentTotal > + KB_GRAPH_DATABASE_INT_MAX - component.amount_minor_units + ) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'component amount total exceeds the database integer range', + path: ['components'], + }) + return + } + componentTotal += component.amount_minor_units + } + if (componentTotal !== value.amount_minor_units) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'amount_minor_units must equal the component total', + path: ['amount_minor_units'], + }) + } + }) + +export const kbGraphTerminalResultSchema = z + .object({ + contract_version: z.literal(KB_GRAPH_CONTRACT_VERSION), + result_id: z.string().trim().min(1), + build_id: uuidString, + kb_id: uuidString, + owner_id: uuidString, + run_id: safeIdentifier, + source_content_digest: sha256String, + graph_name: safeIdentifier, + status: z.enum([ + 'SUCCEEDED', + 'FAILED', + 'CANCELLED', + 'TIMED_OUT', + 'NEEDS_HUMAN_REVIEW', + ]), + edge_count: z.number().int().min(0).default(0), + error_code: safeIdentifier.nullable().default(null), + failed_document_count: z.number().int().min(0).default(0), + graphml_artifact: graphMlArtifactSchema.nullable().default(null), + metered_cost: meteredCostSchema.nullable().default(null), + node_count: z.number().int().min(0).default(0), + processed_document_count: z.number().int().min(0).default(0), + }) + .strict() + .superRefine((value, context) => { + if (value.result_id !== `${value.build_id}:${value.run_id}`) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'result_id must be derived from build_id and run_id', + path: ['result_id'], + }) + } + if (value.graph_name !== `klickeruzh:kb:${value.kb_id}:${value.build_id}`) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'graph_name must match the pinned KB/build identity', + path: ['graph_name'], + }) + } + if (value.status === 'SUCCEEDED') { + if (value.graphml_artifact === null) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'SUCCEEDED results require a GraphML artifact', + path: ['graphml_artifact'], + }) + } + if (value.metered_cost === null) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'SUCCEEDED results require metered_cost', + path: ['metered_cost'], + }) + } + if (value.error_code !== null) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'SUCCEEDED results cannot carry an error_code', + path: ['error_code'], + }) + } + } else if (value.error_code === null) { + context.addIssue({ + code: z.ZodIssueCode.custom, + message: 'non-success terminal results require error_code', + path: ['error_code'], + }) + } + }) + +export type KbGraphTerminalResult = z.infer + +export interface KbGraphTerminalResultExpectation { + buildId: string + kbId: string + ownerId: string + resultId: string + runId?: string + estimatedMinorUnits: number +} + +export type KbGraphTerminalResultValidation = + | { ok: true; result: KbGraphTerminalResult } + | { ok: false; errors: string[] } + +export function validateKbGraphTerminalResult( + value: unknown, + expectation: KbGraphTerminalResultExpectation +): KbGraphTerminalResultValidation { + const parsed = kbGraphTerminalResultSchema.safeParse(value) + if (!parsed.success) { + return { + ok: false, + errors: parsed.error.issues.map( + (issue) => `${issue.path.join('.') || ''}: ${issue.message}` + ), + } + } + + const result = parsed.data + const errors: string[] = [] + + if (result.build_id !== expectation.buildId) { + errors.push( + `build_id mismatch: expected ${expectation.buildId}, got ${result.build_id}` + ) + } + if (result.kb_id !== expectation.kbId) { + errors.push( + `kb_id mismatch: expected ${expectation.kbId}, got ${result.kb_id}` + ) + } + if (result.owner_id !== expectation.ownerId) { + errors.push( + `owner_id mismatch: expected ${expectation.ownerId}, got ${result.owner_id}` + ) + } + if (result.result_id !== expectation.resultId) { + errors.push( + `result_id mismatch: expected ${expectation.resultId}, got ${result.result_id}` + ) + } + if (expectation.runId !== undefined && result.run_id !== expectation.runId) { + errors.push( + `run_id mismatch: expected ${expectation.runId}, got ${result.run_id}` + ) + } + + const expectedGraphName = `klickeruzh:kb:${expectation.kbId}:${expectation.buildId}` + if (result.graph_name !== expectedGraphName) { + errors.push( + `graph_name mismatch: expected ${expectedGraphName}, got ${result.graph_name}` + ) + } + + if ( + !Number.isInteger(expectation.estimatedMinorUnits) || + expectation.estimatedMinorUnits < 0 || + expectation.estimatedMinorUnits > KB_GRAPH_DATABASE_INT_MAX + ) { + errors.push( + `estimatedMinorUnits must be a non-negative integer, got ${expectation.estimatedMinorUnits}` + ) + } else if ( + result.metered_cost !== null && + result.metered_cost.amount_minor_units > expectation.estimatedMinorUnits + ) { + errors.push( + `metered cost ${result.metered_cost.amount_minor_units} exceeds estimated reservation ${expectation.estimatedMinorUnits}` + ) + } + + if (errors.length > 0) { + return { ok: false, errors } + } + return { ok: true, result } +} diff --git a/packages/graphql/src/services/knowledge.ts b/packages/graphql/src/services/knowledge.ts new file mode 100644 index 0000000000..1b600ef97f --- /dev/null +++ b/packages/graphql/src/services/knowledge.ts @@ -0,0 +1,2312 @@ +import { + BlobSASPermissions, + BlobServiceClient, + generateBlobSASQueryParameters, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import { + computeKBContentDigest, + getKnowledgeGraphName, + getPublishedKnowledgeGraph, + hashKBContentDigestEntries, + KnowledgeGraphNotPublishedError, + type PublishedKnowledgeGraph, + readKnowledgeGraphNeighbors, + readKnowledgeGraphOverview, + searchKnowledgeGraph, +} from '@klicker-uzh/knowledge-graph' +import * as DB from '@klicker-uzh/prisma/client' +import type { + DeleteKBResourceInput, + IngestKBResourceInput, + KnowledgeGraphResponse, +} from '@klicker-uzh/types' +import { + MAX_KB_RESOURCE_COUNT, + MAX_KB_SOURCE_SIZE_BYTES, + MAX_KB_TOTAL_SIZE_BYTES, +} from '@klicker-uzh/types' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' +import { normalizePublicHttpUrl } from '@klicker-uzh/util/public-url' +import { createHash, randomUUID } from 'crypto' +import { GraphQLError } from 'graphql' +import { validate as validateUuid } from 'uuid' +import type { ContextWithUser } from '../lib/context.js' +import { + getKBGraphRemainingQuota, + releaseKBGraphCostReservation, + reserveKBGraphCost, + settleKBGraphBuildCost, +} from './knowledgeGraphAccounting.js' +import { + getKBGraphBillingLabel, + getKBGraphCostConfiguration, + requireKBGraphCostConfiguration, +} from './knowledgeGraphCost.js' + +const MAX_KB_FILE_SIZE_BYTES = MAX_KB_SOURCE_SIZE_BYTES +const KB_DELETE_QUEUE_CONCURRENCY = 8 +const KB_DEFAULT_PAGE_SIZE = 20 +const KB_MAX_PAGE_SIZE = 50 +const KB_BULK_DELETE_LIMIT = 50 +const KB_CURSOR_VERSION = 1 +const KB_MCP_SERVER_NAME = 'KB' +const KB_MCP_CHAT_MODES = ['tutor', 'explainer'] as const +const KB_FILE_TYPES: Record = { + pdf: ['application/pdf'], + txt: ['text/plain'], + md: ['text/plain'], +} + +type KBPaginationKind = 'knowledge-bases' | 'resources' + +interface KBPaginationCursor { + version: number + kind: KBPaginationKind + filterHash: string + timestamp: string + id: string +} + +export interface KBPageInfo { + hasNextPage: boolean + endCursor: string | null +} + +export interface KBMetrics { + visibleResourceCount: number + visibleSizeBytes: number + unknownSizeResourceCount: number + quotaResourceCount: number + quotaSizeBytes: number + resourceLimit: number + storageLimitBytes: number + pendingCleanupCount: number + pendingCleanupSizeBytes: number + reservedResourceCount: number + reservedSizeBytes: number + linkedConsumerCount: number +} + +export interface KBWithMetrics extends DB.KB { + metrics: KBMetrics +} + +export interface KBConnection { + items: KBWithMetrics[] + pageInfo: KBPageInfo + totalCount: number +} + +export interface KBResourceConnection { + items: Array + pageInfo: KBPageInfo + totalCount: number +} + +function invalidPaginationInput(message: string): never { + throw new GraphQLError(message, { + extensions: { code: 'BAD_USER_INPUT' }, + }) +} + +function normalizePageSize(first: number | null | undefined) { + const pageSize = first ?? KB_DEFAULT_PAGE_SIZE + if ( + !Number.isSafeInteger(pageSize) || + pageSize < 1 || + pageSize > KB_MAX_PAGE_SIZE + ) { + invalidPaginationInput('KB page size is invalid') + } + return pageSize +} + +function normalizeSearch(search: string | null | undefined) { + const normalized = search?.trim().replace(/\s+/g, ' ') ?? '' + if (normalized.length > 200) { + invalidPaginationInput('KB search is too long') + } + return normalized +} + +function getFilterHash(filters: Record) { + return createHash('sha256') + .update(JSON.stringify(filters)) + .digest('base64url') +} + +function encodePaginationCursor(cursor: KBPaginationCursor) { + return Buffer.from(JSON.stringify(cursor)).toString('base64url') +} + +function decodePaginationCursor( + value: string | null | undefined, + expectedKind: KBPaginationKind, + expectedFilterHash: string +) { + if (!value) return null + if (value.length > 2048 || !/^[A-Za-z0-9_-]+$/.test(value)) { + invalidPaginationInput('KB pagination cursor is invalid') + } + + try { + const parsed = JSON.parse( + Buffer.from(value, 'base64url').toString('utf8') + ) as Partial + const timestamp = new Date(parsed.timestamp ?? '') + if ( + parsed.version !== KB_CURSOR_VERSION || + parsed.kind !== expectedKind || + parsed.filterHash !== expectedFilterHash || + !validateUuid(parsed.id ?? '') || + Number.isNaN(timestamp.getTime()) || + timestamp.toISOString() !== parsed.timestamp + ) { + invalidPaginationInput('KB pagination cursor is invalid') + } + return { + timestamp, + id: parsed.id!, + } + } catch (error) { + if (error instanceof GraphQLError) throw error + invalidPaginationInput('KB pagination cursor is invalid') + } +} + +function createPaginationResult( + items: T[], + pageSize: number, + cursorForItem: (item: T) => KBPaginationCursor, + totalCount: number +) { + const hasNextPage = items.length > pageSize + const pageItems = hasNextPage ? items.slice(0, pageSize) : items + const lastItem = pageItems.at(-1) + return { + items: pageItems, + pageInfo: { + hasNextPage, + endCursor: lastItem + ? encodePaginationCursor(cursorForItem(lastItem)) + : null, + }, + totalCount, + } +} + +function getKbContainerName(userId: string) { + return `kb-${userId}` +} + +function getKbBlobContainer(userId: string) { + const accountName = process.env.BLOB_STORAGE_ACCOUNT_NAME + const accessKey = process.env.BLOB_STORAGE_ACCESS_KEY + if (!accountName || !accessKey) { + throw new GraphQLError('Blob storage is not configured') + } + + const credential = new StorageSharedKeyCredential(accountName, accessKey) + const accountUrl = getBlobStorageAccountUrl( + accountName, + process.env.BLOB_STORAGE_ACCOUNT_URL + ) + const internalAccountUrl = getBlobStorageAccountUrl( + accountName, + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? accountUrl + ) + const serviceClient = new BlobServiceClient(internalAccountUrl, credential) + + return { + containerClient: serviceClient.getContainerClient( + getKbContainerName(userId) + ), + accountUrl, + credential, + } +} + +function validateKbFile({ + fileName, + contentType, + sizeBytes, +}: { + fileName: string + contentType: string + sizeBytes: number +}) { + if ( + !Number.isSafeInteger(sizeBytes) || + sizeBytes <= 0 || + sizeBytes > MAX_KB_FILE_SIZE_BYTES + ) { + throw new GraphQLError('KB file size is invalid') + } + + const extension = fileName.trim().split('.').pop()?.toLowerCase() + const normalizedContentType = contentType.trim().toLowerCase() + if ( + !extension || + !KB_FILE_TYPES[extension]?.includes(normalizedContentType) + ) { + throw new GraphQLError('KB file type is not supported') + } + + return { extension, contentType: normalizedContentType } +} + +function validateKbResourceTitle(title: string) { + const normalizedTitle = title.trim() + if (!normalizedTitle) { + throw new GraphQLError('KB resource title is required') + } + return normalizedTitle +} + +async function getKbQuotaUsage( + prisma: DB.Prisma.TransactionClient, + kbId: string +) { + const [resources, unknownSizeResources, uploadTickets] = await Promise.all([ + prisma.kBResource.aggregate({ + where: { kbId }, + _count: { _all: true }, + _sum: { sizeBytes: true }, + }), + prisma.kBResource.count({ + where: { kbId, sizeBytes: null }, + }), + prisma.kBUploadTicket.aggregate({ + where: { kbId }, + _count: { _all: true }, + _sum: { sizeBytes: true }, + }), + ]) + + return { + resourceCount: resources._count._all + uploadTickets._count._all, + sizeBytes: + (resources._sum.sizeBytes ?? 0) + + unknownSizeResources * MAX_KB_FILE_SIZE_BYTES + + (uploadTickets._sum.sizeBytes ?? 0), + } +} + +async function assertKbQuotaAvailable( + prisma: DB.Prisma.TransactionClient, + { + kbId, + resourceCount = 0, + sizeBytes = 0, + }: { + kbId: string + resourceCount?: number + sizeBytes?: number + } +) { + const usage = await getKbQuotaUsage(prisma, kbId) + if (usage.resourceCount + resourceCount > MAX_KB_RESOURCE_COUNT) { + throw new GraphQLError('KB resource limit reached', { + extensions: { code: 'KB_RESOURCE_LIMIT_REACHED' }, + }) + } + if (usage.sizeBytes + sizeBytes > MAX_KB_TOTAL_SIZE_BYTES) { + throw new GraphQLError('KB storage limit reached', { + extensions: { code: 'KB_STORAGE_LIMIT_REACHED' }, + }) + } +} + +async function assertKbPreviewAccess(ctx: ContextWithUser) { + const user = await ctx.prisma.user.findUnique({ + where: { id: ctx.user.sub }, + select: { privatePreview: true }, + }) + if (!user?.privatePreview) { + throw new GraphQLError('KB workspace preview access is required', { + extensions: { code: 'KB_PREVIEW_ACCESS_REQUIRED' }, + }) + } +} + +function assertKbIngestionEnabled() { + if (process.env.KB_INGESTION_DISABLED === 'true') { + throw new GraphQLError('KB ingestion is currently disabled', { + extensions: { code: 'KB_INGESTION_DISABLED' }, + }) + } +} + +function assertKbGraphGenerationEnabled() { + if (process.env.KB_GRAPH_DISABLED === 'true') { + throw new GraphQLError('KB graph generation is currently disabled', { + extensions: { code: 'KB_GRAPH_DISABLED' }, + }) + } +} + +async function getOwnedKbOrThrow(ctx: ContextWithUser, id: string) { + const kb = await ctx.prisma.kB.findFirst({ + where: { id, deletedAt: null }, + }) + if (!kb || kb.ownerId !== ctx.user.sub) { + throw new GraphQLError('KB not found') + } + return kb +} + +async function getOwnedKbResourceOrThrow(ctx: ContextWithUser, id: string) { + const resource = await ctx.prisma.kBResource.findFirst({ + where: { + id, + deletedAt: null, + kb: { ownerId: ctx.user.sub, deletedAt: null }, + }, + }) + if (!resource) { + throw new GraphQLError('KB resource not found') + } + return resource +} + +async function lockOwnedKbOrThrow( + prisma: DB.Prisma.TransactionClient, + id: string, + ownerId: string +) { + const lockedKb = await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KB" + WHERE "id" = CAST(${id} AS UUID) + AND "ownerId" = CAST(${ownerId} AS UUID) + AND "deletedAt" IS NULL + FOR UPDATE + ` + if (lockedKb.length === 0) { + throw new GraphQLError('KB not found') + } +} + +async function lockOwnedKbResourceOrThrow( + prisma: DB.Prisma.TransactionClient, + id: string, + ownerId: string +) { + const lockedResource = await prisma.$queryRaw>` + SELECT resource."id" + FROM "public"."KBResource" AS resource + INNER JOIN "public"."KB" AS kb ON kb."id" = resource."kbId" + WHERE resource."id" = CAST(${id} AS UUID) + AND kb."ownerId" = CAST(${ownerId} AS UUID) + AND resource."deletedAt" IS NULL + AND kb."deletedAt" IS NULL + FOR UPDATE OF resource + ` + if (lockedResource.length === 0) { + throw new GraphQLError('KB resource not found') + } +} + +async function lockKbResourceInKbOrThrow( + prisma: DB.Prisma.TransactionClient, + kbId: string, + resourceId: string +) { + const lockedResource = await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KBResource" + WHERE "id" = CAST(${resourceId} AS UUID) + AND "kbId" = CAST(${kbId} AS UUID) + AND "deletedAt" IS NULL + FOR UPDATE + ` + if (lockedResource.length === 0) { + throw new GraphQLError('KB resource not found') + } +} + +async function lockOwnedKbForResourceOrThrow( + prisma: DB.Prisma.TransactionClient, + resourceId: string, + ownerId: string +) { + const lockedKb = await prisma.$queryRaw>` + SELECT kb."id" AS "kbId" + FROM "public"."KB" AS kb + INNER JOIN "public"."KBResource" AS resource ON resource."kbId" = kb."id" + WHERE resource."id" = CAST(${resourceId} AS UUID) + AND kb."ownerId" = CAST(${ownerId} AS UUID) + AND kb."deletedAt" IS NULL + AND resource."deletedAt" IS NULL + FOR UPDATE OF kb + ` + if (lockedKb.length === 0) { + throw new GraphQLError('KB resource not found') + } + return lockedKb[0]!.kbId +} + +async function lockOwnedChatbotOrThrow( + prisma: DB.Prisma.TransactionClient, + id: string, + ownerId: string +) { + const lockedChatbot = await prisma.$queryRaw>` + SELECT "id" + FROM "public"."Chatbot" + WHERE "id" = CAST(${id} AS UUID) + AND "ownerId" = CAST(${ownerId} AS UUID) + FOR UPDATE + ` + if (lockedChatbot.length === 0) { + throw new GraphQLError('Chatbot not found') + } +} + +async function getKbMcpServerOrThrow(prisma: DB.Prisma.TransactionClient) { + const mcpServer = await prisma.chatbotMCPServer.findUnique({ + where: { name: KB_MCP_SERVER_NAME }, + select: { id: true, isActive: true }, + }) + if (!mcpServer || !mcpServer.isActive) { + throw new GraphQLError('Knowledge base retrieval is not configured') + } + return mcpServer +} + +function createKbMetrics({ + visibleResourceCount = 0, + visibleSizeBytes = 0, + visibleUnknownSizeCount = 0, + retainedResourceCount = 0, + retainedSizeBytes = 0, + retainedUnknownSizeCount = 0, + reservedResourceCount = 0, + reservedSizeBytes = 0, + linkedConsumerCount = 0, +}: Partial<{ + visibleResourceCount: number + visibleSizeBytes: number + visibleUnknownSizeCount: number + retainedResourceCount: number + retainedSizeBytes: number + retainedUnknownSizeCount: number + reservedResourceCount: number + reservedSizeBytes: number + linkedConsumerCount: number +}> = {}): KBMetrics { + const quotaRetainedSizeBytes = + retainedSizeBytes + retainedUnknownSizeCount * MAX_KB_FILE_SIZE_BYTES + const quotaVisibleSizeBytes = + visibleSizeBytes + visibleUnknownSizeCount * MAX_KB_FILE_SIZE_BYTES + + return { + visibleResourceCount, + visibleSizeBytes, + unknownSizeResourceCount: visibleUnknownSizeCount, + quotaResourceCount: retainedResourceCount + reservedResourceCount, + quotaSizeBytes: quotaRetainedSizeBytes + reservedSizeBytes, + resourceLimit: MAX_KB_RESOURCE_COUNT, + storageLimitBytes: MAX_KB_TOTAL_SIZE_BYTES, + pendingCleanupCount: retainedResourceCount - visibleResourceCount, + pendingCleanupSizeBytes: quotaRetainedSizeBytes - quotaVisibleSizeBytes, + reservedResourceCount, + reservedSizeBytes, + linkedConsumerCount, + } +} + +async function getKbMetricsMap( + prisma: DB.Prisma.TransactionClient | ContextWithUser['prisma'], + kbIds: string[] +) { + if (kbIds.length === 0) return new Map() + + const [ + visibleResources, + visibleUnknownSizes, + retainedResources, + retainedUnknownSizes, + uploadTickets, + linkedConsumers, + ] = await Promise.all([ + prisma.kBResource.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds }, deletedAt: null }, + _count: { _all: true }, + _sum: { sizeBytes: true }, + }), + prisma.kBResource.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds }, deletedAt: null, sizeBytes: null }, + _count: { _all: true }, + }), + prisma.kBResource.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds } }, + _count: { _all: true }, + _sum: { sizeBytes: true }, + }), + prisma.kBResource.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds }, sizeBytes: null }, + _count: { _all: true }, + }), + prisma.kBUploadTicket.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds } }, + _count: { _all: true }, + _sum: { sizeBytes: true }, + }), + prisma.kBChatbot.groupBy({ + by: ['kbId'], + where: { kbId: { in: kbIds }, isEnabled: true }, + _count: { _all: true }, + }), + ]) + + const visibleByKb = new Map(visibleResources.map((row) => [row.kbId, row])) + const visibleUnknownByKb = new Map( + visibleUnknownSizes.map((row) => [row.kbId, row._count._all]) + ) + const retainedByKb = new Map(retainedResources.map((row) => [row.kbId, row])) + const retainedUnknownByKb = new Map( + retainedUnknownSizes.map((row) => [row.kbId, row._count._all]) + ) + const ticketsByKb = new Map(uploadTickets.map((row) => [row.kbId, row])) + const consumersByKb = new Map( + linkedConsumers.map((row) => [row.kbId, row._count._all]) + ) + + return new Map( + kbIds.map((kbId) => { + const visible = visibleByKb.get(kbId) + const retained = retainedByKb.get(kbId) + const tickets = ticketsByKb.get(kbId) + return [ + kbId, + createKbMetrics({ + visibleResourceCount: visible?._count._all, + visibleSizeBytes: visible?._sum.sizeBytes ?? 0, + visibleUnknownSizeCount: visibleUnknownByKb.get(kbId), + retainedResourceCount: retained?._count._all, + retainedSizeBytes: retained?._sum.sizeBytes ?? 0, + retainedUnknownSizeCount: retainedUnknownByKb.get(kbId), + reservedResourceCount: tickets?._count._all, + reservedSizeBytes: tickets?._sum.sizeBytes ?? 0, + linkedConsumerCount: consumersByKb.get(kbId), + }), + ] + }) + ) +} + +async function getKbMetrics( + prisma: DB.Prisma.TransactionClient | ContextWithUser['prisma'], + kbId: string +): Promise { + const metrics = await getKbMetricsMap(prisma, [kbId]) + return metrics.get(kbId) ?? createKbMetrics() +} + +export async function getUserKbsConnection( + { + first, + after, + search, + }: { + first?: number | null + after?: string | null + search?: string | null + }, + ctx: ContextWithUser +): Promise { + await assertKbPreviewAccess(ctx) + const pageSize = normalizePageSize(first) + const normalizedSearch = normalizeSearch(search) + const filterHash = getFilterHash({ + ownerId: ctx.user.sub, + search: normalizedSearch, + }) + const cursor = decodePaginationCursor(after, 'knowledge-bases', filterHash) + const searchWhere: DB.Prisma.KBWhereInput = normalizedSearch + ? { + OR: [ + { name: { contains: normalizedSearch, mode: 'insensitive' } }, + { + description: { + contains: normalizedSearch, + mode: 'insensitive', + }, + }, + ], + } + : {} + const where: DB.Prisma.KBWhereInput = { + ownerId: ctx.user.sub, + deletedAt: null, + ...searchWhere, + ...(cursor + ? { + AND: [ + searchWhere, + { + OR: [ + { updatedAt: { lt: cursor.timestamp } }, + { + updatedAt: cursor.timestamp, + id: { lt: cursor.id }, + }, + ], + }, + ], + } + : {}), + } + + const [items, totalCount] = await Promise.all([ + ctx.prisma.kB.findMany({ + where, + orderBy: [{ updatedAt: 'desc' }, { id: 'desc' }], + take: pageSize + 1, + }), + ctx.prisma.kB.count({ + where: { + ownerId: ctx.user.sub, + deletedAt: null, + ...searchWhere, + }, + }), + ]) + const metrics = await getKbMetricsMap( + ctx.prisma, + items.map(({ id }) => id) + ) + const itemsWithMetrics = items.map((kb) => ({ + ...kb, + resources: [], + metrics: metrics.get(kb.id) ?? createKbMetrics(), + })) + + return createPaginationResult( + itemsWithMetrics, + pageSize, + (kb) => ({ + version: KB_CURSOR_VERSION, + kind: 'knowledge-bases', + filterHash, + timestamp: kb.updatedAt.toISOString(), + id: kb.id, + }), + totalCount + ) +} + +export async function getKb({ id }: { id: string }, ctx: ContextWithUser) { + await assertKbPreviewAccess(ctx) + const kb = await ctx.prisma.kB.findFirst({ + where: { id, ownerId: ctx.user.sub, deletedAt: null }, + }) + if (!kb) { + throw new GraphQLError('KB not found') + } + return { + ...kb, + metrics: await getKbMetrics(ctx.prisma, kb.id), + } satisfies KBWithMetrics +} + +export async function getKbResourcesConnection( + { + kbId, + first, + after, + search, + type, + status, + }: { + kbId: string + first?: number | null + after?: string | null + search?: string | null + type?: DB.KBResourceType | null + status?: DB.KBIngestionStatus | null + }, + ctx: ContextWithUser +): Promise { + await assertKbPreviewAccess(ctx) + await getOwnedKbOrThrow(ctx, kbId) + const pageSize = normalizePageSize(first) + const normalizedSearch = normalizeSearch(search) + const filterHash = getFilterHash({ + ownerId: ctx.user.sub, + kbId, + search: normalizedSearch, + type: type ?? null, + status: status ?? null, + }) + const cursor = decodePaginationCursor(after, 'resources', filterHash) + const operationStatusResourceIds = status + ? await ctx.prisma.$queryRaw>` + SELECT resource."id" + FROM "public"."KBResource" AS resource + INNER JOIN "public"."KBIngestionRun" AS run + ON run."id" = resource."ingestionAttemptId" + WHERE resource."kbId" = CAST(${kbId} AS UUID) + AND run."status" = CAST(${status} AS "KBIngestionStatus") + ` + : null + const searchWhere: DB.Prisma.KBResourceWhereInput = normalizedSearch + ? { + OR: [ + { title: { contains: normalizedSearch, mode: 'insensitive' } }, + { + originalFilename: { + contains: normalizedSearch, + mode: 'insensitive', + }, + }, + { + sourceUrl: { + contains: normalizedSearch, + mode: 'insensitive', + }, + }, + ], + } + : {} + const baseWhere: DB.Prisma.KBResourceWhereInput = { + kbId, + kb: { + is: { + ownerId: ctx.user.sub, + deletedAt: null, + }, + }, + deletedAt: null, + ...(type ? { type } : {}), + ...(operationStatusResourceIds + ? { id: { in: operationStatusResourceIds.map(({ id }) => id) } } + : {}), + ...searchWhere, + } + const where: DB.Prisma.KBResourceWhereInput = cursor + ? { + AND: [ + baseWhere, + { + OR: [ + { createdAt: { lt: cursor.timestamp } }, + { + createdAt: cursor.timestamp, + id: { lt: cursor.id }, + }, + ], + }, + ], + } + : baseWhere + + const [items, totalCount] = await Promise.all([ + ctx.prisma.kBResource.findMany({ + where, + orderBy: [{ createdAt: 'desc' }, { id: 'desc' }], + take: pageSize + 1, + }), + ctx.prisma.kBResource.count({ where: baseWhere }), + ]) + const currentAttemptIds = items.flatMap(({ ingestionAttemptId }) => + ingestionAttemptId ? [ingestionAttemptId] : [] + ) + const currentRuns = + currentAttemptIds.length === 0 + ? [] + : await ctx.prisma.kBIngestionRun.findMany({ + where: { id: { in: currentAttemptIds } }, + }) + const currentRunsById = new Map(currentRuns.map((run) => [run.id, run])) + const itemsWithCurrentRuns = items.map((resource) => { + const currentRun = resource.ingestionAttemptId + ? currentRunsById.get(resource.ingestionAttemptId) + : undefined + + // A platform refresh appends a historic ledger row but must not replace + // the lecturer operation currently recorded on the resource. + return { + ...resource, + ingestionRuns: currentRun ? [currentRun] : [], + } + }) + + return createPaginationResult( + itemsWithCurrentRuns, + pageSize, + (resource) => ({ + version: KB_CURSOR_VERSION, + kind: 'resources', + filterHash, + timestamp: resource.createdAt.toISOString(), + id: resource.id, + }), + totalCount + ) +} + +export async function getKbChatbotBindings( + { kbId }: { kbId: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + await getOwnedKbOrThrow(ctx, kbId) + + const chatbots = await ctx.prisma.chatbot.findMany({ + where: { ownerId: ctx.user.sub }, + select: { + id: true, + name: true, + knowledgeBases: { + where: { isEnabled: true }, + select: { + kb: { select: { id: true, name: true } }, + }, + take: 1, + }, + }, + orderBy: { name: 'asc' }, + }) + + return chatbots.map((chatbot) => ({ + chatbotId: chatbot.id, + chatbotName: chatbot.name, + enabledKbId: chatbot.knowledgeBases[0]?.kb.id ?? null, + enabledKbName: chatbot.knowledgeBases[0]?.kb.name ?? null, + })) +} + +export async function attachKbToChatbot( + { kbId, chatbotId }: { kbId: string; chatbotId: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + return ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + await lockOwnedChatbotOrThrow(prisma, chatbotId, ctx.user.sub) + const mcpServer = await getKbMcpServerOrThrow(prisma) + + await prisma.kBChatbot.updateMany({ + where: { + chatbotId, + kbId: { not: kbId }, + isEnabled: true, + }, + data: { isEnabled: false }, + }) + await prisma.kBChatbot.upsert({ + where: { kbId_chatbotId: { kbId, chatbotId } }, + create: { kbId, chatbotId, isEnabled: true }, + update: { isEnabled: true }, + }) + + for (const chatMode of KB_MCP_CHAT_MODES) { + await prisma.chatbotMCPConfig.upsert({ + where: { + chatbotId_mcpServerId_chatMode: { + chatbotId, + mcpServerId: mcpServer.id, + chatMode, + }, + }, + create: { + chatbotId, + mcpServerId: mcpServer.id, + chatMode, + allowedTools: ['doc_query'], + priority: 0, + isEnabled: true, + }, + update: { + allowedTools: ['doc_query'], + priority: 0, + isEnabled: true, + }, + }) + } + + const [chatbot, kb] = await Promise.all([ + prisma.chatbot.findUniqueOrThrow({ + where: { id: chatbotId }, + select: { name: true }, + }), + prisma.kB.findUniqueOrThrow({ + where: { id: kbId }, + select: { name: true }, + }), + ]) + return { + chatbotId, + chatbotName: chatbot.name, + enabledKbId: kbId, + enabledKbName: kb.name, + } + }) +} + +export async function detachKbFromChatbot( + { kbId, chatbotId }: { kbId: string; chatbotId: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + return ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + await lockOwnedChatbotOrThrow(prisma, chatbotId, ctx.user.sub) + + await prisma.kBChatbot.deleteMany({ where: { kbId, chatbotId } }) + const enabledBinding = await prisma.kBChatbot.findFirst({ + where: { chatbotId, isEnabled: true }, + select: { id: true }, + }) + if (!enabledBinding) { + const mcpServer = await prisma.chatbotMCPServer.findUnique({ + where: { name: KB_MCP_SERVER_NAME }, + select: { id: true }, + }) + if (mcpServer) { + await prisma.chatbotMCPConfig.updateMany({ + where: { chatbotId, mcpServerId: mcpServer.id }, + data: { isEnabled: false }, + }) + } + } + + return true + }) +} + +export async function getKbResourceIngestionRuns( + { resourceId }: { resourceId: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + await getOwnedKbResourceOrThrow(ctx, resourceId) + + return ctx.prisma.kBIngestionRun.findMany({ + where: { + resourceId, + resource: { + deletedAt: null, + kb: { ownerId: ctx.user.sub, deletedAt: null }, + }, + }, + orderBy: { createdAt: 'desc' }, + take: 5, + }) +} + +export async function createKb( + { + name, + description, + }: { + name: string + description?: string | null + }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + const normalizedName = name.trim() + if (!normalizedName) { + throw new GraphQLError('KB name is required') + } + + return ctx.prisma.kB.create({ + data: { + name: normalizedName, + description, + ownerId: ctx.user.sub, + }, + }) +} + +async function recordDeletionQueueFailure( + input: DeleteKBResourceInput, + ctx: ContextWithUser +) { + await ctx.prisma.$transaction(async (prisma) => { + const resourceUpdate = await prisma.kBResource.updateMany({ + where: { + id: input.resourceId, + deletedAt: { not: null }, + ingestionOperation: DB.KBIngestionOperation.DELETE, + ingestionAttemptId: input.deletionAttemptId, + resourceVersion: input.resourceVersion, + externalOperationId: null, + }, + data: { + status: DB.KBResourceStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + errorCode: 'DELETION_QUEUE_FAILED', + }, + }) + if (resourceUpdate.count !== 1) return + + await prisma.kBIngestionRun.updateMany({ + where: { + id: input.deletionAttemptId, + resourceId: input.resourceId, + operation: DB.KBIngestionOperation.DELETE, + resourceVersion: input.resourceVersion, + status: { + in: [DB.KBIngestionStatus.QUEUED, DB.KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: DB.KBIngestionStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + errorCode: 'DELETION_QUEUE_FAILED', + }, + }) + }) +} + +async function queueKbDeletions( + inputs: DeleteKBResourceInput[], + ctx: ContextWithUser +) { + for ( + let start = 0; + start < inputs.length; + start += KB_DELETE_QUEUE_CONCURRENCY + ) { + const batch = inputs.slice(start, start + KB_DELETE_QUEUE_CONCURRENCY) + const results = await Promise.allSettled( + batch.map((input) => ctx.tasks.deleteKBResource.runNoWait(input)) + ) + await Promise.allSettled( + results.map((result, index) => + result.status === 'rejected' + ? recordDeletionQueueFailure(batch[index]!, ctx) + : Promise.resolve() + ) + ) + } +} + +export async function deleteKb({ id }: { id: string }, ctx: ContextWithUser) { + await assertKbPreviewAccess(ctx) + const { kb, deletionInputs } = await ctx.prisma.$transaction( + async (prisma) => { + await lockOwnedKbOrThrow(prisma, id, ctx.user.sub) + const graphState = await prisma.kB.findUniqueOrThrow({ + where: { id }, + select: { activeGraphBuildId: true }, + }) + if (graphState.activeGraphBuildId) { + throw new GraphQLError('KB cannot be deleted while a graph build runs') + } + await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KBResource" + WHERE "kbId" = CAST(${id} AS UUID) + AND "deletedAt" IS NULL + FOR UPDATE + ` + await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KBUploadTicket" + WHERE "kbId" = CAST(${id} AS UUID) + FOR UPDATE + ` + const resources = await prisma.kBResource.findMany({ + where: { kbId: id, deletedAt: null }, + }) + + if ( + resources.some( + ({ status }) => + status === DB.KBResourceStatus.QUEUED || + status === DB.KBResourceStatus.PROCESSING + ) + ) { + throw new GraphQLError('KB cannot be deleted') + } + if ( + resources.some( + ({ resourceVersion }) => resourceVersion >= 2_147_483_647 + ) + ) { + throw new GraphQLError('KB resource version limit reached') + } + + const deletedAt = new Date() + const deletionInputs = resources.map((resource) => ({ + resourceId: resource.id, + kbId: id, + deletionAttemptId: randomUUID(), + resourceVersion: resource.resourceVersion + 1, + })) + + await prisma.kB.update({ + where: { id }, + data: { + deletedAt, + deletedById: ctx.user.sub, + publishedGraphBuildId: null, + }, + }) + const bindingCandidates = await prisma.kBChatbot.findMany({ + where: { kbId: id, isEnabled: true }, + select: { chatbotId: true }, + orderBy: { chatbotId: 'asc' }, + }) + for (const { chatbotId } of bindingCandidates) { + await lockOwnedChatbotOrThrow(prisma, chatbotId, ctx.user.sub) + } + const bindings = await prisma.kBChatbot.findMany({ + where: { kbId: id, isEnabled: true }, + select: { chatbotId: true }, + }) + if (bindings.length > 0) { + await prisma.kBChatbot.updateMany({ + where: { kbId: id, isEnabled: true }, + data: { isEnabled: false }, + }) + const chatbotIds = bindings.map(({ chatbotId }) => chatbotId) + const remainingBindings = await prisma.kBChatbot.findMany({ + where: { + chatbotId: { in: chatbotIds }, + isEnabled: true, + }, + select: { chatbotId: true }, + }) + const stillEnabled = new Set( + remainingBindings.map(({ chatbotId }) => chatbotId) + ) + const unboundChatbotIds = chatbotIds.filter( + (chatbotId) => !stillEnabled.has(chatbotId) + ) + const mcpServer = await prisma.chatbotMCPServer.findUnique({ + where: { name: KB_MCP_SERVER_NAME }, + select: { id: true }, + }) + if (mcpServer && unboundChatbotIds.length > 0) { + await prisma.chatbotMCPConfig.updateMany({ + where: { + mcpServerId: mcpServer.id, + chatbotId: { in: unboundChatbotIds }, + }, + data: { isEnabled: false }, + }) + } + } + + for (const input of deletionInputs) { + await prisma.kBResource.update({ + where: { id: input.resourceId }, + data: { + deletedAt, + deletedById: ctx.user.sub, + status: DB.KBResourceStatus.QUEUED, + statusMessage: null, + ingestionOperation: DB.KBIngestionOperation.DELETE, + ingestionAttemptId: input.deletionAttemptId, + resourceVersion: input.resourceVersion, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + await prisma.kBIngestionRun.create({ + data: { + id: input.deletionAttemptId, + resourceId: input.resourceId, + operation: DB.KBIngestionOperation.DELETE, + resourceVersion: input.resourceVersion, + }, + }) + } + + const kb = await prisma.kB.findUniqueOrThrow({ + where: { id }, + }) + return { kb, deletionInputs } + } + ) + await queueKbDeletions(deletionInputs, ctx) + return kb +} + +export async function requestKbFileUpload( + { + kbId, + fileName, + contentType, + sizeBytes, + }: { + kbId: string + fileName: string + contentType: string + sizeBytes: number + }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + assertKbIngestionEnabled() + await getOwnedKbOrThrow(ctx, kbId) + const validated = validateKbFile({ fileName, contentType, sizeBytes }) + const { accountUrl, containerClient, credential } = getKbBlobContainer( + ctx.user.sub + ) + await containerClient.createIfNotExists() + + const blobId = randomUUID() + const blobName = `${blobId}.${validated.extension}` + const expiresOn = new Date(Date.now() + 15 * 60 * 1000) + await ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + await assertKbQuotaAvailable(prisma, { + kbId, + resourceCount: 1, + sizeBytes, + }) + await prisma.kBUploadTicket.create({ + data: { + id: blobId, + kbId, + blobName, + sizeBytes, + expiresAt: expiresOn, + }, + }) + }) + const permissions = BlobSASPermissions.parse('cw') + const queryParams = generateBlobSASQueryParameters( + { + containerName: containerClient.containerName, + blobName, + permissions, + expiresOn, + }, + credential + ) + + return { + uploadSasURL: `${accountUrl}?${queryParams.toString()}`, + containerName: containerClient.containerName, + blobName, + } +} + +function assertMatchingConfirmedBlob( + resource: DB.KBResource, + { + kbId, + blobName, + title, + originalFilename, + mimeType, + sizeBytes, + }: { + kbId: string + blobName: string + title: string + originalFilename: string + mimeType: string + sizeBytes: number + } +) { + if ( + resource.kbId !== kbId || + resource.type !== DB.KBResourceType.BLOB || + resource.blobName !== blobName + ) { + throw new GraphQLError('KB blob name is invalid') + } + if ( + resource.title !== title || + resource.originalFilename !== originalFilename || + resource.mimeType !== mimeType || + resource.sizeBytes !== sizeBytes + ) { + throw new GraphQLError('KB upload ticket is invalid', { + extensions: { code: 'KB_UPLOAD_TICKET_MISMATCH' }, + }) + } +} + +export async function confirmKbFileUpload( + { + kbId, + blobName, + title, + originalFilename, + mimeType, + sizeBytes, + }: { + kbId: string + blobName: string + title: string + originalFilename: string + mimeType: string + sizeBytes: number + }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + const validated = validateKbFile({ + fileName: originalFilename, + contentType: mimeType, + sizeBytes, + }) + const separator = blobName.lastIndexOf('.') + const blobId = blobName.slice(0, separator) + const blobExtension = blobName.slice(separator + 1).toLowerCase() + if ( + separator <= 0 || + !validateUuid(blobId) || + blobExtension !== validated.extension + ) { + throw new GraphQLError('KB blob name is invalid') + } + const normalizedTitle = validateKbResourceTitle(title) + + const existingResource = await ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + return prisma.kBResource.findFirst({ + where: { + id: blobId, + deletedAt: null, + kb: { ownerId: ctx.user.sub, deletedAt: null }, + }, + }) + }) + if (existingResource) { + assertMatchingConfirmedBlob(existingResource, { + kbId, + blobName, + title: normalizedTitle, + originalFilename, + mimeType: validated.contentType, + sizeBytes, + }) + return existingResource + } + + const { accountUrl, containerClient } = getKbBlobContainer(ctx.user.sub) + const blobClient = containerClient.getBlobClient(blobName) + if (!(await blobClient.exists())) { + throw new GraphQLError('KB blob was not found') + } + + const properties = await blobClient.getProperties() + if ( + properties.contentLength !== sizeBytes || + properties.contentType?.trim().toLowerCase() !== validated.contentType + ) { + await blobClient.deleteIfExists() + throw new GraphQLError('KB blob metadata is invalid') + } + + return ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + const racedResource = await prisma.kBResource.findFirst({ + where: { id: blobId, deletedAt: null }, + }) + if (racedResource) { + assertMatchingConfirmedBlob(racedResource, { + kbId, + blobName, + title: normalizedTitle, + originalFilename, + mimeType: validated.contentType, + sizeBytes, + }) + return racedResource + } + + const ticket = await prisma.kBUploadTicket.findFirst({ + where: { + id: blobId, + kbId, + blobName, + expiresAt: { gt: new Date() }, + }, + select: { id: true, sizeBytes: true }, + }) + if (!ticket || (ticket.sizeBytes !== 0 && ticket.sizeBytes !== sizeBytes)) { + throw new GraphQLError('KB upload ticket is invalid', { + extensions: { code: 'KB_UPLOAD_TICKET_MISMATCH' }, + }) + } + if (ticket.sizeBytes === 0) { + await assertKbQuotaAvailable(prisma, { kbId, sizeBytes }) + } + + const resource = await prisma.kBResource.create({ + data: { + id: blobId, + kbId, + type: DB.KBResourceType.BLOB, + title: normalizedTitle, + originalFilename, + mimeType: validated.contentType, + sizeBytes, + blobName, + blobHref: `${accountUrl}/${containerClient.containerName}/${blobName}`, + status: DB.KBResourceStatus.ADDED, + }, + }) + await prisma.kBUploadTicket.delete({ where: { id: ticket.id } }) + return resource + }) +} + +export async function createKbUrlResource( + { + kbId, + url, + title, + }: { + kbId: string + url: string + title: string + }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + assertKbIngestionEnabled() + await getOwnedKbOrThrow(ctx, kbId) + + let sourceUrl: string + try { + sourceUrl = normalizePublicHttpUrl(url) + } catch { + throw new GraphQLError('KB resource URL is invalid') + } + + return ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + await assertKbQuotaAvailable(prisma, { + kbId, + resourceCount: 1, + sizeBytes: MAX_KB_FILE_SIZE_BYTES, + }) + return prisma.kBResource.create({ + data: { + kbId, + type: DB.KBResourceType.URL, + title: validateKbResourceTitle(title), + sourceUrl, + status: DB.KBResourceStatus.ADDED, + }, + }) + }) +} + +export async function deleteKbResource( + { id }: { id: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + const { resource, deletionInput } = await ctx.prisma.$transaction( + async (prisma) => { + const kbId = await lockOwnedKbForResourceOrThrow(prisma, id, ctx.user.sub) + await lockOwnedKbResourceOrThrow(prisma, id, ctx.user.sub) + const resource = await prisma.kBResource.findUniqueOrThrow({ + where: { id }, + }) + + if ( + resource.status === DB.KBResourceStatus.QUEUED || + resource.status === DB.KBResourceStatus.PROCESSING + ) { + throw new GraphQLError('KB resource cannot be deleted') + } + if (resource.resourceVersion >= 2_147_483_647) { + throw new GraphQLError('KB resource version limit reached') + } + + const deletionInput = { + resourceId: resource.id, + kbId, + deletionAttemptId: randomUUID(), + resourceVersion: resource.resourceVersion + 1, + } satisfies DeleteKBResourceInput + const deletedResource = await prisma.kBResource.update({ + where: { id }, + data: { + deletedAt: new Date(), + deletedById: ctx.user.sub, + status: DB.KBResourceStatus.QUEUED, + statusMessage: null, + ingestionOperation: DB.KBIngestionOperation.DELETE, + ingestionAttemptId: deletionInput.deletionAttemptId, + resourceVersion: deletionInput.resourceVersion, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + await prisma.kBIngestionRun.create({ + data: { + id: deletionInput.deletionAttemptId, + resourceId: resource.id, + operation: DB.KBIngestionOperation.DELETE, + resourceVersion: deletionInput.resourceVersion, + }, + }) + return { resource: deletedResource, deletionInput } + } + ) + await queueKbDeletions([deletionInput], ctx) + return resource +} + +export async function deleteKbResources( + { kbId, ids }: { kbId: string; ids: string[] }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + if ( + ids.length === 0 || + ids.length > KB_BULK_DELETE_LIMIT || + new Set(ids).size !== ids.length || + ids.some((id) => !validateUuid(id)) + ) { + invalidPaginationInput('KB bulk deletion selection is invalid') + } + // Explicit code-unit order: this fixes the child lock order, so it must not + // vary with the runtime locale. + const sortedIds = [...ids].sort((a, b) => (a < b ? -1 : a > b ? 1 : 0)) + + const { resources, deletionInputs } = await ctx.prisma.$transaction( + async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + for (const resourceId of sortedIds) { + await lockKbResourceInKbOrThrow(prisma, kbId, resourceId) + } + + const resourceRows = await prisma.kBResource.findMany({ + where: { + id: { in: sortedIds }, + kbId, + deletedAt: null, + }, + }) + if (resourceRows.length !== sortedIds.length) { + throw new GraphQLError('KB resource not found') + } + if ( + resourceRows.some( + ({ status }) => + status === DB.KBResourceStatus.QUEUED || + status === DB.KBResourceStatus.PROCESSING + ) + ) { + throw new GraphQLError('KB resources cannot be deleted', { + extensions: { code: 'KB_RESOURCE_ACTIVE' }, + }) + } + if ( + resourceRows.some( + ({ resourceVersion }) => resourceVersion >= 2_147_483_647 + ) + ) { + throw new GraphQLError('KB resource version limit reached') + } + + const byId = new Map( + resourceRows.map((resource) => [resource.id, resource]) + ) + const orderedResources = sortedIds.map((id) => byId.get(id)!) + const deletedAt = new Date() + const deletionInputs = orderedResources.map((resource) => ({ + resourceId: resource.id, + kbId, + deletionAttemptId: randomUUID(), + resourceVersion: resource.resourceVersion + 1, + })) + const resources: DB.KBResource[] = [] + for (const [index, resource] of orderedResources.entries()) { + const input = deletionInputs[index]! + resources.push( + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { + deletedAt, + deletedById: ctx.user.sub, + status: DB.KBResourceStatus.QUEUED, + statusMessage: null, + ingestionOperation: DB.KBIngestionOperation.DELETE, + ingestionAttemptId: input.deletionAttemptId, + resourceVersion: input.resourceVersion, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + ) + await prisma.kBIngestionRun.create({ + data: { + id: input.deletionAttemptId, + resourceId: resource.id, + operation: DB.KBIngestionOperation.DELETE, + resourceVersion: input.resourceVersion, + }, + }) + } + return { resources, deletionInputs } + } + ) + + await queueKbDeletions(deletionInputs, ctx) + return resources +} + +export async function ingestKbResource( + { id }: { id: string }, + ctx: ContextWithUser +) { + await assertKbPreviewAccess(ctx) + assertKbIngestionEnabled() + const resource = await getOwnedKbResourceOrThrow(ctx, id) + if ( + resource.status !== DB.KBResourceStatus.ADDED && + resource.status !== DB.KBResourceStatus.READY && + resource.status !== DB.KBResourceStatus.FAILED + ) { + throw new GraphQLError('KB resource cannot be ingested') + } + if (resource.resourceVersion >= 2_147_483_647) { + throw new GraphQLError('KB resource version limit reached') + } + + const ingestionAttemptId = randomUUID() + const resourceVersion = resource.resourceVersion + 1 + const basePayload = { + resourceId: resource.id, + kbId: resource.kbId, + title: resource.title, + ingestionAttemptId, + resourceVersion, + } + let payload: IngestKBResourceInput + if (resource.type === DB.KBResourceType.BLOB) { + if ( + !resource.blobName || + !resource.mimeType || + resource.sizeBytes === null + ) { + throw new GraphQLError('KB blob metadata is invalid') + } + payload = { + ...basePayload, + type: DB.KBResourceType.BLOB, + blobName: resource.blobName, + containerName: getKbContainerName(ctx.user.sub), + mimeType: resource.mimeType, + sizeBytes: resource.sizeBytes, + } + } else { + if (!resource.sourceUrl) { + throw new GraphQLError('KB resource URL is invalid') + } + payload = { + ...basePayload, + type: DB.KBResourceType.URL, + sourceUrl: resource.sourceUrl, + } + } + + await ctx.prisma.$transaction(async (prisma) => { + const claim = await prisma.kBResource.updateMany({ + where: { + id: resource.id, + status: resource.status, + ingestionAttemptId: resource.ingestionAttemptId, + deletedAt: null, + kb: { ownerId: ctx.user.sub, deletedAt: null }, + }, + data: { + status: DB.KBResourceStatus.QUEUED, + statusMessage: null, + ingestionOperation: DB.KBIngestionOperation.UPSERT, + ingestionAttemptId, + resourceVersion, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + if (claim.count !== 1) { + throw new GraphQLError('KB resource cannot be ingested') + } + await prisma.kBIngestionRun.create({ + data: { + id: ingestionAttemptId, + resourceId: resource.id, + operation: DB.KBIngestionOperation.UPSERT, + resourceVersion, + }, + }) + }) + + try { + await ctx.tasks.ingestKBResource.runNoWait(payload) + } catch { + const finishedAt = new Date() + await ctx.prisma.$transaction(async (prisma) => { + const failed = await prisma.kBResource.updateMany({ + where: { + id: resource.id, + status: DB.KBResourceStatus.QUEUED, + ingestionAttemptId, + }, + data: { + status: DB.KBResourceStatus.FAILED, + statusMessage: 'The ingestion operation could not be queued.', + errorCode: 'QUEUE_DISPATCH_FAILED', + }, + }) + if (failed.count === 1) { + await prisma.kBIngestionRun.update({ + where: { id: ingestionAttemptId }, + data: { + status: DB.KBIngestionStatus.FAILED, + statusMessage: 'The ingestion operation could not be queued.', + errorCode: 'QUEUE_DISPATCH_FAILED', + finishedAt, + }, + }) + } + }) + throw new GraphQLError('KB ingestion could not be queued') + } + + return ctx.prisma.kBResource.findUniqueOrThrow({ + where: { id: resource.id }, + }) +} + +export interface KBKnowledgeGraphConfig { + kbId: string + isEnabled: boolean + buildId: string | null + status: DB.KBGraphBuildStatus | null + statusMessage: string | null + qualityTier: DB.KBGraphQualityTier | null + sourceContentDigest: string | null + activeBuildId: string | null + publishedBuildId: string | null + isStale: boolean + startedAt: Date | null + finishedAt: Date | null + createdAt: Date | null + updatedAt: Date | null + costConfigurationReady: boolean + costCurrency: string | null + quotaCurrency: string | null + billingLabel: string | null + standardEstimateMinorUnits: number | null + highEstimateMinorUnits: number | null + estimatedCostMinorUnits: number | null + actualCostMinorUnits: number | null + actualInputTokens: number | null + actualOutputTokens: number | null + actualEmbeddingTokens: number | null + actualRequestCount: number | null + maxCostMinorUnits: number | null + costStatus: DB.KBGraphCostStatus | null + semesterKey: string | null + semesterQuotaMinorUnits: number | null + semesterReservedMinorUnits: number | null + semesterSettledMinorUnits: number | null + remainingSemesterQuotaMinorUnits: number | null + worstCaseRemainingMinorUnits: number | null +} + +function getKBGraphArtifactBlobName(buildId: string): string { + return `knowledge-graphs/${buildId}.graphml` +} + +const KB_GRAPH_BUILD_CONFIG_SELECT = { + id: true, + status: true, + statusMessage: true, + qualityTier: true, + sourceContentDigest: true, + startedAt: true, + finishedAt: true, + createdAt: true, + updatedAt: true, + estimatedCostMinorUnits: true, + actualCostMinorUnits: true, + actualInputTokens: true, + actualOutputTokens: true, + actualEmbeddingTokens: true, + actualRequestCount: true, + costCurrency: true, + costStatus: true, + errorCode: true, + quotaId: true, + quota: { + select: { + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + settledMinorUnits: true, + }, + }, +} satisfies DB.Prisma.KBGraphBuildSelect + +export function getKBGraphBuildConfig( + kb: { + id: string + knowledgeGraphEnabled: boolean + activeGraphBuildId: string | null + publishedGraphBuildId: string | null + }, + build: { + id: string + status: DB.KBGraphBuildStatus + statusMessage: string | null + qualityTier: DB.KBGraphQualityTier + sourceContentDigest: string + startedAt: Date | null + finishedAt: Date | null + createdAt: Date + updatedAt: Date + estimatedCostMinorUnits: number | null + actualCostMinorUnits: number | null + actualInputTokens: number | null + actualOutputTokens: number | null + actualEmbeddingTokens: number | null + actualRequestCount: number | null + costCurrency: string | null + costStatus: DB.KBGraphCostStatus | null + quotaId: string | null + quota: { + currency: string + limitMinorUnits: number + reservedMinorUnits: number + settledMinorUnits: number + } | null + } | null, + isStale: boolean, + quota: { + currency: string + limitMinorUnits: number + reservedMinorUnits: number + settledMinorUnits: number + } | null, + costConfiguration: ReturnType +): KBKnowledgeGraphConfig { + const quotaConfigurationMatches = + quota === null || + (quota.currency === costConfiguration.currency && + quota.limitMinorUnits === costConfiguration.semesterQuotaMinorUnits) + const costConfigurationReady = + costConfiguration.ready && quotaConfigurationMatches + const remainingSemesterQuotaMinorUnits = getKBGraphRemainingQuota( + quota, + costConfiguration + ) + const worstCaseRemainingMinorUnits = + remainingSemesterQuotaMinorUnits !== null && + costConfiguration.maxCostMinorUnits !== null + ? remainingSemesterQuotaMinorUnits - costConfiguration.maxCostMinorUnits + : null + return { + kbId: kb.id, + isEnabled: kb.knowledgeGraphEnabled, + buildId: build?.id ?? null, + status: build?.status ?? null, + statusMessage: build?.statusMessage ?? null, + qualityTier: build?.qualityTier ?? null, + sourceContentDigest: build?.sourceContentDigest ?? null, + activeBuildId: kb.activeGraphBuildId, + publishedBuildId: kb.publishedGraphBuildId, + isStale, + startedAt: build?.startedAt ?? null, + finishedAt: build?.finishedAt ?? null, + createdAt: build?.createdAt ?? null, + updatedAt: build?.updatedAt ?? null, + costConfigurationReady, + costCurrency: build?.costCurrency ?? costConfiguration.currency, + quotaCurrency: quota?.currency ?? costConfiguration.currency, + billingLabel: getKBGraphBillingLabel(costConfiguration), + standardEstimateMinorUnits: costConfiguration.standardEstimateMinorUnits, + highEstimateMinorUnits: costConfiguration.highEstimateMinorUnits, + estimatedCostMinorUnits: + build?.estimatedCostMinorUnits ?? + costConfiguration.standardEstimateMinorUnits, + actualCostMinorUnits: build?.actualCostMinorUnits ?? null, + actualInputTokens: build?.actualInputTokens ?? null, + actualOutputTokens: build?.actualOutputTokens ?? null, + actualEmbeddingTokens: build?.actualEmbeddingTokens ?? null, + actualRequestCount: build?.actualRequestCount ?? null, + maxCostMinorUnits: costConfiguration.maxCostMinorUnits, + costStatus: build?.costStatus ?? null, + semesterKey: costConfiguration.semesterKey, + semesterQuotaMinorUnits: + quota?.limitMinorUnits ?? costConfiguration.semesterQuotaMinorUnits, + semesterReservedMinorUnits: quota?.reservedMinorUnits ?? 0, + semesterSettledMinorUnits: quota?.settledMinorUnits ?? 0, + remainingSemesterQuotaMinorUnits, + worstCaseRemainingMinorUnits, + } +} + +export async function getKbKnowledgeGraphConfig( + { kbId }: { kbId: string }, + ctx: ContextWithUser +): Promise { + await assertKbPreviewAccess(ctx) + const kb = await getOwnedKbOrThrow(ctx, kbId) + const costConfiguration = getKBGraphCostConfiguration() + const [build, publishedBuild] = await Promise.all([ + ctx.prisma.kBGraphBuild.findFirst({ + where: { kbId: kb.id }, + select: KB_GRAPH_BUILD_CONFIG_SELECT, + orderBy: [{ createdAt: 'desc' }, { id: 'desc' }], + }), + kb.publishedGraphBuildId + ? ctx.prisma.kBGraphBuild.findFirst({ + where: { + id: kb.publishedGraphBuildId, + kbId: kb.id, + status: DB.KBGraphBuildStatus.SUCCEEDED, + }, + select: { sourceContentDigest: true }, + }) + : Promise.resolve(null), + ]) + const quota = await ctx.prisma.kBGraphQuota.findUnique({ + where: { + ownerId_semesterKey: { + ownerId: kb.ownerId, + semesterKey: costConfiguration.semesterKey, + }, + }, + select: { + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + settledMinorUnits: true, + }, + }) + const isStale = + publishedBuild !== null + ? publishedBuild.sourceContentDigest !== + (await computeKBContentDigest(ctx.prisma, kb.id)) + : false + return getKBGraphBuildConfig(kb, build, isStale, quota, costConfiguration) +} + +async function readOwnedPublishedKBGraph( + kbId: string, + ctx: ContextWithUser, + read: (graph: PublishedKnowledgeGraph) => Promise +) { + await assertKbPreviewAccess(ctx) + await getOwnedKbOrThrow(ctx, kbId) + try { + return await read(await getPublishedKnowledgeGraph(ctx.prisma, kbId)) + } catch (error) { + if (error instanceof KnowledgeGraphNotPublishedError) { + throw new GraphQLError('KB knowledge graph is not published', { + extensions: { code: `KB_GRAPH_${error.code}` }, + }) + } + throw error + } +} + +export async function getKbKnowledgeGraphOverview( + { kbId }: { kbId: string }, + ctx: ContextWithUser +) { + return readOwnedPublishedKBGraph(kbId, ctx, readKnowledgeGraphOverview) +} + +export async function searchKbKnowledgeGraph( + { kbId, query }: { kbId: string; query: string }, + ctx: ContextWithUser +) { + return readOwnedPublishedKBGraph(kbId, ctx, (graph) => + searchKnowledgeGraph(graph, query) + ) +} + +export async function getKbKnowledgeGraphNeighbors( + { kbId, nodeId }: { kbId: string; nodeId: string }, + ctx: ContextWithUser +) { + return readOwnedPublishedKBGraph(kbId, ctx, (graph) => + readKnowledgeGraphNeighbors(graph, nodeId) + ) +} + +export async function setKbKnowledgeGraphEnabled( + { kbId, enabled }: { kbId: string; enabled: boolean }, + ctx: ContextWithUser +): Promise { + await assertKbPreviewAccess(ctx) + if (enabled) { + assertKbGraphGenerationEnabled() + requireKBGraphCostConfiguration() + } + + await ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + await prisma.kB.update({ + where: { id: kbId }, + data: { knowledgeGraphEnabled: enabled }, + }) + }) + + return getKbKnowledgeGraphConfig({ kbId }, ctx) +} + +export async function settleKbKnowledgeGraphResult( + prisma: DB.PrismaClient, + { + buildId, + result, + allowLateSuccess, + }: { buildId: string; result: unknown; allowLateSuccess?: boolean }, + finishedAt = new Date() +) { + return prisma.$transaction((transaction) => + settleKBGraphBuildCost(transaction, { + buildId, + result, + finishedAt, + allowLateSuccess, + }) + ) +} + +type GraphBuildSnapshotResource = { + id: string + title: string + type: DB.KBResourceType + sourceUrl: string | null + blobName: string | null + activeContentSha256: string | null +} + +function validateGraphBuildSnapshotResource( + resource: GraphBuildSnapshotResource +) { + if (!resource.activeContentSha256) { + throw new GraphQLError('KB graph source is not serving content') + } + if (resource.type === DB.KBResourceType.BLOB && !resource.blobName) { + throw new GraphQLError('KB graph blob source is invalid') + } + if (resource.type === DB.KBResourceType.URL && !resource.sourceUrl) { + throw new GraphQLError('KB graph URL source is invalid') + } + return resource.activeContentSha256 +} + +export async function rebuildKbKnowledgeGraph( + { + kbId, + qualityTier: requestedQualityTier, + }: { + kbId: string + qualityTier?: DB.KBGraphQualityTier | null + }, + ctx: ContextWithUser +): Promise { + const qualityTier = requestedQualityTier ?? DB.KBGraphQualityTier.STANDARD + await assertKbPreviewAccess(ctx) + assertKbGraphGenerationEnabled() + const result = await ctx.prisma.$transaction(async (prisma) => { + await lockOwnedKbOrThrow(prisma, kbId, ctx.user.sub) + const kb = await prisma.kB.findUniqueOrThrow({ + where: { id: kbId }, + select: { + id: true, + knowledgeGraphEnabled: true, + activeGraphBuildId: true, + publishedGraphBuildId: true, + }, + }) + + if (!kb.knowledgeGraphEnabled) { + throw new GraphQLError('KB knowledge graph is not enabled', { + extensions: { code: 'KB_GRAPH_NOT_ENABLED' }, + }) + } + + if (kb.activeGraphBuildId) { + const activeBuild = await prisma.kBGraphBuild.findFirst({ + where: { id: kb.activeGraphBuildId, kbId }, + select: KB_GRAPH_BUILD_CONFIG_SELECT, + }) + if ( + activeBuild && + (activeBuild.status === DB.KBGraphBuildStatus.QUEUED || + activeBuild.status === DB.KBGraphBuildStatus.PROCESSING) + ) { + return { kb, build: activeBuild, queueBuildId: null } + } + if (activeBuild?.errorCode === 'KB_GRAPH_DISPATCH_AMBIGUOUS') { + throw new GraphQLError( + 'The previous KB graph dispatch requires manual review before another build can start.', + { extensions: { code: 'KB_GRAPH_DISPATCH_AMBIGUOUS' } } + ) + } + await prisma.kB.updateMany({ + where: { id: kbId, activeGraphBuildId: kb.activeGraphBuildId }, + data: { activeGraphBuildId: null }, + }) + } + + const resources = await prisma.kBResource.findMany({ + where: { + kbId, + deletedAt: null, + activeContentSha256: { not: null }, + }, + select: { + id: true, + title: true, + type: true, + sourceUrl: true, + blobName: true, + activeContentSha256: true, + }, + orderBy: { id: 'asc' }, + }) + if (resources.length === 0) { + throw new GraphQLError('KB has no active graph sources', { + extensions: { code: 'KB_GRAPH_EMPTY' }, + }) + } + + const validatedResources = resources.map((resource) => ({ + resource, + contentSha256: validateGraphBuildSnapshotResource(resource), + })) + const sourceContentDigest = hashKBContentDigestEntries( + validatedResources.map(({ resource, contentSha256 }) => ({ + resourceId: resource.id, + contentSha256, + })) + ) + const buildId = randomUUID() + const reservation = await reserveKBGraphCost(prisma, { + ownerId: ctx.user.sub, + qualityTier, + }) + const build = await prisma.kBGraphBuild.create({ + data: { + id: buildId, + kbId, + requestedById: ctx.user.sub, + qualityTier, + sourceContentDigest, + graphName: getKnowledgeGraphName(kbId, buildId), + graphmlBlobName: getKBGraphArtifactBlobName(buildId), + estimatedCostMinorUnits: reservation.estimatedCostMinorUnits, + costCurrency: reservation.currency, + costPricingVersion: reservation.pricingVersion, + costStatus: DB.KBGraphCostStatus.RESERVED, + semesterKey: reservation.semesterKey, + quotaId: reservation.quotaId, + sources: { + create: validatedResources.map(({ resource, contentSha256 }) => ({ + resourceId: resource.id, + title: resource.title, + type: resource.type, + sourceUrl: resource.sourceUrl, + blobName: resource.blobName, + contentSha256, + })), + }, + }, + select: KB_GRAPH_BUILD_CONFIG_SELECT, + }) + const claimed = await prisma.kB.updateMany({ + where: { id: kbId, activeGraphBuildId: null }, + data: { activeGraphBuildId: buildId }, + }) + if (claimed.count !== 1) { + throw new Error('KB graph build slot could not be claimed') + } + return { + kb: { ...kb, activeGraphBuildId: buildId }, + build, + queueBuildId: buildId, + } + }) + + if (result.queueBuildId) { + try { + await ctx.tasks.buildKBGraph.runNoWait({ buildId: result.queueBuildId }) + } catch { + const finishedAt = new Date() + await ctx.prisma.$transaction(async (prisma) => { + const failed = await prisma.kBGraphBuild.updateMany({ + where: { + id: result.queueBuildId!, + kbId, + externalOperationId: null, + dispatchClaimedAt: null, + status: DB.KBGraphBuildStatus.QUEUED, + }, + data: { + status: DB.KBGraphBuildStatus.FAILED, + statusMessage: 'The KB graph build could not be queued.', + errorCode: 'KB_GRAPH_QUEUE_DISPATCH_FAILED', + finishedAt, + }, + }) + if (failed.count === 1) { + await releaseKBGraphCostReservation(prisma, result.queueBuildId!) + await prisma.kB.updateMany({ + where: { id: kbId, activeGraphBuildId: result.queueBuildId! }, + data: { activeGraphBuildId: null }, + }) + } + }) + throw new GraphQLError('KB graph build could not be queued') + } + } + + const isStale = + result.build.status === DB.KBGraphBuildStatus.SUCCEEDED + ? result.build.sourceContentDigest !== + (await computeKBContentDigest(ctx.prisma, kbId)) + : false + const costConfiguration = getKBGraphCostConfiguration() + const quota = await ctx.prisma.kBGraphQuota.findUnique({ + where: { + ownerId_semesterKey: { + ownerId: ctx.user.sub, + semesterKey: costConfiguration.semesterKey, + }, + }, + select: { + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + settledMinorUnits: true, + }, + }) + return getKBGraphBuildConfig( + result.kb, + result.build, + isStale, + quota, + costConfiguration + ) +} diff --git a/packages/graphql/src/services/knowledgeGraphAccounting.ts b/packages/graphql/src/services/knowledgeGraphAccounting.ts new file mode 100644 index 0000000000..472c5227d9 --- /dev/null +++ b/packages/graphql/src/services/knowledgeGraphAccounting.ts @@ -0,0 +1,687 @@ +import { computeKBContentDigest } from '@klicker-uzh/knowledge-graph' +import * as DB from '@klicker-uzh/prisma/client' +import { GraphQLError } from 'graphql' +import { randomUUID } from 'node:crypto' +import { + KB_GRAPH_DATABASE_INT_MAX, + validateKbGraphTerminalResult, + type KbGraphTerminalResult, +} from './kbGraphContract.js' +import { + getKBGraphCostConfiguration, + getKBGraphEstimate, + requireKBGraphCostConfiguration, +} from './knowledgeGraphCost.js' + +export type KBGraphCostReservation = { + quotaId: string + semesterKey: string + currency: string + pricingVersion: string + estimatedCostMinorUnits: number + maxCostMinorUnits: number +} + +type KBGraphCostTransaction = DB.Prisma.TransactionClient + +const RESERVATION_HOLD_STATUSES = [DB.KBGraphCostStatus.RESERVED] as const +const SETTLEMENT_HOLD_STATUSES = [ + DB.KBGraphCostStatus.RESERVED, + DB.KBGraphCostStatus.NEEDS_HUMAN_REVIEW, +] as const + +async function lockQuota( + prisma: KBGraphCostTransaction, + quotaId: string +): Promise { + await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KBGraphQuota" + WHERE "id" = CAST(${quotaId} AS UUID) + FOR UPDATE + ` +} + +export async function reserveKBGraphCost( + prisma: KBGraphCostTransaction, + { + ownerId, + qualityTier, + env = process.env, + now = new Date(), + }: { + ownerId: string + qualityTier: DB.KBGraphQualityTier + env?: NodeJS.ProcessEnv + now?: Date + } +): Promise { + const config = requireKBGraphCostConfiguration(env, now) + const estimatedCostMinorUnits = getKBGraphEstimate(qualityTier, config) + if (estimatedCostMinorUnits === null) { + throw new GraphQLError('KB graph cost estimate is not configured', { + extensions: { code: 'KB_GRAPH_COST_CONFIGURATION_MISSING' }, + }) + } + + const candidateQuotaId = randomUUID() + await prisma.$executeRaw` + INSERT INTO "public"."KBGraphQuota" + ("id", "ownerId", "semesterKey", "currency", "limitMinorUnits", "updatedAt") + VALUES + (CAST(${candidateQuotaId} AS UUID), CAST(${ownerId} AS UUID), + ${config.semesterKey}, ${config.currency}, + ${config.semesterQuotaMinorUnits}, ${now}) + ON CONFLICT ("ownerId", "semesterKey") DO NOTHING + ` + const quota = await prisma.kBGraphQuota.findUniqueOrThrow({ + where: { + ownerId_semesterKey: { + ownerId, + semesterKey: config.semesterKey, + }, + }, + select: { id: true }, + }) + await lockQuota(prisma, quota.id) + + const lockedQuota = await prisma.kBGraphQuota.findUniqueOrThrow({ + where: { id: quota.id }, + select: { + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + settledMinorUnits: true, + }, + }) + if ( + lockedQuota.currency !== config.currency || + lockedQuota.limitMinorUnits !== config.semesterQuotaMinorUnits + ) { + throw new GraphQLError( + 'KB graph quota configuration changed mid-semester', + { + extensions: { code: 'KB_GRAPH_QUOTA_CONFIGURATION_CHANGED' }, + } + ) + } + + const usedMinorUnits = + lockedQuota.reservedMinorUnits + lockedQuota.settledMinorUnits + if (usedMinorUnits + estimatedCostMinorUnits > lockedQuota.limitMinorUnits) { + throw new GraphQLError('KB graph semester quota is insufficient', { + extensions: { + code: 'KB_GRAPH_QUOTA_EXCEEDED', + remainingMinorUnits: Math.max( + 0, + lockedQuota.limitMinorUnits - usedMinorUnits + ), + }, + }) + } + + await prisma.kBGraphQuota.update({ + where: { id: quota.id }, + data: { reservedMinorUnits: { increment: estimatedCostMinorUnits } }, + }) + + return { + quotaId: quota.id, + semesterKey: config.semesterKey, + currency: config.currency, + pricingVersion: config.pricingVersion, + estimatedCostMinorUnits, + maxCostMinorUnits: config.maxCostMinorUnits, + } +} + +export async function releaseKBGraphCostReservation( + prisma: KBGraphCostTransaction, + buildId: string +): Promise { + const build = await prisma.kBGraphBuild.findUnique({ + where: { id: buildId }, + select: { + quotaId: true, + estimatedCostMinorUnits: true, + costStatus: true, + }, + }) + if ( + !build || + !RESERVATION_HOLD_STATUSES.includes( + build.costStatus as (typeof RESERVATION_HOLD_STATUSES)[number] + ) || + build.estimatedCostMinorUnits === null + ) { + return false + } + + if (build.quotaId) { + await lockQuota(prisma, build.quotaId) + } + const updated = await prisma.kBGraphBuild.updateMany({ + where: { + id: buildId, + costStatus: { in: [...RESERVATION_HOLD_STATUSES] }, + }, + data: { costStatus: DB.KBGraphCostStatus.RELEASED }, + }) + if (updated.count !== 1) return false + + if (build.quotaId) { + const quotaUpdated = await prisma.kBGraphQuota.updateMany({ + where: { + id: build.quotaId, + reservedMinorUnits: { gte: build.estimatedCostMinorUnits }, + }, + data: { + reservedMinorUnits: { decrement: build.estimatedCostMinorUnits }, + }, + }) + if (quotaUpdated.count !== 1) { + throw new Error('KB graph quota reservation could not be released') + } + } + return true +} + +export type KBGraphCostSettlementOutcome = + | 'SETTLED' + | 'RELEASED' + | 'NEEDS_HUMAN_REVIEW' + | 'DUPLICATE' + +type KBGraphCostBuild = { + id: string + kbId: string + externalOperationId: string | null + sourceContentDigest: string + graphName: string + graphmlBlobName: string | null + createdAt: Date + estimatedCostMinorUnits: number | null + costCurrency: string | null + costStatus: DB.KBGraphCostStatus | null + quotaId: string | null + status: DB.KBGraphBuildStatus + errorCode: string | null + cleanupStartedAt: Date | null + cleanedAt: Date | null + kb: { + ownerId: string + activeGraphBuildId: string | null + deletedAt: Date | null + } +} + +async function markCostNeedsHumanReview( + prisma: KBGraphCostTransaction, + build: KBGraphCostBuild, + message: string, + errorCode: string, + finishedAt: Date +): Promise { + // KB before quota: every other path that touches both rows takes them in this + // order, and reversing it here would let two settlements deadlock. + await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KB" + WHERE "id" = CAST(${build.kbId} AS UUID) + FOR UPDATE + ` + if (build.quotaId) await lockQuota(prisma, build.quotaId) + const updated = await prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + costStatus: { in: [...RESERVATION_HOLD_STATUSES] }, + }, + data: { + status: DB.KBGraphBuildStatus.FAILED, + statusMessage: message, + errorCode, + costStatus: DB.KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + finishedAt, + }, + }) + if (updated.count === 1) { + await prisma.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + return 'NEEDS_HUMAN_REVIEW' + } + return 'DUPLICATE' +} + +function terminalResultError(result: KbGraphTerminalResult): string { + return ( + result.error_code ?? + `KB graph provider returned terminal status ${result.status}` + ) +} + +type KBGraphLateSuccessResolution = + | { eligible: true } + | { + eligible: false + status: DB.KBGraphBuildStatus + statusMessage: string + errorCode: string + } + +async function reconcileLateKBGraphSuccess( + prisma: KBGraphCostTransaction, + build: KBGraphCostBuild +): Promise { + const lockedKb = await prisma.$queryRaw< + Array<{ + id: string + activeGraphBuildId: string | null + deletedAt: Date | null + }> + >` + SELECT "id", "activeGraphBuildId", "deletedAt" + FROM "public"."KB" + WHERE "id" = CAST(${build.kbId} AS UUID) + FOR UPDATE + ` + const currentKb = lockedKb[0] + if ( + lockedKb.length !== 1 || + !currentKb || + currentKb.deletedAt !== null || + currentKb.activeGraphBuildId !== null + ) { + return { + eligible: false, + status: DB.KBGraphBuildStatus.SUPERSEDED, + statusMessage: 'A newer KB graph build replaced this timed-out build.', + errorCode: 'KB_GRAPH_LATE_SUCCESS_SUPERSEDED', + } + } + + const newerBuild = await prisma.kBGraphBuild.findFirst({ + where: { kbId: build.kbId, createdAt: { gt: build.createdAt } }, + select: { id: true }, + }) + if (newerBuild) { + return { + eligible: false, + status: DB.KBGraphBuildStatus.SUPERSEDED, + statusMessage: 'A newer KB graph build replaced this timed-out build.', + errorCode: 'KB_GRAPH_LATE_SUCCESS_SUPERSEDED', + } + } + + // Resource refreshes lock their resource row. Lock the serving set after the + // KB row so the digest cannot change between the eligibility check and + // settlement claim. + await prisma.$queryRaw>` + SELECT resource."id" + FROM "public"."KBResource" AS resource + WHERE resource."kbId" = CAST(${build.kbId} AS UUID) + AND resource."deletedAt" IS NULL + ORDER BY resource."id" ASC + FOR UPDATE OF resource + ` + const currentDigest = await computeKBContentDigest(prisma, build.kbId) + if (currentDigest !== build.sourceContentDigest) { + return { + eligible: false, + status: DB.KBGraphBuildStatus.FAILED, + statusMessage: 'The KB changed before the timed-out build completed.', + errorCode: 'KB_GRAPH_LATE_SUCCESS_STALE', + } + } + + return { eligible: true } +} + +export async function settleKBGraphBuildCost( + prisma: KBGraphCostTransaction, + { + buildId, + result: rawResult, + finishedAt = new Date(), + allowLateSuccess = false, + }: { + buildId: string + result: unknown + finishedAt?: Date + allowLateSuccess?: boolean + } +): Promise { + const build = await prisma.kBGraphBuild.findUnique({ + where: { id: buildId }, + select: { + id: true, + kbId: true, + externalOperationId: true, + sourceContentDigest: true, + graphName: true, + graphmlBlobName: true, + createdAt: true, + estimatedCostMinorUnits: true, + costCurrency: true, + costStatus: true, + quotaId: true, + status: true, + errorCode: true, + cleanupStartedAt: true, + cleanedAt: true, + kb: { + select: { + ownerId: true, + activeGraphBuildId: true, + deletedAt: true, + }, + }, + }, + }) + if (!build) { + throw new GraphQLError('KB graph build not found', { + extensions: { code: 'KB_GRAPH_BUILD_NOT_FOUND' }, + }) + } + if ( + !SETTLEMENT_HOLD_STATUSES.includes( + build.costStatus as (typeof SETTLEMENT_HOLD_STATUSES)[number] + ) + ) { + return 'DUPLICATE' + } + if ( + build.estimatedCostMinorUnits === null || + build.costCurrency === null || + build.externalOperationId === null || + build.quotaId === null + ) { + return markCostNeedsHumanReview( + prisma, + build, + 'The KB graph result could not be matched to a complete reservation.', + 'KB_GRAPH_RESERVATION_INCOMPLETE', + finishedAt + ) + } + + const expectedResultId = `${build.id}:${build.externalOperationId}` + const validation = validateKbGraphTerminalResult(rawResult, { + buildId: build.id, + kbId: build.kbId, + ownerId: build.kb.ownerId, + resultId: expectedResultId, + runId: build.externalOperationId, + estimatedMinorUnits: build.estimatedCostMinorUnits, + }) + if (!validation.ok) { + return markCostNeedsHumanReview( + prisma, + build, + 'The KB graph provider result failed contract validation.', + 'KB_GRAPH_RESULT_CONTRACT_INVALID', + finishedAt + ) + } + + const result = validation.result + if ( + result.source_content_digest !== build.sourceContentDigest || + result.graph_name !== build.graphName || + (result.graphml_artifact !== null && + (result.graphml_artifact.blob_name !== build.graphmlBlobName || + result.graphml_artifact.container_name !== `kb-${build.kb.ownerId}`)) + ) { + return markCostNeedsHumanReview( + prisma, + build, + 'The KB graph provider result did not match the pinned build identity.', + 'KB_GRAPH_RESULT_IDENTITY_INVALID', + finishedAt + ) + } + + if ( + result.status === 'SUCCEEDED' && + (build.cleanupStartedAt !== null || build.cleanedAt !== null) + ) { + if (build.costStatus === DB.KBGraphCostStatus.NEEDS_HUMAN_REVIEW) { + return 'NEEDS_HUMAN_REVIEW' + } + return markCostNeedsHumanReview( + prisma, + build, + 'A successful KB graph result arrived after artifact cleanup started.', + 'KB_GRAPH_RESULT_AFTER_CLEANUP', + finishedAt + ) + } + + if (result.metered_cost !== null) { + if (result.metered_cost.currency !== build.costCurrency) { + return markCostNeedsHumanReview( + prisma, + build, + 'The KB graph result currency did not match its reservation.', + 'KB_GRAPH_RESULT_CURRENCY_MISMATCH', + finishedAt + ) + } + + const usage = { + inputTokens: 0, + outputTokens: 0, + embeddingTokens: 0, + requestCount: 0, + } + for (const component of result.metered_cost.components) { + if ( + usage.inputTokens > + KB_GRAPH_DATABASE_INT_MAX - component.input_tokens || + usage.outputTokens > + KB_GRAPH_DATABASE_INT_MAX - component.output_tokens || + usage.embeddingTokens > + KB_GRAPH_DATABASE_INT_MAX - component.embedding_tokens || + usage.requestCount > KB_GRAPH_DATABASE_INT_MAX - component.request_count + ) { + return markCostNeedsHumanReview( + prisma, + build, + 'The KB graph result metering exceeded the database integer range.', + 'KB_GRAPH_RESULT_METERING_OVERFLOW', + finishedAt + ) + } + usage.inputTokens += component.input_tokens + usage.outputTokens += component.output_tokens + usage.embeddingTokens += component.embedding_tokens + usage.requestCount += component.request_count + } + + const lateSuccess = + allowLateSuccess && + result.status === 'SUCCEEDED' && + build.status === DB.KBGraphBuildStatus.FAILED && + build.errorCode === 'KB_GRAPH_TIMEOUT' + ? await reconcileLateKBGraphSuccess(prisma, build) + : null + const succeeded = result.status === 'SUCCEEDED' + const lateRejection = lateSuccess?.eligible === false ? lateSuccess : null + let publishSuccess = + succeeded && + lateRejection === null && + (build.kb.activeGraphBuildId === build.id || + lateSuccess?.eligible === true) + const settledStatus = publishSuccess + ? DB.KBGraphBuildStatus.SUCCEEDED + : (lateRejection?.status ?? + (succeeded + ? DB.KBGraphBuildStatus.SUCCEEDED + : DB.KBGraphBuildStatus.FAILED)) + const settledStatusMessage = publishSuccess + ? null + : (lateRejection?.statusMessage ?? + (succeeded ? null : terminalResultError(result))) + const settledErrorCode = publishSuccess + ? null + : (lateRejection?.errorCode ?? + (succeeded ? null : (result.error_code ?? `KB_GRAPH_${result.status}`))) + await lockQuota(prisma, build.quotaId!) + const updated = await prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + costStatus: { in: [...SETTLEMENT_HOLD_STATUSES] }, + ...(succeeded ? { cleanupStartedAt: null, cleanedAt: null } : {}), + }, + data: { + status: settledStatus, + statusMessage: settledStatusMessage, + errorCode: settledErrorCode, + actualCostMinorUnits: result.metered_cost.amount_minor_units, + actualInputTokens: usage.inputTokens, + actualOutputTokens: usage.outputTokens, + actualEmbeddingTokens: usage.embeddingTokens, + actualRequestCount: usage.requestCount, + costStatus: DB.KBGraphCostStatus.SETTLED, + meteredCost: result.metered_cost, + finishedAt, + }, + }) + if (updated.count !== 1) { + const current = await prisma.kBGraphBuild.findUnique({ + where: { id: build.id }, + select: { cleanupStartedAt: true, cleanedAt: true }, + }) + return current?.cleanupStartedAt || current?.cleanedAt + ? 'NEEDS_HUMAN_REVIEW' + : 'DUPLICATE' + } + + const quotaUpdated = await prisma.kBGraphQuota.updateMany({ + where: { + id: build.quotaId!, + reservedMinorUnits: { gte: build.estimatedCostMinorUnits }, + }, + data: { + reservedMinorUnits: { decrement: build.estimatedCostMinorUnits }, + settledMinorUnits: { + increment: result.metered_cost.amount_minor_units, + }, + }, + }) + if (quotaUpdated.count !== 1) { + throw new Error('KB graph quota reservation could not be settled') + } + if (publishSuccess && lateSuccess?.eligible === true) { + const claimed = await prisma.kB.updateMany({ + where: { + id: build.kbId, + deletedAt: null, + activeGraphBuildId: null, + }, + data: { activeGraphBuildId: build.id }, + }) + if (claimed.count !== 1) { + publishSuccess = false + const superseded = await prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + costStatus: DB.KBGraphCostStatus.SETTLED, + status: DB.KBGraphBuildStatus.SUCCEEDED, + }, + data: { + status: DB.KBGraphBuildStatus.SUPERSEDED, + statusMessage: + 'A newer KB graph build replaced this timed-out build.', + errorCode: 'KB_GRAPH_LATE_SUCCESS_SUPERSEDED', + }, + }) + if (superseded.count !== 1) { + throw new Error( + 'KB graph late success could not be marked as superseded' + ) + } + } + } + if (publishSuccess) { + const published = await prisma.kB.updateMany({ + where: { + id: build.kbId, + deletedAt: null, + activeGraphBuildId: build.id, + }, + data: { + activeGraphBuildId: null, + publishedGraphBuildId: build.id, + }, + }) + if (published.count !== 1) { + throw new Error('KB graph build changed while accepting late success') + } + } else { + await prisma.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + } + return 'SETTLED' + } + + if (result.status === 'SUCCEEDED') { + return markCostNeedsHumanReview( + prisma, + build, + 'A successful KB graph result did not include metering.', + 'KB_GRAPH_RESULT_METERING_MISSING', + finishedAt + ) + } + + if (build.costStatus === DB.KBGraphCostStatus.NEEDS_HUMAN_REVIEW) { + return 'NEEDS_HUMAN_REVIEW' + } + if (!(await releaseKBGraphCostReservation(prisma, build.id))) { + return 'DUPLICATE' + } + await prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + costStatus: DB.KBGraphCostStatus.RELEASED, + status: { + in: [DB.KBGraphBuildStatus.QUEUED, DB.KBGraphBuildStatus.PROCESSING], + }, + }, + data: { + status: DB.KBGraphBuildStatus.FAILED, + statusMessage: terminalResultError(result), + errorCode: result.error_code ?? `KB_GRAPH_${result.status}`, + finishedAt, + }, + }) + await prisma.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + return 'RELEASED' +} + +export type KBGraphQuotaSummary = { + currency: string + limitMinorUnits: number + reservedMinorUnits: number + settledMinorUnits: number +} + +export function getKBGraphRemainingQuota( + quota: KBGraphQuotaSummary | null, + config: ReturnType +): number | null { + if (quota) { + return ( + quota.limitMinorUnits - quota.reservedMinorUnits - quota.settledMinorUnits + ) + } + if (config.semesterQuotaMinorUnits === null) return null + return config.semesterQuotaMinorUnits +} diff --git a/packages/graphql/src/services/knowledgeGraphCost.ts b/packages/graphql/src/services/knowledgeGraphCost.ts new file mode 100644 index 0000000000..e6ea9fc224 --- /dev/null +++ b/packages/graphql/src/services/knowledgeGraphCost.ts @@ -0,0 +1,197 @@ +import * as DB from '@klicker-uzh/prisma/client' +import { GraphQLError } from 'graphql' +import { KB_GRAPH_DATABASE_INT_MAX } from './kbGraphContract.js' + +const SEMESTER_KEY_PATTERN = /^\d{4}-(?:H1|H2)$/ +const CURRENCY_PATTERN = /^[A-Z]{3}$/ + +export const KB_GRAPH_COST_CURRENCY_DEFAULT = 'CHF' +export const KB_GRAPH_BILLING_MODE_DEFAULT = 'SEMESTER_QUOTA' +export const KB_GRAPH_PRICING_VERSION_DEFAULT = 'unconfigured' + +export type KBGraphBillingMode = 'SEMESTER_QUOTA' | 'PROVIDER_BILLED' + +export type KBGraphCostConfiguration = { + currency: string + standardEstimateMinorUnits: number | null + highEstimateMinorUnits: number | null + maxCostMinorUnits: number | null + semesterQuotaMinorUnits: number | null + pricingVersion: string + billingMode: KBGraphBillingMode + semesterKey: string + ready: boolean +} + +function parseMinorUnits(env: NodeJS.ProcessEnv, name: string): number | null { + const value = env[name]?.trim() + if (value === undefined || value === '') return null + if (!/^(?:0|[1-9]\d*)$/.test(value)) { + throw new Error(`${name} must be a non-negative integer`) + } + const parsed = Number(value) + if (!Number.isSafeInteger(parsed) || parsed > KB_GRAPH_DATABASE_INT_MAX) { + throw new Error(`${name} is outside the supported range`) + } + return parsed +} + +function requirePositive(value: number | null, name: string): number { + if (value === null || value === 0) { + throw new Error(`${name} must be a positive integer`) + } + return value +} + +function parseCurrency(env: NodeJS.ProcessEnv): string { + const currency = + env.KB_GRAPH_COST_CURRENCY?.trim() ?? KB_GRAPH_COST_CURRENCY_DEFAULT + if (!CURRENCY_PATTERN.test(currency)) { + throw new Error( + 'KB_GRAPH_COST_CURRENCY must be a three-letter uppercase code' + ) + } + return currency +} + +function parsePricingVersion(env: NodeJS.ProcessEnv): string { + const pricingVersion = + env.KB_GRAPH_COST_PRICING_VERSION?.trim() ?? + KB_GRAPH_PRICING_VERSION_DEFAULT + if (!/^[A-Za-z0-9._-]{1,100}$/.test(pricingVersion)) { + throw new Error('KB_GRAPH_COST_PRICING_VERSION is invalid') + } + return pricingVersion +} + +function parseBillingMode(env: NodeJS.ProcessEnv): KBGraphBillingMode { + const billingMode = + env.KB_GRAPH_BILLING_MODE?.trim() ?? KB_GRAPH_BILLING_MODE_DEFAULT + if (billingMode !== 'SEMESTER_QUOTA' && billingMode !== 'PROVIDER_BILLED') { + throw new Error( + 'KB_GRAPH_BILLING_MODE must be SEMESTER_QUOTA or PROVIDER_BILLED' + ) + } + return billingMode +} + +export function getKBGraphSemesterKey( + now: Date = new Date(), + env: NodeJS.ProcessEnv = process.env +): string { + const configured = env.KB_GRAPH_SEMESTER_KEY?.trim() + if (configured !== undefined && !SEMESTER_KEY_PATTERN.test(configured)) { + throw new Error('KB_GRAPH_SEMESTER_KEY must use YYYY-H1 or YYYY-H2') + } + if (configured) return configured + + const half = now.getUTCMonth() < 6 ? 'H1' : 'H2' + return `${now.getUTCFullYear()}-${half}` +} + +export function getKBGraphCostConfiguration( + env: NodeJS.ProcessEnv = process.env, + now: Date = new Date() +): KBGraphCostConfiguration { + const standardEstimateMinorUnits = parseMinorUnits( + env, + 'KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS' + ) + const highEstimateMinorUnits = parseMinorUnits( + env, + 'KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS' + ) + const maxCostMinorUnits = parseMinorUnits( + env, + 'KB_GRAPH_MAX_COST_MINOR_UNITS' + ) + const semesterQuotaMinorUnits = parseMinorUnits( + env, + 'KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS' + ) + + if ( + maxCostMinorUnits !== null && + standardEstimateMinorUnits !== null && + standardEstimateMinorUnits > maxCostMinorUnits + ) { + throw new Error( + 'KB_GRAPH_MAX_COST_MINOR_UNITS must cover the standard estimate' + ) + } + if ( + maxCostMinorUnits !== null && + highEstimateMinorUnits !== null && + highEstimateMinorUnits > maxCostMinorUnits + ) { + throw new Error( + 'KB_GRAPH_MAX_COST_MINOR_UNITS must cover the high estimate' + ) + } + + return { + currency: parseCurrency(env), + standardEstimateMinorUnits, + highEstimateMinorUnits, + maxCostMinorUnits, + semesterQuotaMinorUnits, + pricingVersion: parsePricingVersion(env), + billingMode: parseBillingMode(env), + semesterKey: getKBGraphSemesterKey(now, env), + ready: [ + standardEstimateMinorUnits, + highEstimateMinorUnits, + maxCostMinorUnits, + semesterQuotaMinorUnits, + ].every((value) => value !== null && value > 0), + } +} + +export function requireKBGraphCostConfiguration( + env: NodeJS.ProcessEnv = process.env, + now: Date = new Date() +) { + const config = getKBGraphCostConfiguration(env, now) + if (!config.ready) { + throw new GraphQLError('KB graph cost configuration is incomplete', { + extensions: { code: 'KB_GRAPH_COST_CONFIGURATION_MISSING' }, + }) + } + + return { + ...config, + standardEstimateMinorUnits: requirePositive( + config.standardEstimateMinorUnits, + 'KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS' + ), + highEstimateMinorUnits: requirePositive( + config.highEstimateMinorUnits, + 'KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS' + ), + maxCostMinorUnits: requirePositive( + config.maxCostMinorUnits, + 'KB_GRAPH_MAX_COST_MINOR_UNITS' + ), + semesterQuotaMinorUnits: requirePositive( + config.semesterQuotaMinorUnits, + 'KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS' + ), + } +} + +export function getKBGraphEstimate( + qualityTier: DB.KBGraphQualityTier, + config: KBGraphCostConfiguration +): number | null { + return qualityTier === DB.KBGraphQualityTier.HIGH + ? config.highEstimateMinorUnits + : config.standardEstimateMinorUnits +} + +export function getKBGraphBillingLabel( + config: KBGraphCostConfiguration +): string { + return config.billingMode === 'PROVIDER_BILLED' + ? 'PROVIDER_BILLED' + : 'SEMESTER_QUOTA' +} diff --git a/packages/graphql/src/services/knowledgeSourceGateway.ts b/packages/graphql/src/services/knowledgeSourceGateway.ts new file mode 100644 index 0000000000..75ffac2164 --- /dev/null +++ b/packages/graphql/src/services/knowledgeSourceGateway.ts @@ -0,0 +1,140 @@ +import { + BlobServiceClient, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import { + KBResourceStatus, + KBResourceType, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' +import { createHash, timingSafeEqual } from 'node:crypto' + +const KB_SOURCE_GATEWAY_STORAGE_TIMEOUT_MS = 30_000 + +type KBSourceGatewayError = { + statusCode: 401 | 404 | 502 | 503 + body: { error: string } +} + +type KBSourceGatewaySuccess = { + statusCode: 200 + contentLength: number + contentType: string + stream: NodeJS.ReadableStream +} + +export type KBSourceGatewayResult = + | KBSourceGatewayError + | KBSourceGatewaySuccess + +function isAuthorized(authorization: string | undefined, secret: string) { + const expected = createHash('sha256') + .update(`Bearer ${secret}`, 'utf8') + .digest() + const provided = createHash('sha256') + .update(authorization ?? '', 'utf8') + .digest() + return timingSafeEqual(expected, provided) +} + +function getBlobClient({ + accountName, + accessKey, + accountUrl, + containerName, + blobName, +}: { + accountName: string + accessKey: string + accountUrl?: string + containerName: string + blobName: string +}) { + const credential = new StorageSharedKeyCredential(accountName, accessKey) + const serviceClient = new BlobServiceClient( + getBlobStorageAccountUrl(accountName, accountUrl), + credential + ) + return serviceClient.getContainerClient(containerName).getBlobClient(blobName) +} + +export async function handleKBSourceGateway({ + prisma, + resourceId, + resourceVersion, + authorization, + env = process.env, +}: { + prisma: PrismaClient + resourceId: string + resourceVersion: number + authorization?: string + env?: NodeJS.ProcessEnv +}): Promise { + const gatewaySecret = env.KB_SOURCE_GATEWAY_KEY?.trim() + const accountName = env.BLOB_STORAGE_ACCOUNT_NAME?.trim() + const accessKey = env.BLOB_STORAGE_ACCESS_KEY?.trim() + if (!gatewaySecret || !accountName || !accessKey) { + return { statusCode: 503, body: { error: 'Service unavailable' } } + } + if (!isAuthorized(authorization, gatewaySecret)) { + return { statusCode: 401, body: { error: 'Unauthorized' } } + } + + const resource = await prisma.kBResource.findFirst({ + where: { + id: resourceId, + resourceVersion, + deletedAt: null, + type: KBResourceType.BLOB, + contentSha256: { not: null }, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + select: { + blobName: true, + mimeType: true, + sizeBytes: true, + kb: { select: { ownerId: true } }, + }, + }) + if ( + !resource?.blobName || + !resource.mimeType || + resource.sizeBytes === null + ) { + return { statusCode: 404, body: { error: 'Resource not found' } } + } + + try { + const blobClient = getBlobClient({ + accountName, + accessKey, + accountUrl: + env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? env.BLOB_STORAGE_ACCOUNT_URL, + containerName: `kb-${resource.kb.ownerId}`, + blobName: resource.blobName, + }) + const response = await blobClient.download(0, undefined, { + abortSignal: AbortSignal.timeout(KB_SOURCE_GATEWAY_STORAGE_TIMEOUT_MS), + }) + if ( + !response.readableStreamBody || + response.contentLength !== resource.sizeBytes || + response.contentType?.trim().toLowerCase() !== resource.mimeType + ) { + return { statusCode: 502, body: { error: 'Source unavailable' } } + } + + return { + statusCode: 200, + contentLength: resource.sizeBytes, + contentType: resource.mimeType, + stream: response.readableStreamBody, + } + } catch { + return { statusCode: 502, body: { error: 'Source unavailable' } } + } +} diff --git a/packages/graphql/src/services/knowledgeWebhooks.ts b/packages/graphql/src/services/knowledgeWebhooks.ts new file mode 100644 index 0000000000..c18866165d --- /dev/null +++ b/packages/graphql/src/services/knowledgeWebhooks.ts @@ -0,0 +1,521 @@ +import { + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import { createKBIngestionWebhookSignature } from '@klicker-uzh/util' +import { timingSafeEqual } from 'node:crypto' + +export { signKBIngestionWebhook } from '@klicker-uzh/util' + +const SIGNATURE_MAX_AGE_SECONDS = 300 +const MAX_RESOURCE_VERSION = 2_147_483_647 +const UUID_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/ +const SHA256_PATTERN = /^[a-f0-9]{64}$/ +const CANONICAL_TIMESTAMP_PATTERN = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/ +const EVENT_TYPES = [ + 'resource.processing_started', + 'resource.processing_progress', + 'resource.processing_succeeded', + 'resource.processing_failed', + 'resource.content_refreshed', + 'resource.subresources_updated', + 'kb.metrics_updated', +] as const + +type WebhookHeaders = Record +type OperationStatusEventType = (typeof EVENT_TYPES)[number] + +type OperationStatusEvent = { + eventId: string + eventType: OperationStatusEventType + occurredAt: string + operation_id: string + external_resource_id: string + resource_version: number + serving: { + active_resource_version: number | null + active_sha256: string | null + } + error_code: string | null + statusDetail: string | null + correlation_id: string +} + +type KBIngestionWebhookResult = { + statusCode: number + body: { ok: true } | { error: string } +} + +function getHeader(headers: WebhookHeaders, name: string) { + const value = headers[name] + return typeof value === 'string' ? value : undefined +} + +// The canonical JSON below is what the webhook signature is verified against, so +// key order must be byte-stable and locale-independent rather than collated. +function compareCodeUnits(a: string, b: string) { + return a < b ? -1 : a > b ? 1 : 0 +} + +function hasExactKeys(value: Record, keys: string[]) { + const actualKeys = Object.keys(value).sort(compareCodeUnits) + const expectedKeys = [...keys].sort(compareCodeUnits) + return ( + actualKeys.length === expectedKeys.length && + actualKeys.every((key, index) => key === expectedKeys[index]) + ) +} + +function isString(value: unknown, minLength: number, maxLength: number) { + return ( + typeof value === 'string' && + value.length >= minLength && + value.length <= maxLength + ) +} + +function isResourceVersion(value: unknown): value is number { + return ( + typeof value === 'number' && + Number.isSafeInteger(value) && + value > 0 && + value <= MAX_RESOURCE_VERSION + ) +} + +function isNullableResourceVersion(value: unknown): value is number | null { + return value === null || isResourceVersion(value) +} + +function isNullableSha256(value: unknown): value is string | null { + return ( + value === null || (typeof value === 'string' && SHA256_PATTERN.test(value)) + ) +} + +function isCanonicalTimestamp(value: unknown): value is string { + if (typeof value !== 'string' || !CANONICAL_TIMESTAMP_PATTERN.test(value)) { + return false + } + const parsed = new Date(value) + return ( + Number.isFinite(parsed.getTime()) && + parsed.toISOString().replace('.000Z', 'Z') === value + ) +} + +function canonicalJson(value: unknown): string { + if (Array.isArray(value)) { + return `[${value.map(canonicalJson).join(',')}]` + } + if (value !== null && typeof value === 'object') { + const record = value as Record + return `{${Object.keys(record) + .sort(compareCodeUnits) + .map((key) => `${JSON.stringify(key)}:${canonicalJson(record[key])}`) + .join(',')}}` + } + return JSON.stringify(value) ?? 'null' +} + +function parsePayload(rawBody: Buffer): OperationStatusEvent | null { + let value: unknown + try { + value = JSON.parse(rawBody.toString('utf8')) + } catch { + return null + } + if (!value || typeof value !== 'object' || Array.isArray(value)) return null + + const payload = value as Record + if ( + !hasExactKeys(payload, [ + 'eventId', + 'eventType', + 'occurredAt', + 'operation_id', + 'external_resource_id', + 'resource_version', + 'serving', + 'error_code', + 'statusDetail', + 'correlation_id', + ]) || + typeof payload.eventId !== 'string' || + !UUID_PATTERN.test(payload.eventId) || + !EVENT_TYPES.includes(payload.eventType as OperationStatusEventType) || + !isCanonicalTimestamp(payload.occurredAt) || + !isString(payload.operation_id, 1, 255) || + !isString(payload.external_resource_id, 1, 512) || + !isResourceVersion(payload.resource_version) || + !payload.serving || + typeof payload.serving !== 'object' || + Array.isArray(payload.serving) || + !hasExactKeys(payload.serving as Record, [ + 'active_resource_version', + 'active_sha256', + ]) || + !isNullableResourceVersion( + (payload.serving as Record).active_resource_version + ) || + !isNullableSha256( + (payload.serving as Record).active_sha256 + ) || + (payload.error_code !== null && !isString(payload.error_code, 0, 128)) || + (payload.statusDetail !== null && + !isString(payload.statusDetail, 0, 512)) || + !isString(payload.correlation_id, 1, 255) + ) { + return null + } + + const parsed = payload as OperationStatusEvent + if (Buffer.from(canonicalJson(parsed), 'utf8').compare(rawBody) !== 0) { + return null + } + return parsed +} + +function signatureMatches({ + rawBody, + signature, + timestamp, + secrets, +}: { + rawBody: Buffer + signature: string + timestamp: string + secrets: string[] +}) { + const provided = Buffer.from(signature, 'hex') + let matches = false + for (const secret of secrets) { + const expected = Buffer.from( + createKBIngestionWebhookSignature({ + rawBody, + secret, + timestamp, + }), + 'hex' + ) + matches = + (expected.length === provided.length && + timingSafeEqual(expected, provided)) || + matches + } + return matches +} + +function transitionForEvent(payload: OperationStatusEvent) { + switch (payload.eventType) { + case 'resource.processing_started': + case 'resource.processing_progress': + return { + resourceStatus: KBResourceStatus.PROCESSING, + runStatus: KBIngestionStatus.PROCESSING, + statusMessage: payload.statusDetail, + finishedAt: undefined, + } + case 'resource.processing_succeeded': + return { + resourceStatus: KBResourceStatus.READY, + runStatus: KBIngestionStatus.SUCCEEDED, + statusMessage: payload.statusDetail, + finishedAt: new Date(payload.occurredAt), + } + case 'resource.processing_failed': + return { + resourceStatus: KBResourceStatus.FAILED, + runStatus: KBIngestionStatus.FAILED, + statusMessage: + payload.statusDetail ?? 'The ingestion operation failed.', + finishedAt: new Date(payload.occurredAt), + } + case 'resource.content_refreshed': + case 'resource.subresources_updated': + case 'kb.metrics_updated': + return null + } +} + +export async function handleKBIngestionWebhook({ + prisma, + rawBody, + headers, + env = process.env, + now = () => new Date(), +}: { + prisma: PrismaClient + rawBody: Buffer + headers: WebhookHeaders + env?: NodeJS.ProcessEnv + now?: () => Date +}): Promise { + const currentSecret = env.KB_WEBHOOK_SECRET + if (!currentSecret) { + return { statusCode: 503, body: { error: 'Service unavailable' } } + } + const secrets = [ + currentSecret, + ...(env.KB_WEBHOOK_PREVIOUS_SECRET ? [env.KB_WEBHOOK_PREVIOUS_SECRET] : []), + ] + + const eventIdHeader = getHeader(headers, 'x-ingestion-event-id') + const eventTypeHeader = getHeader(headers, 'x-ingestion-event-type') + const timestampHeader = getHeader(headers, 'x-ingestion-timestamp') + const signatureHeader = getHeader(headers, 'x-ingestion-signature') + if ( + !eventIdHeader || + !UUID_PATTERN.test(eventIdHeader) || + !eventTypeHeader || + !EVENT_TYPES.includes(eventTypeHeader as OperationStatusEventType) || + !timestampHeader || + !/^(0|[1-9]\d*)$/.test(timestampHeader) || + !signatureHeader || + !/^[a-f\d]{64}$/.test(signatureHeader) + ) { + return { statusCode: 401, body: { error: 'Unauthorized' } } + } + + const timestamp = Number(timestampHeader) + const currentTimestamp = Math.floor(now().getTime() / 1000) + if ( + !Number.isSafeInteger(timestamp) || + Math.abs(currentTimestamp - timestamp) > SIGNATURE_MAX_AGE_SECONDS || + !signatureMatches({ + rawBody, + signature: signatureHeader, + timestamp: timestampHeader, + secrets, + }) + ) { + return { statusCode: 401, body: { error: 'Unauthorized' } } + } + + const payload = parsePayload(rawBody) + if ( + !payload || + payload.eventId !== eventIdHeader || + payload.eventType !== eventTypeHeader || + !UUID_PATTERN.test(payload.external_resource_id) + ) { + return { statusCode: 400, body: { error: 'Invalid request' } } + } + + const transition = transitionForEvent(payload) + await prisma.$transaction(async (tx) => { + if (payload.eventType === 'resource.content_refreshed') { + const activeResourceVersion = payload.serving.active_resource_version + const activeContentSha256 = payload.serving.active_sha256 + if ( + activeResourceVersion === null || + activeContentSha256 === null || + activeResourceVersion !== payload.resource_version || + payload.error_code !== null + ) { + return + } + + const resources = await tx.$queryRaw< + Array<{ + id: string + deletedAt: Date | null + activeResourceVersion: number | null + ingestedAt: Date | null + }> + >` + SELECT + resource."id", + resource."deletedAt", + resource."activeResourceVersion", + resource."ingestedAt" + FROM "public"."KBResource" AS resource + WHERE resource."id" = CAST(${payload.external_resource_id} AS UUID) + FOR UPDATE + ` + const resource = resources[0] + if (!resource || resource.deletedAt !== null) { + return + } + + const existingRun = await tx.kBIngestionRun.findFirst({ + where: { + resourceId: payload.external_resource_id, + externalOperationId: payload.operation_id, + }, + select: { id: true }, + }) + if (existingRun) { + return + } + + const occurredAt = new Date(payload.occurredAt) + const refreshWasSuperseded = + resource.activeResourceVersion !== null && + (resource.activeResourceVersion > activeResourceVersion || + (resource.activeResourceVersion === activeResourceVersion && + resource.ingestedAt !== null && + resource.ingestedAt >= occurredAt)) + + await tx.kBIngestionRun.create({ + data: { + id: payload.eventId, + operation: KBIngestionOperation.UPSERT, + status: refreshWasSuperseded + ? KBIngestionStatus.SUPERSEDED + : KBIngestionStatus.SUCCEEDED, + resourceVersion: payload.resource_version, + contentSha256: activeContentSha256, + externalOperationId: payload.operation_id, + statusMessage: refreshWasSuperseded + ? 'The platform refresh was superseded by a newer serving revision.' + : payload.statusDetail, + finishedAt: occurredAt, + resourceId: payload.external_resource_id, + }, + }) + if (!refreshWasSuperseded) { + await tx.kBResource.update({ + where: { id: resource.id }, + data: { + activeResourceVersion, + activeContentSha256, + ingestedAt: occurredAt, + }, + }) + } + return + } + + const resource = await tx.kBResource.findFirst({ + where: { + id: payload.external_resource_id, + resourceVersion: payload.resource_version, + externalOperationId: payload.operation_id, + ingestionAttemptId: { not: null }, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + select: { + ingestionAttemptId: true, + contentSha256: true, + ingestionOperation: true, + }, + }) + if (!resource?.ingestionAttemptId) { + return + } + + const run = await tx.kBIngestionRun.findUnique({ + where: { id: resource.ingestionAttemptId }, + select: { status: true, operation: true }, + }) + if (!run || run.operation !== resource.ingestionOperation) { + return + } + const servingMatchesCurrent = + resource.ingestionOperation === KBIngestionOperation.DELETE + ? payload.serving.active_resource_version === null && + payload.serving.active_sha256 === null + : payload.serving.active_resource_version === + payload.resource_version && + payload.serving.active_sha256 !== null && + payload.serving.active_sha256 === resource.contentSha256 + const servingState = { + activeResourceVersion: payload.serving.active_resource_version, + activeContentSha256: payload.serving.active_sha256, + } + + if (!transition) { + await tx.kBResource.updateMany({ + where: { + id: payload.external_resource_id, + resourceVersion: payload.resource_version, + externalOperationId: payload.operation_id, + ingestionAttemptId: resource.ingestionAttemptId, + ingestionOperation: resource.ingestionOperation, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + data: + servingMatchesCurrent && run.status === KBIngestionStatus.SUCCEEDED + ? { + ...servingState, + status: KBResourceStatus.READY, + statusMessage: null, + errorCode: null, + ingestedAt: new Date(payload.occurredAt), + } + : servingState, + }) + return + } + + const resourceStatus = + payload.eventType === 'resource.processing_succeeded' && + !servingMatchesCurrent + ? KBResourceStatus.PROCESSING + : transition.resourceStatus + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: payload.external_resource_id, + resourceVersion: payload.resource_version, + externalOperationId: payload.operation_id, + ingestionAttemptId: resource.ingestionAttemptId, + ingestionOperation: resource.ingestionOperation, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + data: { + ...servingState, + status: resourceStatus, + statusMessage: transition.statusMessage, + errorCode: payload.error_code, + ...(resourceStatus === KBResourceStatus.READY + ? { ingestedAt: new Date(payload.occurredAt) } + : {}), + }, + }) + if (resourceUpdate.count !== 1) { + return + } + + const sourceRunStatuses = + transition.runStatus === KBIngestionStatus.SUCCEEDED + ? [ + KBIngestionStatus.QUEUED, + KBIngestionStatus.PROCESSING, + KBIngestionStatus.SUCCEEDED, + ] + : [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING] + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: resource.ingestionAttemptId, + resourceId: payload.external_resource_id, + resourceVersion: payload.resource_version, + operation: resource.ingestionOperation, + status: { in: sourceRunStatuses }, + }, + data: { + status: transition.runStatus, + statusMessage: transition.statusMessage, + errorCode: payload.error_code, + ...(payload.eventType === 'resource.processing_started' + ? { startedAt: new Date(payload.occurredAt) } + : {}), + ...(transition.finishedAt ? { finishedAt: transition.finishedAt } : {}), + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion run transition could not be correlated') + } + }) + + return { statusCode: 200, body: { ok: true } } +} diff --git a/packages/graphql/src/types/app.ts b/packages/graphql/src/types/app.ts index 74189ef22e..dd8bd3c773 100644 --- a/packages/graphql/src/types/app.ts +++ b/packages/graphql/src/types/app.ts @@ -13,6 +13,22 @@ import type { SingleQuestionResponseLiveQuiz, } from '@klicker-uzh/types' +type KBGraphMeteredCost = { + currency: string + amount_minor_units: number + components: Array<{ + provider: string + model: string + amount_minor_units: number + pricing_version: string + embedding_tokens: number + input_tokens: number + output_tokens: number + request_count: number + }> + metering_source: 'provider_reported' | 'configured_pricing' +} + export type PrismaMigrationClient = Omit< InstanceType, '$connect' | '$disconnect' | '$on' | '$transaction' | '$use' | '$extends' @@ -43,6 +59,7 @@ declare global { type PrismaGroupActivityResults = GroupActivityResults type PrismaActivityLogModificationDetails = ActivityLogModificationDetails type PrismaAssessmentReportSnapshot = AssessmentReportSnapshot + type PrismaKBGraphMeteredCost = KBGraphMeteredCost } } // #endregion diff --git a/packages/graphql/test/helpers.ts b/packages/graphql/test/helpers.ts index 84c4580edf..41aa3280c3 100644 --- a/packages/graphql/test/helpers.ts +++ b/packages/graphql/test/helpers.ts @@ -13,7 +13,6 @@ import { } from '@/services/microLearning.js' import { handlePublishScheduledPracticeQuiz } from '@/services/practiceQuizzes.js' import type { Hatchet } from '@hatchet-dev/typescript-sdk' -import { hatchetClient } from '@klicker-uzh/hatchet' import { prisma } from '@klicker-uzh/prisma' import { AnswerCollection, @@ -32,10 +31,12 @@ import { UserRole, } from '@klicker-uzh/prisma/client' import { + BuildKBGraphInput, DisplayMode, ElementData, ElementInstanceOptions, ElementInstanceResults, + IngestKBResourceInput, } from '@klicker-uzh/types' import { getInitialInstanceResults, @@ -127,8 +128,14 @@ export async function testInitialization( }) const pubSub = createPubSub() - const redisExec = new Redis({ host: '127.0.0.1', port: 6379 }) - const redisAssessmentExec = new Redis({ host: '127.0.0.1', port: 6380 }) + const redisExec = new Redis({ + host: process.env.REDIS_HOST ?? '127.0.0.1', + port: Number(process.env.REDIS_PORT ?? 6379), + }) + const redisAssessmentExec = new Redis({ + host: process.env.REDIS_ASSESSMENT_HOST ?? '127.0.0.1', + port: Number(process.env.REDIS_ASSESSMENT_PORT ?? 6380), + }) const hatchetCtx = { hatchet, @@ -141,6 +148,27 @@ export async function testInitialization( // initialize tasks to be called const tasks = { + ingestKBResource: hatchet.task({ + name: 'ingest-kb-resource', + fn: async (input: IngestKBResourceInput) => { + console.info('KB ingestion dispatch stub triggered', input) + return { success: true } + }, + }), + buildKBGraph: hatchet.task({ + name: 'build-kb-knowledge-graph', + fn: async (input: BuildKBGraphInput) => { + console.info('KB graph build dispatch stub triggered', input) + return { success: true } + }, + }), + deleteKBResource: hatchet.task({ + name: 'delete-kb-resource', + fn: async (input) => { + console.info('KB deletion dispatch stub triggered', input) + return { success: true } + }, + }), createAuditLogEntry: hatchet.task({ name: 'create-audit-log-entry', fn: async ({ @@ -356,6 +384,9 @@ export async function testCleanup(prisma: PrismaClient) { ) } + // upload tickets intentionally restrict KB deletion until retention cleanup + await prisma.kBUploadTicket.deleteMany() + // delete all users, participants and user groups / participant groups that have been added for the test run await prisma.user.deleteMany() await prisma.participant.deleteMany() @@ -382,6 +413,7 @@ export async function initializePrisma() { try { // create EventEmitter for test context const emitter = new EventEmitter() + const { hatchetClient } = await import('@klicker-uzh/hatchet') return { prisma, hatchet: hatchetClient, emitter } } catch (error) { diff --git a/packages/graphql/test/kbGraphContract.test.ts b/packages/graphql/test/kbGraphContract.test.ts new file mode 100644 index 0000000000..4be6b2464d --- /dev/null +++ b/packages/graphql/test/kbGraphContract.test.ts @@ -0,0 +1,400 @@ +import { describe, expect, it } from 'vitest' +import { + KB_GRAPH_CONTRACT_VERSION, + type KbGraphTerminalResult, + validateKbGraphTerminalResult, +} from '../src/services/kbGraphContract.js' + +const validResult: KbGraphTerminalResult = { + contract_version: 'klicker-kb-graph/v1', + result_id: '11111111-1111-4111-8111-111111111111:hatchet-run-001', + build_id: '11111111-1111-4111-8111-111111111111', + kb_id: '22222222-2222-4222-8222-222222222222', + owner_id: '44444444-4444-4444-8444-444444444444', + run_id: 'hatchet-run-001', + source_content_digest: + '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef', + graph_name: + 'klickeruzh:kb:22222222-2222-4222-8222-222222222222:11111111-1111-4111-8111-111111111111', + status: 'SUCCEEDED', + node_count: 10, + edge_count: 20, + processed_document_count: 3, + failed_document_count: 0, + error_code: null, + graphml_artifact: { + container_name: 'kb-44444444-4444-4444-8444-444444444444', + blob_name: 'knowledge-graphs/11111111-1111-4111-8111-111111111111.graphml', + }, + metered_cost: { + currency: 'CHF', + amount_minor_units: 120, + components: [ + { + provider: 'openai', + model: 'gpt-5.6-luna', + amount_minor_units: 120, + pricing_version: '2026-08', + input_tokens: 1000, + output_tokens: 500, + embedding_tokens: 0, + request_count: 2, + }, + ], + metering_source: 'provider_reported', + }, +} + +const validExpectation = { + buildId: '11111111-1111-4111-8111-111111111111', + kbId: '22222222-2222-4222-8222-222222222222', + ownerId: '44444444-4444-4444-8444-444444444444', + resultId: '11111111-1111-4111-8111-111111111111:hatchet-run-001', + runId: 'hatchet-run-001', + estimatedMinorUnits: 200, +} + +describe('kbGraphContract', () => { + it('exports the v1 contract version constant', () => { + expect(KB_GRAPH_CONTRACT_VERSION).toBe('klicker-kb-graph/v1') + }) + + it('accepts a valid terminal result matching the expectation', () => { + const validation = validateKbGraphTerminalResult( + validResult, + validExpectation + ) + + expect(validation.ok).toBe(true) + if (validation.ok) { + expect(validation.result.build_id).toBe( + '11111111-1111-4111-8111-111111111111' + ) + expect(validation.result.metered_cost?.amount_minor_units).toBe(120) + } + }) + + it('rejects a result with a mismatched build id', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + build_id: '33333333-3333-4333-8333-333333333333', + result_id: '33333333-3333-4333-8333-333333333333:hatchet-run-001', + graph_name: + 'klickeruzh:kb:22222222-2222-4222-8222-222222222222:33333333-3333-4333-8333-333333333333', + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('build_id mismatch') + } + }) + + it('rejects a result with a mismatched kb id', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + kb_id: '55555555-5555-4555-8555-555555555555', + graph_name: + 'klickeruzh:kb:55555555-5555-4555-8555-555555555555:11111111-1111-4111-8111-111111111111', + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('kb_id mismatch') + } + }) + + it('rejects a result with a mismatched owner id', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + owner_id: '66666666-6666-4666-8666-666666666666', + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('owner_id mismatch') + } + }) + + it('rejects a result with a mismatched result id', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + run_id: 'hatchet-run-002', + result_id: '11111111-1111-4111-8111-111111111111:hatchet-run-002', + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('result_id mismatch') + } + }) + + it('rejects a result with a mismatched run id', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + run_id: 'run-other', + result_id: '11111111-1111-4111-8111-111111111111:run-other', + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('run_id mismatch') + } + }) + + it('rejects a result with a malformed metered cost', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + metered_cost: { + ...validResult.metered_cost!, + amount_minor_units: -5, + }, + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('amount_minor_units') + } + }) + + it('rejects a result with a negative node count', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, node_count: -1 }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('node_count') + } + }) + + it('rejects a result with a non-integer edge count', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, edge_count: 1.5 }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('edge_count') + } + }) + + it('rejects a result with a negative component amount', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + metered_cost: { + ...validResult.metered_cost!, + components: [ + { + ...validResult.metered_cost!.components[0], + amount_minor_units: -1, + }, + ], + }, + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('amount_minor_units') + } + }) + + it('rejects a result with a non-integer component token count', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + metered_cost: { + ...validResult.metered_cost!, + components: [ + { + ...validResult.metered_cost!.components[0], + input_tokens: 1.5, + }, + ], + }, + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('input_tokens') + } + }) + + it('rejects component counters above the database integer range', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + metered_cost: { + ...validResult.metered_cost!, + components: [ + { + ...validResult.metered_cost!.components[0], + input_tokens: 2_147_483_648, + }, + ], + }, + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('input_tokens') + } + }) + + it('rejects a result that exceeds the estimated reservation', () => { + const validation = validateKbGraphTerminalResult(validResult, { + ...validExpectation, + estimatedMinorUnits: 100, + }) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain( + 'exceeds estimated reservation' + ) + } + }) + + it('accepts a result whose metered cost equals the reservation', () => { + const validation = validateKbGraphTerminalResult(validResult, { + ...validExpectation, + estimatedMinorUnits: 120, + }) + + expect(validation.ok).toBe(true) + }) + + it('rejects a negative estimated reservation', () => { + const validation = validateKbGraphTerminalResult(validResult, { + ...validExpectation, + estimatedMinorUnits: -1, + }) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('estimatedMinorUnits') + } + }) + + it('rejects a non-integer estimated reservation', () => { + const validation = validateKbGraphTerminalResult(validResult, { + ...validExpectation, + estimatedMinorUnits: 1.5, + }) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('estimatedMinorUnits') + } + }) + + it('rejects a result with an unknown contract version', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, contract_version: 'klicker-kb-graph/v2' }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('contract_version') + } + }) + + it('rejects a result with an unknown status', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, status: 'RUNNING' }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('status') + } + }) + + it('rejects a result with an unknown extra property', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, unexpected_field: true }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain('unexpected_field') + } + }) + + it('rejects a successful result without a graph artifact', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, graphml_artifact: null }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain( + 'SUCCEEDED results require a GraphML artifact' + ) + } + }) + + it('rejects a non-success result without an error code', () => { + const validation = validateKbGraphTerminalResult( + { ...validResult, status: 'FAILED', error_code: null }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain( + 'non-success terminal results require error_code' + ) + } + }) + + it('rejects metered components whose amounts do not add up', () => { + const validation = validateKbGraphTerminalResult( + { + ...validResult, + metered_cost: { + ...validResult.metered_cost!, + amount_minor_units: 121, + }, + }, + validExpectation + ) + + expect(validation.ok).toBe(false) + if (!validation.ok) { + expect(validation.errors.join(' ')).toContain( + 'amount_minor_units must equal the component total' + ) + } + }) +}) diff --git a/packages/graphql/test/knowledge.test.ts b/packages/graphql/test/knowledge.test.ts new file mode 100644 index 0000000000..14a3124433 --- /dev/null +++ b/packages/graphql/test/knowledge.test.ts @@ -0,0 +1,2544 @@ +import { BlobServiceClient } from '@azure/storage-blob' +import type { Hatchet } from '@hatchet-dev/typescript-sdk' +import { prisma as prismaClient } from '@klicker-uzh/prisma' +import { + KBGraphBuildStatus, + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + KBResourceType, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import { + MAX_KB_RESOURCE_COUNT, + MAX_KB_TOTAL_SIZE_BYTES, +} from '@klicker-uzh/types' +import { randomUUID } from 'crypto' +import { EventEmitter } from 'events' +import { readFileSync } from 'fs' +import { buildSchema, parse, validate } from 'graphql' +import { vi } from 'vitest' +import type { ContextWithUser } from '../src/lib/context.js' +import { + attachKbToChatbot, + confirmKbFileUpload, + createKb, + createKbUrlResource, + deleteKb, + deleteKbResource, + deleteKbResources, + detachKbFromChatbot, + getKb, + getKbChatbotBindings, + getKbResourceIngestionRuns, + getKbResourcesConnection, + getUserKbsConnection, + ingestKbResource, + rebuildKbKnowledgeGraph, + requestKbFileUpload, + setKbKnowledgeGraphEnabled, +} from '../src/services/knowledge.js' +import { seedCourse, testCleanup, testInitialization } from './helpers.js' + +function createDeferred() { + let resolve!: (value: T) => void + const promise = new Promise((resolvePromise) => { + resolve = resolvePromise + }) + return { promise, resolve } +} + +function legacyUrlResources(kbId: string, count: number) { + return Array.from({ length: count }, (_, index) => ({ + kbId, + type: KBResourceType.URL, + title: `Legacy URL ${index}`, + sourceUrl: `https://example.com/legacy-${index}`, + })) +} + +function withIngestionClaimSignal( + ctx: ContextWithUser, + onClaim: () => void +): ContextWithUser { + const prisma = ctx.prisma.$extends({ + query: { + kBResource: { + updateMany({ args, query }) { + onClaim() + return query(args) + }, + }, + }, + }) + return { ...ctx, prisma: prisma as unknown as PrismaClient } +} + +function withTombstonePause( + ctx: ContextWithUser, + model: 'kB' | 'kBResource', + onTombstone: () => void, + continueTombstone: Promise +): ContextWithUser { + const queryExtension = { + async update({ + args, + query, + }: { + args: { data: { deletedAt?: unknown } } + query: (args: unknown) => Promise + }) { + if (args.data.deletedAt) { + onTombstone() + await continueTombstone + } + return query(args) + }, + } + const prisma = ctx.prisma.$extends({ + query: (model === 'kB' + ? { kB: queryExtension } + : { kBResource: queryExtension }) as never, + }) + return { ...ctx, prisma: prisma as unknown as PrismaClient } +} + +function withKbBindingSnapshotPause( + ctx: ContextWithUser, + kbId: string, + onSnapshot: () => void, + continueSnapshot: Promise +): ContextWithUser { + const prisma = ctx.prisma.$extends({ + query: { + kBChatbot: { + async findMany({ args, query }) { + const result = await query(args) + if (args.where?.kbId === kbId && args.where.isEnabled === true) { + onSnapshot() + await continueSnapshot + } + return result + }, + }, + }, + }) + return { ...ctx, prisma: prisma as unknown as PrismaClient } +} + +describe('Knowledge base GraphQL contract', () => { + it('requires the resource id for ingestion', () => { + const schema = buildSchema( + readFileSync( + new URL('../src/public/schema.graphql', import.meta.url), + 'utf8' + ) + ) + const document = parse(` + mutation { + ingestKbResource { + id + } + } + `) + + expect(validate(schema, document).map(({ message }) => message)).toEqual([ + 'Field "ingestKbResource" argument "id" of type "ID!" is required, but it was not provided.', + ]) + }) +}) + +describe('Integration tests for knowledge base CRUD', () => { + let prisma: PrismaClient + let hatchet: Hatchet + let emitter: EventEmitter + let userOneCtx: ContextWithUser + let userTwoCtx: ContextWithUser + let nonPreviewCtx: ContextWithUser + let previousBlobAccountName: string | undefined + let previousBlobAccessKey: string | undefined + let previousBlobAccountUrl: string | undefined + let previousBlobInternalAccountUrl: string | undefined + let containerName: string + let requestedBlobName: string + let createIfNotExists: ReturnType + let blobExists: ReturnType + let getBlobProperties: ReturnType + let deleteBlobIfExists: ReturnType + let getBlobClient: ReturnType + let blobServiceUrl: string + const graphCostEnvironmentKeys = [ + 'KB_GRAPH_DISABLED', + 'KB_GRAPH_COST_CURRENCY', + 'KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS', + 'KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS', + 'KB_GRAPH_MAX_COST_MINOR_UNITS', + 'KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS', + 'KB_GRAPH_COST_PRICING_VERSION', + 'KB_GRAPH_SEMESTER_KEY', + ] as const + let previousGraphCostEnvironment: Partial< + Record<(typeof graphCostEnvironmentKeys)[number], string | undefined> + > + + beforeAll(async () => { + prisma = prismaClient + await testCleanup(prisma) + hatchet = { + task: vi.fn(() => ({ runNoWait: vi.fn() })), + } as unknown as Hatchet + emitter = new EventEmitter() + }) + + afterAll(async () => { + await testCleanup(prisma) + await prisma.$disconnect() + }) + + beforeEach(async () => { + const initialized = await testInitialization(prisma, hatchet, emitter) + userOneCtx = initialized.userOneCtx + userTwoCtx = initialized.userTwoCtx + nonPreviewCtx = initialized.userThreeCtx + // the shared fixture users default to privatePreview: false; the KB workspace + // gate requires preview access, so opt both active owners in explicitly and + // leave nonPreviewCtx's backing user at the default (privatePreview: false) + await prisma.user.updateMany({ + where: { id: { in: [userOneCtx.user.sub, userTwoCtx.user.sub] } }, + data: { privatePreview: true }, + }) + await prisma.chatbotMCPServer.upsert({ + where: { name: 'KB' }, + create: { + name: 'KB', + description: 'Knowledge base retrieval', + url: 'http://localhost:1417/mcp', + authType: 'scope_token', + isActive: true, + }, + update: { + authType: 'scope_token', + isActive: true, + }, + }) + + previousBlobAccountName = process.env.BLOB_STORAGE_ACCOUNT_NAME + previousBlobAccessKey = process.env.BLOB_STORAGE_ACCESS_KEY + previousBlobAccountUrl = process.env.BLOB_STORAGE_ACCOUNT_URL + previousBlobInternalAccountUrl = + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL + process.env.BLOB_STORAGE_ACCOUNT_NAME = 'kbtestaccount' + process.env.BLOB_STORAGE_ACCESS_KEY = Buffer.alloc(32).toString('base64') + delete process.env.BLOB_STORAGE_ACCOUNT_URL + delete process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL + + previousGraphCostEnvironment = Object.fromEntries( + graphCostEnvironmentKeys.map((key) => [key, process.env[key]]) + ) + Object.assign(process.env, { + KB_GRAPH_COST_CURRENCY: 'CHF', + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '100', + KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS: '200', + KB_GRAPH_MAX_COST_MINOR_UNITS: '200', + KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '500', + KB_GRAPH_COST_PRICING_VERSION: 'test-v1', + KB_GRAPH_SEMESTER_KEY: '2026-H2', + }) + + containerName = '' + requestedBlobName = '' + blobServiceUrl = '' + createIfNotExists = vi.fn().mockResolvedValue({ succeeded: true }) + blobExists = vi.fn().mockResolvedValue(true) + getBlobProperties = vi.fn().mockResolvedValue({ + contentLength: 1024, + contentType: 'application/pdf', + }) + deleteBlobIfExists = vi.fn().mockResolvedValue({ succeeded: true }) + const blobClient = { + get url() { + return `https://kbtestaccount.blob.core.windows.net/${containerName}/${requestedBlobName}` + }, + exists: blobExists, + getProperties: getBlobProperties, + deleteIfExists: deleteBlobIfExists, + } + getBlobClient = vi.fn().mockImplementation((blobName: string) => { + requestedBlobName = blobName + return blobClient + }) + const containerClient = { + get containerName() { + return containerName + }, + createIfNotExists, + getBlobClient, + } + vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ).mockImplementation(function (this: BlobServiceClient, name: string) { + blobServiceUrl = this.url + containerName = name + return containerClient as never + }) + }) + + afterEach(async () => { + vi.restoreAllMocks() + if (previousBlobAccountName === undefined) { + delete process.env.BLOB_STORAGE_ACCOUNT_NAME + } else { + process.env.BLOB_STORAGE_ACCOUNT_NAME = previousBlobAccountName + } + if (previousBlobAccessKey === undefined) { + delete process.env.BLOB_STORAGE_ACCESS_KEY + } else { + process.env.BLOB_STORAGE_ACCESS_KEY = previousBlobAccessKey + } + if (previousBlobAccountUrl === undefined) { + delete process.env.BLOB_STORAGE_ACCOUNT_URL + } else { + process.env.BLOB_STORAGE_ACCOUNT_URL = previousBlobAccountUrl + } + if (previousBlobInternalAccountUrl === undefined) { + delete process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL + } else { + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL = + previousBlobInternalAccountUrl + } + for (const key of graphCostEnvironmentKeys) { + const value = previousGraphCostEnvironment[key] + if (value === undefined) delete process.env[key] + else process.env[key] = value + } + await testCleanup(prisma) + }) + + it('requires graph opt-in and the kill switch before dispatching a build', async () => { + const kb = await createKb({ name: 'Graph controls' }, userOneCtx) + + await expect( + rebuildKbKnowledgeGraph({ kbId: kb.id }, userOneCtx) + ).rejects.toMatchObject({ + extensions: { code: 'KB_GRAPH_NOT_ENABLED' }, + }) + + process.env.KB_GRAPH_DISABLED = 'true' + await expect( + setKbKnowledgeGraphEnabled({ kbId: kb.id, enabled: true }, userOneCtx) + ).rejects.toMatchObject({ + extensions: { code: 'KB_GRAPH_DISABLED' }, + }) + + process.env.KB_GRAPH_DISABLED = 'false' + await setKbKnowledgeGraphEnabled({ kbId: kb.id, enabled: true }, userOneCtx) + await prisma.kBResource.create({ + data: { + kbId: kb.id, + type: KBResourceType.URL, + title: 'Graph source', + sourceUrl: 'https://example.com/graph-source', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: 'a'.repeat(64), + }, + }) + + const config = await rebuildKbKnowledgeGraph({ kbId: kb.id }, userOneCtx) + + expect(config).toMatchObject({ + status: KBGraphBuildStatus.QUEUED, + costConfigurationReady: true, + estimatedCostMinorUnits: 100, + semesterReservedMinorUnits: 100, + }) + }) + + it('fences rebuilds after an accepted but uncorrelated graph dispatch', async () => { + const kb = await createKb({ name: 'Ambiguous graph dispatch' }, userOneCtx) + await setKbKnowledgeGraphEnabled({ kbId: kb.id, enabled: true }, userOneCtx) + await prisma.kBResource.create({ + data: { + kbId: kb.id, + type: KBResourceType.URL, + title: 'Graph source', + sourceUrl: 'https://example.com/ambiguous-graph-source', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: 'b'.repeat(64), + }, + }) + + const initial = await rebuildKbKnowledgeGraph({ kbId: kb.id }, userOneCtx) + expect(initial.buildId).not.toBeNull() + await prisma.kBGraphBuild.update({ + where: { id: initial.buildId! }, + data: { + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_DISPATCH_AMBIGUOUS', + statusMessage: + 'The external KB graph workflow may have been accepted but could not be correlated; manual review is required.', + finishedAt: new Date(), + }, + }) + + await expect( + rebuildKbKnowledgeGraph({ kbId: kb.id }, userOneCtx) + ).rejects.toMatchObject({ + extensions: { code: 'KB_GRAPH_DISPATCH_AMBIGUOUS' }, + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kb.id } }) + ).resolves.toMatchObject({ activeGraphBuildId: initial.buildId }) + await expect( + prisma.kBGraphBuild.count({ where: { kbId: kb.id } }) + ).resolves.toBe(1) + }) + + it('creates and lists only the current users knowledge bases', async () => { + const created = await createKb( + { name: 'Finance notes', description: 'Course material' }, + userOneCtx + ) + await createKb({ name: 'Other owner' }, userTwoCtx) + + const userKbs = (await getUserKbsConnection({}, userOneCtx)).items + + expect(userKbs).toHaveLength(1) + expect(userKbs[0]).toMatchObject({ + id: created.id, + name: 'Finance notes', + description: 'Course material', + ownerId: userOneCtx.user.sub, + }) + }) + + it('returns owned knowledge base metadata without an unbounded child list', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + + const kb = await getKb({ id: created.id }, userOneCtx) + + expect(kb.id).toBe(created.id) + }) + + it('lists owned chatbots with their enabled knowledge base', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const otherKb = await createKb({ name: 'Other notes' }, userTwoCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + const otherCourse = await seedCourse({}, userTwoCtx) + const otherChatbot = await prisma.chatbot.create({ + data: { + name: 'Other tutor', + ownerId: userTwoCtx.user.sub, + courseId: otherCourse.id, + }, + }) + await prisma.kBChatbot.create({ + data: { kbId: otherKb.id, chatbotId: otherChatbot.id }, + }) + + await attachKbToChatbot({ kbId: kb.id, chatbotId: chatbot.id }, userOneCtx) + const bindings = await getKbChatbotBindings({ kbId: kb.id }, userOneCtx) + + expect(bindings).toEqual([ + { + chatbotId: chatbot.id, + chatbotName: 'Finance tutor', + enabledKbId: kb.id, + enabledKbName: 'Finance notes', + }, + ]) + }) + + it('replaces the enabled binding and provisions only doc_query', async () => { + const firstKb = await createKb({ name: 'First KB' }, userOneCtx) + const secondKb = await createKb({ name: 'Second KB' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + + await attachKbToChatbot( + { kbId: firstKb.id, chatbotId: chatbot.id }, + userOneCtx + ) + await attachKbToChatbot( + { kbId: secondKb.id, chatbotId: chatbot.id }, + userOneCtx + ) + + const links = await prisma.kBChatbot.findMany({ + where: { chatbotId: chatbot.id }, + orderBy: { kbId: 'asc' }, + }) + expect(links).toHaveLength(2) + expect(links.filter(({ isEnabled }) => isEnabled)).toEqual([ + expect.objectContaining({ kbId: secondKb.id }), + ]) + + const configurations = await prisma.chatbotMCPConfig.findMany({ + where: { chatbotId: chatbot.id, mcpServer: { name: 'KB' } }, + orderBy: { chatMode: 'asc' }, + }) + expect(configurations).toHaveLength(2) + expect(configurations).toEqual([ + expect.objectContaining({ + chatMode: 'explainer', + allowedTools: ['doc_query'], + isEnabled: true, + }), + expect.objectContaining({ + chatMode: 'tutor', + allowedTools: ['doc_query'], + isEnabled: true, + }), + ]) + }) + + it('serializes concurrent replacements to one enabled binding', async () => { + const firstKb = await createKb({ name: 'First KB' }, userOneCtx) + const secondKb = await createKb({ name: 'Second KB' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + + await Promise.all([ + attachKbToChatbot( + { kbId: firstKb.id, chatbotId: chatbot.id }, + userOneCtx + ), + attachKbToChatbot( + { kbId: secondKb.id, chatbotId: chatbot.id }, + userOneCtx + ), + ]) + + await expect( + prisma.kBChatbot.count({ + where: { chatbotId: chatbot.id, isEnabled: true }, + }) + ).resolves.toBe(1) + }) + + it('rejects bindings across ownership boundaries', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const otherKb = await createKb({ name: 'Other notes' }, userTwoCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + const otherCourse = await seedCourse({}, userTwoCtx) + const otherChatbot = await prisma.chatbot.create({ + data: { + name: 'Other tutor', + ownerId: userTwoCtx.user.sub, + courseId: otherCourse.id, + }, + }) + + await expect( + attachKbToChatbot({ kbId: otherKb.id, chatbotId: chatbot.id }, userOneCtx) + ).rejects.toThrow('KB not found') + await expect( + attachKbToChatbot({ kbId: kb.id, chatbotId: otherChatbot.id }, userOneCtx) + ).rejects.toThrow('Chatbot not found') + }) + + it('fails closed when knowledge retrieval is inactive', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await prisma.chatbotMCPServer.update({ + where: { name: 'KB' }, + data: { isActive: false }, + }) + + await expect( + attachKbToChatbot({ kbId: kb.id, chatbotId: chatbot.id }, userOneCtx) + ).rejects.toThrow('Knowledge base retrieval is not configured') + await expect( + prisma.kBChatbot.count({ where: { chatbotId: chatbot.id } }) + ).resolves.toBe(0) + }) + + it('accepts the existing KB server row during the compatible rollout', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await prisma.chatbotMCPServer.update({ + where: { name: 'KB' }, + data: { + authType: 'bearer', + authSecret: 'encrypted-legacy-auth-placeholder', + }, + }) + + await expect( + attachKbToChatbot({ kbId: kb.id, chatbotId: chatbot.id }, userOneCtx) + ).resolves.toMatchObject({ + chatbotId: chatbot.id, + enabledKbId: kb.id, + }) + }) + + it('detaches the binding and disables KB MCP configurations', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await attachKbToChatbot({ kbId: kb.id, chatbotId: chatbot.id }, userOneCtx) + + await detachKbFromChatbot( + { kbId: kb.id, chatbotId: chatbot.id }, + userOneCtx + ) + + await expect( + prisma.kBChatbot.count({ where: { chatbotId: chatbot.id } }) + ).resolves.toBe(0) + const configurations = await prisma.chatbotMCPConfig.findMany({ + where: { chatbotId: chatbot.id, mcpServer: { name: 'KB' } }, + }) + expect(configurations).toHaveLength(2) + expect(configurations.every(({ isEnabled }) => !isEnabled)).toBe(true) + }) + + it('disables chatbot retrieval when its knowledge base is tombstoned', async () => { + const kb = await createKb({ name: 'Finance notes' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await attachKbToChatbot({ kbId: kb.id, chatbotId: chatbot.id }, userOneCtx) + + await deleteKb({ id: kb.id }, userOneCtx) + + await expect( + prisma.kBChatbot.findUnique({ + where: { kbId_chatbotId: { kbId: kb.id, chatbotId: chatbot.id } }, + }) + ).resolves.toMatchObject({ isEnabled: false }) + const configurations = await prisma.chatbotMCPConfig.findMany({ + where: { chatbotId: chatbot.id, mcpServer: { name: 'KB' } }, + }) + expect(configurations).toHaveLength(2) + expect(configurations.every(({ isEnabled }) => !isEnabled)).toBe(true) + }) + + it('preserves retrieval when another knowledge base is attached during deletion', async () => { + const firstKb = await createKb({ name: 'First KB' }, userOneCtx) + const secondKb = await createKb({ name: 'Second KB' }, userOneCtx) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Finance tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await attachKbToChatbot( + { kbId: firstKb.id, chatbotId: chatbot.id }, + userOneCtx + ) + const bindingSnapshotted = createDeferred() + const finishDeletion = createDeferred() + const deleteCtx = withKbBindingSnapshotPause( + userOneCtx, + firstKb.id, + () => bindingSnapshotted.resolve(undefined), + finishDeletion.promise + ) + + const deletion = deleteKb({ id: firstKb.id }, deleteCtx) + await bindingSnapshotted.promise + await attachKbToChatbot( + { kbId: secondKb.id, chatbotId: chatbot.id }, + userOneCtx + ) + finishDeletion.resolve(undefined) + await deletion + + await expect( + prisma.kBChatbot.findFirst({ + where: { chatbotId: chatbot.id, isEnabled: true }, + select: { kbId: true }, + }) + ).resolves.toEqual({ kbId: secondKb.id }) + const configurations = await prisma.chatbotMCPConfig.findMany({ + where: { chatbotId: chatbot.id, mcpServer: { name: 'KB' } }, + }) + expect(configurations).toHaveLength(2) + expect(configurations.every(({ isEnabled }) => isEnabled)).toBe(true) + }) + + it('rejects an empty knowledge base name', async () => { + await expect(createKb({ name: ' ' }, userOneCtx)).rejects.toThrow( + 'KB name is required' + ) + + await expect(getUserKbsConnection({}, userOneCtx)).resolves.toMatchObject({ + items: [], + }) + }) + + it('hides an owned knowledge base behind a durable tombstone', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + + await deleteKb({ id: created.id }, userOneCtx) + + await expect(getUserKbsConnection({}, userOneCtx)).resolves.toMatchObject({ + items: [], + }) + await expect(getKb({ id: created.id }, userOneCtx)).rejects.toThrow( + 'KB not found' + ) + await expect( + prisma.kB.findUnique({ where: { id: created.id } }) + ).resolves.toMatchObject({ + deletedById: userOneCtx.user.sub, + deletedAt: expect.any(Date), + }) + }) + + it('tombstones blob resources without deleting storage synchronously', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: 'd6c22240-7380-4bbf-8c7a-2f907b8e2677.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + }, + }) + const runNoWait = vi.spyOn(userOneCtx.tasks.deleteKBResource, 'runNoWait') + + await deleteKb({ id: created.id }, userOneCtx) + + expect(deleteBlobIfExists).not.toHaveBeenCalled() + expect(runNoWait).toHaveBeenCalledOnce() + await expect( + prisma.kB.findUnique({ where: { id: created.id } }) + ).resolves.toMatchObject({ deletedAt: expect.any(Date) }) + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + deletedAt: expect.any(Date), + ingestionOperation: KBIngestionOperation.DELETE, + status: KBResourceStatus.QUEUED, + resourceVersion: 1, + }) + }) + + it.each([ + KBResourceStatus.QUEUED, + KBResourceStatus.PROCESSING, + ])('does not delete a knowledge base with a %s resource', async (status) => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: '83fa9dfa-d796-4f8e-868f-b87a220127b3.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + status, + }, + }) + + await expect(deleteKb({ id: created.id }, userOneCtx)).rejects.toThrow( + 'KB cannot be deleted' + ) + + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kB.findUnique({ where: { id: created.id } }) + ).resolves.toBeTruthy() + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toBeTruthy() + }) + + it('denies reads and deletion to a foreign owner without revealing existence', async () => { + const created = await createKb({ name: 'Private notes' }, userOneCtx) + + await expect(getKb({ id: created.id }, userTwoCtx)).rejects.toThrow( + 'KB not found' + ) + await expect(deleteKb({ id: created.id }, userTwoCtx)).rejects.toThrow( + 'KB not found' + ) + await expect( + prisma.kB.findUnique({ where: { id: created.id } }) + ).resolves.toBeTruthy() + }) + + it('rejects invalid file uploads and foreign knowledge bases before storage access', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'malware.exe', + contentType: 'application/octet-stream', + sizeBytes: 1024, + }, + userOneCtx + ) + ).rejects.toThrow('KB file type is not supported') + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'slides.pptx', + contentType: + 'application/vnd.openxmlformats-officedocument.presentationml.presentation', + sizeBytes: 25 * 1024 * 1024 + 1, + }, + userOneCtx + ) + ).rejects.toThrow('KB file size is invalid') + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'slides.pptx', + contentType: + 'application/vnd.openxmlformats-officedocument.presentationml.presentation', + sizeBytes: 1024, + }, + userOneCtx + ) + ).rejects.toThrow('KB file type is not supported') + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.md', + contentType: 'text/plain', + sizeBytes: 1024, + }, + userOneCtx + ) + ).resolves.toMatchObject({ blobName: expect.stringMatching(/\.md$/) }) + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userTwoCtx + ) + ).rejects.toThrow('KB not found') + }) + + it('issues a private blob-scoped upload ticket without creating a resource', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + process.env.BLOB_STORAGE_ACCOUNT_URL = + 'https://blob.klicker.localhost/kbtestaccount/' + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL = + 'http://kb-poc-azurite:10000/kbtestaccount/' + + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + + expect(containerName).toBe(`kb-${userOneCtx.user.sub}`) + expect(blobServiceUrl).toBe('http://kb-poc-azurite:10000/kbtestaccount') + expect(createIfNotExists).toHaveBeenCalledWith() + expect(ticket.containerName).toBe(containerName) + expect(ticket.blobName).toMatch( + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\.pdf$/ + ) + const uploadUrl = new URL(ticket.uploadSasURL) + expect(uploadUrl.origin).toBe('https://blob.klicker.localhost') + expect(uploadUrl.pathname).toBe('/kbtestaccount') + expect(uploadUrl.searchParams.get('sp')).toBe('cw') + expect(uploadUrl.searchParams.get('sr')).toBe('b') + const expiry = Date.parse(uploadUrl.searchParams.get('se') ?? '') + expect(expiry).toBeGreaterThan(Date.now() + 14 * 60 * 1000) + expect(expiry).toBeLessThanOrEqual(Date.now() + 15 * 60 * 1000 + 1000) + await expect( + prisma.kBResource.count({ where: { kbId: created.id } }) + ).resolves.toBe(0) + const persistedTicket = await prisma.kBUploadTicket.findUniqueOrThrow({ + where: { id: ticket.blobName.slice(0, -4) }, + }) + expect(persistedTicket).toMatchObject({ + kbId: created.id, + blobName: ticket.blobName, + sizeBytes: 1024, + }) + expect(Math.abs(persistedTicket.expiresAt.getTime() - expiry)).toBeLessThan( + 1000 + ) + }) + + it('does not issue an upload ticket after whole-KB deletion wins the lock', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const deletionStarted = createDeferred() + const finishDeletion = createDeferred() + const storageReady = createDeferred() + const deleteCtx = withTombstonePause( + userOneCtx, + 'kB', + () => deletionStarted.resolve(undefined), + finishDeletion.promise + ) + createIfNotExists.mockImplementationOnce(async () => { + storageReady.resolve(undefined) + return { succeeded: true } + }) + + const deletion = deleteKb({ id: created.id }, deleteCtx) + await deletionStarted.promise + const uploadRequest = requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + await storageReady.promise + finishDeletion.resolve(undefined) + + await expect(deletion).resolves.toMatchObject({ id: created.id }) + await expect(uploadRequest).rejects.toThrow('KB not found') + await expect( + prisma.kBUploadTicket.count({ where: { kbId: created.id } }) + ).resolves.toBe(0) + }) + + it('reserves the final resource slot and rejects concurrent claims beyond it', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + await prisma.kBResource.createMany({ + data: Array.from({ length: MAX_KB_RESOURCE_COUNT - 1 }, (_, index) => ({ + kbId: created.id, + type: KBResourceType.URL, + title: `Resource ${index}`, + sourceUrl: `https://example.com/resource-${index}`, + sizeBytes: 1, + })), + }) + + const requests = await Promise.allSettled([ + requestKbFileUpload( + { + kbId: created.id, + fileName: 'first.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ), + requestKbFileUpload( + { + kbId: created.id, + fileName: 'second.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ), + ]) + + expect( + requests.filter(({ status }) => status === 'fulfilled') + ).toHaveLength(1) + const rejected = requests.find(({ status }) => status === 'rejected') + expect(rejected).toMatchObject({ + status: 'rejected', + reason: expect.objectContaining({ + extensions: { code: 'KB_RESOURCE_LIMIT_REACHED' }, + }), + }) + await expect( + prisma.kBUploadTicket.count({ where: { kbId: created.id } }) + ).resolves.toBe(1) + }) + + it('reserves the final byte-quota placeholder and rejects concurrent URL claims beyond it', async () => { + const created = await createKb({ name: 'Legacy URLs' }, userOneCtx) + await prisma.kBResource.createMany({ + data: legacyUrlResources(created.id, 19), + }) + + const requests = await Promise.allSettled([ + createKbUrlResource( + { + kbId: created.id, + title: 'First', + url: 'https://example.com/concurrent-first', + }, + userOneCtx + ), + createKbUrlResource( + { + kbId: created.id, + title: 'Second', + url: 'https://example.com/concurrent-second', + }, + userOneCtx + ), + ]) + + expect( + requests.filter(({ status }) => status === 'fulfilled') + ).toHaveLength(1) + const rejected = requests.find(({ status }) => status === 'rejected') + expect(rejected).toMatchObject({ + status: 'rejected', + reason: expect.objectContaining({ + extensions: { code: 'KB_STORAGE_LIMIT_REACHED' }, + }), + }) + await expect( + prisma.kBResource.count({ + where: { + kbId: created.id, + sourceUrl: { startsWith: 'https://example.com/concurrent' }, + }, + }) + ).resolves.toBe(1) + }) + + it('retains tombstones in quota usage until hard cleanup removes them', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const rows = Array.from({ length: MAX_KB_RESOURCE_COUNT }, (_, index) => ({ + kbId: created.id, + type: KBResourceType.URL, + title: `Resource ${index}`, + sourceUrl: `https://example.com/resource-${index}`, + sizeBytes: 1, + ...(index === 0 + ? { deletedAt: new Date(), deletedById: userOneCtx.user.sub } + : {}), + })) + await prisma.kBResource.createMany({ data: rows }) + + const request = () => + requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + await expect(request()).rejects.toMatchObject({ + extensions: { code: 'KB_RESOURCE_LIMIT_REACHED' }, + }) + + await prisma.kBResource.deleteMany({ + where: { kbId: created.id, deletedAt: { not: null } }, + }) + await expect(request()).resolves.toMatchObject({ + blobName: expect.stringMatching(/\.pdf$/), + }) + }) + + it('conservatively reserves 25 MiB for retained resources with unknown size', async () => { + const created = await createKb({ name: 'Legacy URLs' }, userOneCtx) + await prisma.kBResource.createMany({ + data: legacyUrlResources(created.id, 20), + }) + + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1, + }, + userOneCtx + ) + ).rejects.toMatchObject({ + extensions: { code: 'KB_STORAGE_LIMIT_REACHED' }, + }) + }) + + it('charges a URL resource its unknown-size placeholder against the byte quota', async () => { + const atCap = await createKb({ name: 'Legacy URLs at cap' }, userOneCtx) + await prisma.kBResource.createMany({ + data: legacyUrlResources(atCap.id, 20), + }) + + await expect( + createKbUrlResource( + { + kbId: atCap.id, + title: 'One too many', + url: 'https://example.com/one-too-many', + }, + userOneCtx + ) + ).rejects.toMatchObject({ + extensions: { code: 'KB_STORAGE_LIMIT_REACHED' }, + }) + + const underCap = await createKb( + { name: 'Legacy URLs under cap' }, + userOneCtx + ) + await prisma.kBResource.createMany({ + data: legacyUrlResources(underCap.id, 19), + }) + + await expect( + createKbUrlResource( + { + kbId: underCap.id, + title: 'Fits under cap', + url: 'https://example.com/fits-under-cap', + }, + userOneCtx + ) + ).resolves.toMatchObject({ + kbId: underCap.id, + type: KBResourceType.URL, + }) + }) + + it('counts upload reservations toward the byte quota without double counting confirmation', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.URL, + title: 'Large resource', + sourceUrl: 'https://example.com/large', + sizeBytes: MAX_KB_TOTAL_SIZE_BYTES - 1024, + }, + }) + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + + await expect( + requestKbFileUpload( + { + kbId: created.id, + fileName: 'extra.pdf', + contentType: 'application/pdf', + sizeBytes: 1, + }, + userOneCtx + ) + ).rejects.toMatchObject({ + extensions: { code: 'KB_STORAGE_LIMIT_REACHED' }, + }) + + await expect( + confirmKbFileUpload( + { + kbId: created.id, + blobName: ticket.blobName, + title: 'Notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + ).resolves.toMatchObject({ sizeBytes: 1024 }) + await expect( + prisma.kBResource.aggregate({ + where: { kbId: created.id }, + _sum: { sizeBytes: true }, + }) + ).resolves.toMatchObject({ + _sum: { sizeBytes: MAX_KB_TOTAL_SIZE_BYTES }, + }) + }) + + it('rejects confirmation metadata that differs from the reserved upload size', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + getBlobProperties.mockResolvedValue({ + contentLength: 2048, + contentType: 'application/pdf', + }) + + await expect( + confirmKbFileUpload( + { + kbId: created.id, + blobName: ticket.blobName, + title: 'Notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 2048, + }, + userOneCtx + ) + ).rejects.toMatchObject({ + extensions: { code: 'KB_UPLOAD_TICKET_MISMATCH' }, + }) + await expect( + prisma.kBUploadTicket.findUnique({ + where: { id: ticket.blobName.slice(0, -4) }, + }) + ).resolves.toBeTruthy() + }) + + it('confirms a matching blob idempotently', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + process.env.BLOB_STORAGE_ACCOUNT_URL = + 'https://blob.klicker.localhost/kbtestaccount' + process.env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL = + 'http://kb-poc-azurite:10000/kbtestaccount' + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + const args = { + kbId: created.id, + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + } + + const first = await confirmKbFileUpload(args, userOneCtx) + const second = await confirmKbFileUpload(args, userOneCtx) + await expect( + confirmKbFileUpload({ ...args, title: 'Changed title' }, userOneCtx) + ).rejects.toMatchObject({ + extensions: { code: 'KB_UPLOAD_TICKET_MISMATCH' }, + }) + + expect(first.id).toBe(ticket.blobName.slice(0, -4)) + expect(first.blobHref).toBe( + `https://blob.klicker.localhost/kbtestaccount/${containerName}/${ticket.blobName}` + ) + expect(second.id).toBe(first.id) + expect(blobExists).toHaveBeenCalledOnce() + expect(getBlobProperties).toHaveBeenCalledOnce() + await expect( + prisma.kBResource.count({ where: { blobName: ticket.blobName } }) + ).resolves.toBe(1) + await expect( + prisma.kBUploadTicket.findUnique({ where: { id: first.id } }) + ).resolves.toBeNull() + }) + + it('safely converts a legacy zero-size upload ticket under the KB quota lock', async () => { + const created = await createKb({ name: 'Legacy upload' }, userOneCtx) + const blobId = randomUUID() + const blobName = `${blobId}.pdf` + await prisma.kBUploadTicket.create({ + data: { + id: blobId, + kbId: created.id, + blobName, + sizeBytes: 0, + expiresAt: new Date(Date.now() + 60_000), + }, + }) + + await expect( + confirmKbFileUpload( + { + kbId: created.id, + blobName, + title: 'Legacy notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + ).resolves.toMatchObject({ + id: blobId, + sizeBytes: 1024, + }) + await expect( + prisma.kBUploadTicket.findUnique({ where: { id: blobId } }) + ).resolves.toBeNull() + }) + + it('rejects an expired upload ticket while preserving its blob for cleanup', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + await prisma.kBUploadTicket.update({ + where: { id: ticket.blobName.slice(0, -4) }, + data: { expiresAt: new Date(Date.now() - 1) }, + }) + + await expect( + confirmKbFileUpload( + { + kbId: created.id, + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + ).rejects.toThrow('KB upload ticket is invalid') + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kBUploadTicket.findUnique({ + where: { id: ticket.blobName.slice(0, -4) }, + }) + ).resolves.toBeTruthy() + }) + + it('does not reveal a foreign resource through blob confirmation', async () => { + const ownedKb = await createKb({ name: 'Owned notes' }, userOneCtx) + const foreignKb = await createKb({ name: 'Foreign notes' }, userTwoCtx) + const foreignBlobId = 'a38eec07-5125-40b2-a245-019d58eab5d1' + await prisma.kBResource.create({ + data: { + id: foreignBlobId, + kbId: foreignKb.id, + type: KBResourceType.BLOB, + title: 'Foreign file', + originalFilename: 'foreign.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: `${foreignBlobId}.pdf`, + blobHref: + 'https://kbtestaccount.blob.core.windows.net/foreign/foreign.pdf', + }, + }) + blobExists.mockResolvedValue(false) + + const confirm = (blobName: string) => + confirmKbFileUpload( + { + kbId: ownedKb.id, + blobName, + title: 'Probe', + originalFilename: 'probe.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + + await expect(confirm(`${foreignBlobId}.pdf`)).rejects.toThrow( + 'KB blob was not found' + ) + await expect( + confirm('b151cb31-064b-49c0-b53b-fe732171660f.pdf') + ).rejects.toThrow('KB blob was not found') + }) + + it('does not delete the winning blob during concurrent cross-KB confirmation', async () => { + const firstKb = await createKb({ name: 'First notes' }, userOneCtx) + const secondKb = await createKb({ name: 'Second notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: firstKb.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + const confirm = (kbId: string) => + confirmKbFileUpload( + { + kbId, + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + + const results = await Promise.allSettled([ + confirm(firstKb.id), + confirm(secondKb.id), + ]) + + expect(results.filter(({ status }) => status === 'fulfilled')).toHaveLength( + 1 + ) + expect(results.filter(({ status }) => status === 'rejected')).toHaveLength( + 1 + ) + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kBResource.count({ where: { blobName: ticket.blobName } }) + ).resolves.toBe(1) + }) + + it('does not delete an existing blob on cross-KB metadata mismatch', async () => { + const firstKb = await createKb({ name: 'First notes' }, userOneCtx) + const secondKb = await createKb({ name: 'Second notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: firstKb.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + const args = { + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + } + + const resource = await confirmKbFileUpload( + { ...args, kbId: firstKb.id }, + userOneCtx + ) + getBlobProperties.mockClear() + deleteBlobIfExists.mockClear() + + await expect( + confirmKbFileUpload( + { ...args, kbId: secondKb.id, sizeBytes: 1025 }, + userOneCtx + ) + ).rejects.toThrow('KB blob name is invalid') + expect(getBlobProperties).not.toHaveBeenCalled() + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kBResource.findUniqueOrThrow({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + kbId: firstKb.id, + blobName: ticket.blobName, + sizeBytes: 1024, + }) + }) + + it('returns one resource for concurrent confirmation retries', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + const args = { + kbId: created.id, + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + } + + const [first, second] = await Promise.all([ + confirmKbFileUpload(args, userOneCtx), + confirmKbFileUpload(args, userOneCtx), + ]) + + expect(second.id).toBe(first.id) + await expect( + prisma.kBResource.count({ where: { blobName: ticket.blobName } }) + ).resolves.toBe(1) + }) + + it('rejects absent blobs and deletes mismatched uploads', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const ticket = await requestKbFileUpload( + { + kbId: created.id, + fileName: 'notes.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ) + const args = { + kbId: created.id, + blobName: ticket.blobName, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + } + blobExists.mockResolvedValueOnce(false) + await expect(confirmKbFileUpload(args, userOneCtx)).rejects.toThrow( + 'KB blob was not found' + ) + + getBlobProperties.mockResolvedValue({ + contentLength: 1025, + contentType: 'application/pdf', + }) + + await expect(confirmKbFileUpload(args, userOneCtx)).rejects.toThrow( + 'KB blob metadata is invalid' + ) + expect(deleteBlobIfExists).toHaveBeenCalledOnce() + await expect( + prisma.kBResource.count({ where: { kbId: created.id } }) + ).resolves.toBe(0) + }) + + it('validates URL resources and denies foreign knowledge bases', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + + await expect( + createKbUrlResource( + { kbId: created.id, title: 'Invalid', url: 'not-a-url' }, + userOneCtx + ) + ).rejects.toThrow('KB resource URL is invalid') + await expect( + createKbUrlResource( + { kbId: created.id, title: 'FTP', url: 'ftp://example.com/file' }, + userOneCtx + ) + ).rejects.toThrow('KB resource URL is invalid') + await expect( + createKbUrlResource( + { + kbId: created.id, + title: 'Private', + url: 'http://169.254.169.254/latest/meta-data', + }, + userOneCtx + ) + ).rejects.toThrow('KB resource URL is invalid') + await expect( + createKbUrlResource( + { + kbId: created.id, + title: 'Credentials', + url: 'https://user:password@example.com/file', + }, + userOneCtx + ) + ).rejects.toThrow('KB resource URL is invalid') + await expect( + createKbUrlResource( + { + kbId: created.id, + title: 'Foreign', + url: 'https://example.com', + }, + userTwoCtx + ) + ).rejects.toThrow('KB not found') + }) + + it('creates and deletes an owned URL resource', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/watch?id=123', + }, + userOneCtx + ) + + expect(resource).toMatchObject({ + kbId: created.id, + title: 'Lecture recording', + type: KBResourceType.URL, + sourceUrl: 'https://video.example.com/watch?id=123', + }) + await expect( + deleteKbResource({ id: resource.id }, userTwoCtx) + ).rejects.toThrow('KB resource not found') + + await deleteKbResource({ id: resource.id }, userOneCtx) + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + deletedById: userOneCtx.user.sub, + deletedAt: expect.any(Date), + ingestionOperation: KBIngestionOperation.DELETE, + status: KBResourceStatus.QUEUED, + resourceVersion: 1, + }) + await expect( + getKbResourcesConnection({ kbId: created.id }, userOneCtx) + ).resolves.toMatchObject({ items: [] }) + }) + + it('keeps a tombstone hidden when queueing its delete task fails', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/watch?id=123', + }, + userOneCtx + ) + vi.spyOn( + userOneCtx.tasks.deleteKBResource, + 'runNoWait' + ).mockRejectedValueOnce(new Error('queue unavailable')) + + await expect( + deleteKbResource({ id: resource.id }, userOneCtx) + ).resolves.toMatchObject({ id: resource.id }) + + const tombstone = await prisma.kBResource.findUniqueOrThrow({ + where: { id: resource.id }, + }) + expect(tombstone).toMatchObject({ + deletedAt: expect.any(Date), + status: KBResourceStatus.QUEUED, + errorCode: 'DELETION_QUEUE_FAILED', + }) + await expect( + prisma.kBIngestionRun.findUniqueOrThrow({ + where: { id: tombstone.ingestionAttemptId! }, + }) + ).resolves.toMatchObject({ + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.QUEUED, + errorCode: 'DELETION_QUEUE_FAILED', + }) + await expect( + getKbResourcesConnection({ kbId: created.id }, userOneCtx) + ).resolves.toMatchObject({ items: [] }) + }) + + it('returns only the five newest ingestion runs to the resource owner', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/watch?id=123', + }, + userOneCtx + ) + const runIds = Array.from({ length: 6 }, () => randomUUID()) + + for (const [index, id] of runIds.entries()) { + await prisma.kBIngestionRun.create({ + data: { + id, + resourceId: resource.id, + resourceVersion: index + 1, + status: KBIngestionStatus.SUCCEEDED, + createdAt: new Date(Date.UTC(2026, 6, 27, 10, index)), + }, + }) + } + + await expect( + getKbResourceIngestionRuns({ resourceId: resource.id }, userTwoCtx) + ).rejects.toThrow('KB resource not found') + await expect( + getKbResourceIngestionRuns({ resourceId: resource.id }, userOneCtx) + ).resolves.toMatchObject( + runIds + .slice(1) + .reverse() + .map((id, index) => ({ + id, + resourceVersion: 6 - index, + })) + ) + }) + + it('keeps the current lecturer attempt visible after a platform refresh', async () => { + const kb = await createKb({ name: 'Refresh projection test' }, userOneCtx) + const lecturerAttemptId = randomUUID() + const platformRefreshId = randomUUID() + const resource = await prisma.kBResource.create({ + data: { + kbId: kb.id, + type: KBResourceType.URL, + title: 'Refreshable resource', + sourceUrl: 'https://example.com/refreshable-resource', + status: KBResourceStatus.PROCESSING, + ingestionAttemptId: lecturerAttemptId, + resourceVersion: 2, + }, + }) + await prisma.kBIngestionRun.createMany({ + data: [ + { + id: lecturerAttemptId, + resourceId: resource.id, + resourceVersion: 2, + status: KBIngestionStatus.PROCESSING, + createdAt: new Date('2026-08-01T08:00:00.000Z'), + }, + { + id: platformRefreshId, + resourceId: resource.id, + resourceVersion: 1, + status: KBIngestionStatus.SUCCEEDED, + externalOperationId: 'platform-refresh-operation', + createdAt: new Date('2026-08-01T08:01:00.000Z'), + }, + ], + }) + + await expect( + getKbResourcesConnection({ kbId: kb.id }, userOneCtx) + ).resolves.toMatchObject({ + items: [ + { + id: resource.id, + ingestionRuns: [ + { id: lecturerAttemptId, status: KBIngestionStatus.PROCESSING }, + ], + }, + ], + }) + await expect( + getKbResourcesConnection( + { kbId: kb.id, status: KBIngestionStatus.PROCESSING }, + userOneCtx + ) + ).resolves.toMatchObject({ items: [{ id: resource.id }] }) + await expect( + getKbResourcesConnection( + { kbId: kb.id, status: KBIngestionStatus.SUCCEEDED }, + userOneCtx + ) + ).resolves.toMatchObject({ items: [] }) + }) + + it('defers blob storage deletion to asynchronous cleanup', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: '8d2140ef-04b4-41cb-a5a9-ff25381f9fdb.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + }, + }) + await expect( + deleteKbResource({ id: resource.id }, userOneCtx) + ).resolves.toMatchObject({ id: resource.id }) + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ deletedAt: expect.any(Date) }) + }) + + it.each([ + KBResourceStatus.QUEUED, + KBResourceStatus.PROCESSING, + ])('does not delete a %s blob resource', async (status) => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: 'bc1b27e4-b616-4403-8223-e6e8b3136c7e.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + status, + }, + }) + + await expect( + deleteKbResource({ id: resource.id }, userOneCtx) + ).rejects.toThrow('KB resource cannot be deleted') + + expect(deleteBlobIfExists).not.toHaveBeenCalled() + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toBeTruthy() + }) + + it('serializes resource deletion against a concurrent ingestion claim', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: 'ad135b54-bc2a-4888-b356-631e5a76627c.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + }, + }) + const deletionStarted = createDeferred() + const finishDeletion = createDeferred() + const deleteCtx = withTombstonePause( + userOneCtx, + 'kBResource', + () => deletionStarted.resolve(undefined), + finishDeletion.promise + ) + const claimStarted = createDeferred() + const ingestCtx = withIngestionClaimSignal(userOneCtx, () => + claimStarted.resolve(undefined) + ) + const runNoWait = vi + .spyOn(ingestCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const deletion = deleteKbResource({ id: resource.id }, deleteCtx) + await deletionStarted.promise + const ingestion = expect( + ingestKbResource({ id: resource.id }, ingestCtx) + ).rejects.toThrow('KB resource cannot be ingested') + await claimStarted.promise + + expect(runNoWait).not.toHaveBeenCalled() + finishDeletion.resolve(undefined) + await expect(deletion).resolves.toMatchObject({ id: resource.id }) + await ingestion + + expect(deleteBlobIfExists).not.toHaveBeenCalled() + expect(runNoWait).not.toHaveBeenCalled() + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ deletedAt: expect.any(Date) }) + }) + + it('serializes knowledge base deletion against a concurrent ingestion claim', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: '0f1320e3-0458-4874-a949-bc093be069fb.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + }, + }) + const deletionStarted = createDeferred() + const finishDeletion = createDeferred() + const deleteCtx = withTombstonePause( + userOneCtx, + 'kB', + () => deletionStarted.resolve(undefined), + finishDeletion.promise + ) + const claimStarted = createDeferred() + const ingestCtx = withIngestionClaimSignal(userOneCtx, () => + claimStarted.resolve(undefined) + ) + const runNoWait = vi + .spyOn(ingestCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const deletion = deleteKb({ id: created.id }, deleteCtx) + await deletionStarted.promise + const ingestion = expect( + ingestKbResource({ id: resource.id }, ingestCtx) + ).rejects.toThrow('KB resource cannot be ingested') + await claimStarted.promise + + expect(runNoWait).not.toHaveBeenCalled() + finishDeletion.resolve(undefined) + await expect(deletion).resolves.toMatchObject({ id: created.id }) + await ingestion + + expect(deleteBlobIfExists).not.toHaveBeenCalled() + expect(runNoWait).not.toHaveBeenCalled() + await expect( + prisma.kB.findUnique({ where: { id: created.id } }) + ).resolves.toMatchObject({ deletedAt: expect.any(Date) }) + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ deletedAt: expect.any(Date) }) + }) + + it('paginates owned knowledge bases with tied timestamps and filter-bound cursors', async () => { + const timestamp = new Date('2026-07-28T12:00:00.000Z') + const ids = Array.from({ length: 4 }, () => randomUUID()) + .sort() + .reverse() + await prisma.kB.createMany({ + data: ids.map((id, index) => ({ + id, + ownerId: userOneCtx.user.sub, + name: index === 0 ? 'Finance handbook' : `Course notes ${index}`, + createdAt: timestamp, + updatedAt: timestamp, + })), + }) + await createKb({ name: 'Other owner' }, userTwoCtx) + + const firstPage = await getUserKbsConnection({ first: 2 }, userOneCtx) + expect(firstPage.items.map(({ id }) => id)).toEqual(ids.slice(0, 2)) + expect(firstPage).toMatchObject({ + totalCount: 4, + pageInfo: { hasNextPage: true }, + }) + expect(firstPage.pageInfo.endCursor).toBeTruthy() + + const secondPage = await getUserKbsConnection( + { first: 2, after: firstPage.pageInfo.endCursor }, + userOneCtx + ) + expect(secondPage.items.map(({ id }) => id)).toEqual(ids.slice(2)) + expect(secondPage).toMatchObject({ + totalCount: 4, + pageInfo: { hasNextPage: false }, + }) + await expect( + getUserKbsConnection( + { + first: 2, + after: firstPage.pageInfo.endCursor, + search: 'finance', + }, + userOneCtx + ) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + await expect( + getUserKbsConnection( + { first: 2, after: firstPage.pageInfo.endCursor }, + userTwoCtx + ) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + await expect( + getUserKbsConnection({ after: 'not+a+cursor' }, userOneCtx) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + + const searchResult = await getUserKbsConnection( + { search: ' FINANCE ' }, + userOneCtx + ) + expect(searchResult.items.map(({ name }) => name)).toEqual([ + 'Finance handbook', + ]) + expect(searchResult.totalCount).toBe(1) + }) + + it('keeps resource pagination stable across status updates and hides tombstones', async () => { + const kb = await createKb({ name: 'Scale test' }, userOneCtx) + const timestamp = new Date('2026-07-28T13:00:00.000Z') + const ids = Array.from({ length: 5 }, () => randomUUID()) + .sort() + .reverse() + await prisma.kBResource.createMany({ + data: ids.map((id, index) => ({ + id, + kbId: kb.id, + type: index % 2 === 0 ? KBResourceType.URL : KBResourceType.BLOB, + title: index === 0 ? 'Finance syllabus' : `Resource ${index}`, + sourceUrl: + index % 2 === 0 ? `https://example.com/resource-${index}` : null, + originalFilename: index % 2 === 1 ? `resource-${index}.pdf` : null, + mimeType: index % 2 === 1 ? 'application/pdf' : null, + sizeBytes: index % 2 === 1 ? 100 + index : null, + blobName: index % 2 === 1 ? `${id}.pdf` : null, + blobHref: + index % 2 === 1 + ? `https://kbtestaccount.blob.core.windows.net/container/${id}.pdf` + : null, + createdAt: timestamp, + updatedAt: timestamp, + })), + }) + await prisma.kBResource.update({ + where: { id: ids[4] }, + data: { + deletedAt: new Date(), + deletedById: userOneCtx.user.sub, + }, + }) + + const firstPage = await getKbResourcesConnection( + { kbId: kb.id, first: 2 }, + userOneCtx + ) + expect(firstPage.items.map(({ id }) => id)).toEqual(ids.slice(0, 2)) + expect(firstPage.totalCount).toBe(4) + + const currentAttemptId = randomUUID() + await prisma.kBResource.update({ + where: { id: ids[0] }, + data: { + status: KBResourceStatus.PROCESSING, + ingestionAttemptId: currentAttemptId, + }, + }) + await prisma.kBIngestionRun.createMany({ + data: [ + { + id: randomUUID(), + resourceId: ids[0]!, + resourceVersion: 1, + status: KBIngestionStatus.FAILED, + createdAt: new Date('2026-07-28T13:01:00.000Z'), + }, + { + id: currentAttemptId, + resourceId: ids[0]!, + resourceVersion: 2, + status: KBIngestionStatus.PROCESSING, + createdAt: new Date('2026-07-28T13:02:00.000Z'), + }, + { + id: randomUUID(), + resourceId: ids[1]!, + resourceVersion: 1, + status: KBIngestionStatus.SUCCEEDED, + createdAt: new Date('2026-07-28T13:02:00.000Z'), + }, + ], + }) + const secondPage = await getKbResourcesConnection( + { + kbId: kb.id, + first: 2, + after: firstPage.pageInfo.endCursor, + }, + userOneCtx + ) + expect(secondPage.items.map(({ id }) => id)).toEqual(ids.slice(2, 4)) + expect( + new Set([ + ...firstPage.items.map(({ id }) => id), + ...secondPage.items.map(({ id }) => id), + ]).size + ).toBe(4) + expect(secondPage.pageInfo.hasNextPage).toBe(false) + + await expect( + getKbResourcesConnection( + { + kbId: kb.id, + after: firstPage.pageInfo.endCursor, + type: KBResourceType.BLOB, + }, + userOneCtx + ) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + await expect( + getKbResourcesConnection( + { kbId: kb.id, after: firstPage.pageInfo.endCursor }, + userTwoCtx + ) + ).rejects.toThrow('KB not found') + + const filtered = await getKbResourcesConnection( + { + kbId: kb.id, + search: 'finance', + type: KBResourceType.URL, + status: KBIngestionStatus.PROCESSING, + }, + userOneCtx + ) + expect(filtered.items.map(({ id }) => id)).toEqual([ids[0]]) + expect(filtered.totalCount).toBe(1) + }) + + it('returns exact visible, retained, reserved, cleanup, and limit metrics', async () => { + const kb = await createKb({ name: 'Metrics test' }, userOneCtx) + await prisma.kBResource.createMany({ + data: [ + { + kbId: kb.id, + type: KBResourceType.BLOB, + title: 'Visible file', + sizeBytes: 100, + }, + { + kbId: kb.id, + type: KBResourceType.URL, + title: 'Visible URL', + }, + { + kbId: kb.id, + type: KBResourceType.BLOB, + title: 'Pending cleanup', + sizeBytes: 50, + deletedAt: new Date(), + deletedById: userOneCtx.user.sub, + }, + ], + }) + await prisma.kBUploadTicket.create({ + data: { + id: randomUUID(), + kbId: kb.id, + blobName: `${randomUUID()}.pdf`, + sizeBytes: 25, + expiresAt: new Date(Date.now() + 60_000), + }, + }) + + const result = await getKb({ id: kb.id }, userOneCtx) + expect(result.metrics).toEqual({ + visibleResourceCount: 2, + visibleSizeBytes: 100, + unknownSizeResourceCount: 1, + quotaResourceCount: 4, + quotaSizeBytes: 25 * 1024 * 1024 + 175, + resourceLimit: MAX_KB_RESOURCE_COUNT, + storageLimitBytes: MAX_KB_TOTAL_SIZE_BYTES, + pendingCleanupCount: 1, + pendingCleanupSizeBytes: 50, + reservedResourceCount: 1, + reservedSizeBytes: 25, + linkedConsumerCount: 0, + }) + }) + + it('attributes the six-grouped-query metrics to each KB independently when multiple KBs share one connection page', async () => { + // Three KBs for the same owner, each with a deliberately distinct + // composition, to prove getKbMetricsMap's per-kbId groupBy attribution + // doesn't bleed a sibling KB's rows into another's metrics. + const kbA = await createKb({ name: 'Alpha KB' }, userOneCtx) + const kbB = await createKb({ name: 'Beta KB' }, userOneCtx) + const kbC = await createKb({ name: 'Gamma KB' }, userOneCtx) + + await prisma.kBResource.createMany({ + data: [ + // Alpha: 2 visible resources (one unknown-size), 1 tombstone, 1 reservation, 1 linked consumer + { + kbId: kbA.id, + type: KBResourceType.BLOB, + title: 'Alpha visible blob', + sizeBytes: 100, + }, + { + kbId: kbA.id, + type: KBResourceType.URL, + title: 'Alpha visible url (unknown size)', + }, + { + kbId: kbA.id, + type: KBResourceType.BLOB, + title: 'Alpha tombstone', + sizeBytes: 30, + deletedAt: new Date(), + deletedById: userOneCtx.user.sub, + }, + // Beta: 1 visible resource, 2 unknown-size tombstones, no reservation, no consumer + { + kbId: kbB.id, + type: KBResourceType.BLOB, + title: 'Beta visible blob', + sizeBytes: 200, + }, + { + kbId: kbB.id, + type: KBResourceType.BLOB, + title: 'Beta tombstone (unknown size) 1', + deletedAt: new Date(), + deletedById: userOneCtx.user.sub, + }, + { + kbId: kbB.id, + type: KBResourceType.BLOB, + title: 'Beta tombstone (unknown size) 2', + deletedAt: new Date(), + deletedById: userOneCtx.user.sub, + }, + // Gamma: no resources at all, only an upload reservation + ], + }) + await prisma.kBUploadTicket.createMany({ + data: [ + { + id: randomUUID(), + kbId: kbA.id, + blobName: `${randomUUID()}.pdf`, + sizeBytes: 10, + expiresAt: new Date(Date.now() + 60_000), + }, + { + id: randomUUID(), + kbId: kbC.id, + blobName: `${randomUUID()}.pdf`, + sizeBytes: 5, + expiresAt: new Date(Date.now() + 60_000), + }, + ], + }) + const course = await seedCourse({}, userOneCtx) + const chatbot = await prisma.chatbot.create({ + data: { + name: 'Alpha tutor', + ownerId: userOneCtx.user.sub, + courseId: course.id, + }, + }) + await prisma.kBChatbot.create({ + data: { kbId: kbA.id, chatbotId: chatbot.id, isEnabled: true }, + }) + + const page = await getUserKbsConnection({ first: 20 }, userOneCtx) + const byId = new Map(page.items.map((item) => [item.id, item])) + expect(byId.has(kbA.id)).toBe(true) + expect(byId.has(kbB.id)).toBe(true) + expect(byId.has(kbC.id)).toBe(true) + + expect(byId.get(kbA.id)?.metrics).toEqual({ + visibleResourceCount: 2, + visibleSizeBytes: 100, + unknownSizeResourceCount: 1, + quotaResourceCount: 4, + quotaSizeBytes: 25 * 1024 * 1024 + 140, + resourceLimit: MAX_KB_RESOURCE_COUNT, + storageLimitBytes: MAX_KB_TOTAL_SIZE_BYTES, + pendingCleanupCount: 1, + pendingCleanupSizeBytes: 30, + reservedResourceCount: 1, + reservedSizeBytes: 10, + linkedConsumerCount: 1, + }) + expect(byId.get(kbB.id)?.metrics).toEqual({ + visibleResourceCount: 1, + visibleSizeBytes: 200, + unknownSizeResourceCount: 0, + quotaResourceCount: 3, + quotaSizeBytes: 25 * 1024 * 1024 * 2 + 200, + resourceLimit: MAX_KB_RESOURCE_COUNT, + storageLimitBytes: MAX_KB_TOTAL_SIZE_BYTES, + pendingCleanupCount: 2, + pendingCleanupSizeBytes: 25 * 1024 * 1024 * 2, + reservedResourceCount: 0, + reservedSizeBytes: 0, + linkedConsumerCount: 0, + }) + expect(byId.get(kbC.id)?.metrics).toEqual({ + visibleResourceCount: 0, + visibleSizeBytes: 0, + unknownSizeResourceCount: 0, + quotaResourceCount: 1, + quotaSizeBytes: 5, + resourceLimit: MAX_KB_RESOURCE_COUNT, + storageLimitBytes: MAX_KB_TOTAL_SIZE_BYTES, + pendingCleanupCount: 0, + pendingCleanupSizeBytes: 0, + reservedResourceCount: 1, + reservedSizeBytes: 5, + linkedConsumerCount: 0, + }) + + // cross-check the single-KB detail path against the multi-KB catalog + // path: both call the same getKbMetricsMap, so a KB's metrics must be + // identical however many sibling KBs are aggregated alongside it. + expect((await getKb({ id: kbB.id }, userOneCtx)).metrics).toEqual( + byId.get(kbB.id)?.metrics + ) + }) + + it('bulk deletes a bounded selection with one independently queued attempt per resource', async () => { + const kb = await createKb({ name: 'Bulk delete' }, userOneCtx) + const resources = await Promise.all( + ['First', 'Second'].map((title) => + createKbUrlResource( + { + kbId: kb.id, + title, + url: `https://example.com/${title.toLowerCase()}`, + }, + userOneCtx + ) + ) + ) + const runNoWait = vi + .spyOn(userOneCtx.tasks.deleteKBResource, 'runNoWait') + .mockRejectedValueOnce(new Error('queue unavailable')) + .mockResolvedValueOnce({} as never) + + const deleted = await deleteKbResources( + { kbId: kb.id, ids: resources.map(({ id }) => id).reverse() }, + userOneCtx + ) + + expect(deleted).toHaveLength(2) + expect(runNoWait).toHaveBeenCalledTimes(2) + const persisted = await prisma.kBResource.findMany({ + where: { id: { in: resources.map(({ id }) => id) } }, + orderBy: { id: 'asc' }, + }) + expect(persisted.every(({ deletedAt }) => deletedAt !== null)).toBe(true) + expect( + persisted.filter(({ errorCode }) => errorCode === 'DELETION_QUEUE_FAILED') + ).toHaveLength(1) + expect( + await prisma.kBIngestionRun.count({ + where: { + resourceId: { in: resources.map(({ id }) => id) }, + operation: KBIngestionOperation.DELETE, + }, + }) + ).toBe(2) + }) + + it('rejects an invalid or unsafe bulk deletion atomically', async () => { + const kb = await createKb({ name: 'Bulk guards' }, userOneCtx) + const foreignKb = await createKb({ name: 'Foreign' }, userTwoCtx) + const safe = await createKbUrlResource( + { + kbId: kb.id, + title: 'Safe', + url: 'https://example.com/safe', + }, + userOneCtx + ) + const active = await createKbUrlResource( + { + kbId: kb.id, + title: 'Active', + url: 'https://example.com/active', + }, + userOneCtx + ) + await prisma.kBResource.update({ + where: { id: active.id }, + data: { status: KBResourceStatus.PROCESSING }, + }) + const foreign = await createKbUrlResource( + { + kbId: foreignKb.id, + title: 'Foreign', + url: 'https://example.com/foreign', + }, + userTwoCtx + ) + + await expect( + deleteKbResources({ kbId: kb.id, ids: [safe.id, active.id] }, userOneCtx) + ).rejects.toMatchObject({ extensions: { code: 'KB_RESOURCE_ACTIVE' } }) + await expect( + deleteKbResources({ kbId: kb.id, ids: [safe.id, foreign.id] }, userOneCtx) + ).rejects.toThrow('KB resource not found') + await expect( + deleteKbResources({ kbId: kb.id, ids: [safe.id, safe.id] }, userOneCtx) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + await expect( + deleteKbResources( + { + kbId: kb.id, + ids: Array.from({ length: 51 }, () => randomUUID()), + }, + userOneCtx + ) + ).rejects.toMatchObject({ extensions: { code: 'BAD_USER_INPUT' } }) + await expect( + prisma.kBResource.findUnique({ where: { id: safe.id } }) + ).resolves.toMatchObject({ deletedAt: null }) + }) + + it('rejects every KB entry point for a non-preview user', async () => { + const nonPreviewUser = await prisma.user.findUnique({ + where: { id: nonPreviewCtx.user.sub }, + }) + expect(nonPreviewUser?.privatePreview).toBe(false) + + const kbId = randomUUID() + const resourceId = randomUUID() + const chatbotId = randomUUID() + + const entryPoints: Array<() => Promise> = [ + () => createKb({ name: 'Blocked KB' }, nonPreviewCtx), + () => deleteKb({ id: kbId }, nonPreviewCtx), + () => + createKbUrlResource( + { kbId, url: 'https://example.com/blocked', title: 'Blocked' }, + nonPreviewCtx + ), + () => deleteKbResource({ id: resourceId }, nonPreviewCtx), + () => deleteKbResources({ kbId, ids: [resourceId] }, nonPreviewCtx), + () => ingestKbResource({ id: resourceId }, nonPreviewCtx), + () => + requestKbFileUpload( + { + kbId, + fileName: 'blocked.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + nonPreviewCtx + ), + () => + confirmKbFileUpload( + { + kbId, + blobName: `${randomUUID()}.pdf`, + title: 'Blocked', + originalFilename: 'blocked.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + }, + nonPreviewCtx + ), + () => attachKbToChatbot({ kbId, chatbotId }, nonPreviewCtx), + () => detachKbFromChatbot({ kbId, chatbotId }, nonPreviewCtx), + () => getUserKbsConnection({}, nonPreviewCtx), + () => getKb({ id: kbId }, nonPreviewCtx), + () => getKbResourcesConnection({ kbId }, nonPreviewCtx), + () => getKbChatbotBindings({ kbId }, nonPreviewCtx), + () => getKbResourceIngestionRuns({ resourceId }, nonPreviewCtx), + ] + + expect(entryPoints).toHaveLength(15) + for (const callEntryPoint of entryPoints) { + await expect(callEntryPoint()).rejects.toMatchObject({ + extensions: { code: 'KB_PREVIEW_ACCESS_REQUIRED' }, + }) + } + }) + + it('refuses new KB content dispatch while the ingestion kill switch is enabled', async () => { + const kb = await createKb({ name: 'Kill switch KB' }, userOneCtx) + const existingResource = await createKbUrlResource( + { kbId: kb.id, title: 'Existing', url: 'https://example.com/existing' }, + userOneCtx + ) + + vi.stubEnv('KB_INGESTION_DISABLED', 'true') + try { + const blockedCalls: Array<() => Promise> = [ + () => + createKbUrlResource( + { + kbId: kb.id, + title: 'Blocked', + url: 'https://example.com/blocked', + }, + userOneCtx + ), + () => + requestKbFileUpload( + { + kbId: kb.id, + fileName: 'blocked.pdf', + contentType: 'application/pdf', + sizeBytes: 1024, + }, + userOneCtx + ), + () => ingestKbResource({ id: existingResource.id }, userOneCtx), + ] + for (const callBlockedEntryPoint of blockedCalls) { + await expect(callBlockedEntryPoint()).rejects.toMatchObject({ + extensions: { code: 'KB_INGESTION_DISABLED' }, + }) + } + + // reads and deletion of already-registered content stay live + await expect(getKb({ id: kb.id }, userOneCtx)).resolves.toMatchObject({ + id: kb.id, + }) + const deleted = await deleteKbResource( + { id: existingResource.id }, + userOneCtx + ) + expect(deleted.id).toBe(existingResource.id) + } finally { + vi.unstubAllEnvs() + } + }) +}) diff --git a/packages/graphql/test/knowledgeGraphAccounting.test.ts b/packages/graphql/test/knowledgeGraphAccounting.test.ts new file mode 100644 index 0000000000..783c456c7d --- /dev/null +++ b/packages/graphql/test/knowledgeGraphAccounting.test.ts @@ -0,0 +1,775 @@ +import { hashKBContentDigestEntries } from '@klicker-uzh/knowledge-graph' +import { prisma as prismaClient } from '@klicker-uzh/prisma' +import { + KBGraphBuildStatus, + KBGraphCostStatus, + KBGraphQualityTier, + KBResourceStatus, + KBResourceType, +} from '@klicker-uzh/prisma/client' +import { randomUUID } from 'node:crypto' +import { afterAll, afterEach, beforeEach, describe, expect, it } from 'vitest' +import { + releaseKBGraphCostReservation, + reserveKBGraphCost, + settleKBGraphBuildCost, +} from '../src/services/knowledgeGraphAccounting.js' + +const NOW = new Date('2026-08-15T19:30:00.000Z') +const SOURCE_CONTENT_DIGEST = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' +const LATE_RESOURCE_ID = '17af8b84-58bf-4a92-8f8b-197556ed98f4' +const LATE_CONTENT_SHA256 = + '2c26b46b68ffc68ff99b453c1d30413413422f164490f3d7c1d7d7d6d4b1f6b3' +const LATE_SOURCE_CONTENT_DIGEST = hashKBContentDigestEntries([ + { resourceId: LATE_RESOURCE_ID, contentSha256: LATE_CONTENT_SHA256 }, +]) + +const costEnv = { + KB_GRAPH_COST_CURRENCY: 'CHF', + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '100', + KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS: '200', + KB_GRAPH_MAX_COST_MINOR_UNITS: '200', + KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '200', + KB_GRAPH_COST_PRICING_VERSION: 'test-v1', + KB_GRAPH_SEMESTER_KEY: '2026-H2', +} + +const prisma = prismaClient +let ownerId: string +let kbId: string + +async function createBuild({ + costStatus = KBGraphCostStatus.RESERVED, + active = true, + status = KBGraphBuildStatus.PROCESSING, + errorCode = null, + createdAt, + sourceContentDigest = SOURCE_CONTENT_DIGEST, + cleanupStartedAt = null, + cleanedAt = null, +}: { + costStatus?: KBGraphCostStatus + active?: boolean + status?: KBGraphBuildStatus + errorCode?: string | null + createdAt?: Date + sourceContentDigest?: string + cleanupStartedAt?: Date | null + cleanedAt?: Date | null +} = {}) { + const buildId = randomUUID() + const runId = `run-${buildId}` + const graphmlBlobName = `knowledge-graphs/${buildId}.graphml` + const quota = await prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + select: { id: true }, + }) + const build = await prisma.kBGraphBuild.create({ + data: { + id: buildId, + kbId, + requestedById: ownerId, + status, + qualityTier: KBGraphQualityTier.STANDARD, + sourceContentDigest, + graphName: `klickeruzh:kb:${kbId}:${buildId}`, + graphmlBlobName, + estimatedCostMinorUnits: 100, + costCurrency: 'CHF', + costPricingVersion: 'test-v1', + costStatus, + semesterKey: '2026-H2', + quotaId: quota.id, + externalOperationId: runId, + errorCode, + ...(createdAt ? { createdAt } : {}), + cleanupStartedAt, + cleanedAt, + }, + }) + if (active) { + await prisma.kB.update({ + where: { id: kbId }, + data: { activeGraphBuildId: buildId }, + }) + } + return { build, runId, graphmlBlobName } +} + +function successfulResult({ + buildId, + runId, + amountMinorUnits = 60, + graphmlBlobName, + sourceContentDigest = SOURCE_CONTENT_DIGEST, +}: { + buildId: string + runId: string + amountMinorUnits?: number + graphmlBlobName: string + sourceContentDigest?: string +}) { + return { + contract_version: 'klicker-kb-graph/v1', + result_id: `${buildId}:${runId}`, + build_id: buildId, + kb_id: kbId, + owner_id: ownerId, + run_id: runId, + source_content_digest: sourceContentDigest, + graph_name: `klickeruzh:kb:${kbId}:${buildId}`, + status: 'SUCCEEDED', + edge_count: 1, + failed_document_count: 0, + graphml_artifact: { + container_name: `kb-${ownerId}`, + blob_name: graphmlBlobName, + }, + metered_cost: { + currency: 'CHF', + amount_minor_units: amountMinorUnits, + components: [ + { + provider: 'test-provider', + model: 'test-model', + amount_minor_units: amountMinorUnits, + pricing_version: 'test-v1', + embedding_tokens: 7, + input_tokens: 11, + output_tokens: 13, + request_count: 2, + }, + ], + metering_source: 'configured_pricing', + }, + node_count: 2, + processed_document_count: 1, + error_code: null, + } +} + +describe('KB graph cost accounting', () => { + beforeEach(async () => { + ownerId = randomUUID() + kbId = randomUUID() + await prisma.user.create({ + data: { + id: ownerId, + email: `${ownerId}@example.org`, + shortname: `kb-graph-${ownerId.slice(0, 8)}`, + }, + }) + await prisma.kB.create({ + data: { + id: kbId, + ownerId, + name: 'Accounting test KB', + }, + }) + }) + + afterEach(async () => { + await prisma.user.delete({ where: { id: ownerId } }) + }) + + afterAll(async () => { + await prisma.$disconnect() + }) + + it('serializes concurrent reservations against the same semester quota', async () => { + const results = await Promise.allSettled([ + prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: { ...costEnv, KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '150' }, + now: NOW, + }) + ), + prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: { ...costEnv, KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '150' }, + now: NOW, + }) + ), + ]) + + expect( + results.filter((result) => result.status === 'fulfilled') + ).toHaveLength(1) + expect( + results.filter((result) => result.status === 'rejected') + ).toHaveLength(1) + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ + limitMinorUnits: 150, + reservedMinorUnits: 100, + settledMinorUnits: 0, + }) + }) + + it('settles a valid result once and publishes only the validated build', async () => { + const reservation = await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build, runId, graphmlBlobName } = await createBuild() + expect(reservation.estimatedCostMinorUnits).toBe(100) + + const result = successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + }) + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result, + finishedAt: NOW, + }) + ) + ).resolves.toBe('SETTLED') + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result, + finishedAt: NOW, + }) + ) + ).resolves.toBe('DUPLICATE') + + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ + reservedMinorUnits: 0, + settledMinorUnits: 60, + }) + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.SUCCEEDED, + costStatus: KBGraphCostStatus.SETTLED, + actualCostMinorUnits: 60, + actualInputTokens: 11, + actualOutputTokens: 13, + actualEmbeddingTokens: 7, + actualRequestCount: 2, + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ + activeGraphBuildId: null, + publishedGraphBuildId: build.id, + }) + }) + + it('accepts and publishes a late success when the pinned digest still matches', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + await prisma.kBResource.create({ + data: { + id: LATE_RESOURCE_ID, + kbId, + type: KBResourceType.URL, + title: 'Late success resource', + sourceUrl: 'https://content.example.org/late.pdf', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: LATE_CONTENT_SHA256, + }, + }) + const { build, runId, graphmlBlobName } = await createBuild({ + active: false, + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + createdAt: new Date(NOW.getTime() - 5 * 60 * 1000), + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }) + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }), + finishedAt: NOW, + allowLateSuccess: true, + }) + ) + ).resolves.toBe('SETTLED') + + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.SUCCEEDED, + costStatus: KBGraphCostStatus.SETTLED, + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ + activeGraphBuildId: null, + publishedGraphBuildId: build.id, + }) + }) + + it('validates late-result metering before claiming the active graph slot', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + await prisma.kBResource.create({ + data: { + id: LATE_RESOURCE_ID, + kbId, + type: KBResourceType.URL, + title: 'Late currency mismatch resource', + sourceUrl: 'https://content.example.org/currency.pdf', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: LATE_CONTENT_SHA256, + }, + }) + const { build, runId, graphmlBlobName } = await createBuild({ + active: false, + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + createdAt: new Date(NOW.getTime() - 5 * 60 * 1000), + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }) + const result = successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }) + result.metered_cost!.currency = 'EUR' + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result, + finishedAt: NOW, + allowLateSuccess: true, + }) + ) + ).resolves.toBe('NEEDS_HUMAN_REVIEW') + + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + errorCode: 'KB_GRAPH_RESULT_CURRENCY_MISMATCH', + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ activeGraphBuildId: null }) + }) + + it('settles a late success without publishing when the KB digest is stale', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + await prisma.kBResource.create({ + data: { + id: LATE_RESOURCE_ID, + kbId, + type: KBResourceType.URL, + title: 'Stale late success resource', + sourceUrl: 'https://content.example.org/stale.pdf', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: `${LATE_CONTENT_SHA256.slice(0, -1)}0`, + }, + }) + const { build, runId, graphmlBlobName } = await createBuild({ + active: false, + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + createdAt: new Date(NOW.getTime() - 5 * 60 * 1000), + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }) + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }), + finishedAt: NOW, + allowLateSuccess: true, + }) + ) + ).resolves.toBe('SETTLED') + + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.SETTLED, + errorCode: 'KB_GRAPH_LATE_SUCCESS_STALE', + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ + activeGraphBuildId: null, + publishedGraphBuildId: null, + }) + }) + + it('settles a late success without publishing when a newer build exists', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + await prisma.kBResource.create({ + data: { + id: LATE_RESOURCE_ID, + kbId, + type: KBResourceType.URL, + title: 'Superseded late success resource', + sourceUrl: 'https://content.example.org/superseded.pdf', + status: KBResourceStatus.READY, + activeResourceVersion: 1, + activeContentSha256: LATE_CONTENT_SHA256, + }, + }) + const { build, runId, graphmlBlobName } = await createBuild({ + active: false, + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + createdAt: new Date(NOW.getTime() - 5 * 60 * 1000), + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }) + await createBuild({ + active: false, + status: KBGraphBuildStatus.QUEUED, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + createdAt: NOW, + }) + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + sourceContentDigest: LATE_SOURCE_CONTENT_DIGEST, + }), + finishedAt: NOW, + allowLateSuccess: true, + }) + ) + ).resolves.toBe('SETTLED') + + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.SUPERSEDED, + costStatus: KBGraphCostStatus.SETTLED, + errorCode: 'KB_GRAPH_LATE_SUCCESS_SUPERSEDED', + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ + activeGraphBuildId: null, + publishedGraphBuildId: null, + }) + }) + + it('holds invalid results until a valid late success reconciles the reservation', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build, runId, graphmlBlobName } = await createBuild() + const invalidResult = { + ...successfulResult({ buildId: build.id, runId, graphmlBlobName }), + owner_id: randomUUID(), + } + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: invalidResult, + finishedAt: NOW, + }) + ) + ).resolves.toBe('NEEDS_HUMAN_REVIEW') + + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ + reservedMinorUnits: 100, + settledMinorUnits: 0, + }) + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + errorCode: 'KB_GRAPH_RESULT_CONTRACT_INVALID', + }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ activeGraphBuildId: null }) + + await expect( + prisma.$transaction((tx) => releaseKBGraphCostReservation(tx, build.id)) + ).resolves.toBe(false) + + const lateFailure = { + ...successfulResult({ buildId: build.id, runId, graphmlBlobName }), + status: 'FAILED', + error_code: 'KB_GRAPH_PROVIDER_FAILED', + graphml_artifact: null, + metered_cost: null, + } + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: lateFailure, + finishedAt: NOW, + }) + ) + ).resolves.toBe('NEEDS_HUMAN_REVIEW') + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 100, settledMinorUnits: 0 }) + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + }), + finishedAt: NOW, + }) + ) + ).resolves.toBe('SETTLED') + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 0, settledMinorUnits: 60 }) + }) + + it('releases a reservation exactly once when dispatch never starts', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build } = await createBuild() + + await expect( + prisma.$transaction((tx) => releaseKBGraphCostReservation(tx, build.id)) + ).resolves.toBe(true) + await expect( + prisma.$transaction((tx) => releaseKBGraphCostReservation(tx, build.id)) + ).resolves.toBe(false) + + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 0, settledMinorUnits: 0 }) + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ costStatus: KBGraphCostStatus.RELEASED }) + }) + + it('settles metered non-success results without publishing the build', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build, runId, graphmlBlobName } = await createBuild() + const failedResult = { + ...successfulResult({ buildId: build.id, runId, graphmlBlobName }), + status: 'FAILED', + error_code: 'KB_GRAPH_PROVIDER_FAILED', + graphml_artifact: null, + } + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: failedResult, + finishedAt: NOW, + }) + ) + ).resolves.toBe('SETTLED') + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.SETTLED, + errorCode: 'KB_GRAPH_PROVIDER_FAILED', + actualCostMinorUnits: 60, + }) + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 0, settledMinorUnits: 60 }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ + activeGraphBuildId: null, + publishedGraphBuildId: null, + }) + }) + + it('holds metering whose aggregate counters exceed the database integer range', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build, runId, graphmlBlobName } = await createBuild() + const result = successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + }) + result.metered_cost!.components.push({ + ...result.metered_cost!.components[0]!, + amount_minor_units: 0, + input_tokens: 2_147_483_647, + }) + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result, + finishedAt: NOW, + }) + ) + ).resolves.toBe('NEEDS_HUMAN_REVIEW') + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + errorCode: 'KB_GRAPH_RESULT_METERING_OVERFLOW', + }) + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 100, settledMinorUnits: 0 }) + }) + + it('does not publish a valid result after cleanup has started', async () => { + await prisma.$transaction((tx) => + reserveKBGraphCost(tx, { + ownerId, + qualityTier: KBGraphQualityTier.STANDARD, + env: costEnv, + now: NOW, + }) + ) + const { build, runId, graphmlBlobName } = await createBuild({ + cleanupStartedAt: NOW, + }) + + await expect( + prisma.$transaction((tx) => + settleKBGraphBuildCost(tx, { + buildId: build.id, + result: successfulResult({ + buildId: build.id, + runId, + graphmlBlobName, + }), + finishedAt: NOW, + }) + ) + ).resolves.toBe('NEEDS_HUMAN_REVIEW') + await expect( + prisma.kBGraphBuild.findUniqueOrThrow({ where: { id: build.id } }) + ).resolves.toMatchObject({ + status: KBGraphBuildStatus.FAILED, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + errorCode: 'KB_GRAPH_RESULT_AFTER_CLEANUP', + actualCostMinorUnits: null, + }) + await expect( + prisma.kBGraphQuota.findUniqueOrThrow({ + where: { ownerId_semesterKey: { ownerId, semesterKey: '2026-H2' } }, + }) + ).resolves.toMatchObject({ reservedMinorUnits: 100, settledMinorUnits: 0 }) + await expect( + prisma.kB.findUniqueOrThrow({ where: { id: kbId } }) + ).resolves.toMatchObject({ publishedGraphBuildId: null }) + }) +}) diff --git a/packages/graphql/test/knowledgeGraphConfig.test.ts b/packages/graphql/test/knowledgeGraphConfig.test.ts new file mode 100644 index 0000000000..99c2d6a3a9 --- /dev/null +++ b/packages/graphql/test/knowledgeGraphConfig.test.ts @@ -0,0 +1,75 @@ +import { + KBGraphBuildStatus, + KBGraphQualityTier, +} from '@klicker-uzh/prisma/client' +import { describe, expect, it } from 'vitest' +import { getKBGraphBuildConfig } from '../src/services/knowledge.js' +import { getKBGraphCostConfiguration } from '../src/services/knowledgeGraphCost.js' + +const costEnv = { + KB_GRAPH_COST_CURRENCY: 'CHF', + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '100', + KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS: '200', + KB_GRAPH_MAX_COST_MINOR_UNITS: '250', + KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '1000', + KB_GRAPH_COST_PRICING_VERSION: 'test-v1', + KB_GRAPH_SEMESTER_KEY: '2026-H2', +} + +const build: Parameters[1] = { + id: '11111111-1111-4111-8111-111111111111', + status: KBGraphBuildStatus.SUCCEEDED, + statusMessage: null, + qualityTier: KBGraphQualityTier.STANDARD, + sourceContentDigest: 'source-digest', + startedAt: null, + finishedAt: null, + createdAt: new Date('2026-08-15T00:00:00.000Z'), + updatedAt: new Date('2026-08-15T00:00:00.000Z'), + estimatedCostMinorUnits: 100, + actualCostMinorUnits: 60, + actualInputTokens: 11, + actualOutputTokens: 13, + actualEmbeddingTokens: 7, + actualRequestCount: 2, + costCurrency: 'CHF', + costStatus: null, + quotaId: '22222222-2222-4222-8222-222222222222', + quota: { + currency: 'CHF', + limitMinorUnits: 1000, + reservedMinorUnits: 0, + settledMinorUnits: 60, + }, +} + +describe('KB knowledge graph config', () => { + it('reports quota configuration drift and keeps quota currency separate', () => { + const costConfiguration = getKBGraphCostConfiguration(costEnv) + const result = getKBGraphBuildConfig( + { + id: '33333333-3333-4333-8333-333333333333', + knowledgeGraphEnabled: true, + activeGraphBuildId: null, + publishedGraphBuildId: build.id, + }, + { + ...build, + costCurrency: 'EUR', + }, + false, + { + currency: 'USD', + limitMinorUnits: 900, + reservedMinorUnits: 100, + settledMinorUnits: 50, + }, + costConfiguration + ) + + expect(result.costConfigurationReady).toBe(false) + expect(result.costCurrency).toBe('EUR') + expect(result.quotaCurrency).toBe('USD') + expect(result.remainingSemesterQuotaMinorUnits).toBe(750) + }) +}) diff --git a/packages/graphql/test/knowledgeGraphCost.test.ts b/packages/graphql/test/knowledgeGraphCost.test.ts new file mode 100644 index 0000000000..f74f2f9188 --- /dev/null +++ b/packages/graphql/test/knowledgeGraphCost.test.ts @@ -0,0 +1,86 @@ +import { KBGraphQualityTier } from '@klicker-uzh/prisma/client' +import { GraphQLError } from 'graphql' +import { describe, expect, it } from 'vitest' +import { + getKBGraphCostConfiguration, + getKBGraphEstimate, + getKBGraphSemesterKey, + requireKBGraphCostConfiguration, +} from '../src/services/knowledgeGraphCost.js' + +const configuredEnv = { + KB_GRAPH_COST_CURRENCY: 'CHF', + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '100', + KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS: '200', + KB_GRAPH_MAX_COST_MINOR_UNITS: '250', + KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS: '1000', + KB_GRAPH_COST_PRICING_VERSION: '2026-08', + KB_GRAPH_BILLING_MODE: 'SEMESTER_QUOTA', + KB_GRAPH_SEMESTER_KEY: '2026-H2', +} + +describe('knowledge graph cost configuration', () => { + it('fails closed when monetary configuration is absent', () => { + const config = getKBGraphCostConfiguration( + {}, + new Date('2026-08-15T00:00:00.000Z') + ) + + expect(config.ready).toBe(false) + expect(config.currency).toBe('CHF') + expect(config.semesterKey).toBe('2026-H2') + }) + + it('reads integer minor-unit limits and tier estimates', () => { + const config = getKBGraphCostConfiguration(configuredEnv) + + expect(config).toMatchObject({ + ready: true, + standardEstimateMinorUnits: 100, + highEstimateMinorUnits: 200, + maxCostMinorUnits: 250, + semesterQuotaMinorUnits: 1000, + semesterKey: '2026-H2', + }) + expect(getKBGraphEstimate(KBGraphQualityTier.STANDARD, config)).toBe(100) + expect(getKBGraphEstimate(KBGraphQualityTier.HIGH, config)).toBe(200) + }) + + it('does not expose zero-value monetary settings as ready', () => { + const config = getKBGraphCostConfiguration({ + ...configuredEnv, + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '0', + }) + + expect(config.ready).toBe(false) + expect(() => requireKBGraphCostConfiguration(configuredEnv)).not.toThrow() + expect(() => + requireKBGraphCostConfiguration({ + ...configuredEnv, + KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS: '0', + }) + ).toThrow(GraphQLError) + }) + + it('rejects a maximum below a configured tier estimate', () => { + expect(() => + getKBGraphCostConfiguration({ + ...configuredEnv, + KB_GRAPH_MAX_COST_MINOR_UNITS: '99', + }) + ).toThrow('must cover the standard estimate') + }) + + it('rejects malformed semester keys', () => { + expect(() => + getKBGraphSemesterKey(new Date('2026-08-15T00:00:00.000Z'), { + KB_GRAPH_SEMESTER_KEY: '2026-fall', + }) + ).toThrow('YYYY-H1 or YYYY-H2') + }) + + it('requires a complete configuration before reservation', () => { + expect(() => requireKBGraphCostConfiguration({})).toThrow(GraphQLError) + expect(() => requireKBGraphCostConfiguration(configuredEnv)).not.toThrow() + }) +}) diff --git a/packages/graphql/test/knowledgeIngestion.test.ts b/packages/graphql/test/knowledgeIngestion.test.ts new file mode 100644 index 0000000000..2a76ce5eca --- /dev/null +++ b/packages/graphql/test/knowledgeIngestion.test.ts @@ -0,0 +1,475 @@ +import type { Hatchet } from '@hatchet-dev/typescript-sdk' +import { prisma as prismaClient } from '@klicker-uzh/prisma' +import { KBResourceType, PrismaClient } from '@klicker-uzh/prisma/client' +import { EventEmitter } from 'events' +import { vi } from 'vitest' +import type { ContextWithUser } from '../src/lib/context.js' +import { + createKb, + createKbUrlResource, + ingestKbResource, +} from '../src/services/knowledge.js' +import { testCleanup, testInitialization } from './helpers.js' + +const UUID_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/ + +describe('Integration tests for knowledge base ingestion', () => { + let prisma: PrismaClient + let hatchet: Hatchet + let emitter: EventEmitter + let userOneCtx: ContextWithUser + let userTwoCtx: ContextWithUser + + beforeAll(async () => { + prisma = prismaClient + await testCleanup(prisma) + hatchet = { + task: vi.fn(() => ({ runNoWait: vi.fn() })), + } as unknown as Hatchet + emitter = new EventEmitter() + }) + + afterAll(async () => { + await testCleanup(prisma) + await prisma.$disconnect() + }) + + beforeEach(async () => { + const initialized = await testInitialization(prisma, hatchet, emitter) + userOneCtx = initialized.userOneCtx + userTwoCtx = initialized.userTwoCtx + await prisma.user.updateMany({ + where: { id: { in: [userOneCtx.user.sub, userTwoCtx.user.sub] } }, + data: { privatePreview: true }, + }) + }) + + afterEach(async () => { + vi.restoreAllMocks() + await testCleanup(prisma) + }) + + it('queues an owned URL resource with a fresh attempt', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const queued = await ingestKbResource({ id: resource.id }, userOneCtx) + + expect(queued).toMatchObject({ + status: 'QUEUED', + ingestionAttemptId: expect.stringMatching(UUID_PATTERN), + resourceVersion: 1, + }) + expect(runNoWait).toHaveBeenCalledWith({ + resourceId: resource.id, + kbId: created.id, + type: 'URL', + title: 'Lecture recording', + sourceUrl: 'https://video.example.com/course', + ingestionAttemptId: queued.ingestionAttemptId, + resourceVersion: 1, + }) + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + status: 'QUEUED', + ingestionAttemptId: queued.ingestionAttemptId, + resourceVersion: 1, + }) + await expect( + prisma.kBIngestionRun.findUnique({ + where: { id: queued.ingestionAttemptId! }, + }) + ).resolves.toMatchObject({ + resourceId: resource.id, + resourceVersion: 1, + status: 'QUEUED', + }) + }) + + it('claims a new attempt while preserving active serving metadata', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const oldAttemptId = '1f9aa27b-ee62-4b52-9c76-5f9f024347fd' + const ingestedAt = new Date('2026-07-19T12:00:00.000Z') + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { + status: 'READY', + statusMessage: 'Previous ingestion completed', + ingestedAt, + ingestionAttemptId: oldAttemptId, + resourceVersion: 2, + contentSha256: 'a'.repeat(64), + externalOperationId: 'old-operation-id', + externalOperationStartedAt: new Date('2026-07-19T11:30:00.000Z'), + activeResourceVersion: 2, + activeContentSha256: 'a'.repeat(64), + }, + }) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const queued = await ingestKbResource({ id: resource.id }, userOneCtx) + + expect(queued).toMatchObject({ + status: 'QUEUED', + statusMessage: null, + ingestedAt, + resourceVersion: 3, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + activeResourceVersion: 2, + activeContentSha256: 'a'.repeat(64), + }) + expect(queued.ingestionAttemptId).toMatch(UUID_PATTERN) + expect(queued.ingestionAttemptId).not.toBe(oldAttemptId) + expect(runNoWait).toHaveBeenCalledWith( + expect.objectContaining({ + ingestionAttemptId: queued.ingestionAttemptId, + resourceVersion: 3, + }) + ) + }) + + it('queues a READY blob resource with its private container location', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.BLOB, + title: 'Finance notes', + originalFilename: 'notes.pdf', + mimeType: 'application/pdf', + sizeBytes: 1024, + blobName: '79a40d25-78cf-4bde-9661-a07747d7b715.pdf', + blobHref: + 'https://kbtestaccount.blob.core.windows.net/container/notes.pdf', + status: 'READY', + }, + }) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const queued = await ingestKbResource({ id: resource.id }, userOneCtx) + + expect(queued.status).toBe('QUEUED') + expect(runNoWait).toHaveBeenCalledWith({ + resourceId: resource.id, + kbId: created.id, + type: 'BLOB', + title: 'Finance notes', + blobName: resource.blobName, + containerName: `kb-${userOneCtx.user.sub}`, + ingestionAttemptId: queued.ingestionAttemptId, + resourceVersion: 1, + mimeType: 'application/pdf', + sizeBytes: 1024, + }) + }) + + it('denies foreign or already active resources without dispatching', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + await expect( + ingestKbResource({ id: resource.id }, userTwoCtx) + ).rejects.toThrow('KB resource not found') + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { status: 'PROCESSING' }, + }) + await expect( + ingestKbResource({ id: resource.id }, userOneCtx) + ).rejects.toThrow('KB resource cannot be ingested') + expect(runNoWait).not.toHaveBeenCalled() + }) + + it('claims a resource once when ingestion requests race', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + + const results = await Promise.allSettled([ + ingestKbResource({ id: resource.id }, userOneCtx), + ingestKbResource({ id: resource.id }, userOneCtx), + ]) + + expect( + results.filter((result) => result.status === 'fulfilled') + ).toHaveLength(1) + expect( + results.filter((result) => result.status === 'rejected') + ).toHaveLength(1) + expect(runNoWait).toHaveBeenCalledTimes(1) + const dispatchedAttemptId = ( + runNoWait.mock.calls[0]?.[0] as unknown as { + ingestionAttemptId: string + } + ).ingestionAttemptId + expect(dispatchedAttemptId).toMatch(UUID_PATTERN) + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + status: 'QUEUED', + ingestionAttemptId: dispatchedAttemptId, + resourceVersion: 1, + }) + await expect( + prisma.kBIngestionRun.count({ where: { resourceId: resource.id } }) + ).resolves.toBe(1) + }) + + it('rejects an ABA claim when the observed attempt changes at the same status', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const observedAttemptId = '60bf5833-1a03-4586-a9a0-f7e1ea0f7eef' + const newerAttemptId = '9c739b93-4f48-4f0d-bac1-5db4e27c821d' + const resource = await prisma.kBResource.create({ + data: { + kbId: created.id, + type: KBResourceType.URL, + title: 'Lecture recording', + sourceUrl: 'https://video.example.com/course', + status: 'READY', + ingestionAttemptId: observedAttemptId, + }, + }) + const runNoWait = vi + .spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait') + .mockResolvedValue({} as never) + const kbResource = prisma.kBResource + const abaCtx = { + ...userOneCtx, + prisma: { + user: { + findUnique: userOneCtx.prisma.user.findUnique.bind( + userOneCtx.prisma.user + ), + }, + kBResource: { + findFirst: kbResource.findFirst.bind(kbResource), + }, + $transaction: async ( + callback: (tx: { + kBResource: { + updateMany: ( + args: Parameters[0] + ) => Promise<{ count: number }> + } + kBIngestionRun: { + create: ( + args: Parameters[0] + ) => Promise + } + }) => Promise + ) => { + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { + ingestionAttemptId: newerAttemptId, + statusMessage: 'Newer same-status attempt', + }, + }) + return prisma.$transaction(async (tx) => + callback({ + kBResource: { + updateMany: async (args) => tx.kBResource.updateMany(args), + }, + kBIngestionRun: { + create: async (args) => tx.kBIngestionRun.create(args), + }, + }) + ) + }, + }, + } as unknown as ContextWithUser + + await expect(ingestKbResource({ id: resource.id }, abaCtx)).rejects.toThrow( + 'KB resource cannot be ingested' + ) + expect(runNoWait).not.toHaveBeenCalled() + await expect( + prisma.kBResource.findUniqueOrThrow({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + status: 'READY', + statusMessage: 'Newer same-status attempt', + ingestionAttemptId: newerAttemptId, + }) + }) + + it('records a failed attempt when Hatchet queue dispatch fails', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const oldAttemptId = '3b894217-e5dc-4d39-a94d-b21b08f4725e' + const oldIngestedAt = new Date('2026-07-18T09:00:00.000Z') + const oldExternalStartedAt = new Date('2026-07-18T08:30:00.000Z') + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { + status: 'FAILED', + statusMessage: 'Previous external run failed', + ingestedAt: oldIngestedAt, + ingestionAttemptId: oldAttemptId, + resourceVersion: 2, + contentSha256: 'b'.repeat(64), + externalOperationId: 'previous-operation-id', + externalOperationStartedAt: oldExternalStartedAt, + activeResourceVersion: 1, + activeContentSha256: 'c'.repeat(64), + }, + }) + vi.spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait').mockRejectedValue( + new Error('Hatchet unavailable') + ) + + await expect( + ingestKbResource({ id: resource.id }, userOneCtx) + ).rejects.toThrow('KB ingestion could not be queued') + const failed = await prisma.kBResource.findUniqueOrThrow({ + where: { id: resource.id }, + }) + expect(failed).toMatchObject({ + status: 'FAILED', + statusMessage: 'The ingestion operation could not be queued.', + ingestedAt: oldIngestedAt, + ingestionAttemptId: expect.stringMatching(UUID_PATTERN), + resourceVersion: 3, + contentSha256: null, + externalOperationId: null, + externalOperationStartedAt: null, + activeResourceVersion: 1, + activeContentSha256: 'c'.repeat(64), + errorCode: 'QUEUE_DISPATCH_FAILED', + }) + expect(failed.ingestionAttemptId).not.toBe(oldAttemptId) + await expect( + prisma.kBIngestionRun.findUniqueOrThrow({ + where: { id: failed.ingestionAttemptId! }, + }) + ).resolves.toMatchObject({ + status: 'FAILED', + resourceId: resource.id, + resourceVersion: 3, + errorCode: 'QUEUE_DISPATCH_FAILED', + finishedAt: expect.any(Date), + }) + }) + + it('does not roll back a resource that advanced after dispatch began', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + vi.spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait').mockImplementation( + async () => { + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { status: 'PROCESSING' }, + }) + throw new Error('Hatchet response lost') + } + ) + + await expect( + ingestKbResource({ id: resource.id }, userOneCtx) + ).rejects.toThrow('KB ingestion could not be queued') + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ status: 'PROCESSING' }) + }) + + it('does not let a stale dispatch failure roll back a newer queued attempt', async () => { + const created = await createKb({ name: 'Finance notes' }, userOneCtx) + const resource = await createKbUrlResource( + { + kbId: created.id, + title: 'Lecture recording', + url: 'https://video.example.com/course', + }, + userOneCtx + ) + const newerAttemptId = '7adf2e60-82b8-436a-90bd-ae6eb142385a' + const newerStartedAt = new Date('2026-07-20T08:30:00.000Z') + vi.spyOn(userOneCtx.tasks.ingestKBResource, 'runNoWait').mockImplementation( + async () => { + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { + ingestionAttemptId: newerAttemptId, + resourceVersion: 2, + externalOperationId: 'newer-operation-id', + externalOperationStartedAt: newerStartedAt, + statusMessage: 'Newer attempt accepted', + }, + }) + throw new Error('Stale Hatchet response lost') + } + ) + + await expect( + ingestKbResource({ id: resource.id }, userOneCtx) + ).rejects.toThrow('KB ingestion could not be queued') + await expect( + prisma.kBResource.findUnique({ where: { id: resource.id } }) + ).resolves.toMatchObject({ + status: 'QUEUED', + statusMessage: 'Newer attempt accepted', + ingestionAttemptId: newerAttemptId, + resourceVersion: 2, + externalOperationId: 'newer-operation-id', + externalOperationStartedAt: newerStartedAt, + }) + }) +}) diff --git a/packages/graphql/test/knowledgeSourceGateway.test.ts b/packages/graphql/test/knowledgeSourceGateway.test.ts new file mode 100644 index 0000000000..2bdfb3fcc1 --- /dev/null +++ b/packages/graphql/test/knowledgeSourceGateway.test.ts @@ -0,0 +1,440 @@ +import { BlobServiceClient } from '@azure/storage-blob' +import type { Hatchet } from '@hatchet-dev/typescript-sdk' +import { prisma as prismaClient } from '@klicker-uzh/prisma' +import { + KBResourceStatus, + KBResourceType, + PrismaClient, +} from '@klicker-uzh/prisma/client' +import { randomUUID } from 'crypto' +import { EventEmitter } from 'events' +import { Readable } from 'node:stream' +import { + afterAll, + afterEach, + beforeAll, + describe, + expect, + it, + vi, +} from 'vitest' +import type { ContextWithUser } from '../src/lib/context.js' +import { handleKBSourceGateway } from '../src/services/knowledgeSourceGateway.js' +import { testCleanup, testInitialization } from './helpers.js' + +const RESOURCE_ID = '7f3e2a10-9c4b-4d8e-b1a6-5e0f9d2c7b3a' +const RESOURCE_VERSION = 3 +const env = { + KB_SOURCE_GATEWAY_KEY: 'gateway-key', + BLOB_STORAGE_ACCOUNT_NAME: 'kbaccount', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), +} + +function prismaWithResource(resource: Record | null) { + return { + kBResource: { + findFirst: vi.fn().mockResolvedValue(resource), + }, + } +} + +afterEach(() => { + vi.restoreAllMocks() +}) + +describe('KB source gateway', () => { + it('fails closed when gateway configuration is incomplete', async () => { + const prisma = prismaWithResource(null) + + await expect( + handleKBSourceGateway({ + prisma: prisma as never, + resourceId: RESOURCE_ID, + resourceVersion: RESOURCE_VERSION, + authorization: 'Bearer gateway-key', + env: {}, + }) + ).resolves.toEqual({ + statusCode: 503, + body: { error: 'Service unavailable' }, + }) + expect(prisma.kBResource.findFirst).not.toHaveBeenCalled() + }) + + it('rejects unauthorized callers before looking up source metadata', async () => { + const prisma = prismaWithResource(null) + + await expect( + handleKBSourceGateway({ + prisma: prisma as never, + resourceId: RESOURCE_ID, + resourceVersion: RESOURCE_VERSION, + authorization: 'Bearer wrong-key', + env, + }) + ).resolves.toEqual({ + statusCode: 401, + body: { error: 'Unauthorized' }, + }) + expect(prisma.kBResource.findFirst).not.toHaveBeenCalled() + }) + + // NOTE: a prior test here ("streams the exact active blob resource + // version") mirror-asserted the implementation's `findFirst` where-clause + // against a fully mocked Prisma client via `toHaveBeenCalledWith`. That + // proved only that the code passes the literal object it always would -- + // it never ran against real rows, so a broken filter (e.g. wrong status + // set, missing a clause) would still pass. It has been replaced by the + // real-PostgreSQL "KB source gateway authz filter (real database)" suite + // below, which seeds actual KBResource rows and proves each clause one at + // a time. The streaming-layer concerns that test also touched (blob + // client wiring, container-name derivation) are re-proven there against a + // real DB-joined owner id. + + it('does not expose an unavailable resource version', async () => { + const prisma = prismaWithResource(null) + const getContainerClient = vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ) + + await expect( + handleKBSourceGateway({ + prisma: prisma as never, + resourceId: RESOURCE_ID, + resourceVersion: RESOURCE_VERSION, + authorization: 'Bearer gateway-key', + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + expect(getContainerClient).not.toHaveBeenCalled() + }) + + it('rejects blob metadata drift instead of streaming changed bytes', async () => { + const download = vi.fn().mockResolvedValue({ + contentLength: 8, + contentType: 'application/pdf', + readableStreamBody: Readable.from([Buffer.from('changed!')]), + }) + vi.spyOn(BlobServiceClient.prototype, 'getContainerClient').mockReturnValue( + { + getBlobClient: vi.fn().mockReturnValue({ download }), + } as never + ) + const prisma = prismaWithResource({ + blobName: `${RESOURCE_ID}.pdf`, + mimeType: 'application/pdf', + sizeBytes: 7, + kb: { ownerId: 'owner-id' }, + }) + + await expect( + handleKBSourceGateway({ + prisma: prisma as never, + resourceId: RESOURCE_ID, + resourceVersion: RESOURCE_VERSION, + authorization: 'Bearer gateway-key', + env, + }) + ).resolves.toEqual({ + statusCode: 502, + body: { error: 'Source unavailable' }, + }) + }) +}) + +// The mocked-Prisma tests above cover the gateway's non-DB concerns (fail +// closed on missing config, HMAC authorization, blob metadata drift). The +// suite below seeds real rows in Postgres and proves the `findFirst` +// where-clause's actual semantics, one clause at a time: +// id === resourceId +// resourceVersion === resourceVersion (exact, not >=) +// deletedAt is null (tombstoned resources are excluded) +// type === BLOB (URL resources are excluded) +// contentSha256 is not null (digest has been computed) +// status in [QUEUED, PROCESSING] (READY/FAILED/ADDED are excluded) +// There is no owner/foreign-KB clause at all -- the function takes no +// caller identity to check against an owner in the first place. The test near +// the end documents this system-to-system gateway-key trust model explicitly. +describe('KB source gateway authz filter (real database)', () => { + let prisma: PrismaClient + let userOneCtx: ContextWithUser + let userTwoCtx: ContextWithUser + let kbId: string + + const authorization = 'Bearer gateway-key' + + beforeAll(async () => { + prisma = prismaClient + await testCleanup(prisma) + const hatchet = { + task: vi.fn(() => ({ runNoWait: vi.fn() })), + } as unknown as Hatchet + const initialized = await testInitialization( + prisma, + hatchet, + new EventEmitter() + ) + userOneCtx = initialized.userOneCtx + userTwoCtx = initialized.userTwoCtx + const kb = await prisma.kB.create({ + data: { name: 'Gateway fixture', ownerId: userOneCtx.user.sub }, + }) + kbId = kb.id + }) + + afterAll(async () => { + await testCleanup(prisma) + await prisma.$disconnect() + }) + + function mockBlobDownload(bytes: string, mimeType: string) { + const download = vi.fn().mockResolvedValue({ + contentLength: Buffer.byteLength(bytes), + contentType: mimeType, + readableStreamBody: Readable.from([Buffer.from(bytes)]), + }) + const getBlobClient = vi.fn().mockReturnValue({ download }) + const getContainerClient = vi.fn().mockReturnValue({ getBlobClient }) + vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ).mockImplementation(getContainerClient) + return { getContainerClient, getBlobClient } + } + + async function createGatewayResource({ + status = KBResourceStatus.QUEUED, + type = KBResourceType.BLOB, + resourceVersion = 3, + contentSha256 = 'f'.repeat(64), + kbId: kbIdOverride, + }: { + status?: KBResourceStatus + type?: KBResourceType + resourceVersion?: number + contentSha256?: string | null + kbId?: string + } = {}) { + return prisma.kBResource.create({ + data: { + kbId: kbIdOverride ?? kbId, + type, + title: 'Lecture', + blobName: `${randomUUID()}.pdf`, + mimeType: 'application/pdf', + sizeBytes: 7, + contentSha256, + status, + resourceVersion, + }, + }) + } + + it('serves the blob when every clause of the filter is satisfied, deriving the container from the real owner join', async () => { + const resource = await createGatewayResource({ + status: KBResourceStatus.QUEUED, + resourceVersion: 3, + }) + const { getContainerClient, getBlobClient } = mockBlobDownload( + 'lecture', + 'application/pdf' + ) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: 3, + authorization, + env, + }) + ).resolves.toMatchObject({ + statusCode: 200, + contentLength: 7, + contentType: 'application/pdf', + }) + expect(getContainerClient).toHaveBeenCalledWith(`kb-${userOneCtx.user.sub}`) + expect(getBlobClient).toHaveBeenCalledWith(resource.blobName) + }) + + it('also serves a resource that is still PROCESSING -- the other allowed status value', async () => { + const resource = await createGatewayResource({ + status: KBResourceStatus.PROCESSING, + }) + mockBlobDownload('lecture', 'application/pdf') + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toMatchObject({ statusCode: 200 }) + }) + + it.each([ + KBResourceStatus.ADDED, + KBResourceStatus.READY, + KBResourceStatus.FAILED, + ])('rejects status %s even though every other clause matches', async (status) => { + const resource = await createGatewayResource({ status }) + const getContainerClient = vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + expect(getContainerClient).not.toHaveBeenCalled() + }) + + it('rejects a resource whose content digest has not been computed yet', async () => { + const resource = await createGatewayResource({ contentSha256: null }) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + }) + + it('rejects a URL-type resource even if every other clause matches', async () => { + const resource = await createGatewayResource({ type: KBResourceType.URL }) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + }) + + it('rejects a stale resource-version request once the row has moved on to a newer version', async () => { + const resource = await createGatewayResource({ resourceVersion: 5 }) + const getContainerClient = vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: 3, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + expect(getContainerClient).not.toHaveBeenCalled() + + mockBlobDownload('lecture', 'application/pdf') + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: 5, + authorization, + env, + }) + ).resolves.toMatchObject({ statusCode: 200 }) + }) + + it('rejects a request for an id with no matching resource row', async () => { + await createGatewayResource() + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: randomUUID(), + resourceVersion: 3, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + }) + + it('derives the blob container from the resource owner under the system-to-system gateway-key trust model', async () => { + const otherOwnerKb = await prisma.kB.create({ + data: { name: 'Different owner', ownerId: userTwoCtx.user.sub }, + }) + const resource = await createGatewayResource({ + kbId: otherOwnerKb.id, + }) + const { getContainerClient } = mockBlobDownload( + 'lecture', + 'application/pdf' + ) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toMatchObject({ statusCode: 200 }) + expect(getContainerClient).toHaveBeenCalledWith(`kb-${userTwoCtx.user.sub}`) + }) + + it('rejects a tombstoned resource before accessing blob storage', async () => { + const resource = await createGatewayResource() + await prisma.kBResource.update({ + where: { id: resource.id }, + data: { deletedAt: new Date(), deletedById: userOneCtx.user.sub }, + }) + const getContainerClient = vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ) + + await expect( + handleKBSourceGateway({ + prisma, + resourceId: resource.id, + resourceVersion: resource.resourceVersion, + authorization, + env, + }) + ).resolves.toEqual({ + statusCode: 404, + body: { error: 'Resource not found' }, + }) + expect(getContainerClient).not.toHaveBeenCalled() + }) +}) diff --git a/packages/graphql/test/knowledgeWebhooks.test.ts b/packages/graphql/test/knowledgeWebhooks.test.ts new file mode 100644 index 0000000000..7165d25239 --- /dev/null +++ b/packages/graphql/test/knowledgeWebhooks.test.ts @@ -0,0 +1,737 @@ +import { prisma as prismaClient } from '@klicker-uzh/prisma' +import { + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + KBResourceType, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import { + handleKBIngestionWebhook, + signKBIngestionWebhook, +} from '../src/services/knowledgeWebhooks.js' + +const SECRET = 'kb-webhook-test-secret' +const PREVIOUS_SECRET = 'kb-webhook-previous-test-secret' +const OWNER_ID = 'c08036f0-5354-47dc-aac0-408a89c251a5' +const EVENT_ID = 'e8a1b2c3-d4e5-4f60-9a7b-8c9d0e1f2a3b' +const OTHER_EVENT_ID = 'f92f85a3-bbbc-47cb-8739-f93ed85bdce5' +const REFRESH_EVENT_ID = 'a2d2c4e8-8e04-49d7-9a5c-0cb485cf57c2' +const INGESTION_ATTEMPT_ID = 'e69e7cbd-c301-41d4-b653-bb645576d637' +const OPERATION_ID = 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND' +const REFRESH_OPERATION_ID = 'op_01K2X8K3M9QZ4R7T6V5W1Y0BND' +const RESOURCE_VERSION = 3 +const CONTENT_SHA256 = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' +const REFRESH_CONTENT_SHA256 = 'f'.repeat(64) +const OCCURRED_AT = '2026-07-12T14:04:52Z' + +type EventType = + | 'resource.processing_started' + | 'resource.processing_progress' + | 'resource.processing_succeeded' + | 'resource.processing_failed' + | 'resource.content_refreshed' + | 'resource.subresources_updated' + | 'kb.metrics_updated' + +function canonicalJson(value: unknown): string { + if (Array.isArray(value)) { + return `[${value.map(canonicalJson).join(',')}]` + } + if (value !== null && typeof value === 'object') { + const record = value as Record + return `{${Object.keys(record) + .sort() + .map((key) => `${JSON.stringify(key)}:${canonicalJson(record[key])}`) + .join(',')}}` + } + return JSON.stringify(value) ?? 'null' +} + +describe('KB ingestion webhook contract', () => { + let prisma: PrismaClient + let resourceId: string + + function event( + eventType: EventType, + overrides: Record = {} + ) { + return { + eventId: EVENT_ID, + eventType, + occurredAt: OCCURRED_AT, + operation_id: OPERATION_ID, + external_resource_id: resourceId, + resource_version: RESOURCE_VERSION, + serving: { + active_resource_version: + eventType === 'resource.processing_succeeded' || + eventType === 'resource.content_refreshed' + ? RESOURCE_VERSION + : null, + active_sha256: + eventType === 'resource.processing_succeeded' || + eventType === 'resource.content_refreshed' + ? CONTENT_SHA256 + : null, + }, + error_code: + eventType === 'resource.processing_failed' + ? 'source_fetch_failed' + : null, + statusDetail: null, + correlation_id: INGESTION_ATTEMPT_ID, + ...overrides, + } + } + + function createRequest( + payload: Record, + { + secret = SECRET, + timestamp = Math.floor(Date.now() / 1000), + }: { secret?: string; timestamp?: number | string } = {} + ) { + const rawBody = Buffer.from(canonicalJson(payload)) + return { + rawBody, + headers: signKBIngestionWebhook({ + eventId: String(payload.eventId), + eventType: String(payload.eventType), + rawBody, + secret, + timestamp, + }), + } + } + + async function getResource() { + return prisma.kBResource.findUniqueOrThrow({ where: { id: resourceId } }) + } + + async function getRun() { + return prisma.kBIngestionRun.findUniqueOrThrow({ + where: { id: INGESTION_ATTEMPT_ID }, + }) + } + + async function getRunByExternalOperation(externalOperationId: string) { + return prisma.kBIngestionRun.findFirstOrThrow({ + where: { resourceId, externalOperationId }, + }) + } + + beforeAll(async () => { + prisma = prismaClient + }) + + beforeEach(async () => { + await prisma.user.deleteMany({ where: { id: OWNER_ID } }) + await prisma.user.create({ + data: { + id: OWNER_ID, + email: 'kb-webhook@example.com', + shortname: 'kb-webhook', + }, + }) + const kb = await prisma.kB.create({ + data: { name: 'Webhook test KB', ownerId: OWNER_ID }, + }) + const resource = await prisma.kBResource.create({ + data: { + kbId: kb.id, + type: KBResourceType.URL, + title: 'Webhook test resource', + sourceUrl: 'https://example.com/resource', + status: KBResourceStatus.QUEUED, + ingestionAttemptId: INGESTION_ATTEMPT_ID, + resourceVersion: RESOURCE_VERSION, + contentSha256: CONTENT_SHA256, + externalOperationId: OPERATION_ID, + }, + }) + resourceId = resource.id + await prisma.kBIngestionRun.create({ + data: { + id: INGESTION_ATTEMPT_ID, + resourceId, + resourceVersion: RESOURCE_VERSION, + contentSha256: CONTENT_SHA256, + externalOperationId: OPERATION_ID, + }, + }) + }) + + afterEach(async () => { + await prisma.user.deleteMany({ where: { id: OWNER_ID } }) + }) + + afterAll(async () => { + await prisma.$disconnect() + }) + + it('accepts the canonical processing-started event', async () => { + const request = createRequest(event('resource.processing_started')) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + statusMessage: null, + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.PROCESSING, + startedAt: new Date(OCCURRED_AT), + }) + }) + + it('accepts progress and persists only its safe status detail', async () => { + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { status: KBResourceStatus.PROCESSING }, + }) + const request = createRequest( + event('resource.processing_progress', { + statusDetail: 'Extracting text', + }) + ) + + await handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + statusMessage: 'Extracting text', + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.PROCESSING, + statusMessage: 'Extracting text', + }) + }) + + it('marks the exact serving version and digest ready at occurredAt', async () => { + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { status: KBResourceStatus.PROCESSING }, + }) + const request = createRequest(event('resource.processing_succeeded')) + + await handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.READY, + statusMessage: null, + ingestedAt: new Date(OCCURRED_AT), + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: CONTENT_SHA256, + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.SUCCEEDED, + finishedAt: new Date(OCCURRED_AT), + }) + }) + + it('marks a delete succeeded only when no resource remains served', async () => { + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { + deletedAt: new Date(OCCURRED_AT), + ingestionOperation: KBIngestionOperation.DELETE, + contentSha256: null, + activeResourceVersion: 2, + activeContentSha256: 'a'.repeat(64), + }, + }) + await prisma.kBIngestionRun.update({ + where: { id: INGESTION_ATTEMPT_ID }, + data: { + operation: KBIngestionOperation.DELETE, + contentSha256: null, + }, + }) + const request = createRequest( + event('resource.processing_succeeded', { + serving: { + active_resource_version: null, + active_sha256: null, + }, + }) + ) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.READY, + activeResourceVersion: null, + activeContentSha256: null, + }) + await expect(getRun()).resolves.toMatchObject({ + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.SUCCEEDED, + finishedAt: new Date(OCCURRED_AT), + }) + }) + + it('records success while a different digest is still serving', async () => { + const request = createRequest( + event('resource.processing_succeeded', { + serving: { + active_resource_version: RESOURCE_VERSION, + active_sha256: '0'.repeat(64), + }, + }) + ) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + ingestedAt: null, + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: '0'.repeat(64), + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.SUCCEEDED, + finishedAt: new Date(OCCURRED_AT), + }) + }) + + it('maps a failed event to a sanitized local failure', async () => { + const request = createRequest(event('resource.processing_failed')) + + await handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.FAILED, + statusMessage: 'The ingestion operation failed.', + errorCode: 'source_fetch_failed', + ingestedAt: null, + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.FAILED, + errorCode: 'source_fetch_failed', + finishedAt: new Date(OCCURRED_AT), + }) + }) + + it.each([ + 'resource.subresources_updated', + 'kb.metrics_updated', + ] satisfies EventType[])('authenticates reserved %s events as successful no-ops', async (eventType) => { + const request = createRequest(event(eventType)) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.QUEUED, + }) + }) + + it('marks a succeeded replacement ready when a later serving event cuts over', async () => { + const succeeded = createRequest( + event('resource.processing_succeeded', { + serving: { + active_resource_version: RESOURCE_VERSION - 1, + active_sha256: 'a'.repeat(64), + }, + }) + ) + await handleKBIngestionWebhook({ + prisma, + ...succeeded, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + activeResourceVersion: RESOURCE_VERSION - 1, + activeContentSha256: 'a'.repeat(64), + }) + + const cutover = createRequest( + event('resource.subresources_updated', { + serving: { + active_resource_version: RESOURCE_VERSION, + active_sha256: CONTENT_SHA256, + }, + }) + ) + await handleKBIngestionWebhook({ + prisma, + ...cutover, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.READY, + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: CONTENT_SHA256, + ingestedAt: new Date(OCCURRED_AT), + }) + }) + + it('records a platform refresh and advances only the serving identity', async () => { + const candidateContentSha256 = 'c'.repeat(64) + const candidateOperationId = 'op_01L2X8K3M9QZ4R7T6V5W1Y0BND' + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { + status: KBResourceStatus.PROCESSING, + resourceVersion: RESOURCE_VERSION + 1, + contentSha256: candidateContentSha256, + externalOperationId: candidateOperationId, + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: CONTENT_SHA256, + ingestedAt: new Date('2026-07-11T14:04:52Z'), + }, + }) + await prisma.kBIngestionRun.update({ + where: { id: INGESTION_ATTEMPT_ID }, + data: { + status: KBIngestionStatus.PROCESSING, + resourceVersion: RESOURCE_VERSION + 1, + contentSha256: candidateContentSha256, + externalOperationId: candidateOperationId, + }, + }) + const request = createRequest( + event('resource.content_refreshed', { + eventId: REFRESH_EVENT_ID, + operation_id: REFRESH_OPERATION_ID, + serving: { + active_resource_version: RESOURCE_VERSION, + active_sha256: REFRESH_CONTENT_SHA256, + }, + correlation_id: 'platform-weekly-refresh', + }) + ) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + ingestionAttemptId: INGESTION_ATTEMPT_ID, + resourceVersion: RESOURCE_VERSION + 1, + contentSha256: candidateContentSha256, + externalOperationId: candidateOperationId, + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: REFRESH_CONTENT_SHA256, + ingestedAt: new Date(OCCURRED_AT), + }) + await expect(getRun()).resolves.toMatchObject({ + status: KBIngestionStatus.PROCESSING, + externalOperationId: candidateOperationId, + }) + await expect( + getRunByExternalOperation(REFRESH_OPERATION_ID) + ).resolves.toMatchObject({ + id: REFRESH_EVENT_ID, + operation: KBIngestionOperation.UPSERT, + status: KBIngestionStatus.SUCCEEDED, + resourceVersion: RESOURCE_VERSION, + contentSha256: REFRESH_CONTENT_SHA256, + finishedAt: new Date(OCCURRED_AT), + }) + }) + + it('deduplicates repeated platform refresh delivery by external operation', async () => { + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { + status: KBResourceStatus.READY, + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: CONTENT_SHA256, + ingestedAt: new Date('2026-07-11T14:04:52Z'), + }, + }) + const request = createRequest( + event('resource.content_refreshed', { + eventId: REFRESH_EVENT_ID, + operation_id: REFRESH_OPERATION_ID, + serving: { + active_resource_version: RESOURCE_VERSION, + active_sha256: REFRESH_CONTENT_SHA256, + }, + correlation_id: 'platform-weekly-refresh', + }) + ) + + await Promise.all( + [request, request].map((refreshRequest) => + handleKBIngestionWebhook({ + prisma, + ...refreshRequest, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ) + ) + + await expect( + prisma.kBIngestionRun.count({ + where: { + resourceId, + externalOperationId: REFRESH_OPERATION_ID, + }, + }) + ).resolves.toBe(1) + await expect(getResource()).resolves.toMatchObject({ + activeResourceVersion: RESOURCE_VERSION, + activeContentSha256: REFRESH_CONTENT_SHA256, + ingestedAt: new Date(OCCURRED_AT), + }) + }) + + it('records an older platform refresh as superseded without regressing serving', async () => { + const currentServingSha256 = 'd'.repeat(64) + await prisma.kBResource.update({ + where: { id: resourceId }, + data: { + status: KBResourceStatus.READY, + resourceVersion: RESOURCE_VERSION + 1, + activeResourceVersion: RESOURCE_VERSION + 1, + activeContentSha256: currentServingSha256, + ingestedAt: new Date('2026-07-13T14:04:52Z'), + }, + }) + const request = createRequest( + event('resource.content_refreshed', { + eventId: REFRESH_EVENT_ID, + operation_id: REFRESH_OPERATION_ID, + serving: { + active_resource_version: RESOURCE_VERSION, + active_sha256: REFRESH_CONTENT_SHA256, + }, + correlation_id: 'platform-weekly-refresh', + }) + ) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + + await expect(getResource()).resolves.toMatchObject({ + activeResourceVersion: RESOURCE_VERSION + 1, + activeContentSha256: currentServingSha256, + ingestedAt: new Date('2026-07-13T14:04:52Z'), + }) + await expect( + getRunByExternalOperation(REFRESH_OPERATION_ID) + ).resolves.toMatchObject({ + status: KBIngestionStatus.SUPERSEDED, + statusMessage: + 'The platform refresh was superseded by a newer serving revision.', + }) + }) + + it('accepts the previous secret during key rotation', async () => { + const request = createRequest(event('resource.processing_started'), { + secret: PREVIOUS_SECRET, + }) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { + KB_WEBHOOK_SECRET: SECRET, + KB_WEBHOOK_PREVIOUS_SECRET: PREVIOUS_SECRET, + }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.PROCESSING, + }) + }) + + it('rejects an invalid signature', async () => { + const request = createRequest(event('resource.processing_started')) + + await expect( + handleKBIngestionWebhook({ + prisma, + rawBody: request.rawBody, + headers: { + ...request.headers, + 'x-ingestion-signature': '0'.repeat(64), + }, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ + statusCode: 401, + body: { error: 'Unauthorized' }, + }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.QUEUED, + }) + }) + + it('rejects a stale timestamp', async () => { + const now = new Date('2026-07-26T18:00:00Z') + const request = createRequest(event('resource.processing_started'), { + timestamp: Math.floor(now.getTime() / 1000) - 301, + }) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + now: () => now, + }) + ).resolves.toEqual({ + statusCode: 401, + body: { error: 'Unauthorized' }, + }) + }) + + it('rejects header and payload envelope mismatches', async () => { + const request = createRequest(event('resource.processing_started')) + + await expect( + handleKBIngestionWebhook({ + prisma, + rawBody: request.rawBody, + headers: { + ...request.headers, + 'x-ingestion-event-id': OTHER_EVENT_ID, + }, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ + statusCode: 400, + body: { error: 'Invalid request' }, + }) + }) + + it('rejects non-canonical or extended payload bytes', async () => { + const payload = event('resource.processing_started', { + unexpected: true, + }) + const rawBody = Buffer.from(JSON.stringify(payload, null, 2)) + const headers = signKBIngestionWebhook({ + eventId: EVENT_ID, + eventType: 'resource.processing_started', + rawBody, + secret: SECRET, + timestamp: Math.floor(Date.now() / 1000), + }) + + await expect( + handleKBIngestionWebhook({ + prisma, + rawBody, + headers, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ + statusCode: 400, + body: { error: 'Invalid request' }, + }) + }) + + it('does not let a stale version or operation mutate the current attempt', async () => { + const request = createRequest( + event('resource.processing_succeeded', { + operation_id: 'op_stale', + resource_version: RESOURCE_VERSION - 1, + serving: { + active_resource_version: RESOURCE_VERSION - 1, + active_sha256: CONTENT_SHA256, + }, + }) + ) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: { KB_WEBHOOK_SECRET: SECRET }, + }) + ).resolves.toEqual({ statusCode: 200, body: { ok: true } }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.QUEUED, + ingestedAt: null, + }) + }) + + it('does not let a racing started event regress ready', async () => { + const started = createRequest(event('resource.processing_started')) + const succeeded = createRequest(event('resource.processing_succeeded')) + + await Promise.all([ + handleKBIngestionWebhook({ + prisma, + ...started, + env: { KB_WEBHOOK_SECRET: SECRET }, + }), + handleKBIngestionWebhook({ + prisma, + ...succeeded, + env: { KB_WEBHOOK_SECRET: SECRET }, + }), + ]) + + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.READY, + }) + }) + + it('returns 503 without revealing details when the current secret is missing', async () => { + const request = createRequest(event('resource.processing_started')) + + await expect( + handleKBIngestionWebhook({ + prisma, + ...request, + env: {}, + }) + ).resolves.toEqual({ + statusCode: 503, + body: { error: 'Service unavailable' }, + }) + await expect(getResource()).resolves.toMatchObject({ + status: KBResourceStatus.QUEUED, + }) + }) +}) diff --git a/packages/hatchet/package.json b/packages/hatchet/package.json index 1d9686961d..f9e2ff156e 100644 --- a/packages/hatchet/package.json +++ b/packages/hatchet/package.json @@ -8,9 +8,12 @@ "dist" ], "dependencies": { + "@azure/storage-blob": "12.25.0", "@hatchet-dev/typescript-sdk": "1.9.4", + "@klicker-uzh/knowledge-graph": "workspace:*", "@klicker-uzh/prisma": "workspace:*", - "@klicker-uzh/types": "workspace:*" + "@klicker-uzh/types": "workspace:*", + "@klicker-uzh/util": "workspace:*" }, "devDependencies": { "@parcel/watcher": "~2.4.1", @@ -24,7 +27,8 @@ "rollup": "~4.34.9", "rollup-plugin-copy": "~3.5.0", "tsx": "~4.19.4", - "typescript": "~6.0.3" + "typescript": "~6.0.3", + "vitest": "~3.2.4" }, "scripts": { "build": "run-s --npm-path pnpm build:ts", @@ -34,7 +38,9 @@ "dev": "run-p --npm-path pnpm dev:ts", "dev:infisical": "../../util/_run_with_infisical.sh --env dev pnpm run dev", "dev:offline": "pnpm run dev", - "dev:ts": "cross-env NODE_ENV=development rollup -c --watch" + "dev:ts": "cross-env NODE_ENV=development rollup -c --watch", + "test": "vitest run", + "test:watch": "vitest" }, "engines": { "node": "=24" diff --git a/packages/hatchet/src/client.ts b/packages/hatchet/src/client.ts index f8c113c9a8..04a57357bb 100644 --- a/packages/hatchet/src/client.ts +++ b/packages/hatchet/src/client.ts @@ -1,10 +1,24 @@ import { HatchetClient } from '@hatchet-dev/typescript-sdk' +import { HatchetLogger } from '@hatchet-dev/typescript-sdk/clients/hatchet-client/index.js' import type { LogLevel } from '@hatchet-dev/typescript-sdk/util/logger/logger.js' const globalForHatchet = global as unknown as { hatchetClient: HatchetClient } const validLogLevels = ['INFO', 'OFF', 'DEBUG', 'WARN', 'ERROR'] +function createHatchetLogger(context: string, logLevel?: LogLevel) { + const logger = new HatchetLogger(context, logLevel) as HatchetLogger & { + undefined: () => void + } + + // `tsx --watch` sends development control messages through worker threads. + // Hatchet 1.9.4 assumes every heartbeat message has a log-level `type`, so + // messages without that shape become a call to the logger's `undefined` key. + logger.undefined = () => undefined + + return logger +} + function setupClient() { const hatchet = HatchetClient.init({ token: process.env.HATCHET_CLIENT_TOKEN, @@ -22,6 +36,7 @@ function setupClient() { ) ? (process.env.HATCHET_LOG_LEVEL as LogLevel) : 'INFO', + logger: createHatchetLogger, }) return hatchet diff --git a/packages/hatchet/src/index.ts b/packages/hatchet/src/index.ts index 1ae70e486e..698a287023 100644 --- a/packages/hatchet/src/index.ts +++ b/packages/hatchet/src/index.ts @@ -1,11 +1,39 @@ -import { Priority, type HatchetClient } from '@hatchet-dev/typescript-sdk' +import { + ConcurrencyLimitStrategy, + Priority, + type Context, + type HatchetClient, +} from '@hatchet-dev/typescript-sdk' import { prisma } from '@klicker-uzh/prisma' -import type { HatchetHandlers } from '@klicker-uzh/types' +import type { + BuildKBGraphInput, + DeleteKBResourceInput, + HatchetHandlers, + IngestKBResourceInput, +} from '@klicker-uzh/types' import type EventEmitter from 'events' import type { PubSub } from 'graphql-yoga' import type { Redis } from 'ioredis' +import { + dispatchKBGraphBuild, + markKBGraphBuildDispatchFailed, + monitorActiveKBGraphBuilds, +} from './kbGraphIngestion.js' +import { + dispatchKBDeletion, + dispatchKBIngestion, + failKBIngestionDispatch, + monitorActiveKBIngestions, + retainFailedKBDeletionDispatch, +} from './kbIngestion.js' +import { maintainKBResources } from './kbMaintenance.js' export * from './client.js' +export * from './kbGraphIngestion.js' +export * from './kbGraphIngestionApi.js' +export * from './kbIngestion.js' +export * from './kbIngestionApi.js' +export * from './kbMaintenance.js' export type { HatchetHandlers } from '@klicker-uzh/types' @@ -17,6 +45,8 @@ export function prepareHatchetTasks({ redisAssessmentExec, redisCache, handlers, + getKBGraphTerminalResult, + settleKBGraphTerminalResult, }: { hatchet: HatchetClient pubSub: PubSub @@ -25,6 +55,13 @@ export function prepareHatchetTasks({ redisAssessmentExec: Redis redisCache?: Redis handlers: HatchetHandlers + getKBGraphTerminalResult: (runId: string) => Promise + settleKBGraphTerminalResult: (input: { + buildId: string + result: unknown + finishedAt: Date + allowLateSuccess?: boolean + }) => Promise<'SETTLED' | 'RELEASED' | 'NEEDS_HUMAN_REVIEW' | 'DUPLICATE'> }) { const globalContext = { hatchet, @@ -56,6 +93,80 @@ export function prepareHatchetTasks({ ctx.logger.info(`Audit log entry: ${info}`, args) }, }) + + const ingestKBResourceDefinition = { + name: 'ingest-kb-resource', + retries: 3, + fn: async ( + input: IngestKBResourceInput, + ctx: Context + ) => { + await ctx.logger.info('KB ingestion dispatch started', { + resourceId: input.resourceId, + kbId: input.kbId, + type: input.type, + }) + await dispatchKBIngestion(input, { + prisma, + logger: ctx.logger, + }) + return { success: true } + }, + onFailure: { + retries: 3, + fn: async (input: IngestKBResourceInput) => { + await failKBIngestionDispatch({ input, prisma }) + }, + }, + } + const ingestKBResource = hatchet.task(ingestKBResourceDefinition) + const deleteKBResourceDefinition = { + name: 'delete-kb-resource', + retries: 3, + fn: async ( + input: DeleteKBResourceInput, + ctx: Context + ) => { + await ctx.logger.info('KB deletion dispatch started', { + resourceId: input.resourceId, + kbId: input.kbId, + }) + await dispatchKBDeletion(input, { + prisma, + logger: ctx.logger, + }) + return { success: true } + }, + onFailure: { + retries: 3, + fn: async (input: DeleteKBResourceInput) => { + await retainFailedKBDeletionDispatch({ input, prisma }) + }, + }, + } + const deleteKBResource = hatchet.task(deleteKBResourceDefinition) + + const buildKBGraphDefinition = { + name: 'build-kb-knowledge-graph', + retries: 3, + fn: async (input: BuildKBGraphInput, ctx: Context) => { + await ctx.logger.info('KB graph build dispatch started', { + buildId: input.buildId, + }) + await dispatchKBGraphBuild(input, { + prisma, + logger: ctx.logger, + }) + return { success: true } + }, + onFailure: { + retries: 3, + fn: async (input: BuildKBGraphInput) => { + await markKBGraphBuildDispatchFailed(input, prisma) + }, + }, + } + const buildKBGraph = hatchet.task(buildKBGraphDefinition) // #endregion // ! ACTIVITY PUBLICATION TASKS @@ -274,6 +385,52 @@ export function prepareHatchetTasks({ }, }) + const monitorKBIngestions = hatchet.task({ + name: 'monitor-kb-ingestions', + onCrons: ['* * * * *'], + concurrency: { + expression: '"monitor-kb-ingestions"', + maxRuns: 1, + limitStrategy: ConcurrencyLimitStrategy.CANCEL_NEWEST, + }, + fn: async () => monitorActiveKBIngestions({ prisma }), + }) + + const monitorKBGraphBuilds = hatchet.task({ + name: 'monitor-kb-graph-builds', + onCrons: ['* * * * *'], + concurrency: { + expression: '"monitor-kb-graph-builds"', + maxRuns: 1, + limitStrategy: ConcurrencyLimitStrategy.CANCEL_NEWEST, + }, + fn: async (_, ctx) => + monitorActiveKBGraphBuilds({ + prisma, + logger: ctx.logger, + getTerminalResult: getKBGraphTerminalResult, + settleTerminalResult: settleKBGraphTerminalResult, + }), + }) + + const maintainKBResourcesTask = hatchet.task({ + name: 'maintain-kb-resources', + onCrons: ['*/15 * * * *'], + concurrency: { + expression: '"maintain-kb-resources"', + maxRuns: 1, + limitStrategy: ConcurrencyLimitStrategy.CANCEL_NEWEST, + }, + fn: async (_, ctx) => + maintainKBResources({ + prisma, + logger: ctx.logger, + enqueueKBGraphBuild: async (buildId) => { + await buildKBGraph.runNoWait({ buildId }) + }, + }), + }) + // ? temporarily paused workflow, since the functionality is currently not available and needs fixing const sendPushNotifications = hatchet.task({ name: 'send-push-notifications', @@ -302,6 +459,12 @@ export function prepareHatchetTasks({ endExpiredMicroLearning, aggregateLiveQuizBlockResultsStandard, aggregateLiveQuizBlockResultsAssessment, + ingestKBResource, + deleteKBResource, + buildKBGraph, + monitorKBIngestions, + monitorKBGraphBuilds, + maintainKBResources: maintainKBResourcesTask, createAuditLogEntry, } } diff --git a/packages/hatchet/src/kbGraphIngestion.ts b/packages/hatchet/src/kbGraphIngestion.ts new file mode 100644 index 0000000000..9097e9a673 --- /dev/null +++ b/packages/hatchet/src/kbGraphIngestion.ts @@ -0,0 +1,1416 @@ +import { getKnowledgeGraphName } from '@klicker-uzh/knowledge-graph' +import { + KBGraphBuildStatus, + KBGraphCostStatus, + type KBGraphBuildSource, + type Prisma, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import type { BuildKBGraphInput } from '@klicker-uzh/types' +import { + KB_GRAPH_BUILD_METADATA_KEY, + KB_GRAPH_KB_METADATA_KEY, + cancelExternalKBGraphRunBestEffort, + getExternalKBGraphClient, + getExternalKBGraphConfig, + getKBGraphArtifactBlobName, + getKBGraphOwnerContainerName, + getKBGraphQualityConfig, + getKBGraphSourceUrl, + getKBGraphTimeoutSeconds, + recoverExternalKBGraphRun, + type ExternalKBGraphClient, + type ExternalKBGraphPayload, + type KBGraphLogger, + type ExternalKBGraphRequestOptions, +} from './kbGraphIngestionApi.js' + +const KB_GRAPH_MONITOR_BATCH_SIZE = 32 +const KB_GRAPH_MONITOR_CONCURRENCY = 8 +const KB_GRAPH_MONITOR_INTERVAL_MS = 15 * 60 * 1000 +const KB_GRAPH_MONITOR_PROVIDER_TIMEOUT_MS = 10_000 +const activeProviderOperations = new Set>() + +class ProviderOperationLimitError extends Error { + constructor() { + super('KB graph provider operation concurrency limit reached') + this.name = 'ProviderOperationLimitError' + } +} + +export const KB_GRAPH_DISPATCH_AMBIGUOUS_CODE = 'KB_GRAPH_DISPATCH_AMBIGUOUS' +const KB_GRAPH_DISPATCH_FAILED_CODE = 'KB_GRAPH_DISPATCH_FAILED' +// A dispatch claim is written just before the provider call, so a claimed build +// with no external operation id is ambiguous for as long as the first attempt may +// still be inside that call. Within this window a duplicate task run must leave +// the build alone: asking the provider too early answers "no run yet", and acting +// on that answer would release the reservation while the real run is starting and +// goes on to spend. Only a claim older than the window is treated as abandoned. +const KB_GRAPH_DISPATCH_CLAIM_GRACE_MS = 15 * 60 * 1000 + +type KBGraphPrisma = Pick< + PrismaClient, + | '$queryRaw' + | '$transaction' + | 'kB' + | 'kBGraphBuild' + | 'kBGraphQuota' + | 'kBResource' +> + +type KBGraphDispatchRecord = { + id: string + kbId: string + sourceContentDigest: string + graphName: string + graphmlBlobName: string | null + qualityTier: Parameters[0] + createdAt: Date + kb: { + ownerId: string + knowledgeGraphEnabled: boolean + } + sources: Array< + Pick< + KBGraphBuildSource, + 'resourceId' | 'type' | 'sourceUrl' | 'blobName' | 'contentSha256' + > + > +} + +type KBGraphReservationRecord = { + estimatedCostMinorUnits: number | null + costCurrency: string | null + costPricingVersion: string | null + semesterKey: string | null + costStatus: KBGraphCostStatus | null + quotaId: string | null + quota: { + id: string + ownerId: string + semesterKey: string + currency: string + limitMinorUnits: number + reservedMinorUnits: number + } | null + kb: { ownerId: string } +} + +export type DispatchKBGraphDependencies = { + prisma: KBGraphPrisma + client?: ExternalKBGraphClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBGraphLogger + providerOperationTimeoutMs?: number + getSourceUrl?: ( + source: KBGraphDispatchRecord['sources'][number], + options: { ownerId: string; env: NodeJS.ProcessEnv; now: () => Date } + ) => string +} + +export type MonitorKBGraphBuildsDependencies = { + prisma: KBGraphPrisma + client?: ExternalKBGraphClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBGraphLogger + providerOperationTimeoutMs?: number + getTerminalResult?: ( + runId: string, + options?: ExternalKBGraphRequestOptions + ) => Promise + settleTerminalResult?: (input: { + buildId: string + result: unknown + finishedAt: Date + allowLateSuccess?: boolean + }) => Promise<'SETTLED' | 'RELEASED' | 'NEEDS_HUMAN_REVIEW' | 'DUPLICATE'> +} + +function graphIdentifiers(build: Pick) { + return { buildId: build.id, kbId: build.kbId } +} + +async function logInfoBestEffort( + logger: KBGraphLogger | undefined, + message: string, + identifiers: Record +): Promise { + try { + await logger?.info?.(message, identifiers) + } catch { + // External state has already been persisted; logging must not undo it. + } +} + +async function logErrorBestEffort( + logger: KBGraphLogger | undefined, + message: string, + identifiers: Record +): Promise { + try { + await logger?.error?.(message, identifiers) + } catch { + // Reconciliation must remain retryable when the logger is unavailable. + } +} + +function isActiveBuildStatus(status: KBGraphBuildStatus) { + return ( + status === KBGraphBuildStatus.QUEUED || + status === KBGraphBuildStatus.PROCESSING + ) +} + +async function releaseKBGraphReservationInTransaction( + prisma: Prisma.TransactionClient, + buildId: string, + // A held reservation normally only leaves RESERVED. Resolving an ambiguous + // dispatch also has to unwind a hold that was already parked for review, once + // the provider has confirmed that no run of that build id ever existed. + releasableCostStatuses: KBGraphCostStatus[] = [KBGraphCostStatus.RESERVED] +): Promise { + const build = await prisma.kBGraphBuild.findUnique({ + where: { id: buildId }, + select: { + quotaId: true, + estimatedCostMinorUnits: true, + costStatus: true, + }, + }) + if ( + !build || + build.costStatus === null || + !releasableCostStatuses.includes(build.costStatus) || + build.estimatedCostMinorUnits === null + ) { + return + } + + if (build.quotaId) { + await prisma.$queryRaw>` + SELECT "id" + FROM "public"."KBGraphQuota" + WHERE "id" = CAST(${build.quotaId} AS UUID) + FOR UPDATE + ` + } + const updated = await prisma.kBGraphBuild.updateMany({ + where: { + id: buildId, + costStatus: build.costStatus, + }, + data: { costStatus: KBGraphCostStatus.RELEASED }, + }) + if (updated.count !== 1 || !build.quotaId) return + + const quotaUpdated = await prisma.kBGraphQuota.updateMany({ + where: { + id: build.quotaId, + reservedMinorUnits: { gte: build.estimatedCostMinorUnits }, + }, + data: { + reservedMinorUnits: { decrement: build.estimatedCostMinorUnits }, + }, + }) + if (quotaUpdated.count !== 1) { + throw new Error('KB graph quota reservation could not be released') + } +} + +function getGraphMonitorBatchOffset(total: number, now: Date) { + if (total <= KB_GRAPH_MONITOR_BATCH_SIZE) { + return 0 + } + const runNumber = Math.floor(now.getTime() / KB_GRAPH_MONITOR_INTERVAL_MS) + const pageCount = Math.ceil(total / KB_GRAPH_MONITOR_BATCH_SIZE) + return (runNumber % pageCount) * KB_GRAPH_MONITOR_BATCH_SIZE +} + +async function withProviderTimeout( + operationFactory: (signal: AbortSignal) => Promise, + timeoutMilliseconds: number | undefined +): Promise { + if (activeProviderOperations.size >= KB_GRAPH_MONITOR_CONCURRENCY) { + throw new ProviderOperationLimitError() + } + + const abortController = new AbortController() + const operation = operationFactory(abortController.signal) + let trackedOperation: Promise + trackedOperation = operation.finally(() => { + activeProviderOperations.delete(trackedOperation) + }) + activeProviderOperations.add(trackedOperation) + + if (timeoutMilliseconds === undefined) { + return trackedOperation + } + + let timeout: ReturnType | undefined + let timedOut = false + const timeoutError = new Error('KB graph provider operation timed out') + try { + return await Promise.race([ + trackedOperation, + new Promise((_, reject) => { + timeout = setTimeout(() => { + timedOut = true + abortController.abort(timeoutError) + reject(timeoutError) + }, timeoutMilliseconds) + }), + ]) + } catch (error) { + if (!timedOut) { + throw error + } + // The transport must honor the signal before capacity can be reused. This + // keeps the worker-wide ceiling valid even when a request times out. + await trackedOperation.catch(() => undefined) + throw timeoutError + } finally { + if (timeout) clearTimeout(timeout) + } +} + +async function runWithConcurrency( + items: T[], + task: (item: T) => Promise +): Promise { + let nextIndex = 0 + const workerCount = Math.min(KB_GRAPH_MONITOR_CONCURRENCY, items.length) + await Promise.all( + Array.from({ length: workerCount }, async () => { + while (nextIndex < items.length) { + const item = items[nextIndex]! + nextIndex += 1 + await task(item) + } + }) + ) +} + +function isDispatchableBuild(build: { + id: string + status: KBGraphBuildStatus + externalOperationId: string | null + dispatchClaimedAt: Date | null + graphmlBlobName: string | null + estimatedCostMinorUnits: number | null + costCurrency: string | null + costPricingVersion: string | null + semesterKey: string | null + costStatus: KBGraphCostStatus | null + quotaId: string | null + quota: KBGraphReservationRecord['quota'] + kb: { + ownerId: string + deletedAt: Date | null + activeGraphBuildId: string | null + knowledgeGraphEnabled: boolean + } + sources: KBGraphDispatchRecord['sources'] +}) { + return ( + isActiveBuildStatus(build.status) && + build.externalOperationId === null && + build.dispatchClaimedAt === null && + hasCompleteKBGraphReservation(build) && + build.kb.deletedAt === null && + build.kb.activeGraphBuildId === build.id && + build.kb.knowledgeGraphEnabled && + build.sources.length > 0 && + build.graphmlBlobName !== null + ) +} + +function hasCompleteKBGraphReservation( + build: Pick< + KBGraphReservationRecord, + | 'estimatedCostMinorUnits' + | 'costCurrency' + | 'costPricingVersion' + | 'semesterKey' + | 'costStatus' + | 'quotaId' + | 'quota' + | 'kb' + > +): boolean { + return ( + build.costStatus === KBGraphCostStatus.RESERVED && + build.estimatedCostMinorUnits !== null && + build.estimatedCostMinorUnits > 0 && + build.costCurrency !== null && + build.costCurrency.length > 0 && + build.costPricingVersion !== null && + build.costPricingVersion.length > 0 && + build.semesterKey !== null && + build.semesterKey.length > 0 && + build.quotaId !== null && + build.quota !== null && + build.quota.id === build.quotaId && + build.quota.ownerId === build.kb.ownerId && + build.quota.semesterKey === build.semesterKey && + build.quota.currency === build.costCurrency && + build.quota.limitMinorUnits > 0 && + build.quota.reservedMinorUnits >= build.estimatedCostMinorUnits + ) +} + +function isUnstartedActiveBuild(build: { + id: string + status: KBGraphBuildStatus + externalOperationId: string | null + dispatchClaimedAt: Date | null + kb: { deletedAt: Date | null; activeGraphBuildId: string | null } +}) { + return ( + isActiveBuildStatus(build.status) && + build.externalOperationId === null && + build.dispatchClaimedAt === null && + build.kb.deletedAt === null && + build.kb.activeGraphBuildId === build.id + ) +} + +function getDispatchGateFailure( + build: KBGraphReservationRecord & { + kb: KBGraphReservationRecord['kb'] & { knowledgeGraphEnabled: boolean } + }, + env: NodeJS.ProcessEnv +): { statusMessage: string; errorCode: string } | null { + if (env.KB_GRAPH_DISABLED === 'true') { + return { + statusMessage: 'KB graph generation is currently disabled.', + errorCode: 'KB_GRAPH_DISABLED', + } + } + if (!build.kb.knowledgeGraphEnabled) { + return { + statusMessage: 'KB graph generation is not enabled for this KB.', + errorCode: 'KB_GRAPH_NOT_ENABLED', + } + } + if (!hasCompleteKBGraphReservation(build)) { + return { + statusMessage: + 'The KB graph build has no complete cost reservation and requires review.', + errorCode: 'KB_GRAPH_RESERVATION_INCOMPLETE', + } + } + return null +} + +async function failKBGraphBuildBeforeDispatch( + prisma: KBGraphPrisma, + { + buildId, + kbId, + statusMessage, + errorCode, + }: { + buildId: string + kbId: string + statusMessage: string + errorCode: string + }, + finishedAt: Date +): Promise { + await prisma.$transaction(async (tx) => { + const current = await tx.kBGraphBuild.findUnique({ + where: { id: buildId }, + select: { + externalOperationId: true, + dispatchClaimedAt: true, + costStatus: true, + status: true, + kb: { + select: { deletedAt: true, activeGraphBuildId: true }, + }, + }, + }) + if ( + !current || + !isUnstartedActiveBuild({ + id: buildId, + status: current.status, + externalOperationId: current.externalOperationId, + dispatchClaimedAt: current.dispatchClaimedAt, + kb: current.kb, + }) + ) { + return + } + + const failed = await tx.kBGraphBuild.updateMany({ + where: { + id: buildId, + kbId, + externalOperationId: null, + dispatchClaimedAt: null, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + }, + data: { + status: KBGraphBuildStatus.FAILED, + statusMessage, + errorCode, + finishedAt, + }, + }) + if (failed.count !== 1) return + + if ( + current.costStatus === KBGraphCostStatus.RESERVED && + errorCode !== 'KB_GRAPH_RESERVATION_INCOMPLETE' + ) { + await releaseKBGraphReservationInTransaction(tx, buildId) + } else if ( + current.costStatus === null || + (current.costStatus === KBGraphCostStatus.RESERVED && + errorCode === 'KB_GRAPH_RESERVATION_INCOMPLETE') + ) { + await tx.kBGraphBuild.updateMany({ + where: { id: buildId, costStatus: current.costStatus }, + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + } + await tx.kB.updateMany({ + where: { id: kbId, activeGraphBuildId: buildId }, + data: { activeGraphBuildId: null }, + }) + }) +} + +export function buildExternalKBGraphPayload( + build: KBGraphDispatchRecord, + sourceUrls: string[], + env: NodeJS.ProcessEnv = process.env +): ExternalKBGraphPayload { + if (sourceUrls.length !== build.sources.length) { + throw new Error('KB graph source URL count does not match') + } + if (build.graphmlBlobName !== getKBGraphArtifactBlobName(build.id)) { + throw new Error('KB graph artifact path is invalid') + } + + const quality = getKBGraphQualityConfig(build.qualityTier, env) + return { + course_id: build.id, + storage_name: build.id, + sources: build.sources.map((source, index) => ({ + source_id: source.resourceId, + source_url: sourceUrls[index]!, + expected_content_sha256: source.contentSha256, + })), + upload_markdown: false, + export_to_falkordb: true, + falkordb_graph_name: build.graphName, + speed_mode: quality.speedMode, + generation_model: quality.generationModel, + cleaning_model: quality.cleaningModel, + klicker_graph_build: { + build_id: build.id, + kb_id: build.kbId, + owner_id: build.kb.ownerId, + source_content_digest: build.sourceContentDigest, + graphml_container_name: getKBGraphOwnerContainerName(build.kb.ownerId), + graphml_blob_name: build.graphmlBlobName, + }, + } +} + +function validateBuildIdentity(build: KBGraphDispatchRecord): void { + if (build.graphmlBlobName !== getKBGraphArtifactBlobName(build.id)) { + throw new Error('KB graph artifact path is invalid') + } + if ( + build.graphName !== getKnowledgeGraphName(build.kbId, build.id) || + build.sources.some( + (source) => + !source.contentSha256 || + (source.type === 'BLOB' && !source.blobName) || + (source.type === 'URL' && !source.sourceUrl) + ) + ) { + throw new Error('KB graph build snapshot is invalid') + } +} + +/** + * Outcome of asking the provider whether an accepted-but-uncorrelated dispatch + * actually produced a run. + * + * - `CORRELATED`: the run exists and the build now carries its id again. + * - `RELEASED`: the provider has no such run, so nothing external is spending; + * the reservation is released and the KB build slot is freed for a rebuild. + * - `HELD`: the provider could not be asked, so a run may still be spending and + * both the reservation and the build slot stay fenced until the next attempt. + */ +export type KBGraphDispatchAmbiguityResolution = + | 'CORRELATED' + | 'RELEASED' + | 'HELD' + +/** + * Resolves a build whose dispatch was claimed but never correlated. The provider + * lookup decides: only a definitive "no run for this build id" may unwind the + * hold, because releasing a reservation for a run that is still generating would + * both under-charge the lecturer's quota and allow a second external run. + */ +export async function resolveAmbiguousKBGraphDispatch( + build: { id: string; kbId: string; createdAt: Date }, + dependencies: Pick< + DispatchKBGraphDependencies, + | 'prisma' + | 'client' + | 'env' + | 'logger' + | 'now' + | 'providerOperationTimeoutMs' + > +): Promise { + const env = dependencies.env ?? process.env + const now = dependencies.now ?? (() => new Date()) + const identifiers = graphIdentifiers(build) + + let recoveredRun: Awaited> + try { + const config = getExternalKBGraphConfig(env) + const client = dependencies.client ?? getExternalKBGraphClient(env) + recoveredRun = await withProviderTimeout( + (signal) => + recoverExternalKBGraphRun({ + client, + workflowName: config.workflowName, + additionalMetadata: { + [KB_GRAPH_BUILD_METADATA_KEY]: build.id, + [KB_GRAPH_KB_METADATA_KEY]: build.kbId, + }, + recoveryAnchor: build.createdAt, + requestOptions: { signal }, + }), + dependencies.providerOperationTimeoutMs + ) + } catch { + await logErrorBestEffort( + dependencies.logger, + 'KB graph dispatch ambiguity could not be resolved against the provider', + identifiers + ) + return 'HELD' + } + + if (recoveredRun) { + const correlated = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId: null, + dispatchClaimedAt: { not: null }, + kb: { deletedAt: null, activeGraphBuildId: build.id }, + }, + data: { + externalOperationId: recoveredRun.runId, + dispatchClaimedAt: null, + externalStartedAt: recoveredRun.startedAt, + startedAt: recoveredRun.startedAt, + status: KBGraphBuildStatus.PROCESSING, + statusMessage: null, + errorCode: null, + finishedAt: null, + }, + }) + if (correlated.count !== 1) { + return 'HELD' + } + await logInfoBestEffort( + dependencies.logger, + 'KB graph dispatch ambiguity resolved by correlating the external run', + identifiers + ) + return 'CORRELATED' + } + + await dependencies.prisma.$transaction(async (tx) => { + const failed = await tx.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId: null, + // Second money fence: re-checked at write time so a claim refreshed by a + // concurrent dispatch between the provider lookup and this update keeps + // its reservation instead of being released underneath a starting run. + dispatchClaimedAt: { + lte: new Date(now().getTime() - KB_GRAPH_DISPATCH_CLAIM_GRACE_MS), + }, + }, + data: { + status: KBGraphBuildStatus.FAILED, + statusMessage: 'The external KB graph workflow could not be started.', + errorCode: KB_GRAPH_DISPATCH_FAILED_CODE, + finishedAt: new Date(), + }, + }) + if (failed.count !== 1) { + return + } + await releaseKBGraphReservationInTransaction(tx, build.id, [ + KBGraphCostStatus.RESERVED, + KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + ]) + await tx.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + }) + await logInfoBestEffort( + dependencies.logger, + 'KB graph dispatch ambiguity resolved: the provider has no run for this build', + identifiers + ) + return 'RELEASED' +} + +export async function dispatchKBGraphBuild( + input: BuildKBGraphInput, + dependencies: DispatchKBGraphDependencies +): Promise { + const env = dependencies.env ?? process.env + const now = dependencies.now ?? (() => new Date()) + const build = await dependencies.prisma.kBGraphBuild.findUnique({ + where: { id: input.buildId }, + select: { + id: true, + kbId: true, + sourceContentDigest: true, + graphName: true, + graphmlBlobName: true, + qualityTier: true, + status: true, + externalOperationId: true, + dispatchClaimedAt: true, + estimatedCostMinorUnits: true, + costCurrency: true, + costPricingVersion: true, + semesterKey: true, + costStatus: true, + quotaId: true, + quota: { + select: { + id: true, + ownerId: true, + semesterKey: true, + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + }, + }, + createdAt: true, + kb: { + select: { + ownerId: true, + deletedAt: true, + activeGraphBuildId: true, + knowledgeGraphEnabled: true, + }, + }, + sources: { + select: { + resourceId: true, + type: true, + sourceUrl: true, + blobName: true, + contentSha256: true, + }, + orderBy: { resourceId: 'asc' }, + }, + }, + }) + if (!build) { + return undefined + } + if (build.dispatchClaimedAt !== null && build.externalOperationId === null) { + if ( + now().getTime() - build.dispatchClaimedAt.getTime() < + KB_GRAPH_DISPATCH_CLAIM_GRACE_MS + ) { + // A sibling task run for the same build is probably still inside the + // provider call. Leave its claim, its reservation, and its status exactly + // as they are; the graph monitor revisits the build once the claim ages + // past the grace. + await logInfoBestEffort( + dependencies.logger, + 'KB graph dispatch is already claimed and may still be in flight', + graphIdentifiers(build) + ) + return undefined + } + // Ask the provider before parking the build for review: the earlier attempt + // may have produced a run that simply lost its id, and an unresolvable hold + // fences the KB slot and the lecturer's quota with no way out. + const resolution = await resolveAmbiguousKBGraphDispatch( + build, + dependencies + ) + if (resolution === 'HELD') { + await markKBGraphBuildDispatchFailed(input, dependencies.prisma) + } + return undefined + } + if (isUnstartedActiveBuild(build)) { + const gateFailure = getDispatchGateFailure(build, env) + if (gateFailure) { + await failKBGraphBuildBeforeDispatch( + dependencies.prisma, + { + buildId: build.id, + kbId: build.kbId, + ...gateFailure, + }, + now() + ) + return undefined + } + } + if (!isDispatchableBuild(build)) return undefined + + const dispatchBuild: KBGraphDispatchRecord = { + id: build.id, + kbId: build.kbId, + sourceContentDigest: build.sourceContentDigest, + graphName: build.graphName, + graphmlBlobName: build.graphmlBlobName, + qualityTier: build.qualityTier, + createdAt: build.createdAt, + kb: { + ownerId: build.kb.ownerId, + knowledgeGraphEnabled: build.kb.knowledgeGraphEnabled, + }, + sources: build.sources, + } + const identifiers = graphIdentifiers(dispatchBuild) + + try { + validateBuildIdentity(dispatchBuild) + const config = getExternalKBGraphConfig(env) + const client = dependencies.client ?? getExternalKBGraphClient(env) + const additionalMetadata = { + [KB_GRAPH_BUILD_METADATA_KEY]: dispatchBuild.id, + [KB_GRAPH_KB_METADATA_KEY]: dispatchBuild.kbId, + } + const recoveredRun = await recoverExternalKBGraphRun({ + client, + workflowName: config.workflowName, + additionalMetadata, + recoveryAnchor: dispatchBuild.createdAt, + }) + + let runId: string + let startedAt: Date + if (recoveredRun) { + runId = recoveredRun.runId + startedAt = recoveredRun.startedAt + const claimed = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: dispatchBuild.id, + kbId: dispatchBuild.kbId, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + externalOperationId: null, + dispatchClaimedAt: null, + kb: { + deletedAt: null, + activeGraphBuildId: dispatchBuild.id, + }, + }, + data: { dispatchClaimedAt: startedAt }, + }) + if (claimed.count !== 1) { + return undefined + } + } else { + const current = await dependencies.prisma.kBGraphBuild.findUnique({ + where: { id: dispatchBuild.id }, + select: { + id: true, + status: true, + externalOperationId: true, + dispatchClaimedAt: true, + estimatedCostMinorUnits: true, + costCurrency: true, + costPricingVersion: true, + semesterKey: true, + costStatus: true, + quotaId: true, + quota: { + select: { + id: true, + ownerId: true, + semesterKey: true, + currency: true, + limitMinorUnits: true, + reservedMinorUnits: true, + }, + }, + kb: { + select: { + ownerId: true, + deletedAt: true, + activeGraphBuildId: true, + knowledgeGraphEnabled: true, + }, + }, + }, + }) + const gateFailure = current ? getDispatchGateFailure(current, env) : null + if (!current || !isUnstartedActiveBuild(current) || gateFailure) { + if (current && isUnstartedActiveBuild(current) && gateFailure) { + await failKBGraphBuildBeforeDispatch( + dependencies.prisma, + { + buildId: current.id, + kbId: dispatchBuild.kbId, + ...gateFailure, + }, + now() + ) + } + return undefined + } + const getSourceUrl = dependencies.getSourceUrl ?? getKBGraphSourceUrl + const sourceUrls = dispatchBuild.sources.map((source) => + getSourceUrl(source, { + ownerId: dispatchBuild.kb.ownerId, + env, + now, + }) + ) + const payload = buildExternalKBGraphPayload( + dispatchBuild, + sourceUrls, + env + ) + startedAt = now() + const claimed = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: dispatchBuild.id, + kbId: dispatchBuild.kbId, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + externalOperationId: null, + dispatchClaimedAt: null, + kb: { + deletedAt: null, + activeGraphBuildId: dispatchBuild.id, + }, + }, + data: { dispatchClaimedAt: startedAt }, + }) + if (claimed.count !== 1) { + return undefined + } + const run = await client.runNoWait(config.workflowName, payload, { + additionalMetadata, + }) + runId = await run.getWorkflowRunId() + } + + const persisted = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: dispatchBuild.id, + kbId: dispatchBuild.kbId, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + externalOperationId: null, + dispatchClaimedAt: startedAt, + kb: { + deletedAt: null, + activeGraphBuildId: dispatchBuild.id, + }, + }, + data: { + externalOperationId: runId, + dispatchClaimedAt: null, + externalStartedAt: startedAt, + startedAt, + statusMessage: null, + errorCode: null, + }, + }) + if (persisted.count === 1) { + await logInfoBestEffort( + dependencies.logger, + 'External KB graph build dispatched', + identifiers + ) + return runId + } + + const current = await dependencies.prisma.kBGraphBuild.findUnique({ + where: { id: dispatchBuild.id }, + select: { externalOperationId: true }, + }) + if (current?.externalOperationId === runId) { + return runId + } + + await cancelExternalKBGraphRunBestEffort({ + client, + runId, + identifiers, + logger: dependencies.logger, + }) + return undefined + } catch { + await logErrorBestEffort( + dependencies.logger, + 'External KB graph build dispatch failed', + identifiers + ) + throw new Error('External KB graph build dispatch failed') + } +} + +async function finishKBGraphBuild({ + build, + status, + statusMessage, + errorCode, + prisma, + finishedAt, +}: { + build: { + id: string + kbId: string + externalOperationId: string + } + status: typeof KBGraphBuildStatus.SUCCEEDED | typeof KBGraphBuildStatus.FAILED + statusMessage: string | null + errorCode: string | null + prisma: KBGraphPrisma + finishedAt: Date +}) { + await prisma.$transaction(async (tx) => { + const updated = await tx.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId: build.externalOperationId, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + }, + data: { + status, + statusMessage, + errorCode, + finishedAt, + }, + }) + if (updated.count !== 1) { + return + } + + await tx.kBGraphBuild.updateMany({ + where: { + id: build.id, + costStatus: { + in: [ + KBGraphCostStatus.RESERVED, + KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + ], + }, + }, + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + + await tx.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + }) +} + +type TimedOutKBGraphBuild = { + id: string + kbId: string + sourceContentDigest: string + createdAt: Date + externalOperationId: string +} + +async function recordIneligibleLateSuccess( + build: TimedOutKBGraphBuild, + prisma: Pick, + status: + | typeof KBGraphBuildStatus.FAILED + | typeof KBGraphBuildStatus.SUPERSEDED, + statusMessage: string, + errorCode: string +): Promise { + await prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId: build.externalOperationId, + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + cleanedAt: null, + cleanupStartedAt: null, + }, + data: { status, statusMessage, errorCode }, + }) +} + +async function monitorTimedOutKBGraphBuilds( + builds: TimedOutKBGraphBuild[], + dependencies: MonitorKBGraphBuildsDependencies, + client: ExternalKBGraphClient, + now: () => Date, + providerOperationTimeoutMs: number +): Promise { + await runWithConcurrency(builds, async (build) => { + const identifiers = graphIdentifiers(build) + try { + const externalStatus = await withProviderTimeout( + (signal) => + client.runs.get_status(build.externalOperationId, { signal }), + providerOperationTimeoutMs + ) + if (externalStatus !== 'COMPLETED') { + return + } + + if (dependencies.getTerminalResult && dependencies.settleTerminalResult) { + const terminalResult = await withProviderTimeout( + (signal) => + dependencies.getTerminalResult!(build.externalOperationId, { + signal, + }), + providerOperationTimeoutMs + ) + await dependencies.settleTerminalResult({ + buildId: build.id, + result: terminalResult, + finishedAt: now(), + allowLateSuccess: true, + }) + } else { + await recordIneligibleLateSuccess( + build, + dependencies.prisma, + KBGraphBuildStatus.FAILED, + 'The external workflow completed after timeout without a versioned terminal result.', + 'KB_GRAPH_RESULT_REQUIRED' + ) + } + } catch { + await logErrorBestEffort( + dependencies.logger, + 'Timed-out external KB graph build monitor failed', + identifiers + ) + } + }) +} + +export async function monitorActiveKBGraphBuilds( + dependencies: MonitorKBGraphBuildsDependencies +): Promise { + const env = dependencies.env ?? process.env + const now = dependencies.now ?? (() => new Date()) + const timeoutMilliseconds = getKBGraphTimeoutSeconds(env) * 1000 + const providerOperationTimeoutMs = + dependencies.providerOperationTimeoutMs ?? + KB_GRAPH_MONITOR_PROVIDER_TIMEOUT_MS + const sweepNow = now() + const activeWhere = { + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + externalOperationId: { not: null }, + externalStartedAt: { not: null }, + } satisfies Prisma.KBGraphBuildWhereInput + const activeCount = await dependencies.prisma.kBGraphBuild.count({ + where: activeWhere, + }) + const activeOffset = getGraphMonitorBatchOffset(activeCount, sweepNow) + const builds = await dependencies.prisma.kBGraphBuild.findMany({ + where: activeWhere, + select: { + id: true, + kbId: true, + externalOperationId: true, + externalStartedAt: true, + }, + orderBy: { createdAt: 'asc' }, + ...(activeOffset > 0 ? { skip: activeOffset } : {}), + take: KB_GRAPH_MONITOR_BATCH_SIZE, + }) + const timedOutWhere = { + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + externalOperationId: { not: null }, + cleanedAt: null, + cleanupStartedAt: null, + } satisfies Prisma.KBGraphBuildWhereInput + const timedOutCount = await dependencies.prisma.kBGraphBuild.count({ + where: timedOutWhere, + }) + const timedOutOffset = getGraphMonitorBatchOffset(timedOutCount, sweepNow) + const timedOutBuilds = await dependencies.prisma.kBGraphBuild.findMany({ + where: timedOutWhere, + select: { + id: true, + kbId: true, + sourceContentDigest: true, + createdAt: true, + externalOperationId: true, + }, + orderBy: { createdAt: 'asc' }, + ...(timedOutOffset > 0 ? { skip: timedOutOffset } : {}), + take: KB_GRAPH_MONITOR_BATCH_SIZE, + }) + // Runs ahead of the early return: a build parked on an ambiguous dispatch has + // neither a correlated run nor a timeout, so no other sweep would revisit it. + await recheckAmbiguousKBGraphDispatches( + { ...dependencies, providerOperationTimeoutMs }, + sweepNow + ) + + if (builds.length === 0 && timedOutBuilds.length === 0) { + return + } + + const client = dependencies.client ?? getExternalKBGraphClient(env) + await runWithConcurrency(builds, async (build) => { + if (!build.externalOperationId || !build.externalStartedAt) { + return + } + const externalOperationId = build.externalOperationId + const identifiers = graphIdentifiers(build) + try { + const externalStatus = await withProviderTimeout( + (signal) => client.runs.get_status(externalOperationId, { signal }), + providerOperationTimeoutMs + ) + const observedAt = now() + if ( + externalStatus === 'COMPLETED' || + externalStatus === 'FAILED' || + externalStatus === 'CANCELLED' + ) { + if ( + !dependencies.getTerminalResult || + !dependencies.settleTerminalResult + ) { + const statusMessage = + externalStatus === 'COMPLETED' + ? 'The external workflow completed without a versioned terminal result.' + : `The external workflow ended with ${externalStatus.toLowerCase()} without a versioned terminal result.` + await finishKBGraphBuild({ + build: { + ...build, + externalOperationId, + }, + status: KBGraphBuildStatus.FAILED, + statusMessage, + errorCode: 'KB_GRAPH_RESULT_REQUIRED', + prisma: dependencies.prisma, + finishedAt: observedAt, + }) + return + } + + const terminalResult = await withProviderTimeout( + (signal) => + dependencies.getTerminalResult!(externalOperationId, { signal }), + providerOperationTimeoutMs + ) + await dependencies.settleTerminalResult({ + buildId: build.id, + result: terminalResult, + finishedAt: observedAt, + }) + return + } + + if ( + observedAt.getTime() - build.externalStartedAt.getTime() > + timeoutMilliseconds + ) { + await withProviderTimeout( + (signal) => + cancelExternalKBGraphRunBestEffort({ + client, + runId: externalOperationId, + identifiers, + logger: dependencies.logger, + requestOptions: { signal }, + }), + providerOperationTimeoutMs + ) + await finishKBGraphBuild({ + build: { + ...build, + externalOperationId, + }, + status: KBGraphBuildStatus.FAILED, + statusMessage: 'External KB graph workflow timed out.', + errorCode: 'KB_GRAPH_TIMEOUT', + prisma: dependencies.prisma, + finishedAt: observedAt, + }) + return + } + + if (externalStatus === 'RUNNING') { + await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId, + status: KBGraphBuildStatus.QUEUED, + }, + data: { + status: KBGraphBuildStatus.PROCESSING, + statusMessage: null, + }, + }) + } + } catch { + await logErrorBestEffort( + dependencies.logger, + 'External KB graph build monitor failed', + identifiers + ) + } + }) + await monitorTimedOutKBGraphBuilds( + timedOutBuilds.flatMap((build) => + build.externalOperationId === null + ? [] + : [{ ...build, externalOperationId: build.externalOperationId }] + ), + dependencies, + client, + now, + providerOperationTimeoutMs + ) +} + +/** + * Retries the provider lookup for every build still parked on an ambiguous + * dispatch. A provider outage is transient, so the hold is a waiting state rather + * than a permanent one: each sweep either correlates the run or, once the + * provider is reachable and reports no run, releases the reservation and frees + * the KB build slot so the lecturer can rebuild without an operator. + */ +export async function recheckAmbiguousKBGraphDispatches( + dependencies: Pick< + MonitorKBGraphBuildsDependencies, + 'prisma' | 'client' | 'env' | 'logger' | 'providerOperationTimeoutMs' + >, + sweepNow: Date +): Promise { + const ambiguousWhere = { + errorCode: KB_GRAPH_DISPATCH_AMBIGUOUS_CODE, + externalOperationId: null, + // Only claims older than the in-flight grace are candidates: a fresher claim + // may belong to a dispatch that is still inside the provider call. + dispatchClaimedAt: { + lte: new Date(sweepNow.getTime() - KB_GRAPH_DISPATCH_CLAIM_GRACE_MS), + }, + } satisfies Prisma.KBGraphBuildWhereInput + const ambiguousCount = await dependencies.prisma.kBGraphBuild.count({ + where: ambiguousWhere, + }) + const ambiguousOffset = getGraphMonitorBatchOffset(ambiguousCount, sweepNow) + const ambiguousBuilds = await dependencies.prisma.kBGraphBuild.findMany({ + where: ambiguousWhere, + select: { id: true, kbId: true, createdAt: true }, + orderBy: { createdAt: 'asc' }, + ...(ambiguousOffset > 0 ? { skip: ambiguousOffset } : {}), + take: KB_GRAPH_MONITOR_BATCH_SIZE, + }) + await runWithConcurrency(ambiguousBuilds, async (build) => { + await resolveAmbiguousKBGraphDispatch(build, { + ...dependencies, + now: () => sweepNow, + }) + }) +} + +export async function markKBGraphBuildDispatchFailed( + input: BuildKBGraphInput, + prisma: KBGraphPrisma +): Promise { + const finishedAt = new Date() + await prisma.$transaction(async (tx) => { + const build = await tx.kBGraphBuild.findUnique({ + where: { id: input.buildId }, + select: { + id: true, + kbId: true, + dispatchClaimedAt: true, + costStatus: true, + }, + }) + if (!build) return + + const failed = await tx.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + externalOperationId: null, + dispatchClaimedAt: + build.dispatchClaimedAt === null ? null : { not: null }, + status: { + in: [KBGraphBuildStatus.QUEUED, KBGraphBuildStatus.PROCESSING], + }, + }, + data: { + status: KBGraphBuildStatus.FAILED, + statusMessage: + build.dispatchClaimedAt === null + ? 'The external KB graph workflow could not be started.' + : 'The external KB graph workflow may have been accepted but could not be correlated; manual review is required.', + errorCode: + build.dispatchClaimedAt === null + ? 'KB_GRAPH_DISPATCH_FAILED' + : 'KB_GRAPH_DISPATCH_AMBIGUOUS', + finishedAt, + }, + }) + if (failed.count === 1) { + if ( + build.dispatchClaimedAt === null && + build.costStatus === KBGraphCostStatus.RESERVED + ) { + await releaseKBGraphReservationInTransaction(tx, build.id) + } else if ( + build.costStatus === null || + build.costStatus === KBGraphCostStatus.RESERVED + ) { + await tx.kBGraphBuild.updateMany({ + where: { id: build.id, costStatus: build.costStatus }, + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + } + if (build.dispatchClaimedAt === null) { + await tx.kB.updateMany({ + where: { id: build.kbId, activeGraphBuildId: build.id }, + data: { activeGraphBuildId: null }, + }) + } + } + }) +} diff --git a/packages/hatchet/src/kbGraphIngestionApi.ts b/packages/hatchet/src/kbGraphIngestionApi.ts new file mode 100644 index 0000000000..a4c07f1b0c --- /dev/null +++ b/packages/hatchet/src/kbGraphIngestionApi.ts @@ -0,0 +1,543 @@ +import { + BlobSASPermissions, + BlobServiceClient, + generateBlobSASQueryParameters, + SASProtocol, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import { HatchetClient } from '@hatchet-dev/typescript-sdk' +import { isIP } from 'node:net' +import { getKnowledgeGraphConfig } from '@klicker-uzh/knowledge-graph' +import { + KBGraphQualityTier, + KBResourceType, + type KBGraphBuildSource, +} from '@klicker-uzh/prisma/client' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' + +const DEFAULT_KB_GRAPH_TIMEOUT_SECONDS = 6 * 60 * 60 +const KB_GRAPH_BLOB_SAS_CLOCK_SKEW_MS = 5 * 60 * 1000 +const KB_GRAPH_ARTIFACT_PREFIX = 'knowledge-graphs' +// Only chart-owned (ConfigMap) keys may arm the all-or-nothing startup gate. +// `KB_GRAPH_HATCHET_CLIENT_TOKEN` lives in the out-of-repo general-worker secret, +// so listing it would let a secret rollout on its own halt every unrelated job on +// that worker before the chart values completing the configuration have landed. +// Once one of these keys is set the token is still required. +const KB_GRAPH_CONFIGURATION_ENVIRONMENT_VARIABLES = [ + 'KB_GRAPH_HATCHET_CLIENT_HOST_PORT', + 'KB_GRAPH_HATCHET_API_URL', + 'KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY', + 'KB_GRAPH_HATCHET_WORKFLOW_NAME', + 'KB_GRAPH_TIMEOUT_SECONDS', + 'KB_GRAPH_STANDARD_GENERATION_MODEL', + 'KB_GRAPH_STANDARD_CLEANING_MODEL', + 'KB_GRAPH_HIGH_GENERATION_MODEL', + 'KB_GRAPH_HIGH_CLEANING_MODEL', +] as const + +export const KB_GRAPH_BUILD_METADATA_KEY = 'klickerKBGraphBuildId' +export const KB_GRAPH_KB_METADATA_KEY = 'klickerKBGraphKbId' + +function isLoopbackHost(hostname: string): boolean { + return ( + hostname === 'localhost' || + hostname.endsWith('.localhost') || + (isIP(hostname) === 4 && hostname.startsWith('127.')) || + hostname === '[::1]' + ) +} + +type ExternalHatchetTLSStrategy = 'tls' | 'mtls' | 'none' + +export type ExternalKBGraphPayload = { + course_id: string + storage_name: string + sources: Array<{ + source_id: string + source_url: string + expected_content_sha256: string + }> + upload_markdown: false + export_to_falkordb: true + falkordb_graph_name: string + speed_mode: 'balanced' | 'quality' + generation_model: string + cleaning_model: string + klicker_graph_build: { + build_id: string + kb_id: string + owner_id: string + source_content_digest: string + graphml_container_name: string + graphml_blob_name: string + } +} + +export type ExternalKBGraphStatus = + | 'QUEUED' + | 'RUNNING' + | 'COMPLETED' + | 'FAILED' + | 'CANCELLED' + +export type ExternalKBGraphRequestOptions = { + signal?: AbortSignal +} + +export type ExternalKBGraphClient = { + runNoWait: ( + workflowName: string, + input: ExternalKBGraphPayload, + options: { additionalMetadata: Record } + ) => Promise<{ getWorkflowRunId: () => Promise }> + runs: { + get: ( + runId: string, + options?: ExternalKBGraphRequestOptions + ) => Promise<{ + run: { output: unknown } + }> + get_status: ( + runId: string, + options?: ExternalKBGraphRequestOptions + ) => Promise + list: ( + options: { + workflowNames: string[] + additionalMetadata: Record + onlyTasks: boolean + includePayloads: boolean + limit: number + since: Date + }, + requestOptions?: ExternalKBGraphRequestOptions + ) => Promise<{ + rows: Array<{ + workflowRunExternalId: string + createdAt: string + additionalMetadata?: Record + }> + }> + cancel: ( + options: { ids: string[] }, + requestOptions?: ExternalKBGraphRequestOptions + ) => Promise + } +} + +export type KBGraphLogger = { + info?: ( + message: string, + metadata?: Record + ) => unknown | Promise + error?: ( + message: string, + metadata?: Record + ) => unknown | Promise +} + +export type ExternalKBGraphConfig = { + client: { + token: string + host_port: string + api_url: string + namespace: '' + tls_config: { tls_strategy: ExternalHatchetTLSStrategy } + } + workflowName: string +} + +export type KBGraphQualityConfig = { + speedMode: 'balanced' | 'quality' + generationModel: string + cleaningModel: string +} + +export type RecoveredExternalKBGraphRun = { + runId: string + startedAt: Date +} + +let externalKBGraphClient: ExternalKBGraphClient | undefined + +function requireEnvironmentVariable( + env: NodeJS.ProcessEnv, + name: string +): string { + const value = env[name]?.trim() + if (!value) { + throw new Error(`${name} must be configured`) + } + return value +} + +export function getKBGraphArtifactBlobName(buildId: string): string { + return `${KB_GRAPH_ARTIFACT_PREFIX}/${buildId}.graphml` +} + +export function getKBGraphOwnerContainerName(ownerId: string): string { + return `kb-${ownerId}` +} + +export function getKBGraphTimeoutSeconds( + env: NodeJS.ProcessEnv = process.env +): number { + const configuredValue = env.KB_GRAPH_TIMEOUT_SECONDS + if (configuredValue === undefined) { + return DEFAULT_KB_GRAPH_TIMEOUT_SECONDS + } + if (!/^[1-9]\d*$/.test(configuredValue)) { + throw new Error('KB_GRAPH_TIMEOUT_SECONDS must be a positive integer') + } + + const timeoutSeconds = Number(configuredValue) + if (!Number.isSafeInteger(timeoutSeconds)) { + throw new Error('KB_GRAPH_TIMEOUT_SECONDS must be a positive integer') + } + return timeoutSeconds +} + +export function getExternalKBGraphConfig( + env: NodeJS.ProcessEnv = process.env +): ExternalKBGraphConfig { + const tlsStrategy = requireEnvironmentVariable( + env, + 'KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY' + ) + if ( + tlsStrategy !== 'tls' && + tlsStrategy !== 'mtls' && + tlsStrategy !== 'none' + ) { + throw new Error( + 'KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY must be tls, mtls, or none' + ) + } + + return { + client: { + token: requireEnvironmentVariable(env, 'KB_GRAPH_HATCHET_CLIENT_TOKEN'), + host_port: requireEnvironmentVariable( + env, + 'KB_GRAPH_HATCHET_CLIENT_HOST_PORT' + ), + api_url: requireEnvironmentVariable(env, 'KB_GRAPH_HATCHET_API_URL'), + namespace: '', + tls_config: { tls_strategy: tlsStrategy }, + }, + workflowName: requireEnvironmentVariable( + env, + 'KB_GRAPH_HATCHET_WORKFLOW_NAME' + ), + } +} + +export function getKBGraphQualityConfig( + qualityTier: KBGraphQualityTier, + env: NodeJS.ProcessEnv = process.env +): KBGraphQualityConfig { + switch (qualityTier) { + case KBGraphQualityTier.STANDARD: + return { + speedMode: 'balanced', + generationModel: requireEnvironmentVariable( + env, + 'KB_GRAPH_STANDARD_GENERATION_MODEL' + ), + cleaningModel: requireEnvironmentVariable( + env, + 'KB_GRAPH_STANDARD_CLEANING_MODEL' + ), + } + case KBGraphQualityTier.HIGH: + return { + speedMode: 'quality', + generationModel: requireEnvironmentVariable( + env, + 'KB_GRAPH_HIGH_GENERATION_MODEL' + ), + cleaningModel: requireEnvironmentVariable( + env, + 'KB_GRAPH_HIGH_CLEANING_MODEL' + ), + } + } +} + +/** + * Keep the existing worker usable before graph integration is configured, but + * fail its startup on an incomplete graph configuration instead of consuming + * lecturer-triggered build retries with a predictable environment error. + */ +export function validateKBGraphWorkerConfig( + env: NodeJS.ProcessEnv = process.env +): void { + const graphIntegrationConfigured = + KB_GRAPH_CONFIGURATION_ENVIRONMENT_VARIABLES.some((name) => + env[name]?.trim() + ) + if (!graphIntegrationConfigured) { + return + } + + getExternalKBGraphConfig(env) + getKnowledgeGraphConfig(env) + getKBGraphTimeoutSeconds(env) + getKBGraphQualityConfig(KBGraphQualityTier.STANDARD, env) + getKBGraphQualityConfig(KBGraphQualityTier.HIGH, env) +} + +export function getExternalKBGraphClient( + env: NodeJS.ProcessEnv = process.env +): ExternalKBGraphClient { + if (!externalKBGraphClient) { + const hatchetClient = HatchetClient.init( + getExternalKBGraphConfig(env).client + ) + externalKBGraphClient = { + runNoWait: (workflowName, input, options) => + hatchetClient.runNoWait(workflowName, input, options), + runs: { + get: async (runId, requestOptions) => { + const response = await hatchetClient.api.v1WorkflowRunGet( + runId, + requestOptions + ) + return response.data + }, + get_status: async (runId, requestOptions) => { + const response = await hatchetClient.api.v1WorkflowRunGetStatus( + runId, + requestOptions + ) + return response.data + }, + list: async (options, requestOptions) => { + const workflowIds = await Promise.all( + options.workflowNames.map(async (workflowName) => { + const response = await hatchetClient.api.workflowList( + hatchetClient.tenantId, + { name: workflowName }, + requestOptions + ) + const workflowId = response.data.rows?.[0]?.metadata.id + if (!workflowId) { + throw new Error( + `Hatchet workflow was not found: ${workflowName}` + ) + } + return workflowId + }) + ) + const response = await hatchetClient.api.v1WorkflowRunList( + hatchetClient.tenantId, + { + limit: options.limit, + since: options.since.toISOString(), + additional_metadata: Object.entries( + options.additionalMetadata + ).map(([key, value]) => `${key}:${value}`), + workflow_ids: workflowIds, + only_tasks: options.onlyTasks, + include_payloads: options.includePayloads, + }, + requestOptions + ) + return { + rows: response.data.rows.map((row) => ({ + workflowRunExternalId: row.workflowRunExternalId, + createdAt: row.createdAt, + additionalMetadata: row.additionalMetadata as + | Record + | undefined, + })), + } + }, + cancel: (options, requestOptions) => + hatchetClient.api.v1TaskCancel( + hatchetClient.tenantId, + { externalIds: options.ids }, + requestOptions + ), + }, + } + } + return externalKBGraphClient! +} + +export async function getKBGraphTerminalResult( + runId: string, + clientOrOptions?: ExternalKBGraphClient | ExternalKBGraphRequestOptions +): Promise { + const client = + clientOrOptions && 'runs' in clientOrOptions + ? clientOrOptions + : getExternalKBGraphClient() + const requestOptions = + clientOrOptions && 'runs' in clientOrOptions ? undefined : clientOrOptions + const details = requestOptions + ? await client.runs.get(runId, requestOptions) + : await client.runs.get(runId) + return details.run.output +} + +export function getKBGraphSourceUrl( + source: Pick, + { + ownerId, + env = process.env, + now = () => new Date(), + }: { + ownerId: string + env?: NodeJS.ProcessEnv + now?: () => Date + } +): string { + if (source.type === KBResourceType.URL) { + if (!source.sourceUrl) { + throw new Error('KB graph URL source is invalid') + } + return source.sourceUrl + } + if (source.type !== KBResourceType.BLOB || !source.blobName) { + throw new Error('KB graph blob source is invalid') + } + + const accountName = requireEnvironmentVariable( + env, + 'BLOB_STORAGE_ACCOUNT_NAME' + ) + const accessKey = requireEnvironmentVariable(env, 'BLOB_STORAGE_ACCESS_KEY') + const credential = new StorageSharedKeyCredential(accountName, accessKey) + // The external LightRAG worker needs a host-reachable account URL. Production + // uses the public HTTPS endpoint; local development can override this with a + // loopback or .localhost Azurite endpoint without changing browser uploads. + const graphAccountUrl = getBlobStorageAccountUrl( + accountName, + env.KB_GRAPH_BLOB_ACCOUNT_URL ?? env.BLOB_STORAGE_ACCOUNT_URL + ) + const graphAccount = new URL(graphAccountUrl) + const graphAccountHost = graphAccount.hostname + const isLocalDevEndpoint = isLoopbackHost(graphAccountHost) + if (graphAccount.protocol === 'http:' && !isLocalDevEndpoint) { + throw new Error( + 'KB graph Blob account URL must use HTTPS outside local development' + ) + } + const serviceClient = new BlobServiceClient(graphAccountUrl, credential) + const containerName = getKBGraphOwnerContainerName(ownerId) + const blobClient = serviceClient + .getContainerClient(containerName) + .getBlobClient(source.blobName) + const currentTime = now() + const expiresOn = new Date( + currentTime.getTime() + + (getKBGraphTimeoutSeconds(env) * 1000 + KB_GRAPH_BLOB_SAS_CLOCK_SKEW_MS) + ) + const sas = generateBlobSASQueryParameters( + { + containerName, + blobName: source.blobName, + permissions: BlobSASPermissions.parse('r'), + ...(isLocalDevEndpoint ? {} : { protocol: SASProtocol.Https }), + startsOn: new Date( + currentTime.getTime() - KB_GRAPH_BLOB_SAS_CLOCK_SKEW_MS + ), + expiresOn, + }, + credential + ).toString() + + return `${blobClient.url}?${sas}` +} + +export async function recoverExternalKBGraphRun({ + client, + workflowName, + additionalMetadata, + recoveryAnchor, + requestOptions, +}: { + client: ExternalKBGraphClient + workflowName: string + additionalMetadata: Record + recoveryAnchor: Date + requestOptions?: ExternalKBGraphRequestOptions +}): Promise { + const buildId = additionalMetadata[KB_GRAPH_BUILD_METADATA_KEY] + if (!buildId) { + throw new Error('KB graph build recovery metadata is missing') + } + + const existingRuns = await client.runs.list( + { + workflowNames: [workflowName], + // Hatchet treats metadata filters as OR. Query on the unique build id and + // verify the rest locally before recovering a run. + additionalMetadata: { [KB_GRAPH_BUILD_METADATA_KEY]: buildId }, + onlyTasks: false, + includePayloads: false, + // An empty answer is taken as proof that the provider never accepted a run + // for this build (it releases the cost reservation), so the page must be + // large enough that a matching row can never be truncated away. + limit: 10, + since: new Date( + recoveryAnchor.getTime() - KB_GRAPH_BLOB_SAS_CLOCK_SKEW_MS + ), + }, + requestOptions + ) + const recoveredRun = existingRuns.rows.find((run) => + Object.entries(additionalMetadata).every( + ([key, value]) => run.additionalMetadata?.[key] === value + ) + ) + if (!recoveredRun) { + return undefined + } + + return { + runId: recoveredRun.workflowRunExternalId, + startedAt: new Date(recoveredRun.createdAt), + } +} + +async function logErrorBestEffort( + logger: KBGraphLogger | undefined, + message: string, + identifiers: Record +): Promise { + try { + await logger?.error?.(message, identifiers) + } catch { + // State reconciliation must not fail merely because logging is unavailable. + } +} + +export async function cancelExternalKBGraphRunBestEffort({ + client, + runId, + identifiers, + logger, + requestOptions, +}: { + client: ExternalKBGraphClient + runId: string + identifiers: Record + logger?: KBGraphLogger + requestOptions?: ExternalKBGraphRequestOptions +}): Promise { + try { + if (requestOptions) { + await client.runs.cancel({ ids: [runId] }, requestOptions) + } else { + await client.runs.cancel({ ids: [runId] }) + } + } catch { + await logErrorBestEffort( + logger, + 'External KB graph workflow cancellation failed', + identifiers + ) + } +} diff --git a/packages/hatchet/src/kbIngestion.ts b/packages/hatchet/src/kbIngestion.ts new file mode 100644 index 0000000000..e0d7b20817 --- /dev/null +++ b/packages/hatchet/src/kbIngestion.ts @@ -0,0 +1,962 @@ +import { + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + type Prisma, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import type { + DeleteKBResourceInput, + IngestKBResourceInput, +} from '@klicker-uzh/types' +import { + MAX_KB_SOURCE_SIZE_BYTES, + MAX_KB_TOTAL_SIZE_BYTES, +} from '@klicker-uzh/types' +import { + buildKBIngestionSource, + createKBIngestionApiClient, + getKBIngestionProjectId, + prepareKBIngestionSource, + type KBIngestionApiClient, + type KBIngestionSource, +} from './kbIngestionApi.js' + +const KB_INGESTION_POLL_CONCURRENCY = 8 +const KB_INGESTION_POLL_LIMIT = 32 + +export type KBIngestionLogger = { + info?: ( + message: string, + metadata?: Record + ) => unknown | Promise + error?: ( + message: string, + metadata?: Record + ) => unknown | Promise +} + +type KBIngestionPrisma = PrismaClient + +export type DispatchKBIngestionDependencies = { + prisma: KBIngestionPrisma + client?: KBIngestionApiClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBIngestionLogger + prepareSource?: ( + input: IngestKBResourceInput, + env: NodeJS.ProcessEnv + ) => Promise +} + +export type MonitorKBIngestionsDependencies = { + prisma: KBIngestionPrisma + client?: KBIngestionApiClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBIngestionLogger +} + +export type DispatchKBDeletionDependencies = { + prisma: KBIngestionPrisma + client?: KBIngestionApiClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBIngestionLogger +} + +async function lockPersistedKbScope( + prisma: Prisma.TransactionClient, + input: IngestKBResourceInput +): Promise { + const locked = await prisma.$queryRaw>` + SELECT kb."id" + FROM "public"."KB" AS kb + INNER JOIN "public"."KBResource" AS resource ON resource."kbId" = kb."id" + WHERE kb."id" = CAST(${input.kbId} AS UUID) + AND resource."id" = CAST(${input.resourceId} AS UUID) + AND kb."deletedAt" IS NULL + AND resource."deletedAt" IS NULL + FOR UPDATE OF kb + ` + return locked.length === 1 +} + +async function logErrorBestEffort( + logger: KBIngestionLogger | undefined, + message: string, + identifiers: Record +): Promise { + try { + await logger?.error?.(message, identifiers) + } catch { + // Error handling must continue when the logger transport is unavailable. + } +} + +async function logInfoBestEffort( + logger: KBIngestionLogger | undefined, + message: string, + identifiers: Record +): Promise { + try { + await logger?.info?.(message, identifiers) + } catch { + // A completed dispatch must not fail when the logger transport is unavailable. + } +} + +export function validateKBIngestionWorkerConfig( + env: NodeJS.ProcessEnv = process.env +): void { + const configuredValues = [ + env.KB_INGESTION_API_URL, + env.KB_INGESTION_API_KEY, + ].filter((value) => value?.trim()) + if (configuredValues.length > 0) { + createKBIngestionApiClient({ env }) + } +} + +async function persistPreparedSource({ + input, + prisma, + source, + env, +}: { + input: IngestKBResourceInput + prisma: KBIngestionPrisma + source: KBIngestionSource + env: NodeJS.ProcessEnv +}): Promise { + const persisted = await prisma.$transaction(async (tx) => { + if (!(await lockPersistedKbScope(tx, input))) { + return false + } + const currentResource = await tx.kBResource.findFirst({ + where: { + id: input.resourceId, + kbId: input.kbId, + deletedAt: null, + kb: { deletedAt: null }, + }, + select: { sizeBytes: true }, + }) + if (!currentResource) { + return false + } + const [resources, unknownSizeResources, uploadTickets] = await Promise.all([ + tx.kBResource.aggregate({ + where: { kbId: input.kbId }, + _sum: { sizeBytes: true }, + }), + tx.kBResource.count({ + where: { kbId: input.kbId, sizeBytes: null }, + }), + tx.kBUploadTicket.aggregate({ + where: { kbId: input.kbId }, + _sum: { sizeBytes: true }, + }), + ]) + const retainedSizeBytes = + (resources._sum.sizeBytes ?? 0) + + unknownSizeResources * MAX_KB_SOURCE_SIZE_BYTES + const projectedSizeBytes = + retainedSizeBytes + + (uploadTickets._sum.sizeBytes ?? 0) - + (currentResource.sizeBytes ?? MAX_KB_SOURCE_SIZE_BYTES) + + source.sizeBytes + if (projectedSizeBytes > MAX_KB_TOTAL_SIZE_BYTES) { + const finishedAt = new Date() + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + kbId: input.kbId, + deletedAt: null, + ingestionAttemptId: input.ingestionAttemptId, + resourceVersion: input.resourceVersion, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + externalOperationId: null, + }, + data: { + status: KBResourceStatus.FAILED, + statusMessage: 'The knowledge base storage limit was reached.', + errorCode: 'KB_STORAGE_LIMIT_REACHED', + }, + }) + if (resourceUpdate.count === 1) { + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.ingestionAttemptId, + resourceId: input.resourceId, + operation: KBIngestionOperation.UPSERT, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: KBIngestionStatus.FAILED, + statusMessage: 'The knowledge base storage limit was reached.', + errorCode: 'KB_STORAGE_LIMIT_REACHED', + finishedAt, + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion source could not be correlated') + } + } + return false + } + + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + kbId: input.kbId, + deletedAt: null, + kb: { deletedAt: null }, + ingestionAttemptId: input.ingestionAttemptId, + resourceVersion: input.resourceVersion, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + contentSha256: null, + externalOperationId: null, + }, + data: { + contentSha256: source.contentSha256, + mimeType: source.mimeType, + sizeBytes: source.sizeBytes, + }, + }) + if (resourceUpdate.count !== 1) { + return false + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.ingestionAttemptId, + resourceId: input.resourceId, + operation: KBIngestionOperation.UPSERT, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { contentSha256: source.contentSha256 }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion source could not be correlated') + } + return true + }) + if (persisted) { + return source + } + + const current = await prisma.kBResource.findUnique({ + where: { id: input.resourceId }, + select: { + ingestionAttemptId: true, + resourceVersion: true, + contentSha256: true, + mimeType: true, + sizeBytes: true, + kbId: true, + deletedAt: true, + kb: { select: { deletedAt: true } }, + }, + }) + if ( + current?.ingestionAttemptId !== input.ingestionAttemptId || + current.resourceVersion !== input.resourceVersion || + current.kbId !== input.kbId || + current.deletedAt !== null || + current.kb.deletedAt !== null || + !current.contentSha256 || + !current.mimeType || + current.sizeBytes === null + ) { + return undefined + } + return buildKBIngestionSource( + input, + current.mimeType, + current.contentSha256, + current.sizeBytes, + env + ) +} + +export async function dispatchKBIngestion( + input: IngestKBResourceInput, + dependencies: DispatchKBIngestionDependencies +): Promise { + const env = dependencies.env ?? process.env + const now = dependencies.now ?? (() => new Date()) + const identifiers = { + resourceId: input.resourceId, + kbId: input.kbId, + ingestionAttemptId: input.ingestionAttemptId, + } + + try { + const resource = await dependencies.prisma.kBResource.findUnique({ + where: { id: input.resourceId }, + select: { + status: true, + ingestionAttemptId: true, + resourceVersion: true, + contentSha256: true, + mimeType: true, + sizeBytes: true, + kbId: true, + deletedAt: true, + kb: { select: { deletedAt: true } }, + externalOperationId: true, + }, + }) + if ( + !resource || + resource.kbId !== input.kbId || + resource.deletedAt !== null || + resource.kb.deletedAt !== null || + resource.ingestionAttemptId !== input.ingestionAttemptId || + resource.resourceVersion !== input.resourceVersion || + (resource.status !== KBResourceStatus.QUEUED && + resource.status !== KBResourceStatus.PROCESSING) + ) { + return undefined + } + if (resource.externalOperationId) { + return resource.externalOperationId + } + + let source = + resource.contentSha256 && resource.mimeType && resource.sizeBytes + ? buildKBIngestionSource( + input, + resource.mimeType, + resource.contentSha256, + resource.sizeBytes, + env + ) + : undefined + if (!source) { + const prepareSource = + dependencies.prepareSource ?? + ((sourceInput, sourceEnv) => + prepareKBIngestionSource(sourceInput, sourceEnv)) + const preparedSource = await prepareSource(input, env) + source = await persistPreparedSource({ + input, + prisma: dependencies.prisma, + source: preparedSource, + env, + }) + if (!source) { + return undefined + } + } + + const client = dependencies.client ?? createKBIngestionApiClient({ env }) + const startedAt = now() + const operationId = await client.acceptResource({ + resourceId: input.resourceId, + kbId: input.kbId, + resourceVersion: input.resourceVersion, + ingestionAttemptId: input.ingestionAttemptId, + source, + }) + const persisted = await dependencies.prisma.$transaction(async (tx) => { + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + ingestionAttemptId: input.ingestionAttemptId, + resourceVersion: input.resourceVersion, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + contentSha256: source.contentSha256, + externalOperationId: null, + }, + data: { + externalOperationId: operationId, + externalOperationStartedAt: startedAt, + }, + }) + if (resourceUpdate.count !== 1) { + return false + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.ingestionAttemptId, + resourceId: input.resourceId, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + externalOperationId: operationId, + startedAt, + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion operation could not be correlated') + } + return true + }) + if (!persisted) { + const currentResource = await dependencies.prisma.kBResource.findUnique({ + where: { id: input.resourceId }, + select: { + ingestionAttemptId: true, + resourceVersion: true, + externalOperationId: true, + }, + }) + if ( + currentResource?.ingestionAttemptId === input.ingestionAttemptId && + currentResource.resourceVersion === input.resourceVersion && + currentResource.externalOperationId === operationId + ) { + return operationId + } + + await logErrorBestEffort( + dependencies.logger, + 'Accepted KB ingestion operation could not be correlated', + identifiers + ) + return undefined + } + + await logInfoBestEffort( + dependencies.logger, + 'KB ingestion operation accepted', + identifiers + ) + return operationId + } catch { + await logErrorBestEffort( + dependencies.logger, + 'KB ingestion dispatch failed', + identifiers + ) + throw new Error('KB ingestion dispatch failed') + } +} + +export async function failKBIngestionDispatch({ + input, + prisma, +}: { + input: IngestKBResourceInput + prisma: KBIngestionPrisma +}): Promise { + const finishedAt = new Date() + await prisma.$transaction(async (tx) => { + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + ingestionAttemptId: input.ingestionAttemptId, + resourceVersion: input.resourceVersion, + externalOperationId: null, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + data: { + status: KBResourceStatus.FAILED, + statusMessage: 'The ingestion operation could not be started.', + errorCode: 'INGESTION_DISPATCH_FAILED', + }, + }) + if (resourceUpdate.count !== 1) { + return + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.ingestionAttemptId, + resourceId: input.resourceId, + operation: KBIngestionOperation.UPSERT, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: KBIngestionStatus.FAILED, + statusMessage: 'The ingestion operation could not be started.', + errorCode: 'INGESTION_DISPATCH_FAILED', + finishedAt, + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion dispatch failure could not be correlated') + } + }) +} + +export async function dispatchKBDeletion( + input: DeleteKBResourceInput, + dependencies: DispatchKBDeletionDependencies +): Promise { + const env = dependencies.env ?? process.env + const identifiers = { + resourceId: input.resourceId, + kbId: input.kbId, + deletionAttemptId: input.deletionAttemptId, + } + + try { + const resource = await dependencies.prisma.kBResource.findUnique({ + where: { id: input.resourceId }, + select: { + kbId: true, + deletedAt: true, + ingestionOperation: true, + ingestionAttemptId: true, + resourceVersion: true, + externalOperationId: true, + }, + }) + if ( + !resource?.deletedAt || + resource.kbId !== input.kbId || + resource.ingestionOperation !== KBIngestionOperation.DELETE || + resource.ingestionAttemptId !== input.deletionAttemptId || + resource.resourceVersion !== input.resourceVersion + ) { + return undefined + } + if (resource.externalOperationId) { + return resource.externalOperationId + } + + const client = dependencies.client ?? createKBIngestionApiClient({ env }) + const startedAt = (dependencies.now ?? (() => new Date()))() + const operationId = await client.deleteResource(input) + const persisted = await dependencies.prisma.$transaction(async (tx) => { + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: input.deletionAttemptId, + resourceVersion: input.resourceVersion, + externalOperationId: null, + }, + data: { + externalOperationId: operationId, + externalOperationStartedAt: startedAt, + statusMessage: null, + errorCode: null, + }, + }) + if (resourceUpdate.count !== 1) { + return false + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.deletionAttemptId, + resourceId: input.resourceId, + operation: KBIngestionOperation.DELETE, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + externalOperationId: operationId, + startedAt, + statusMessage: null, + errorCode: null, + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB deletion operation could not be correlated') + } + return true + }) + if (!persisted) { + const current = await dependencies.prisma.kBResource.findUnique({ + where: { id: input.resourceId }, + select: { + ingestionAttemptId: true, + resourceVersion: true, + externalOperationId: true, + }, + }) + if ( + current?.ingestionAttemptId === input.deletionAttemptId && + current?.resourceVersion === input.resourceVersion && + current?.externalOperationId === operationId + ) { + return operationId + } + await logErrorBestEffort( + dependencies.logger, + 'Accepted KB deletion operation could not be correlated', + identifiers + ) + return undefined + } + + await logInfoBestEffort( + dependencies.logger, + 'KB deletion operation accepted', + identifiers + ) + return operationId + } catch { + await logErrorBestEffort( + dependencies.logger, + 'KB deletion dispatch failed', + identifiers + ) + throw new Error('KB deletion dispatch failed') + } +} + +export async function retainFailedKBDeletionDispatch({ + input, + prisma, +}: { + input: DeleteKBResourceInput + prisma: KBIngestionPrisma +}): Promise { + await prisma.$transaction(async (tx) => { + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: input.resourceId, + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: input.deletionAttemptId, + resourceVersion: input.resourceVersion, + externalOperationId: null, + }, + data: { + status: KBResourceStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + errorCode: 'DELETION_DISPATCH_FAILED', + }, + }) + if (resourceUpdate.count !== 1) { + return + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: input.deletionAttemptId, + resourceId: input.resourceId, + operation: KBIngestionOperation.DELETE, + resourceVersion: input.resourceVersion, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: KBIngestionStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + errorCode: 'DELETION_DISPATCH_FAILED', + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB deletion retry state could not be correlated') + } + }) +} + +function mapOperationStatus( + status: 'accepted' | 'running' | 'succeeded' | 'failed' | 'superseded' +) { + switch (status) { + case 'accepted': + return { + resourceStatus: KBResourceStatus.QUEUED, + runStatus: KBIngestionStatus.QUEUED, + statusMessage: null, + terminal: false, + } + case 'running': + return { + resourceStatus: KBResourceStatus.PROCESSING, + runStatus: KBIngestionStatus.PROCESSING, + statusMessage: null, + terminal: false, + } + case 'succeeded': + return { + resourceStatus: KBResourceStatus.READY, + runStatus: KBIngestionStatus.SUCCEEDED, + statusMessage: null, + terminal: true, + } + case 'failed': + return { + resourceStatus: KBResourceStatus.FAILED, + runStatus: KBIngestionStatus.FAILED, + statusMessage: 'The ingestion operation failed.', + terminal: true, + } + case 'superseded': + return { + resourceStatus: KBResourceStatus.FAILED, + runStatus: KBIngestionStatus.SUPERSEDED, + statusMessage: 'The ingestion operation was superseded.', + terminal: true, + } + } +} + +async function reconcileResource({ + resource, + client, + prisma, + env, + logger, +}: { + resource: { + id: string + kbId: string + ingestionAttemptId: string | null + resourceVersion: number + contentSha256: string | null + externalOperationId: string | null + ingestionOperation: KBIngestionOperation + } + client: KBIngestionApiClient + prisma: KBIngestionPrisma + env: NodeJS.ProcessEnv + logger?: KBIngestionLogger +}) { + const { ingestionAttemptId, contentSha256, externalOperationId } = resource + const ingestionOperation = + resource.ingestionOperation ?? KBIngestionOperation.UPSERT + if ( + !ingestionAttemptId || + !externalOperationId || + (ingestionOperation === KBIngestionOperation.UPSERT && !contentSha256) + ) { + return + } + const identifiers = { + resourceId: resource.id, + kbId: resource.kbId, + ingestionAttemptId, + } + + try { + const operation = await client.getOperation(externalOperationId) + if ( + operation.operationId !== externalOperationId || + operation.projectId !== getKBIngestionProjectId(env) || + operation.producer !== 'klicker' || + operation.externalResourceId !== resource.id || + operation.resourceVersion !== resource.resourceVersion || + operation.expectedSha256 !== + (ingestionOperation === KBIngestionOperation.DELETE + ? null + : contentSha256) || + (ingestionOperation === KBIngestionOperation.DELETE + ? operation.operation !== 'delete' + : operation.operation === 'delete') + ) { + await logErrorBestEffort( + logger, + 'KB ingestion operation correlation failed', + identifiers + ) + return + } + + if ( + ingestionOperation === KBIngestionOperation.UPSERT && + operation.status === 'succeeded' && + operation.observedSha256 !== contentSha256 + ) { + await logErrorBestEffort( + logger, + 'KB ingestion observed digest correlation failed', + identifiers + ) + return + } + + const transition = mapOperationStatus(operation.status) + const servingMatchesCurrent = + ingestionOperation === KBIngestionOperation.DELETE + ? operation.serving.activeResourceVersion === null && + operation.serving.activeSha256 === null + : operation.serving.activeResourceVersion === + resource.resourceVersion && + operation.serving.activeSha256 === contentSha256 + if (operation.status === 'succeeded' && !servingMatchesCurrent) { + await logInfoBestEffort( + logger, + 'KB ingestion succeeded while serving cutover is pending', + identifiers + ) + } + const resourceStatus = + operation.status === 'succeeded' && !servingMatchesCurrent + ? KBResourceStatus.PROCESSING + : transition.resourceStatus + const sourceStatuses = + operation.status === 'accepted' + ? [KBResourceStatus.QUEUED] + : [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING] + const sourceRunStatuses = + operation.status === 'accepted' + ? [KBIngestionStatus.QUEUED] + : operation.status === 'succeeded' + ? [ + KBIngestionStatus.QUEUED, + KBIngestionStatus.PROCESSING, + KBIngestionStatus.SUCCEEDED, + ] + : [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING] + const operationUpdatedAt = new Date(operation.updatedAt) + await prisma.$transaction(async (tx) => { + const resourceUpdate = await tx.kBResource.updateMany({ + where: { + id: resource.id, + ingestionAttemptId, + resourceVersion: resource.resourceVersion, + contentSha256: + ingestionOperation === KBIngestionOperation.DELETE + ? null + : contentSha256, + externalOperationId, + ingestionOperation, + status: { + in: sourceStatuses, + }, + }, + data: { + status: resourceStatus, + statusMessage: transition.statusMessage, + errorCode: operation.errorCode, + activeResourceVersion: operation.serving.activeResourceVersion, + activeContentSha256: operation.serving.activeSha256, + ...(resourceStatus === KBResourceStatus.READY + ? { ingestedAt: operationUpdatedAt } + : {}), + }, + }) + if (resourceUpdate.count !== 1) { + return + } + const runUpdate = await tx.kBIngestionRun.updateMany({ + where: { + id: ingestionAttemptId, + resourceId: resource.id, + operation: ingestionOperation, + resourceVersion: resource.resourceVersion, + status: { in: sourceRunStatuses }, + }, + data: { + status: transition.runStatus, + statusMessage: transition.statusMessage, + errorCode: operation.errorCode, + ...(transition.terminal ? { finishedAt: operationUpdatedAt } : {}), + }, + }) + if (runUpdate.count !== 1) { + throw new Error('KB ingestion operation state could not be correlated') + } + }) + } catch { + await logErrorBestEffort( + logger, + 'KB ingestion operation reconciliation failed', + identifiers + ) + } +} + +export async function monitorActiveKBIngestions( + dependencies: MonitorKBIngestionsDependencies +): Promise { + const env = dependencies.env ?? process.env + const activeWhere = { + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + ingestionAttemptId: { not: null }, + externalOperationId: { not: null }, + OR: [ + { + ingestionOperation: KBIngestionOperation.UPSERT, + contentSha256: { not: null }, + deletedAt: null, + }, + { + ingestionOperation: KBIngestionOperation.DELETE, + deletedAt: { not: null }, + }, + ], + } + const activeCount = await dependencies.prisma.kBResource.count({ + where: activeWhere, + }) + if (activeCount === 0) { + return + } + + const pollWindow = Math.floor( + (dependencies.now?.() ?? new Date()).getTime() / 60_000 + ) + const skip = + ((pollWindow % activeCount) * KB_INGESTION_POLL_LIMIT) % activeCount + const query = { + where: activeWhere, + orderBy: { id: 'asc' as const }, + select: { + id: true, + kbId: true, + ingestionAttemptId: true, + resourceVersion: true, + contentSha256: true, + externalOperationId: true, + ingestionOperation: true, + }, + } + const resources = await dependencies.prisma.kBResource.findMany({ + ...query, + skip, + take: KB_INGESTION_POLL_LIMIT, + }) + if (resources.length < KB_INGESTION_POLL_LIMIT && skip > 0) { + resources.push( + ...(await dependencies.prisma.kBResource.findMany({ + ...query, + take: Math.min(KB_INGESTION_POLL_LIMIT - resources.length, skip), + })) + ) + } + const client = dependencies.client ?? createKBIngestionApiClient({ env }) + + for ( + let start = 0; + start < resources.length; + start += KB_INGESTION_POLL_CONCURRENCY + ) { + await Promise.all( + resources + .slice(start, start + KB_INGESTION_POLL_CONCURRENCY) + .map((resource) => + reconcileResource({ + resource, + client, + prisma: dependencies.prisma, + env, + logger: dependencies.logger, + }) + ) + ) + } +} diff --git a/packages/hatchet/src/kbIngestionApi.ts b/packages/hatchet/src/kbIngestionApi.ts new file mode 100644 index 0000000000..f7674bc9db --- /dev/null +++ b/packages/hatchet/src/kbIngestionApi.ts @@ -0,0 +1,581 @@ +import { + BlobServiceClient, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import type { IngestKBResourceInput } from '@klicker-uzh/types' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' +import { + isPublicIPv4Address, + normalizePublicHttpUrl, +} from '@klicker-uzh/util/public-url' +import { createHash } from 'node:crypto' +import { lookup } from 'node:dns/promises' +import { request as httpRequest, type IncomingMessage } from 'node:http' +import { request as httpsRequest } from 'node:https' +import type { LookupFunction } from 'node:net' + +const KB_INGESTION_PROJECT_ID = 'klicker-course-materials' +const KB_INGESTION_PRODUCER = 'klicker' +const KB_INGESTION_REQUEST_TIMEOUT_MS = 10_000 +const KB_SOURCE_FETCH_TIMEOUT_MS = 30_000 +const MAX_KB_SOURCE_BYTES = 25 * 1024 * 1024 +const MAX_KB_SOURCE_REDIRECTS = 3 +const SUPPORTED_INGESTION_MIME_TYPES = new Set([ + 'application/pdf', + 'text/plain', +]) +const SHA256_PATTERN = /^[a-f0-9]{64}$/ + +export type KBIngestionSource = { + kind: 'blob' | 'url' + url: string + mimeType: string + displayName: string + contentSha256: string + sizeBytes: number +} + +export type KBOperationStatus = + | 'accepted' + | 'running' + | 'succeeded' + | 'failed' + | 'superseded' + +export type KBOperationStatusResponse = { + operationId: string + status: KBOperationStatus + operation: 'create' | 'update' | 'delete' + projectId: string + producer: string + externalResourceId: string + resourceVersion: number + expectedSha256: string | null + observedSha256: string | null + serving: { + activeResourceVersion: number | null + activeSha256: string | null + } + errorCode: string | null + correlationId: string + createdAt: string + updatedAt: string +} + +export type AcceptKBResourceInput = { + resourceId: string + kbId: string + resourceVersion: number + ingestionAttemptId: string + source: KBIngestionSource +} + +export type DeleteKBResourceInput = { + resourceId: string + kbId: string + resourceVersion: number + deletionAttemptId: string +} + +export type KBIngestionApiClient = { + acceptResource: (input: AcceptKBResourceInput) => Promise + deleteResource: (input: DeleteKBResourceInput) => Promise + getOperation: (operationId: string) => Promise +} + +export type KBSourcePreparationDependencies = { + resolvePublicIPv4?: (hostname: string) => Promise + requestPinnedUrl?: (url: URL, address: string) => Promise +} + +export function getKBIngestionProjectId( + env: NodeJS.ProcessEnv = process.env +): string { + return env.KB_INGESTION_PROJECT_ID?.trim() || KB_INGESTION_PROJECT_ID +} + +type KBIngestionFetch = ( + input: string | URL, + init?: RequestInit +) => Promise> + +function requireEnvironmentVariable( + env: NodeJS.ProcessEnv, + name: string +): string { + const value = env[name]?.trim() + if (!value) { + throw new Error(`${name} must be configured`) + } + return value +} + +function getOrigin(env: NodeJS.ProcessEnv, name: string): string { + const rawValue = requireEnvironmentVariable(env, name) + let value: URL + try { + value = new URL(rawValue) + } catch { + throw new Error(`${name} must be an HTTP(S) origin`) + } + if ( + (value.protocol !== 'http:' && value.protocol !== 'https:') || + value.username || + value.password || + value.pathname !== '/' || + value.search || + value.hash + ) { + throw new Error(`${name} must be an HTTP(S) origin`) + } + return value.origin +} + +// Key comparison guards an exact-shape contract check, so the order must be +// byte-stable code units rather than locale-dependent collation. +function compareCodeUnits(a: string, b: string) { + return a < b ? -1 : a > b ? 1 : 0 +} + +function hasExactKeys(value: Record, keys: string[]) { + const actualKeys = Object.keys(value).sort(compareCodeUnits) + const expectedKeys = [...keys].sort(compareCodeUnits) + return ( + actualKeys.length === keys.length && + actualKeys.every((key, index) => key === expectedKeys[index]) + ) +} + +function isNullableSha256(value: unknown): value is string | null { + return ( + value === null || (typeof value === 'string' && SHA256_PATTERN.test(value)) + ) +} + +function isBoundedString(value: unknown, maxLength: number): value is string { + return ( + typeof value === 'string' && value.length > 0 && value.length <= maxLength + ) +} + +function isAwareDateTime(value: unknown): value is string { + return ( + typeof value === 'string' && + /(?:[zZ]|[+-]\d{2}:\d{2})$/.test(value) && + Number.isFinite(Date.parse(value)) + ) +} + +function isPositiveSafeInteger(value: unknown): value is number { + return typeof value === 'number' && Number.isSafeInteger(value) && value > 0 +} + +function isNullablePositiveSafeInteger(value: unknown): value is number | null { + return value === null || isPositiveSafeInteger(value) +} + +function parseAcceptedOperation(value: unknown): string { + if ( + !value || + typeof value !== 'object' || + Array.isArray(value) || + !hasExactKeys(value as Record, ['operation_id']) + ) { + throw new Error('Ingestion API returned an invalid response') + } + const operationId = (value as Record).operation_id + if (!isBoundedString(operationId, 255)) { + throw new Error('Ingestion API returned an invalid response') + } + return operationId +} + +function parseOperationStatus(value: unknown): KBOperationStatusResponse { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('Ingestion API returned an invalid response') + } + const operation = value as Record + if ( + !hasExactKeys(operation, [ + 'operation_id', + 'status', + 'operation', + 'project_id', + 'producer', + 'external_resource_id', + 'resource_version', + 'expected_sha256', + 'observed_sha256', + 'serving', + 'error_code', + 'correlation_id', + 'created_at', + 'updated_at', + ]) || + !isBoundedString(operation.operation_id, 255) || + typeof operation.status !== 'string' || + !['accepted', 'running', 'succeeded', 'failed', 'superseded'].includes( + operation.status + ) || + typeof operation.operation !== 'string' || + !['create', 'update', 'delete'].includes(operation.operation) || + !isBoundedString(operation.project_id, 255) || + !isBoundedString(operation.producer, 255) || + !isBoundedString(operation.external_resource_id, 512) || + !isPositiveSafeInteger(operation.resource_version) || + !isNullableSha256(operation.expected_sha256) || + !isNullableSha256(operation.observed_sha256) || + !operation.serving || + typeof operation.serving !== 'object' || + Array.isArray(operation.serving) || + !hasExactKeys(operation.serving as Record, [ + 'active_resource_version', + 'active_sha256', + ]) || + !isNullablePositiveSafeInteger( + (operation.serving as Record).active_resource_version + ) || + !isNullableSha256( + (operation.serving as Record).active_sha256 + ) || + (operation.error_code !== null && + !isBoundedString(operation.error_code, 128)) || + !isBoundedString(operation.correlation_id, 255) || + !isAwareDateTime(operation.created_at) || + !isAwareDateTime(operation.updated_at) + ) { + throw new Error('Ingestion API returned an invalid response') + } + + const serving = operation.serving as Record + return { + operationId: operation.operation_id, + status: operation.status as KBOperationStatus, + operation: operation.operation as KBOperationStatusResponse['operation'], + projectId: operation.project_id, + producer: operation.producer, + externalResourceId: operation.external_resource_id, + resourceVersion: operation.resource_version, + expectedSha256: operation.expected_sha256, + observedSha256: operation.observed_sha256, + serving: { + activeResourceVersion: serving.active_resource_version as number | null, + activeSha256: serving.active_sha256 as string | null, + }, + errorCode: operation.error_code as string | null, + correlationId: operation.correlation_id, + createdAt: operation.created_at, + updatedAt: operation.updated_at, + } +} + +export function createKBIngestionApiClient({ + env = process.env, + fetchRequest = fetch, +}: { + env?: NodeJS.ProcessEnv + fetchRequest?: KBIngestionFetch +} = {}): KBIngestionApiClient { + const apiOrigin = getOrigin(env, 'KB_INGESTION_API_URL') + const apiKey = requireEnvironmentVariable(env, 'KB_INGESTION_API_KEY') + const projectId = getKBIngestionProjectId(env) + + async function request( + path: string, + init: RequestInit, + expectedStatus: number + ) { + try { + const response = await fetchRequest(new URL(path, apiOrigin), { + ...init, + headers: { + Authorization: `Bearer ${apiKey}`, + ...init.headers, + }, + signal: AbortSignal.timeout(KB_INGESTION_REQUEST_TIMEOUT_MS), + }) + if (!response.ok || response.status !== expectedStatus) { + throw new Error('Ingestion API request failed') + } + return await response.json() + } catch { + throw new Error('Ingestion API request failed') + } + } + + return { + async acceptResource(input) { + const value = await request( + '/v1/resources', + { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'Idempotency-Key': input.ingestionAttemptId, + }, + body: JSON.stringify({ + project_id: projectId, + producer: KB_INGESTION_PRODUCER, + external_resource_id: input.resourceId, + resource_version: input.resourceVersion, + scope: { kb_id: input.kbId }, + source: { + kind: input.source.kind, + url: input.source.url, + mime_type: input.source.mimeType, + display_name: input.source.displayName, + }, + content_sha256: input.source.contentSha256, + }), + }, + 202 + ) + return parseAcceptedOperation(value) + }, + + async deleteResource(input) { + const value = await request( + `/v1/resources/${encodeURIComponent(input.resourceId)}`, + { + method: 'DELETE', + headers: { + 'Content-Type': 'application/json', + 'Idempotency-Key': input.deletionAttemptId, + }, + body: JSON.stringify({ + project_id: projectId, + producer: KB_INGESTION_PRODUCER, + resource_version: input.resourceVersion, + scope: { kb_id: input.kbId }, + }), + }, + 202 + ) + return parseAcceptedOperation(value) + }, + + async getOperation(operationId) { + const value = await request( + `/v1/operations/${encodeURIComponent(operationId)}`, + { method: 'GET' }, + 200 + ) + return parseOperationStatus(value) + }, + } +} + +async function sha256Stream( + stream: AsyncIterable, + maxBytes: number +): Promise<{ contentSha256: string; sizeBytes: number }> { + const hash = createHash('sha256') + let sizeBytes = 0 + for await (const value of stream) { + const chunk = Buffer.isBuffer(value) ? value : Buffer.from(value as never) + sizeBytes += chunk.length + if (sizeBytes > maxBytes) { + throw new Error('KB ingestion source is too large') + } + hash.update(chunk) + } + if (sizeBytes === 0) { + throw new Error('KB ingestion source is empty') + } + return { contentSha256: hash.digest('hex'), sizeBytes } +} + +async function prepareBlobSource( + input: Extract, + env: NodeJS.ProcessEnv +): Promise { + const accountName = requireEnvironmentVariable( + env, + 'BLOB_STORAGE_ACCOUNT_NAME' + ) + const accessKey = requireEnvironmentVariable(env, 'BLOB_STORAGE_ACCESS_KEY') + const credential = new StorageSharedKeyCredential(accountName, accessKey) + const blobClient = new BlobServiceClient( + getBlobStorageAccountUrl( + accountName, + env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? env.BLOB_STORAGE_ACCOUNT_URL + ), + credential + ) + .getContainerClient(input.containerName) + .getBlobClient(input.blobName) + const response = await blobClient.download(0, undefined, { + abortSignal: AbortSignal.timeout(KB_SOURCE_FETCH_TIMEOUT_MS), + }) + const mimeType = response.contentType?.trim().toLowerCase() + if ( + !response.readableStreamBody || + response.contentLength !== input.sizeBytes || + mimeType !== input.mimeType || + !SUPPORTED_INGESTION_MIME_TYPES.has(mimeType) + ) { + throw new Error('KB ingestion source is invalid') + } + const digest = await sha256Stream( + response.readableStreamBody as AsyncIterable, + MAX_KB_SOURCE_BYTES + ) + if (digest.sizeBytes !== input.sizeBytes) { + throw new Error('KB ingestion source is invalid') + } + + return buildKBIngestionSource( + input, + mimeType, + digest.contentSha256, + digest.sizeBytes, + env + ) +} + +export async function resolvePublicIPv4(hostname: string): Promise { + const addresses = await lookup(hostname, { all: true, family: 4 }) + if ( + addresses.length === 0 || + addresses.some(({ address }) => !isPublicIPv4Address(address)) + ) { + throw new Error('KB ingestion source URL is invalid') + } + return addresses[0]!.address +} + +function requestPinnedUrl(url: URL, address: string): Promise { + const request = url.protocol === 'https:' ? httpsRequest : httpRequest + const pinnedLookup: LookupFunction = (_hostname, _options, callback) => { + callback(null, address, 4) + } + + return new Promise((resolve, reject) => { + const sourceRequest = request( + url, + { + headers: { + Accept: [...SUPPORTED_INGESTION_MIME_TYPES].join(', '), + Connection: 'close', + }, + lookup: pinnedLookup, + signal: AbortSignal.timeout(KB_SOURCE_FETCH_TIMEOUT_MS), + }, + resolve + ) + sourceRequest.on('error', reject) + sourceRequest.end() + }) +} + +async function preparePublicUrlSource( + input: Extract, + dependencies: KBSourcePreparationDependencies +): Promise { + let currentUrl = new URL(normalizePublicHttpUrl(input.sourceUrl)) + for ( + let redirectCount = 0; + redirectCount <= MAX_KB_SOURCE_REDIRECTS; + redirectCount++ + ) { + const address = await (dependencies.resolvePublicIPv4 ?? resolvePublicIPv4)( + currentUrl.hostname + ) + const response = await (dependencies.requestPinnedUrl ?? requestPinnedUrl)( + currentUrl, + address + ) + if ( + response.statusCode && + [301, 302, 303, 307, 308].includes(response.statusCode) + ) { + const location = response.headers.location + response.resume() + if (!location || redirectCount === MAX_KB_SOURCE_REDIRECTS) { + throw new Error('KB ingestion source redirect is invalid') + } + currentUrl = new URL( + normalizePublicHttpUrl(new URL(location, currentUrl).toString()) + ) + continue + } + if (response.statusCode !== 200) { + response.resume() + throw new Error('KB ingestion source could not be fetched') + } + + const rawMimeType = response.headers['content-type'] + const mimeType = (Array.isArray(rawMimeType) ? rawMimeType[0] : rawMimeType) + ?.split(';', 1)[0] + .trim() + .toLowerCase() + if (!mimeType || !SUPPORTED_INGESTION_MIME_TYPES.has(mimeType)) { + response.resume() + throw new Error('KB ingestion source type is not supported') + } + const declaredSize = Number(response.headers['content-length']) + if ( + response.headers['content-length'] !== undefined && + (!Number.isSafeInteger(declaredSize) || + declaredSize <= 0 || + declaredSize > MAX_KB_SOURCE_BYTES) + ) { + response.resume() + throw new Error('KB ingestion source is too large') + } + + const digest = await sha256Stream(response, MAX_KB_SOURCE_BYTES) + return buildKBIngestionSource( + input, + mimeType, + digest.contentSha256, + digest.sizeBytes + ) + } + throw new Error('KB ingestion source redirect is invalid') +} + +export async function prepareKBIngestionSource( + input: IngestKBResourceInput, + env: NodeJS.ProcessEnv = process.env, + dependencies: KBSourcePreparationDependencies = {} +): Promise { + return input.type === 'BLOB' + ? prepareBlobSource(input, env) + : preparePublicUrlSource(input, dependencies) +} + +export function buildKBIngestionSource( + input: IngestKBResourceInput, + mimeType: string, + contentSha256: string, + sizeBytes: number, + env: NodeJS.ProcessEnv = process.env +): KBIngestionSource { + if ( + !SUPPORTED_INGESTION_MIME_TYPES.has(mimeType) || + !SHA256_PATTERN.test(contentSha256) || + !Number.isSafeInteger(sizeBytes) || + sizeBytes <= 0 || + sizeBytes > MAX_KB_SOURCE_BYTES + ) { + throw new Error('KB ingestion source is invalid') + } + + const url = + input.type === 'BLOB' + ? new URL( + `/api/ingestion/resources/${input.resourceId}/versions/${input.resourceVersion}`, + getOrigin(env, 'KB_SOURCE_GATEWAY_URL') + ).toString() + : normalizePublicHttpUrl(input.sourceUrl) + + return { + kind: input.type === 'BLOB' ? 'blob' : 'url', + url, + mimeType, + displayName: input.title, + contentSha256, + sizeBytes, + } +} diff --git a/packages/hatchet/src/kbMaintenance.ts b/packages/hatchet/src/kbMaintenance.ts new file mode 100644 index 0000000000..65213d2121 --- /dev/null +++ b/packages/hatchet/src/kbMaintenance.ts @@ -0,0 +1,862 @@ +import { + BlobServiceClient, + StorageSharedKeyCredential, +} from '@azure/storage-blob' +import { + deleteKnowledgeGraph, + getKnowledgeGraphName, +} from '@klicker-uzh/knowledge-graph' +import { + KBGraphBuildStatus, + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + KBResourceType, + type Prisma, + type PrismaClient, +} from '@klicker-uzh/prisma/client' +import type { + DeleteKBResourceInput, + IngestKBResourceInput, +} from '@klicker-uzh/types' +import { getBlobStorageAccountUrl } from '@klicker-uzh/util' +import { randomUUID } from 'node:crypto' +import { getKBGraphArtifactBlobName } from './kbGraphIngestionApi.js' +import { + dispatchKBDeletion, + dispatchKBIngestion, + type KBIngestionLogger, +} from './kbIngestion.js' +import { + createKBIngestionApiClient, + type KBIngestionApiClient, +} from './kbIngestionApi.js' + +const KB_MAINTENANCE_BATCH_SIZE = 32 +const KB_MAINTENANCE_CONCURRENCY = 8 +// The maintenance task itself runs on this cadence (see the `maintain-kb-resources` +// cron in `index.ts`), so a QUEUED/UPSERT row with no `externalOperationId` that is +// older than one interval has necessarily survived at least one full sweep without +// being dispatched or reconciled, i.e. it is stranded rather than merely in flight. +export const KB_MAINTENANCE_INTERVAL_MS = 15 * 60 * 1000 +const KB_UPLOAD_RETENTION_GRACE_MS = 24 * 60 * 60 * 1000 +// How long a retired FalkorDB graph (neither active nor published) is kept before +// the serving projection is dropped. The GraphML archive is on a separate clock. +const KB_GRAPH_RETENTION_GRACE_MS = 24 * 60 * 60 * 1000 +// ADR 0015: deleting a knowledge base starts a 30-day recovery grace, after which +// its archived GraphML versions are purged. Until then every archived version is +// restorable, so a lecturer can be rolled back to any earlier successful graph. +const KB_GRAPH_DELETION_GRACE_MS = 30 * 24 * 60 * 60 * 1000 +const KB_BLOB_DELETE_TIMEOUT_MS = 30_000 +const KB_TERMINAL_DELETION_STATUSES = [ + KBIngestionStatus.FAILED, + KBIngestionStatus.SUPERSEDED, +] +// A build reaches one of these statuses only after the external run completed and +// exported its GraphML, so its artifact is a real, restorable graph version. +// SUPERSEDED is reached solely by a late success that a newer build outran +// (`KB_GRAPH_LATE_SUCCESS_SUPERSEDED`), which still produced a valid export. +const KB_GRAPH_ARCHIVED_ARTIFACT_STATUSES: KBGraphBuildStatus[] = [ + KBGraphBuildStatus.SUCCEEDED, + KBGraphBuildStatus.SUPERSEDED, +] + +type KBMaintenanceDependencies = { + prisma: PrismaClient + client?: KBIngestionApiClient + env?: NodeJS.ProcessEnv + now?: () => Date + logger?: KBIngestionLogger + deleteBlob?: (ownerId: string, blobName: string) => Promise + deleteGraph?: (graphName: string) => Promise + // Maintenance holds no task handles, so the caller supplies the re-enqueue. + enqueueKBGraphBuild?: (buildId: string) => Promise +} + +async function logMaintenanceError( + logger: KBIngestionLogger | undefined, + message: string, + identifiers: Record +) { + try { + await logger?.error?.(message, identifiers) + } catch { + // Maintenance must remain retryable when logging is unavailable. + } +} + +async function logInvalidRetryPayload( + logger: KBIngestionLogger | undefined, + resource: { id: string; kbId: string }, + ingestionAttemptId: string +) { + await logMaintenanceError(logger, 'KB ingestion retry payload is invalid', { + resourceId: resource.id, + kbId: resource.kbId, + ingestionAttemptId, + }) +} + +async function runBounded( + values: T[], + callback: (value: T) => Promise +) { + for ( + let start = 0; + start < values.length; + start += KB_MAINTENANCE_CONCURRENCY + ) { + await Promise.all( + values.slice(start, start + KB_MAINTENANCE_CONCURRENCY).map(callback) + ) + } +} + +function getBatchOffset(total: number, now: Date) { + if (total <= KB_MAINTENANCE_BATCH_SIZE) { + return 0 + } + const runNumber = Math.floor(now.getTime() / KB_MAINTENANCE_INTERVAL_MS) + const pageCount = Math.ceil(total / KB_MAINTENANCE_BATCH_SIZE) + return (runNumber % pageCount) * KB_MAINTENANCE_BATCH_SIZE +} + +async function deleteKBBlob( + ownerId: string, + blobName: string, + env: NodeJS.ProcessEnv +) { + const accountName = env.BLOB_STORAGE_ACCOUNT_NAME?.trim() + const accessKey = env.BLOB_STORAGE_ACCESS_KEY?.trim() + if (!accountName || !accessKey) { + throw new Error('Blob storage is not configured') + } + const credential = new StorageSharedKeyCredential(accountName, accessKey) + const serviceClient = new BlobServiceClient( + getBlobStorageAccountUrl( + accountName, + env.BLOB_STORAGE_INTERNAL_ACCOUNT_URL ?? env.BLOB_STORAGE_ACCOUNT_URL + ), + credential + ) + await serviceClient + .getContainerClient(`kb-${ownerId}`) + .getBlobClient(blobName) + .deleteIfExists({ + abortSignal: AbortSignal.timeout(KB_BLOB_DELETE_TIMEOUT_MS), + }) +} + +function isExpectedKBGraphArtifactName(blobName: string, buildId: string) { + return blobName === getKBGraphArtifactBlobName(buildId) +} + +export async function maintainKBResources( + dependencies: KBMaintenanceDependencies +): Promise { + const env = dependencies.env ?? process.env + const now = (dependencies.now ?? (() => new Date()))() + const deleteBlob = + dependencies.deleteBlob ?? + ((ownerId, blobName) => deleteKBBlob(ownerId, blobName, env)) + const deleteGraph = dependencies.deleteGraph ?? deleteKnowledgeGraph + + const deletionRetryWhere = { + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: { not: null }, + OR: [{ externalOperationId: null }, { status: KBResourceStatus.FAILED }], + } satisfies Prisma.KBResourceWhereInput + const deletionRetryCount = await dependencies.prisma.kBResource.count({ + where: deletionRetryWhere, + }) + const deletionRetryOffset = getBatchOffset(deletionRetryCount, now) + const deletionRetries = await dependencies.prisma.kBResource.findMany({ + where: deletionRetryWhere, + select: { + id: true, + kbId: true, + status: true, + ingestionAttemptId: true, + resourceVersion: true, + externalOperationId: true, + ingestionRuns: { + where: { + operation: KBIngestionOperation.DELETE, + status: { in: KB_TERMINAL_DELETION_STATUSES }, + }, + select: { id: true }, + }, + }, + orderBy: { id: 'asc' }, + ...(deletionRetryOffset > 0 ? { skip: deletionRetryOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + const retryableDeletions = deletionRetries.filter( + (resource) => + !resource.externalOperationId || + (resource.status === KBResourceStatus.FAILED && + resource.ingestionRuns.some( + ({ id }) => id === resource.ingestionAttemptId + )) + ) + if (retryableDeletions.length > 0) { + await runBounded(retryableDeletions, async (resource) => { + let deletionAttemptId = resource.ingestionAttemptId! + if (resource.externalOperationId) { + const retryAttemptId = randomUUID() + const claimed = await dependencies.prisma.$transaction(async (tx) => { + const update = await tx.kBResource.updateMany({ + where: { + id: resource.id, + deletedAt: { not: null }, + status: KBResourceStatus.FAILED, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: resource.ingestionAttemptId, + resourceVersion: resource.resourceVersion, + externalOperationId: resource.externalOperationId, + ingestionRuns: { + some: { + id: resource.ingestionAttemptId!, + operation: KBIngestionOperation.DELETE, + status: { in: KB_TERMINAL_DELETION_STATUSES }, + }, + }, + }, + data: { + status: KBResourceStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + ingestionAttemptId: retryAttemptId, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + if (update.count !== 1) { + return false + } + await tx.kBIngestionRun.create({ + data: { + id: retryAttemptId, + resourceId: resource.id, + operation: KBIngestionOperation.DELETE, + resourceVersion: resource.resourceVersion, + }, + }) + return true + }) + if (!claimed) { + return + } + deletionAttemptId = retryAttemptId + } + const input = { + resourceId: resource.id, + kbId: resource.kbId, + deletionAttemptId, + resourceVersion: resource.resourceVersion, + } satisfies DeleteKBResourceInput + try { + const client = + dependencies.client ?? createKBIngestionApiClient({ env }) + await dispatchKBDeletion(input, { + prisma: dependencies.prisma, + client, + env, + now: () => now, + logger: dependencies.logger, + }) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB deletion retry failed', + { + resourceId: resource.id, + kbId: resource.kbId, + deletionAttemptId, + } + ) + } + }) + } + + // Recovers UPSERT dispatches stranded when the process crashed between the + // commit that claims a fresh `ingestionAttemptId` (status QUEUED, + // externalOperationId null) and the enqueue of the ingestion task. Such rows + // are invisible to `monitorActiveKBIngestions` (which requires a non-null + // externalOperationId) and cannot be re-ingested or deleted through the API + // while QUEUED, so they would otherwise be stuck forever. + const upsertRetryStaleBefore = new Date( + now.getTime() - KB_MAINTENANCE_INTERVAL_MS + ) + const upsertRetryWhere = { + deletedAt: null, + ingestionOperation: KBIngestionOperation.UPSERT, + status: KBResourceStatus.QUEUED, + externalOperationId: null, + ingestionAttemptId: { not: null }, + updatedAt: { lte: upsertRetryStaleBefore }, + } satisfies Prisma.KBResourceWhereInput + const upsertRetryCount = await dependencies.prisma.kBResource.count({ + where: upsertRetryWhere, + }) + const upsertRetryOffset = getBatchOffset(upsertRetryCount, now) + const upsertRetries = await dependencies.prisma.kBResource.findMany({ + where: upsertRetryWhere, + select: { + id: true, + kbId: true, + title: true, + type: true, + blobName: true, + mimeType: true, + sizeBytes: true, + sourceUrl: true, + ingestionAttemptId: true, + resourceVersion: true, + kb: { select: { ownerId: true } }, + }, + orderBy: { id: 'asc' }, + ...(upsertRetryOffset > 0 ? { skip: upsertRetryOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + if (upsertRetries.length > 0) { + await runBounded(upsertRetries, async (resource) => { + // The same attempt id is reused (never a new one, never a status + // transition): the external platform dedupes on this id via its + // Idempotency-Key, so re-dispatching is safe even if a previous crash + // happened after the platform had already accepted the operation. + const ingestionAttemptId = resource.ingestionAttemptId + if (!ingestionAttemptId) { + return + } + const basePayload = { + resourceId: resource.id, + kbId: resource.kbId, + title: resource.title, + ingestionAttemptId, + resourceVersion: resource.resourceVersion, + } + let payload: IngestKBResourceInput + if (resource.type === KBResourceType.BLOB) { + if ( + !resource.blobName || + !resource.mimeType || + resource.sizeBytes === null + ) { + await logInvalidRetryPayload( + dependencies.logger, + resource, + ingestionAttemptId + ) + return + } + payload = { + ...basePayload, + type: KBResourceType.BLOB, + blobName: resource.blobName, + containerName: `kb-${resource.kb.ownerId}`, + mimeType: resource.mimeType, + sizeBytes: resource.sizeBytes, + } + } else { + if (!resource.sourceUrl) { + await logInvalidRetryPayload( + dependencies.logger, + resource, + ingestionAttemptId + ) + return + } + payload = { + ...basePayload, + type: KBResourceType.URL, + sourceUrl: resource.sourceUrl, + } + } + try { + const client = + dependencies.client ?? createKBIngestionApiClient({ env }) + await dispatchKBIngestion(payload, { + prisma: dependencies.prisma, + client, + env, + now: () => now, + logger: dependencies.logger, + }) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB ingestion retry failed', + { + resourceId: resource.id, + kbId: resource.kbId, + ingestionAttemptId, + } + ) + } + }) + } + + // The graph-build analogue of the UPSERT recovery above. `rebuildKbKnowledgeGraph` + // commits the reservation and the build-slot claim, then enqueues the task; a + // crash in between leaves a QUEUED build with neither an externalOperationId nor + // a dispatch claim. `monitorActiveKBGraphBuilds` skips it (it requires a + // correlated run) and it has no finishedAt for the retention sweep, so the KB's + // build slot and the lecturer's quota reservation would stay held forever. + const enqueueKBGraphBuild = dependencies.enqueueKBGraphBuild + if (enqueueKBGraphBuild) { + const graphDispatchStaleBefore = new Date( + now.getTime() - KB_MAINTENANCE_INTERVAL_MS + ) + const graphDispatchRetryWhere = { + status: KBGraphBuildStatus.QUEUED, + externalOperationId: null, + dispatchClaimedAt: null, + createdAt: { lte: graphDispatchStaleBefore }, + } satisfies Prisma.KBGraphBuildWhereInput + const graphDispatchRetryCount = + await dependencies.prisma.kBGraphBuild.count({ + where: graphDispatchRetryWhere, + }) + const graphDispatchRetryOffset = getBatchOffset( + graphDispatchRetryCount, + now + ) + const graphDispatchRetries = + await dependencies.prisma.kBGraphBuild.findMany({ + where: graphDispatchRetryWhere, + select: { id: true, kbId: true }, + orderBy: { id: 'asc' }, + ...(graphDispatchRetryOffset > 0 + ? { skip: graphDispatchRetryOffset } + : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(graphDispatchRetries, async (build) => { + try { + // The build id is already the external idempotency key and the real + // provider call is fenced by `dispatchClaimedAt`, so re-enqueuing the + // same id can never start a second external run or a second charge. + await enqueueKBGraphBuild(build.id) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB graph build dispatch retry failed', + { buildId: build.id, kbId: build.kbId } + ) + } + }) + } + + const expiredBefore = new Date(now.getTime() - KB_UPLOAD_RETENTION_GRACE_MS) + const expiredTicketWhere = { + expiresAt: { lte: expiredBefore }, + } satisfies Prisma.KBUploadTicketWhereInput + const expiredTicketCount = await dependencies.prisma.kBUploadTicket.count({ + where: expiredTicketWhere, + }) + const expiredTicketOffset = getBatchOffset(expiredTicketCount, now) + const expiredTickets = await dependencies.prisma.kBUploadTicket.findMany({ + where: expiredTicketWhere, + select: { + id: true, + blobName: true, + expiresAt: true, + kb: { select: { ownerId: true } }, + }, + orderBy: { id: 'asc' }, + ...(expiredTicketOffset > 0 ? { skip: expiredTicketOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(expiredTickets, async (ticket) => { + try { + await deleteBlob(ticket.kb.ownerId, ticket.blobName) + await dependencies.prisma.kBUploadTicket.deleteMany({ + where: { + id: ticket.id, + blobName: ticket.blobName, + expiresAt: ticket.expiresAt, + }, + }) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB upload cleanup failed', + { uploadTicketId: ticket.id } + ) + } + }) + + const expiredGraphBefore = new Date( + now.getTime() - KB_GRAPH_RETENTION_GRACE_MS + ) + const graphCleanupWhere = { + status: { + in: [ + KBGraphBuildStatus.SUCCEEDED, + KBGraphBuildStatus.FAILED, + KBGraphBuildStatus.SUPERSEDED, + ], + }, + finishedAt: { lte: expiredGraphBefore }, + cleanedAt: null, + OR: [ + { cleanupStartedAt: null }, + { cleanupStartedAt: { lt: expiredGraphBefore } }, + ], + } satisfies Prisma.KBGraphBuildWhereInput + const retainedGraphCount = await dependencies.prisma.kBGraphBuild.count({ + where: graphCleanupWhere, + }) + const retainedGraphOffset = getBatchOffset(retainedGraphCount, now) + const retainedGraphBuilds = await dependencies.prisma.kBGraphBuild.findMany({ + where: graphCleanupWhere, + select: { + id: true, + kbId: true, + status: true, + graphName: true, + graphmlBlobName: true, + kb: { + select: { + ownerId: true, + activeGraphBuildId: true, + publishedGraphBuildId: true, + }, + }, + }, + orderBy: { finishedAt: 'asc' }, + ...(retainedGraphOffset > 0 ? { skip: retainedGraphOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(retainedGraphBuilds, async (build) => { + if ( + build.kb.activeGraphBuildId === build.id || + build.kb.publishedGraphBuildId === build.id + ) { + return + } + if (build.graphName !== getKnowledgeGraphName(build.kbId, build.id)) { + await logMaintenanceError( + dependencies.logger, + 'KB graph cleanup rejected an invalid graph name', + { buildId: build.id, kbId: build.kbId } + ) + return + } + if ( + build.graphmlBlobName !== null && + !isExpectedKBGraphArtifactName(build.graphmlBlobName, build.id) + ) { + await logMaintenanceError( + dependencies.logger, + 'KB graph cleanup rejected an invalid artifact name', + { buildId: build.id, kbId: build.kbId } + ) + return + } + + const claimed = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + status: { + in: [ + KBGraphBuildStatus.SUCCEEDED, + KBGraphBuildStatus.FAILED, + KBGraphBuildStatus.SUPERSEDED, + ], + }, + finishedAt: { lte: expiredGraphBefore }, + cleanedAt: null, + OR: [ + { cleanupStartedAt: null }, + { cleanupStartedAt: { lt: expiredGraphBefore } }, + ], + kb: { + OR: [ + { activeGraphBuildId: null }, + { activeGraphBuildId: { not: build.id } }, + ], + AND: [ + { + OR: [ + { publishedGraphBuildId: null }, + { publishedGraphBuildId: { not: build.id } }, + ], + }, + ], + }, + }, + data: { cleanupStartedAt: now }, + }) + if (claimed.count !== 1) { + return + } + + // Only the serving projection is retired here. A build that exported a + // GraphML keeps it until its knowledge base is deleted and the recovery + // grace expires, so an earlier version stays restorable; a build that never + // produced an export has nothing worth retaining. + const purgeArchive = !KB_GRAPH_ARCHIVED_ARTIFACT_STATUSES.includes( + build.status + ) + try { + if (purgeArchive && build.graphmlBlobName) { + await deleteBlob(build.kb.ownerId, build.graphmlBlobName) + } + await deleteGraph(build.graphName) + const cleaned = await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + cleanedAt: null, + cleanupStartedAt: now, + status: { + in: [ + KBGraphBuildStatus.SUCCEEDED, + KBGraphBuildStatus.FAILED, + KBGraphBuildStatus.SUPERSEDED, + ], + }, + kb: { + OR: [ + { activeGraphBuildId: null }, + { activeGraphBuildId: { not: build.id } }, + ], + AND: [ + { + OR: [ + { publishedGraphBuildId: null }, + { publishedGraphBuildId: { not: build.id } }, + ], + }, + ], + }, + }, + data: { + cleanedAt: now, + ...(purgeArchive ? { graphmlPurgedAt: now } : {}), + }, + }) + if (cleaned.count !== 1) { + throw new Error('KB graph cleanup claim was lost') + } + } catch { + try { + await dependencies.prisma.kBGraphBuild.updateMany({ + where: { + id: build.id, + kbId: build.kbId, + cleanedAt: null, + cleanupStartedAt: now, + }, + data: { cleanupStartedAt: null }, + }) + } catch { + // A stale claim is reclaimable on a later sweep after the grace window. + } + await logMaintenanceError( + dependencies.logger, + 'KB graph cleanup failed', + { + buildId: build.id, + kbId: build.kbId, + } + ) + } + }) + + // ADR 0015: once the deletion recovery grace has expired there is nothing left + // to restore, so every remaining artifact of that knowledge base goes. This + // pass deliberately ignores `activeGraphBuildId`/`publishedGraphBuildId`: the + // KB is gone, and a build still holding the slot when it was deleted would + // otherwise keep its graph and archive forever. + const purgeArchiveBefore = new Date( + now.getTime() - KB_GRAPH_DELETION_GRACE_MS + ) + const archivePurgeWhere = { + graphmlPurgedAt: null, + kb: { deletedAt: { lte: purgeArchiveBefore } }, + } satisfies Prisma.KBGraphBuildWhereInput + const archivePurgeCount = await dependencies.prisma.kBGraphBuild.count({ + where: archivePurgeWhere, + }) + const archivePurgeOffset = getBatchOffset(archivePurgeCount, now) + const purgeableArchives = await dependencies.prisma.kBGraphBuild.findMany({ + where: archivePurgeWhere, + select: { + id: true, + kbId: true, + graphName: true, + graphmlBlobName: true, + cleanedAt: true, + kb: { select: { ownerId: true } }, + }, + orderBy: { id: 'asc' }, + ...(archivePurgeOffset > 0 ? { skip: archivePurgeOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(purgeableArchives, async (build) => { + if (build.graphName !== getKnowledgeGraphName(build.kbId, build.id)) { + await logMaintenanceError( + dependencies.logger, + 'KB graph archive purge rejected an invalid graph name', + { buildId: build.id, kbId: build.kbId } + ) + return + } + if ( + build.graphmlBlobName !== null && + !isExpectedKBGraphArtifactName(build.graphmlBlobName, build.id) + ) { + await logMaintenanceError( + dependencies.logger, + 'KB graph archive purge rejected an invalid artifact name', + { buildId: build.id, kbId: build.kbId } + ) + return + } + try { + if (build.graphmlBlobName) { + await deleteBlob(build.kb.ownerId, build.graphmlBlobName) + } + if (build.cleanedAt === null) { + await deleteGraph(build.graphName) + } + // The ledger row survives the purge (ADR 0013 keeps its cost evidence); + // only the stamps recording that the artifacts are gone are written. + await dependencies.prisma.kBGraphBuild.updateMany({ + where: { id: build.id, kbId: build.kbId, graphmlPurgedAt: null }, + data: { + graphmlPurgedAt: now, + ...(build.cleanedAt === null ? { cleanedAt: now } : {}), + }, + }) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB graph archive purge failed', + { buildId: build.id, kbId: build.kbId } + ) + } + }) + + const deletedResourceWhere = { + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + activeResourceVersion: null, + activeContentSha256: null, + ingestionAttemptId: { not: null }, + ingestionRuns: { + some: { + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.SUCCEEDED, + }, + }, + } satisfies Prisma.KBResourceWhereInput + const deletedResourceCount = await dependencies.prisma.kBResource.count({ + where: deletedResourceWhere, + }) + const deletedResourceOffset = getBatchOffset(deletedResourceCount, now) + const deletedResources = await dependencies.prisma.kBResource.findMany({ + where: deletedResourceWhere, + select: { + id: true, + type: true, + blobName: true, + ingestionAttemptId: true, + kb: { select: { ownerId: true } }, + ingestionRuns: { + where: { + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.SUCCEEDED, + }, + select: { id: true }, + }, + }, + orderBy: { id: 'asc' }, + ...(deletedResourceOffset > 0 ? { skip: deletedResourceOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(deletedResources, async (resource) => { + if ( + !resource.ingestionAttemptId || + !resource.ingestionRuns.some( + ({ id }) => id === resource.ingestionAttemptId + ) + ) { + return + } + try { + if (resource.type === KBResourceType.BLOB) { + if (!resource.blobName) { + throw new Error('KB blob metadata is invalid') + } + await deleteBlob(resource.kb.ownerId, resource.blobName) + } + await dependencies.prisma.kBResource.deleteMany({ + where: { + id: resource.id, + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: resource.ingestionAttemptId, + activeResourceVersion: null, + activeContentSha256: null, + ingestionRuns: { + some: { + id: resource.ingestionAttemptId, + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.SUCCEEDED, + }, + }, + }, + }) + } catch { + await logMaintenanceError( + dependencies.logger, + 'KB resource cleanup failed', + { resourceId: resource.id } + ) + } + }) + + // `KBGraphBuild.kb` cascades, so hard-deleting a KB would take its build ledger + // — metered cost, pricing version and actual provider usage — with it. ADR 0013 + // requires that evidence to stay auditable, so a KB that ever ran a graph build + // keeps its tombstone row permanently and only its artifacts are purged above. + // KBs that never built a graph are removed once the recovery grace has expired. + const deletedKbWhere = { + deletedAt: { not: null, lte: purgeArchiveBefore }, + resources: { none: {} }, + uploadTickets: { none: {} }, + chatbots: { none: { isEnabled: true } }, + graphBuilds: { none: {} }, + } satisfies Prisma.KBWhereInput + const deletedKbCount = await dependencies.prisma.kB.count({ + where: deletedKbWhere, + }) + const deletedKbOffset = getBatchOffset(deletedKbCount, now) + const deletedKbs = await dependencies.prisma.kB.findMany({ + where: deletedKbWhere, + select: { id: true }, + orderBy: { id: 'asc' }, + ...(deletedKbOffset > 0 ? { skip: deletedKbOffset } : {}), + take: KB_MAINTENANCE_BATCH_SIZE, + }) + await runBounded(deletedKbs, async (kb) => { + try { + await dependencies.prisma.kB.deleteMany({ + where: { + id: kb.id, + ...deletedKbWhere, + }, + }) + } catch { + await logMaintenanceError(dependencies.logger, 'KB cleanup failed', { + kbId: kb.id, + }) + } + }) +} diff --git a/packages/hatchet/test/kbGraphIngestion.test.ts b/packages/hatchet/test/kbGraphIngestion.test.ts new file mode 100644 index 0000000000..0a765f1f05 --- /dev/null +++ b/packages/hatchet/test/kbGraphIngestion.test.ts @@ -0,0 +1,1272 @@ +import { hashKBContentDigestEntries } from '@klicker-uzh/knowledge-graph' +import { + KBGraphBuildStatus, + KBGraphCostStatus, + KBGraphQualityTier, + KBResourceType, +} from '@klicker-uzh/prisma/client' +import { describe, expect, it, vi } from 'vitest' +import { + buildExternalKBGraphPayload, + dispatchKBGraphBuild, + markKBGraphBuildDispatchFailed, + monitorActiveKBGraphBuilds, +} from '../src/kbGraphIngestion.js' +import { + getKBGraphSourceUrl, + getKBGraphTerminalResult, + KB_GRAPH_BUILD_METADATA_KEY, + KB_GRAPH_KB_METADATA_KEY, + validateKBGraphWorkerConfig, + type ExternalKBGraphClient, +} from '../src/kbGraphIngestionApi.js' + +const NOW = new Date('2026-08-01T12:00:00.000Z') +const CREATED_AT = new Date('2026-08-01T11:55:00.000Z') +// Older than the 15-minute in-flight grace, so the claiming attempt is treated as +// abandoned rather than as a dispatch that may still be inside the provider call. +const STALE_CLAIMED_AT = new Date(NOW.getTime() - 16 * 60 * 1000) +const FRESH_CLAIMED_AT = new Date(NOW.getTime() - 60 * 1000) +const BUILD_ID = 'd1ec25e9-71ae-449f-88c5-7872f0b1a875' +const KB_ID = '842f262d-3482-43aa-956a-68f0c52184dd' +const OWNER_ID = 'fb5c14dc-853a-4acb-b146-080e84c4b7df' +const QUOTA_ID = '82b6f7a1-7aa7-4fc1-b7a0-648ba4c64e90' +const RESOURCE_ID = '17af8b84-58bf-4a92-8f8b-197556ed98f4' +const CONTENT_SHA256 = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' +const SOURCE_DIGEST = hashKBContentDigestEntries([ + { resourceId: RESOURCE_ID, contentSha256: CONTENT_SHA256 }, +]) +const SOURCE_URL = 'https://content.example.org/public-paper.pdf?version=1' + +const externalEnv = { + KB_GRAPH_HATCHET_CLIENT_TOKEN: 'external-token', + KB_GRAPH_HATCHET_CLIENT_HOST_PORT: 'hatchet-engine.other:7070', + KB_GRAPH_HATCHET_API_URL: 'http://hatchet-api.other:8080', + KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY: 'none', + KB_GRAPH_HATCHET_WORKFLOW_NAME: 'course-kg-ingestion', + KB_GRAPH_TIMEOUT_SECONDS: '3600', + KB_GRAPH_STANDARD_GENERATION_MODEL: 'klickeruzh/azure/gpt-5.4', + KB_GRAPH_STANDARD_CLEANING_MODEL: 'klickeruzh/azure/gpt-4.1-nano', + KB_GRAPH_HIGH_GENERATION_MODEL: 'klickeruzh/azure/gpt-5.4-high', + KB_GRAPH_HIGH_CLEANING_MODEL: 'klickeruzh/azure/gpt-4.1', + KB_FALKORDB_HOST: 'falkordb.other', + KB_FALKORDB_PORT: '6379', + KB_FALKORDB_TLS: 'false', + KB_FALKORDB_QUERY_TIMEOUT_MS: '5000', +} + +function createBuild(overrides: Record = {}) { + return { + id: BUILD_ID, + kbId: KB_ID, + sourceContentDigest: SOURCE_DIGEST, + graphName: `klickeruzh:kb:${KB_ID}:${BUILD_ID}`, + graphmlBlobName: `knowledge-graphs/${BUILD_ID}.graphml`, + qualityTier: KBGraphQualityTier.STANDARD, + createdAt: CREATED_AT, + status: KBGraphBuildStatus.QUEUED, + externalOperationId: null, + dispatchClaimedAt: null, + costStatus: KBGraphCostStatus.RESERVED, + estimatedCostMinorUnits: 100, + costCurrency: 'CHF', + costPricingVersion: 'test-v1', + semesterKey: '2026-H2', + quotaId: QUOTA_ID, + quota: { + id: QUOTA_ID, + ownerId: OWNER_ID, + semesterKey: '2026-H2', + currency: 'CHF', + limitMinorUnits: 1000, + reservedMinorUnits: 100, + }, + kb: { + ownerId: OWNER_ID, + deletedAt: null, + activeGraphBuildId: BUILD_ID, + knowledgeGraphEnabled: true, + }, + sources: [ + { + resourceId: RESOURCE_ID, + type: KBResourceType.URL, + sourceUrl: SOURCE_URL, + blobName: null, + contentSha256: CONTENT_SHA256, + }, + ], + ...overrides, + } +} + +function createClient({ + runId = 'external-run-id', + rows = [], +}: { + runId?: string + rows?: Array<{ + workflowRunExternalId: string + createdAt: string + additionalMetadata?: Record + }> +} = {}) { + return { + runs: { + get: vi.fn().mockResolvedValue({ run: { output: null } }), + get_status: vi.fn().mockResolvedValue('QUEUED'), + list: vi.fn().mockResolvedValue({ rows }), + cancel: vi.fn().mockResolvedValue({}), + }, + runNoWait: vi.fn().mockResolvedValue({ + getWorkflowRunId: vi.fn().mockResolvedValue(runId), + }), + } as unknown as ExternalKBGraphClient +} + +function createDispatchPrisma({ + build = createBuild(), + updateCount = 1, + rereadExternalOperationId, +}: { + build?: ReturnType | null + updateCount?: number + rereadExternalOperationId?: string | null +} = {}) { + const findUnique = vi.fn().mockResolvedValue(build) + if (rereadExternalOperationId !== undefined) { + findUnique + .mockResolvedValueOnce(build) + .mockResolvedValueOnce({ externalOperationId: rereadExternalOperationId }) + } + const prisma = { + kBGraphBuild: { + findUnique, + updateMany: vi.fn().mockResolvedValue({ count: updateCount }), + }, + kBGraphQuota: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kB: { updateMany: vi.fn().mockResolvedValue({ count: 1 }) }, + $queryRaw: vi.fn().mockResolvedValue([{ id: QUOTA_ID }]), + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + return prisma +} + +function createMonitorPrisma( + builds: Array>, + { + activeBuildCount = builds.length, + timedOutBuilds = [], + timedOutBuildCount = timedOutBuilds.length, + newerBuild = null, + servingResources = [], + ambiguousBuilds = [], + ambiguousBuildCount = ambiguousBuilds.length, + }: { + activeBuildCount?: number + timedOutBuilds?: Array> + timedOutBuildCount?: number + newerBuild?: { id: string } | null + servingResources?: Array<{ + id: string + activeContentSha256: string | null + }> + ambiguousBuilds?: Array> + ambiguousBuildCount?: number + } = {} +) { + const prisma = { + kBGraphBuild: { + findMany: vi + .fn() + .mockResolvedValueOnce(builds) + .mockResolvedValueOnce(timedOutBuilds) + .mockResolvedValueOnce(ambiguousBuilds), + count: vi + .fn() + .mockResolvedValueOnce(activeBuildCount) + .mockResolvedValueOnce(timedOutBuildCount) + .mockResolvedValueOnce(ambiguousBuildCount), + findFirst: vi.fn().mockResolvedValue(newerBuild), + findUnique: vi.fn().mockResolvedValue({ + quotaId: QUOTA_ID, + estimatedCostMinorUnits: 100, + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + }), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBGraphQuota: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kB: { + findUnique: vi + .fn() + .mockResolvedValue({ activeGraphBuildId: null, deletedAt: null }), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBResource: { findMany: vi.fn().mockResolvedValue(servingResources) }, + $queryRaw: vi + .fn() + .mockResolvedValue([ + { id: KB_ID, activeGraphBuildId: null, deletedAt: null }, + ]), + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + return prisma +} + +describe('KB graph external dispatch', () => { + it('reads the versioned terminal payload from the external run output', async () => { + const client = createClient() + const result = { contract_version: 'klicker-kb-graph/v1' } + vi.mocked(client.runs.get).mockResolvedValue({ run: { output: result } }) + + await expect( + getKBGraphTerminalResult('external-run-id', client) + ).resolves.toEqual(result) + }) + + it('allows an unconfigured worker but rejects a partial graph integration', () => { + expect(() => validateKBGraphWorkerConfig({})).not.toThrow() + // The out-of-repo worker secret alone must not arm the gate: doing so would + // stop every unrelated general-worker job if the secret lands first. + expect(() => + validateKBGraphWorkerConfig({ + KB_GRAPH_HATCHET_CLIENT_TOKEN: 'external-token', + }) + ).not.toThrow() + expect(() => + validateKBGraphWorkerConfig({ + KB_GRAPH_HATCHET_WORKFLOW_NAME: 'course-kg-ingestion', + }) + ).toThrow('KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY must be configured') + expect(() => validateKBGraphWorkerConfig(externalEnv)).not.toThrow() + }) + + it('uses a loopback Blob endpoint for host-side graph sources', () => { + const sourceUrl = getKBGraphSourceUrl( + { + type: KBResourceType.BLOB, + sourceUrl: null, + blobName: 'slides/private.pdf', + }, + { + ownerId: OWNER_ID, + env: { + BLOB_STORAGE_ACCOUNT_NAME: 'klickertest', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), + BLOB_STORAGE_ACCOUNT_URL: 'https://blob.example.org', + KB_GRAPH_BLOB_ACCOUNT_URL: 'http://127.0.0.1:10003/klickerdev', + KB_GRAPH_TIMEOUT_SECONDS: '3600', + }, + now: () => NOW, + } + ) + + const parsed = new URL(sourceUrl) + expect(`${parsed.origin}${parsed.pathname}`).toBe( + `http://127.0.0.1:10003/klickerdev/kb-${OWNER_ID}/slides/private.pdf` + ) + expect(parsed.searchParams.get('sp')).toBe('r') + expect(parsed.searchParams.get('spr')).toBeNull() + }) + + it('rejects a cleartext non-local Blob endpoint', () => { + expect(() => + getKBGraphSourceUrl( + { + type: KBResourceType.BLOB, + sourceUrl: null, + blobName: 'slides/private.pdf', + }, + { + ownerId: OWNER_ID, + env: { + BLOB_STORAGE_ACCOUNT_NAME: 'klickertest', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), + KB_GRAPH_BLOB_ACCOUNT_URL: 'http://blob.example.org', + KB_GRAPH_TIMEOUT_SECONDS: '3600', + }, + now: () => NOW, + } + ) + ).toThrow( + 'KB graph Blob account URL must use HTTPS outside local development' + ) + }) + + it('rejects a loopback-looking hostname outside local development', () => { + expect(() => + getKBGraphSourceUrl( + { + type: KBResourceType.BLOB, + sourceUrl: null, + blobName: 'slides/private.pdf', + }, + { + ownerId: OWNER_ID, + env: { + BLOB_STORAGE_ACCOUNT_NAME: 'klickertest', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), + KB_GRAPH_BLOB_ACCOUNT_URL: 'http://127.example.com', + KB_GRAPH_TIMEOUT_SECONDS: '3600', + }, + now: () => NOW, + } + ) + ).toThrow( + 'KB graph Blob account URL must use HTTPS outside local development' + ) + }) + + it('fails a queued build closed when the global graph kill switch is enabled', async () => { + const prisma = createDispatchPrisma() + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: { ...externalEnv, KB_GRAPH_DISABLED: 'true' }, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphQuota.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { reservedMinorUnits: { decrement: 100 } }, + }) + ) + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it('rechecks the persisted KB opt-in before starting the external run', async () => { + const prisma = createDispatchPrisma() + const initialBuild = createBuild() + const optedOutBuild = createBuild({ + kb: { ...initialBuild.kb, knowledgeGraphEnabled: false }, + }) + prisma.kBGraphBuild.findUnique + .mockReset() + .mockResolvedValueOnce(initialBuild) + .mockResolvedValueOnce(optedOutBuild) + .mockResolvedValue(optedOutBuild) + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ errorCode: 'KB_GRAPH_NOT_ENABLED' }), + }) + ) + }) + + it('holds a pre-accounting build for review instead of dispatching it', async () => { + const build = createBuild({ costStatus: null }) + const prisma = createDispatchPrisma({ build }) + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW, + }), + }) + ) + }) + + it('holds a reservation with incomplete quota identity before dispatch', async () => { + const build = createBuild({ quota: null }) + const prisma = createDispatchPrisma({ build }) + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + errorCode: 'KB_GRAPH_RESERVATION_INCOMPLETE', + }), + }) + ) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + ) + }) + + it('builds the pinned external manifest field-for-field', () => { + const build = createBuild() + + expect( + buildExternalKBGraphPayload(build, [SOURCE_URL], externalEnv) + ).toEqual({ + course_id: BUILD_ID, + storage_name: BUILD_ID, + sources: [ + { + source_id: RESOURCE_ID, + source_url: SOURCE_URL, + expected_content_sha256: CONTENT_SHA256, + }, + ], + upload_markdown: false, + export_to_falkordb: true, + falkordb_graph_name: `klickeruzh:kb:${KB_ID}:${BUILD_ID}`, + speed_mode: 'balanced', + generation_model: 'klickeruzh/azure/gpt-5.4', + cleaning_model: 'klickeruzh/azure/gpt-4.1-nano', + klicker_graph_build: { + build_id: BUILD_ID, + kb_id: KB_ID, + owner_id: OWNER_ID, + source_content_digest: SOURCE_DIGEST, + graphml_container_name: `kb-${OWNER_ID}`, + graphml_blob_name: `knowledge-graphs/${BUILD_ID}.graphml`, + }, + }) + }) + + it('creates an exact-blob, read-only, HTTPS-only SAS for a private source', () => { + const sourceUrl = getKBGraphSourceUrl( + { + type: KBResourceType.BLOB, + sourceUrl: null, + blobName: 'slides/private.pdf', + }, + { + ownerId: OWNER_ID, + env: { + BLOB_STORAGE_ACCOUNT_NAME: 'klickertest', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), + KB_GRAPH_TIMEOUT_SECONDS: '3600', + }, + now: () => NOW, + } + ) + + const parsed = new URL(sourceUrl) + expect(`${parsed.origin}${parsed.pathname}`).toBe( + `https://klickertest.blob.core.windows.net/kb-${OWNER_ID}/slides/private.pdf` + ) + expect(parsed.searchParams.get('sp')).toBe('r') + expect(parsed.searchParams.get('spr')).toBe('https') + expect(new Date(parsed.searchParams.get('st')!).toISOString()).toBe( + '2026-08-01T11:55:00.000Z' + ) + expect(new Date(parsed.searchParams.get('se')!).toISOString()).toBe( + '2026-08-01T13:05:00.000Z' + ) + }) + + it('persists a single external correlation for the active KB build', async () => { + const prisma = createDispatchPrisma() + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBe('external-run-id') + + expect(vi.mocked(client.runNoWait)).toHaveBeenCalledWith( + 'course-kg-ingestion', + expect.objectContaining({ + course_id: BUILD_ID, + storage_name: BUILD_ID, + sources: [ + expect.objectContaining({ + source_id: RESOURCE_ID, + expected_content_sha256: CONTENT_SHA256, + }), + ], + }), + { + additionalMetadata: { + [KB_GRAPH_BUILD_METADATA_KEY]: BUILD_ID, + [KB_GRAPH_KB_METADATA_KEY]: KB_ID, + }, + } + ) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + id: BUILD_ID, + kbId: KB_ID, + externalOperationId: null, + }), + data: expect.objectContaining({ + externalOperationId: 'external-run-id', + externalStartedAt: NOW, + startedAt: NOW, + }), + }) + ) + }) + + it('holds the reservation when provider acceptance cannot be correlated', async () => { + const prisma = createDispatchPrisma() + const client = createClient() + vi.mocked(client.runNoWait).mockResolvedValue({ + getWorkflowRunId: vi + .fn() + .mockRejectedValue(new Error('run id unavailable')), + }) + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).rejects.toThrow('External KB graph build dispatch failed') + + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { dispatchClaimedAt: NOW }, + }) + ) + + const ambiguousPrisma = createDispatchPrisma({ + build: createBuild({ dispatchClaimedAt: STALE_CLAIMED_AT }), + }) + await markKBGraphBuildDispatchFailed( + { buildId: BUILD_ID }, + ambiguousPrisma as never + ) + + expect(ambiguousPrisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(ambiguousPrisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + ) + expect(ambiguousPrisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('correlates an accepted-but-uncorrelated dispatch instead of parking it', async () => { + const prisma = createDispatchPrisma({ + build: createBuild({ dispatchClaimedAt: STALE_CLAIMED_AT }), + }) + const client = createClient({ + rows: [ + { + workflowRunExternalId: 'recovered-run-id', + createdAt: NOW.toISOString(), + additionalMetadata: { + [KB_GRAPH_BUILD_METADATA_KEY]: BUILD_ID, + [KB_GRAPH_KB_METADATA_KEY]: KB_ID, + }, + }, + ], + }) + + await dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + + // The run the earlier attempt lost is adopted, so no second run is started + // and the build leaves the ambiguous state on its own. + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + externalOperationId: 'recovered-run-id', + status: KBGraphBuildStatus.PROCESSING, + errorCode: null, + }), + }) + ) + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('leaves a freshly claimed dispatch alone instead of releasing its money', async () => { + const prisma = createDispatchPrisma({ + build: createBuild({ dispatchClaimedAt: FRESH_CLAIMED_AT }), + }) + const client = createClient({ rows: [] }) + + await dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + + // A duplicate task run for the same build must not act on the provider's + // "no run yet": the first attempt may still be inside its dispatch call and + // about to start a run that spends. Nothing is asked, nothing is written. + expect(client.runs.list).not.toHaveBeenCalled() + expect(client.runNoWait).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).not.toHaveBeenCalled() + expect(prisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('releases the hold when the provider has no run for the claimed build', async () => { + const prisma = createDispatchPrisma({ + build: createBuild({ dispatchClaimedAt: STALE_CLAIMED_AT }), + }) + const client = createClient({ rows: [] }) + + await dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + + // Nothing external was ever accepted, so this is an ordinary dispatch + // failure: the quota is given back and the KB build slot is freed. + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + errorCode: 'KB_GRAPH_DISPATCH_FAILED', + }), + }) + ) + expect(prisma.kBGraphQuota.updateMany).toHaveBeenCalled() + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it('keeps the ambiguous hold when the provider lookup itself fails', async () => { + const prisma = createDispatchPrisma({ + build: createBuild({ dispatchClaimedAt: STALE_CLAIMED_AT }), + }) + const client = createClient() + vi.mocked(client.runs.list).mockRejectedValue(new Error('provider down')) + + await dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + + // A run may still be generating and spending, so neither the quota nor the + // build slot may be handed back on an unanswered lookup. + expect(prisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + errorCode: 'KB_GRAPH_DISPATCH_AMBIGUOUS', + }), + }) + ) + }) + + it('does not release a reservation when gate compensation loses the dispatch claim race', async () => { + const prisma = createDispatchPrisma({ updateCount: 0 }) + const client = createClient() + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: { ...externalEnv, KB_GRAPH_DISABLED: 'true' }, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(prisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('recovers a matching external build before creating a duplicate run', async () => { + const prisma = createDispatchPrisma() + const client = createClient({ + runId: 'new-run-id', + rows: [ + { + workflowRunExternalId: 'recovered-run-id', + createdAt: '2026-08-01T11:57:00.000Z', + additionalMetadata: { + [KB_GRAPH_BUILD_METADATA_KEY]: BUILD_ID, + [KB_GRAPH_KB_METADATA_KEY]: KB_ID, + }, + }, + ], + }) + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBe('recovered-run-id') + + expect(vi.mocked(client.runNoWait)).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + externalOperationId: 'recovered-run-id', + externalStartedAt: new Date('2026-08-01T11:57:00.000Z'), + }), + }) + ) + }) + + it('cancels an orphaned external run when the guarded correlation loses its race', async () => { + const prisma = createDispatchPrisma({ updateCount: 0 }) + prisma.kBGraphBuild.updateMany + .mockResolvedValueOnce({ count: 1 }) + .mockResolvedValueOnce({ count: 0 }) + const client = createClient({ runId: 'orphaned-run-id' }) + + await expect( + dispatchKBGraphBuild( + { buildId: BUILD_ID }, + { + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getSourceUrl: () => SOURCE_URL, + } + ) + ).resolves.toBeUndefined() + + expect(vi.mocked(client.runs.cancel)).toHaveBeenCalledWith({ + ids: ['orphaned-run-id'], + }) + }) +}) + +describe('KB graph external reconciliation', () => { + it('fails closed when a completed build has no versioned terminal result', async () => { + const prisma = createMonitorPrisma([ + { + id: BUILD_ID, + kbId: KB_ID, + externalOperationId: 'external-run-id', + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + }, + ]) + const client = createClient() + vi.mocked(client.runs.get_status).mockResolvedValue('COMPLETED') + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + }) + + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ id: BUILD_ID }), + data: expect.objectContaining({ + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_RESULT_REQUIRED', + finishedAt: NOW, + }), + }) + ) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + ) + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it('hands a completed build to the versioned terminal-result settlement path', async () => { + const prisma = createMonitorPrisma([ + { + id: BUILD_ID, + kbId: KB_ID, + externalOperationId: 'external-run-id', + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + }, + ]) + const client = createClient() + vi.mocked(client.runs.get_status).mockResolvedValue('COMPLETED') + const getTerminalResult = vi.fn().mockResolvedValue({ status: 'SUCCEEDED' }) + const settleTerminalResult = vi.fn().mockResolvedValue('SETTLED') + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getTerminalResult, + settleTerminalResult, + }) + + expect(getTerminalResult).toHaveBeenCalledWith( + 'external-run-id', + expect.objectContaining({ signal: expect.any(AbortSignal) }) + ) + expect(settleTerminalResult).toHaveBeenCalledWith({ + buildId: BUILD_ID, + result: { status: 'SUCCEEDED' }, + finishedAt: NOW, + }) + expect(prisma.kBGraphBuild.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('times out a running build, cancels it, and releases its active slot', async () => { + const prisma = createMonitorPrisma([ + { + id: BUILD_ID, + kbId: KB_ID, + externalOperationId: 'external-run-id', + externalStartedAt: new Date('2026-08-01T10:00:00.000Z'), + }, + ]) + const client = createClient() + vi.mocked(client.runs.get_status).mockResolvedValue('RUNNING') + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + }) + + expect(vi.mocked(client.runs.cancel)).toHaveBeenCalledWith( + { ids: ['external-run-id'] }, + expect.objectContaining({ signal: expect.any(AbortSignal) }) + ) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + }), + }) + ) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { costStatus: KBGraphCostStatus.NEEDS_HUMAN_REVIEW }, + }) + ) + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it.each([ + { buildCount: 33, elapsedIntervals: 1, expectedSkip: 32 }, + { buildCount: 65, elapsedIntervals: 2, expectedSkip: 64 }, + ])('rotates the timed-out graph backstop window for $buildCount builds', async ({ + buildCount, + elapsedIntervals, + expectedSkip, + }) => { + const prisma = createMonitorPrisma([], { + timedOutBuildCount: buildCount, + }) + const client = createClient() + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => new Date(NOW.getTime() + elapsedIntervals * 15 * 60 * 1000), + }) + + expect(prisma.kBGraphBuild.findMany).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ skip: expectedSkip, take: 32 }) + ) + }) + + it.each([ + { buildCount: 33, elapsedIntervals: 1, expectedSkip: 32 }, + { buildCount: 65, elapsedIntervals: 2, expectedSkip: 64 }, + ])('rotates the active graph monitor window for $buildCount builds', async ({ + buildCount, + elapsedIntervals, + expectedSkip, + }) => { + const prisma = createMonitorPrisma([], { activeBuildCount: buildCount }) + const client = createClient() + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => new Date(NOW.getTime() + elapsedIntervals * 15 * 60 * 1000), + }) + + expect(prisma.kBGraphBuild.findMany).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ skip: expectedSkip, take: 32 }) + ) + }) + + it('limits active provider status checks to eight at a time', async () => { + const builds = Array.from({ length: 16 }, (_, index) => ({ + id: `build-${index}`, + kbId: KB_ID, + externalOperationId: `external-run-${index}`, + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + })) + const prisma = createMonitorPrisma(builds) + const client = createClient() + let activeCalls = 0 + let maxActiveCalls = 0 + vi.mocked(client.runs.get_status).mockImplementation(async () => { + activeCalls += 1 + maxActiveCalls = Math.max(maxActiveCalls, activeCalls) + await Promise.resolve() + activeCalls -= 1 + return 'QUEUED' + }) + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + }) + + expect(maxActiveCalls).toBe(8) + }) + + it('aborts timed-out provider operations before admitting the next build', async () => { + const builds = Array.from({ length: 9 }, (_, index) => ({ + id: `build-${index}`, + kbId: KB_ID, + externalOperationId: `external-run-${index}`, + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + })) + const prisma = createMonitorPrisma(builds) + const client = createClient() + let activeCalls = 0 + let maxActiveCalls = 0 + let callCount = 0 + vi.mocked(client.runs.get_status).mockImplementation((_runId, options) => { + callCount += 1 + activeCalls += 1 + maxActiveCalls = Math.max(maxActiveCalls, activeCalls) + if (callCount === 9) { + activeCalls -= 1 + return Promise.resolve('QUEUED') + } + + return new Promise<'QUEUED'>((_, reject) => { + const abort = () => { + activeCalls -= 1 + reject(options?.signal?.reason) + } + options?.signal?.addEventListener('abort', abort, { once: true }) + }) + }) + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + providerOperationTimeoutMs: 5, + }) + + expect(callCount).toBe(9) + expect(maxActiveCalls).toBe(8) + expect(activeCalls).toBe(0) + }) + + it('continues independent builds when one provider status call hangs', async () => { + const hungBuild = { + id: 'build-hung', + kbId: KB_ID, + externalOperationId: 'external-run-hung', + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + } + const readyBuild = { + id: 'build-ready', + kbId: KB_ID, + externalOperationId: 'external-run-ready', + externalStartedAt: new Date('2026-08-01T11:59:00.000Z'), + } + const prisma = createMonitorPrisma([hungBuild, readyBuild]) + const client = createClient() + vi.mocked(client.runs.get_status).mockImplementation((runId, options) => + runId === 'external-run-hung' + ? new Promise<'RUNNING'>((_, reject) => { + options?.signal?.addEventListener( + 'abort', + () => reject(options.signal?.reason), + { once: true } + ) + }) + : Promise.resolve('RUNNING') + ) + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + providerOperationTimeoutMs: 5, + }) + + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ id: 'build-ready' }), + data: expect.objectContaining({ + status: KBGraphBuildStatus.PROCESSING, + }), + }) + ) + expect(prisma.kBGraphBuild.updateMany).not.toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ id: 'build-hung' }), + }) + ) + }) + + it('frees a parked ambiguous build once the provider confirms no run exists', async () => { + const prisma = createMonitorPrisma([], { + ambiguousBuilds: [{ id: BUILD_ID, kbId: KB_ID, createdAt: CREATED_AT }], + }) + const client = createClient({ rows: [] }) + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + }) + + // The hold is a waiting state, not a permanent one: the sweep gives the + // lecturer their quota and build slot back without an operator step. + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + errorCode: 'KB_GRAPH_DISPATCH_FAILED', + }), + }) + ) + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it('does not publish a late completion without a versioned terminal result', async () => { + const prisma = createMonitorPrisma([], { + timedOutBuilds: [ + { + id: BUILD_ID, + kbId: KB_ID, + sourceContentDigest: SOURCE_DIGEST, + createdAt: CREATED_AT, + externalOperationId: 'external-run-id', + }, + ], + servingResources: [ + { id: RESOURCE_ID, activeContentSha256: CONTENT_SHA256 }, + ], + }) + const client = createClient() + vi.mocked(client.runs.get_status).mockResolvedValue('COMPLETED') + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + }) + + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_TIMEOUT', + cleanedAt: null, + cleanupStartedAt: null, + }), + data: expect.objectContaining({ + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_RESULT_REQUIRED', + }), + }) + ) + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) + + it('settles a late completion through the versioned terminal-result handoff', async () => { + const prisma = createMonitorPrisma([], { + timedOutBuilds: [ + { + id: BUILD_ID, + kbId: KB_ID, + sourceContentDigest: SOURCE_DIGEST, + createdAt: CREATED_AT, + externalOperationId: 'external-run-id', + }, + ], + servingResources: [ + { id: RESOURCE_ID, activeContentSha256: CONTENT_SHA256 }, + ], + }) + const client = createClient() + vi.mocked(client.runs.get_status).mockResolvedValue('COMPLETED') + const getTerminalResult = vi.fn().mockResolvedValue({ status: 'SUCCEEDED' }) + const settleTerminalResult = vi.fn().mockResolvedValue('SETTLED') + + await monitorActiveKBGraphBuilds({ + prisma: prisma as never, + client, + env: externalEnv, + now: () => NOW, + getTerminalResult, + settleTerminalResult, + }) + + expect(getTerminalResult).toHaveBeenCalledWith( + 'external-run-id', + expect.objectContaining({ signal: expect.any(AbortSignal) }) + ) + expect(settleTerminalResult).toHaveBeenCalledWith({ + buildId: BUILD_ID, + result: { status: 'SUCCEEDED' }, + finishedAt: NOW, + allowLateSuccess: true, + }) + expect(prisma.kBGraphBuild.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) +}) + +describe('KB graph build failure guard', () => { + it('marks only an uncorrelated active build as failed and releases its slot', async () => { + const prisma = createDispatchPrisma() + + await markKBGraphBuildDispatchFailed({ buildId: BUILD_ID }, prisma as never) + + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + id: BUILD_ID, + kbId: KB_ID, + externalOperationId: null, + }), + data: expect.objectContaining({ + status: KBGraphBuildStatus.FAILED, + errorCode: 'KB_GRAPH_DISPATCH_FAILED', + }), + }) + ) + expect(prisma.kB.updateMany).toHaveBeenCalledWith({ + where: { id: KB_ID, activeGraphBuildId: BUILD_ID }, + data: { activeGraphBuildId: null }, + }) + }) + + it('does not release a reservation when failure compensation loses the dispatch claim race', async () => { + const prisma = createDispatchPrisma({ updateCount: 0 }) + + await markKBGraphBuildDispatchFailed({ buildId: BUILD_ID }, prisma as never) + + expect(prisma.kBGraphQuota.updateMany).not.toHaveBeenCalled() + expect(prisma.kB.updateMany).not.toHaveBeenCalled() + }) +}) diff --git a/packages/hatchet/test/kbIngestion.test.ts b/packages/hatchet/test/kbIngestion.test.ts new file mode 100644 index 0000000000..51d0e0b0ef --- /dev/null +++ b/packages/hatchet/test/kbIngestion.test.ts @@ -0,0 +1,1074 @@ +import { + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, +} from '@klicker-uzh/prisma/client' +import type { + DeleteKBResourceInput, + IngestKBResourceInput, +} from '@klicker-uzh/types' +import { + MAX_KB_SOURCE_SIZE_BYTES, + MAX_KB_TOTAL_SIZE_BYTES, +} from '@klicker-uzh/types' +import { describe, expect, it, vi } from 'vitest' +import { + dispatchKBDeletion, + dispatchKBIngestion, + failKBIngestionDispatch, + monitorActiveKBIngestions, + retainFailedKBDeletionDispatch, + validateKBIngestionWorkerConfig, +} from '../src/kbIngestion.js' +import type { + KBIngestionApiClient, + KBIngestionSource, + KBOperationStatusResponse, +} from '../src/kbIngestionApi.js' + +const RESOURCE_ID = '7f3e2a10-9c4b-4d8e-b1a6-5e0f9d2c7b3a' +const KB_ID = 'c2a91f74-6e0b-4c3d-8f5a-1b9e7d4a2c60' +const ATTEMPT_ID = 'b5d4c3a2-1f0e-4d9c-8b7a-6e5f4d3c2b1a' +const OPERATION_ID = 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND' +const CONTENT_SHA256 = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' +const NOW = new Date('2026-07-26T12:00:00.000Z') + +const input = { + resourceId: RESOURCE_ID, + kbId: KB_ID, + title: 'Lecture 1', + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + type: 'URL', + sourceUrl: 'https://example.com/lecture.txt', +} satisfies IngestKBResourceInput + +const deletionInput = { + resourceId: RESOURCE_ID, + kbId: KB_ID, + deletionAttemptId: ATTEMPT_ID, + resourceVersion: 4, +} satisfies DeleteKBResourceInput + +const source = { + kind: 'url', + url: input.sourceUrl, + mimeType: 'text/plain', + displayName: input.title, + contentSha256: CONTENT_SHA256, + sizeBytes: 1024, +} satisfies KBIngestionSource + +function operation( + overrides: Partial = {} +): KBOperationStatusResponse { + return { + operationId: OPERATION_ID, + status: 'running', + operation: 'update', + projectId: 'klicker-course-materials', + producer: 'klicker', + externalResourceId: RESOURCE_ID, + resourceVersion: 3, + expectedSha256: CONTENT_SHA256, + observedSha256: null, + serving: { + activeResourceVersion: 2, + activeSha256: 'a'.repeat(64), + }, + errorCode: null, + correlationId: 'correlation-id', + createdAt: '2026-07-26T11:59:00Z', + updatedAt: '2026-07-26T12:00:00Z', + ...overrides, + } +} + +function client( + overrides: Partial = {} +): KBIngestionApiClient { + return { + acceptResource: vi.fn().mockResolvedValue(OPERATION_ID), + deleteResource: vi.fn().mockResolvedValue(OPERATION_ID), + getOperation: vi.fn().mockResolvedValue(operation()), + ...overrides, + } +} + +function dispatchPrisma( + resource: Record, + updateResults: number[] = [1, 1], + quota: { + resourceBytes?: number + unknownSizeCount?: number + ticketBytes?: number + } = {} +) { + const persistedResource = { + kbId: KB_ID, + deletedAt: null, + kb: { deletedAt: null }, + sizeBytes: 1024, + ...resource, + } + const prisma = { + kBResource: { + findUnique: vi.fn().mockResolvedValue(persistedResource), + findFirst: vi.fn().mockResolvedValue(persistedResource), + aggregate: vi.fn().mockResolvedValue({ + _sum: { sizeBytes: quota.resourceBytes ?? 1024 }, + }), + count: vi.fn().mockResolvedValue(quota.unknownSizeCount ?? 0), + updateMany: vi.fn().mockImplementation(async () => ({ + count: updateResults.shift() ?? 0, + })), + }, + kBUploadTicket: { + aggregate: vi.fn().mockResolvedValue({ + _sum: { sizeBytes: quota.ticketBytes ?? 0 }, + }), + }, + kBIngestionRun: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + $queryRaw: vi.fn().mockResolvedValue([{ id: KB_ID }]), + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + return prisma +} + +describe('KB ingestion dispatch', () => { + it('does not require external API config until the integration is configured', () => { + expect(() => validateKBIngestionWorkerConfig({})).not.toThrow() + }) + + it('fails fast when external API config is only partially configured', () => { + expect(() => + validateKBIngestionWorkerConfig({ + KB_INGESTION_API_URL: 'https://ingestion.example', + }) + ).toThrow('KB_INGESTION_API_KEY must be configured') + }) + + it('prepares source bytes, awaits API acceptance, and persists correlation', async () => { + const prisma = dispatchPrisma({ + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + externalOperationId: null, + }) + const apiClient = client() + const prepareSource = vi.fn().mockResolvedValue(source) + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource, + now: () => NOW, + }) + ).resolves.toBe(OPERATION_ID) + + expect(prepareSource).toHaveBeenCalledWith(input, process.env) + expect(prisma.kBResource.updateMany).toHaveBeenNthCalledWith(1, { + where: { + id: RESOURCE_ID, + kbId: KB_ID, + deletedAt: null, + kb: { deletedAt: null }, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + contentSha256: null, + externalOperationId: null, + }, + data: { + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + }, + }) + expect(apiClient.acceptResource).toHaveBeenCalledWith({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 3, + ingestionAttemptId: ATTEMPT_ID, + source, + }) + expect(prisma.kBResource.updateMany).toHaveBeenNthCalledWith(2, { + where: { + id: RESOURCE_ID, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + contentSha256: CONTENT_SHA256, + externalOperationId: null, + }, + data: { + externalOperationId: OPERATION_ID, + externalOperationStartedAt: NOW, + }, + }) + }) + + it('reuses persisted source identity on an idempotent task retry', async () => { + const prisma = dispatchPrisma( + { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + externalOperationId: null, + }, + [1] + ) + const apiClient = client() + const prepareSource = vi.fn() + + await dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource, + now: () => NOW, + }) + + expect(prepareSource).not.toHaveBeenCalled() + expect(apiClient.acceptResource).toHaveBeenCalledWith( + expect.objectContaining({ source }) + ) + }) + + it('returns an already correlated operation without another API call', async () => { + const prisma = dispatchPrisma({ + status: KBResourceStatus.PROCESSING, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + externalOperationId: OPERATION_ID, + }) + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + }) + ).resolves.toBe(OPERATION_ID) + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('does not dispatch a stale attempt or version', async () => { + const prisma = dispatchPrisma({ + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + contentSha256: null, + mimeType: null, + sizeBytes: null, + externalOperationId: null, + }) + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + }) + ).resolves.toBeUndefined() + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('rejects a payload whose KB scope does not match the persisted resource', async () => { + const prisma = dispatchPrisma({ + kbId: '5190edaa-2e7e-4828-a209-968a597e65b9', + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + sizeBytes: null, + externalOperationId: null, + }) + const apiClient = client() + const prepareSource = vi.fn() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource, + }) + ).resolves.toBeUndefined() + + expect(prepareSource).not.toHaveBeenCalled() + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('replaces the previous URL size without double counting at the quota boundary', async () => { + const prisma = dispatchPrisma( + { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + sizeBytes: 1000, + externalOperationId: null, + }, + [1, 1], + { resourceBytes: MAX_KB_TOTAL_SIZE_BYTES - 500 } + ) + const boundarySource = { ...source, sizeBytes: 1500 } + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource: vi.fn().mockResolvedValue(boundarySource), + }) + ).resolves.toBe(OPERATION_ID) + + expect(apiClient.acceptResource).toHaveBeenCalledWith( + expect.objectContaining({ source: boundarySource }) + ) + }) + + it('replaces the conservative reservation for a legacy unknown-size URL', async () => { + const prisma = dispatchPrisma( + { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + sizeBytes: null, + externalOperationId: null, + }, + [1, 1], + { + resourceBytes: 0, + unknownSizeCount: 20, + } + ) + const observedSource = { ...source, sizeBytes: 1 } + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource: vi.fn().mockResolvedValue(observedSource), + }) + ).resolves.toBe(OPERATION_ID) + + expect(MAX_KB_SOURCE_SIZE_BYTES * 20).toBe(MAX_KB_TOTAL_SIZE_BYTES) + expect(apiClient.acceptResource).toHaveBeenCalledWith( + expect.objectContaining({ source: observedSource }) + ) + }) + + it('records a stable failure before dispatch when a URL replacement exceeds quota', async () => { + const prisma = dispatchPrisma( + { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + sizeBytes: 1000, + externalOperationId: null, + }, + [1], + { resourceBytes: MAX_KB_TOTAL_SIZE_BYTES - 500 } + ) + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource: vi + .fn() + .mockResolvedValue({ ...source, sizeBytes: 1501 }), + }) + ).resolves.toBeUndefined() + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith({ + where: expect.objectContaining({ + id: RESOURCE_ID, + kbId: KB_ID, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + }), + data: { + status: KBResourceStatus.FAILED, + statusMessage: 'The knowledge base storage limit was reached.', + errorCode: 'KB_STORAGE_LIMIT_REACHED', + }, + }) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith({ + where: expect.objectContaining({ + id: ATTEMPT_ID, + resourceId: RESOURCE_ID, + resourceVersion: 3, + }), + data: { + status: KBIngestionStatus.FAILED, + statusMessage: 'The knowledge base storage limit was reached.', + errorCode: 'KB_STORAGE_LIMIT_REACHED', + finishedAt: expect.any(Date), + }, + }) + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('rolls back an over-limit failure that cannot correlate its run', async () => { + const prisma = dispatchPrisma( + { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: null, + mimeType: null, + sizeBytes: 1000, + externalOperationId: null, + }, + [1], + { resourceBytes: MAX_KB_TOTAL_SIZE_BYTES } + ) + prisma.kBIngestionRun.updateMany.mockResolvedValueOnce({ count: 0 }) + const apiClient = client() + + await expect( + dispatchKBIngestion(input, { + prisma: prisma as never, + client: apiClient, + prepareSource: vi + .fn() + .mockResolvedValue({ ...source, sizeBytes: 1001 }), + }) + ).rejects.toThrow('KB ingestion dispatch failed') + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('marks only an unaccepted current attempt failed after task retries', async () => { + const prisma = { + kBResource: { updateMany: vi.fn().mockResolvedValue({ count: 1 }) }, + kBIngestionRun: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + + await failKBIngestionDispatch({ input, prisma: prisma as never }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith({ + where: { + id: RESOURCE_ID, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + externalOperationId: null, + status: { + in: [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + data: { + status: KBResourceStatus.FAILED, + statusMessage: 'The ingestion operation could not be started.', + errorCode: 'INGESTION_DISPATCH_FAILED', + }, + }) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith({ + where: { + id: ATTEMPT_ID, + resourceId: RESOURCE_ID, + operation: 'UPSERT', + resourceVersion: 3, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: KBIngestionStatus.FAILED, + statusMessage: 'The ingestion operation could not be started.', + errorCode: 'INGESTION_DISPATCH_FAILED', + finishedAt: expect.any(Date), + }, + }) + }) +}) + +describe('KB deletion dispatch', () => { + it('dispatches a current tombstone and persists its operation correlation', async () => { + const prisma = dispatchPrisma({ + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }) + const apiClient = client() + + await expect( + dispatchKBDeletion(deletionInput, { + prisma: prisma as never, + client: apiClient, + now: () => NOW, + }) + ).resolves.toBe(OPERATION_ID) + + expect(apiClient.deleteResource).toHaveBeenCalledWith(deletionInput) + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith({ + where: { + id: RESOURCE_ID, + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }, + data: { + externalOperationId: OPERATION_ID, + externalOperationStartedAt: NOW, + statusMessage: null, + errorCode: null, + }, + }) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith({ + where: { + id: ATTEMPT_ID, + resourceId: RESOURCE_ID, + operation: KBIngestionOperation.DELETE, + resourceVersion: 4, + status: { + in: [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + externalOperationId: OPERATION_ID, + startedAt: NOW, + statusMessage: null, + errorCode: null, + }, + }) + }) + + it('does not redispatch a stale deletion attempt', async () => { + const prisma = dispatchPrisma({ + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: '77996ac1-ad9a-4379-8ff8-2a07d2184a31', + resourceVersion: 4, + externalOperationId: null, + }) + const apiClient = client() + + await expect( + dispatchKBDeletion(deletionInput, { + prisma: prisma as never, + client: apiClient, + }) + ).resolves.toBeUndefined() + expect(apiClient.deleteResource).not.toHaveBeenCalled() + }) + + it('does not dispatch a deletion for a different knowledge base', async () => { + const prisma = dispatchPrisma({ + kbId: '4dad13f2-1c45-47b3-b08a-1bc9cf4c5c47', + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }) + const apiClient = client() + + await expect( + dispatchKBDeletion(deletionInput, { + prisma: prisma as never, + client: apiClient, + }) + ).resolves.toBeUndefined() + expect(apiClient.deleteResource).not.toHaveBeenCalled() + }) + + it('keeps a failed deletion hidden and retryable', async () => { + const prisma = dispatchPrisma({}, [1]) + + await retainFailedKBDeletionDispatch({ + input: deletionInput, + prisma: prisma as never, + }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBResourceStatus.QUEUED, + errorCode: 'DELETION_DISPATCH_FAILED', + }), + }) + ) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBIngestionStatus.QUEUED, + errorCode: 'DELETION_DISPATCH_FAILED', + }), + }) + ) + }) +}) + +describe('KB ingestion reconciliation', () => { + function monitorPrisma(resources: Record[]) { + const prisma = { + kBResource: { + count: vi.fn().mockResolvedValue(resources.length), + findMany: vi + .fn() + .mockImplementation(async ({ skip = 0, take = resources.length }) => + resources.slice(skip, skip + take) + ), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBIngestionRun: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + return prisma + } + + const activeResource = { + id: RESOURCE_ID, + kbId: KB_ID, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + externalOperationId: OPERATION_ID, + } + + it('reconciles a succeeded delete only after serving is empty', async () => { + const deletedResource = { + ...activeResource, + resourceVersion: 4, + contentSha256: null, + ingestionOperation: KBIngestionOperation.DELETE, + } + const prisma = monitorPrisma([deletedResource]) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + status: 'succeeded', + operation: 'delete', + resourceVersion: 4, + expectedSha256: null, + observedSha256: null, + serving: { + activeResourceVersion: null, + activeSha256: null, + }, + }) + ), + }), + }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + ingestionOperation: KBIngestionOperation.DELETE, + contentSha256: null, + }), + data: expect.objectContaining({ + status: KBResourceStatus.READY, + activeResourceVersion: null, + activeContentSha256: null, + }), + }) + ) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ + operation: KBIngestionOperation.DELETE, + }), + data: expect.objectContaining({ + status: KBIngestionStatus.SUCCEEDED, + }), + }) + ) + }) + + it('keeps a succeeded delete processing while old content still serves', async () => { + const deletedResource = { + ...activeResource, + resourceVersion: 4, + contentSha256: null, + ingestionOperation: KBIngestionOperation.DELETE, + } + const prisma = monitorPrisma([deletedResource]) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + status: 'succeeded', + operation: 'delete', + resourceVersion: 4, + expectedSha256: null, + observedSha256: null, + serving: { + activeResourceVersion: 3, + activeSha256: CONTENT_SHA256, + }, + }) + ), + }), + }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBResourceStatus.PROCESSING, + }), + }) + ) + }) + + it.each([ + ['accepted', KBResourceStatus.QUEUED, KBIngestionStatus.QUEUED, null], + [ + 'running', + KBResourceStatus.PROCESSING, + KBIngestionStatus.PROCESSING, + null, + ], + [ + 'failed', + KBResourceStatus.FAILED, + KBIngestionStatus.FAILED, + 'The ingestion operation failed.', + ], + [ + 'superseded', + KBResourceStatus.FAILED, + KBIngestionStatus.SUPERSEDED, + 'The ingestion operation was superseded.', + ], + ] as const)('maps %s operation status to %s', async (externalStatus, localStatus, runStatus, statusMessage) => { + const prisma = monitorPrisma([activeResource]) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi + .fn() + .mockResolvedValue(operation({ status: externalStatus })), + }), + }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith({ + where: { + id: RESOURCE_ID, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + externalOperationId: OPERATION_ID, + ingestionOperation: 'UPSERT', + status: { + in: + externalStatus === 'accepted' + ? [KBResourceStatus.QUEUED] + : [KBResourceStatus.QUEUED, KBResourceStatus.PROCESSING], + }, + }, + data: { + status: localStatus, + statusMessage, + errorCode: null, + activeResourceVersion: 2, + activeContentSha256: 'a'.repeat(64), + }, + }) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith({ + where: { + id: ATTEMPT_ID, + resourceId: RESOURCE_ID, + operation: 'UPSERT', + resourceVersion: 3, + status: { + in: + externalStatus === 'accepted' + ? [KBIngestionStatus.QUEUED] + : [KBIngestionStatus.QUEUED, KBIngestionStatus.PROCESSING], + }, + }, + data: { + status: runStatus, + statusMessage, + errorCode: null, + ...(externalStatus === 'failed' || externalStatus === 'superseded' + ? { finishedAt: new Date('2026-07-26T12:00:00Z') } + : {}), + }, + }) + }) + + it('marks a succeeded operation ready with an ingestion timestamp', async () => { + const prisma = monitorPrisma([activeResource]) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + status: 'succeeded', + observedSha256: CONTENT_SHA256, + serving: { + activeResourceVersion: 3, + activeSha256: CONTENT_SHA256, + }, + }) + ), + }), + }) + + const update = prisma.kBResource.updateMany.mock.calls[0]![0] + expect(update.data).toMatchObject({ + status: KBResourceStatus.READY, + statusMessage: null, + activeResourceVersion: 3, + activeContentSha256: CONTENT_SHA256, + errorCode: null, + }) + expect(update.data.ingestedAt).toEqual(new Date('2026-07-26T12:00:00.000Z')) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith({ + where: { + id: ATTEMPT_ID, + resourceId: RESOURCE_ID, + operation: 'UPSERT', + resourceVersion: 3, + status: { + in: [ + KBIngestionStatus.QUEUED, + KBIngestionStatus.PROCESSING, + KBIngestionStatus.SUCCEEDED, + ], + }, + }, + data: { + status: KBIngestionStatus.SUCCEEDED, + statusMessage: null, + errorCode: null, + finishedAt: new Date('2026-07-26T12:00:00.000Z'), + }, + }) + }) + + it('refuses success when the observed digest does not match', async () => { + const prisma = monitorPrisma([activeResource]) + const logger = { error: vi.fn() } + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + status: 'succeeded', + observedSha256: null, + serving: { + activeResourceVersion: 3, + activeSha256: CONTENT_SHA256, + }, + }) + ), + }), + logger, + }) + + expect(prisma.kBResource.updateMany).not.toHaveBeenCalled() + expect(logger.error).toHaveBeenCalledWith( + 'KB ingestion observed digest correlation failed', + { + resourceId: RESOURCE_ID, + kbId: KB_ID, + ingestionAttemptId: ATTEMPT_ID, + } + ) + }) + + it.each([ + { + activeResourceVersion: 2, + activeSha256: 'a'.repeat(64), + }, + { + activeResourceVersion: 3, + activeSha256: 'a'.repeat(64), + }, + ])('records successful operation while serving cutover is pending', async (serving) => { + const prisma = monitorPrisma([activeResource]) + const logger = { info: vi.fn() } + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + status: 'succeeded', + observedSha256: CONTENT_SHA256, + serving, + }) + ), + }), + logger, + }) + + expect(prisma.kBResource.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBResourceStatus.PROCESSING, + activeResourceVersion: serving.activeResourceVersion, + activeContentSha256: serving.activeSha256, + }), + }) + ) + expect(prisma.kBIngestionRun.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: expect.objectContaining({ + status: KBIngestionStatus.SUCCEEDED, + }), + }) + ) + expect(logger.info).toHaveBeenCalledWith( + 'KB ingestion succeeded while serving cutover is pending', + { + resourceId: RESOURCE_ID, + kbId: KB_ID, + ingestionAttemptId: ATTEMPT_ID, + } + ) + }) + + it('refuses a status response that does not match every correlation field', async () => { + const prisma = monitorPrisma([activeResource]) + const logger = { error: vi.fn() } + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ + getOperation: vi.fn().mockResolvedValue( + operation({ + externalResourceId: '99c15e36-62ce-4982-88f8-1f50ed9bf61e', + }) + ), + }), + logger, + }) + + expect(prisma.kBResource.updateMany).not.toHaveBeenCalled() + expect(logger.error).toHaveBeenCalledWith( + 'KB ingestion operation correlation failed', + { + resourceId: RESOURCE_ID, + kbId: KB_ID, + ingestionAttemptId: ATTEMPT_ID, + } + ) + }) + + it('bounds concurrent operation polls to eight', async () => { + const resources = Array.from({ length: 17 }, (_, index) => ({ + ...activeResource, + id: `00000000-0000-4000-8000-${String(index).padStart(12, '0')}`, + externalOperationId: `${OPERATION_ID}_${index}`, + })) + const prisma = monitorPrisma(resources) + let active = 0 + let peak = 0 + const getOperation = vi.fn().mockImplementation(async (operationId) => { + active += 1 + peak = Math.max(peak, active) + await Promise.resolve() + active -= 1 + const index = Number(operationId.split('_').at(-1)) + return operation({ + operationId, + externalResourceId: resources[index]!.id, + }) + }) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ getOperation }), + }) + + expect(getOperation).toHaveBeenCalledTimes(17) + expect(peak).toBe(8) + }) + + it('rotates a bounded 32-resource reconciliation window each minute', async () => { + const resources = Array.from({ length: 49 }, (_, index) => ({ + ...activeResource, + id: `00000000-0000-4000-8000-${String(index).padStart(12, '0')}`, + externalOperationId: `${OPERATION_ID}_${index}`, + })) + const prisma = { + kBResource: { + count: vi.fn().mockResolvedValue(resources.length), + findMany: vi.fn().mockImplementation(async ({ skip = 0, take }) => { + return resources.slice(skip, skip + take) + }), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBIngestionRun: { + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + const getOperation = vi.fn().mockImplementation(async (operationId) => { + const index = Number(operationId.split('_').at(-1)) + return operation({ + operationId, + externalResourceId: resources[index]!.id, + }) + }) + + await monitorActiveKBIngestions({ + prisma: prisma as never, + client: client({ getOperation }), + now: () => new Date(60_000), + }) + + expect(prisma.kBResource.findMany).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ skip: 32, take: 32 }) + ) + expect(prisma.kBResource.findMany).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ take: 15 }) + ) + expect(getOperation).toHaveBeenCalledTimes(32) + }) +}) diff --git a/packages/hatchet/test/kbIngestionApi.test.ts b/packages/hatchet/test/kbIngestionApi.test.ts new file mode 100644 index 0000000000..7f32c73c59 --- /dev/null +++ b/packages/hatchet/test/kbIngestionApi.test.ts @@ -0,0 +1,418 @@ +import { BlobServiceClient } from '@azure/storage-blob' +import type { IngestKBResourceInput } from '@klicker-uzh/types' +import { Readable } from 'node:stream' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + buildKBIngestionSource, + createKBIngestionApiClient, + prepareKBIngestionSource, + resolvePublicIPv4, +} from '../src/kbIngestionApi.js' + +const RESOURCE_ID = '7f3e2a10-9c4b-4d8e-b1a6-5e0f9d2c7b3a' +const KB_ID = 'c2a91f74-6e0b-4c3d-8f5a-1b9e7d4a2c60' +const ATTEMPT_ID = 'b5d4c3a2-1f0e-4d9c-8b7a-6e5f4d3c2b1a' +const CONTENT_SHA256 = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' + +const env = { + KB_INGESTION_API_URL: 'https://ingestion.example', + KB_INGESTION_API_KEY: 'api-key', + KB_SOURCE_GATEWAY_URL: 'http://klicker-backend.stg-klicker.svc:3000', + BLOB_STORAGE_ACCOUNT_NAME: 'kbaccount', + BLOB_STORAGE_ACCESS_KEY: Buffer.alloc(32).toString('base64'), + BLOB_STORAGE_ACCOUNT_URL: 'https://blob.klicker.kb-poc.localhost/kbaccount', + BLOB_STORAGE_INTERNAL_ACCOUNT_URL: 'http://kb-poc-azurite:10000/kbaccount', +} + +const source = { + kind: 'blob', + url: `http://klicker-backend.stg-klicker.svc:3000/api/ingestion/resources/${RESOURCE_ID}/versions/3`, + mimeType: 'application/pdf', + displayName: 'Lecture 1', + contentSha256: CONTENT_SHA256, + sizeBytes: 1024, +} as const + +const operationResponse = { + operation_id: 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND', + status: 'succeeded', + operation: 'update', + project_id: 'klicker-course-materials', + producer: 'klicker', + external_resource_id: RESOURCE_ID, + resource_version: 3, + expected_sha256: CONTENT_SHA256, + observed_sha256: CONTENT_SHA256, + serving: { + active_resource_version: 3, + active_sha256: CONTENT_SHA256, + }, + error_code: null, + correlation_id: ATTEMPT_ID, + created_at: '2026-07-12T14:03:21Z', + updated_at: '2026-07-12T14:04:52Z', +} + +afterEach(() => { + vi.restoreAllMocks() +}) + +describe('canonical ingestion API client', () => { + it('sends the create fixture field-for-field and awaits 202 acceptance', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 202, + json: vi.fn().mockResolvedValue({ + operation_id: 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND', + }), + }) + const client = createKBIngestionApiClient({ + env, + fetchRequest, + }) + + await expect( + client.acceptResource({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 3, + ingestionAttemptId: ATTEMPT_ID, + source, + }) + ).resolves.toBe('op_01J2X8K3M9QZ4R7T6V5W1Y0BND') + + expect(fetchRequest).toHaveBeenCalledOnce() + const [url, request] = fetchRequest.mock.calls[0]! + expect(url.toString()).toBe('https://ingestion.example/v1/resources') + expect(request).toMatchObject({ + method: 'POST', + headers: { + Authorization: 'Bearer api-key', + 'Content-Type': 'application/json', + 'Idempotency-Key': ATTEMPT_ID, + }, + }) + expect(JSON.parse(request.body)).toEqual({ + project_id: 'klicker-course-materials', + producer: 'klicker', + external_resource_id: RESOURCE_ID, + resource_version: 3, + scope: { kb_id: KB_ID }, + source: { + kind: 'blob', + url: source.url, + mime_type: 'application/pdf', + display_name: 'Lecture 1', + }, + content_sha256: CONTENT_SHA256, + }) + }) + + it('sends the canonical delete request with a stable idempotency key', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 202, + json: vi.fn().mockResolvedValue({ + operation_id: 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND', + }), + }) + const client = createKBIngestionApiClient({ env, fetchRequest }) + + await expect( + client.deleteResource({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 4, + deletionAttemptId: ATTEMPT_ID, + }) + ).resolves.toBe('op_01J2X8K3M9QZ4R7T6V5W1Y0BND') + + const [url, request] = fetchRequest.mock.calls[0]! + expect(url.toString()).toBe( + `https://ingestion.example/v1/resources/${RESOURCE_ID}` + ) + expect(request).toMatchObject({ + method: 'DELETE', + headers: { + Authorization: 'Bearer api-key', + 'Content-Type': 'application/json', + 'Idempotency-Key': ATTEMPT_ID, + }, + }) + expect(JSON.parse(request.body)).toEqual({ + project_id: 'klicker-course-materials', + producer: 'klicker', + resource_version: 4, + scope: { kb_id: KB_ID }, + }) + }) + + it('parses the canonical operation response for reconciliation', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 200, + json: vi.fn().mockResolvedValue(operationResponse), + }) + + await expect( + createKBIngestionApiClient({ env, fetchRequest }).getOperation( + 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND' + ) + ).resolves.toEqual({ + operationId: 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND', + status: 'succeeded', + operation: 'update', + projectId: 'klicker-course-materials', + producer: 'klicker', + externalResourceId: RESOURCE_ID, + resourceVersion: 3, + expectedSha256: CONTENT_SHA256, + observedSha256: CONTENT_SHA256, + serving: { + activeResourceVersion: 3, + activeSha256: CONTENT_SHA256, + }, + errorCode: null, + correlationId: ATTEMPT_ID, + createdAt: '2026-07-12T14:03:21Z', + updatedAt: '2026-07-12T14:04:52Z', + }) + }) + + it('rejects response drift and hides remote response details', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 202, + json: vi.fn().mockResolvedValue({ + operation_id: 'operation-id', + unexpected: true, + }), + }) + + await expect( + createKBIngestionApiClient({ env, fetchRequest }).acceptResource({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 3, + ingestionAttemptId: ATTEMPT_ID, + source, + }) + ).rejects.toThrow('Ingestion API returned an invalid response') + }) + + it('rejects a successful response with the wrong endpoint status', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 200, + json: vi.fn().mockResolvedValue({ + operation_id: 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND', + }), + }) + + await expect( + createKBIngestionApiClient({ env, fetchRequest }).acceptResource({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 3, + ingestionAttemptId: ATTEMPT_ID, + source, + }) + ).rejects.toThrow('Ingestion API request failed') + }) + + it('rejects timestamps without the contract-required timezone', async () => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 200, + json: vi.fn().mockResolvedValue({ + ...operationResponse, + created_at: '2026-07-12T14:03:21', + }), + }) + + await expect( + createKBIngestionApiClient({ env, fetchRequest }).getOperation( + operationResponse.operation_id + ) + ).rejects.toThrow('Ingestion API returned an invalid response') + }) + + it.each([ + { status: ['succeeded'] }, + { operation: ['update'] }, + ])('rejects non-string operation enums', async (override) => { + const fetchRequest = vi.fn().mockResolvedValue({ + ok: true, + status: 200, + json: vi.fn().mockResolvedValue({ + ...operationResponse, + ...override, + }), + }) + + await expect( + createKBIngestionApiClient({ env, fetchRequest }).getOperation( + operationResponse.operation_id + ) + ).rejects.toThrow('Ingestion API returned an invalid response') + }) +}) + +describe('ingestion source preparation', () => { + const blobInput = { + resourceId: RESOURCE_ID, + kbId: KB_ID, + title: 'Lecture 1', + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + type: 'BLOB', + containerName: 'kb-owner', + blobName: `${RESOURCE_ID}.pdf`, + mimeType: 'application/pdf', + sizeBytes: 7, + } satisfies IngestKBResourceInput + + it('hashes immutable blob bytes and builds the authenticated gateway URL', async () => { + let blobServiceUrl = '' + const getBlobClient = vi.fn().mockReturnValue({ + download: vi.fn().mockResolvedValue({ + contentLength: 7, + contentType: 'application/pdf', + readableStreamBody: Readable.from([Buffer.from('lecture')]), + }), + }) + vi.spyOn( + BlobServiceClient.prototype, + 'getContainerClient' + ).mockImplementation(function (this: BlobServiceClient) { + blobServiceUrl = this.url + return { getBlobClient } as never + }) + + await expect(prepareKBIngestionSource(blobInput, env)).resolves.toEqual({ + kind: 'blob', + url: source.url, + mimeType: 'application/pdf', + displayName: 'Lecture 1', + contentSha256: + '6bc636ff0103a2888fb38ca3c2bf3b1371110ceac5a104a519d85d39207732b0', + sizeBytes: 7, + }) + expect(blobServiceUrl).toBe('http://kb-poc-azurite:10000/kbaccount') + }) + + it('pins every public URL hop and hashes only supported response bytes', async () => { + const urlInput = { + resourceId: RESOURCE_ID, + kbId: KB_ID, + title: 'Lecture notes', + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + type: 'URL', + sourceUrl: 'https://example.com/notes', + } satisfies IngestKBResourceInput + const redirect = Object.assign(Readable.from([]), { + statusCode: 302, + headers: { location: 'https://cdn.example.com/notes.txt' }, + }) + const content = Object.assign(Readable.from([Buffer.from('notes')]), { + statusCode: 200, + headers: { + 'content-type': 'text/plain; charset=utf-8', + 'content-length': '5', + }, + }) + const resolvePublicIPv4 = vi + .fn() + .mockResolvedValueOnce('93.184.216.34') + .mockResolvedValueOnce('93.184.216.35') + const requestPinnedUrl = vi + .fn() + .mockResolvedValueOnce(redirect) + .mockResolvedValueOnce(content) + + await expect( + prepareKBIngestionSource(urlInput, env, { + resolvePublicIPv4, + requestPinnedUrl, + }) + ).resolves.toEqual({ + kind: 'url', + url: 'https://example.com/notes', + mimeType: 'text/plain', + displayName: 'Lecture notes', + contentSha256: + 'ab5aa97074c454a0632057e704220d9a6678fbf773a0a5806fc09b8173b07309', + sizeBytes: 5, + }) + expect(resolvePublicIPv4).toHaveBeenNthCalledWith(1, 'example.com') + expect(resolvePublicIPv4).toHaveBeenNthCalledWith(2, 'cdn.example.com') + expect(requestPinnedUrl).toHaveBeenNthCalledWith( + 1, + new URL('https://example.com/notes'), + '93.184.216.34' + ) + }) + + it('rejects persisted source identity with a non-canonical digest', () => { + expect(() => + buildKBIngestionSource( + blobInput, + 'application/pdf', + 'not-a-digest', + 1024, + env + ) + ).toThrow('KB ingestion source is invalid') + }) +}) + +// The test above exercises `preparePublicUrlSource` only through injected +// `resolvePublicIPv4`/`requestPinnedUrl` fakes, so the real DNS-rebinding +// guard in `resolvePublicIPv4` (SSRF protection: reject loopback, private, +// and link-local/metadata addresses even when they arrive via a normal- +// looking hostname) never actually ran. `resolvePublicIPv4` was exported +// (pure `export` keyword addition, no logic change) so it can be called +// directly here. Every case below is self-contained: `dns.lookup()` never +// performs a network round trip for an IP-literal "hostname" -- Node +// resolves it locally regardless of family -- so this needs no external +// DNS or network egress, verified empirically (0ms lookups) before writing +// these tests. +describe('resolvePublicIPv4 SSRF guard (real function, no fakes)', () => { + it.each([ + ['localhost', 'loopback hostname (resolves to 127.0.0.1 locally)'], + ['127.0.0.1', 'loopback literal'], + ['10.0.0.1', 'private class A'], + ['192.168.1.1', 'private class C'], + ['169.254.169.254', 'link-local / cloud metadata address'], + ])('rejects %s (%s)', async (hostname) => { + await expect(resolvePublicIPv4(hostname)).rejects.toThrow( + 'KB ingestion source URL is invalid' + ) + }) + + it('rejects an IPv6 literal presented as the hostname', async () => { + // Node's dns.lookup(hostname, { family: 4 }) does NOT fail or filter an + // IP-literal "hostname" to family 4 -- for a literal address it just + // returns that address verbatim, family 6 and all (confirmed directly: + // `dns.lookup('::1', { all: true, family: 4 })` resolves to + // `[{ address: '::1', family: 6 }]`, not an error and not an IPv4 + // address). The real rejection therefore has to come from the + // `isPublicIPv4Address` classification afterwards (it requires + // `isIP(value) === 4`), not from the forced-family lookup itself. This + // pins that the real function still rejects it end to end. + await expect(resolvePublicIPv4('::1')).rejects.toThrow( + 'KB ingestion source URL is invalid' + ) + }) + + it('accepts a public IPv4 literal, exercising the real classification logic with no DNS/network dependency', async () => { + // dns.lookup() short-circuits for IP literals without any I/O (0ms in a + // local trial), so this is not a disguised real-network test -- it + // proves the real success path of resolvePublicIPv4 deterministically. + await expect(resolvePublicIPv4('93.184.216.34')).resolves.toBe( + '93.184.216.34' + ) + }) +}) diff --git a/packages/hatchet/test/kbMaintenance.test.ts b/packages/hatchet/test/kbMaintenance.test.ts new file mode 100644 index 0000000000..89d197ec4f --- /dev/null +++ b/packages/hatchet/test/kbMaintenance.test.ts @@ -0,0 +1,861 @@ +import { + KBGraphBuildStatus, + KBIngestionOperation, + KBIngestionStatus, + KBResourceStatus, + KBResourceType, +} from '@klicker-uzh/prisma/client' +import { describe, expect, it, vi } from 'vitest' +import type { KBIngestionApiClient } from '../src/kbIngestionApi.js' +import { + KB_MAINTENANCE_INTERVAL_MS, + maintainKBResources, +} from '../src/kbMaintenance.js' + +const RESOURCE_ID = '7f3e2a10-9c4b-4d8e-b1a6-5e0f9d2c7b3a' +const KB_ID = 'c2a91f74-6e0b-4c3d-8f5a-1b9e7d4a2c60' +const OWNER_ID = 'f490ce41-bd11-42c1-b601-74bdbcd4d3d7' +const ATTEMPT_ID = 'b5d4c3a2-1f0e-4d9c-8b7a-6e5f4d3c2b1a' +const OPERATION_ID = 'op_01J2X8K3M9QZ4R7T6V5W1Y0BND' +const CONTENT_SHA256 = + '9b74c9897bac770ffc029102a200c5de11ba9dbd0e0f28c991eb64b0fb54d96e' +const NOW = new Date('2026-07-27T12:00:00.000Z') +const GRAPH_DELETION_GRACE_MS = 30 * 24 * 60 * 60 * 1000 +const BUILD_ID = 'f1fbd1fd-aabb-4dd4-8e64-2f9a13a971a6' +const GRAPH_NAME = `klickeruzh:kb:${KB_ID}:${BUILD_ID}` +const GRAPHML_BLOB_NAME = `knowledge-graphs/${BUILD_ID}.graphml` + +function client(): KBIngestionApiClient { + return { + acceptResource: vi.fn().mockResolvedValue(OPERATION_ID), + deleteResource: vi.fn().mockResolvedValue(OPERATION_ID), + getOperation: vi.fn(), + } +} + +function maintenancePrisma({ + pendingDispatch = [], + pendingDispatchCount = pendingDispatch.length, + pendingUpsertRetries = [], + pendingUpsertRetryCount = pendingUpsertRetries.length, + expiredTickets = [], + expiredTicketCount = expiredTickets.length, + deletedResources = [], + deletedResourceCount = deletedResources.length, + deletedKbs = [], + deletedKbCount = deletedKbs.length, + pendingGraphDispatch, + pendingGraphDispatchCount, + retainedGraphBuilds = [], + retainedGraphBuildCount = retainedGraphBuilds.length, + purgeableArchives = [], + purgeableArchiveCount = purgeableArchives.length, + currentResource, +}: { + pendingDispatch?: unknown[] + pendingDispatchCount?: number + pendingUpsertRetries?: unknown[] + pendingUpsertRetryCount?: number + expiredTickets?: unknown[] + expiredTicketCount?: number + deletedResources?: unknown[] + deletedResourceCount?: number + deletedKbs?: unknown[] + deletedKbCount?: number + pendingGraphDispatch?: unknown[] + pendingGraphDispatchCount?: number + retainedGraphBuilds?: unknown[] + retainedGraphBuildCount?: number + purgeableArchives?: unknown[] + purgeableArchiveCount?: number + currentResource?: unknown +} = {}) { + // The sweep queries KBGraphBuild once per pass, in order. The stranded-dispatch + // pass only runs when the caller wires `enqueueKBGraphBuild`, which is what + // supplying `pendingGraphDispatch` stands for here. + const graphBuildPages: Array<{ count: number; rows: unknown[] }> = [ + ...(pendingGraphDispatch + ? [ + { + count: pendingGraphDispatchCount ?? pendingGraphDispatch.length, + rows: pendingGraphDispatch, + }, + ] + : []), + { count: retainedGraphBuildCount, rows: retainedGraphBuilds }, + { count: purgeableArchiveCount, rows: purgeableArchives }, + ] + const graphBuildCount = vi.fn() + const graphBuildFindMany = vi.fn() + for (const page of graphBuildPages) { + graphBuildCount.mockResolvedValueOnce(page.count) + graphBuildFindMany.mockResolvedValueOnce(page.rows) + } + const prisma = { + kBResource: { + count: vi + .fn() + .mockResolvedValueOnce(pendingDispatchCount) + .mockResolvedValueOnce(pendingUpsertRetryCount) + .mockResolvedValueOnce(deletedResourceCount), + findMany: vi + .fn() + .mockResolvedValueOnce(pendingDispatch) + .mockResolvedValueOnce(pendingUpsertRetries) + .mockResolvedValueOnce(deletedResources), + findUnique: vi.fn().mockResolvedValue(currentResource), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + deleteMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBIngestionRun: { + create: vi.fn().mockResolvedValue({}), + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBUploadTicket: { + count: vi.fn().mockResolvedValue(expiredTicketCount), + findMany: vi.fn().mockResolvedValue(expiredTickets), + deleteMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kB: { + count: vi.fn().mockResolvedValue(deletedKbCount), + findMany: vi.fn().mockResolvedValue(deletedKbs), + deleteMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + kBGraphBuild: { + count: graphBuildCount, + findMany: graphBuildFindMany, + updateMany: vi.fn().mockResolvedValue({ count: 1 }), + }, + $transaction: vi.fn(), + } + prisma.$transaction.mockImplementation(async (callback) => callback(prisma)) + return prisma +} + +describe('KB retention maintenance', () => { + it('retries an undispatched tombstone with its stable attempt id', async () => { + const pending = { + id: RESOURCE_ID, + kbId: KB_ID, + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + ingestionRuns: [], + } + const prisma = maintenancePrisma({ + pendingDispatch: [pending], + currentResource: { + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }, + }) + const apiClient = client() + + await maintainKBResources({ + prisma: prisma as never, + client: apiClient, + now: () => NOW, + }) + + expect(apiClient.deleteResource).toHaveBeenCalledWith({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + deletionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + }) + }) + + it.each([ + KBIngestionStatus.FAILED, + KBIngestionStatus.SUPERSEDED, + ])('starts a fresh attempt after a terminal external delete result (%s)', async (terminalStatus) => { + const failedOperationId = 'op_01J2X8K3M9QZ4R7T6V5W1Y0OLD' + const failed = { + id: RESOURCE_ID, + kbId: KB_ID, + status: KBResourceStatus.FAILED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: failedOperationId, + ingestionRuns: [{ id: ATTEMPT_ID, status: terminalStatus }], + } + const prisma = maintenancePrisma({ pendingDispatch: [failed] }) + prisma.kBResource.findMany.mockReset() + prisma.kBResource.findMany + .mockImplementationOnce(async (args) => { + const terminalStatuses = args.select.ingestionRuns.where.status.in + return terminalStatuses.includes(terminalStatus) ? [failed] : [] + }) + .mockResolvedValueOnce([]) // no stranded UPSERT dispatches + .mockResolvedValueOnce([]) // no hard-deletable resources + prisma.kBResource.updateMany + .mockReset() + .mockImplementationOnce(async (args) => { + const terminalStatuses = args.where.ingestionRuns.some.status.in + return { count: terminalStatuses.includes(terminalStatus) ? 1 : 0 } + }) + .mockResolvedValue({ count: 1 }) + prisma.kBResource.findUnique.mockImplementation(async () => { + const retryAttemptId = + prisma.kBIngestionRun.create.mock.calls[0]?.[0].data.id + return { + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: retryAttemptId, + resourceVersion: 4, + externalOperationId: null, + } + }) + const apiClient = client() + + await maintainKBResources({ + prisma: prisma as never, + client: apiClient, + now: () => NOW, + }) + + const retryAttemptId = + prisma.kBIngestionRun.create.mock.calls[0]?.[0].data.id + expect(retryAttemptId).toEqual(expect.any(String)) + expect(retryAttemptId).not.toBe(ATTEMPT_ID) + expect(prisma.kBResource.findMany).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ + select: expect.objectContaining({ + ingestionRuns: expect.objectContaining({ + where: { + operation: KBIngestionOperation.DELETE, + status: { + in: [KBIngestionStatus.FAILED, KBIngestionStatus.SUPERSEDED], + }, + }, + }), + }), + }) + ) + expect(prisma.kBResource.updateMany).toHaveBeenNthCalledWith(1, { + where: { + id: RESOURCE_ID, + deletedAt: { not: null }, + status: KBResourceStatus.FAILED, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: failedOperationId, + ingestionRuns: { + some: { + id: ATTEMPT_ID, + operation: KBIngestionOperation.DELETE, + status: { + in: [KBIngestionStatus.FAILED, KBIngestionStatus.SUPERSEDED], + }, + }, + }, + }, + data: { + status: KBResourceStatus.QUEUED, + statusMessage: 'The deletion operation is awaiting retry.', + ingestionAttemptId: retryAttemptId, + externalOperationId: null, + externalOperationStartedAt: null, + errorCode: null, + }, + }) + expect(apiClient.deleteResource).toHaveBeenCalledWith({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + deletionAttemptId: retryAttemptId, + resourceVersion: 4, + }) + }) + + it('continues independent cleanup when deletion dispatch is not configured', async () => { + const expiresAt = new Date(NOW.getTime() - 25 * 60 * 60 * 1000) + const pending = { + id: RESOURCE_ID, + kbId: KB_ID, + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + ingestionRuns: [], + } + const ticket = { + id: '77996ac1-ad9a-4379-8ff8-2a07d2184a31', + blobName: 'abandoned.pdf', + expiresAt, + kb: { ownerId: OWNER_ID }, + } + const prisma = maintenancePrisma({ + pendingDispatch: [pending], + currentResource: { + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }, + expiredTickets: [ticket], + }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + env: {}, + now: () => NOW, + deleteBlob, + }) + + expect(deleteBlob).toHaveBeenCalledWith(OWNER_ID, ticket.blobName) + expect(prisma.kBUploadTicket.deleteMany).toHaveBeenCalledOnce() + }) + + it.each([ + { pendingDispatchCount: 33, elapsedIntervals: 1, expectedSkip: 32 }, + { pendingDispatchCount: 65, elapsedIntervals: 2, expectedSkip: 64 }, + ])('rotates bounded retries so retained failures cannot starve later rows ($pendingDispatchCount rows)', async ({ + pendingDispatchCount, + elapsedIntervals, + expectedSkip, + }) => { + const pending = { + id: RESOURCE_ID, + kbId: KB_ID, + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + ingestionRuns: [], + } + const prisma = maintenancePrisma({ + pendingDispatch: [pending], + pendingDispatchCount, + currentResource: { + kbId: KB_ID, + deletedAt: NOW, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 4, + externalOperationId: null, + }, + }) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => new Date(NOW.getTime() + elapsedIntervals * 15 * 60 * 1000), + }) + + expect(prisma.kBResource.findMany).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ skip: expectedSkip, take: 32 }) + ) + }) + + it('deletes an abandoned blob before consuming its expired ticket', async () => { + const expiresAt = new Date(NOW.getTime() - 24 * 60 * 60 * 1000 - 1) + const ticket = { + id: ATTEMPT_ID, + blobName: `${ATTEMPT_ID}.pdf`, + expiresAt, + kb: { ownerId: OWNER_ID }, + } + const prisma = maintenancePrisma({ expiredTickets: [ticket] }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + }) + + expect(deleteBlob).toHaveBeenCalledWith(OWNER_ID, ticket.blobName) + expect(prisma.kBUploadTicket.deleteMany).toHaveBeenCalledWith({ + where: { + id: ticket.id, + blobName: ticket.blobName, + expiresAt, + }, + }) + expect(deleteBlob.mock.invocationCallOrder[0]).toBeLessThan( + prisma.kBUploadTicket.deleteMany.mock.invocationCallOrder[0]! + ) + expect(prisma.kBUploadTicket.findMany).toHaveBeenCalledWith({ + where: { + expiresAt: { + lte: new Date(NOW.getTime() - 24 * 60 * 60 * 1000), + }, + }, + select: { + id: true, + blobName: true, + expiresAt: true, + kb: { select: { ownerId: true } }, + }, + orderBy: { id: 'asc' }, + take: 32, + }) + }) + + it('retains an upload ticket when blob cleanup fails', async () => { + const ticket = { + id: ATTEMPT_ID, + blobName: `${ATTEMPT_ID}.pdf`, + expiresAt: new Date(NOW.getTime() - 25 * 60 * 60 * 1000), + kb: { ownerId: OWNER_ID }, + } + const prisma = maintenancePrisma({ expiredTickets: [ticket] }) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob: vi.fn().mockRejectedValue(new Error('storage unavailable')), + }) + + expect(prisma.kBUploadTicket.deleteMany).not.toHaveBeenCalled() + }) + + it('hard-deletes a tombstoned blob only after its current delete succeeded', async () => { + const resource = { + id: RESOURCE_ID, + type: KBResourceType.BLOB, + blobName: `${RESOURCE_ID}.pdf`, + ingestionAttemptId: ATTEMPT_ID, + kb: { ownerId: OWNER_ID }, + ingestionRuns: [{ id: ATTEMPT_ID }], + } + const prisma = maintenancePrisma({ deletedResources: [resource] }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + }) + + expect(deleteBlob).toHaveBeenCalledWith(OWNER_ID, resource.blobName) + expect(prisma.kBResource.deleteMany).toHaveBeenCalledWith({ + where: { + id: RESOURCE_ID, + deletedAt: { not: null }, + ingestionOperation: KBIngestionOperation.DELETE, + ingestionAttemptId: ATTEMPT_ID, + activeResourceVersion: null, + activeContentSha256: null, + ingestionRuns: { + some: { + id: ATTEMPT_ID, + operation: KBIngestionOperation.DELETE, + status: KBIngestionStatus.SUCCEEDED, + }, + }, + }, + }) + expect(deleteBlob.mock.invocationCallOrder[0]).toBeLessThan( + prisma.kBResource.deleteMany.mock.invocationCallOrder[0]! + ) + }) + + it('does not clean a resource whose current delete run has not succeeded', async () => { + const prisma = maintenancePrisma({ + deletedResources: [ + { + id: RESOURCE_ID, + type: KBResourceType.URL, + blobName: null, + ingestionAttemptId: ATTEMPT_ID, + kb: { ownerId: OWNER_ID }, + ingestionRuns: [{ id: '3139acdc-4639-4af3-9f6d-458867e09d98' }], + }, + ], + }) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob: vi.fn(), + }) + + expect(prisma.kBResource.deleteMany).not.toHaveBeenCalled() + }) + + it('hard-deletes a tombstoned URL without touching blob storage', async () => { + const resource = { + id: RESOURCE_ID, + type: KBResourceType.URL, + blobName: null, + ingestionAttemptId: ATTEMPT_ID, + kb: { ownerId: OWNER_ID }, + ingestionRuns: [{ id: ATTEMPT_ID }], + } + const prisma = maintenancePrisma({ deletedResources: [resource] }) + const deleteBlob = vi.fn() + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + }) + + expect(deleteBlob).not.toHaveBeenCalled() + expect(prisma.kBResource.deleteMany).toHaveBeenCalledOnce() + }) + + it('finalizes a pending knowledge base only after all children are gone', async () => { + const prisma = maintenancePrisma({ deletedKbs: [{ id: KB_ID }] }) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob: vi.fn(), + }) + + expect(prisma.kB.deleteMany).toHaveBeenCalledWith({ + where: expect.objectContaining({ + id: KB_ID, + resources: { none: {} }, + uploadTickets: { none: {} }, + chatbots: { none: { isEnabled: true } }, + // A build ledger row carries the settled cost evidence and cascades from + // the KB, so only a knowledge base that never built a graph is removed, + // and only once the recovery grace has expired. + graphBuilds: { none: {} }, + deletedAt: { + not: null, + lte: new Date(NOW.getTime() - GRAPH_DELETION_GRACE_MS), + }, + }), + }) + }) + + it('retires an unreferenced successful graph but keeps its GraphML archive', async () => { + const prisma = maintenancePrisma({ + retainedGraphBuilds: [ + { + id: BUILD_ID, + kbId: KB_ID, + status: KBGraphBuildStatus.SUCCEEDED, + graphName: GRAPH_NAME, + graphmlBlobName: GRAPHML_BLOB_NAME, + kb: { + ownerId: OWNER_ID, + activeGraphBuildId: null, + publishedGraphBuildId: null, + }, + }, + ], + }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + const deleteGraph = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + deleteGraph, + }) + + // The serving projection is reconstructible and goes after the short grace; + // the archive is the durable record and stays while the KB exists, so an + // earlier successful version remains restorable. + expect(deleteGraph).toHaveBeenCalledWith(GRAPH_NAME) + expect(deleteBlob).not.toHaveBeenCalled() + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: expect.objectContaining({ id: BUILD_ID, cleanedAt: null }), + data: { cleanedAt: NOW }, + }) + ) + expect(prisma.kBGraphBuild.updateMany.mock.calls[0]?.[0]).toEqual( + expect.objectContaining({ + where: expect.objectContaining({ + id: BUILD_ID, + cleanedAt: null, + OR: expect.arrayContaining([{ cleanupStartedAt: null }]), + }), + data: { cleanupStartedAt: NOW }, + }) + ) + }) + + it('purges the pinned artifact of a build that never produced an export', async () => { + const prisma = maintenancePrisma({ + retainedGraphBuilds: [ + { + id: BUILD_ID, + kbId: KB_ID, + status: KBGraphBuildStatus.FAILED, + graphName: GRAPH_NAME, + graphmlBlobName: GRAPHML_BLOB_NAME, + kb: { + ownerId: OWNER_ID, + activeGraphBuildId: null, + publishedGraphBuildId: null, + }, + }, + ], + }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + const deleteGraph = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + deleteGraph, + }) + + expect(deleteBlob).toHaveBeenCalledWith(OWNER_ID, GRAPHML_BLOB_NAME) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith( + expect.objectContaining({ + data: { cleanedAt: NOW, graphmlPurgedAt: NOW }, + }) + ) + }) + + it('purges a retained archive only once the deletion recovery grace expired', async () => { + const prisma = maintenancePrisma({ + purgeableArchives: [ + { + id: BUILD_ID, + kbId: KB_ID, + graphName: GRAPH_NAME, + graphmlBlobName: GRAPHML_BLOB_NAME, + cleanedAt: NOW, + kb: { ownerId: OWNER_ID }, + }, + ], + }) + const deleteBlob = vi.fn().mockResolvedValue(undefined) + const deleteGraph = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + deleteBlob, + deleteGraph, + }) + + // The pass is selected purely on how long the KB has been deleted, and it + // leaves the ledger row in place so the settled cost stays auditable. + expect(prisma.kBGraphBuild.findMany).toHaveBeenLastCalledWith( + expect.objectContaining({ + where: { + graphmlPurgedAt: null, + kb: { + deletedAt: { + lte: new Date(NOW.getTime() - GRAPH_DELETION_GRACE_MS), + }, + }, + }, + }) + ) + expect(deleteBlob).toHaveBeenCalledWith(OWNER_ID, GRAPHML_BLOB_NAME) + expect(prisma.kBGraphBuild.updateMany).toHaveBeenCalledWith({ + where: { id: BUILD_ID, kbId: KB_ID, graphmlPurgedAt: null }, + data: { graphmlPurgedAt: NOW }, + }) + }) + + it('re-enqueues a graph build stranded between its reservation and its dispatch', async () => { + const prisma = maintenancePrisma({ + pendingGraphDispatch: [{ id: BUILD_ID, kbId: KB_ID }], + }) + const enqueueKBGraphBuild = vi.fn().mockResolvedValue(undefined) + + await maintainKBResources({ + prisma: prisma as never, + client: client(), + now: () => NOW, + enqueueKBGraphBuild, + }) + + expect(prisma.kBGraphBuild.findMany).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ + where: { + status: KBGraphBuildStatus.QUEUED, + externalOperationId: null, + dispatchClaimedAt: null, + createdAt: { + lte: new Date(NOW.getTime() - KB_MAINTENANCE_INTERVAL_MS), + }, + }, + }) + ) + // Recovery reuses the stored build id, which is already the external + // idempotency key, so no second run and no second charge can start. + expect(enqueueKBGraphBuild).toHaveBeenCalledWith(BUILD_ID) + }) + + it('retries a stranded UPSERT dispatch stuck in the crash window with its stable attempt id', async () => { + const stranded = { + id: RESOURCE_ID, + kbId: KB_ID, + title: 'Lecture 1', + type: KBResourceType.URL, + blobName: null, + mimeType: null, + sizeBytes: null, + sourceUrl: 'https://example.com/lecture.txt', + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + kb: { ownerId: OWNER_ID }, + } + const prisma = maintenancePrisma({ + pendingUpsertRetries: [stranded], + currentResource: { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + kbId: KB_ID, + deletedAt: null, + kb: { deletedAt: null }, + externalOperationId: null, + }, + }) + const apiClient = client() + + await maintainKBResources({ + prisma: prisma as never, + client: apiClient, + now: () => NOW, + }) + + expect(apiClient.acceptResource).toHaveBeenCalledWith( + expect.objectContaining({ + resourceId: RESOURCE_ID, + kbId: KB_ID, + resourceVersion: 3, + ingestionAttemptId: ATTEMPT_ID, + source: expect.objectContaining({ + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + }), + }) + ) + // Recovery reuses the existing attempt: no new attempt id, no status + // transition performed by the maintenance sweep itself. + expect(prisma.kBIngestionRun.create).not.toHaveBeenCalled() + }) + + it('excludes upsert rows that are fresh, still in flight, tombstoned, or mid-deletion', async () => { + const prisma = maintenancePrisma() + const apiClient = client() + + await maintainKBResources({ + prisma: prisma as never, + client: apiClient, + now: () => NOW, + }) + + expect(prisma.kBResource.findMany).toHaveBeenNthCalledWith(2, { + where: { + deletedAt: null, + ingestionOperation: KBIngestionOperation.UPSERT, + status: KBResourceStatus.QUEUED, + externalOperationId: null, + ingestionAttemptId: { not: null }, + updatedAt: { + lte: new Date(NOW.getTime() - KB_MAINTENANCE_INTERVAL_MS), + }, + }, + select: { + id: true, + kbId: true, + title: true, + type: true, + blobName: true, + mimeType: true, + sizeBytes: true, + sourceUrl: true, + ingestionAttemptId: true, + resourceVersion: true, + kb: { select: { ownerId: true } }, + }, + orderBy: { id: 'asc' }, + take: 32, + }) + expect(apiClient.acceptResource).not.toHaveBeenCalled() + }) + + it('re-dispatches the same attempt id across repeated sweeps without minting a new attempt', async () => { + const stranded = { + id: RESOURCE_ID, + kbId: KB_ID, + title: 'Lecture 1', + type: KBResourceType.URL, + blobName: null, + mimeType: null, + sizeBytes: null, + sourceUrl: 'https://example.com/lecture.txt', + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + kb: { ownerId: OWNER_ID }, + } + const currentResource = { + status: KBResourceStatus.QUEUED, + ingestionAttemptId: ATTEMPT_ID, + resourceVersion: 3, + contentSha256: CONTENT_SHA256, + mimeType: 'text/plain', + sizeBytes: 1024, + kbId: KB_ID, + deletedAt: null, + kb: { deletedAt: null }, + externalOperationId: null, + } + const apiClient = client() + + // Two independent sweeps against the same still-stranded row (e.g. a + // second crash before the first sweep's dispatch could be correlated): + // both must reuse the identical attempt id, which the ingestion API + // dedupes on via its Idempotency-Key, so the double dispatch is harmless. + const firstPrisma = maintenancePrisma({ + pendingUpsertRetries: [stranded], + currentResource, + }) + await maintainKBResources({ + prisma: firstPrisma as never, + client: apiClient, + now: () => NOW, + }) + + const secondPrisma = maintenancePrisma({ + pendingUpsertRetries: [stranded], + currentResource, + }) + await maintainKBResources({ + prisma: secondPrisma as never, + client: apiClient, + now: () => NOW, + }) + + expect(apiClient.acceptResource).toHaveBeenCalledTimes(2) + for (const call of vi.mocked(apiClient.acceptResource).mock.calls) { + expect(call[0]).toMatchObject({ ingestionAttemptId: ATTEMPT_ID }) + } + expect(firstPrisma.kBIngestionRun.create).not.toHaveBeenCalled() + expect(secondPrisma.kBIngestionRun.create).not.toHaveBeenCalled() + }) +}) diff --git a/packages/hatchet/vitest.config.ts b/packages/hatchet/vitest.config.ts new file mode 100644 index 0000000000..837d9ce393 --- /dev/null +++ b/packages/hatchet/vitest.config.ts @@ -0,0 +1,20 @@ +import { defineConfig } from 'vitest/config' + +export default defineConfig({ + test: { + globals: true, + environment: 'node', + testTimeout: 30000, + silent: false, + reporters: ['verbose'], + pool: 'forks', + poolOptions: { + forks: { + singleFork: true, + }, + }, + }, + resolve: { + conditions: ['node', 'import', 'default'], + }, +}) diff --git a/packages/i18n/messages/de.ts b/packages/i18n/messages/de.ts index c7af7c142c..756e50e11c 100644 --- a/packages/i18n/messages/de.ts +++ b/packages/i18n/messages/de.ts @@ -854,6 +854,15 @@ Andere Teilnehmende sehen nur Dein öffentliches **Teilnehmendenprofil**, einsch activeContext: 'Nutzt den aktuellen Seitenkontext', questionContext: 'Frage {currentStep}/{totalSteps}', noCourseChatbot: 'Für diesen Kurs ist noch kein Kurs-Chatbot verfügbar.', + retrieval: { + searching: 'Vorlesungsinhalte werden nach „{query}“ durchsucht…', + errorTitle: 'Suche nicht verfügbar', + errorDescription: + 'Die Vorlesungsinhalte konnten nicht durchsucht werden. Bitte versuchen Sie es erneut.', + contentTitle: 'Vorlesungsinhalte', + questionLabel: 'Frage', + noContent: 'Keine Inhalte verfügbar', + }, }, insights: { noCourseDataAvailable: @@ -1502,6 +1511,222 @@ Da die KlickerUZH-App noch nicht im iOS-App-Store verfügbar ist, folgen Sie die cohortSizeLabel: 'Vergleichsgruppe: {count} aktive Teilnehmende', }, }, + kb: { + title: 'Wissensdatenbanken', + create: 'Wissensdatenbank erstellen', + nameLabel: 'Name', + descriptionLabel: 'Beschreibung (optional)', + createSuccess: 'Wissensdatenbank wurde erstellt.', + createError: 'Die Wissensdatenbank konnte nicht erstellt werden.', + deleteTitle: 'Wissensdatenbank löschen', + deleteDescription: + '„{name}“ wird sofort ausgeblendet. Gespeicherte Dateien und der externe Index werden im Hintergrund entfernt. Diese Aktion kann nicht rückgängig gemacht werden.', + deleteSuccess: + 'Wissensdatenbank wurde entfernt. Die Bereinigung läuft im Hintergrund.', + deleteError: 'Die Wissensdatenbank konnte nicht gelöscht werden.', + emptyTitle: 'Noch keine Wissensdatenbanken', + emptyDescription: + 'Erstellen Sie eine Wissensdatenbank, um erste Ressourcen hinzuzufügen.', + noDescription: 'Keine Beschreibung', + loadError: 'Die Wissensdatenbanken konnten nicht geladen werden.', + searchKnowledgeBases: 'Wissensdatenbanken suchen', + searchKnowledgeBasesPlaceholder: 'Nach Name oder Beschreibung suchen', + noSearchResults: 'Keine passenden Wissensdatenbanken', + noSearchResultsDescription: + 'Versuchen Sie es mit einem anderen Namen oder einer anderen Beschreibung.', + searchResultCount: + '{count, plural, =0 {Keine Wissensdatenbanken} one {# Wissensdatenbank} other {# Wissensdatenbanken}}', + catalogMetrics: + '{resources, plural, one {# Ressource} other {# Ressourcen}} · {chatbots, plural, one {# verknüpfter Chatbot} other {# verknüpfte Chatbots}}', + loadMore: 'Weitere Wissensdatenbanken laden', + notFound: 'Die Wissensdatenbank konnte nicht gefunden werden.', + backToList: 'Zurück zu den Wissensdatenbanken', + metricsTitle: 'Nutzung und Verknüpfungen', + metricVisibleResources: 'Sichtbare Ressourcen', + metricReservedResources: + '{count, plural, =0 {Keine Upload-Reservierungen} one {# Upload-Reservierung} other {# Upload-Reservierungen}}', + metricStorage: 'Speicherlimit', + metricStorageBreakdown: + '{visible} sichtbar · {reserved} für Uploads reserviert', + unknownSizesReserved: + '{count, plural, one {# ältere Ressource reserviert bis zu 25 MiB} other {# ältere Ressourcen reservieren je bis zu 25 MiB}}', + metricPendingCleanup: 'Ausstehende Bereinigung', + metricPendingCleanupSize: '{size} warten auf die Bereinigung', + metricLinkedConsumers: 'Verknüpfte Chatbots', + metricQuotaResources: '{count} Ressourcen werden dem Limit angerechnet', + quotaReleaseMessage: + 'Gelöschte Ressourcen werden dem Limit weiterhin angerechnet, bis die Bereinigung im Hintergrund abgeschlossen ist.', + fileUploadTitle: 'Datei hochladen', + fileUploadDescription: 'Fügen Sie Kursmaterial von Ihrem Computer hinzu.', + fileDropPrompt: 'Datei hier ablegen oder zum Auswählen klicken', + fileUploadFormats: 'PDF, TXT oder MD · maximal 25 MB', + uploading: 'Wird hochgeladen…', + fileUploadSuccess: 'Datei wurde zur Wissensdatenbank hinzugefügt.', + fileUploadError: 'Die Datei konnte nicht hochgeladen werden.', + fileRejected: 'Wählen Sie eine unterstützte Datei mit maximal 25 MB.', + linkTitle: 'Link hinzufügen', + linkDescription: + 'Registrieren Sie eine Website oder Medienressource für die Verarbeitung.', + resourceTitleLabel: 'Titel', + urlLabel: 'URL', + invalidUrl: + 'Geben Sie eine gültige URL ein, die mit http:// oder https:// beginnt.', + linkSuccess: 'Link wurde zur Wissensdatenbank hinzugefügt.', + linkError: 'Der Link konnte nicht hinzugefügt werden.', + resourcesTitle: 'Ressourcen', + resourcesLoadError: 'Die Ressourcen konnten nicht geladen werden.', + searchResources: 'Ressourcen suchen', + searchResourcesPlaceholder: 'Titel, Dateiname oder URL suchen', + filterType: 'Typ', + filterStatus: 'Letzte Verarbeitung', + filterAll: 'Alle', + typeFile: 'Datei', + typeUrl: 'Link', + noResourceResults: 'Keine Ressourcen entsprechen diesen Filtern.', + resourceResultCount: + '{count, plural, =0 {Keine Ressourcen} one {# Ressource} other {# Ressourcen}}', + selectAllPage: 'Bis zu 50 verfügbare Ressourcen auswählen', + selectResource: '„{title}“ auswählen', + loadMoreResources: 'Weitere Ressourcen laden', + noResources: 'Es wurden noch keine Ressourcen hinzugefügt.', + updatedAt: 'Aktualisiert {date}', + statusAdded: 'Hinzugefügt', + statusQueued: 'In Warteschlange', + statusProcessing: 'In Verarbeitung', + statusReady: 'Bereit', + statusFailed: 'Fehlgeschlagen', + deleteResourceTitle: 'Ressource löschen', + deleteResourceDescription: + '„{title}“ wird sofort ausgeblendet. Die gespeicherte Datei und der externe Index werden im Hintergrund entfernt. Diese Aktion kann nicht rückgängig gemacht werden.', + deleteResourceSuccess: + 'Ressource wurde entfernt. Die Bereinigung läuft im Hintergrund.', + deleteResourceError: 'Die Ressource konnte nicht gelöscht werden.', + bulkDelete: 'Ausgewählte löschen ({count})', + bulkDeleteTitle: + '{count, plural, one {# Ressource löschen} other {# Ressourcen löschen}}', + bulkDeleteConfirm: + '{count, plural, one {Ressource löschen} other {# Ressourcen löschen}}', + bulkDeleteDescription: + '{count, plural, one {Die ausgewählte Ressource wird sofort ausgeblendet. Gespeicherte Dateien und externe Indizes werden im Hintergrund entfernt. Diese Aktion kann nicht rückgängig gemacht werden.} other {Die # ausgewählten Ressourcen werden sofort ausgeblendet. Gespeicherte Dateien und externe Indizes werden im Hintergrund entfernt. Diese Aktion kann nicht rückgängig gemacht werden.}}', + bulkDeleteSuccess: + '{count, plural, one {Ressource wurde entfernt. Die Bereinigung läuft im Hintergrund.} other {# Ressourcen wurden entfernt. Die Bereinigung läuft im Hintergrund.}}', + bulkDeleteError: + 'Das Löschen konnte nicht bestätigt werden. Aktualisieren Sie die Liste, bevor Sie es erneut versuchen.', + ingestResource: 'Verarbeiten', + retryIngestion: 'Erneut versuchen', + reingestResource: 'Neu verarbeiten', + ingestResourceSuccess: 'Ressource wurde zur Verarbeitung eingeplant.', + ingestResourceError: + 'Die Ressource konnte nicht zur Verarbeitung eingeplant werden.', + operationStatus: 'Letzte Verarbeitung', + operationInProgress: + 'Dieser Vorgang läuft im Hintergrund. Sie können diese Seite verlassen.', + backgroundOperationsMessage: + 'Ein oder mehrere Vorgänge laufen noch. Die Status werden automatisch aktualisiert und Sie können diese Seite sicher verlassen.', + servingStatus: 'Für KI verfügbar', + notServing: 'Noch nicht verfügbar', + servingCurrentVersion: 'Aktuelle Version {version}', + servingPreviousVersion: 'Version {version} bleibt verfügbar', + servingSince: 'Verfügbar seit {date}', + version: 'Version {version}', + recentAttempts: 'Letzte Versuche', + noRecentAttempts: 'Noch keine Verarbeitungsversuche.', + historyLoadError: 'Die letzten Versuche konnten nicht geladen werden.', + runStatusQueued: 'In Warteschlange', + runStatusProcessing: 'In Verarbeitung', + runStatusSucceeded: 'Erfolgreich', + runStatusFailed: 'Fehlgeschlagen', + runStatusSuperseded: 'Ersetzt', + ingestionStartError: + 'Der Verarbeitungsvorgang konnte nicht gestartet werden.', + storageLimitError: + 'Diese Ressource überschreitet das Speicherlimit von 500 MiB für die Wissensdatenbank.', + resourceLimitError: + 'Diese Wissensdatenbank hat ihr Limit von 100 Ressourcen erreicht.', + uploadMismatchError: + 'Die hochgeladene Datei stimmt nicht mehr mit ihrer Upload-Reservierung überein. Laden Sie sie erneut hoch.', + ingestionFailed: 'Der Verarbeitungsvorgang ist fehlgeschlagen.', + ingestionSuperseded: 'Der Verarbeitungsvorgang wurde ersetzt.', + inspectResource: 'Details', + inspectorTitle: 'Ressourcendetails', + sourceType: 'Quelltyp', + sourceLocation: 'Quell-URL', + fileName: 'Ursprünglicher Dateiname', + mimeType: 'Medientyp', + fileSize: 'Dateigrösse', + createdAt: 'Erstellt', + chatbotsTitle: 'Verknüpfte Chatbots', + chatbotsDescription: + 'Wählen Sie, welcher Chatbot diese Wissensdatenbank verwenden kann. Ein Chatbot kann jeweils eine Wissensdatenbank verwenden.', + chatbotsLoadError: 'Die Chatbots konnten nicht geladen werden.', + noChatbots: + 'Erstellen Sie einen Chatbot, bevor Sie eine Wissensdatenbank verknüpfen.', + chatbotSelectLabel: 'Chatbot', + chatbotSelectPlaceholder: 'Chatbot auswählen', + attachChatbot: 'Chatbot verknüpfen', + replaceChatbot: 'Wissensdatenbank ersetzen', + detachChatbot: 'Verknüpfung aufheben', + linkedChatbots: 'Verwendet diese Wissensdatenbank', + noLinkedChatbots: 'Kein Chatbot verwendet diese Wissensdatenbank.', + chatbotReplacementWarning: + 'Dieser Chatbot verwendet derzeit „{kbName}“. Durch die Verknüpfung wird diese Wissensdatenbank ersetzt.', + chatbotAttachSuccess: 'Chatbot wurde mit der Wissensdatenbank verknüpft.', + chatbotAttachError: 'Der Chatbot konnte nicht verknüpft werden.', + chatbotDetachSuccess: 'Die Verknüpfung des Chatbots wurde aufgehoben.', + chatbotDetachError: + 'Die Verknüpfung des Chatbots konnte nicht aufgehoben werden.', + previewAccessError: + 'Der Wissensdatenbank-Arbeitsbereich ist für Ihr Konto noch nicht verfügbar.', + graphTitle: 'Wissensgraph', + graphDescription: + 'Erstellen Sie aus den Ressourcen der Wissensdatenbank einen Graphen und prüfen Sie das veröffentlichte Ergebnis.', + graphQualityTierLabel: 'Qualität des Aufbaus', + graphQualityStandard: 'Standard (geringere Kosten)', + graphQualityHigh: 'Hoch (höhere Kosten)', + graphBuild: 'Graph erstellen', + graphRebuild: 'Graph neu erstellen', + graphBuildCost: 'Geschätzte Kosten für diesen Aufbau: {amount}.', + graphEnableLabel: 'Wissensgraph für diese Wissensdatenbank aktivieren', + graphEnabledDescription: + 'Ein veröffentlichter Graph kann von aktivierten Chatbot-Verknüpfungen dieser Wissensdatenbank verwendet werden.', + graphDisabledDescription: + 'Aktivieren Sie die Wissensdatenbank, bevor Sie einen Graphen erstellen oder für Studierende bereitstellen.', + graphCostUnavailable: + 'Die Kostenkontrollen für Graphen sind noch nicht konfiguriert. Der Aufbau bleibt deaktiviert.', + graphEnableError: + 'Die Einstellung des Wissensgraphen konnte nicht aktualisiert werden.', + graphBillingLabel: 'Abrechnungsmodus', + graphBillingSemesterQuota: 'Semesterkontingent', + graphBillingProvider: 'Durch Anbieter abgerechnet', + graphRemainingQuota: 'Verbleibendes Semesterkontingent', + graphWorstCaseBalance: 'Kontostand nach dem maximalen Aufbau', + graphMaxCost: 'Maximal reservierte Kosten', + graphCostStatus: 'Kostenreservierung', + graphCostStatusReserved: 'Reserviert', + graphCostStatusSettled: 'Abgerechnet', + graphCostStatusReleased: 'Freigegeben', + graphCostStatusNeedsHumanReview: 'Zur manuellen Prüfung zurückgehalten', + graphActualCost: 'Tatsächliche Kosten', + graphActualUsage: + 'Tatsächliche Nutzung: {requests} Anfragen, {inputTokens} Eingabetoken, {outputTokens} Ausgabetoken, {embeddingTokens} Embedding-Token.', + graphStatusLabel: 'Status', + graphStatusEmpty: 'Kein Aufbau', + graphStatusQueued: 'In Warteschlange', + graphStatusProcessing: 'In Verarbeitung', + graphStatusSucceeded: 'Erfolgreich', + graphStatusFailed: 'Fehlgeschlagen', + graphStale: 'Veraltet', + graphBuildId: 'Aufbau {buildId}', + graphLoading: 'Graphstatus wird geladen…', + graphLoadError: 'Der Graphstatus konnte nicht geladen werden.', + graphRetry: 'Erneut versuchen', + graphBuildError: 'Der Graphaufbau konnte nicht gestartet werden.', + graphPreviewTitle: 'Veröffentlichter Graph', + graphPreviewUnavailable: + 'Erstellen und veröffentlichen Sie einen Graphen, bevor Sie die Dozierendenansicht öffnen.', + ingestionDisabledError: + 'Das Hinzufügen neuer Inhalte zu Wissensdatenbanken ist vorübergehend deaktiviert.', + }, manage: { assistant: { open: 'Assistent', @@ -3790,6 +4015,9 @@ Da die KlickerUZH-App noch nicht im iOS-App-Store verfügbar ist, folgen Sie die disclaimerDeclined: 'Abgelehnt', disclaimerPending: 'Ausstehend', mcpConfigurations: 'MCP-Konfigurationen', + knowledgeBase: 'Wissensdatenbank', + noEnabledKnowledgeBase: + 'Es ist keine Wissensdatenbank verknüpft. Dieser Chatbot kann Kursmaterial nicht durchsuchen.', noMcpConfigurations: 'Keine MCP-Konfigurationen.', mcpServerActive: 'Server aktiv', mcpServerInactive: 'Server inaktiv', diff --git a/packages/i18n/messages/en.ts b/packages/i18n/messages/en.ts index 3383b9b7fb..6b87c94e69 100644 --- a/packages/i18n/messages/en.ts +++ b/packages/i18n/messages/en.ts @@ -847,6 +847,15 @@ Other participants will only see your public **participant profile**, including activeContext: 'Using current page context', questionContext: 'Question {currentStep}/{totalSteps}', noCourseChatbot: 'No course chatbot is available for this course yet.', + retrieval: { + searching: 'Searching lecture content for “{query}”…', + errorTitle: 'Search unavailable', + errorDescription: + 'The lecture content could not be searched. Please try again.', + contentTitle: 'Lecture content', + questionLabel: 'Question', + noContent: 'No content available', + }, }, insights: { noCourseDataAvailable: @@ -1496,6 +1505,212 @@ Since the KlickerUZH app is not yet available in the iOS App Store, follow these cohortSizeLabel: 'Comparison cohort: {count} active participants', }, }, + kb: { + title: 'Knowledge Bases', + create: 'Create knowledge base', + nameLabel: 'Name', + descriptionLabel: 'Description (optional)', + createSuccess: 'Knowledge base created.', + createError: 'The knowledge base could not be created.', + deleteTitle: 'Delete knowledge base', + deleteDescription: + '“{name}” will disappear immediately. Its stored files and external index are removed in the background. This action cannot be undone.', + deleteSuccess: 'Knowledge base removed. Background cleanup is in progress.', + deleteError: 'The knowledge base could not be deleted.', + emptyTitle: 'No knowledge bases yet', + emptyDescription: 'Create a knowledge base to add your first resources.', + noDescription: 'No description', + loadError: 'The knowledge bases could not be loaded.', + searchKnowledgeBases: 'Search knowledge bases', + searchKnowledgeBasesPlaceholder: 'Search by name or description', + noSearchResults: 'No matching knowledge bases', + noSearchResultsDescription: 'Try a different name or description.', + searchResultCount: + '{count, plural, =0 {No knowledge bases} one {# knowledge base} other {# knowledge bases}}', + catalogMetrics: + '{resources, plural, one {# resource} other {# resources}} · {chatbots, plural, one {# connected chatbot} other {# connected chatbots}}', + loadMore: 'Load more knowledge bases', + notFound: 'The knowledge base could not be found.', + backToList: 'Back to knowledge bases', + metricsTitle: 'Usage and connections', + metricVisibleResources: 'Visible resources', + metricReservedResources: + '{count, plural, =0 {No upload reservations} one {# upload reservation} other {# upload reservations}}', + metricStorage: 'Storage quota', + metricStorageBreakdown: + '{visible} visible · {reserved} reserved for uploads', + unknownSizesReserved: + '{count, plural, one {# legacy resource reserves up to 25 MiB} other {# legacy resources reserve up to 25 MiB each}}', + metricPendingCleanup: 'Pending cleanup', + metricPendingCleanupSize: '{size} awaiting cleanup', + metricLinkedConsumers: 'Connected chatbots', + metricQuotaResources: '{count} resources count toward the quota', + quotaReleaseMessage: + 'Deleted resources continue to count toward the quota until background cleanup finishes.', + fileUploadTitle: 'Upload a file', + fileUploadDescription: 'Add course material from your computer.', + fileDropPrompt: 'Drop a file here or click to choose one', + fileUploadFormats: 'PDF, TXT or MD · maximum 25 MB', + uploading: 'Uploading…', + fileUploadSuccess: 'File added to the knowledge base.', + fileUploadError: 'The file could not be uploaded.', + fileRejected: 'Choose a supported file of no more than 25 MB.', + linkTitle: 'Add a link', + linkDescription: 'Register a website or media resource for ingestion.', + resourceTitleLabel: 'Title', + urlLabel: 'URL', + invalidUrl: 'Enter a valid URL starting with http:// or https://.', + linkSuccess: 'Link added to the knowledge base.', + linkError: 'The link could not be added.', + resourcesTitle: 'Resources', + resourcesLoadError: 'The resources could not be loaded.', + searchResources: 'Search resources', + searchResourcesPlaceholder: 'Search title, filename or URL', + filterType: 'Type', + filterStatus: 'Latest ingestion', + filterAll: 'All', + typeFile: 'File', + typeUrl: 'Link', + noResourceResults: 'No resources match these filters.', + resourceResultCount: + '{count, plural, =0 {No resources} one {# resource} other {# resources}}', + selectAllPage: 'Select up to 50 available resources', + selectResource: 'Select “{title}”', + loadMoreResources: 'Load more resources', + noResources: 'No resources have been added yet.', + updatedAt: 'Updated {date}', + statusAdded: 'Added', + statusQueued: 'Queued', + statusProcessing: 'Processing', + statusReady: 'Ready', + statusFailed: 'Failed', + deleteResourceTitle: 'Delete resource', + deleteResourceDescription: + '“{title}” will disappear immediately. Its stored file and external index are removed in the background. This action cannot be undone.', + deleteResourceSuccess: + 'Resource removed. Background cleanup is in progress.', + deleteResourceError: 'The resource could not be deleted.', + bulkDelete: 'Delete selected ({count})', + bulkDeleteTitle: + '{count, plural, one {Delete # resource} other {Delete # resources}}', + bulkDeleteConfirm: + '{count, plural, one {Delete resource} other {Delete # resources}}', + bulkDeleteDescription: + '{count, plural, one {The selected resource will disappear immediately. Stored files and external indexes are removed in the background. This action cannot be undone.} other {The # selected resources will disappear immediately. Stored files and external indexes are removed in the background. This action cannot be undone.}}', + bulkDeleteSuccess: + '{count, plural, one {Resource removed. Background cleanup is in progress.} other {# resources removed. Background cleanup is in progress.}}', + bulkDeleteError: + 'The deletion could not be confirmed. Refresh the list before trying again.', + ingestResource: 'Ingest', + retryIngestion: 'Retry', + reingestResource: 'Re-ingest', + ingestResourceSuccess: 'Resource queued for ingestion.', + ingestResourceError: 'The resource could not be queued for ingestion.', + operationStatus: 'Latest ingestion', + operationInProgress: + 'This operation is running in the background. You can leave this page.', + backgroundOperationsMessage: + 'One or more operations are still running. Statuses update automatically, and you can safely leave this page.', + servingStatus: 'Available to AI', + notServing: 'Not available yet', + servingCurrentVersion: 'Current version {version}', + servingPreviousVersion: 'Version {version} remains available', + servingSince: 'Available since {date}', + version: 'Version {version}', + recentAttempts: 'Recent attempts', + noRecentAttempts: 'No ingestion attempts yet.', + historyLoadError: 'Recent attempts could not be loaded.', + runStatusQueued: 'Queued', + runStatusProcessing: 'Processing', + runStatusSucceeded: 'Succeeded', + runStatusFailed: 'Failed', + runStatusSuperseded: 'Superseded', + ingestionStartError: 'The ingestion operation could not be started.', + storageLimitError: + 'This resource exceeds the 500 MiB knowledge base storage limit.', + resourceLimitError: + 'This knowledge base has reached its limit of 100 resources.', + uploadMismatchError: + 'The uploaded file no longer matches its upload reservation. Please upload it again.', + ingestionFailed: 'The ingestion operation failed.', + ingestionSuperseded: 'The ingestion operation was superseded.', + inspectResource: 'Inspect', + inspectorTitle: 'Resource details', + sourceType: 'Source type', + sourceLocation: 'Source URL', + fileName: 'Original filename', + mimeType: 'Media type', + fileSize: 'File size', + createdAt: 'Created', + chatbotsTitle: 'Connected chatbots', + chatbotsDescription: + 'Choose which chatbot can use this knowledge base. A chatbot can use one knowledge base at a time.', + chatbotsLoadError: 'The chatbots could not be loaded.', + noChatbots: 'Create a chatbot before connecting a knowledge base.', + chatbotSelectLabel: 'Chatbot', + chatbotSelectPlaceholder: 'Choose a chatbot', + attachChatbot: 'Connect chatbot', + replaceChatbot: 'Replace knowledge base', + detachChatbot: 'Disconnect', + linkedChatbots: 'Using this knowledge base', + noLinkedChatbots: 'No chatbot is using this knowledge base.', + chatbotReplacementWarning: + 'This chatbot currently uses “{kbName}”. Connecting it here replaces that knowledge base.', + chatbotAttachSuccess: 'Chatbot connected to the knowledge base.', + chatbotAttachError: 'The chatbot could not be connected.', + chatbotDetachSuccess: 'Chatbot disconnected from the knowledge base.', + chatbotDetachError: 'The chatbot could not be disconnected.', + previewAccessError: + 'The knowledge base workspace is not available for your account yet.', + graphTitle: 'Knowledge graph', + graphDescription: + 'Build a graph from the knowledge base resources and inspect the published result.', + graphQualityTierLabel: 'Build quality', + graphQualityStandard: 'Standard (lower cost)', + graphQualityHigh: 'High (higher cost)', + graphBuild: 'Build graph', + graphRebuild: 'Rebuild graph', + graphBuildCost: 'Estimated cost for this build: {amount}.', + graphEnableLabel: 'Enable the knowledge graph for this KB', + graphEnabledDescription: + 'A published graph can be used by enabled chatbot bindings for this knowledge base.', + graphDisabledDescription: + 'Enable this knowledge base before starting a graph build or serving it to students.', + graphCostUnavailable: + 'Graph cost controls are not configured yet. Building remains disabled.', + graphEnableError: 'The knowledge graph setting could not be updated.', + graphBillingLabel: 'Billing mode', + graphBillingSemesterQuota: 'Semester quota', + graphBillingProvider: 'Provider-billed', + graphRemainingQuota: 'Remaining semester quota', + graphWorstCaseBalance: 'Balance after the maximum build', + graphMaxCost: 'Maximum reserved cost', + graphCostStatus: 'Cost reservation', + graphCostStatusReserved: 'Reserved', + graphCostStatusSettled: 'Settled', + graphCostStatusReleased: 'Released', + graphCostStatusNeedsHumanReview: 'Held for human review', + graphActualCost: 'Actual cost', + graphActualUsage: + 'Actual usage: {requests} requests, {inputTokens} input tokens, {outputTokens} output tokens, {embeddingTokens} embedding tokens.', + graphStatusLabel: 'Status', + graphStatusEmpty: 'No build', + graphStatusQueued: 'Queued', + graphStatusProcessing: 'Processing', + graphStatusSucceeded: 'Succeeded', + graphStatusFailed: 'Failed', + graphStale: 'Stale', + graphBuildId: 'Build {buildId}', + graphLoading: 'Loading graph status…', + graphLoadError: 'The graph status could not be loaded.', + graphRetry: 'Retry', + graphBuildError: 'The graph build could not be started.', + graphPreviewTitle: 'Published graph', + graphPreviewUnavailable: + 'Build and publish a graph before opening the lecturer viewer.', + ingestionDisabledError: + 'Adding new content to knowledge bases is temporarily disabled.', + }, manage: { assistant: { open: 'Assistant', @@ -3729,6 +3944,9 @@ Since the KlickerUZH app is not yet available in the iOS App Store, follow these disclaimerDeclined: 'Declined', disclaimerPending: 'Pending', mcpConfigurations: 'MCP Configurations', + knowledgeBase: 'Knowledge base', + noEnabledKnowledgeBase: + 'No knowledge base is connected. This chatbot cannot search course material.', noMcpConfigurations: 'No MCP configurations.', mcpServerActive: 'Server active', mcpServerInactive: 'Server inactive', diff --git a/packages/i18n/package.json b/packages/i18n/package.json index 0341e2ffa1..f4362e61d5 100644 --- a/packages/i18n/package.json +++ b/packages/i18n/package.json @@ -11,6 +11,7 @@ "typescript": "~6.0.3" }, "peerDependencies": { + "next": "^16.2.10", "next-intl": "^4.13.0" }, "engines": { diff --git a/packages/i18n/request.ts b/packages/i18n/request.ts index 5d17e64026..73153c0cc2 100644 --- a/packages/i18n/request.ts +++ b/packages/i18n/request.ts @@ -3,13 +3,18 @@ import { getRequestConfig } from 'next-intl/server' import { getMessageFallback, onError } from './index' import { routing } from './routing' -async function loadMessages(locale: Locale) { - switch (locale) { - case 'de': - return (await import('./messages/de')).default - default: - return (await import('./messages/en')).default - } +type SupportedLocale = (typeof routing.locales)[number] + +const messageLoaders: Record< + SupportedLocale, + () => Promise<{ default: Record }> +> = { + de: () => import('./messages/de'), + en: () => import('./messages/en'), +} + +function isSupportedLocale(locale: Locale): locale is SupportedLocale { + return routing.locales.some((supportedLocale) => supportedLocale === locale) } export default getRequestConfig(async ({ requestLocale }) => { @@ -17,8 +22,8 @@ export default getRequestConfig(async ({ requestLocale }) => { const requested = (await requestLocale) as Locale // ensure that the incoming locale is valid - let locale: Locale - if (!requested || !routing.locales.includes(requested as any)) { + let locale: SupportedLocale + if (!requested || !isSupportedLocale(requested)) { locale = routing.defaultLocale } else { locale = requested @@ -26,7 +31,7 @@ export default getRequestConfig(async ({ requestLocale }) => { return { locale, - messages: await loadMessages(locale), + messages: (await messageLoaders[locale]()).default, onError, getMessageFallback, } diff --git a/packages/kb-management/package.json b/packages/kb-management/package.json new file mode 100644 index 0000000000..c995fbb240 --- /dev/null +++ b/packages/kb-management/package.json @@ -0,0 +1,34 @@ +{ + "private": true, + "name": "@klicker-uzh/kb-management", + "main": "src/index.ts", + "types": "src/index.ts", + "devDependencies": { + "@types/react": "^19.2.17", + "typescript": "~6.0.3" + }, + "peerDependencies": { + "@apollo/client": "^3.13.8", + "@azure/storage-blob": "^12.25.0", + "@fortawesome/free-solid-svg-icons": "^6.7.2", + "@fortawesome/react-fontawesome": "^0.2.2", + "@klicker-uzh/graphql": "workspace:*", + "@klicker-uzh/i18n": "workspace:*", + "@klicker-uzh/shared-components": "workspace:*", + "@klicker-uzh/types": "workspace:*", + "@uzh-bf/design-system": "4.1.8", + "next": "^16.2.10", + "next-intl": "^4.13.0", + "react": "^19.2.7", + "react-dropzone": "^14.2.9" + }, + "scripts": { + "check": "tsc --noEmit" + }, + "engines": { + "node": "=24" + }, + "volta": { + "extends": "../../package.json" + } +} diff --git a/packages/kb-management/src/KnowledgeBaseDetail.tsx b/packages/kb-management/src/KnowledgeBaseDetail.tsx new file mode 100644 index 0000000000..5a3bbe9f0b --- /dev/null +++ b/packages/kb-management/src/KnowledgeBaseDetail.tsx @@ -0,0 +1,206 @@ +import { useQuery } from '@apollo/client' +import { GetKbDocument } from '@klicker-uzh/graphql/dist/ops' +import { H2, Skeleton, UserNotification } from '@uzh-bf/design-system' +import { useFormatter, useTranslations } from 'next-intl' +import Link from 'next/link' +import React, { useState } from 'react' +import KnowledgeBaseChatbotBindings from './components/KnowledgeBaseChatbotBindings' +import KnowledgeBaseFileDropzone from './components/KnowledgeBaseFileDropzone' +import KnowledgeBaseResourceList from './components/KnowledgeBaseResourceList' +import KnowledgeBaseUrlForm from './components/KnowledgeBaseUrlForm' +import KnowledgeGraphPanel from './components/KnowledgeGraphPanel' +import { getGraphQLErrorCode } from './graphqlError' + +function KnowledgeBaseDetail({ kbId }: { kbId: string }) { + const t = useTranslations() + const format = useFormatter() + const [resourceRefreshKey, setResourceRefreshKey] = useState(0) + const { data, loading, error, refetch } = useQuery(GetKbDocument, { + variables: { id: kbId }, + }) + + if (loading) { + return ( +
+
+ ) + } + + if (error || !data?.getKb) { + return ( +
+ +
+ ) + } + + const metrics = data.getKb.metrics + const formatFileSize = (sizeBytes: number) => { + if (sizeBytes < 1024) return `${format.number(sizeBytes)} B` + if (sizeBytes < 1024 * 1024) { + return `${format.number(sizeBytes / 1024, { + maximumFractionDigits: 1, + })} KiB` + } + return `${format.number(sizeBytes / (1024 * 1024), { + maximumFractionDigits: 1, + })} MiB` + } + const refreshMetrics = () => refetch() + const handleResourceCreated = async () => { + setResourceRefreshKey((current) => current + 1) + await refreshMetrics() + } + + return ( +
+ + {t('kb.backToList')} + +

{data.getKb.name}

+ {data.getKb.description ? ( +

+ {data.getKb.description} +

+ ) : null} + {metrics ? ( +
+

+ {t('kb.metricsTitle')} +

+
+
+
+ {t('kb.metricVisibleResources')} +
+
+ {format.number(metrics.visibleResourceCount)} + + / {format.number(metrics.resourceLimit)} + +
+

+ {t('kb.metricReservedResources', { + count: metrics.reservedResourceCount, + })} +

+
+
+
+ {t('kb.metricStorage')} +
+
+ {formatFileSize(metrics.quotaSizeBytes)} + + / {formatFileSize(metrics.storageLimitBytes)} + +
+

+ {metrics.unknownSizeResourceCount > 0 + ? t('kb.unknownSizesReserved', { + count: metrics.unknownSizeResourceCount, + }) + : t('kb.metricStorageBreakdown', { + visible: formatFileSize(metrics.visibleSizeBytes), + reserved: formatFileSize(metrics.reservedSizeBytes), + })} +

+
+
+
+ {t('kb.metricPendingCleanup')} +
+
+ {format.number(metrics.pendingCleanupCount)} +
+

+ {t('kb.metricPendingCleanupSize', { + size: formatFileSize(metrics.pendingCleanupSizeBytes), + })} +

+
+
+
+ {t('kb.metricLinkedConsumers')} +
+
+ {format.number(metrics.linkedConsumerCount)} +
+

+ {t('kb.metricQuotaResources', { + count: metrics.quotaResourceCount, + })} +

+
+
+ {metrics.pendingCleanupCount > 0 ? ( +

+ {t('kb.quotaReleaseMessage')} +

+ ) : null} +
+ ) : null} +
+ + +
+ + + +
+ ) +} + +export default KnowledgeBaseDetail diff --git a/packages/kb-management/src/KnowledgeBaseManager.tsx b/packages/kb-management/src/KnowledgeBaseManager.tsx new file mode 100644 index 0000000000..9acd32fd86 --- /dev/null +++ b/packages/kb-management/src/KnowledgeBaseManager.tsx @@ -0,0 +1,222 @@ +import { NetworkStatus, useQuery } from '@apollo/client' +import { + GetUserKbsDocument, + type GetUserKbsQuery, +} from '@klicker-uzh/graphql/dist/ops' +import { + Button, + H2, + H3, + Skeleton, + TextField, + UserNotification, +} from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import Link from 'next/link' +import React, { useDeferredValue, useState } from 'react' +import CreateKnowledgeBaseModal from './components/CreateKnowledgeBaseModal' +import DeleteKnowledgeBaseModal from './components/DeleteKnowledgeBaseModal' +import { getGraphQLErrorCode } from './graphqlError' + +const PAGE_SIZE = 20 + +type KnowledgeBaseSummary = + GetUserKbsQuery['getUserKbsConnection']['items'][number] + +function KnowledgeBaseManager() { + const t = useTranslations() + const [createOpen, setCreateOpen] = useState(false) + const [search, setSearch] = useState('') + const deferredSearch = useDeferredValue(search.trim()) + const [deletionTarget, setDeletionTarget] = + useState(null) + const { data, loading, error, fetchMore, refetch, networkStatus } = useQuery( + GetUserKbsDocument, + { + variables: { + first: PAGE_SIZE, + search: deferredSearch || null, + }, + notifyOnNetworkStatusChange: true, + } + ) + const connection = data?.getUserKbsConnection + const knowledgeBases = connection?.items ?? [] + const loadingMore = networkStatus === NetworkStatus.fetchMore + + const loadMore = async () => { + if (!connection?.pageInfo.hasNextPage || loadingMore) return + await fetchMore({ + variables: { after: connection.pageInfo.endCursor }, + updateQuery: (previous, { fetchMoreResult }) => ({ + ...fetchMoreResult, + getUserKbsConnection: { + ...fetchMoreResult.getUserKbsConnection, + items: [ + ...previous.getUserKbsConnection.items, + ...fetchMoreResult.getUserKbsConnection.items, + ], + }, + }), + }) + } + + return ( +
+
+

{t('kb.title')}

+ +
+ +
+ +
+ + {loading && !connection ? ( +
+
+ ) : error ? ( + + ) : knowledgeBases.length === 0 ? ( +
+

+ {deferredSearch ? t('kb.noSearchResults') : t('kb.emptyTitle')} +

+

+ {deferredSearch + ? t('kb.noSearchResultsDescription') + : t('kb.emptyDescription')} +

+ {!deferredSearch ? ( + + ) : null} +
+ ) : ( + <> +

+ {t('kb.searchResultCount', { + count: connection?.totalCount ?? 0, + })} +

+
    + {knowledgeBases.map((kb) => ( +
  • + + + {kb.name} + + + {kb.description || t('kb.noDescription')} + + {kb.metrics ? ( + + {t('kb.catalogMetrics', { + resources: kb.metrics.visibleResourceCount, + chatbots: kb.metrics.linkedConsumerCount, + })} + + ) : null} + +
    + +
    +
    • + ))} +
+ {connection?.pageInfo.hasNextPage ? ( +
+ +
+ ) : null} + + )} + + {createOpen ? ( + setCreateOpen(false)} + onCreated={() => refetch()} + /> + ) : null} + {deletionTarget ? ( + setDeletionTarget(null)} + onDeleted={() => refetch()} + /> + ) : null} +
+ ) +} + +export default KnowledgeBaseManager diff --git a/packages/kb-management/src/components/CreateKnowledgeBaseModal.tsx b/packages/kb-management/src/components/CreateKnowledgeBaseModal.tsx new file mode 100644 index 0000000000..b0cff47b29 --- /dev/null +++ b/packages/kb-management/src/components/CreateKnowledgeBaseModal.tsx @@ -0,0 +1,86 @@ +import { useMutation } from '@apollo/client' +import { CreateKbDocument } from '@klicker-uzh/graphql/dist/ops' +import { Modal, TextareaField, TextField, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React, { useState } from 'react' +import { refreshAfterMutation } from '../refreshAfterMutation' + +function CreateKnowledgeBaseModal({ + onClose, + onCreated, +}: { + onClose: () => void + onCreated: () => Promise +}) { + const t = useTranslations() + const [name, setName] = useState('') + const [description, setDescription] = useState('') + const [createKb, { loading }] = useMutation(CreateKbDocument) + + const handleCreate = async () => { + const trimmedName = name.trim() + if (!trimmedName || loading) return + + try { + await createKb({ + variables: { + name: trimmedName, + description: description.trim() || null, + }, + }) + } catch (error) { + console.error('Failed to create knowledge base', error) + toast({ type: 'error', message: t('kb.createError') }) + return + } + + await refreshAfterMutation(onCreated, 'knowledge bases after creation') + toast({ type: 'success', message: t('kb.createSuccess') }) + onClose() + } + + return ( + +
+ + +
+ + ) +} + +export default CreateKnowledgeBaseModal diff --git a/packages/kb-management/src/components/DeleteKnowledgeBaseModal.tsx b/packages/kb-management/src/components/DeleteKnowledgeBaseModal.tsx new file mode 100644 index 0000000000..a79aff3149 --- /dev/null +++ b/packages/kb-management/src/components/DeleteKnowledgeBaseModal.tsx @@ -0,0 +1,66 @@ +import { useMutation } from '@apollo/client' +import { + DeleteKbDocument, + type GetUserKbsQuery, +} from '@klicker-uzh/graphql/dist/ops' +import { Modal, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React from 'react' +import { refreshAfterMutation } from '../refreshAfterMutation' + +type KnowledgeBaseSummary = + GetUserKbsQuery['getUserKbsConnection']['items'][number] + +function DeleteKnowledgeBaseModal({ + knowledgeBase, + onClose, + onDeleted, +}: { + knowledgeBase: KnowledgeBaseSummary + onClose: () => void + onDeleted: () => Promise +}) { + const t = useTranslations() + const [deleteKb, { loading }] = useMutation(DeleteKbDocument) + + const handleDelete = async () => { + if (loading) return + + try { + await deleteKb({ + variables: { id: knowledgeBase.id }, + }) + } catch (error) { + console.error('Failed to delete knowledge base', error) + toast({ type: 'error', message: t('kb.deleteError') }) + return + } + + await refreshAfterMutation(onDeleted, 'knowledge bases after deletion') + toast({ type: 'success', message: t('kb.deleteSuccess') }) + onClose() + } + + return ( + +

{t('kb.deleteDescription', { name: knowledgeBase.name })}

+ + ) +} + +export default DeleteKnowledgeBaseModal diff --git a/packages/kb-management/src/components/DeleteKnowledgeBaseResourceModal.tsx b/packages/kb-management/src/components/DeleteKnowledgeBaseResourceModal.tsx new file mode 100644 index 0000000000..f6ffb08d16 --- /dev/null +++ b/packages/kb-management/src/components/DeleteKnowledgeBaseResourceModal.tsx @@ -0,0 +1,66 @@ +import { useMutation } from '@apollo/client' +import { + DeleteKbResourceDocument, + type GetKbResourcesQuery, +} from '@klicker-uzh/graphql/dist/ops' +import { Modal, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React from 'react' +import { refreshAfterMutation } from '../refreshAfterMutation' + +type KnowledgeBaseResource = + GetKbResourcesQuery['getKbResources']['items'][number] + +function DeleteKnowledgeBaseResourceModal({ + resource, + onClose, + onDeleted, +}: { + resource: KnowledgeBaseResource + onClose: () => void + onDeleted: () => Promise +}) { + const t = useTranslations() + const [deleteResource, { loading }] = useMutation(DeleteKbResourceDocument) + + const handleDelete = async () => { + if (loading) return + + try { + await deleteResource({ + variables: { id: resource.id }, + }) + } catch (error) { + console.error('Failed to delete KB resource', error) + toast({ type: 'error', message: t('kb.deleteResourceError') }) + return + } + + await refreshAfterMutation(onDeleted, 'KB resources after deletion') + toast({ type: 'success', message: t('kb.deleteResourceSuccess') }) + onClose() + } + + return ( + +

{t('kb.deleteResourceDescription', { title: resource.title })}

+ + ) +} + +export default DeleteKnowledgeBaseResourceModal diff --git a/packages/kb-management/src/components/DeleteKnowledgeBaseResourcesModal.tsx b/packages/kb-management/src/components/DeleteKnowledgeBaseResourcesModal.tsx new file mode 100644 index 0000000000..367874b2e7 --- /dev/null +++ b/packages/kb-management/src/components/DeleteKnowledgeBaseResourcesModal.tsx @@ -0,0 +1,79 @@ +import { useMutation } from '@apollo/client' +import { + DeleteKbResourcesDocument, + type GetKbResourcesQuery, +} from '@klicker-uzh/graphql/dist/ops' +import { Modal, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React from 'react' +import { refreshAfterMutation } from '../refreshAfterMutation' + +type KnowledgeBaseResource = + GetKbResourcesQuery['getKbResources']['items'][number] + +function DeleteKnowledgeBaseResourcesModal({ + kbId, + resources, + onClose, + onDeleted, +}: { + kbId: string + resources: KnowledgeBaseResource[] + onClose: () => void + onDeleted: () => Promise +}) { + const t = useTranslations() + const [deleteResources, { loading }] = useMutation(DeleteKbResourcesDocument) + + const handleDelete = async () => { + if (loading || resources.length === 0 || resources.length > 50) return + + try { + await deleteResources({ + variables: { kbId, ids: resources.map(({ id }) => id) }, + }) + } catch (error) { + console.error('Failed to delete KB resources', error) + toast({ type: 'error', message: t('kb.bulkDeleteError') }) + return + } + + await refreshAfterMutation(onDeleted, 'KB resources after deletion') + toast({ + type: 'success', + message: t('kb.bulkDeleteSuccess', { count: resources.length }), + }) + onClose() + } + + return ( + 50} + onPrimaryAction={handleDelete} + secondaryLabel={t('shared.generic.cancel')} + onSecondaryAction={onClose} + dataContent={{ cy: 'delete-kb-resources-modal' }} + dataCloseButton={{ cy: 'close-delete-kb-resources' }} + dataPrimaryAction={{ cy: 'confirm-delete-kb-resources' }} + dataSecondaryAction={{ cy: 'cancel-delete-kb-resources' }} + className={{ content: 'max-w-xl' }} + > +

{t('kb.bulkDeleteDescription', { count: resources.length })}

+
    + {resources.map((resource) => ( +
  • + {resource.title} +
    • + ))} +
+ + ) +} + +export default DeleteKnowledgeBaseResourcesModal diff --git a/packages/kb-management/src/components/KnowledgeBaseChatbotBindings.tsx b/packages/kb-management/src/components/KnowledgeBaseChatbotBindings.tsx new file mode 100644 index 0000000000..dc520090e8 --- /dev/null +++ b/packages/kb-management/src/components/KnowledgeBaseChatbotBindings.tsx @@ -0,0 +1,206 @@ +import { useMutation, useQuery } from '@apollo/client' +import { + AttachKbToChatbotDocument, + DetachKbFromChatbotDocument, + GetChatbotsInfoDocument, + GetKbChatbotBindingsDocument, +} from '@klicker-uzh/graphql/dist/ops' +import { + Button, + H3, + SelectField, + Skeleton, + UserNotification, + toast, +} from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React, { useState } from 'react' +import { refreshAfterMutation } from '../refreshAfterMutation' + +function KnowledgeBaseChatbotBindings({ + kbId, + onChanged, +}: { + kbId: string + onChanged: () => Promise +}) { + const t = useTranslations() + const [selectedChatbotId, setSelectedChatbotId] = useState< + string | undefined + >() + const { data, loading, error } = useQuery(GetKbChatbotBindingsDocument, { + variables: { kbId }, + }) + const [attachKb, { loading: attaching }] = useMutation( + AttachKbToChatbotDocument + ) + const [detachKb, { loading: detaching }] = useMutation( + DetachKbFromChatbotDocument + ) + const bindings = data?.getKbChatbotBindings ?? [] + const selectedBinding = bindings.find( + ({ chatbotId }) => chatbotId === selectedChatbotId + ) + const linkedBindings = bindings.filter( + ({ enabledKbId }) => enabledKbId === kbId + ) + const replacing = + selectedBinding?.enabledKbId != null && selectedBinding.enabledKbId !== kbId + const mutating = attaching || detaching + + const refetchQueries = [ + { + query: GetKbChatbotBindingsDocument, + variables: { kbId }, + }, + { query: GetChatbotsInfoDocument }, + ] + + const handleAttach = async () => { + if (!selectedChatbotId || mutating) return + + try { + await attachKb({ + variables: { kbId, chatbotId: selectedChatbotId }, + refetchQueries, + }) + } catch (mutationError) { + console.error('Failed to attach KB to chatbot', mutationError) + toast({ type: 'error', message: t('kb.chatbotAttachError') }) + return + } + + await refreshAfterMutation(onChanged, 'KB chatbot bindings after attach') + setSelectedChatbotId(undefined) + toast({ type: 'success', message: t('kb.chatbotAttachSuccess') }) + } + + const handleDetach = async (chatbotId: string) => { + if (mutating) return + + try { + await detachKb({ + variables: { kbId, chatbotId }, + refetchQueries, + }) + } catch (mutationError) { + console.error('Failed to detach KB from chatbot', mutationError) + toast({ type: 'error', message: t('kb.chatbotDetachError') }) + return + } + + await refreshAfterMutation(onChanged, 'KB chatbot bindings after detach') + toast({ type: 'success', message: t('kb.chatbotDetachSuccess') }) + } + + return ( +
+

{t('kb.chatbotsTitle')}

+

+ {t('kb.chatbotsDescription')} +

+ + {loading ? ( +
+ ) +} + +export default KnowledgeBaseChatbotBindings diff --git a/packages/kb-management/src/components/KnowledgeBaseFileDropzone.tsx b/packages/kb-management/src/components/KnowledgeBaseFileDropzone.tsx new file mode 100644 index 0000000000..71ddc2a954 --- /dev/null +++ b/packages/kb-management/src/components/KnowledgeBaseFileDropzone.tsx @@ -0,0 +1,150 @@ +import { useMutation } from '@apollo/client' +import { + ConfirmKbFileUploadDocument, + RequestKbFileUploadDocument, +} from '@klicker-uzh/graphql/dist/ops' +import { H3, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React, { useState } from 'react' +import { useDropzone } from 'react-dropzone' +import { getGraphQLErrorCode } from '../graphqlError' +import { refreshAfterMutation } from '../refreshAfterMutation' + +const MAX_FILE_SIZE = 25 * 1024 * 1024 +const CONTENT_TYPES: Record = { + pdf: 'application/pdf', + txt: 'text/plain', + md: 'text/plain', +} + +const ACCEPTED_FILES = { + 'application/pdf': ['.pdf'], + 'text/plain': ['.txt', '.md'], +} + +function KnowledgeBaseFileDropzone({ + kbId, + onResourceCreated, +}: { + kbId: string + onResourceCreated: () => Promise +}) { + const t = useTranslations() + const [uploading, setUploading] = useState(false) + const [requestUpload] = useMutation(RequestKbFileUploadDocument) + const [confirmUpload] = useMutation(ConfirmKbFileUploadDocument) + + const uploadFile = async (files: File[]) => { + const file = files[0] + if (!file || uploading) return + + const extension = file.name.split('.').pop()?.toLowerCase() + const contentType = extension ? CONTENT_TYPES[extension] : undefined + if (!contentType) { + toast({ type: 'error', message: t('kb.fileRejected') }) + return + } + + setUploading(true) + try { + try { + const { data } = await requestUpload({ + variables: { + kbId, + fileName: file.name, + contentType, + sizeBytes: file.size, + }, + }) + const ticket = data?.requestKbFileUpload + if (!ticket) throw new Error('Upload ticket was not returned') + + const { BlobServiceClient } = await import('@azure/storage-blob') + const serviceClient = new BlobServiceClient(ticket.uploadSasURL) + const blockBlobClient = serviceClient + .getContainerClient(ticket.containerName) + .getBlockBlobClient(ticket.blobName) + await blockBlobClient.uploadData(file, { + blobHTTPHeaders: { blobContentType: contentType }, + }) + + await confirmUpload({ + variables: { + kbId, + blobName: ticket.blobName, + title: file.name, + originalFilename: file.name, + mimeType: contentType, + sizeBytes: file.size, + }, + }) + } catch (error) { + console.error('Failed to upload KB file', error) + const code = getGraphQLErrorCode(error) + const message = + code === 'KB_RESOURCE_LIMIT_REACHED' + ? t('kb.resourceLimitError') + : code === 'KB_STORAGE_LIMIT_REACHED' + ? t('kb.storageLimitError') + : code === 'KB_UPLOAD_TICKET_MISMATCH' + ? t('kb.uploadMismatchError') + : code === 'KB_INGESTION_DISABLED' + ? t('kb.ingestionDisabledError') + : t('kb.fileUploadError') + toast({ type: 'error', message }) + return + } + + await refreshAfterMutation(onResourceCreated, 'KB resources after upload') + toast({ type: 'success', message: t('kb.fileUploadSuccess') }) + } finally { + setUploading(false) + } + } + + const { getInputProps, getRootProps, isDragActive } = useDropzone({ + accept: ACCEPTED_FILES, + disabled: uploading, + maxSize: MAX_FILE_SIZE, + multiple: false, + onDropAccepted: uploadFile, + onDropRejected: () => + toast({ type: 'error', message: t('kb.fileRejected') }), + }) + + return ( +
+

{t('kb.fileUploadTitle')}

+

+ {t('kb.fileUploadDescription')} +

+
+ + + {uploading ? t('kb.uploading') : t('kb.fileDropPrompt')} + + + {t('kb.fileUploadFormats')} + +
+
+ ) +} + +export default KnowledgeBaseFileDropzone diff --git a/packages/kb-management/src/components/KnowledgeBaseResourceList.tsx b/packages/kb-management/src/components/KnowledgeBaseResourceList.tsx new file mode 100644 index 0000000000..9a9e405e47 --- /dev/null +++ b/packages/kb-management/src/components/KnowledgeBaseResourceList.tsx @@ -0,0 +1,1337 @@ +import { + NetworkStatus, + useApolloClient, + useLazyQuery, + useMutation, + useQuery, + type ApolloQueryResult, +} from '@apollo/client' +import { + faFileLines, + faLink, + faSpinner, +} from '@fortawesome/free-solid-svg-icons' +import { FontAwesomeIcon } from '@fortawesome/react-fontawesome' +import { + GetKbResourceIngestionRunsDocument, + GetKbResourcesDocument, + IngestKbResourceDocument, + KbIngestionStatus, + KbResourceStatus, + KbResourceType, + type GetKbResourcesQuery, + type GetKbResourcesQueryVariables, +} from '@klicker-uzh/graphql/dist/ops' +import { + Badge, + Button, + H3, + Modal, + SelectField, + Skeleton, + TextField, + UserNotification, + toast, +} from '@uzh-bf/design-system' +import { useFormatter, useTranslations } from 'next-intl' +import React, { + useCallback, + useDeferredValue, + useEffect, + useMemo, + useRef, + useState, +} from 'react' +import { getGraphQLErrorCode } from '../graphqlError' +import { refreshAfterMutation } from '../refreshAfterMutation' +import DeleteKnowledgeBaseResourceModal from './DeleteKnowledgeBaseResourceModal' +import DeleteKnowledgeBaseResourcesModal from './DeleteKnowledgeBaseResourcesModal' + +const PAGE_SIZE = 20 +const MAX_BULK_SELECTION = 50 +// SelectField's underlying Radix Select forbids an item value of `''` (it is +// reserved to mean "no selection" / show the placeholder), so the "all" +// filter option uses this sentinel and is translated to/from `''` at the +// typeFilter/statusFilter state boundary. +const FILTER_ALL_VALUE = 'all' + +type KnowledgeBaseResource = + GetKbResourcesQuery['getKbResources']['items'][number] + +function getUrlHost(sourceUrl: string | null | undefined) { + if (!sourceUrl) return '—' + try { + return new URL(sourceUrl).host + } catch { + return sourceUrl + } +} + +function isActiveResource(resource: KnowledgeBaseResource) { + return ( + resource.status === KbResourceStatus.Queued || + resource.status === KbResourceStatus.Processing || + resource.latestIngestionRun?.status === KbIngestionStatus.Queued || + resource.latestIngestionRun?.status === KbIngestionStatus.Processing + ) +} + +function RunStatusBadge({ + status, + dataCy, +}: { + status: KbIngestionStatus + dataCy: string +}) { + const t = useTranslations() + const presentation = (() => { + switch (status) { + case KbIngestionStatus.Queued: + return { + label: t('kb.runStatusQueued'), + className: 'border-amber-300 bg-amber-100 text-amber-900', + } + case KbIngestionStatus.Processing: + return { + label: t('kb.runStatusProcessing'), + className: 'border-amber-300 bg-amber-100 text-amber-900', + } + case KbIngestionStatus.Succeeded: + return { + label: t('kb.runStatusSucceeded'), + className: 'border-green-300 bg-green-100 text-green-800', + } + case KbIngestionStatus.Failed: + return { + label: t('kb.runStatusFailed'), + className: 'border-red-300 bg-red-100 text-red-800', + } + case KbIngestionStatus.Superseded: + return { + label: t('kb.runStatusSuperseded'), + className: 'border-slate-300 bg-slate-100 text-slate-700', + } + } + })() + + return ( + + {status === KbIngestionStatus.Processing ? ( + + ) +} + +function RunStatusMessage({ + status, + errorCode, + className, + dataCy, +}: { + status: KbIngestionStatus + errorCode?: string | null + className?: string + dataCy?: string +}) { + const t = useTranslations() + const message = (() => { + if (errorCode === 'QUEUE_DISPATCH_FAILED') { + return t('kb.ingestResourceError') + } + if (errorCode === 'INGESTION_DISPATCH_FAILED') { + return t('kb.ingestionStartError') + } + if (errorCode === 'KB_STORAGE_LIMIT_REACHED') { + return t('kb.storageLimitError') + } + if (status === KbIngestionStatus.Failed) { + return t('kb.ingestionFailed') + } + if (status === KbIngestionStatus.Superseded) { + return t('kb.ingestionSuperseded') + } + return null + })() + + return message ? ( +

+ {message} +

+ ) : null +} + +function KnowledgeBaseResourceHistory({ + resourceId, + refreshKey, +}: { + resourceId: string + refreshKey: number +}) { + const t = useTranslations() + const format = useFormatter() + const detailsRef = useRef(null) + const [loadRuns, { data, loading, error }] = useLazyQuery( + GetKbResourceIngestionRunsDocument, + { variables: { resourceId } } + ) + + useEffect(() => { + if (refreshKey > 0 && detailsRef.current?.open) { + void loadRuns({ fetchPolicy: 'network-only' }) + } + }, [loadRuns, refreshKey]) + + return ( +
{ + if (event.currentTarget.open && !loading) { + void loadRuns({ fetchPolicy: 'network-only' }) + } + }} + > + + {t('kb.recentAttempts')} + + {loading ? ( +

+ {t('shared.generic.loading')} +

+ ) : error ? ( +

+ {t('kb.historyLoadError')} +

+ ) : data?.getKbResourceIngestionRuns.length === 0 ? ( +

+ {t('kb.noRecentAttempts')} +

+ ) : ( +
    + {data?.getKbResourceIngestionRuns.map((run) => ( +
  1. + +
    + + {t('kb.version', { version: run.resourceVersion })} + + +
    + +
    2. + ))} +
+ )} +
+ ) +} + +function OperationProgress({ resource }: { resource: KnowledgeBaseResource }) { + const t = useTranslations() + if (!isActiveResource(resource)) return null + + return ( +
+
+
+
+

+ {t('kb.operationInProgress')} +

+
+ ) +} + +function ResourceServingStatus({ + resource, +}: { + resource: KnowledgeBaseResource +}) { + const t = useTranslations() + + if (resource.activeResourceVersion == null) { + return t('kb.notServing') + } + if ( + resource.activeResourceVersion === resource.resourceVersion && + resource.status === KbResourceStatus.Ready + ) { + return t('kb.servingCurrentVersion', { + version: resource.activeResourceVersion, + }) + } + return t('kb.servingPreviousVersion', { + version: resource.activeResourceVersion, + }) +} + +function KnowledgeBaseResourceList({ + kbId, + refreshKey, + onMetricsChanged, +}: { + kbId: string + refreshKey: number + onMetricsChanged: () => Promise +}) { + const t = useTranslations() + const format = useFormatter() + const [search, setSearch] = useState('') + const deferredSearch = useDeferredValue(search.trim()) + const [typeFilter, setTypeFilter] = useState('') + const [statusFilter, setStatusFilter] = useState('') + const [selectedIds, setSelectedIds] = useState>(new Set()) + const [deletionTarget, setDeletionTarget] = + useState(null) + const [bulkDeletionOpen, setBulkDeletionOpen] = useState(false) + const [inspectorId, setInspectorId] = useState(null) + const [ingestingId, setIngestingId] = useState(null) + const [historyRefreshes, setHistoryRefreshes] = useState< + Record + >({}) + const pollInFlightRef = useRef(false) + const loadMoreInFlightRef = useRef(false) + const refreshAfterLoadMoreRef = useRef(false) + const pollTickRef = useRef(0) + const refreshRequestRef = useRef(0) + const handledRefreshTriggerRef = useRef(null) + // Bounded-polling bookkeeping (P2-1): per-page cursors needed to refetch a + // specific loaded page (index 0's cursor is always `null`), which page + // indexes are known to contain an active row, and whether a full walk has + // established that bookkeeping since the last kb/filter reset. Populated by + // `refreshLoadedResources` (full walk) and extended by `loadMore`; consumed + // by `pollActivePages`, which the 2s interval calls instead of re-walking + // every loaded page. + const pageCursorsRef = useRef<(string | null)[]>([null]) + const activePageIndexesRef = useRef>(new Set()) + const bookkeepingValidRef = useRef(false) + const variables = { + kbId, + first: PAGE_SIZE, + search: deferredSearch || null, + type: typeFilter || null, + status: statusFilter || null, + } + const { data, loading, error, fetchMore, networkStatus, updateQuery } = + useQuery(GetKbResourcesDocument, { + variables, + notifyOnNetworkStatusChange: true, + }) + const apolloClient = useApolloClient() + const [ingestResource] = useMutation(IngestKbResourceDocument) + const connection = data?.getKbResources + const resources = connection?.items ?? [] + const loadedResourceCountRef = useRef(resources.length) + loadedResourceCountRef.current = resources.length + const polling = resources.some(isActiveResource) + const loadingMore = networkStatus === NetworkStatus.fetchMore + const inspectorResource = useMemo( + () => resources.find(({ id }) => id === inspectorId) ?? null, + [inspectorId, resources] + ) + const selectableIds = resources + .filter((resource) => !isActiveResource(resource)) + .map(({ id }) => id) + const bulkSelectableIds = selectableIds.slice(0, MAX_BULK_SELECTION) + const allPageSelected = + bulkSelectableIds.length > 0 && + bulkSelectableIds.every((id) => selectedIds.has(id)) + const selectedResources = resources.filter(({ id }) => selectedIds.has(id)) + + // The per-page fetch variables are identical at every poll site (full walk + // and bounded poll alike) bar the cursor, so build them in one place. + const getPageVariables = useCallback( + (after: string | null) => ({ + kbId, + first: PAGE_SIZE, + after, + search: deferredSearch || null, + type: typeFilter || null, + status: statusFilter || null, + }), + [deferredSearch, kbId, statusFilter, typeFilter] + ) + + const refreshLoadedResources = useCallback(async () => { + // A load-more request owns the result window until fetchMore has appended + // its page and updated the cursor bookkeeping. Skipping a concurrent + // refresh prevents an older target window from replacing the new page. + if (loadMoreInFlightRef.current) { + refreshAfterLoadMoreRef.current = true + return false + } + const requestId = ++refreshRequestRef.current + // The interval can race the render caused by fetchMore. Read the latest + // committed window size so a full walk cannot collapse a newly loaded + // page because its closure still saw the previous length. + const targetCount = Math.max(loadedResourceCountRef.current, PAGE_SIZE) + const refreshedItems: KnowledgeBaseResource[] = [] + const pageCursors: (string | null)[] = [null] + const activePageIndexes = new Set() + let after: string | null = null + let pageIndex = 0 + let refreshedConnection: GetKbResourcesQuery['getKbResources'] | null = null + + try { + do { + const refreshedResult: ApolloQueryResult = + await apolloClient.query< + GetKbResourcesQuery, + GetKbResourcesQueryVariables + >({ + query: GetKbResourcesDocument, + variables: getPageVariables(after), + fetchPolicy: 'no-cache', + }) + if (requestId !== refreshRequestRef.current) return false + + refreshedConnection = refreshedResult.data?.getKbResources ?? null + if (!refreshedConnection) return false + const pageItems = refreshedConnection.items + refreshedItems.push(...pageItems) + if (pageItems.some(isActiveResource)) { + activePageIndexes.add(pageIndex) + } + after = refreshedConnection.pageInfo.endCursor ?? null + pageCursors[pageIndex + 1] = after + pageIndex += 1 + } while ( + refreshedItems.length < targetCount && + refreshedConnection.pageInfo.hasNextPage && + after + ) + + if (requestId !== refreshRequestRef.current || !refreshedConnection) { + return false + } + const finalConnection = refreshedConnection + + updateQuery((previous) => ({ + ...previous, + getKbResources: { + ...finalConnection, + items: refreshedItems, + }, + })) + pageCursorsRef.current = pageCursors + activePageIndexesRef.current = activePageIndexes + bookkeepingValidRef.current = true + return true + } catch (refreshError) { + if ( + requestId !== refreshRequestRef.current || + (refreshError as { name?: string }).name === 'AbortError' + ) { + return false + } + throw refreshError + } + }, [apolloClient, getPageVariables, updateQuery]) + + // Bounded poll for the 2s interval (P2-1): instead of re-walking every + // loaded page, only refetch the first page (the entry point of the cursor + // walk) plus pages already known to contain an active row, using the + // per-page cursors captured by the last full walk. Falls back to a full + // walk (via refreshLoadedResources) whenever that bookkeeping isn't + // established yet for the currently loaded pages, whenever a refetched + // page comes back with a different item count than expected, or whenever + // a non-tail page's fresh end cursor drifts from the cursor recorded for + // the next loaded page -- all are signs that a concurrent insert/delete + // shifted rows across the keyset window, so the stored cursors can no + // longer be trusted and state must be re-established from scratch rather + // than merged (a merge on a shifted window would splice stale rows back + // in and render duplicate resource ids/React keys). The polling effect + // also forces a full walk every 10th tick regardless, bounding the + // staleness of untracked pages and of totalCount/pageInfo to 20s. + const pollActivePages = useCallback(async () => { + if (loadMoreInFlightRef.current) return false + const loadedResourceCount = loadedResourceCountRef.current + const loadedPageCount = Math.max( + 1, + Math.ceil(loadedResourceCount / PAGE_SIZE) + ) + const cursors = pageCursorsRef.current + const tailPageIndex = loadedPageCount - 1 + + if (!bookkeepingValidRef.current || cursors.length < loadedPageCount) { + return refreshLoadedResources() + } + + const requestId = ++refreshRequestRef.current + const targetPages = Array.from( + new Set([0, ...Array.from(activePageIndexesRef.current)]) + ) + .filter((pageIndex) => pageIndex < loadedPageCount) + .sort((a, b) => a - b) + + try { + const fetchedByPage = new Map() + let page0Connection: GetKbResourcesQuery['getKbResources'] | null = null + let tailConnection: GetKbResourcesQuery['getKbResources'] | null = null + + for (const pageIndex of targetPages) { + const { data: polledData } = await apolloClient.query< + GetKbResourcesQuery, + GetKbResourcesQueryVariables + >({ + query: GetKbResourcesDocument, + variables: getPageVariables(cursors[pageIndex] ?? null), + fetchPolicy: 'no-cache', + }) + if (requestId !== refreshRequestRef.current) return false + + const polledConnection = polledData?.getKbResources ?? null + if (!polledConnection) { + bookkeepingValidRef.current = false + return refreshLoadedResources() + } + + const isTailPage = pageIndex === tailPageIndex + const expectedLength = isTailPage + ? loadedResourceCount - pageIndex * PAGE_SIZE + : PAGE_SIZE + if (polledConnection.items.length !== expectedLength) { + bookkeepingValidRef.current = false + return refreshLoadedResources() + } + + // A non-tail page's fresh end cursor is the cursor the next loaded + // page must resume from. If it no longer matches what the last full + // walk (or a prior poll) recorded, a concurrent insert/delete + // absorbed rows across the page boundary and the loaded window has + // shifted -- detected here, before any merge is applied, so a + // shifted page is never spliced while downstream pages stay stale. + if ( + !isTailPage && + (polledConnection.pageInfo.endCursor ?? null) !== + (cursors[pageIndex + 1] ?? null) + ) { + bookkeepingValidRef.current = false + return refreshLoadedResources() + } + + fetchedByPage.set(pageIndex, polledConnection.items) + if (pageIndex === 0) page0Connection = polledConnection + if (isTailPage) tailConnection = polledConnection + } + + if (requestId !== refreshRequestRef.current) return false + + fetchedByPage.forEach((items, pageIndex) => { + if (items.some(isActiveResource)) { + activePageIndexesRef.current.add(pageIndex) + } else { + activePageIndexesRef.current.delete(pageIndex) + } + }) + updateQuery((previous) => { + const items = previous.getKbResources.items.slice() + fetchedByPage.forEach((pageItems, pageIndex) => { + // pageItems.length was already checked against expectedLength + // above (both tail and non-tail); using it directly keeps the + // splice's deleteCount correct for a tail page shorter than + // PAGE_SIZE instead of over-deleting into the next page's rows. + items.splice(pageIndex * PAGE_SIZE, pageItems.length, ...pageItems) + }) + return { + ...previous, + getKbResources: { + ...previous.getKbResources, + items, + // totalCount is connection-level and fresh on any page fetch, + // so page 0's connection (always fetched) always carries it. + totalCount: + page0Connection?.totalCount ?? previous.getKbResources.totalCount, + // pageInfo describes the window after whichever page it came + // from -- taking it from page 0 would corrupt loadMore's + // endCursor unless page 0 is also the tail page, so only take + // it when the tail page was actually fetched this tick; + // otherwise leave it as-is (the periodic full walk bounds its + // staleness). + pageInfo: + tailConnection?.pageInfo ?? previous.getKbResources.pageInfo, + }, + } + }) + return true + } catch (pollError) { + if ( + requestId !== refreshRequestRef.current || + (pollError as { name?: string }).name === 'AbortError' + ) { + return false + } + throw pollError + } + }, [apolloClient, getPageVariables, refreshLoadedResources, updateQuery]) + + useEffect(() => { + if (!polling) return + + // Reset on every (re)start so a filter/kb change or a poll-stop/restart + // doesn't inherit a stale count from a previous run. + pollTickRef.current = 0 + + const pollCurrentPage = async () => { + if (pollInFlightRef.current || loadMoreInFlightRef.current) return + pollInFlightRef.current = true + try { + pollTickRef.current += 1 + // Every 10th tick (20s at the 2s interval below) runs a full walk + // instead of the bounded poll -- this is what discovers activations + // on pages the bounded poll doesn't track, and bounds the staleness + // of totalCount/pageInfo, restoring parity with polling every loaded + // page at a fixed bound. + if (pollTickRef.current % 10 === 0) { + await refreshLoadedResources() + } else { + await pollActivePages() + } + } finally { + pollInFlightRef.current = false + } + } + + const intervalId = window.setInterval(() => { + void pollCurrentPage().catch((pollError) => { + console.error('Failed to poll KB resource operations', pollError) + }) + }, 2000) + return () => { + window.clearInterval(intervalId) + refreshRequestRef.current += 1 + pollInFlightRef.current = false + } + }, [polling, pollActivePages, refreshLoadedResources]) + + useEffect(() => { + refreshRequestRef.current += 1 + setSelectedIds(new Set()) + // The kb/filter context changed, so the query variables changed too -- + // any cursors and active-page indexes recorded for the previous context + // no longer describe this result set. Reset the bounded-poll bookkeeping + // so the next poll re-establishes it with a full walk. + pageCursorsRef.current = [null] + activePageIndexesRef.current = new Set() + bookkeepingValidRef.current = false + }, [deferredSearch, kbId, statusFilter, typeFilter]) + + useEffect(() => { + if (refreshKey === 0) return + const refreshTrigger = `${kbId}:${refreshKey}` + if (handledRefreshTriggerRef.current === refreshTrigger) return + handledRefreshTriggerRef.current = refreshTrigger + void refreshLoadedResources().catch((refreshError) => { + console.error('Failed to refresh KB resources', refreshError) + toast({ type: 'error', message: t('kb.resourcesLoadError') }) + }) + }, [kbId, refreshKey, refreshLoadedResources, t]) + + useEffect(() => { + const currentSelectableIds = new Set( + resources + .filter((resource) => !isActiveResource(resource)) + .map(({ id }) => id) + ) + setSelectedIds((current) => { + const next = new Set( + Array.from(current).filter((id) => currentSelectableIds.has(id)) + ) + return next.size === current.size ? current : next + }) + }, [resources]) + + useEffect(() => { + if (selectedResources.length === 0) { + setBulkDeletionOpen(false) + } + }, [selectedResources.length]) + + const formatFileSize = (sizeBytes: number | null | undefined) => { + if (sizeBytes === null || sizeBytes === undefined) return '—' + if (sizeBytes < 1024) return `${format.number(sizeBytes)} B` + if (sizeBytes < 1024 * 1024) { + return `${format.number(sizeBytes / 1024, { + maximumFractionDigits: 1, + })} KiB` + } + return `${format.number(sizeBytes / (1024 * 1024), { + maximumFractionDigits: 1, + })} MiB` + } + + const getStatusPresentation = (status: KbResourceStatus) => { + switch (status) { + case KbResourceStatus.Added: + return { + label: t('kb.statusAdded'), + className: 'border-slate-300 bg-slate-100 text-slate-700', + } + case KbResourceStatus.Queued: + return { + label: t('kb.statusQueued'), + className: 'border-amber-300 bg-amber-100 text-amber-900', + } + case KbResourceStatus.Processing: + return { + label: t('kb.statusProcessing'), + className: 'border-amber-300 bg-amber-100 text-amber-900', + } + case KbResourceStatus.Ready: + return { + label: t('kb.statusReady'), + className: 'border-green-300 bg-green-100 text-green-800', + } + case KbResourceStatus.Failed: + return { + label: t('kb.statusFailed'), + className: 'border-red-300 bg-red-100 text-red-800', + } + } + } + + const renderStatus = (resource: KnowledgeBaseResource) => { + if (resource.latestIngestionRun) { + return ( + + ) + } + const status = getStatusPresentation(resource.status) + return ( + + {status.label} + + ) + } + + const refreshWorkspace = async () => { + await Promise.all([refreshLoadedResources(), onMetricsChanged()]) + } + + const handleIngest = async (resource: KnowledgeBaseResource) => { + if (ingestingId !== null) return + setIngestingId(resource.id) + try { + try { + await ingestResource({ variables: { id: resource.id } }) + } catch (mutationError) { + console.error('Failed to queue KB resource ingestion', mutationError) + const code = getGraphQLErrorCode(mutationError) + const message = + code === 'KB_INGESTION_DISABLED' + ? t('kb.ingestionDisabledError') + : t('kb.ingestResourceError') + toast({ type: 'error', message }) + return + } + + setSelectedIds((current) => { + const next = new Set(current) + next.delete(resource.id) + return next + }) + + await refreshAfterMutation( + refreshWorkspace, + 'KB resources after ingestion' + ) + setHistoryRefreshes((current) => ({ + ...current, + [resource.id]: (current[resource.id] ?? 0) + 1, + })) + toast({ type: 'success', message: t('kb.ingestResourceSuccess') }) + } finally { + setIngestingId(null) + } + } + + const getIngestActionLabel = (resource: KnowledgeBaseResource) => { + if (resource.status === KbResourceStatus.Added) { + return t('kb.ingestResource') + } + if (resource.status === KbResourceStatus.Failed) { + return t('kb.retryIngestion') + } + return t('kb.reingestResource') + } + + const loadMore = async () => { + if ( + !connection?.pageInfo.hasNextPage || + loadingMore || + loadMoreInFlightRef.current + ) { + return + } + loadMoreInFlightRef.current = true + // Fence any poll/full refresh that captured the old loaded window before + // this request started. Polling also observes loadMoreInFlightRef and + // cannot start another refresh until the appended page is committed. A + // single queued full refresh then reconciles any explicit refresh that + // overlapped this request instead of silently dropping it. + refreshAfterLoadMoreRef.current = true + refreshRequestRef.current += 1 + const newPageIndex = Math.ceil(resources.length / PAGE_SIZE) + const cursorUsed = connection.pageInfo.endCursor ?? null + let newPageEndCursor: string | null = null + let newPageHasActive = false + try { + await fetchMore({ + variables: { after: connection.pageInfo.endCursor }, + updateQuery: (previous, { fetchMoreResult }) => { + newPageEndCursor = + fetchMoreResult.getKbResources.pageInfo.endCursor ?? null + newPageHasActive = + fetchMoreResult.getKbResources.items.some(isActiveResource) + const items = [ + ...previous.getKbResources.items, + ...fetchMoreResult.getKbResources.items, + ] + // Keep promise-only refreshes correct even if an interval fires + // before React renders the appended Apollo result. + loadedResourceCountRef.current = items.length + return { + ...fetchMoreResult, + getKbResources: { + ...fetchMoreResult.getKbResources, + items, + }, + } + }, + }) + // Keep the bounded-poll bookkeeping in sync with the newly loaded page + // so an active row on it is picked up by the next poll without a full + // walk. + pageCursorsRef.current[newPageIndex] = cursorUsed + pageCursorsRef.current[newPageIndex + 1] = newPageEndCursor + if (newPageHasActive) { + activePageIndexesRef.current.add(newPageIndex) + } + } finally { + loadMoreInFlightRef.current = false + if (refreshAfterLoadMoreRef.current) { + refreshAfterLoadMoreRef.current = false + void refreshLoadedResources().catch((refreshError) => { + console.error( + 'Failed to refresh KB resources after loading more', + refreshError + ) + toast({ type: 'error', message: t('kb.resourcesLoadError') }) + }) + } + } + } + + const toggleSelection = (id: string) => { + setSelectedIds((current) => { + const next = new Set(current) + if (next.has(id)) next.delete(id) + else if (next.size < MAX_BULK_SELECTION) next.add(id) + return next + }) + } + + const togglePageSelection = () => { + setSelectedIds(allPageSelected ? new Set() : new Set(bulkSelectableIds)) + } + + return ( +
+
+

{t('kb.resourcesTitle')}

+ {selectedIds.size > 0 ? ( + + ) : null} +
+ +
+ + + setTypeFilter( + newValue === FILTER_ALL_VALUE ? '' : (newValue as KbResourceType) + ) + } + items={[ + { value: FILTER_ALL_VALUE, label: t('kb.filterAll') }, + { value: KbResourceType.Blob, label: t('kb.typeFile') }, + { value: KbResourceType.Url, label: t('kb.typeUrl') }, + ]} + data={{ cy: 'kb-resource-type-filter' }} + className={{ root: 'w-full', select: { trigger: 'w-full' } }} + /> + + setStatusFilter( + newValue === FILTER_ALL_VALUE + ? '' + : (newValue as KbIngestionStatus) + ) + } + items={[ + { value: FILTER_ALL_VALUE, label: t('kb.filterAll') }, + { value: KbIngestionStatus.Queued, label: t('kb.runStatusQueued') }, + { + value: KbIngestionStatus.Processing, + label: t('kb.runStatusProcessing'), + }, + { + value: KbIngestionStatus.Succeeded, + label: t('kb.runStatusSucceeded'), + }, + { value: KbIngestionStatus.Failed, label: t('kb.runStatusFailed') }, + { + value: KbIngestionStatus.Superseded, + label: t('kb.runStatusSuperseded'), + }, + ]} + data={{ cy: 'kb-resource-status-filter' }} + className={{ root: 'w-full', select: { trigger: 'w-full' } }} + /> +
+ + {polling ? ( + + ) : null} + + {loading && !connection ? ( +
+
+ ) : error ? ( + + ) : resources.length === 0 ? ( +
+

+ {deferredSearch || typeFilter || statusFilter + ? t('kb.noResourceResults') + : t('kb.noResources')} +

+ {!deferredSearch && !typeFilter && !statusFilter ? ( + + ) : null} +
+ ) : ( + <> +
+

+ {t('kb.resourceResultCount', { + count: connection?.totalCount ?? 0, + })} +

+ +
+
    + {resources.map((resource) => { + const active = isActiveResource(resource) + return ( +
  • +
    + +
    +
    +
    +
    +
    +
    + {t('kb.operationStatus')} +
    +
    + {renderStatus(resource)} + + {t('kb.version', { + version: resource.resourceVersion, + })} + +
    + {resource.latestIngestionRun ? ( + + ) : null} + +
    +
    +
    + {t('kb.servingStatus')} +
    +
    + +
    + {resource.ingestedAt ? ( +
    + {t('kb.servingSince', { + date: format.dateTime( + new Date(resource.ingestedAt), + { + dateStyle: 'medium', + timeStyle: 'short', + } + ), + })} +
    + ) : null} +
    +
    +
    +
    + + +
    +
    +
    + {t('kb.updatedAt', { + date: format.dateTime(new Date(resource.updatedAt), { + dateStyle: 'medium', + timeStyle: 'short', + }), + })} +
    +
    • + ) + })} +
+ {connection?.pageInfo.hasNextPage ? ( +
+ +
+ ) : null} + + )} + + {inspectorResource ? ( + setInspectorId(null)} + title={t('kb.inspectorTitle')} + primaryLabel={getIngestActionLabel(inspectorResource)} + primaryLoading={ingestingId === inspectorResource.id} + primaryDisabled={ + ingestingId !== null || isActiveResource(inspectorResource) + } + onPrimaryAction={() => handleIngest(inspectorResource)} + secondaryLabel={t('shared.generic.close')} + onSecondaryAction={() => setInspectorId(null)} + dataContent={{ cy: 'kb-resource-inspector' }} + dataCloseButton={{ cy: 'close-kb-resource-inspector' }} + dataPrimaryAction={{ cy: 'ingest-kb-resource-inspector' }} + dataSecondaryAction={{ cy: 'done-kb-resource-inspector' }} + className={{ content: 'max-w-2xl' }} + > +
+
+
+ {inspectorResource.title} +
+
{renderStatus(inspectorResource)}
+ +
+
+
+
+ {t('kb.sourceType')} +
+
+ {inspectorResource.type === KbResourceType.Blob + ? t('kb.typeFile') + : t('kb.typeUrl')} +
+
+
+
+ {t('kb.fileSize')} +
+
+ {formatFileSize(inspectorResource.sizeBytes)} +
+
+
+
+ {t('kb.fileName')} +
+
+ {inspectorResource.originalFilename || '—'} +
+
+
+
+ {t('kb.mimeType')} +
+
+ {inspectorResource.mimeType || '—'} +
+
+
+
+ {t('kb.sourceLocation')} +
+
+ {inspectorResource.sourceUrl || '—'} +
+
+
+
+ {t('kb.createdAt')} +
+
+ {format.dateTime(new Date(inspectorResource.createdAt), { + dateStyle: 'medium', + timeStyle: 'short', + })} +
+
+
+
+ {t('kb.servingStatus')} +
+
+ +
+
+
+ + +
+ + ) : null} + + {deletionTarget ? ( + setDeletionTarget(null)} + onDeleted={async () => { + setSelectedIds((current) => { + const next = new Set(current) + next.delete(deletionTarget.id) + return next + }) + await refreshWorkspace() + }} + /> + ) : null} + + {bulkDeletionOpen && selectedResources.length > 0 ? ( + setBulkDeletionOpen(false)} + onDeleted={async () => { + setSelectedIds(new Set()) + await refreshWorkspace() + }} + /> + ) : null} +
+ ) +} + +export default KnowledgeBaseResourceList diff --git a/packages/kb-management/src/components/KnowledgeBaseUrlForm.tsx b/packages/kb-management/src/components/KnowledgeBaseUrlForm.tsx new file mode 100644 index 0000000000..d5395614ba --- /dev/null +++ b/packages/kb-management/src/components/KnowledgeBaseUrlForm.tsx @@ -0,0 +1,128 @@ +import { useMutation } from '@apollo/client' +import { CreateKbUrlResourceDocument } from '@klicker-uzh/graphql/dist/ops' +import { Button, H3, TextField, toast } from '@uzh-bf/design-system' +import { useTranslations } from 'next-intl' +import React, { type FormEvent, useState } from 'react' +import { getGraphQLErrorCode } from '../graphqlError' +import { refreshAfterMutation } from '../refreshAfterMutation' + +function isValidWebUrl(value: string) { + try { + const parsed = new URL(value) + return parsed.protocol === 'http:' || parsed.protocol === 'https:' + } catch { + return false + } +} + +function KnowledgeBaseUrlForm({ + kbId, + onResourceCreated, +}: { + kbId: string + onResourceCreated: () => Promise +}) { + const t = useTranslations() + const [title, setTitle] = useState('') + const [url, setUrl] = useState('') + const [urlTouched, setUrlTouched] = useState(false) + const [createUrlResource, { loading }] = useMutation( + CreateKbUrlResourceDocument + ) + const urlValid = isValidWebUrl(url.trim()) + const urlInvalid = urlTouched && Boolean(url.trim()) && !urlValid + const valid = Boolean(title.trim()) && urlValid + + const handleSubmit = async (event: FormEvent) => { + event.preventDefault() + if (!valid || loading) return + + try { + await createUrlResource({ + variables: { kbId, title: title.trim(), url: url.trim() }, + }) + } catch (error) { + console.error('Failed to create KB URL resource', error) + const code = getGraphQLErrorCode(error) + const message = + code === 'KB_RESOURCE_LIMIT_REACHED' + ? t('kb.resourceLimitError') + : code === 'KB_STORAGE_LIMIT_REACHED' + ? t('kb.storageLimitError') + : code === 'KB_INGESTION_DISABLED' + ? t('kb.ingestionDisabledError') + : t('kb.linkError') + toast({ type: 'error', message }) + return + } + + await refreshAfterMutation( + onResourceCreated, + 'KB resources after link creation' + ) + setTitle('') + setUrl('') + setUrlTouched(false) + toast({ type: 'success', message: t('kb.linkSuccess') }) + } + + return ( + + ) +} + +export default KnowledgeBaseUrlForm diff --git a/packages/kb-management/src/components/KnowledgeGraphPanel.tsx b/packages/kb-management/src/components/KnowledgeGraphPanel.tsx new file mode 100644 index 0000000000..d2730bea2e --- /dev/null +++ b/packages/kb-management/src/components/KnowledgeGraphPanel.tsx @@ -0,0 +1,540 @@ +'use client' + +import { + ApolloError, + useApolloClient, + useMutation, + useQuery, +} from '@apollo/client' +import type { + GetKbKnowledgeGraphNeighborsQuery, + GetKbKnowledgeGraphOverviewQuery, +} from '@klicker-uzh/graphql/dist/ops' +import { + GetKbKnowledgeGraphConfigDocument, + GetKbKnowledgeGraphNeighborsDocument, + GetKbKnowledgeGraphOverviewDocument, + KbGraphBuildStatus, + KbGraphCostStatus, + KbGraphQualityTier, + RebuildKbKnowledgeGraphDocument, + SearchKbKnowledgeGraphDocument, + SetKbKnowledgeGraphEnabledDocument, +} from '@klicker-uzh/graphql/dist/ops' +import type { KnowledgeGraphDataSource } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import { KnowledgeGraphUnavailableError } from '@klicker-uzh/shared-components/src/knowledgeGraph/knowledgeGraphState' +import type { KnowledgeGraphResponse } from '@klicker-uzh/types' +import { Badge, Button, H3, SelectField, Switch } from '@uzh-bf/design-system' +import { useFormatter, useTranslations } from 'next-intl' +import dynamic from 'next/dynamic' +import React, { useEffect, useMemo, useState } from 'react' + +const KnowledgeGraphViewer = dynamic( + () => + import( + '@klicker-uzh/shared-components/src/knowledgeGraph/KnowledgeGraphViewer' + ).then((module) => module.KnowledgeGraphViewer), + { ssr: false } +) + +type GraphResponse = + | GetKbKnowledgeGraphOverviewQuery['getKbKnowledgeGraphOverview'] + | GetKbKnowledgeGraphNeighborsQuery['getKbKnowledgeGraphNeighbors'] + +function toKnowledgeGraphResponse( + response: GraphResponse +): KnowledgeGraphResponse { + return { + kbId: response.kbId, + buildId: response.buildId, + isStale: response.isStale, + truncated: response.truncated, + nodes: response.nodes.map((node) => ({ + id: node.id, + labels: node.labels, + kind: node.kind, + displayLabel: node.displayLabel, + ...(node.summary == null ? {} : { summary: node.summary }), + ...(node.content == null ? {} : { content: node.content }), + degree: node.degree, + sourceReferences: node.sourceReferences.map((source) => ({ + resourceId: source.resourceId, + title: source.title, + ...(source.reference == null ? {} : { reference: source.reference }), + })), + })), + edges: response.edges.map((edge) => ({ + id: edge.id, + source: edge.source, + target: edge.target, + type: edge.type, + label: edge.label, + properties: edge.properties as Record, + })), + } +} + +function normalizeGraphError(error: unknown): never { + if ( + error instanceof ApolloError && + error.graphQLErrors.some((graphQLError) => + String(graphQLError.extensions?.code ?? '').startsWith('KB_GRAPH_') + ) + ) { + throw new KnowledgeGraphUnavailableError() + } + + throw error +} + +type KnowledgeGraphStatusLabels = { + empty: string + queued: string + processing: string + succeeded: string + failed: string +} + +type KnowledgeGraphCostStatusLabels = { + reserved: string + settled: string + released: string + needsHumanReview: string +} + +function statusLabel( + status: KbGraphBuildStatus | null | undefined, + labels: KnowledgeGraphStatusLabels +) { + switch (status) { + case KbGraphBuildStatus.Queued: + return labels.queued + case KbGraphBuildStatus.Processing: + return labels.processing + case KbGraphBuildStatus.Succeeded: + return labels.succeeded + case KbGraphBuildStatus.Failed: + return labels.failed + default: + return labels.empty + } +} + +function costStatusLabel( + status: KbGraphCostStatus | null | undefined, + labels: KnowledgeGraphCostStatusLabels +) { + switch (status) { + case KbGraphCostStatus.Reserved: + return labels.reserved + case KbGraphCostStatus.Settled: + return labels.settled + case KbGraphCostStatus.Released: + return labels.released + case KbGraphCostStatus.NeedsHumanReview: + return labels.needsHumanReview + default: + return '—' + } +} + +function formatMinorUnits( + format: ReturnType, + amountMinorUnits: number | null | undefined, + currency: string | null | undefined +) { + if (amountMinorUnits == null || currency == null) return '—' + return format.number(amountMinorUnits / 100, { + style: 'currency', + currency, + }) +} + +function KnowledgeGraphPreview({ kbId }: { kbId: string }) { + const t = useTranslations() + const apolloClient = useApolloClient() + const dataSource = useMemo( + () => ({ + overview: async () => { + try { + const { data } = await apolloClient.query({ + query: GetKbKnowledgeGraphOverviewDocument, + variables: { kbId }, + fetchPolicy: 'network-only', + }) + return toKnowledgeGraphResponse(data.getKbKnowledgeGraphOverview) + } catch (error) { + return normalizeGraphError(error) + } + }, + search: async (query) => { + try { + const { data } = await apolloClient.query({ + query: SearchKbKnowledgeGraphDocument, + variables: { kbId, query }, + fetchPolicy: 'network-only', + }) + return toKnowledgeGraphResponse(data.searchKbKnowledgeGraph) + } catch (error) { + return normalizeGraphError(error) + } + }, + neighbors: async (nodeId) => { + try { + const { data } = await apolloClient.query({ + query: GetKbKnowledgeGraphNeighborsDocument, + variables: { kbId, nodeId }, + fetchPolicy: 'network-only', + }) + return toKnowledgeGraphResponse(data.getKbKnowledgeGraphNeighbors) + } catch (error) { + return normalizeGraphError(error) + } + }, + }), + [apolloClient, kbId] + ) + + return ( +
+ +
+ ) +} + +function KnowledgeGraphPanel({ kbId }: { kbId: string }) { + const t = useTranslations() + const format = useFormatter() + const [selectedTier, setSelectedTier] = useState( + KbGraphQualityTier.Standard + ) + const [operationError, setOperationError] = useState(null) + const { data, loading, error, refetch, startPolling, stopPolling } = useQuery( + GetKbKnowledgeGraphConfigDocument, + { + variables: { kbId }, + fetchPolicy: 'network-only', + notifyOnNetworkStatusChange: true, + } + ) + const [rebuildGraph, { loading: isRebuilding }] = useMutation( + RebuildKbKnowledgeGraphDocument + ) + const [setGraphEnabled, { loading: isTogglingEnabled }] = useMutation( + SetKbKnowledgeGraphEnabledDocument + ) + const config = data?.getKbKnowledgeGraphConfig + const formattedBillingLabel = + config?.billingLabel === 'SEMESTER_QUOTA' + ? t('kb.graphBillingSemesterQuota') + : config?.billingLabel === 'PROVIDER_BILLED' + ? t('kb.graphBillingProvider') + : '—' + const isActive = + config?.status === KbGraphBuildStatus.Queued || + config?.status === KbGraphBuildStatus.Processing + const hasPublishedGraph = config?.publishedBuildId != null + const selectedEstimate = + selectedTier === KbGraphQualityTier.High + ? config?.highEstimateMinorUnits + : config?.standardEstimateMinorUnits + const formattedSelectedEstimate = formatMinorUnits( + format, + selectedEstimate, + config?.quotaCurrency + ) + + useEffect(() => { + if (config?.qualityTier != null && !isActive) { + setSelectedTier(config.qualityTier) + } + }, [config?.qualityTier, isActive]) + + useEffect(() => { + if (isActive) { + startPolling(30_000) + } else { + stopPolling() + } + return stopPolling + }, [isActive, startPolling, stopPolling]) + + const tierItems = [ + { + value: KbGraphQualityTier.Standard, + label: t('kb.graphQualityStandard'), + }, + { + value: KbGraphQualityTier.High, + label: t('kb.graphQualityHigh'), + }, + ] + const statusLabels: KnowledgeGraphStatusLabels = { + empty: t('kb.graphStatusEmpty'), + queued: t('kb.graphStatusQueued'), + processing: t('kb.graphStatusProcessing'), + succeeded: t('kb.graphStatusSucceeded'), + failed: t('kb.graphStatusFailed'), + } + const costStatusLabels: KnowledgeGraphCostStatusLabels = { + reserved: t('kb.graphCostStatusReserved'), + settled: t('kb.graphCostStatusSettled'), + released: t('kb.graphCostStatusReleased'), + needsHumanReview: t('kb.graphCostStatusNeedsHumanReview'), + } + + const handleRebuild = async () => { + if (isRebuilding || isActive || !config?.isEnabled) return + + setOperationError(null) + try { + await rebuildGraph({ + variables: { kbId, qualityTier: selectedTier }, + }) + await refetch() + } catch (mutationError) { + console.error('Failed to rebuild KB knowledge graph', { kbId }) + setOperationError(t('kb.graphBuildError')) + } + } + + const handleEnabledChange = async (enabled: boolean) => { + setOperationError(null) + try { + await setGraphEnabled({ variables: { kbId, enabled } }) + try { + await refetch() + } catch { + console.warn('Failed to refresh KB knowledge graph opt-in', { kbId }) + } + } catch { + console.error('Failed to update KB knowledge graph opt-in', { kbId }) + setOperationError(t('kb.graphEnableError')) + } + } + + return ( +
+
+

{t('kb.graphTitle')}

+

+ {t('kb.graphDescription')} +

+
+ + {loading && data === undefined ? ( +

+ {t('kb.graphLoading')} +

+ ) : error || config === undefined ? ( +
+
+ {t('kb.graphLoadError')} + +
+
+ ) : ( + <> +
+ void handleEnabledChange(enabled)} + disabled={isTogglingEnabled} + data={{ cy: 'kb-knowledge-graph-enabled' }} + /> +

+ {config.isEnabled + ? t('kb.graphEnabledDescription') + : t('kb.graphDisabledDescription')} +

+ {!config.costConfigurationReady ? ( +

+ {t('kb.graphCostUnavailable')} +

+ ) : null} +
+ + setSelectedTier(value as KbGraphQualityTier) + } + disabled={ + isActive || + isRebuilding || + !config.isEnabled || + !config.costConfigurationReady + } + data={{ cy: 'kb-knowledge-graph-quality-tier' }} + /> + +
+

+ {t('kb.graphBuildCost', { amount: formattedSelectedEstimate })} +

+
+

+ + {t('kb.graphBillingLabel')}: + {' '} + {formattedBillingLabel} +

+

+ + {t('kb.graphRemainingQuota')}: + {' '} + {formatMinorUnits( + format, + config.remainingSemesterQuotaMinorUnits, + config.quotaCurrency + )} +

+

+ + {t('kb.graphWorstCaseBalance')}: + {' '} + {formatMinorUnits( + format, + config.worstCaseRemainingMinorUnits, + config.quotaCurrency + )} +

+

+ {t('kb.graphMaxCost')}:{' '} + {formatMinorUnits( + format, + config.maxCostMinorUnits, + config.quotaCurrency + )} +

+ {config.costStatus ? ( +

+ + {t('kb.graphCostStatus')}: + {' '} + {costStatusLabel(config.costStatus, costStatusLabels)} +

+ ) : null} + {config.actualCostMinorUnits != null ? ( +

+ + {t('kb.graphActualCost')}: + {' '} + {formatMinorUnits( + format, + config.actualCostMinorUnits, + config.costCurrency + )} +

+ ) : null} +
+ {config.actualRequestCount != null ? ( +

+ {t('kb.graphActualUsage', { + requests: config.actualRequestCount, + inputTokens: config.actualInputTokens ?? 0, + outputTokens: config.actualOutputTokens ?? 0, + embeddingTokens: config.actualEmbeddingTokens ?? 0, + })} +

+ ) : null} +
+
+ + {t('kb.graphStatusLabel')}: + + + {statusLabel(config.status, statusLabels)} + + {config.isStale && hasPublishedGraph ? ( + {t('kb.graphStale')} + ) : null} +
+ {config.buildId ? ( +

+ {t('kb.graphBuildId', { buildId: config.buildId })} +

+ ) : null} +
+
+ + {operationError ? ( +

+ {operationError} +

+ ) : null} + +
+

+ {t('kb.graphPreviewTitle')} +

+ {hasPublishedGraph ? ( + + ) : ( +
+ {t('kb.graphPreviewUnavailable')} +
+ )} +
+ + )} +
+ ) +} + +export default KnowledgeGraphPanel diff --git a/packages/kb-management/src/graphqlError.ts b/packages/kb-management/src/graphqlError.ts new file mode 100644 index 0000000000..df4a956876 --- /dev/null +++ b/packages/kb-management/src/graphqlError.ts @@ -0,0 +1,12 @@ +export function getGraphQLErrorCode(error: unknown) { + if (!error || typeof error !== 'object') return null + const graphQLErrors = ( + error as { + graphQLErrors?: Array<{ extensions?: { code?: unknown } }> + } + ).graphQLErrors + const code = graphQLErrors?.find( + ({ extensions }) => typeof extensions?.code === 'string' + )?.extensions?.code + return typeof code === 'string' ? code : null +} diff --git a/packages/kb-management/src/index.ts b/packages/kb-management/src/index.ts new file mode 100644 index 0000000000..cda5c4b966 --- /dev/null +++ b/packages/kb-management/src/index.ts @@ -0,0 +1,2 @@ +export { default as KnowledgeBaseDetail } from './KnowledgeBaseDetail' +export { default as KnowledgeBaseManager } from './KnowledgeBaseManager' diff --git a/packages/kb-management/src/refreshAfterMutation.ts b/packages/kb-management/src/refreshAfterMutation.ts new file mode 100644 index 0000000000..9d77b025b7 --- /dev/null +++ b/packages/kb-management/src/refreshAfterMutation.ts @@ -0,0 +1,10 @@ +export async function refreshAfterMutation( + refresh: () => Promise, + context: string +) { + try { + await refresh() + } catch (error) { + console.error(`Failed to refresh ${context}`, error) + } +} diff --git a/packages/kb-management/tsconfig.json b/packages/kb-management/tsconfig.json new file mode 100644 index 0000000000..77d13af86e --- /dev/null +++ b/packages/kb-management/tsconfig.json @@ -0,0 +1,20 @@ +{ + "include": ["src/**/*.ts", "src/**/*.tsx"], + "compilerOptions": { + "jsx": "react", + "esModuleInterop": true, + "skipLibCheck": true, + "target": "es2022", + "allowJs": true, + "resolveJsonModule": true, + "moduleDetection": "force", + "isolatedModules": true, + "verbatimModuleSyntax": true, + "strict": true, + "noUncheckedIndexedAccess": true, + "noImplicitOverride": true, + "module": "preserve", + "noEmit": true, + "lib": ["es2022", "dom", "dom.iterable"] + } +} diff --git a/packages/knowledge-graph/package.json b/packages/knowledge-graph/package.json new file mode 100644 index 0000000000..d159860724 --- /dev/null +++ b/packages/knowledge-graph/package.json @@ -0,0 +1,50 @@ +{ + "name": "@klicker-uzh/knowledge-graph", + "version": "3.3.0-alpha.82", + "license": "AGPL-3.0", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "files": [ + "dist" + ], + "dependencies": { + "@klicker-uzh/prisma": "workspace:*", + "@klicker-uzh/types": "workspace:*", + "falkordb": "6.6.2" + }, + "devDependencies": { + "@parcel/watcher": "~2.4.1", + "@rollup/plugin-node-resolve": "~15.3.1", + "@rollup/plugin-typescript": "~12.1.4", + "@types/node": "^24.10.1", + "cross-env": "~7.0.3", + "npm-run-all": "~4.1.5", + "rollup": "~4.34.9", + "typescript": "~6.0.3", + "vitest": "~3.2.4" + }, + "scripts": { + "build": "run-s --npm-path pnpm build:ts", + "build:test": "pnpm run build", + "build:ts": "cross-env NODE_ENV=production rollup -c", + "check": "tsc --noEmit", + "dev": "run-p --npm-path pnpm dev:ts", + "dev:infisical": "../../util/_run_with_infisical.sh --env dev pnpm run dev", + "dev:ts": "cross-env NODE_ENV=development rollup -c --watch", + "test": "vitest run", + "test:watch": "vitest" + }, + "engines": { + "node": "=24" + }, + "volta": { + "extends": "../../package.json" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + } + }, + "type": "module" +} diff --git a/packages/knowledge-graph/rollup.config.js b/packages/knowledge-graph/rollup.config.js new file mode 100644 index 0000000000..2fe8ee169e --- /dev/null +++ b/packages/knowledge-graph/rollup.config.js @@ -0,0 +1,25 @@ +import { nodeResolve } from '@rollup/plugin-node-resolve' +import typescript from '@rollup/plugin-typescript' +import { defineConfig } from 'rollup' + +const config = defineConfig([ + { + input: ['src/index.ts'], + output: { + dir: 'dist', + format: 'esm', + sourcemap: true, + entryFileNames: '[name].js', + }, + plugins: [ + nodeResolve(), + typescript({ + tsconfig: './tsconfig.json', + rootDir: 'src', + }), + ], + external: [/@klicker-uzh*/, /node_modules/], + }, +]) + +export default config diff --git a/packages/knowledge-graph/src/client.ts b/packages/knowledge-graph/src/client.ts new file mode 100644 index 0000000000..2f003e584a --- /dev/null +++ b/packages/knowledge-graph/src/client.ts @@ -0,0 +1,278 @@ +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphResponse, +} from '@klicker-uzh/types' +import { FalkorDB, type Graph } from 'falkordb' + +import { + KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT, + KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT, + KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT, + type KnowledgeGraphConfig, + getKnowledgeGraphConfig, +} from './config.js' +import { + normalizeKnowledgeGraphEdge, + normalizeKnowledgeGraphNode, +} from './normalize.js' +import { type PublishedKnowledgeGraph } from './publication.js' +import { + type KnowledgeGraphEdgeRow, + type KnowledgeGraphNodeRow, + getEdgesForNodeIdsQuery, + getNeighborhoodNodesQuery, + getOverviewNodesQuery, + getSearchNodesQuery, +} from './queries.js' + +type ClientSession = { + client: FalkorDB + config: KnowledgeGraphConfig +} + +let clientSessionPromise: Promise | undefined +let beforeExitRegistered = false + +function handleClientError(): void { + // Keep credentials, connection strings, queries, and raw SDK errors out of + // application logs while retaining a safe operational signal. + console.error('Knowledge graph database connection error') +} + +function closeBeforeExit(): void { + void closeKnowledgeGraphClient() +} + +function registerBeforeExit(): void { + if (!beforeExitRegistered) { + process.once('beforeExit', closeBeforeExit) + beforeExitRegistered = true + } +} + +async function getClientSession(): Promise { + if (clientSessionPromise !== undefined) { + return clientSessionPromise + } + + const config = getKnowledgeGraphConfig() + clientSessionPromise = FalkorDB.connect({ + username: config.username, + password: config.password, + socket: { + host: config.host, + port: config.port, + tls: config.tls, + connectTimeout: config.queryTimeoutMs, + }, + }) + .then((client) => { + client.on('error', handleClientError) + registerBeforeExit() + return { client, config } + }) + .catch((error: unknown) => { + clientSessionPromise = undefined + throw error + }) + + return clientSessionPromise +} + +export async function closeKnowledgeGraphClient(): Promise { + const sessionPromise = clientSessionPromise + clientSessionPromise = undefined + + if (beforeExitRegistered) { + process.removeListener('beforeExit', closeBeforeExit) + beforeExitRegistered = false + } + + if (sessionPromise === undefined) { + return + } + + let session: ClientSession + try { + session = await sessionPromise + } catch { + return + } + session.client.removeListener('error', handleClientError) + await session.client.close() +} + +/** + * Remove one completed build's private FalkorDB graph. Callers must validate + * ownership before invoking this; the client deliberately has no notion of KB + * lifecycle or retention policy. + */ +export async function deleteKnowledgeGraph(graphName: string): Promise { + const { graph } = await graphSession(graphName) + await graph.delete() +} + +// The graph name comes from the published build rather than being recomputed, so +// a build that is being served is always read under the name it was written to. +async function graphSession(graphName: string): Promise<{ + graph: Graph + config: KnowledgeGraphConfig +}> { + const { client, config } = await getClientSession() + return { + graph: client.selectGraph(graphName), + config, + } +} + +async function readRows( + graph: Graph, + config: KnowledgeGraphConfig, + query: { cypher: string; params: Record } +): Promise { + const result = await graph.roQuery(query.cypher, { + params: query.params, + TIMEOUT: config.queryTimeoutMs, + }) + return result.data ?? [] +} + +function sourceMap(context: PublishedKnowledgeGraph) { + return new Map( + context.sources.map((source) => [source.resourceId, source] as const) + ) +} + +function normalizedNodes( + rows: KnowledgeGraphNodeRow[], + context: PublishedKnowledgeGraph +): KnowledgeGraphNode[] { + const nodes = new Map() + const sources = sourceMap(context) + + for (const row of rows) { + const node = normalizeKnowledgeGraphNode(row, sources) + if (node !== null && !nodes.has(node.id)) { + nodes.set(node.id, node) + } + } + + return Array.from(nodes.values()) +} + +function normalizedEdges(rows: KnowledgeGraphEdgeRow[]): KnowledgeGraphEdge[] { + const edges = new Map() + + for (const row of rows) { + const edge = normalizeKnowledgeGraphEdge(row) + if (edge !== null && !edges.has(edge.id)) { + edges.set(edge.id, edge) + } + } + + return Array.from(edges.values()) +} + +function response( + context: PublishedKnowledgeGraph, + nodes: KnowledgeGraphNode[], + edges: KnowledgeGraphEdge[], + truncated: boolean +): KnowledgeGraphResponse { + return { + kbId: context.kbId, + buildId: context.buildId, + isStale: context.isStale, + nodes, + edges, + truncated, + } +} + +export async function readKnowledgeGraphOverview( + context: PublishedKnowledgeGraph +): Promise { + const { graph, config } = await graphSession(context.graphName) + const nodeRows = await readRows( + graph, + config, + getOverviewNodesQuery() + ) + const nodes = normalizedNodes( + nodeRows.slice(0, KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT), + context + ) + const edgeRows = + nodes.length === 0 + ? [] + : await readRows( + graph, + config, + getEdgesForNodeIdsQuery( + nodes.map((node) => node.id), + 'overview' + ) + ) + const edges = normalizedEdges( + edgeRows.slice(0, KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT) + ) + + return response( + context, + nodes, + edges, + nodeRows.length > KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT || + edgeRows.length > KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT + ) +} + +export async function searchKnowledgeGraph( + context: PublishedKnowledgeGraph, + searchText: string +): Promise { + const query = getSearchNodesQuery(searchText) + const { graph, config } = await graphSession(context.graphName) + const rows = await readRows(graph, config, query) + + return response( + context, + normalizedNodes(rows.slice(0, KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT), context), + [], + rows.length > KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT + ) +} + +export async function readKnowledgeGraphNeighbors( + context: PublishedKnowledgeGraph, + nodeId: string +): Promise { + const query = getNeighborhoodNodesQuery(nodeId) + const { graph, config } = await graphSession(context.graphName) + const nodeRows = await readRows(graph, config, query) + const allNodes = normalizedNodes(nodeRows, context).filter( + (node) => node.id !== nodeId + ) + const nodes = allNodes.slice(0, KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT) + const edgeRows = await readRows( + graph, + config, + getEdgesForNodeIdsQuery( + [nodeId, ...nodes.map((node) => node.id)], + 'neighbors' + ) + ) + const edges = normalizedEdges( + edgeRows.slice(0, KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT) + ) + + return response( + context, + nodes, + edges, + allNodes.length > KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT || + edgeRows.length > KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT + ) +} diff --git a/packages/knowledge-graph/src/config.ts b/packages/knowledge-graph/src/config.ts new file mode 100644 index 0000000000..1e079bf30d --- /dev/null +++ b/packages/knowledge-graph/src/config.ts @@ -0,0 +1,103 @@ +export const DEFAULT_KNOWLEDGE_GRAPH_QUERY_TIMEOUT_MS = 5000 + +export const KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT = 250 +export const KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT = 500 +export const KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT = 20 +export const KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT = 100 +export const KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT = 200 + +export type KnowledgeGraphConfig = { + host: string + port: number + username?: string + password?: string + tls: boolean + queryTimeoutMs: number +} + +type KnowledgeGraphEnvironment = Record + +function parseRequiredHost(value: string | undefined): string { + const host = value?.trim() + + if (!host) { + throw new Error('KB_FALKORDB_HOST must be a non-empty value') + } + + return host +} + +function parseIntegerInRange({ + name, + value, + minimum, + maximum, +}: { + name: string + value: string | undefined + minimum: number + maximum: number +}): number { + if (value === undefined || !/^\d+$/.test(value)) { + throw new Error( + `${name} must be an integer between ${minimum} and ${maximum}` + ) + } + + const parsed = Number(value) + if (!Number.isSafeInteger(parsed) || parsed < minimum || parsed > maximum) { + throw new Error( + `${name} must be an integer between ${minimum} and ${maximum}` + ) + } + + return parsed +} + +function parseTls(value: string | undefined): boolean { + if (value === undefined) { + return false + } + + if (value === 'true') { + return true + } + + if (value === 'false') { + return false + } + + throw new Error('KB_FALKORDB_TLS must be either true or false') +} + +function optionalValue(value: string | undefined): string | undefined { + return value === '' ? undefined : value +} + +export function getKnowledgeGraphConfig( + env: KnowledgeGraphEnvironment = process.env +): KnowledgeGraphConfig { + const queryTimeoutMs = + env.KB_FALKORDB_QUERY_TIMEOUT_MS === undefined + ? DEFAULT_KNOWLEDGE_GRAPH_QUERY_TIMEOUT_MS + : parseIntegerInRange({ + name: 'KB_FALKORDB_QUERY_TIMEOUT_MS', + value: env.KB_FALKORDB_QUERY_TIMEOUT_MS, + minimum: 1, + maximum: Number.MAX_SAFE_INTEGER, + }) + + return { + host: parseRequiredHost(env.KB_FALKORDB_HOST), + port: parseIntegerInRange({ + name: 'KB_FALKORDB_PORT', + value: env.KB_FALKORDB_PORT, + minimum: 1, + maximum: 65535, + }), + username: optionalValue(env.KB_FALKORDB_USERNAME), + password: optionalValue(env.KB_FALKORDB_PASSWORD), + tls: parseTls(env.KB_FALKORDB_TLS), + queryTimeoutMs, + } +} diff --git a/packages/knowledge-graph/src/digest.ts b/packages/knowledge-graph/src/digest.ts new file mode 100644 index 0000000000..e77af0aa4f --- /dev/null +++ b/packages/knowledge-graph/src/digest.ts @@ -0,0 +1,68 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' +import { createHash } from 'node:crypto' + +type KBContentDigestPrisma = Pick + +export type KBContentDigestEntry = { + resourceId: string + contentSha256: string +} + +/** + * The KB's content identity: every resource currently serving RAG, pinned by the + * content hash ingestion last published for it. A graph build is made from exactly + * this set, so comparing digests answers "has the KB moved on since this build?". + * + * Computed on demand rather than materialized on KB, so it can never drift from + * the resources it describes. + */ +export async function readKBContentDigestEntries( + prisma: KBContentDigestPrisma, + kbId: string +): Promise { + const resources = await prisma.kBResource.findMany({ + where: { + kbId, + deletedAt: null, + // `status` belongs to the newest ingestion operation. Its predecessor can + // still be serving while that operation is queued or processing. + activeContentSha256: { not: null }, + }, + select: { id: true, activeContentSha256: true }, + orderBy: { id: 'asc' }, + }) + + return resources.flatMap((resource) => + resource.activeContentSha256 === null + ? [] + : [ + { + resourceId: resource.id, + contentSha256: resource.activeContentSha256, + }, + ] + ) +} + +export function hashKBContentDigestEntries( + entries: KBContentDigestEntry[] +): string { + const hash = createHash('sha256') + + // Ordering is fixed by the caller's `orderBy` so the digest is reproducible; + // the separators keep concatenation from aliasing across entry boundaries. + for (const entry of entries) { + hash.update(`${entry.resourceId}:${entry.contentSha256}\n`) + } + + return hash.digest('hex') +} + +export async function computeKBContentDigest( + prisma: KBContentDigestPrisma, + kbId: string +): Promise { + return hashKBContentDigestEntries( + await readKBContentDigestEntries(prisma, kbId) + ) +} diff --git a/packages/knowledge-graph/src/index.ts b/packages/knowledge-graph/src/index.ts new file mode 100644 index 0000000000..eaf9a7d020 --- /dev/null +++ b/packages/knowledge-graph/src/index.ts @@ -0,0 +1,24 @@ +export { + closeKnowledgeGraphClient, + deleteKnowledgeGraph, + readKnowledgeGraphNeighbors, + readKnowledgeGraphOverview, + searchKnowledgeGraph, +} from './client.js' +export * from './config.js' +export { + computeKBContentDigest, + hashKBContentDigestEntries, + readKBContentDigestEntries, +} from './digest.js' +export type { KBContentDigestEntry } from './digest.js' +export { + KnowledgeGraphNotPublishedError, + getKnowledgeGraphName, + getPublishedKnowledgeGraph, +} from './publication.js' +export type { + KnowledgeGraphPublicationCode, + KnowledgeGraphSourceMetadata, + PublishedKnowledgeGraph, +} from './publication.js' diff --git a/packages/knowledge-graph/src/normalize.ts b/packages/knowledge-graph/src/normalize.ts new file mode 100644 index 0000000000..d24544c1df --- /dev/null +++ b/packages/knowledge-graph/src/normalize.ts @@ -0,0 +1,285 @@ +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphSourceReference, +} from '@klicker-uzh/types' + +import type { KnowledgeGraphSourceMetadata } from './publication.js' +import type { KnowledgeGraphEdgeRow, KnowledgeGraphNodeRow } from './queries.js' + +export const KNOWLEDGE_GRAPH_CONTENT_MAX_LENGTH = 8_000 +const KNOWLEDGE_GRAPH_SUMMARY_MAX_LENGTH = 1_000 +const KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH = 300 +const KNOWLEDGE_GRAPH_EDGE_PROPERTY_MAX_LENGTH = 500 + +const SENSITIVE_PROPERTY_KEY = + /embedding|vector|password|secret|token|credential|ingestion|workflow|internal|(?:^|_)(?:url|uri|href|blob|storage|path)(?:_|$)|^(?:source_id|created_at|truncate)$/i +const SENSITIVE_PROPERTY_VALUE = + /(?:^|[^a-z\d])(?:embedding|vector|password|secret|token|credential|ingestion|workflow)(?:[^a-z\d]|$)/i +const SAS_QUERY_PARAMETER = /[?&](?:sig|se|sp|sv)=/i + +type Properties = Record + +function containsSensitiveText(value: string): boolean { + return SENSITIVE_PROPERTY_VALUE.test( + value + .replace(/([A-Z]+)([A-Z][a-z])/g, '$1_$2') + .replace(/([a-z\d])([A-Z])/g, '$1_$2') + ) +} + +function isPlainProperties(value: unknown): value is Properties { + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + return false + } + + const prototype = Object.getPrototypeOf(value) + return prototype === Object.prototype || prototype === null +} + +function isSafeText(value: unknown): value is string { + return ( + typeof value === 'string' && + value.trim().length > 0 && + !containsSensitiveText(value) && + !SAS_QUERY_PARAMETER.test(value) + ) +} + +function safeText(value: unknown, maximumLength: number): string | undefined { + if (!isSafeText(value)) { + return undefined + } + + return value.trim().slice(0, maximumLength) +} + +function firstString( + properties: Properties, + keys: string[], + maximumLength: number +): string | undefined { + for (const key of keys) { + const value = safeText(properties[key], maximumLength) + if (value !== undefined) { + return value + } + } + + return undefined +} + +function internalId(value: unknown): string | undefined { + if (typeof value === 'number' && Number.isSafeInteger(value) && value >= 0) { + return String(value) + } + + if (typeof value === 'bigint' && value >= 0n) { + return String(value) + } + + if (typeof value === 'string' && /^\d+$/.test(value)) { + return value + } + + return undefined +} + +function normalizeLabels(value: unknown): string[] { + if (!Array.isArray(value)) { + return [] + } + + return Array.from( + new Set( + value + .map((label) => safeText(label, KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH)) + .filter((label): label is string => label !== undefined) + ) + ) +} + +function normalizeDegree(value: unknown): number { + if (typeof value === 'number' && Number.isSafeInteger(value) && value >= 0) { + return value + } + + if (typeof value === 'string' && /^\d+$/.test(value)) { + const parsed = Number(value) + return Number.isSafeInteger(parsed) ? parsed : 0 + } + + return 0 +} + +function sourceIds(value: unknown): string[] { + if (typeof value === 'string') { + return [value] + } + + if (!Array.isArray(value)) { + return [] + } + + return value.filter( + (sourceId): sourceId is string => + typeof sourceId === 'string' && sourceId.length > 0 + ) +} + +function sourceReference( + properties: Properties, + sources: ReadonlyMap +): KnowledgeGraphSourceReference[] { + const referenceValue = + properties.reference ?? properties.page ?? properties.page_number + const reference = + typeof referenceValue === 'number' && Number.isFinite(referenceValue) + ? String(referenceValue) + : safeText(referenceValue, KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH) + + const seen = new Set() + const references: KnowledgeGraphSourceReference[] = [] + for (const sourceId of sourceIds(properties.source_id)) { + if (seen.has(sourceId)) { + continue + } + + const source = sources.get(sourceId) + if (!source) { + continue + } + + seen.add(sourceId) + references.push({ + resourceId: source.resourceId, + title: + safeText(source.title, KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH) ?? 'Source', + ...(reference === undefined ? {} : { reference }), + }) + } + + return references +} + +export function normalizeKnowledgeGraphNode( + row: KnowledgeGraphNodeRow, + sources: ReadonlyMap +): KnowledgeGraphNode | null { + const id = internalId(row.id) + if (id === undefined) { + return null + } + + const labels = normalizeLabels(row.labels) + const properties = isPlainProperties(row.properties) ? row.properties : {} + const displayLabel = + firstString( + properties, + ['name', 'title', 'entity'], + KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH + ) ?? `Concept ${id}` + const kind = + firstString( + properties, + ['entity_type'], + KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH + ) ?? + labels[0] ?? + 'Concept' + const summary = firstString( + properties, + ['summary'], + KNOWLEDGE_GRAPH_SUMMARY_MAX_LENGTH + ) + const content = firstString( + properties, + ['description', 'summary', 'content', 'text'], + KNOWLEDGE_GRAPH_CONTENT_MAX_LENGTH + ) + + return { + id, + labels, + kind, + displayLabel, + ...(summary === undefined ? {} : { summary }), + ...(content === undefined ? {} : { content }), + degree: normalizeDegree(row.degree), + sourceReferences: sourceReference(properties, sources), + } +} + +function safeEdgeProperties( + value: unknown +): Record { + if (!isPlainProperties(value)) { + return {} + } + + const properties: Record = {} + for (const [key, propertyValue] of Object.entries(value)) { + const normalizedKey = key + .replace(/([a-z\d])([A-Z])/g, '$1_$2') + .toLowerCase() + if (SENSITIVE_PROPERTY_KEY.test(normalizedKey)) { + continue + } + + if (typeof propertyValue === 'string') { + if (containsSensitiveText(propertyValue)) { + continue + } + + const normalized = safeText( + propertyValue, + KNOWLEDGE_GRAPH_EDGE_PROPERTY_MAX_LENGTH + ) + if (normalized !== undefined) { + properties[key] = normalized + } + continue + } + + if (typeof propertyValue === 'boolean') { + properties[key] = propertyValue + continue + } + + if (typeof propertyValue === 'number' && Number.isFinite(propertyValue)) { + properties[key] = propertyValue + } + } + + return properties +} + +export function normalizeKnowledgeGraphEdge( + row: KnowledgeGraphEdgeRow +): KnowledgeGraphEdge | null { + const id = internalId(row.id) + const source = internalId(row.source) + const target = internalId(row.target) + if (id === undefined || source === undefined || target === undefined) { + return null + } + + const properties = isPlainProperties(row.properties) ? row.properties : {} + const type = + safeText(row.type, KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH) ?? 'RELATED_TO' + const label = + firstString( + properties, + ['label', 'name', 'title'], + KNOWLEDGE_GRAPH_LABEL_MAX_LENGTH + ) ?? type + + return { + id, + source, + target, + type, + label, + properties: safeEdgeProperties(properties), + } +} diff --git a/packages/knowledge-graph/src/publication.ts b/packages/knowledge-graph/src/publication.ts new file mode 100644 index 0000000000..76d27a8819 --- /dev/null +++ b/packages/knowledge-graph/src/publication.ts @@ -0,0 +1,122 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' + +import { computeKBContentDigest } from './digest.js' + +export type KnowledgeGraphSourceMetadata = { + resourceId: string + title: string +} + +export type PublishedKnowledgeGraph = { + kbId: string + buildId: string + graphName: string + /** + * The KB's content has moved on since this build was made. The build keeps + * serving regardless — staleness is a label on the lecturer's views, never an + * outage for students (ADR 0009). + */ + isStale: boolean + sources: KnowledgeGraphSourceMetadata[] +} + +export type KnowledgeGraphPublicationCode = + | 'EMPTY' + | 'QUEUED' + | 'PROCESSING' + | 'FAILED' + +export class KnowledgeGraphNotPublishedError extends Error { + readonly code: KnowledgeGraphPublicationCode + + constructor(code: KnowledgeGraphPublicationCode) { + super('Knowledge graph is not published') + this.name = 'KnowledgeGraphNotPublishedError' + this.code = code + } +} + +/** + * Each build writes its own graph and the KB's published pointer moves to it once + * the build completes, so a graph is never mutated while it is being served. + */ +export function getKnowledgeGraphName(kbId: string, buildId: string): string { + return `klickeruzh:kb:${kbId}:${buildId}` +} + +function unpublishedCode( + latestBuild: { status: string } | null +): KnowledgeGraphPublicationCode { + if (latestBuild === null) { + return 'EMPTY' + } + + if (latestBuild.status === 'QUEUED' || latestBuild.status === 'PROCESSING') { + return latestBuild.status + } + + if (latestBuild.status === 'FAILED') { + return 'FAILED' + } + + return 'EMPTY' +} + +export async function getPublishedKnowledgeGraph( + prisma: PrismaClient, + kbId: string +): Promise { + const kb = await prisma.kB.findFirst({ + where: { id: kbId, deletedAt: null }, + select: { + publishedGraphBuildId: true, + }, + }) + + if (kb === null || kb.publishedGraphBuildId === null) { + const latestBuild = + kb === null + ? null + : await prisma.kBGraphBuild.findFirst({ + where: { kbId }, + select: { status: true }, + orderBy: { createdAt: 'desc' }, + }) + + throw new KnowledgeGraphNotPublishedError(unpublishedCode(latestBuild)) + } + + const build = await prisma.kBGraphBuild.findUnique({ + where: { id: kb.publishedGraphBuildId }, + select: { + id: true, + kbId: true, + status: true, + graphName: true, + sourceContentDigest: true, + sources: { + select: { resourceId: true, title: true }, + orderBy: { resourceId: 'asc' }, + }, + }, + }) + + // The pointer is deliberately not a database relation. Treat it as untrusted + // state: only a completed build belonging to this KB can name a served graph. + if (build === null || build.kbId !== kbId || build.status !== 'SUCCEEDED') { + throw new KnowledgeGraphNotPublishedError('EMPTY') + } + + return { + kbId, + buildId: build.id, + graphName: build.graphName, + isStale: + build.sourceContentDigest !== + (await computeKBContentDigest(prisma, kbId)), + sources: build.sources.map((source) => ({ + resourceId: source.resourceId, + title: source.title, + })), + } +} diff --git a/packages/knowledge-graph/src/queries.ts b/packages/knowledge-graph/src/queries.ts new file mode 100644 index 0000000000..0b9a99c7e6 --- /dev/null +++ b/packages/knowledge-graph/src/queries.ts @@ -0,0 +1,125 @@ +import { + KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT, + KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT, + KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT, +} from './config.js' + +export type KnowledgeGraphNodeRow = { + id: unknown + labels: unknown + properties: unknown + degree: unknown +} + +export type KnowledgeGraphEdgeRow = { + id: unknown + source: unknown + target: unknown + type: unknown + properties: unknown +} + +type FixedQuery = { + cypher: string + params: Record +} + +export class KnowledgeGraphInputError extends Error { + constructor(message: string) { + super(message) + this.name = 'KnowledgeGraphInputError' + } +} + +function validateNodeId(nodeId: string): void { + if (!/^\d+$/.test(nodeId)) { + throw new KnowledgeGraphInputError('Node ID must be a decimal integer') + } +} + +export function getOverviewNodesQuery(): FixedQuery { + return { + cypher: ` + MATCH (n) + OPTIONAL MATCH (n)--(adjacent) + WITH n, count(adjacent) AS degree + RETURN id(n) AS id, labels(n) AS labels, properties(n) AS properties, + degree AS degree + ORDER BY degree DESC, id(n) ASC + LIMIT ${KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT + 1} + `, + params: {}, + } +} + +export function getSearchNodesQuery(searchText: string): FixedQuery { + const normalizedSearchText = searchText.trim() + if (normalizedSearchText.length === 0 || normalizedSearchText.length > 100) { + throw new KnowledgeGraphInputError( + 'Search text must contain between 1 and 100 characters' + ) + } + + return { + cypher: ` + MATCH (n) + WHERE any(candidate IN [n.name, n.title, n.entity] + WHERE candidate IS NOT NULL + AND toLower(toString(candidate)) CONTAINS toLower($searchText)) + OPTIONAL MATCH (n)--(adjacent) + WITH n, count(adjacent) AS degree + RETURN id(n) AS id, labels(n) AS labels, properties(n) AS properties, + degree AS degree + ORDER BY degree DESC, id(n) ASC + LIMIT ${KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT + 1} + `, + params: { searchText: normalizedSearchText }, + } +} + +export function getNeighborhoodNodesQuery(nodeId: string): FixedQuery { + validateNodeId(nodeId) + + return { + cypher: ` + MATCH (center)--(neighbor) + WHERE id(center) = toInteger($nodeId) + WITH DISTINCT neighbor + OPTIONAL MATCH (neighbor)--(adjacent) + WITH neighbor, count(adjacent) AS degree + RETURN id(neighbor) AS id, labels(neighbor) AS labels, + properties(neighbor) AS properties, degree AS degree + ORDER BY degree DESC, id(neighbor) ASC + LIMIT ${KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT + 1} + `, + params: { nodeId }, + } +} + +export function getEdgesForNodeIdsQuery( + nodeIds: string[], + operation: 'overview' | 'neighbors' +): FixedQuery { + nodeIds.forEach(validateNodeId) + + const resultLimit = + operation === 'overview' + ? KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT + 1 + : KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT + 1 + + return { + cypher: ` + MATCH (source)-[relationship]->(target) + WHERE id(source) IN [nodeId IN $nodeIds | toInteger(nodeId)] + AND id(target) IN [nodeId IN $nodeIds | toInteger(nodeId)] + RETURN id(relationship) AS id, id(source) AS source, + id(target) AS target, type(relationship) AS type, + properties(relationship) AS properties + ORDER BY id(relationship) ASC + LIMIT ${resultLimit} + `, + params: { nodeIds }, + } +} diff --git a/packages/knowledge-graph/test/client.test.ts b/packages/knowledge-graph/test/client.test.ts new file mode 100644 index 0000000000..c7993bdd5a --- /dev/null +++ b/packages/knowledge-graph/test/client.test.ts @@ -0,0 +1,222 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const sdk = vi.hoisted(() => ({ + close: vi.fn(), + connect: vi.fn(), + on: vi.fn(), + removeListener: vi.fn(), + roQuery: vi.fn(), + query: vi.fn(), + selectGraph: vi.fn(), +})) + +vi.mock('falkordb', () => ({ + FalkorDB: { connect: sdk.connect }, +})) + +import { + closeKnowledgeGraphClient, + readKnowledgeGraphNeighbors, + readKnowledgeGraphOverview, + searchKnowledgeGraph, +} from '../src/client.js' +import type { PublishedKnowledgeGraph } from '../src/publication.js' +import { + exampleLectureEdgeRows, + exampleLectureNodeRows, + exampleLectureSources, +} from './fixtures/exampleLectureGraph.js' + +const KB_ID = '00000000-0000-4000-8000-000000000001' +const BUILD_ID = '00000000-0000-4000-8000-0000000000b1' + +const context: PublishedKnowledgeGraph = { + kbId: KB_ID, + buildId: BUILD_ID, + graphName: `klickeruzh:kb:${KB_ID}:${BUILD_ID}`, + isStale: false, + sources: exampleLectureSources, +} + +describe('knowledge graph client', () => { + beforeEach(async () => { + await closeKnowledgeGraphClient() + vi.clearAllMocks() + process.env.KB_FALKORDB_HOST = 'falkordb.test' + process.env.KB_FALKORDB_PORT = '6380' + process.env.KB_FALKORDB_USERNAME = 'reader' + process.env.KB_FALKORDB_PASSWORD = 'test-password' + process.env.KB_FALKORDB_TLS = 'true' + process.env.KB_FALKORDB_QUERY_TIMEOUT_MS = '4321' + + sdk.connect.mockResolvedValue({ + close: sdk.close, + on: sdk.on, + removeListener: sdk.removeListener, + selectGraph: sdk.selectGraph, + }) + sdk.selectGraph.mockReturnValue({ + query: sdk.query, + roQuery: sdk.roQuery, + }) + sdk.close.mockResolvedValue(undefined) + }) + + it('connects once with strict socket and credential configuration', async () => { + sdk.roQuery + .mockResolvedValueOnce({ data: exampleLectureNodeRows }) + .mockResolvedValueOnce({ data: exampleLectureEdgeRows }) + .mockResolvedValueOnce({ data: exampleLectureNodeRows }) + + await readKnowledgeGraphOverview(context) + await searchKnowledgeGraph(context, 'Android') + + expect(sdk.connect).toHaveBeenCalledTimes(1) + expect(sdk.connect).toHaveBeenCalledWith({ + username: 'reader', + password: 'test-password', + socket: { + host: 'falkordb.test', + port: 6380, + tls: true, + connectTimeout: 4321, + }, + }) + expect(sdk.on).toHaveBeenCalledWith('error', expect.any(Function)) + expect(sdk.selectGraph).toHaveBeenCalledWith(context.graphName) + }) + + it('uses roQuery with parameters and the configured timeout only', async () => { + sdk.roQuery + .mockResolvedValueOnce({ data: exampleLectureNodeRows }) + .mockResolvedValueOnce({ data: exampleLectureEdgeRows }) + + await readKnowledgeGraphOverview(context) + + expect(sdk.roQuery).toHaveBeenCalledTimes(2) + for (const [cypher, options] of sdk.roQuery.mock.calls) { + expect(cypher).toEqual(expect.any(String)) + expect(options).toMatchObject({ TIMEOUT: 4321 }) + } + expect(sdk.roQuery.mock.calls[1]?.[1]).toMatchObject({ + params: { nodeIds: ['12', '27', '31', '44', '58'] }, + }) + expect(sdk.query).not.toHaveBeenCalled() + }) + + it('closes and resets the reusable client', async () => { + sdk.roQuery.mockResolvedValue({ data: [] }) + + await readKnowledgeGraphOverview(context) + await closeKnowledgeGraphClient() + await readKnowledgeGraphOverview(context) + + expect(sdk.close).toHaveBeenCalledTimes(1) + expect(sdk.connect).toHaveBeenCalledTimes(2) + }) + + it('omits absent optional credentials', async () => { + delete process.env.KB_FALKORDB_USERNAME + delete process.env.KB_FALKORDB_PASSWORD + sdk.roQuery.mockResolvedValue({ data: [] }) + + await readKnowledgeGraphOverview(context) + + expect(sdk.connect).toHaveBeenCalledWith( + expect.objectContaining({ username: undefined, password: undefined }) + ) + }) + + it('normalizes and bounds overview results', async () => { + const extraNodes = Array.from({ length: 249 }, (_, index) => ({ + id: index + 100, + labels: ['Concept'], + properties: { name: `Concept ${index}` }, + degree: 1, + })) + const extraEdges = Array.from({ length: 500 }, (_, index) => ({ + id: index + 1000, + source: 12, + target: 27, + type: 'RELATED_TO', + properties: { position: index }, + })) + sdk.roQuery + .mockResolvedValueOnce({ + data: [...exampleLectureNodeRows, ...extraNodes], + }) + .mockResolvedValueOnce({ + data: [...exampleLectureEdgeRows, ...extraEdges], + }) + + const result = await readKnowledgeGraphOverview(context) + + expect(result).toMatchObject({ + kbId: context.kbId, + buildId: context.buildId, + isStale: false, + truncated: true, + }) + expect(result.nodes).toHaveLength(250) + expect(result.edges).toHaveLength(500) + }) + + it('parameterizes search and returns no arbitrary edge data', async () => { + const userText = 'Android Security' + sdk.roQuery.mockResolvedValueOnce({ data: exampleLectureNodeRows }) + + const result = await searchKnowledgeGraph(context, userText) + + expect(sdk.roQuery.mock.calls[0]?.[0]).not.toContain(userText) + expect(sdk.roQuery.mock.calls[0]?.[1]).toEqual({ + params: { searchText: userText }, + TIMEOUT: 4321, + }) + expect(result.edges).toEqual([]) + }) + + it('returns at most 20 search results and reports truncation', async () => { + const matches = Array.from({ length: 21 }, (_, index) => ({ + id: index, + labels: ['Concept'], + properties: { name: `Result ${index}` }, + degree: 1, + })) + sdk.roQuery.mockResolvedValueOnce({ data: matches }) + + const result = await searchKnowledgeGraph(context, 'Result') + + expect(result.nodes).toHaveLength(20) + expect(result.truncated).toBe(true) + }) + + it('returns at most 100 additional neighborhood nodes and 200 edges', async () => { + const center = exampleLectureNodeRows[0]! + const neighbors = Array.from({ length: 101 }, (_, index) => ({ + id: index + 100, + labels: ['Concept'], + properties: { name: `Neighbor ${index}` }, + degree: 1, + })) + const edges = Array.from({ length: 201 }, (_, index) => ({ + id: index + 1000, + source: 12, + target: index + 100, + type: 'RELATED_TO', + properties: {}, + })) + sdk.roQuery + .mockResolvedValueOnce({ data: [center, ...neighbors] }) + .mockResolvedValueOnce({ data: edges }) + + const result = await readKnowledgeGraphNeighbors(context, '12') + + expect(result.nodes).toHaveLength(100) + expect(result.edges).toHaveLength(200) + expect(result.truncated).toBe(true) + expect(sdk.roQuery.mock.calls[0]?.[0]).not.toContain(context.kbId) + expect(sdk.roQuery.mock.calls[0]?.[1]).toMatchObject({ + params: { nodeId: '12' }, + }) + }) +}) diff --git a/packages/knowledge-graph/test/config.test.ts b/packages/knowledge-graph/test/config.test.ts new file mode 100644 index 0000000000..c177691521 --- /dev/null +++ b/packages/knowledge-graph/test/config.test.ts @@ -0,0 +1,154 @@ +import { describe, expect, it } from 'vitest' +import { + DEFAULT_KNOWLEDGE_GRAPH_QUERY_TIMEOUT_MS, + KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT, + KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT, + KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT, + KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT, + getKnowledgeGraphConfig, +} from '../src/config.js' + +const validEnv = { + KB_FALKORDB_HOST: 'falkordb.ingestion.svc.cluster.local', + KB_FALKORDB_PORT: '6379', +} + +describe('getKnowledgeGraphConfig', () => { + it('parses the required connection settings and applies safe defaults', () => { + expect( + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_TLS: 'true', + }) + ).toEqual({ + host: 'falkordb.ingestion.svc.cluster.local', + port: 6379, + username: undefined, + password: undefined, + tls: true, + queryTimeoutMs: 5000, + }) + }) + + it('preserves optional credentials without requiring them', () => { + expect( + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_USERNAME: 'graph-reader', + KB_FALKORDB_PASSWORD: 'secret-value', + }) + ).toMatchObject({ + username: 'graph-reader', + password: 'secret-value', + tls: false, + }) + }) + + it.each([undefined, '', ' '])('rejects a missing host (%s)', (host) => { + expect(() => + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_HOST: host, + }) + ).toThrow('KB_FALKORDB_HOST') + }) + + it.each([ + undefined, + '', + '0', + '-1', + '1.5', + '65536', + 'not-a-port', + ])('rejects an invalid port (%s)', (port) => { + expect(() => + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_PORT: port, + }) + ).toThrow('KB_FALKORDB_PORT') + }) + + it.each([ + 'TRUE', + '1', + 'yes', + ' false ', + ])('rejects a non-strict TLS value (%s)', (tls) => { + expect(() => + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_TLS: tls, + }) + ).toThrow('KB_FALKORDB_TLS') + }) + + it('accepts an explicit false TLS value', () => { + expect( + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_TLS: 'false', + }).tls + ).toBe(false) + }) + + it('parses a positive safe integer query timeout', () => { + expect( + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_QUERY_TIMEOUT_MS: '12000', + }).queryTimeoutMs + ).toBe(12000) + }) + + it.each([ + '', + '0', + '-1', + '1.5', + 'not-a-timeout', + '9007199254740992', + ])('rejects an invalid or unsafe query timeout (%s)', (queryTimeoutMs) => { + expect(() => + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_QUERY_TIMEOUT_MS: queryTimeoutMs, + }) + ).toThrow('KB_FALKORDB_QUERY_TIMEOUT_MS') + }) + + it('does not expose credentials in validation errors', () => { + const username = 'sensitive-user' + const password = 'sensitive-password' + let thrown: unknown + + try { + getKnowledgeGraphConfig({ + ...validEnv, + KB_FALKORDB_PORT: 'invalid', + KB_FALKORDB_USERNAME: username, + KB_FALKORDB_PASSWORD: password, + }) + } catch (error) { + thrown = error + } + + expect(thrown).toBeInstanceOf(Error) + const message = (thrown as Error).message + expect(message).not.toContain(username) + expect(message).not.toContain(password) + }) +}) + +describe('knowledge graph limits', () => { + it('exports the approved bounded response defaults', () => { + expect(DEFAULT_KNOWLEDGE_GRAPH_QUERY_TIMEOUT_MS).toBe(5000) + expect(KNOWLEDGE_GRAPH_OVERVIEW_NODE_LIMIT).toBe(250) + expect(KNOWLEDGE_GRAPH_OVERVIEW_EDGE_LIMIT).toBe(500) + expect(KNOWLEDGE_GRAPH_SEARCH_NODE_LIMIT).toBe(20) + expect(KNOWLEDGE_GRAPH_NEIGHBOR_NODE_LIMIT).toBe(100) + expect(KNOWLEDGE_GRAPH_NEIGHBOR_EDGE_LIMIT).toBe(200) + }) +}) diff --git a/packages/knowledge-graph/test/digest.test.ts b/packages/knowledge-graph/test/digest.test.ts new file mode 100644 index 0000000000..4cccda86be --- /dev/null +++ b/packages/knowledge-graph/test/digest.test.ts @@ -0,0 +1,114 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' +import { describe, expect, it, vi } from 'vitest' + +import { + computeKBContentDigest, + hashKBContentDigestEntries, +} from '../src/digest.js' + +function mockPrisma( + resources: { + id: string + activeContentSha256: string | null + status?: 'PROCESSING' + }[] +) { + const findMany = vi.fn().mockResolvedValue(resources) + return { + prisma: { kBResource: { findMany } } as unknown as PrismaClient, + findMany, + } +} + +describe('KB content digest', () => { + it('covers every resource that has active serving content', async () => { + const { prisma, findMany } = mockPrisma([]) + + await computeKBContentDigest(prisma, 'kb-id') + + expect(findMany).toHaveBeenCalledWith( + expect.objectContaining({ + where: { + kbId: 'kb-id', + deletedAt: null, + activeContentSha256: { not: null }, + }, + orderBy: { id: 'asc' }, + }) + ) + }) + + it('keeps a serving revision in the digest while its replacement is processing', async () => { + const { prisma } = mockPrisma([ + { + id: 'resource-a', + activeContentSha256: 'sha-a', + status: 'PROCESSING', + }, + ]) + + await expect(computeKBContentDigest(prisma, 'kb-id')).resolves.toBe( + hashKBContentDigestEntries([ + { resourceId: 'resource-a', contentSha256: 'sha-a' }, + ]) + ) + }) + + it('changes when a resource is added, removed, or re-ingested', () => { + const base = hashKBContentDigestEntries([ + { resourceId: 'a', contentSha256: 'sha-a' }, + { resourceId: 'b', contentSha256: 'sha-b' }, + ]) + + expect( + hashKBContentDigestEntries([ + { resourceId: 'a', contentSha256: 'sha-a' }, + { resourceId: 'b', contentSha256: 'sha-b' }, + ]) + ).toBe(base) + + // a resource re-ingested with new content + expect( + hashKBContentDigestEntries([ + { resourceId: 'a', contentSha256: 'sha-a' }, + { resourceId: 'b', contentSha256: 'sha-b-v2' }, + ]) + ).not.toBe(base) + + // a resource removed from the serving set + expect( + hashKBContentDigestEntries([{ resourceId: 'a', contentSha256: 'sha-a' }]) + ).not.toBe(base) + + // a resource added to the serving set + expect( + hashKBContentDigestEntries([ + { resourceId: 'a', contentSha256: 'sha-a' }, + { resourceId: 'b', contentSha256: 'sha-b' }, + { resourceId: 'c', contentSha256: 'sha-c' }, + ]) + ).not.toBe(base) + }) + + it('does not let concatenation alias across entry boundaries', () => { + expect( + hashKBContentDigestEntries([{ resourceId: 'a', contentSha256: 'b:c' }]) + ).not.toBe( + hashKBContentDigestEntries([ + { resourceId: 'a', contentSha256: 'b' }, + { resourceId: 'c', contentSha256: '' }, + ]) + ) + }) + + it('skips resources ingestion has not published content for', async () => { + const { prisma } = mockPrisma([ + { id: 'a', activeContentSha256: 'sha-a' }, + { id: 'b', activeContentSha256: null }, + ]) + + await expect(computeKBContentDigest(prisma, 'kb-id')).resolves.toBe( + hashKBContentDigestEntries([{ resourceId: 'a', contentSha256: 'sha-a' }]) + ) + }) +}) diff --git a/packages/knowledge-graph/test/fixtures/exampleLectureGraph.ts b/packages/knowledge-graph/test/fixtures/exampleLectureGraph.ts new file mode 100644 index 0000000000..e11d4ddc19 --- /dev/null +++ b/packages/knowledge-graph/test/fixtures/exampleLectureGraph.ts @@ -0,0 +1,113 @@ +import type { KnowledgeGraphSourceMetadata } from '../../src/publication.js' +import type { + KnowledgeGraphEdgeRow, + KnowledgeGraphNodeRow, +} from '../../src/queries.js' + +// Sanitized representative data using the property shape observed in the +// approved example-lecture graph. All identifiers and content below are fake. +export const exampleLectureSources: KnowledgeGraphSourceMetadata[] = [ + { resourceId: 'resource-transcript', title: 'Lecture transcript' }, + { resourceId: 'resource-security', title: 'Android security' }, +] + +export const exampleLectureNodeRows: KnowledgeGraphNodeRow[] = [ + { + id: 12, + labels: ['Methode'], + properties: { + name: 'Example method', + entity_id: 'entity-example-method', + entity_type: 'Method', + description: 'A sanitized example method.', + source_id: 'resource-security', + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + degree: 7, + }, + degree: 7, + }, + { + id: 27, + labels: ['Kennzahl'], + properties: { + name: 'Example metric', + entity_id: 'entity-example-metric', + entity_type: 'Metric', + description: 'A sanitized example metric.', + source_id: ['resource-security', 'resource-transcript'], + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + degree: 5, + }, + degree: 5, + }, + { + id: 31, + labels: ['Formel'], + properties: { + name: 'Example formula', + entity_id: 'entity-example-formula', + entity_type: 'Formula', + description: 'A sanitized example formula.', + source_id: 'resource-transcript', + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + degree: 3, + }, + degree: 3, + }, + { + id: 44, + labels: ['Instrument'], + properties: { + name: 'Example instrument', + entity_id: 'entity-example-instrument', + entity_type: 'Instrument', + description: 'A sanitized example instrument.', + source_id: 'resource-security', + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + degree: 2, + }, + degree: 2, + }, + { + id: 58, + labels: ['Konzept'], + properties: { + name: 'Example concept', + entity_id: 'entity-example-concept', + entity_type: 'Concept', + description: 'A sanitized example concept.', + source_id: 'resource-transcript', + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + degree: 1, + }, + degree: 1, + }, +] + +export const exampleLectureEdgeRows: KnowledgeGraphEdgeRow[] = [ + { + id: 91, + source: 12, + target: 27, + type: 'RELATED', + properties: { + weight: 0.91, + description: 'Sanitized relationship description.', + keywords: 'example,metric', + source_id: 'resource-security', + file_path: 'https://example.test/document.pdf', + created_at: '2026-07-20T00:00:00Z', + truncate: false, + }, + }, +] diff --git a/packages/knowledge-graph/test/normalize.test.ts b/packages/knowledge-graph/test/normalize.test.ts new file mode 100644 index 0000000000..490bdbf8f7 --- /dev/null +++ b/packages/knowledge-graph/test/normalize.test.ts @@ -0,0 +1,264 @@ +import { describe, expect, it } from 'vitest' + +import { + KNOWLEDGE_GRAPH_CONTENT_MAX_LENGTH, + normalizeKnowledgeGraphEdge, + normalizeKnowledgeGraphNode, +} from '../src/normalize.js' +import { + exampleLectureEdgeRows, + exampleLectureNodeRows, + exampleLectureSources, +} from './fixtures/exampleLectureGraph.js' + +describe('knowledge graph normalization', () => { + it('normalizes the sanitized example graph using central precedence', () => { + const sources = new Map( + exampleLectureSources.map((source) => [source.resourceId, source]) + ) + + expect( + normalizeKnowledgeGraphNode(exampleLectureNodeRows[0]!, sources) + ).toEqual({ + id: '12', + labels: ['Methode'], + kind: 'Method', + displayLabel: 'Example method', + content: 'A sanitized example method.', + degree: 7, + sourceReferences: [ + { + resourceId: 'resource-security', + title: 'Android security', + }, + ], + }) + + expect(normalizeKnowledgeGraphEdge(exampleLectureEdgeRows[0]!)).toEqual({ + id: '91', + source: '12', + target: '27', + type: 'RELATED', + label: 'RELATED', + properties: { + weight: 0.91, + description: 'Sanitized relationship description.', + keywords: 'example,metric', + }, + }) + }) + + it('uses deterministic node fallbacks', () => { + expect( + normalizeKnowledgeGraphNode( + { id: 42, labels: [], properties: {}, degree: -1 }, + new Map() + ) + ).toEqual({ + id: '42', + labels: [], + kind: 'Concept', + displayLabel: 'Concept 42', + degree: 0, + sourceReferences: [], + }) + }) + + it('uses the approved display, kind, and content property precedence', () => { + const node = normalizeKnowledgeGraphNode( + { + id: 1, + labels: ['FallbackKind'], + properties: { + name: 'Name', + title: 'Title', + entity: 'Entity', + entity_type: 'ExplicitKind', + description: 'Description', + summary: 'Summary', + content: 'Content', + text: 'Text', + }, + degree: 0, + }, + new Map() + ) + + expect(node).toMatchObject({ + displayLabel: 'Name', + kind: 'ExplicitKind', + content: 'Description', + summary: 'Summary', + }) + }) + + it('resolves and deduplicates known source IDs only', () => { + const sources = new Map( + exampleLectureSources.map((source) => [source.resourceId, source]) + ) + const node = normalizeKnowledgeGraphNode( + { + id: 1, + labels: ['Concept'], + properties: { + source_id: [ + 'resource-transcript', + 'unknown-resource', + 'resource-transcript', + ], + page_number: 8, + }, + degree: 0, + }, + sources + ) + + expect(node?.sourceReferences).toEqual([ + { + resourceId: 'resource-transcript', + title: 'Lecture transcript', + reference: '8', + }, + ]) + }) + + it('caps content and removes sensitive, nested, binary, and non-finite edge data', () => { + const node = normalizeKnowledgeGraphNode( + { + id: 1, + labels: ['Concept'], + properties: { + description: 'x'.repeat(KNOWLEDGE_GRAPH_CONTENT_MAX_LENGTH + 100), + }, + degree: 1, + }, + new Map() + ) + expect(node?.content).toHaveLength(KNOWLEDGE_GRAPH_CONTENT_MAX_LENGTH) + + const edge = normalizeKnowledgeGraphEdge({ + id: 9, + source: 1, + target: 2, + type: 'RELATED_TO', + properties: { + confidence: 0.5, + enabled: true, + note: 'safe', + generic: 'contains secret metadata', + embedding: 'hidden', + vector_score: 0.9, + password_hint: 'hidden', + accessToken: 'hidden', + ingestion_run_id: 'hidden', + source_url: 'https://example.test/document.pdf', + signed: 'https://example.test/document.pdf?sv=1&sp=r&sig=secret', + nested: { hidden: true }, + bytes: Buffer.from('hidden'), + invalidNumber: Number.POSITIVE_INFINITY, + }, + }) + + expect(edge?.properties).toEqual({ + confidence: 0.5, + enabled: true, + note: 'safe', + }) + }) + + it('does not return SAS-like values from node display/content fields', () => { + const signedUrl = + 'https://example.test/document.pdf?sv=1&se=tomorrow&sp=r&sig=secret' + const node = normalizeKnowledgeGraphNode( + { + id: 4, + labels: ['Concept'], + properties: { name: signedUrl, description: signedUrl }, + degree: 0, + }, + new Map() + ) + + expect(node).toMatchObject({ + displayLabel: 'Concept 4', + }) + expect(node).not.toHaveProperty('content') + }) + + it('does not return secret-like top-level node or edge text', () => { + const node = normalizeKnowledgeGraphNode( + { + id: 5, + labels: ['secret token'], + properties: { + name: 'api_token=abc', + title: 'Safe fallback title', + entity_type: 'clientSecret=abc', + summary: 'secret metadata', + description: 'clientSecret=abc', + }, + degree: 0, + }, + new Map() + ) + const edge = normalizeKnowledgeGraphEdge({ + id: 9, + source: 5, + target: 6, + type: 'clientSecret=abc', + properties: { label: 'api_token=abc' }, + }) + + expect(node).toEqual({ + id: '5', + labels: [], + kind: 'Concept', + displayLabel: 'Safe fallback title', + degree: 0, + sourceReferences: [], + }) + expect(edge).toEqual({ + id: '9', + source: '5', + target: '6', + type: 'RELATED_TO', + label: 'RELATED_TO', + properties: {}, + }) + }) + + it('filters acronym-camel credentials without blocking safe lookalikes', () => { + const node = normalizeKnowledgeGraphNode( + { + id: 6, + labels: ['Konzept'], + properties: { + name: 'APISecret=abc', + title: 'Tokenization overview', + entity_type: 'JWTToken=abc', + description: 'A secretary coordinates this example.', + }, + degree: 1, + }, + new Map() + ) + const edge = normalizeKnowledgeGraphEdge({ + id: 10, + source: 6, + target: 7, + type: 'JWTToken=abc', + properties: { label: 'secretary' }, + }) + + expect(node).toMatchObject({ + displayLabel: 'Tokenization overview', + kind: 'Konzept', + content: 'A secretary coordinates this example.', + }) + expect(edge).toMatchObject({ + type: 'RELATED_TO', + label: 'secretary', + properties: { label: 'secretary' }, + }) + }) +}) diff --git a/packages/knowledge-graph/test/publication.test.ts b/packages/knowledge-graph/test/publication.test.ts new file mode 100644 index 0000000000..9daba778cf --- /dev/null +++ b/packages/knowledge-graph/test/publication.test.ts @@ -0,0 +1,243 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' +import { describe, expect, it, vi } from 'vitest' + +import { hashKBContentDigestEntries } from '../src/digest.js' +import { + KnowledgeGraphNotPublishedError, + getPublishedKnowledgeGraph, +} from '../src/publication.js' + +type MockKB = { + publishedGraphBuildId: string | null + resources: { id: string; title: string }[] +} | null + +type MockBuild = { + id: string + kbId: string + status: 'QUEUED' | 'PROCESSING' | 'SUCCEEDED' | 'FAILED' + graphName: string + sourceContentDigest: string + sources?: { resourceId: string; title: string }[] +} | null + +function mockPrisma({ + kb, + publishedBuild = null, + latestBuild = null, + servingResources = [], +}: { + kb: MockKB + publishedBuild?: MockBuild + latestBuild?: { status: string } | null + servingResources?: { id: string; activeContentSha256: string | null }[] +}): PrismaClient { + return { + kB: { findFirst: vi.fn().mockResolvedValue(kb) }, + kBGraphBuild: { + findFirst: vi.fn().mockResolvedValue(latestBuild), + findUnique: vi.fn().mockResolvedValue(publishedBuild), + }, + kBResource: { findMany: vi.fn().mockResolvedValue(servingResources) }, + } as unknown as PrismaClient +} + +const RESOURCES = [ + { id: 'resource-a', title: 'First' }, + { id: 'resource-b', title: 'Second' }, +] + +const SOURCES = RESOURCES.map(({ id, title }) => ({ resourceId: id, title })) + +const SERVING = [ + { id: 'resource-a', activeContentSha256: 'sha-a' }, + { id: 'resource-b', activeContentSha256: 'sha-b' }, +] + +const CURRENT_DIGEST = hashKBContentDigestEntries([ + { resourceId: 'resource-a', contentSha256: 'sha-a' }, + { resourceId: 'resource-b', contentSha256: 'sha-b' }, +]) + +describe('knowledge graph publication guard', () => { + it('serves the published build under the name it was written to', async () => { + const prisma = mockPrisma({ + kb: { publishedGraphBuildId: 'build-1', resources: RESOURCES }, + publishedBuild: { + id: 'build-1', + kbId: 'kb-id', + status: 'SUCCEEDED', + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: CURRENT_DIGEST, + sources: SOURCES, + }, + servingResources: SERVING, + }) + + await expect(getPublishedKnowledgeGraph(prisma, 'kb-id')).resolves.toEqual({ + kbId: 'kb-id', + buildId: 'build-1', + graphName: 'klickeruzh:kb:kb-id:build-1', + isStale: false, + sources: [ + { resourceId: 'resource-a', title: 'First' }, + { resourceId: 'resource-b', title: 'Second' }, + ], + }) + }) + + // The rule this inverts: the chatbot-owned predecessor treated a stale graph as + // unpublished and served nothing. + it('keeps serving a stale build, labelled rather than withheld', async () => { + const prisma = mockPrisma({ + kb: { publishedGraphBuildId: 'build-1', resources: RESOURCES }, + publishedBuild: { + id: 'build-1', + kbId: 'kb-id', + status: 'SUCCEEDED', + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: 'digest-from-an-older-content-set', + sources: SOURCES, + }, + servingResources: SERVING, + }) + + await expect(getPublishedKnowledgeGraph(prisma, 'kb-id')).resolves.toEqual( + expect.objectContaining({ buildId: 'build-1', isStale: true }) + ) + }) + + it('keeps serving while a newer build is still running', async () => { + const prisma = mockPrisma({ + kb: { publishedGraphBuildId: 'build-1', resources: RESOURCES }, + publishedBuild: { + id: 'build-1', + kbId: 'kb-id', + status: 'SUCCEEDED', + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: CURRENT_DIGEST, + sources: SOURCES, + }, + latestBuild: { status: 'PROCESSING' }, + servingResources: SERVING, + }) + + await expect(getPublishedKnowledgeGraph(prisma, 'kb-id')).resolves.toEqual( + expect.objectContaining({ buildId: 'build-1', isStale: false }) + ) + }) + + it('uses the build-local source snapshot after a resource changes', async () => { + const snapshot = [{ resourceId: 'resource-a', title: 'Original title' }] + const prisma = mockPrisma({ + kb: { publishedGraphBuildId: 'build-1', resources: RESOURCES }, + publishedBuild: { + id: 'build-1', + kbId: 'kb-id', + status: 'SUCCEEDED', + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: 'digest-from-an-older-content-set', + sources: snapshot, + }, + servingResources: SERVING, + }) + + await expect( + getPublishedKnowledgeGraph(prisma, 'kb-id') + ).resolves.toMatchObject({ + sources: snapshot, + isStale: true, + }) + }) + + it.each([ + [ + 'a queued build', + { + id: 'build-1', + kbId: 'kb-id', + status: 'QUEUED' as const, + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: CURRENT_DIGEST, + }, + ], + [ + 'a failed build', + { + id: 'build-1', + kbId: 'kb-id', + status: 'FAILED' as const, + graphName: 'klickeruzh:kb:kb-id:build-1', + sourceContentDigest: CURRENT_DIGEST, + }, + ], + [ + 'a build owned by another KB', + { + id: 'build-1', + kbId: 'other-kb', + status: 'SUCCEEDED' as const, + graphName: 'klickeruzh:kb:other-kb:build-1', + sourceContentDigest: CURRENT_DIGEST, + }, + ], + ])('rejects a published pointer to %s', async (_, publishedBuild) => { + const promise = getPublishedKnowledgeGraph( + mockPrisma({ + kb: { publishedGraphBuildId: 'build-1', resources: RESOURCES }, + publishedBuild, + }), + 'kb-id' + ) + + await expect(promise).rejects.toMatchObject({ code: 'EMPTY' }) + }) + + it.each([ + ['deleted or missing KB', { kb: null }, 'EMPTY'], + [ + 'KB that has never been built', + { kb: { publishedGraphBuildId: null, resources: RESOURCES } }, + 'EMPTY', + ], + [ + 'first build still queued', + { + kb: { publishedGraphBuildId: null, resources: RESOURCES }, + latestBuild: { status: 'QUEUED' }, + }, + 'QUEUED', + ], + [ + 'first build still processing', + { + kb: { publishedGraphBuildId: null, resources: RESOURCES }, + latestBuild: { status: 'PROCESSING' }, + }, + 'PROCESSING', + ], + [ + 'first build failed', + { + kb: { publishedGraphBuildId: null, resources: RESOURCES }, + latestBuild: { status: 'FAILED' }, + }, + 'FAILED', + ], + [ + 'published pointer with no build behind it', + { + kb: { publishedGraphBuildId: 'build-gone', resources: RESOURCES }, + publishedBuild: null, + }, + 'EMPTY', + ], + ])('rejects a %s', async (_, options, code) => { + const promise = getPublishedKnowledgeGraph(mockPrisma(options), 'kb-id') + + await expect(promise).rejects.toBeInstanceOf( + KnowledgeGraphNotPublishedError + ) + await expect(promise).rejects.toMatchObject({ code }) + }) +}) diff --git a/packages/knowledge-graph/test/queries.test.ts b/packages/knowledge-graph/test/queries.test.ts new file mode 100644 index 0000000000..5a472e7b58 --- /dev/null +++ b/packages/knowledge-graph/test/queries.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it } from 'vitest' + +import { + getEdgesForNodeIdsQuery, + getNeighborhoodNodesQuery, + getOverviewNodesQuery, + getSearchNodesQuery, +} from '../src/queries.js' + +describe('fixed knowledge graph queries', () => { + it('selects one extra overview node and edge to detect truncation', () => { + const nodes = getOverviewNodesQuery() + const edges = getEdgesForNodeIdsQuery(['1', '2'], 'overview') + + expect(nodes.cypher).toContain('LIMIT 251') + expect(edges.cypher).toContain('LIMIT 501') + expect(edges.params).toEqual({ nodeIds: ['1', '2'] }) + }) + + it('keeps search text parameterized and bounded', () => { + const userText = 'Android Security' + const query = getSearchNodesQuery(` ${userText} `) + + expect(query.cypher).toContain('LIMIT 21') + expect(query.cypher).not.toContain(userText) + expect(query.params).toEqual({ searchText: userText }) + }) + + it.each([ + '', + ' ', + 'x'.repeat(101), + ])('rejects invalid search text %j', (searchText) => { + expect(() => getSearchNodesQuery(searchText)).toThrow( + 'Search text must contain between 1 and 100 characters' + ) + }) + + it('keeps decimal neighborhood IDs parameterized and bounded', () => { + const query = getNeighborhoodNodesQuery('12345678901234567890') + + expect(query.cypher).toContain('LIMIT 101') + expect(query.cypher).not.toContain('12345678901234567890') + expect(query.params).toEqual({ nodeId: '12345678901234567890' }) + + const edges = getEdgesForNodeIdsQuery(['1', '2'], 'neighbors') + expect(edges.cypher).toContain('LIMIT 201') + }) + + it.each([ + '-1', + '1.2', + '1 OR 1=1', + ' 1', + '', + ])('rejects non-decimal node ID %j', (nodeId) => { + expect(() => getNeighborhoodNodesQuery(nodeId)).toThrow( + 'Node ID must be a decimal integer' + ) + }) + + it('rejects invalid internally selected IDs before constructing edge reads', () => { + expect(() => + getEdgesForNodeIdsQuery(['1', '2) MATCH (n) RETURN n'], 'overview') + ).toThrow('Node ID must be a decimal integer') + }) +}) diff --git a/packages/knowledge-graph/tsconfig.json b/packages/knowledge-graph/tsconfig.json new file mode 100644 index 0000000000..7ad22ba6a1 --- /dev/null +++ b/packages/knowledge-graph/tsconfig.json @@ -0,0 +1,25 @@ +{ + "include": ["**/**/*"], + "compilerOptions": { + "esModuleInterop": true, + "skipLibCheck": true, + "target": "es2022", + "allowJs": true, + "resolveJsonModule": true, + "moduleDetection": "force", + "isolatedModules": true, + "verbatimModuleSyntax": true, + "strict": true, + "noUncheckedIndexedAccess": true, + "noImplicitOverride": true, + "noErrorTruncation": true, + "module": "NodeNext", + "outDir": "dist", + "sourceMap": true, + "declaration": true, + "composite": true, + "tsBuildInfoFile": "dist/tsconfig.tsbuildinfo", + "declarationMap": true, + "lib": ["es2022"] + } +} diff --git a/packages/prisma-data/package.json b/packages/prisma-data/package.json index 8a500706af..39874495ea 100644 --- a/packages/prisma-data/package.json +++ b/packages/prisma-data/package.json @@ -58,7 +58,10 @@ "seed:raw": "ENV=development run-s --npm-path pnpm seed:test seed:assessment-course", "seed:raw:course-awards": "tsx src/data/seedCourseAwards.ts", "seed:test": "ENV=development tsx src/data/seedTEST.ts", - "test": "vitest run" + "test": "run-s --npm-path pnpm test:node test:vitest", + "test:node": "tsx --test test/*.test.ts", + "test:run": "pnpm run test", + "test:vitest": "vitest run src/scripts/2026-08-23_provision_course_chatbot.test.ts" }, "engines": { "node": "=24" diff --git a/packages/prisma-data/src/data/seedMCPServers.ts b/packages/prisma-data/src/data/seedMCPServers.ts index bcdb400f3e..dc63123c8d 100644 --- a/packages/prisma-data/src/data/seedMCPServers.ts +++ b/packages/prisma-data/src/data/seedMCPServers.ts @@ -11,7 +11,7 @@ interface MCPServerSeed { name: MCP_SERVER_NAMES description: string url: string - authType: 'bearer' | 'basic' | 'none' | 'custom' + authType: 'bearer' | 'basic' | 'none' | 'custom' | 'scope_token' authSecret?: string parameters?: any isActive?: boolean @@ -34,9 +34,9 @@ const MCP_SERVERS: MCPServerSeed[] = [ name: MCP_SERVER_NAMES.KB, description: 'A comprehensive knowledge base for various topics', url: 'http://localhost:1417/mcp', - authType: 'none', + authType: 'scope_token', isActive: true, - passChatbotId: true, + passChatbotId: false, }, ] @@ -58,7 +58,7 @@ const EXAMPLE_CONFIGURATIONS: ChatbotMCPConfigSeed[] = [ chatMode: 'tutor', allowedTools: ['doc_query'], priority: 0, - isEnabled: true, + isEnabled: false, }, { chatbotId: CHATBOT_ID_TEST, @@ -66,7 +66,7 @@ const EXAMPLE_CONFIGURATIONS: ChatbotMCPConfigSeed[] = [ chatMode: 'explainer', allowedTools: ['doc_query'], priority: 0, - isEnabled: true, + isEnabled: false, }, { chatbotId: CHATBOT_ID_TEST, @@ -151,7 +151,7 @@ function validateServerConfig(serverConfig: MCPServerSeed): boolean { } // Validate auth type - const validAuthTypes = ['bearer', 'basic', 'none', 'custom'] + const validAuthTypes = ['bearer', 'basic', 'none', 'custom', 'scope_token'] if (!validAuthTypes.includes(serverConfig.authType)) { console.error( `Invalid auth type for ${serverConfig.name}: ${serverConfig.authType}` @@ -195,6 +195,27 @@ export async function seedMCPServers(prisma: PrismaClient) { }) if (existingServer) { + if (serverConfig.name === MCP_SERVER_NAMES.KB) { + const reconciledServer = await prisma.chatbotMCPServer.update({ + where: { id: existingServer.id }, + data: { + description: serverConfig.description, + url: serverConfig.url, + authType: serverConfig.authType, + authSecret: null, + parameters: serverConfig.parameters || {}, + isActive: serverConfig.isActive ?? true, + passChatbotId: false, + chatbotIdHeader: null, + }, + }) + console.log( + `Reconciled MCP server '${serverConfig.name}' with scoped authentication` + ) + createdServers.push(reconciledServer) + continue + } + console.log( `MCP server '${serverConfig.name}' already exists, skipping` ) @@ -269,6 +290,14 @@ export async function seedChatbotMCPConfigurations( continue } + const enabledBinding = + config.mcpServerName === MCP_SERVER_NAMES.KB + ? await prisma.kBChatbot.findFirst({ + where: { chatbotId: config.chatbotId, isEnabled: true }, + select: { id: true }, + }) + : null + const existingConfig = await prisma.chatbotMCPConfig.findUnique({ where: { chatbotId_mcpServerId_chatMode: { @@ -280,6 +309,21 @@ export async function seedChatbotMCPConfigurations( }) if (existingConfig) { + if (config.mcpServerName === MCP_SERVER_NAMES.KB) { + await prisma.chatbotMCPConfig.update({ + where: { id: existingConfig.id }, + data: { + allowedTools: ['doc_query'], + priority: 0, + isEnabled: Boolean(enabledBinding), + }, + }) + console.log( + `Reconciled ${config.mcpServerName}/${config.chatMode} from its KB binding` + ) + continue + } + console.log( `Configuration for ${config.mcpServerName}/${config.chatMode} already exists, skipping` ) @@ -293,7 +337,10 @@ export async function seedChatbotMCPConfigurations( chatMode: config.chatMode, allowedTools: config.allowedTools, priority: config.priority, - isEnabled: config.isEnabled, + isEnabled: + config.mcpServerName === MCP_SERVER_NAMES.KB + ? Boolean(enabledBinding) + : config.isEnabled, parameters: config.parameters || {}, }, }) diff --git a/packages/prisma-data/test/seedMCPServers.test.ts b/packages/prisma-data/test/seedMCPServers.test.ts new file mode 100644 index 0000000000..62136eb0d7 --- /dev/null +++ b/packages/prisma-data/test/seedMCPServers.test.ts @@ -0,0 +1,77 @@ +import type { PrismaClient } from '@klicker-uzh/prisma/client' +import assert from 'node:assert/strict' +import { describe, test } from 'node:test' +import { seedChatbotMCPConfigurations } from '../src/data/seedMCPServers.js' + +const KB_SERVER = { + id: 'kb-server', + name: 'KB', +} + +function createPrismaMock({ + hasBinding, + hasExistingConfig, +}: { + hasBinding: boolean + hasExistingConfig: boolean +}) { + const updates: Array> = [] + const creates: Array> = [] + + const prisma = { + kBChatbot: { + findFirst: async () => (hasBinding ? { id: 'binding' } : null), + }, + chatbotMCPConfig: { + findUnique: async ({ + where, + }: { + where: { + chatbotId_mcpServerId_chatMode: { chatMode: string } + } + }) => + hasExistingConfig + ? { + id: `config-${where.chatbotId_mcpServerId_chatMode.chatMode}`, + } + : null, + update: async ({ data }: { data: Record }) => { + updates.push(data) + return data + }, + create: async ({ data }: { data: Record }) => { + creates.push(data) + return data + }, + }, + } as unknown as PrismaClient + + return { prisma, updates, creates } +} + +describe('KB chatbot MCP seed reconciliation', () => { + for (const hasBinding of [true, false]) { + for (const hasExistingConfig of [true, false]) { + test(`${hasExistingConfig ? 'updates' : 'creates'} ${hasBinding ? 'enabled' : 'disabled'} tutor and explainer configs`, async () => { + const { prisma, updates, creates } = createPrismaMock({ + hasBinding, + hasExistingConfig, + }) + + await seedChatbotMCPConfigurations(prisma, [KB_SERVER] as Awaited< + ReturnType< + typeof import('../src/data/seedMCPServers.js').seedMCPServers + > + >) + + const writes = hasExistingConfig ? updates : creates + assert.equal(writes.length, 2) + for (const data of writes) { + assert.deepEqual(data.allowedTools, ['doc_query']) + assert.equal(data.priority, 0) + assert.equal(data.isEnabled, hasBinding) + } + }) + } + } +}) diff --git a/packages/prisma/src/prisma/schema/chat.prisma b/packages/prisma/src/prisma/schema/chat.prisma index 509aeafb17..1c4ffa76db 100644 --- a/packages/prisma/src/prisma/schema/chat.prisma +++ b/packages/prisma/src/prisma/schema/chat.prisma @@ -87,7 +87,7 @@ model ChatAttachment { type ChatAttachmentType position Int - imageBase64 String? @db.Text // base64 data URL for display + imageBase64 String? @db.Text // base64 data URL for display imagePreviewBase64 String? @db.Text // compact preview data URL for history rendering imageDescription String? @db.Text // AI-generated description for context injection @@ -132,6 +132,7 @@ model Chatbot { // Relations mcpConfigurations ChatbotMCPConfig[] + knowledgeBases KBChatbot[] threads ChatThread[] usageCredits ChatUsageCredits[] diff --git a/packages/prisma/src/prisma/schema/knowledge.prisma b/packages/prisma/src/prisma/schema/knowledge.prisma new file mode 100644 index 0000000000..f95f98dd20 --- /dev/null +++ b/packages/prisma/src/prisma/schema/knowledge.prisma @@ -0,0 +1,296 @@ +// ----- KNOWLEDGE BASES ----- +// #region + +enum KBResourceType { + BLOB // uploaded file stored in Azure Blob Storage + URL // external web resource (e.g., website, Kaltura MediaSpace video) fetched during ingestion +} + +enum KBResourceStatus { + ADDED // resource registered (blob uploaded or URL added), not yet queued for ingestion + QUEUED + PROCESSING + READY + FAILED +} + +enum KBIngestionStatus { + QUEUED + PROCESSING + SUCCEEDED + FAILED + SUPERSEDED +} + +enum KBIngestionOperation { + UPSERT + DELETE +} + +enum KBGraphBuildStatus { + QUEUED + PROCESSING + SUCCEEDED + FAILED // includes builds failed by the reconciliation timeout + SUPERSEDED +} + +enum KBGraphQualityTier { + STANDARD + HIGH +} + +enum KBGraphCostStatus { + RESERVED + SETTLED + RELEASED + NEEDS_HUMAN_REVIEW +} + +model KB { + id String @id @default(uuid()) @db.Uuid + name String + description String? + + owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade, onUpdate: Cascade) + ownerId String @db.Uuid + + deletedAt DateTime? + deletedBy User? @relation("KBDeletedBy", fields: [deletedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + deletedById String? @db.Uuid + + // at most one build may hold the slot at a time; claimed with a conditional update + activeGraphBuildId String? @db.Uuid + // the build FalkorDB currently serves, kept in place even once its digest goes stale + publishedGraphBuildId String? @db.Uuid + // graph generation is opt-in per KB; the lecturer-level preview flag is a separate gate + knowledgeGraphEnabled Boolean @default(false) + + resources KBResource[] + uploadTickets KBUploadTicket[] + chatbots KBChatbot[] + graphBuilds KBGraphBuild[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([ownerId]) + @@index([deletedAt]) +} + +model KBResource { + id String @id @default(uuid()) @db.Uuid + + type KBResourceType + title String + + // URL resources: location of the external web resource + sourceUrl String? + + // BLOB resources: upload metadata and blob location (resolver-enforced as required for type BLOB) + originalFilename String? + mimeType String? + sizeBytes Int? + blobName String? + blobHref String? + + status KBResourceStatus @default(ADDED) + statusMessage String? + ingestedAt DateTime? + ingestionAttemptId String? @db.Uuid + resourceVersion Int @default(0) + contentSha256 String? + externalOperationId String? + externalOperationStartedAt DateTime? + activeResourceVersion Int? + activeContentSha256 String? + errorCode String? + ingestionOperation KBIngestionOperation @default(UPSERT) + + deletedAt DateTime? + deletedBy User? @relation("KBResourceDeletedBy", fields: [deletedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + deletedById String? @db.Uuid + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + ingestionRuns KBIngestionRun[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([kbId, status]) + @@index([status]) + @@index([deletedAt]) +} + +model KBIngestionRun { + id String @id @db.Uuid + + operation KBIngestionOperation @default(UPSERT) + status KBIngestionStatus @default(QUEUED) + resourceVersion Int + contentSha256 String? + externalOperationId String? + statusMessage String? + errorCode String? + startedAt DateTime? + finishedAt DateTime? + + resource KBResource @relation(fields: [resourceId], references: [id], onDelete: Cascade, onUpdate: Cascade) + resourceId String @db.Uuid + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([resourceId, createdAt]) + @@index([status]) +} + +// Append-only ledger of knowledge graph build attempts, mirroring KBIngestionRun. +// The id is supplied by KlickerUZH and doubles as the idempotency key handed to the +// external generation service, which answers with its own externalOperationId. +model KBGraphBuild { + id String @id @db.Uuid + + status KBGraphBuildStatus @default(QUEUED) + qualityTier KBGraphQualityTier @default(STANDARD) + + // digest over the KB's active serving set at request time; pins what this build represents + sourceContentDigest String + // FalkorDB graph the completed build is written to, in one step at the end + graphName String + // GraphML export retained on Blob for versioning, once the build succeeds + graphmlBlobName String? + + // Cost reservation is part of the build ledger so settlement is idempotent by build id. + estimatedCostMinorUnits Int? + actualCostMinorUnits Int? + actualInputTokens Int? + actualOutputTokens Int? + actualEmbeddingTokens Int? + actualRequestCount Int? + costCurrency String? + costPricingVersion String? + costStatus KBGraphCostStatus? + /// [PrismaKBGraphMeteredCost] + meteredCost Json? + semesterKey String? + quota KBGraphQuota? @relation(fields: [quotaId], references: [id], onDelete: SetNull, onUpdate: Cascade) + quotaId String? @db.Uuid + + externalOperationId String? + externalStartedAt DateTime? + // Durable claim written before the provider call. If the call is accepted + // but its run id cannot be correlated, this prevents releasing the reserve. + dispatchClaimedAt DateTime? + statusMessage String? + errorCode String? + startedAt DateTime? + finishedAt DateTime? + // terminal builds keep their ledger row; maintenance records once their + // external FalkorDB graph has been retired. + // Set before external deletion so late external completions cannot publish + // while retention cleanup is in flight. Stale claims are reclaimable after + // the retention grace window. + cleanupStartedAt DateTime? + cleanedAt DateTime? + // The GraphML archive outlives the serving graph: it is purged on a separate, + // longer clock (the KB deletion recovery grace) so an earlier successful + // version stays restorable while the knowledge base exists. Tracked apart from + // `cleanedAt` because whole-KB hard deletion keys off graph retirement. + graphmlPurgedAt DateTime? + + // builds spend the requesting lecturer's AI budget, so the requester is recorded + requestedBy User? @relation(fields: [requestedById], references: [id], onDelete: SetNull, onUpdate: Cascade) + requestedById String? @db.Uuid + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + // The graph must remain explainable even if a source resource is later + // deleted or replaced, so this is a build-local snapshot rather than a FK. + sources KBGraphBuildSource[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([kbId, createdAt]) + @@index([status]) + @@index([quotaId]) +} + +model KBGraphQuota { + id String @id @default(uuid()) @db.Uuid + + owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade, onUpdate: Cascade) + ownerId String @db.Uuid + + semesterKey String + currency String + limitMinorUnits Int + reservedMinorUnits Int @default(0) + settledMinorUnits Int @default(0) + graphBuilds KBGraphBuild[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([ownerId, semesterKey]) + @@index([ownerId]) +} + +model KBGraphBuildSource { + id String @id @default(uuid()) @db.Uuid + + resourceId String @db.Uuid + title String + type KBResourceType + contentSha256 String + sourceUrl String? + blobName String? + + build KBGraphBuild @relation(fields: [buildId], references: [id], onDelete: Cascade, onUpdate: Cascade) + buildId String @db.Uuid + + createdAt DateTime @default(now()) + + @@unique([buildId, resourceId]) + @@index([resourceId]) +} + +model KBUploadTicket { + id String @id @db.Uuid + + blobName String + sizeBytes Int @default(0) + expiresAt DateTime + + kb KB @relation(fields: [kbId], references: [id], onDelete: Restrict, onUpdate: Cascade) + kbId String @db.Uuid + + createdAt DateTime @default(now()) + + @@unique([kbId, blobName]) + @@index([expiresAt]) +} + +model KBChatbot { + id String @id @default(uuid()) @db.Uuid + + isEnabled Boolean @default(true) + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + chatbot Chatbot @relation(fields: [chatbotId], references: [id], onDelete: Cascade, onUpdate: Cascade) + chatbotId String @db.Uuid + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([kbId, chatbotId]) + @@index([chatbotId]) +} + +// #endregion diff --git a/packages/prisma/src/prisma/schema/migrations/20260825190000_kb_management_foundation/migration.sql b/packages/prisma/src/prisma/schema/migrations/20260825190000_kb_management_foundation/migration.sql new file mode 100644 index 0000000000..d31f9ca721 --- /dev/null +++ b/packages/prisma/src/prisma/schema/migrations/20260825190000_kb_management_foundation/migration.sql @@ -0,0 +1,278 @@ +-- CreateEnum +CREATE TYPE "KBResourceType" AS ENUM ('BLOB', 'URL'); + +-- CreateEnum +CREATE TYPE "KBResourceStatus" AS ENUM ('ADDED', 'QUEUED', 'PROCESSING', 'READY', 'FAILED'); + +-- CreateEnum +CREATE TYPE "KBIngestionStatus" AS ENUM ('QUEUED', 'PROCESSING', 'SUCCEEDED', 'FAILED', 'SUPERSEDED'); + +-- CreateEnum +CREATE TYPE "KBIngestionOperation" AS ENUM ('UPSERT', 'DELETE'); + +-- CreateEnum +CREATE TYPE "KBGraphBuildStatus" AS ENUM ('QUEUED', 'PROCESSING', 'SUCCEEDED', 'FAILED', 'SUPERSEDED'); + +-- CreateEnum +CREATE TYPE "KBGraphQualityTier" AS ENUM ('STANDARD', 'HIGH'); + +-- CreateEnum +CREATE TYPE "KBGraphCostStatus" AS ENUM ('RESERVED', 'SETTLED', 'RELEASED', 'NEEDS_HUMAN_REVIEW'); + +-- CreateTable +CREATE TABLE "KB" ( + "id" UUID NOT NULL, + "name" TEXT NOT NULL, + "description" TEXT, + "ownerId" UUID NOT NULL, + "deletedAt" TIMESTAMP(3), + "deletedById" UUID, + "activeGraphBuildId" UUID, + "publishedGraphBuildId" UUID, + "knowledgeGraphEnabled" BOOLEAN NOT NULL DEFAULT false, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KB_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBResource" ( + "id" UUID NOT NULL, + "type" "KBResourceType" NOT NULL, + "title" TEXT NOT NULL, + "sourceUrl" TEXT, + "originalFilename" TEXT, + "mimeType" TEXT, + "sizeBytes" INTEGER, + "blobName" TEXT, + "blobHref" TEXT, + "status" "KBResourceStatus" NOT NULL DEFAULT 'ADDED', + "statusMessage" TEXT, + "ingestedAt" TIMESTAMP(3), + "ingestionAttemptId" UUID, + "resourceVersion" INTEGER NOT NULL DEFAULT 0, + "contentSha256" TEXT, + "externalOperationId" TEXT, + "externalOperationStartedAt" TIMESTAMP(3), + "activeResourceVersion" INTEGER, + "activeContentSha256" TEXT, + "errorCode" TEXT, + "ingestionOperation" "KBIngestionOperation" NOT NULL DEFAULT 'UPSERT', + "deletedAt" TIMESTAMP(3), + "deletedById" UUID, + "kbId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KBResource_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBIngestionRun" ( + "id" UUID NOT NULL, + "operation" "KBIngestionOperation" NOT NULL DEFAULT 'UPSERT', + "status" "KBIngestionStatus" NOT NULL DEFAULT 'QUEUED', + "resourceVersion" INTEGER NOT NULL, + "contentSha256" TEXT, + "externalOperationId" TEXT, + "statusMessage" TEXT, + "errorCode" TEXT, + "startedAt" TIMESTAMP(3), + "finishedAt" TIMESTAMP(3), + "resourceId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KBIngestionRun_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBGraphBuild" ( + "id" UUID NOT NULL, + "status" "KBGraphBuildStatus" NOT NULL DEFAULT 'QUEUED', + "qualityTier" "KBGraphQualityTier" NOT NULL DEFAULT 'STANDARD', + "sourceContentDigest" TEXT NOT NULL, + "graphName" TEXT NOT NULL, + "graphmlBlobName" TEXT, + "estimatedCostMinorUnits" INTEGER, + "actualCostMinorUnits" INTEGER, + "actualInputTokens" INTEGER, + "actualOutputTokens" INTEGER, + "actualEmbeddingTokens" INTEGER, + "actualRequestCount" INTEGER, + "costCurrency" TEXT, + "costPricingVersion" TEXT, + "costStatus" "KBGraphCostStatus", + "meteredCost" JSONB, + "semesterKey" TEXT, + "quotaId" UUID, + "externalOperationId" TEXT, + "externalStartedAt" TIMESTAMP(3), + "dispatchClaimedAt" TIMESTAMP(3), + "statusMessage" TEXT, + "errorCode" TEXT, + "startedAt" TIMESTAMP(3), + "finishedAt" TIMESTAMP(3), + "cleanupStartedAt" TIMESTAMP(3), + "cleanedAt" TIMESTAMP(3), + "graphmlPurgedAt" TIMESTAMP(3), + "requestedById" UUID, + "kbId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KBGraphBuild_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBGraphQuota" ( + "id" UUID NOT NULL, + "ownerId" UUID NOT NULL, + "semesterKey" TEXT NOT NULL, + "currency" TEXT NOT NULL, + "limitMinorUnits" INTEGER NOT NULL, + "reservedMinorUnits" INTEGER NOT NULL DEFAULT 0, + "settledMinorUnits" INTEGER NOT NULL DEFAULT 0, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KBGraphQuota_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBGraphBuildSource" ( + "id" UUID NOT NULL, + "resourceId" UUID NOT NULL, + "title" TEXT NOT NULL, + "type" "KBResourceType" NOT NULL, + "contentSha256" TEXT NOT NULL, + "sourceUrl" TEXT, + "blobName" TEXT, + "buildId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "KBGraphBuildSource_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBUploadTicket" ( + "id" UUID NOT NULL, + "blobName" TEXT NOT NULL, + "sizeBytes" INTEGER NOT NULL DEFAULT 0, + "expiresAt" TIMESTAMP(3) NOT NULL, + "kbId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "KBUploadTicket_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "KBChatbot" ( + "id" UUID NOT NULL, + "isEnabled" BOOLEAN NOT NULL DEFAULT true, + "kbId" UUID NOT NULL, + "chatbotId" UUID NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "KBChatbot_pkey" PRIMARY KEY ("id") +); + +-- CreateIndex +CREATE INDEX "KB_ownerId_idx" ON "KB"("ownerId"); + +-- CreateIndex +CREATE INDEX "KB_deletedAt_idx" ON "KB"("deletedAt"); + +-- CreateIndex +CREATE INDEX "KBResource_kbId_status_idx" ON "KBResource"("kbId", "status"); + +-- CreateIndex +CREATE INDEX "KBResource_status_idx" ON "KBResource"("status"); + +-- CreateIndex +CREATE INDEX "KBResource_deletedAt_idx" ON "KBResource"("deletedAt"); + +-- CreateIndex +CREATE INDEX "KBIngestionRun_resourceId_createdAt_idx" ON "KBIngestionRun"("resourceId", "createdAt"); + +-- CreateIndex +CREATE INDEX "KBIngestionRun_status_idx" ON "KBIngestionRun"("status"); + +-- CreateIndex +CREATE INDEX "KBGraphBuild_kbId_createdAt_idx" ON "KBGraphBuild"("kbId", "createdAt"); + +-- CreateIndex +CREATE INDEX "KBGraphBuild_status_idx" ON "KBGraphBuild"("status"); + +-- CreateIndex +CREATE INDEX "KBGraphBuild_quotaId_idx" ON "KBGraphBuild"("quotaId"); + +-- CreateIndex +CREATE INDEX "KBGraphQuota_ownerId_idx" ON "KBGraphQuota"("ownerId"); + +-- CreateIndex +CREATE UNIQUE INDEX "KBGraphQuota_ownerId_semesterKey_key" ON "KBGraphQuota"("ownerId", "semesterKey"); + +-- CreateIndex +CREATE INDEX "KBGraphBuildSource_resourceId_idx" ON "KBGraphBuildSource"("resourceId"); + +-- CreateIndex +CREATE UNIQUE INDEX "KBGraphBuildSource_buildId_resourceId_key" ON "KBGraphBuildSource"("buildId", "resourceId"); + +-- CreateIndex +CREATE INDEX "KBUploadTicket_expiresAt_idx" ON "KBUploadTicket"("expiresAt"); + +-- CreateIndex +CREATE UNIQUE INDEX "KBUploadTicket_kbId_blobName_key" ON "KBUploadTicket"("kbId", "blobName"); + +-- CreateIndex +CREATE INDEX "KBChatbot_chatbotId_idx" ON "KBChatbot"("chatbotId"); + +-- CreateIndex +CREATE UNIQUE INDEX "KBChatbot_kbId_chatbotId_key" ON "KBChatbot"("kbId", "chatbotId"); + +-- Enforce the product invariant even across concurrent writers. +CREATE UNIQUE INDEX "KBChatbot_one_enabled_per_chatbot_key" +ON "KBChatbot"("chatbotId") +WHERE "isEnabled" = true; + +-- AddForeignKey +ALTER TABLE "KB" ADD CONSTRAINT "KB_ownerId_fkey" FOREIGN KEY ("ownerId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KB" ADD CONSTRAINT "KB_deletedById_fkey" FOREIGN KEY ("deletedById") REFERENCES "User"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBResource" ADD CONSTRAINT "KBResource_deletedById_fkey" FOREIGN KEY ("deletedById") REFERENCES "User"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBResource" ADD CONSTRAINT "KBResource_kbId_fkey" FOREIGN KEY ("kbId") REFERENCES "KB"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBIngestionRun" ADD CONSTRAINT "KBIngestionRun_resourceId_fkey" FOREIGN KEY ("resourceId") REFERENCES "KBResource"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBGraphBuild" ADD CONSTRAINT "KBGraphBuild_quotaId_fkey" FOREIGN KEY ("quotaId") REFERENCES "KBGraphQuota"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBGraphBuild" ADD CONSTRAINT "KBGraphBuild_requestedById_fkey" FOREIGN KEY ("requestedById") REFERENCES "User"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBGraphBuild" ADD CONSTRAINT "KBGraphBuild_kbId_fkey" FOREIGN KEY ("kbId") REFERENCES "KB"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBGraphQuota" ADD CONSTRAINT "KBGraphQuota_ownerId_fkey" FOREIGN KEY ("ownerId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBGraphBuildSource" ADD CONSTRAINT "KBGraphBuildSource_buildId_fkey" FOREIGN KEY ("buildId") REFERENCES "KBGraphBuild"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBUploadTicket" ADD CONSTRAINT "KBUploadTicket_kbId_fkey" FOREIGN KEY ("kbId") REFERENCES "KB"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBChatbot" ADD CONSTRAINT "KBChatbot_kbId_fkey" FOREIGN KEY ("kbId") REFERENCES "KB"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "KBChatbot" ADD CONSTRAINT "KBChatbot_chatbotId_fkey" FOREIGN KEY ("chatbotId") REFERENCES "Chatbot"("id") ON DELETE CASCADE ON UPDATE CASCADE; diff --git a/packages/prisma/src/prisma/schema/user.prisma b/packages/prisma/src/prisma/schema/user.prisma index 12eb2ce605..1a84243f67 100644 --- a/packages/prisma/src/prisma/schema/user.prisma +++ b/packages/prisma/src/prisma/schema/user.prisma @@ -132,6 +132,11 @@ model User { answerCollections AnswerCollection[] chatbots Chatbot[] chatbotDisclaimers ChatbotDisclaimer[] + kbs KB[] + deletedKbs KB[] @relation("KBDeletedBy") + deletedKbResources KBResource[] @relation("KBResourceDeletedBy") + requestedKbGraphBuilds KBGraphBuild[] + kbGraphQuotas KBGraphQuota[] revokedVerificationRecords VerifiableCredential[] @relation("RevokedVerifiableCredentials") userGroups UserGroup[] @relation("UserGroupMembers") diff --git a/packages/shared-components/package.json b/packages/shared-components/package.json index 5075cb1760..80ce44e2e1 100644 --- a/packages/shared-components/package.json +++ b/packages/shared-components/package.json @@ -3,6 +3,9 @@ "name": "@klicker-uzh/shared-components", "main": "src/index.ts", "types": "src/index.ts", + "dependencies": { + "cytoscape": "3.34.0" + }, "devDependencies": { "@klicker-uzh/graphql": "workspace:*", "@klicker-uzh/markdown": "workspace:*", diff --git a/packages/shared-components/src/knowledgeGraph/KnowledgeGraphDetails.tsx b/packages/shared-components/src/knowledgeGraph/KnowledgeGraphDetails.tsx new file mode 100644 index 0000000000..af77fef56a --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/KnowledgeGraphDetails.tsx @@ -0,0 +1,218 @@ +'use client' + +import type { KnowledgeGraphEdge, KnowledgeGraphNode } from '@klicker-uzh/types' +import React from 'react' +import type { KnowledgeGraphDetailsLabels } from './knowledgeGraphLabels' + +type KnowledgeGraphDetailsProps = { + node?: KnowledgeGraphNode + edge?: KnowledgeGraphEdge + edgeEndpoints?: { source: string; target: string } + isExpanding: boolean + onClose: () => void + onExpand: (nodeId: string) => void + labels: KnowledgeGraphDetailsLabels +} + +function PropertyList({ + properties, + heading, +}: { + properties: Record + heading: string +}) { + const entries = Object.entries(properties) + if (entries.length === 0) { + return null + } + + return ( +
+

+ {heading} +

+
+ {entries.map(([key, value]) => ( +
+
{key}
+
+ {String(value)} +
+
+ ))} +
+
+ ) +} + +export function KnowledgeGraphDetails({ + node, + edge, + edgeEndpoints, + isExpanding, + onClose, + onExpand, + labels, +}: KnowledgeGraphDetailsProps) { + if (node === undefined && edge === undefined) { + return null + } + + const heading = node?.displayLabel ?? edge?.label ?? labels.detailsFallback + + return ( + + ) +} diff --git a/packages/shared-components/src/knowledgeGraph/KnowledgeGraphViewer.tsx b/packages/shared-components/src/knowledgeGraph/KnowledgeGraphViewer.tsx new file mode 100644 index 0000000000..a1ca2ecc67 --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/KnowledgeGraphViewer.tsx @@ -0,0 +1,947 @@ +'use client' + +import cytoscape from 'cytoscape' +import React, { + type FormEvent, + useCallback, + useEffect, + useLayoutEffect, + useMemo, + useReducer, + useRef, + useState, +} from 'react' + +import { KnowledgeGraphDetails } from './KnowledgeGraphDetails' +import { + CYTOSCAPE_STYLE, + cytoscapeEdgeId, + cytoscapeNodeId, + edgeDefinition, + kindStyle, + nodeDefinition, +} from './knowledgeGraphCytoscape' +import { + type KnowledgeGraphViewerLabelOverrides, + type KnowledgeGraphViewerLabels, + resolveKnowledgeGraphLabels, +} from './knowledgeGraphLabels' +import { + type KnowledgeGraphDataSource, + type KnowledgeGraphRequestOperation, + KnowledgeGraphUnavailableError, + initialKnowledgeGraphState, + knowledgeGraphReducer, +} from './knowledgeGraphState' +import { + KNOWLEDGE_GRAPH_MAX_ZOOM, + KNOWLEDGE_GRAPH_MIN_ZOOM, + nextKnowledgeGraphZoom, + relationshipLabels, +} from './knowledgeGraphView' + +type KnowledgeGraphViewerProps = { + dataSource: KnowledgeGraphDataSource + className?: string + unavailableMessage?: string + labels?: KnowledgeGraphViewerLabelOverrides +} + +function isUnavailableError(error: unknown): boolean { + if (error instanceof KnowledgeGraphUnavailableError) { + return true + } + + if (typeof error !== 'object' || error === null) { + return false + } + + const candidate = error as { code?: unknown; status?: unknown } + return candidate.code === 'UNAVAILABLE' || candidate.status === 409 +} + +function safeRequestError( + operation: 'overview' | 'search' | 'neighbors', + labels: KnowledgeGraphViewerLabels +) { + if (operation === 'search') { + return labels.searchUnavailable + } + if (operation === 'neighbors') { + return labels.connectionsUnavailable + } + return labels.graphUnavailable +} + +export function KnowledgeGraphViewer({ + dataSource, + className = '', + unavailableMessage, + labels: labelOverrides, +}: KnowledgeGraphViewerProps) { + const labels = useMemo( + () => resolveKnowledgeGraphLabels(labelOverrides), + [labelOverrides] + ) + const resolvedUnavailableMessage = + unavailableMessage ?? labels.defaultUnavailableMessage + const [state, dispatch] = useReducer( + knowledgeGraphReducer, + initialKnowledgeGraphState + ) + const [searchQuery, setSearchQuery] = useState('') + const containerRef = useRef(null) + const cyRef = useRef(null) + const dataSourceRef = useRef(dataSource) + const stateRef = useRef(state) + const requestIdRef = useRef(0) + const latestRequestIdsRef = useRef< + Record + >({ overview: null, search: null, neighbors: null }) + const sourceGenerationRef = useRef(0) + const mountedDataSourceRef = useRef(null) + const positionsRef = useRef(new Map()) + const renderedBuildIdRef = useRef(null) + const expansionOriginRef = useRef(null) + const pendingFocusRef = useRef(null) + const prefersReducedMotionRef = useRef(false) + const expandNodeRef = useRef<(nodeId: string) => void>(() => undefined) + const labelsRef = useRef(labels) + + dataSourceRef.current = dataSource + stateRef.current = state + labelsRef.current = labels + + const runRequest = useCallback( + async ( + operation: KnowledgeGraphRequestOperation, + input: string | null, + request: () => ReturnType + ) => { + const requestId = ++requestIdRef.current + const sourceGeneration = sourceGenerationRef.current + latestRequestIdsRef.current[operation] = requestId + dispatch({ + type: 'request-started', + operation, + requestId, + ...(input === null ? {} : { input }), + }) + + try { + const graphResponse = await request() + if ( + sourceGenerationRef.current !== sourceGeneration || + latestRequestIdsRef.current[operation] !== requestId + ) { + return null + } + dispatch({ + type: 'request-succeeded', + operation, + requestId, + response: graphResponse, + announcement: + operation === 'search' + ? labelsRef.current.searchResultsLoadedAnnouncement( + graphResponse.nodes.length + ) + : labelsRef.current.conceptsLoadedAnnouncement( + graphResponse.nodes.length + ), + }) + return graphResponse + } catch (error) { + if ( + sourceGenerationRef.current !== sourceGeneration || + latestRequestIdsRef.current[operation] !== requestId + ) { + return null + } + if (isUnavailableError(error)) { + dispatch({ + type: 'request-unavailable', + operation, + requestId, + message: resolvedUnavailableMessage, + ...(input === null ? {} : { input }), + }) + } else { + dispatch({ + type: 'request-failed', + operation, + requestId, + message: safeRequestError(operation, labelsRef.current), + ...(input === null ? {} : { input }), + }) + } + return null + } + }, + [resolvedUnavailableMessage] + ) + + const loadOverview = useCallback(async () => { + await runRequest('overview', null, () => dataSourceRef.current.overview()) + }, [runRequest]) + + const expandNode = useCallback( + async (nodeId: string) => { + expansionOriginRef.current = nodeId + const loadedNodeIds = new Set( + stateRef.current.nodes.map((node) => node.id) + ) + const result = await runRequest('neighbors', nodeId, () => + dataSourceRef.current.neighbors(nodeId) + ) + const hasNewNodes = + result?.nodes.some( + (node) => node.id !== nodeId && !loadedNodeIds.has(node.id) + ) ?? false + if ( + (!hasNewNodes || result === null) && + expansionOriginRef.current === nodeId + ) { + expansionOriginRef.current = null + } + }, + [runRequest] + ) + expandNodeRef.current = (nodeId) => { + void expandNode(nodeId) + } + + useLayoutEffect(() => { + if (mountedDataSourceRef.current === dataSource) { + return + } + + mountedDataSourceRef.current = dataSource + dataSourceRef.current = dataSource + sourceGenerationRef.current += 1 + latestRequestIdsRef.current = { + overview: null, + search: null, + neighbors: null, + } + stateRef.current = initialKnowledgeGraphState + positionsRef.current.clear() + renderedBuildIdRef.current = null + expansionOriginRef.current = null + pendingFocusRef.current = null + cyRef.current?.elements().remove() + setSearchQuery('') + dispatch({ type: 'reset' }) + void loadOverview() + }, [dataSource, loadOverview]) + + useEffect(() => { + const container = containerRef.current + if (container === null) { + return + } + + prefersReducedMotionRef.current = window.matchMedia( + '(prefers-reduced-motion: reduce)' + ).matches + const options: cytoscape.CytoscapeOptions & { + multiClickDebounceTime: number + } = { + container, + elements: [], + style: CYTOSCAPE_STYLE, + layout: { name: 'preset', fit: false }, + minZoom: KNOWLEDGE_GRAPH_MIN_ZOOM, + maxZoom: KNOWLEDGE_GRAPH_MAX_ZOOM, + panningEnabled: true, + userPanningEnabled: true, + zoomingEnabled: true, + userZoomingEnabled: true, + autoungrabify: false, + multiClickDebounceTime: 250, + } + const cy = cytoscape(options) + cyRef.current = cy + + const onNodeTap: cytoscape.EventHandler = (event) => { + const nodeId = String(event.target.data('graphId')) + const displayLabel = String(event.target.data('displayLabel')) + dispatch({ + type: 'select-node', + nodeId, + announcement: + labelsRef.current.selectedConceptAnnouncement(displayLabel), + }) + } + const onNodeDoubleTap: cytoscape.EventHandler = (event) => { + const nodeId = String(event.target.data('graphId')) + expansionOriginRef.current = nodeId + expandNodeRef.current(nodeId) + } + const onEdgeTap: cytoscape.EventHandler = (event) => { + dispatch({ + type: 'select-edge', + edgeId: String(event.target.data('graphId')), + announcement: labelsRef.current.selectedRelationshipAnnouncement, + }) + } + const rememberPosition: cytoscape.EventHandler = (event) => { + positionsRef.current.set( + String(event.target.data('graphId')), + event.target.position() + ) + } + + cy.on('onetap', 'node', onNodeTap) + cy.on('dbltap', 'node', onNodeDoubleTap) + cy.on('onetap', 'edge', onEdgeTap) + cy.on('dragfree', 'node', rememberPosition) + + const resizeObserver = + typeof ResizeObserver === 'undefined' + ? null + : new ResizeObserver(() => cy.resize()) + resizeObserver?.observe(container) + + return () => { + resizeObserver?.disconnect() + cy.off('onetap', 'node', onNodeTap) + cy.off('dbltap', 'node', onNodeDoubleTap) + cy.off('onetap', 'edge', onEdgeTap) + cy.off('dragfree', 'node', rememberPosition) + cy.destroy() + cyRef.current = null + positionsRef.current.clear() + } + }, []) + + useEffect(() => { + const cy = cyRef.current + if (cy === null) { + return + } + + cy.nodes().forEach((node) => { + positionsRef.current.set(String(node.data('graphId')), node.position()) + }) + + const buildChanged = renderedBuildIdRef.current !== state.buildId + if (buildChanged) { + cy.elements().remove() + positionsRef.current.clear() + renderedBuildIdRef.current = state.buildId + } + + const nodeIds = new Set(state.nodes.map((node) => node.id)) + const edgeIds = new Set(state.edges.map((edge) => edge.id)) + const expansionOriginWasLoaded = + expansionOriginRef.current !== null && + !cy.getElementById(cytoscapeNodeId(expansionOriginRef.current)).empty() + cy.edges() + .filter((edge) => !edgeIds.has(String(edge.data('graphId')))) + .remove() + cy.nodes() + .filter((node) => !nodeIds.has(String(node.data('graphId')))) + .remove() + + const newNodeIds = new Set() + for (const node of state.nodes) { + const existing = cy.getElementById(cytoscapeNodeId(node.id)) + const definition = nodeDefinition(node) + if (existing.empty()) { + cy.add(definition) + newNodeIds.add(node.id) + } else { + existing.data(definition.data) + const savedPosition = positionsRef.current.get(node.id) + if (savedPosition !== undefined) { + existing.position(savedPosition) + } + } + } + + for (const edge of state.edges) { + const existing = cy.getElementById(cytoscapeEdgeId(edge.id)) + const endpointsExist = + !cy.getElementById(cytoscapeNodeId(edge.source)).empty() && + !cy.getElementById(cytoscapeNodeId(edge.target)).empty() + if (existing.empty() && endpointsExist) { + cy.add(edgeDefinition(edge)) + } else if (!existing.empty()) { + existing.data(edgeDefinition(edge).data) + } + } + + const newNodes = cy + .nodes() + .filter((node) => newNodeIds.has(String(node.data('graphId')))) + if (newNodes.empty()) { + if (expansionOriginWasLoaded) { + expansionOriginRef.current = null + } + return + } + + const origin = + expansionOriginRef.current === null + ? cy.collection() + : cy.getElementById(cytoscapeNodeId(expansionOriginRef.current)) + const viewportExtent = cy.extent() + const originPosition = origin.empty() + ? { + x: (viewportExtent.x1 + viewportExtent.x2) / 2, + y: (viewportExtent.y1 + viewportExtent.y2) / 2, + } + : origin.position() + newNodes.forEach((node, index) => { + const angle = (index / Math.max(newNodes.length, 1)) * Math.PI * 2 + node.position({ + x: originPosition.x + Math.cos(angle) * 100, + y: originPosition.y + Math.sin(angle) * 100, + }) + }) + + const isInitialLayout = buildChanged + const subsetEdges = cy.edges().filter((edge) => { + return ( + newNodeIds.has(String(edge.source().data('graphId'))) && + newNodeIds.has(String(edge.target().data('graphId'))) + ) + }) + const layoutElements = isInitialLayout + ? cy.elements() + : newNodes.union(subsetEdges) + const layout = layoutElements.layout({ + name: 'cose', + animate: !prefersReducedMotionRef.current, + randomize: isInitialLayout, + fit: false, + padding: 30, + nodeRepulsion: 6_000, + idealEdgeLength: 90, + }) + layout.one('layoutstop', () => { + cy.nodes().forEach((node) => { + positionsRef.current.set(String(node.data('graphId')), node.position()) + }) + if (isInitialLayout) { + cy.fit(cy.elements(), 40) + } + }) + layout.run() + if (expansionOriginWasLoaded) { + expansionOriginRef.current = null + } + }, [state.buildId, state.edges, state.nodes]) + + useEffect(() => { + const cy = cyRef.current + if (cy === null) { + return + } + + cy.elements().unselect() + const selectedId = state.selectedNodeId ?? state.selectedEdgeId + if (selectedId !== null) { + const cytoscapeId = + state.selectedNodeId === null + ? cytoscapeEdgeId(selectedId) + : cytoscapeNodeId(selectedId) + cy.getElementById(cytoscapeId).select() + } + + if ( + state.focusedNodeId !== null && + pendingFocusRef.current === state.focusedNodeId + ) { + const focusedNode = cy.getElementById( + cytoscapeNodeId(state.focusedNodeId) + ) + if (!focusedNode.empty()) { + cy.center(focusedNode) + pendingFocusRef.current = null + } + } + }, [state.focusedNodeId, state.selectedEdgeId, state.selectedNodeId]) + + const indexes = useMemo(() => { + return { + nodes: new Map(state.nodes.map((node) => [node.id, node])), + edges: new Map(state.edges.map((edge) => [edge.id, edge])), + } + }, [state.edges, state.nodes]) + const selectedNode = + state.selectedNodeId === null + ? undefined + : indexes.nodes.get(state.selectedNodeId) + const selectedEdge = + state.selectedEdgeId === null + ? undefined + : indexes.edges.get(state.selectedEdgeId) + const selectedEdgeEndpoints = + selectedEdge === undefined + ? undefined + : relationshipLabels(selectedEdge, indexes.nodes) + const relationshipEntries = useMemo( + () => + state.edges.map((edge) => ({ + edge, + ...relationshipLabels(edge, indexes.nodes), + })), + [indexes.nodes, state.edges] + ) + + const legendEntries = useMemo(() => { + const kinds = new Map>() + for (const node of state.nodes) { + if (!kinds.has(node.kind)) { + kinds.set(node.kind, kindStyle(node.kind)) + } + } + return Array.from(kinds.entries()).slice(0, 8) + }, [state.nodes]) + + function focusNode(nodeId: string) { + pendingFocusRef.current = nodeId + const displayLabel = + indexes.nodes.get(nodeId)?.displayLabel ?? labels.details.concept + dispatch({ + type: 'focus-search-result', + nodeId, + announcement: labels.selectedConceptAnnouncement(displayLabel), + }) + } + + const searchGraph = useCallback( + async (query: string) => { + const loadedNodeIds = new Set( + stateRef.current.nodes.map((node) => node.id) + ) + const result = await runRequest('search', query, () => + dataSourceRef.current.search(query) + ) + const firstResult = result?.nodes[0] + if (firstResult === undefined) { + return + } + + pendingFocusRef.current = firstResult.id + if (!loadedNodeIds.has(firstResult.id)) { + await expandNode(firstResult.id) + } + }, + [expandNode, runRequest] + ) + + async function handleSearch(event: FormEvent) { + event.preventDefault() + const query = searchQuery.trim() + if (query.length === 0 || query.length > 100) { + return + } + + await searchGraph(query) + } + + function retryFailedRequest() { + const failedRequest = stateRef.current.failedRequest + if (failedRequest === null || failedRequest.operation === 'overview') { + void loadOverview() + return + } + + if (failedRequest.operation === 'search') { + if (failedRequest.input !== null) { + setSearchQuery(failedRequest.input) + void searchGraph(failedRequest.input) + } + return + } + + if (failedRequest.input !== null) { + void expandNode(failedRequest.input) + } + } + + function fitGraph() { + const cy = cyRef.current + if (cy !== null && !cy.elements().empty()) { + cy.fit(cy.elements(), 40) + } + } + + function changeZoom(scale: number) { + const cy = cyRef.current + if (cy === null) { + return + } + + cy.zoom({ + level: nextKnowledgeGraphZoom( + cy.zoom(), + scale, + cy.minZoom(), + cy.maxZoom() + ), + renderedPosition: { x: cy.width() / 2, y: cy.height() / 2 }, + }) + } + + function resetLayout() { + const cy = cyRef.current + if (cy === null || cy.nodes().empty()) { + return + } + + const layout = cy.elements().layout({ + name: 'cose', + animate: !prefersReducedMotionRef.current, + randomize: true, + fit: false, + padding: 30, + nodeRepulsion: 6_000, + idealEdgeLength: 90, + }) + layout.run() + } + + const showFullError = state.status === 'error' && state.nodes.length === 0 + const isSearching = state.activeRequestIds.search !== null + const isExpanding = state.activeRequestIds.neighbors !== null + + return ( +
+
+
+
void handleSearch(event)} + className="flex gap-2" + > + + setSearchQuery(event.target.value)} + placeholder={labels.searchPlaceholder} + className="min-h-11 min-w-0 flex-1 rounded border border-[#E9E9E9] px-3 py-2 text-base text-[#121212] placeholder:text-[#666666] focus:border-[#0028A5] focus:outline-none focus:ring-2 focus:ring-[#BDC9E8]" + data-cy="knowledge-graph-search" + /> + +
+ + {state.truncated ? ( +

+ {labels.truncatedNotice} +

+ ) : null} + + {state.errorMessage !== null && !showFullError ? ( +
+ {state.errorMessage} + +
+ ) : null} +
+ +
+ + +
+ {state.searchResults.length === 0 ? null : ( +
+

+ {labels.searchResults} +

+
    + {state.searchResults.map((node) => ( +
  • + +
    • + ))} +
+
+ )} + +
+

+ {labels.loadedConcepts(state.nodes.length)} +

+ {state.nodes.length === 0 ? ( +

+ {labels.noConceptsLoaded} +

+ ) : ( +
    + {state.nodes.map((node) => ( +
  • + +
    • + ))} +
+ )} +
+ +
+

+ {labels.loadedRelationships(relationshipEntries.length)} +

+ {relationshipEntries.length === 0 ? ( +

+ {labels.noRelationshipsLoaded} +

+ ) : ( +
    + {relationshipEntries.map(({ edge, source, target }) => ( +
  • + +
    • + ))} +
+ )} +
+
+
+ + + dispatch({ + type: 'close-details', + announcement: labels.detailsClosedAnnouncement, + }) + } + onExpand={(nodeId) => void expandNode(nodeId)} + /> + +

+ {state.announcement} +

+
+ ) +} + +export default KnowledgeGraphViewer diff --git a/packages/shared-components/src/knowledgeGraph/knowledgeGraphCytoscape.ts b/packages/shared-components/src/knowledgeGraph/knowledgeGraphCytoscape.ts new file mode 100644 index 0000000000..9a1ff86ee0 --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/knowledgeGraphCytoscape.ts @@ -0,0 +1,138 @@ +import type { KnowledgeGraphEdge, KnowledgeGraphNode } from '@klicker-uzh/types' +import type cytoscape from 'cytoscape' + +const UZH_KIND_STYLES = [ + { + color: '#BDC9E8', + borderColor: '#001E7C', + shape: 'ellipse', + legendClassName: 'rounded-full bg-[#BDC9E8] border-[#001E7C]', + shapeLabelKey: 'shapeCircle', + }, + { + color: '#F78CAA', + borderColor: '#8F0A2E', + shape: 'diamond', + legendClassName: 'rotate-45 bg-[#F78CAA] border-[#8F0A2E]', + shapeLabelKey: 'shapeDiamond', + }, + { + color: '#FFE9B5', + borderColor: '#A27200', + shape: 'round-rectangle', + legendClassName: 'rounded bg-[#FFE9B5] border-[#A27200]', + shapeLabelKey: 'shapeRoundedSquare', + }, + { + color: '#E7E7E7', + borderColor: '#4D4D4D', + shape: 'hexagon', + legendClassName: 'rounded-sm bg-[#E7E7E7] border-[#4D4D4D]', + shapeLabelKey: 'shapeHexagon', + }, +] as const + +export const CYTOSCAPE_STYLE: cytoscape.StylesheetJson = [ + { + selector: 'node', + style: { + width: 48, + height: 48, + shape: + 'data(shape)' as cytoscape.Css.PropertyValueNode, + 'background-color': 'data(color)', + 'border-color': 'data(borderColor)', + 'border-width': 2, + label: 'data(displayLabel)', + color: '#121212', + 'font-family': 'Source Sans 3, Source Sans Pro, sans-serif', + 'font-size': 12, + 'font-weight': 600, + 'text-wrap': 'wrap', + 'text-max-width': '120px', + 'text-valign': 'bottom', + 'text-margin-y': 8, + 'overlay-opacity': 0, + }, + }, + { + selector: 'node:selected', + style: { + 'background-color': '#0028A5', + 'border-color': '#001452', + 'border-width': 4, + 'underlay-color': '#BDC9E8', + 'underlay-opacity': 0.45, + 'underlay-padding': 8, + }, + }, + { + selector: 'edge', + style: { + width: 1.5, + 'line-color': '#A3A3A3', + 'target-arrow-color': '#A3A3A3', + 'target-arrow-shape': 'triangle', + 'curve-style': 'bezier', + 'overlay-opacity': 0, + }, + }, + { + selector: 'edge:selected', + style: { + width: 3, + 'line-color': '#0028A5', + 'target-arrow-color': '#0028A5', + }, + }, +] + +export function kindStyle(kind: string) { + let hash = 0 + for (const character of kind) { + hash = (hash * 31 + character.charCodeAt(0)) >>> 0 + } + return UZH_KIND_STYLES[hash % UZH_KIND_STYLES.length]! +} + +export function cytoscapeNodeId(nodeId: string) { + return `node:${nodeId}` +} + +export function cytoscapeEdgeId(edgeId: string) { + return `edge:${edgeId}` +} + +export function nodeDefinition( + node: KnowledgeGraphNode +): cytoscape.NodeDefinition { + const style = kindStyle(node.kind) + return { + group: 'nodes', + data: { + id: cytoscapeNodeId(node.id), + graphId: node.id, + displayLabel: node.displayLabel, + kind: node.kind, + color: style.color, + borderColor: style.borderColor, + shape: style.shape, + }, + } +} + +export function edgeDefinition( + edge: KnowledgeGraphEdge +): cytoscape.EdgeDefinition { + return { + group: 'edges', + data: { + id: cytoscapeEdgeId(edge.id), + graphId: edge.id, + source: cytoscapeNodeId(edge.source), + target: cytoscapeNodeId(edge.target), + label: edge.label, + type: edge.type, + }, + } +} diff --git a/packages/shared-components/src/knowledgeGraph/knowledgeGraphLabels.ts b/packages/shared-components/src/knowledgeGraph/knowledgeGraphLabels.ts new file mode 100644 index 0000000000..9989367f28 --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/knowledgeGraphLabels.ts @@ -0,0 +1,152 @@ +export type KnowledgeGraphDetailsLabels = { + detailsFallback: string + ariaLabel: string + relationship: string + concept: string + closeAriaLabel: string + type: string + connections: string + summary: string + content: string + sources: string + loadingConnections: string + expandConnections: string + from: string + to: string + properties: string +} + +export type KnowledgeGraphViewerLabels = { + explorerAriaLabel: string + searchAriaLabel: string + searchLabel: string + searchPlaceholder: string + searching: string + search: string + truncatedNotice: string + retry: string + canvasAriaLabel: string + zoomInAriaLabel: string + zoomIn: string + zoomOutAriaLabel: string + zoomOut: string + fitView: string + resetLayout: string + legendAriaLabel: string + conceptTypes: string + shapeCircle: string + shapeDiamond: string + shapeRoundedSquare: string + shapeHexagon: string + loading: string + unavailableTitle: string + notReadyTitle: string + checkAgain: string + searchResults: string + loadedConcepts: (count: number) => string + noConceptsLoaded: string + loadedRelationships: (count: number) => string + noRelationshipsLoaded: string + selectRelationshipAriaLabel: ( + source: string, + target: string, + label: string + ) => string + searchUnavailable: string + connectionsUnavailable: string + graphUnavailable: string + searchResultsLoadedAnnouncement: (count: number) => string + conceptsLoadedAnnouncement: (count: number) => string + selectedConceptAnnouncement: (label: string) => string + selectedRelationshipAnnouncement: string + detailsClosedAnnouncement: string + defaultUnavailableMessage: string + details: KnowledgeGraphDetailsLabels +} + +export type KnowledgeGraphViewerLabelOverrides = Partial< + Omit +> & { + details?: Partial +} + +export const DEFAULT_KNOWLEDGE_GRAPH_LABELS: KnowledgeGraphViewerLabels = { + explorerAriaLabel: 'Knowledge graph explorer', + searchAriaLabel: 'Search the knowledge graph', + searchLabel: 'Search concepts', + searchPlaceholder: 'Search concepts…', + searching: 'Searching…', + search: 'Search', + truncatedNotice: + 'This bounded view shows the most connected concepts. Search to explore the complete graph.', + retry: 'Retry', + canvasAriaLabel: + 'Interactive knowledge graph. Use the concept and relationship lists below for keyboard navigation.', + zoomInAriaLabel: 'Zoom in on the knowledge graph', + zoomIn: 'Zoom in', + zoomOutAriaLabel: 'Zoom out of the knowledge graph', + zoomOut: 'Zoom out', + fitView: 'Fit view', + resetLayout: 'Reset layout', + legendAriaLabel: 'Concept type legend', + conceptTypes: 'Concept types', + shapeCircle: 'circle', + shapeDiamond: 'diamond', + shapeRoundedSquare: 'rounded square', + shapeHexagon: 'hexagon', + loading: 'Loading knowledge graph…', + unavailableTitle: 'Knowledge graph unavailable', + notReadyTitle: 'Knowledge graph not ready', + checkAgain: 'Check again', + searchResults: 'Search results', + loadedConcepts: (count) => `Loaded concepts (${count})`, + noConceptsLoaded: 'No concepts loaded.', + loadedRelationships: (count) => `Loaded relationships (${count})`, + noRelationshipsLoaded: 'No relationships loaded.', + selectRelationshipAriaLabel: (source, target, label) => + `Select relationship ${source} to ${target}: ${label}`, + searchUnavailable: 'Search is temporarily unavailable. Try again.', + connectionsUnavailable: 'Connections are temporarily unavailable. Try again.', + graphUnavailable: 'The knowledge graph is temporarily unavailable.', + searchResultsLoadedAnnouncement: (count) => `${count} search results loaded.`, + conceptsLoadedAnnouncement: (count) => `${count} concepts loaded.`, + selectedConceptAnnouncement: (label) => `Selected ${label}.`, + selectedRelationshipAnnouncement: 'Selected relationship.', + detailsClosedAnnouncement: 'Details closed.', + defaultUnavailableMessage: + 'The knowledge graph is not available for the current resource selection.', + details: { + detailsFallback: 'Details', + ariaLabel: 'Knowledge graph details', + relationship: 'Relationship', + concept: 'Concept', + closeAriaLabel: 'Close details', + type: 'Type', + connections: 'Connections', + summary: 'Summary', + content: 'Content', + sources: 'Sources', + loadingConnections: 'Loading connections…', + expandConnections: 'Expand connections', + from: 'From', + to: 'To', + properties: 'Properties', + }, +} + +export function resolveKnowledgeGraphLabels( + overrides?: KnowledgeGraphViewerLabelOverrides +): KnowledgeGraphViewerLabels { + if (overrides === undefined) { + return DEFAULT_KNOWLEDGE_GRAPH_LABELS + } + + return { + ...DEFAULT_KNOWLEDGE_GRAPH_LABELS, + ...overrides, + details: { + ...DEFAULT_KNOWLEDGE_GRAPH_LABELS.details, + ...overrides.details, + }, + } +} diff --git a/packages/shared-components/src/knowledgeGraph/knowledgeGraphState.ts b/packages/shared-components/src/knowledgeGraph/knowledgeGraphState.ts new file mode 100644 index 0000000000..e7bbe207b2 --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/knowledgeGraphState.ts @@ -0,0 +1,332 @@ +import type { + KnowledgeGraphEdge, + KnowledgeGraphNode, + KnowledgeGraphResponse, +} from '@klicker-uzh/types' + +export type KnowledgeGraphDataSource = { + overview: () => Promise + search: (query: string) => Promise + neighbors: (nodeId: string) => Promise +} + +export class KnowledgeGraphUnavailableError extends Error { + constructor(message = 'Knowledge graph is unavailable') { + super(message) + this.name = 'KnowledgeGraphUnavailableError' + } +} + +export type KnowledgeGraphRequestOperation = 'overview' | 'search' | 'neighbors' + +export type KnowledgeGraphFailedRequest = { + operation: KnowledgeGraphRequestOperation + input: string | null +} + +export type KnowledgeGraphViewerStatus = + | 'idle' + | 'loading' + | 'ready' + | 'error' + | 'unavailable' + +type ActiveRequestIds = Record + +export type KnowledgeGraphState = { + kbId: string | null + buildId: string | null + isStale: boolean + nodes: KnowledgeGraphNode[] + edges: KnowledgeGraphEdge[] + truncated: boolean + selectedNodeId: string | null + selectedEdgeId: string | null + focusedNodeId: string | null + searchResults: KnowledgeGraphNode[] + status: KnowledgeGraphViewerStatus + errorMessage: string | null + unavailableMessage: string | null + failedRequest: KnowledgeGraphFailedRequest | null + announcement: string + activeRequestIds: ActiveRequestIds +} + +export type KnowledgeGraphAction = + | { + type: 'request-started' + operation: KnowledgeGraphRequestOperation + requestId: number + input?: string + } + | { + type: 'request-succeeded' + operation: KnowledgeGraphRequestOperation + requestId: number + response: KnowledgeGraphResponse + announcement?: string + } + | { + type: 'request-failed' + operation: KnowledgeGraphRequestOperation + requestId: number + message: string + input?: string + } + | { + type: 'request-unavailable' + operation: KnowledgeGraphRequestOperation + requestId: number + message: string + input?: string + } + | { type: 'select-node'; nodeId: string; announcement?: string } + | { type: 'select-edge'; edgeId: string; announcement?: string } + | { type: 'focus-search-result'; nodeId: string; announcement?: string } + | { type: 'close-details'; announcement?: string } + | { type: 'clear-search' } + | { type: 'reset' } + +const emptyActiveRequestIds: ActiveRequestIds = { + overview: null, + search: null, + neighbors: null, +} + +export const initialKnowledgeGraphState: KnowledgeGraphState = { + kbId: null, + buildId: null, + isStale: false, + nodes: [], + edges: [], + truncated: false, + selectedNodeId: null, + selectedEdgeId: null, + focusedNodeId: null, + searchResults: [], + status: 'idle', + errorMessage: null, + unavailableMessage: null, + failedRequest: null, + announcement: '', + activeRequestIds: emptyActiveRequestIds, +} + +function deduplicateById( + current: T[], + incoming: T[] +): T[] { + const entries = new Map() + for (const entry of current) { + entries.set(entry.id, entry) + } + for (const entry of incoming) { + entries.set(entry.id, entry) + } + return Array.from(entries.values()) +} + +function replacesCurrentGraph( + state: KnowledgeGraphState, + incoming: KnowledgeGraphResponse +): boolean { + return ( + state.buildId === null || + state.buildId !== incoming.buildId || + state.kbId !== incoming.kbId + ) +} + +function replaceKnowledgeGraphResponse( + state: KnowledgeGraphState, + incoming: KnowledgeGraphResponse +): KnowledgeGraphState { + return { + ...state, + kbId: incoming.kbId, + buildId: incoming.buildId, + isStale: incoming.isStale, + nodes: deduplicateById([], incoming.nodes), + edges: deduplicateById([], incoming.edges), + truncated: incoming.truncated, + selectedNodeId: null, + selectedEdgeId: null, + focusedNodeId: null, + searchResults: [], + } +} + +export function mergeKnowledgeGraphResponse( + state: KnowledgeGraphState, + incoming: KnowledgeGraphResponse +): KnowledgeGraphState { + if (replacesCurrentGraph(state, incoming)) { + return replaceKnowledgeGraphResponse(state, incoming) + } + + return { + ...state, + isStale: incoming.isStale, + nodes: deduplicateById(state.nodes, incoming.nodes), + edges: deduplicateById(state.edges, incoming.edges), + truncated: state.truncated || incoming.truncated, + } +} + +function withoutActiveRequest( + state: KnowledgeGraphState, + operation: KnowledgeGraphRequestOperation +): ActiveRequestIds { + return { ...state.activeRequestIds, [operation]: null } +} + +function isStaleRequest( + state: KnowledgeGraphState, + operation: KnowledgeGraphRequestOperation, + requestId: number +): boolean { + return state.activeRequestIds[operation] !== requestId +} + +function hasNode(state: KnowledgeGraphState, nodeId: string): boolean { + return state.nodes.some((node) => node.id === nodeId) +} + +function hasEdge(state: KnowledgeGraphState, edgeId: string): boolean { + return state.edges.some((edge) => edge.id === edgeId) +} + +export function knowledgeGraphReducer( + state: KnowledgeGraphState, + action: KnowledgeGraphAction +): KnowledgeGraphState { + switch (action.type) { + case 'request-started': + return { + ...state, + status: action.operation === 'overview' ? 'loading' : state.status, + errorMessage: null, + unavailableMessage: + action.operation === 'overview' ? null : state.unavailableMessage, + failedRequest: null, + searchResults: action.operation === 'search' ? [] : state.searchResults, + activeRequestIds: { + ...state.activeRequestIds, + [action.operation]: action.requestId, + }, + } + + case 'request-succeeded': { + if (isStaleRequest(state, action.operation, action.requestId)) { + return state + } + + const replacesGraph = + action.operation === 'overview' || + replacesCurrentGraph(state, action.response) + const nextGraph = replacesGraph + ? replaceKnowledgeGraphResponse(state, action.response) + : mergeKnowledgeGraphResponse(state, action.response) + const firstSearchResult = + action.operation === 'search' ? action.response.nodes[0] : undefined + + return { + ...nextGraph, + status: 'ready', + errorMessage: null, + unavailableMessage: null, + failedRequest: null, + selectedNodeId: firstSearchResult?.id ?? nextGraph.selectedNodeId, + selectedEdgeId: + firstSearchResult === undefined ? nextGraph.selectedEdgeId : null, + focusedNodeId: firstSearchResult?.id ?? nextGraph.focusedNodeId, + searchResults: + action.operation === 'search' + ? deduplicateById([], action.response.nodes) + : nextGraph.searchResults, + announcement: + action.announcement ?? + (action.operation === 'search' + ? `${action.response.nodes.length} search results loaded.` + : `${action.response.nodes.length} concepts loaded.`), + activeRequestIds: replacesGraph + ? emptyActiveRequestIds + : withoutActiveRequest(state, action.operation), + } + } + + case 'request-failed': + if (isStaleRequest(state, action.operation, action.requestId)) { + return state + } + return { + ...state, + status: action.operation === 'overview' ? 'error' : state.status, + errorMessage: action.message, + failedRequest: { + operation: action.operation, + input: action.input ?? null, + }, + announcement: action.message, + activeRequestIds: withoutActiveRequest(state, action.operation), + } + + case 'request-unavailable': + if (isStaleRequest(state, action.operation, action.requestId)) { + return state + } + return { + ...initialKnowledgeGraphState, + status: 'unavailable', + unavailableMessage: action.message, + failedRequest: { + operation: action.operation, + input: action.input ?? null, + }, + announcement: action.message, + } + + case 'select-node': + case 'focus-search-result': + if (!hasNode(state, action.nodeId)) { + return state + } + return { + ...state, + selectedNodeId: action.nodeId, + selectedEdgeId: null, + focusedNodeId: action.nodeId, + announcement: + action.announcement ?? + `Selected ${ + state.nodes.find((node) => node.id === action.nodeId) + ?.displayLabel ?? 'concept' + }.`, + } + + case 'select-edge': + if (!hasEdge(state, action.edgeId)) { + return state + } + return { + ...state, + selectedNodeId: null, + selectedEdgeId: action.edgeId, + announcement: action.announcement ?? 'Selected relationship.', + } + + case 'close-details': + return { + ...state, + selectedNodeId: null, + selectedEdgeId: null, + announcement: action.announcement ?? 'Details closed.', + } + + case 'clear-search': + return { ...state, searchResults: [] } + + case 'reset': + return initialKnowledgeGraphState + } +} diff --git a/packages/shared-components/src/knowledgeGraph/knowledgeGraphView.ts b/packages/shared-components/src/knowledgeGraph/knowledgeGraphView.ts new file mode 100644 index 0000000000..38e4e15e75 --- /dev/null +++ b/packages/shared-components/src/knowledgeGraph/knowledgeGraphView.ts @@ -0,0 +1,23 @@ +import type { KnowledgeGraphEdge, KnowledgeGraphNode } from '@klicker-uzh/types' + +export const KNOWLEDGE_GRAPH_MIN_ZOOM = 0.15 +export const KNOWLEDGE_GRAPH_MAX_ZOOM = 3 + +export function nextKnowledgeGraphZoom( + currentZoom: number, + scale: number, + minZoom = KNOWLEDGE_GRAPH_MIN_ZOOM, + maxZoom = KNOWLEDGE_GRAPH_MAX_ZOOM +): number { + return Math.min(maxZoom, Math.max(minZoom, currentZoom * scale)) +} + +export function relationshipLabels( + edge: KnowledgeGraphEdge, + nodes: Map +): { source: string; target: string } { + return { + source: nodes.get(edge.source)?.displayLabel ?? edge.source, + target: nodes.get(edge.target)?.displayLabel ?? edge.target, + } +} diff --git a/packages/types/src/hatchet.ts b/packages/types/src/hatchet.ts index ece87f22a5..b028cd42fe 100644 --- a/packages/types/src/hatchet.ts +++ b/packages/types/src/hatchet.ts @@ -1,6 +1,7 @@ import type { Context, HatchetClient, + JsonObject, TaskWorkflowDeclaration, } from '@hatchet-dev/typescript-sdk/index.js' import type { PrismaClient } from '@klicker-uzh/prisma/client' @@ -18,6 +19,44 @@ export interface HatchetHandlerGlobalContext { prisma: PrismaClient } +export const MAX_KB_RESOURCE_COUNT = 100 +export const MAX_KB_SOURCE_SIZE_BYTES = 25 * 1024 * 1024 +export const MAX_KB_TOTAL_SIZE_BYTES = 500 * 1024 * 1024 + +type IngestKBResourceInputBase = JsonObject & { + resourceId: string + kbId: string + title: string + ingestionAttemptId: string + resourceVersion: number +} + +export type IngestKBResourceInput = IngestKBResourceInputBase & + ( + | { + type: 'BLOB' + blobName: string + containerName: string + mimeType: string + sizeBytes: number + } + | { + type: 'URL' + sourceUrl: string + } + ) + +export type DeleteKBResourceInput = JsonObject & { + resourceId: string + kbId: string + deletionAttemptId: string + resourceVersion: number +} + +export type BuildKBGraphInput = JsonObject & { + buildId: string +} + // Shared contract for Hatchet task handler injections. export interface HatchetHandlers { handleSendTeamsNotification: ( @@ -94,6 +133,15 @@ export interface HatchetHandlers { // Contract for the tasks that are passed into the GraphQL context. export interface PreparedHatchetTasks { + ingestKBResource: TaskWorkflowDeclaration< + IngestKBResourceInput, + { success: boolean } + > + deleteKBResource: TaskWorkflowDeclaration< + DeleteKBResourceInput, + { success: boolean } + > + buildKBGraph: TaskWorkflowDeclaration createAuditLogEntry: TaskWorkflowDeclaration< { message: Record & { diff --git a/packages/types/src/index.ts b/packages/types/src/index.ts index 5f0aedeee0..3ab03b205e 100644 --- a/packages/types/src/index.ts +++ b/packages/types/src/index.ts @@ -16,6 +16,9 @@ import type { export * from './assessmentReport.js' export * from './hatchet.js' +// ----- KNOWLEDGE GRAPH TYPES ----- +export * from './knowledgeGraph.js' + // ----- ACTIVITY LOG TYPES ----- // #region export enum ActivityLogModificationFieldType { @@ -1055,5 +1058,6 @@ export enum PointCorrectionType { // #endregion export * from './chatContext.js' +export * from './knowledgeGraph.js' export * from './manageAssistant.js' export * from './studentMcp.js' diff --git a/packages/types/src/knowledgeGraph.ts b/packages/types/src/knowledgeGraph.ts new file mode 100644 index 0000000000..9d0c538684 --- /dev/null +++ b/packages/types/src/knowledgeGraph.ts @@ -0,0 +1,35 @@ +export type KnowledgeGraphSourceReference = { + resourceId: string + title: string + reference?: string +} + +export type KnowledgeGraphNode = { + id: string + labels: string[] + kind: string + displayLabel: string + summary?: string + content?: string + degree: number + sourceReferences: KnowledgeGraphSourceReference[] +} + +export type KnowledgeGraphEdge = { + id: string + source: string + target: string + type: string + label: string + properties: Record +} + +export type KnowledgeGraphResponse = { + kbId: string + buildId: string + // stale graphs keep serving; only lecturer-facing views surface the label + isStale: boolean + nodes: KnowledgeGraphNode[] + edges: KnowledgeGraphEdge[] + truncated: boolean +} diff --git a/packages/util/package.json b/packages/util/package.json index 5235ad6a6e..1ed495fea2 100644 --- a/packages/util/package.json +++ b/packages/util/package.json @@ -57,6 +57,10 @@ "./client-auth": { "types": "./dist/clientAuth.d.ts", "default": "./dist/clientAuth.js" + }, + "./public-url": { + "types": "./dist/publicUrl.d.ts", + "default": "./dist/publicUrl.js" } }, "type": "module" diff --git a/packages/util/rollup.config.js b/packages/util/rollup.config.js index f077b64495..3b2ce6b965 100644 --- a/packages/util/rollup.config.js +++ b/packages/util/rollup.config.js @@ -5,7 +5,12 @@ import { defineConfig } from 'rollup' const config = defineConfig([ { // Main build configuration - input: ['src/index.ts', 'src/auth.ts', 'src/clientAuth.ts'], + input: [ + 'src/index.ts', + 'src/auth.ts', + 'src/clientAuth.ts', + 'src/publicUrl.ts', + ], output: { dir: 'dist', format: 'esm', diff --git a/packages/util/src/blobStorage.ts b/packages/util/src/blobStorage.ts new file mode 100644 index 0000000000..06308e5b25 --- /dev/null +++ b/packages/util/src/blobStorage.ts @@ -0,0 +1,27 @@ +export function getBlobStorageAccountUrl( + accountName: string, + configuredAccountUrl?: string +) { + const accountUrl = + configuredAccountUrl?.trim() || + `https://${accountName.trim()}.blob.core.windows.net` + + let parsedUrl: URL + try { + parsedUrl = new URL(accountUrl) + } catch { + throw new Error('Blob storage account URL is invalid') + } + + if (!['http:', 'https:'].includes(parsedUrl.protocol)) { + throw new Error('Blob storage account URL is invalid') + } + + // Trimmed without a regex: a backtracking `\/+$` on a configured value is a + // polynomial-time pattern, and a plain scan is both linear and clearer. + let end = accountUrl.length + while (end > 0 && accountUrl.charCodeAt(end - 1) === 47) { + end -= 1 + } + return accountUrl.slice(0, end) +} diff --git a/packages/util/src/index.ts b/packages/util/src/index.ts index 2748df7ef6..0254691112 100644 --- a/packages/util/src/index.ts +++ b/packages/util/src/index.ts @@ -28,5 +28,11 @@ export * from './clientAuth.js' // export everything from the crypto module export * from './crypto.js' +// export everything from the KB webhook module +export * from './kbWebhook.js' + +// export Blob Storage endpoint resolution +export * from './blobStorage.js' + // export everything from the identity module export * from './identity.js' diff --git a/packages/util/src/kbWebhook.ts b/packages/util/src/kbWebhook.ts new file mode 100644 index 0000000000..3d0177ed78 --- /dev/null +++ b/packages/util/src/kbWebhook.ts @@ -0,0 +1,45 @@ +import { createHmac } from 'node:crypto' + +export function createKBIngestionWebhookSignature({ + rawBody, + secret, + timestamp, +}: { + rawBody: Buffer + secret: string + timestamp: number | string +}) { + const timestampHeader = String(timestamp) + return createHmac('sha256', secret) + .update( + Buffer.concat([Buffer.from(`${timestampHeader}.`, 'utf8'), rawBody]) + ) + .digest('hex') +} + +export function signKBIngestionWebhook({ + eventId, + eventType, + rawBody, + secret, + timestamp, +}: { + eventId: string + eventType: string + rawBody: Buffer + secret: string + timestamp: number | string +}) { + const timestampHeader = String(timestamp) + + return { + 'x-ingestion-event-id': eventId, + 'x-ingestion-event-type': eventType, + 'x-ingestion-timestamp': timestampHeader, + 'x-ingestion-signature': createKBIngestionWebhookSignature({ + rawBody, + secret, + timestamp: timestampHeader, + }), + } +} diff --git a/packages/util/src/publicUrl.ts b/packages/util/src/publicUrl.ts new file mode 100644 index 0000000000..229386e679 --- /dev/null +++ b/packages/util/src/publicUrl.ts @@ -0,0 +1,92 @@ +import { BlockList, isIP } from 'node:net' + +const BLOCKED_IPV4_ADDRESSES = new BlockList() + +for (const [network, prefix] of [ + ['0.0.0.0', 8], + ['10.0.0.0', 8], + ['100.64.0.0', 10], + ['127.0.0.0', 8], + ['169.254.0.0', 16], + ['172.16.0.0', 12], + ['192.0.0.0', 24], + ['192.0.2.0', 24], + ['192.88.99.0', 24], + ['192.168.0.0', 16], + ['198.18.0.0', 15], + ['198.51.100.0', 24], + ['203.0.113.0', 24], + ['224.0.0.0', 4], + ['240.0.0.0', 4], +] as const) { + BLOCKED_IPV4_ADDRESSES.addSubnet(network, prefix, 'ipv4') +} + +const BLOCKED_HOSTNAME_SUFFIXES = [ + '.home.arpa', + '.internal', + '.invalid', + '.local', + '.localhost', + '.onion', + '.test', +] +const BLOCKED_HOSTNAMES = new Set( + BLOCKED_HOSTNAME_SUFFIXES.map((suffix) => suffix.slice(1)) +) +const SECRET_QUERY_PARAMETERS = new Set([ + 'access_token', + 'api_key', + 'code', + 'key', + 'sas', + 'sig', + 'signature', + 'token', +]) + +export function isPublicIPv4Address(value: string): boolean { + return isIP(value) === 4 && !BLOCKED_IPV4_ADDRESSES.check(value, 'ipv4') +} + +export function normalizePublicHttpUrl(value: string): string { + let parsedUrl: URL + try { + parsedUrl = new URL(value.trim()) + } catch { + throw new Error('URL is invalid') + } + + if ( + (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') || + parsedUrl.username || + parsedUrl.password || + parsedUrl.hash || + (parsedUrl.port !== '' && + parsedUrl.port !== '80' && + parsedUrl.port !== '443') || + [...parsedUrl.searchParams.keys()].some((key) => + SECRET_QUERY_PARAMETERS.has(key.toLowerCase()) + ) + ) { + throw new Error('URL is invalid') + } + + const hostname = parsedUrl.hostname + .replace(/^\[(.*)\]$/, '$1') + .replace(/\.$/, '') + .toLowerCase() + const ipVersion = isIP(hostname) + if ( + !hostname || + (ipVersion === 0 && !hostname.includes('.')) || + BLOCKED_HOSTNAMES.has(hostname) || + BLOCKED_HOSTNAME_SUFFIXES.some((suffix) => hostname.endsWith(suffix)) || + (ipVersion === 4 && !isPublicIPv4Address(hostname)) || + ipVersion === 6 + ) { + throw new Error('URL is invalid') + } + + return parsedUrl.toString() +} diff --git a/packages/util/test/blobStorage.test.ts b/packages/util/test/blobStorage.test.ts new file mode 100644 index 0000000000..ca05ceb057 --- /dev/null +++ b/packages/util/test/blobStorage.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from 'vitest' +import { getBlobStorageAccountUrl } from '../src/blobStorage.js' + +describe('getBlobStorageAccountUrl', () => { + it('uses the Azure account endpoint by default', () => { + expect(getBlobStorageAccountUrl('klicker')).toBe( + 'https://klicker.blob.core.windows.net' + ) + }) + + it('accepts and normalizes an emulator endpoint', () => { + expect( + getBlobStorageAccountUrl( + 'devstoreaccount1', + ' https://blob.klicker.localhost/devstoreaccount1/ ' + ) + ).toBe('https://blob.klicker.localhost/devstoreaccount1') + }) + + it.each([ + 'not a URL', + 'file:///tmp/blob', + ])('rejects an invalid account URL: %s', (accountUrl) => { + expect(() => getBlobStorageAccountUrl('klicker', accountUrl)).toThrow( + 'Blob storage account URL is invalid' + ) + }) +}) diff --git a/packages/util/test/kbWebhook.test.ts b/packages/util/test/kbWebhook.test.ts new file mode 100644 index 0000000000..d96d82b4db --- /dev/null +++ b/packages/util/test/kbWebhook.test.ts @@ -0,0 +1,52 @@ +import { createHmac } from 'node:crypto' +import { describe, expect, it } from 'vitest' +import { signKBIngestionWebhook } from '../src/kbWebhook.js' + +describe('KB ingestion webhook signing', () => { + it('returns the canonical envelope headers and signs exact raw bytes', () => { + const rawBody = Buffer.from('{"eventId":"event-id"}') + + expect( + signKBIngestionWebhook({ + eventId: 'event-id', + eventType: 'resource.processing_started', + rawBody, + secret: 'secret', + timestamp: 1_721_488_400, + }) + ).toEqual({ + 'x-ingestion-event-id': 'event-id', + 'x-ingestion-event-type': 'resource.processing_started', + 'x-ingestion-timestamp': '1721488400', + 'x-ingestion-signature': createHmac('sha256', 'secret') + .update(Buffer.concat([Buffer.from('1721488400.'), rawBody])) + .digest('hex'), + }) + }) + + it('produces different signatures for equivalent JSON with different raw bytes', () => { + const compactBody = Buffer.from('{"eventId":"event-id"}') + const spacedBody = Buffer.from('{ "eventId": "event-id" }') + + expect(JSON.parse(compactBody.toString('utf8'))).toEqual( + JSON.parse(spacedBody.toString('utf8')) + ) + + const compactSignature = signKBIngestionWebhook({ + eventId: 'event-id', + eventType: 'resource.processing_started', + rawBody: compactBody, + secret: 'secret', + timestamp: 1_721_488_400, + })['x-ingestion-signature'] + const spacedSignature = signKBIngestionWebhook({ + eventId: 'event-id', + eventType: 'resource.processing_started', + rawBody: spacedBody, + secret: 'secret', + timestamp: 1_721_488_400, + })['x-ingestion-signature'] + + expect(compactSignature).not.toBe(spacedSignature) + }) +}) diff --git a/packages/util/test/publicUrl.test.ts b/packages/util/test/publicUrl.test.ts new file mode 100644 index 0000000000..af2a278e2e --- /dev/null +++ b/packages/util/test/publicUrl.test.ts @@ -0,0 +1,59 @@ +import { describe, expect, it } from 'vitest' +import { + isPublicIPv4Address, + normalizePublicHttpUrl, +} from '../src/publicUrl.js' + +describe('normalizePublicHttpUrl', () => { + it('normalizes public HTTP and HTTPS URLs', () => { + expect(normalizePublicHttpUrl(' https://example.com/notes?q=1 ')).toBe( + 'https://example.com/notes?q=1' + ) + expect(normalizePublicHttpUrl('http://8.8.8.8/resource')).toBe( + 'http://8.8.8.8/resource' + ) + }) + + it.each([ + 'ftp://example.com/file', + 'https://user:password@example.com/file', + 'https://example.com/file#section', + 'https://example.com/file?token=sensitive', + 'https://example.com/file?API_KEY=sensitive', + 'https://example.com:8443/file', + 'http://localhost:3000/admin', + 'http://metadata/admin', + 'http://service.internal/admin', + 'http://127.0.0.1/admin', + 'http://2130706433/admin', + 'http://169.254.169.254/latest/meta-data', + 'http://10.0.0.1/admin', + 'http://172.16.0.1/admin', + 'http://192.168.0.1/admin', + 'http://[::1]/admin', + 'https://content.example.test/file', + ])('rejects non-public URL %s', (url) => { + expect(() => normalizePublicHttpUrl(url)).toThrow('URL is invalid') + }) +}) + +describe('isPublicIPv4Address', () => { + it.each([ + '8.8.8.8', + '1.1.1.1', + ])('accepts public IPv4 address %s', (address) => { + expect(isPublicIPv4Address(address)).toBe(true) + }) + + it.each([ + '127.0.0.1', + '169.254.169.254', + '10.0.0.1', + '172.16.0.1', + '192.168.0.1', + '::1', + 'not-an-address', + ])('rejects non-public IPv4 address %s', (address) => { + expect(isPublicIPv4Address(address)).toBe(false) + }) +}) diff --git a/playwright/tests/Y-manage-assistant.spec.ts b/playwright/tests/Y-manage-assistant.spec.ts index 49a17ce79a..2fc74cf4f2 100644 --- a/playwright/tests/Y-manage-assistant.spec.ts +++ b/playwright/tests/Y-manage-assistant.spec.ts @@ -316,6 +316,8 @@ test.describe('Manage Assistant — Per-surface suggestions', () => { }) => { await mockManageChatStream(page) const assistant = await openManageAssistantWidget(page) + // The conversation starters render in their own section next to the + // welcome message, not inside it, so scope surface assertions here. const suggestions = assistant.getByTestId('chat-welcome-suggestions') for (const text of [ diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 60a5cdd7ec..80a1a9041c 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -361,13 +361,13 @@ importers: version: 4.0.30(zod@3.25.76) '@assistant-ui/react': specifier: 0.15.1 - version: 0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)) + version: 0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(use-sync-external-store@1.6.0(react@19.2.7)) '@assistant-ui/react-ai-sdk': specifier: 1.3.41 - version: 1.3.41(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7)(zod@3.25.76)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) + version: 1.3.41(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zod@3.25.76)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) '@assistant-ui/react-markdown': specifier: 0.14.8 - version: 0.14.8(@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) + version: 0.14.8(@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(use-sync-external-store@1.6.0(react@19.2.7)))(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) '@fortawesome/fontawesome-svg-core': specifier: 6.7.2 version: 6.7.2 @@ -383,6 +383,9 @@ importers: '@klicker-uzh/i18n': specifier: workspace:* version: link:../../packages/i18n + '@klicker-uzh/knowledge-graph': + specifier: workspace:* + version: link:../../packages/knowledge-graph '@klicker-uzh/markdown': specifier: workspace:* version: link:../../packages/markdown @@ -886,6 +889,9 @@ importers: '@klicker-uzh/i18n': specifier: workspace:* version: link:../../packages/i18n + '@klicker-uzh/kb-management': + specifier: workspace:* + version: link:../../packages/kb-management '@klicker-uzh/markdown': specifier: workspace:* version: link:../../packages/markdown @@ -1789,6 +1795,9 @@ importers: '@klicker-uzh/grading': specifier: workspace:* version: link:../grading + '@klicker-uzh/knowledge-graph': + specifier: workspace:* + version: link:../knowledge-graph '@klicker-uzh/prisma': specifier: workspace:* version: link:../prisma @@ -1982,15 +1991,24 @@ importers: packages/hatchet: dependencies: + '@azure/storage-blob': + specifier: 12.25.0 + version: 12.25.0 '@hatchet-dev/typescript-sdk': specifier: 1.9.4 version: 1.9.4 + '@klicker-uzh/knowledge-graph': + specifier: workspace:* + version: link:../knowledge-graph '@klicker-uzh/prisma': specifier: workspace:* version: link:../prisma '@klicker-uzh/types': specifier: workspace:* version: link:../types + '@klicker-uzh/util': + specifier: workspace:* + version: link:../util devDependencies: '@parcel/watcher': specifier: ~2.4.1 @@ -2028,17 +2046,112 @@ importers: typescript: specifier: ~6.0.3 version: 6.0.3 + vitest: + specifier: ~3.2.4 + version: 3.2.4(@types/debug@4.1.12)(@types/node@24.12.4)(jiti@2.7.0)(lightningcss@1.30.1)(terser@5.49.0)(tsx@4.19.4)(yaml@2.9.0) packages/i18n: dependencies: + next: + specifier: ^16.2.10 + version: 16.2.10(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) + next-intl: + specifier: ^4.13.0 + version: 4.13.0(next@16.2.10(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7))(react@19.2.7)(typescript@6.0.3) + devDependencies: + typescript: + specifier: ~6.0.3 + version: 6.0.3 + + packages/kb-management: + dependencies: + '@apollo/client': + specifier: ^3.13.8 + version: 3.13.8(@types/react@19.2.17)(graphql-ws@6.0.6(graphql@16.11.0)(ws@8.18.3))(graphql@16.11.0)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) + '@azure/storage-blob': + specifier: ^12.25.0 + version: 12.25.0 + '@fortawesome/free-solid-svg-icons': + specifier: ^6.7.2 + version: 6.7.2 + '@fortawesome/react-fontawesome': + specifier: ^0.2.2 + version: 0.2.2(@fortawesome/fontawesome-svg-core@6.7.2)(react@19.2.7) + '@klicker-uzh/graphql': + specifier: workspace:* + version: link:../graphql + '@klicker-uzh/i18n': + specifier: workspace:* + version: link:../i18n + '@klicker-uzh/shared-components': + specifier: workspace:* + version: link:../shared-components + '@klicker-uzh/types': + specifier: workspace:* + version: link:../types + '@uzh-bf/design-system': + specifier: 4.1.8 + version: 4.1.8(@fortawesome/fontawesome-svg-core@6.7.2)(@fortawesome/free-regular-svg-icons@6.7.2)(@fortawesome/free-solid-svg-icons@6.7.2)(@fortawesome/react-fontawesome@0.2.2(@fortawesome/fontawesome-svg-core@6.7.2)(react@19.2.7))(@tailwindcss/postcss@4.1.12)(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(class-variance-authority@0.7.1)(clsx@2.1.1)(dayjs@1.11.20)(formik@2.4.9(@types/react@19.2.17)(react@19.2.7))(lucide-react@0.522.0(react@19.2.7))(postcss@8.5.14)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(tailwind-merge@3.3.1)(tailwindcss-animate@1.0.7(tailwindcss@4.1.12))(tailwindcss-radix@4.0.2(tailwindcss@4.1.12))(tailwindcss@4.1.12)(tw-animate-css@1.3.7)(yup@1.6.1) + next: + specifier: ^16.2.10 + version: 16.2.10(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) next-intl: specifier: ^4.13.0 - version: 4.13.0(next@15.5.18(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7))(react@19.2.7)(typescript@6.0.3) + version: 4.13.0(next@16.2.10(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7))(react@19.2.7)(typescript@6.0.3) + react: + specifier: ^19.2.7 + version: 19.2.7 + react-dropzone: + specifier: ^14.2.9 + version: 14.2.9(react@19.2.7) devDependencies: + '@types/react': + specifier: ^19.2.17 + version: 19.2.17 typescript: specifier: ~6.0.3 version: 6.0.3 + packages/knowledge-graph: + dependencies: + '@klicker-uzh/prisma': + specifier: workspace:* + version: link:../prisma + '@klicker-uzh/types': + specifier: workspace:* + version: link:../types + falkordb: + specifier: 6.6.2 + version: 6.6.2(@opentelemetry/api@1.9.1) + devDependencies: + '@parcel/watcher': + specifier: ~2.4.1 + version: 2.4.1 + '@rollup/plugin-node-resolve': + specifier: ~15.3.1 + version: 15.3.1(rollup@4.34.9) + '@rollup/plugin-typescript': + specifier: ~12.1.4 + version: 12.1.4(rollup@4.34.9)(tslib@2.8.1)(typescript@6.0.3) + '@types/node': + specifier: ^24.10.1 + version: 24.12.4 + cross-env: + specifier: ~7.0.3 + version: 7.0.3 + npm-run-all: + specifier: ~4.1.5 + version: 4.1.5 + rollup: + specifier: ~4.34.9 + version: 4.34.9 + typescript: + specifier: ~6.0.3 + version: 6.0.3 + vitest: + specifier: ~3.2.4 + version: 3.2.4(@types/debug@4.1.12)(@types/node@24.12.4)(jiti@2.7.0)(lightningcss@1.30.1)(terser@5.49.0)(tsx@4.19.4)(yaml@2.9.0) + packages/markdown: dependencies: '@fortawesome/fontawesome-svg-core': @@ -2331,6 +2444,9 @@ importers: '@uzh-bf/design-system': specifier: 4.1.8 version: 4.1.8(@fortawesome/fontawesome-svg-core@6.7.2)(@fortawesome/free-regular-svg-icons@6.7.2)(@fortawesome/free-solid-svg-icons@6.7.2)(@fortawesome/react-fontawesome@0.2.2(@fortawesome/fontawesome-svg-core@6.7.2)(react@19.2.7))(@tailwindcss/postcss@4.1.12)(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(class-variance-authority@0.7.1)(clsx@2.1.1)(dayjs@1.11.20)(formik@2.4.9(@types/react@19.2.17)(react@19.2.7))(lucide-react@0.522.0(react@19.2.7))(postcss@8.5.14)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(tailwind-merge@3.3.1)(tailwindcss-animate@1.0.7(tailwindcss@4.1.12))(tailwindcss-radix@4.0.2(tailwindcss@4.1.12))(tailwindcss@4.1.12)(tw-animate-css@1.3.7)(yup@1.6.1) + cytoscape: + specifier: 3.34.0 + version: 3.34.0 dayjs: specifier: ^1.11.20 version: 1.11.20 @@ -6064,6 +6180,10 @@ packages: '@js-sdsl/ordered-map@4.4.2': resolution: {integrity: sha512-iUKgm52T8HOE/makSxjqoWhe95ZJA1/G1sYsGev2JDKUSS14KAgg1LHb+Ba+IPow0xflbnSkOsZcO08C7w1gYw==} + '@js-temporal/polyfill@0.5.1': + resolution: {integrity: sha512-hloP58zRVCRSpgDxmqCWJNlizAlUgJFqG2ypq79DCvyv9tHjRYMDOcPFjzfl/A1/YxDvRCZz8wvZvmapQnKwFQ==} + engines: {node: '>=12'} + '@jsep-plugin/assignment@1.3.0': resolution: {integrity: sha512-VVgV+CXrhbMI3aSusQyclHkenWSAm95WaiKrMxRFam3JSUiIaQjoMIw2sEs/OX4XifnqeQUN4DYbJjlA8EfktQ==} engines: {node: '>= 10.16.0'} @@ -8779,6 +8899,42 @@ packages: react-native: optional: true + '@redis/bloom@5.12.1': + resolution: {integrity: sha512-PUUfv+ms7jgPSBVoo/DN4AkPHj4D5TZSd6SbJX7egzBplkYUcKmHRE8RKia7UtZ8bSQbLguLvxVO+asKtQfZWA==} + engines: {node: '>= 18.19.0'} + peerDependencies: + '@redis/client': ^5.12.1 + + '@redis/client@5.12.1': + resolution: {integrity: sha512-7aPGWeqA3uFm43o19umzdl16CEjK/JQGtSXVPevplTaOU3VJA/rseBC1QvYUz9lLDIMBimc4SW/zrW4S89BaCA==} + engines: {node: '>= 18.19.0'} + peerDependencies: + '@node-rs/xxhash': ^1.1.0 + '@opentelemetry/api': '>=1 <2' + peerDependenciesMeta: + '@node-rs/xxhash': + optional: true + '@opentelemetry/api': + optional: true + + '@redis/json@5.12.1': + resolution: {integrity: sha512-eOze75esLve4vfqDel7aMX08CNaiLLQS2fV8mpRN9NxPe1rVR4vQyYiW/OgtGUysF6QOr9ANhfxABKNOJfXdKg==} + engines: {node: '>= 18.19.0'} + peerDependencies: + '@redis/client': ^5.12.1 + + '@redis/search@5.12.1': + resolution: {integrity: sha512-ItlxbxC9cKI6IU1TLWoczwJCRb6TdmkEpWv05UrPawqaAnWGRu3rcIqsc5vN483T2fSociuyV1UkWIL5I4//2w==} + engines: {node: '>= 18.19.0'} + peerDependencies: + '@redis/client': ^5.12.1 + + '@redis/time-series@5.12.1': + resolution: {integrity: sha512-c6JL6E3EcZJuNqKFz+KM+l9l5mpcQiKvTwgA3blt5glWJ8hjDk0yeHN3beE/MpqYIQ8UEX44ItQzgkE/gCBELQ==} + engines: {node: '>= 18.19.0'} + peerDependencies: + '@redis/client': ^5.12.1 + '@redocly/ajv@8.11.2': resolution: {integrity: sha512-io1JpnwtIcvojV7QKDUSIuMN/ikdOUd1ReEnUnMKGfDVridQZ31J0MmIuqwuRjWDZfmvr+Q0MqCcfHM2gTivOg==} @@ -12141,6 +12297,10 @@ packages: csv-parse@6.1.0: resolution: {integrity: sha512-CEE+jwpgLn+MmtCpVcPtiCZpVtB6Z2OKPTr34pycYYoL7sxdOkXDdQ4lRiw6ioC0q6BLqhc6cKweCVvral8yhw==} + cytoscape@3.34.0: + resolution: {integrity: sha512-62rNSrioXw93uliKFBwjukeQyeWwH2PqDrTac31r2P6464u3AUvTk0xS4LVvT251g7IgkFunrI48ZEZGjywSOg==} + engines: {node: '>=0.10'} + d3-array@3.2.4: resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==} engines: {node: '>=12'} @@ -13105,6 +13265,10 @@ packages: extend@3.0.2: resolution: {integrity: sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==} + falkordb@6.6.2: + resolution: {integrity: sha512-f1LUKA7DGXNUbdRQlB22UQdwRO3WHiXrAzKLNsKyvjLSVH3MTl2qcA+6w2juosPBV6sJiX9okBwbXSXlIHiXFA==} + engines: {node: '>=20.0.0'} + fast-check@3.23.2: resolution: {integrity: sha512-h5+1OzzfCC3Ef7VbtKdcv7zsstUQwUDlYpUTvjeUsJAssPgLn7QzbboPtL5ro04Mq0rPOsMzl7q5hIbRs2wD1A==} engines: {node: '>=8.0.0'} @@ -13493,6 +13657,10 @@ packages: generate-password@1.7.1: resolution: {integrity: sha512-9bVYY+16m7W7GczRBDqXE+VVuCX+bWNrfYKC/2p2JkZukFb2sKxT6E3zZ3mJGz7GMe5iRK0A/WawSL3jQfJuNQ==} + generic-pool@3.9.0: + resolution: {integrity: sha512-hymDOu5B53XvN4QT9dBmZxPX4CWhBPPLguTZ9MMFeFa/Kg0xWVfylOVNlJji/E7yTZWFd/q9GO5TxDLq156D7g==} + engines: {node: '>= 4'} + gensync@1.0.0-beta.2: resolution: {integrity: sha512-3hN7NaskYvMDLQY55gnW3NQ+mesEAepTqlg+VEbj7zzqEMBVNhzcGYYeqFo/TlYz6eQiFcp1HcsCZO+nGgS8zg==} engines: {node: '>=6.9.0'} @@ -14725,6 +14893,9 @@ packages: jsbi@3.2.5: resolution: {integrity: sha512-aBE4n43IPvjaddScbvWRA2YlTzKEynHzu7MqOyTipdHucf/VxS63ViCjxYRg86M8Rxwbt/GfzHl1kKERkt45fQ==} + jsbi@4.3.2: + resolution: {integrity: sha512-9fqMSQbhJykSeii05nxKl4m6Eqn2P6rOlYiS+C5Dr/HPIU/7yZxu5qzbs40tgaFORiw2Amd0mirjxatXYMkIew==} + jsc-safe-url@0.2.4: resolution: {integrity: sha512-0wM3YBWtYePOjfyXQH5MWQ8H7sdk5EXSwZvmSLKk2RboVQ2Bu239jycHDz5J/8Blf3K0Qnoy2b6xD+z10MFB+Q==} @@ -17900,6 +18071,10 @@ packages: resolution: {integrity: sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==} engines: {node: '>=4'} + redis@5.12.1: + resolution: {integrity: sha512-LDsoVvb/CpoV9EN3FXvgvSHNJWuCIzl9MiO3ppOevuGLpSGJhwfQjpEwfFJcQvNSddHADDdZaWx0HnmMxRXG7g==} + engines: {node: '>= 18.19.0'} + redoc@2.5.0: resolution: {integrity: sha512-NpYsOZ1PD9qFdjbLVBZJWptqE+4Y6TkUuvEOqPUmoH7AKOmPcE+hYjotLxQNTqVoWL4z0T2uxILmcc8JGDci+Q==} engines: {node: '>=6.9', npm: '>=3.0.0'} @@ -21011,45 +21186,45 @@ snapshots: transitivePeerDependencies: - encoding - '@assistant-ui/core@0.2.21(@assistant-ui/store@0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37)(react@19.2.7)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': + '@assistant-ui/core@0.2.21(@assistant-ui/store@0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': dependencies: '@assistant-ui/store': 0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7) '@assistant-ui/tap': 0.9.7(@types/react@19.2.17)(react@19.2.7) - assistant-stream: 0.3.30 + assistant-stream: 0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)) nanoid: 6.0.0 optionalDependencies: '@types/react': 19.2.17 - assistant-cloud: 0.1.37 + assistant-cloud: 0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)) react: 19.2.7 zustand: 5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)) transitivePeerDependencies: - ioredis - redis - '@assistant-ui/core@0.3.1(@assistant-ui/store@0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37)(react@19.2.7)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': + '@assistant-ui/core@0.3.1(@assistant-ui/store@0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': dependencies: '@assistant-ui/store': 0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7) '@assistant-ui/tap': 0.9.7(@types/react@19.2.17)(react@19.2.7) - assistant-stream: 0.3.30 + assistant-stream: 0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)) nanoid: 6.0.0 optionalDependencies: '@types/react': 19.2.17 - assistant-cloud: 0.1.37 + assistant-cloud: 0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)) react: 19.2.7 zustand: 5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)) transitivePeerDependencies: - ioredis - redis - '@assistant-ui/react-ai-sdk@1.3.41(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7)(zod@3.25.76)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': + '@assistant-ui/react-ai-sdk@1.3.41(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zod@3.25.76)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))': dependencies: '@ai-sdk/mcp': 2.0.25(zod@3.25.76) '@ai-sdk/react': 4.0.37(react@19.2.7)(zod@3.25.76) - '@assistant-ui/core': 0.2.21(@assistant-ui/store@0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37)(react@19.2.7)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) + '@assistant-ui/core': 0.2.21(@assistant-ui/store@0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) '@assistant-ui/store': 0.2.22(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7) ai: 7.0.37(zod@3.25.76) - assistant-cloud: 0.1.37 - assistant-stream: 0.3.30 + assistant-cloud: 0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)) + assistant-stream: 0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)) react: 19.2.7 optionalDependencies: '@types/react': 19.2.17 @@ -21060,9 +21235,9 @@ snapshots: - zod - zustand - '@assistant-ui/react-markdown@0.14.8(@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)))(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)': + '@assistant-ui/react-markdown@0.14.8(@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(use-sync-external-store@1.6.0(react@19.2.7)))(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)': dependencies: - '@assistant-ui/react': 0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7)) + '@assistant-ui/react': 0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(use-sync-external-store@1.6.0(react@19.2.7)) '@radix-ui/react-primitive': 2.1.10(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) '@radix-ui/react-use-callback-ref': 1.1.4(@types/react@19.2.17)(react@19.2.7) classnames: 2.5.1 @@ -21075,9 +21250,9 @@ snapshots: - react-dom - supports-color - '@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))': + '@assistant-ui/react@0.15.1(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(immer@10.1.1)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(use-sync-external-store@1.6.0(react@19.2.7))': dependencies: - '@assistant-ui/core': 0.3.1(@assistant-ui/store@0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37)(react@19.2.7)(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) + '@assistant-ui/core': 0.3.1(@assistant-ui/store@0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7))(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(assistant-cloud@0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)))(react@19.2.7)(redis@5.12.1(@opentelemetry/api@1.9.1))(zustand@5.0.14(@types/react@19.2.17)(immer@10.1.1)(react@19.2.7)(use-sync-external-store@1.6.0(react@19.2.7))) '@assistant-ui/store': 0.3.1(@assistant-ui/tap@0.9.7(@types/react@19.2.17)(react@19.2.7))(@types/react@19.2.17)(react@19.2.7) '@assistant-ui/tap': 0.9.7(@types/react@19.2.17)(react@19.2.7) '@radix-ui/primitive': 1.1.7 @@ -21088,8 +21263,8 @@ snapshots: '@radix-ui/react-use-callback-ref': 1.1.4(@types/react@19.2.17)(react@19.2.7) '@radix-ui/react-use-controllable-state': 1.2.6(@types/react@19.2.17)(react@19.2.7) '@radix-ui/react-use-escape-keydown': 1.1.5(@types/react@19.2.17)(react@19.2.7) - assistant-cloud: 0.1.37 - assistant-stream: 0.3.30 + assistant-cloud: 0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)) + assistant-stream: 0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)) nanoid: 6.0.0 radix-ui: 1.6.7(@types/react-dom@19.2.3(@types/react@19.2.17))(@types/react@19.2.17)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) react: 19.2.7 @@ -24776,7 +24951,7 @@ snapshots: https-proxy-agent: 7.0.6 jose: 5.9.4 js-yaml: 4.1.1 - lodash: 4.18.1 + lodash: 4.17.21 scuid: 1.1.0 tslib: 2.8.1 yaml-ast-parser: 0.0.43 @@ -25385,6 +25560,10 @@ snapshots: '@js-sdsl/ordered-map@4.4.2': {} + '@js-temporal/polyfill@0.5.1': + dependencies: + jsbi: 4.3.2 + '@jsep-plugin/assignment@1.3.0(jsep@1.4.0)': dependencies: jsep: 1.4.0 @@ -28211,6 +28390,28 @@ snapshots: - '@types/react' - immer + '@redis/bloom@5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1))': + dependencies: + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + + '@redis/client@5.12.1(@opentelemetry/api@1.9.1)': + dependencies: + cluster-key-slot: 1.1.2 + optionalDependencies: + '@opentelemetry/api': 1.9.1 + + '@redis/json@5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1))': + dependencies: + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + + '@redis/search@5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1))': + dependencies: + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + + '@redis/time-series@5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1))': + dependencies: + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + '@redocly/ajv@8.11.2': dependencies: fast-deep-equal: 3.1.3 @@ -30702,18 +30903,20 @@ snapshots: assertion-error@2.0.1: {} - assistant-cloud@0.1.37: + assistant-cloud@0.1.37(redis@5.12.1(@opentelemetry/api@1.9.1)): dependencies: - assistant-stream: 0.3.30 + assistant-stream: 0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)) transitivePeerDependencies: - ioredis - redis - assistant-stream@0.3.30: + assistant-stream@0.3.30(redis@5.12.1(@opentelemetry/api@1.9.1)): dependencies: '@standard-schema/spec': 1.1.0 nanoid: 6.0.0 secure-json-parse: 4.1.0 + optionalDependencies: + redis: 5.12.1(@opentelemetry/api@1.9.1) ast-types-flow@0.0.8: {} @@ -30732,7 +30935,7 @@ snapshots: async@2.6.4: dependencies: - lodash: 4.18.1 + lodash: 4.17.21 async@3.2.6: {} @@ -31670,7 +31873,7 @@ snapshots: git-raw-commits: 2.0.11 git-remote-origin-url: 2.0.0 git-semver-tags: 4.1.1 - lodash: 4.18.1 + lodash: 4.17.21 normalize-package-data: 3.0.3 q: 1.5.1 read-pkg: 3.0.0 @@ -31706,7 +31909,7 @@ snapshots: dateformat: 3.0.3 handlebars: 4.7.9 json-stringify-safe: 5.0.1 - lodash: 4.18.1 + lodash: 4.17.21 meow: 8.1.2 semver: 6.3.1 split: 1.0.1 @@ -31735,7 +31938,7 @@ snapshots: dependencies: JSONStream: 1.3.5 is-text-path: 1.0.1 - lodash: 4.18.1 + lodash: 4.17.21 meow: 8.1.2 split2: 3.2.2 through2: 4.0.2 @@ -32078,6 +32281,8 @@ snapshots: csv-parse@6.1.0: {} + cytoscape@3.34.0: {} + d3-array@3.2.4: dependencies: internmap: 2.0.3 @@ -33418,6 +33623,18 @@ snapshots: extend@3.0.2: {} + falkordb@6.6.2(@opentelemetry/api@1.9.1): + dependencies: + '@js-temporal/polyfill': 0.5.1 + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + cluster-key-slot: 1.1.2 + generic-pool: 3.9.0 + lodash: 4.17.21 + redis: 5.12.1(@opentelemetry/api@1.9.1) + transitivePeerDependencies: + - '@node-rs/xxhash' + - '@opentelemetry/api' + fast-check@3.23.2: dependencies: pure-rand: 6.1.0 @@ -33926,6 +34143,8 @@ snapshots: generate-password@1.7.1: {} + generic-pool@3.9.0: {} + gensync@1.0.0-beta.2: {} get-caller-file@2.0.5: {} @@ -34001,7 +34220,7 @@ snapshots: git-raw-commits@2.0.11: dependencies: dargs: 7.0.0 - lodash: 4.18.1 + lodash: 4.17.21 meow: 8.1.2 split2: 3.2.2 through2: 4.0.2 @@ -34771,7 +34990,7 @@ snapshots: cli-cursor: 3.1.0 cli-width: 3.0.0 figures: 3.2.0 - lodash: 4.18.1 + lodash: 4.17.21 mute-stream: 0.0.8 ora: 5.4.1 run-async: 2.4.1 @@ -35355,6 +35574,8 @@ snapshots: jsbi@3.2.5: {} + jsbi@4.3.2: {} + jsc-safe-url@0.2.4: {} jsep@1.4.0: {} @@ -37057,23 +37278,6 @@ snapshots: next-intl-swc-plugin-extractor@4.13.0: {} - next-intl@4.13.0(next@15.5.18(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7))(react@19.2.7)(typescript@6.0.3): - dependencies: - '@formatjs/intl-localematcher': 0.8.10 - '@parcel/watcher': 2.4.1 - '@swc/core': 1.15.43 - icu-minify: 4.13.0 - negotiator: 1.0.0 - next: 15.5.18(@babel/core@7.24.5)(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7) - next-intl-swc-plugin-extractor: 4.13.0 - po-parser: 2.1.1 - react: 19.2.7 - use-intl: 4.13.0(react@19.2.7) - optionalDependencies: - typescript: 6.0.3 - transitivePeerDependencies: - - '@swc/helpers' - next-intl@4.13.0(next@16.2.10(@babel/core@7.28.3)(@opentelemetry/api@1.9.1)(@playwright/test@1.58.2)(react-dom@19.2.7(react@19.2.7))(react@19.2.7))(react@19.2.7)(typescript@6.0.3): dependencies: '@formatjs/intl-localematcher': 0.8.10 @@ -38773,7 +38977,7 @@ snapshots: pretty-error@4.0.0: dependencies: - lodash: 4.18.1 + lodash: 4.17.21 renderkid: 3.0.0 pretty-format@29.7.0: @@ -39627,6 +39831,17 @@ snapshots: dependencies: redis-errors: 1.2.0 + redis@5.12.1(@opentelemetry/api@1.9.1): + dependencies: + '@redis/bloom': 5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1)) + '@redis/client': 5.12.1(@opentelemetry/api@1.9.1) + '@redis/json': 5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1)) + '@redis/search': 5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1)) + '@redis/time-series': 5.12.1(@redis/client@5.12.1(@opentelemetry/api@1.9.1)) + transitivePeerDependencies: + - '@node-rs/xxhash' + - '@opentelemetry/api' + redoc@2.5.0(core-js@3.45.1)(mobx@6.13.7)(react-dom@19.2.7(react@19.2.7))(react@19.2.7)(styled-components@6.1.19(react-dom@19.2.7(react@19.2.7))(react@19.2.7)): dependencies: '@redocly/openapi-core': 1.34.5 @@ -39917,7 +40132,7 @@ snapshots: css-select: 4.3.0 dom-converter: 0.2.0 htmlparser2: 6.1.0 - lodash: 4.18.1 + lodash: 4.17.21 strip-ansi: 6.0.1 repeat-string@1.6.1: {} @@ -42578,7 +42793,7 @@ snapshots: fast-json-stable-stringify: 2.1.0 fs-extra: 9.1.0 glob: 7.2.3 - lodash: 4.18.1 + lodash: 4.17.21 pretty-bytes: 5.6.0 rollup: 2.79.2 source-map: 0.8.0-beta.0 diff --git a/project/2026-07-15-pr-5174-kb-poc-plan.md b/project/2026-07-15-pr-5174-kb-poc-plan.md new file mode 100644 index 0000000000..211b6e26ef --- /dev/null +++ b/project/2026-07-15-pr-5174-kb-poc-plan.md @@ -0,0 +1,353 @@ +# KB Management POC — Plan + +Caveman form. Junior-executable. Read whole plan before slice 1. + +## Plan Identity + +- Plan: `project/2026-07-15-pr-5174-kb-poc-plan.md` +- Branch: `kb-poc` (worktree `trees/kb-poc`), base + target: `v3-ai` (staging release branch, deploys to STG first) +- PR: [#5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) (draft; carries this plan first, then slices) +- History: + - [PR #5078](https://github.com/uzh-bf/klicker-uzh/pull/5078) — full-scale KB control-plane prototype (`codex/kb-management-ui` → `v3-ai`). Too large for MVP. This POC is the slim mergeable path on the SAME line (`v3-ai`), NOT a replacement of the KB direction. Reference for shapes; do NOT cherry-pick wholesale. + - Review of 5078: `project/2026-07-07_pr5078_kb_review.md` on the 5078 branch — catalog of its bugs/gaps. POC avoids them by design (see Decisions). + - `project/KB_PLAN.md` (merged, `03ca4aaa5`) — older plan, KB UI inside `frontend-manage`. Location intent honored (UI mounts in manage); implementation superseded: UI is a **reusable package**, not app-local components. + +## Goal + +Thin end-to-end tracer for knowledge base management, delivered as a **reusable React package** mounted in `frontend-manage`: + +1. Data model (slim) — reviewable on its own. +2. `@klicker-uzh/kb-management` package (buildless TS src, like `shared-components`), mounted at a route in `frontend-manage`. +3. File upload to Azure Blob Storage (SAS pattern, private containers). +4. "Ingest" button → dispatches a Hatchet task (the trigger end of the ingestion contract). +5. Signed webhook receiver → the real (future) ingestion service calls it back to update resource status. +6. Nice UX: auto-refresh (polling), status badges, empty/loading states. + +The POC builds and independently verifies **both ends of the ingestion contract** (dispatch + status-webhook). The real ingestion pipeline (built separately) fills the middle later and reads blobs via workload identity. + +## Non-Goals (POC) + +- No standalone app. UI is a package mounted in `frontend-manage`. (Reverses the earlier separate-app idea — see Grill Round 2.) +- No simulated ingestion worker. The Hatchet task handler is a documented stub pending the real ingestion service. Status movement is verified by manually calling the signed webhook. +- No blob read-SAS. The future ingestion service reads blobs via Azure workload identity (`rs-workload-identity`), out of POC scope. +- No chat-runtime consumption of KBs. +- No website/snippet/klicker-object resources. Files only. +- No metadata profiles, graph settings, refresh scheduling, chatbot/course links. +- No deploy workflows, Helm chart, image builds (package deploys inside manage's existing image). +- No Playwright/Cypress E2E (data-cy attrs mandatory anyway, tests later). + +## Decisions + +Round 1 (D1-D10) drafted from research; Round 2 rulings (2026-07-15) overrode several. Table shows final state. + +| # | Decision | Why | +|---|---|---| +| D1 | **Reusable package `@klicker-uzh/kb-management`** (not a standalone app), mounted at a route in `frontend-manage` | Round 2 ruling: package importable in other React apps, extractable later. Kills the app tax AND both auth blockers (manage is already authed). | +| D2 | Buildless package: `main`/`types` → `src/index.ts`, add to manage `transpilePackages` | Mirrors `packages/shared-components` (ships raw TS, no rollup/dist). No `check-types.yml` build-list entry needed. | +| D3 | Target `v3-ai` (staging release branch) | Round 2 ruling: deploy to STG first via v3-ai. 5078 is on same line → KB names/shapes converge cheaply. | +| D4 | Model names `KB`, `KBResource` — same as 5078, slim fields | Convergence with 5078 on the shared v3-ai line. | +| D5 | Worker/dispatch never writes status to DB. Status updates ONLY via signed webhook from the ingestion service | Preserves production topology: ingestion is an external service that calls back. Tracer must exercise webhook path. | +| D6 | Webhook receiver = express route in `backend-docker`, handler in `packages/graphql` | Needs Prisma; mirrors 5078 shape (`handleKBIngestionWebhook`), reviewed as sound. | +| D7 | Private blob containers, per-user (`kb-`), blob-scoped write SAS. Read access = workload identity (future service) | 5078/media-library flaw: public-read containers. KB docs are private data. Round 2 confirmed workload-identity read path. | +| D8 | DB row created AFTER upload confirmed + server-side `exists()` check (2 mutations) | Media-library flaw: row-before-upload orphans rows on failed PUT. | +| D9 | Auto-refresh via Apollo `pollInterval`, not subscriptions | Subscriptions plumbed but unused repo-wide (0 `useSubscription` call sites). Polling = proven (`cockpit.tsx:57`). | +| D10 | `Int sizeBytes`, not `BigInt` | 5078 bot finding: BigInt JSON serialization crash. 25MB cap fits Int. | +| D11 | No webhook inbox table (POC) | Idempotency via status-transition guard. Inbox = hardening later. | +| D12 | Upload allowlist: pdf, txt, md, docx, pptx | Round 2 ruling: doc-processing (future ingestion) handles docx/pptx. | +| D13 | Two resource kinds via single-table discriminator: `type KBResourceType { BLOB, URL }`. Blob-only fields nullable; `sourceUrl` for URL kind. Per-kind required fields enforced in resolvers (Prisma has no CHECK constraints). | S1 gate ruling: a resource is an uploaded blob OR a web resource (e.g., Kaltura MediaSpace video) fetched by the ingestion service. Separate models = overkill for POC. Initial status renamed `UPLOADED` -> `ADDED` to fit both kinds. | + +## Grill Round 2 — Open Decision Rulings (2026-07-15) + +User rulings that reshaped Round 1: + +1. **Not a separate app — a reusable package** (other React apps may import it; extractable). → D1, D2. Removes app scaffold, routing, Dockerfile, deploy pipeline, and BOTH auth blockers below. +2. **No simulated ingestion** — real ingestion built separately as it can support it. → Non-goal; S5 dispatches only, handler is a stub. +3. **Ingestion reads blobs via workload identity.** → D7; POC mints no read-SAS. +4. **Base = `v3-ai`** (staging release branch, STG first). → D3; branch rebased onto v3-ai. +5. **Shared-auth edits acceptable** on v3-ai feature branch — but moot: package-in-manage needs none. +6. **Allow docx + pptx** (doc-processing supported). → D12. +7. **(S1 gate, 2026-07-15)** A KB resource can be a website/web resource (e.g., Kaltura MediaSpace videos — ingestion fetches and processes them) or an uploaded blob. → D13; model amended in place before S2. + +## Avoided Blockers (were Round 1 BLOCKERs, now moot via D1) + +- Backend JWT origin-gate (`apps/backend-docker/src/app.ts:84-99`, reads lecturer cookie only for `manage`/`control` origins) — NOT touched. Package runs inside manage's origin. +- `apps/auth` redirect-host allowlist (`apps/auth/src/lib/constants.ts:21-27`) — NOT touched. No new subdomain. + +Still live: + +- Repo HMAC examples are NOT timing-safe (`apps/auth/src/pages/api/discourse.ts:37` uses `!==`). Do not copy. Use `crypto.timingSafeEqual` with length guard. +- Media-library upload flow reusable but flawed (public containers, row-before-upload, no delete, no size/type validation). Copy pattern, fix flaws (D7, D8, D12). + +## Research (evidence, file:line — verified on v3 @ e05743901; re-confirm on v3-ai if a path shifted) + +### Package pattern (buildless, the model for kb-management) + +- `packages/shared-components/package.json`: `"main": "src/index.ts"`, `"types": "src/index.ts"` — no build step, ships raw TS. +- `apps/frontend-manage/next.config.mjs:25-26`: `nextConfig.transpilePackages = [...existing, 'formik']` — workspace packages transpiled by Next, no dist needed. Add `@klicker-uzh/kb-management` here. +- CI typecheck on v3-ai: `.github/workflows/check-types.yml:51` `turbo run build --filter="...[base...HEAD]"` — dependency-aware change-detection, no hardcoded list. kb-management auto-covered. + +### Blob upload (media library pattern) + +- Flow: mutation mints SAS → browser PUTs direct to Azure. Backend never sees bytes. +- SAS mint service: `packages/graphql/src/services/elements.ts:1109-1168` (`getFileUploadSas`). Creds `StorageSharedKeyCredential` from env; `generateBlobSASQueryParameters`, 15 min expiry. +- Frontend PUT: `apps/frontend-manage/src/components/common/MediaLibrary.tsx:67-79` — `new BlobServiceClient(uploadSasURL)` → `getBlockBlobClient().uploadData(file, { blockSize: 4MB })`. +- Dropzone: `MediaLibrary.tsx:93-124` (react-dropzone). +- Env: `BLOB_STORAGE_ACCOUNT_NAME`, `BLOB_STORAGE_ACCESS_KEY` (`elements.ts:1113-1120`). +- SDK `@azure/storage-blob` v12.25.0 in `packages/graphql` and `apps/frontend-manage`. +- Flaws to NOT copy: container `access: 'blob'` = public read (`elements.ts:1127`); row before PUT (`elements.ts:1152`); no delete; no size/type validation. + +### Hatchet + +- Client singleton: `packages/hatchet/src/client.ts:9` (`HATCHET_CLIENT_TOKEN`, `HATCHET_CLIENT_HOST_PORT`). +- Tasks in `prepareHatchetTasks()`: `packages/hatchet/src/index.ts:12`, return dict `:291-307`. Template: `create-audit-log-entry` block `:41-58`. +- Types mirrored: `packages/types/src/hatchet.ts:96-138` (`PreparedHatchetTasks`). +- Trigger from resolver: `ctx.tasks..runNoWait([...])` — example `packages/graphql/src/services/courses.ts:1416`. Wiring: `apps/backend-docker/src/index.ts:106-114`, `packages/graphql/src/lib/context.ts:33-35`. +- Worker: `apps/hatchet-worker-general/src/index.ts:60-143` auto-registers all tasks. No new worker app. +- Local dev: hatchet-lite in docker compose. Gotcha (memory): run worker WITHOUT `tsx --watch` — watch kills workers ("workflow not found"). + +### Webhook receiver + +- backend-docker HTTP surface: only `/healthz` + `/api/graphql` (`apps/backend-docker/src/app.ts:190-194`). Express app `:40`. +- 5078 route shape (mirror): `express.raw({ type: 'application/json', limit })` → `handleKBIngestionWebhook({ prisma, rawBody, headers })` from `@klicker-uzh/graphql`, return `{statusCode, body}`. +- 5078 crypto (reimplement slim): HMAC-SHA256 over `` `${timestamp}.${rawBody}` ``, `crypto.timingSafeEqual` + length guard, 300s timestamp tolerance. + +### Host app (frontend-manage) integration + +- Auth guard: `apps/frontend-manage/src/components/Layout.tsx:24-36` (`useQuery(UserProfileDocument)`, redirect `/login`). KB route sits inside this — authed for free. +- Apollo: `apps/frontend-manage/src/lib/apollo.ts` — `credentials: 'include'`, `x-graphql-yoga-csrf: 'true'`, SSR hydration. Package components use manage's `ApolloProvider`. +- Ops live centrally: `packages/graphql/src/graphql/ops/*.graphql` → codegen → import `@klicker-uzh/graphql/dist/ops`. +- i18n: manage uses `packages/i18n` (`en.ts`/`de.ts`, `kb.*` keys). Package consumes `useTranslations` from host. +- Polling precedent: `frontend-control/src/pages/session/[id].tsx:68` (1s), `frontend-manage/.../cockpit.tsx:57` (2s), `evaluation.tsx:17` (5s). +- data-cy on interactive elements — Playwright `testIdAttribute: 'data-cy'` (`playwright/playwright.config.ts:41`). + +## Architecture (POC loop) + +``` +frontend-manage (already authed lecturer session) renders @klicker-uzh/kb-management + 1. MRequestKbFileUpload -> GraphQL: validate type/size, mint blob-scoped SAS (no DB row) + 2. PUT bytes -> Azure Blob (private container kb-) + 3. MConfirmKbFileUpload -> GraphQL: exists() check, create KBResource (type BLOB, ADDED) + (alt) MCreateKbUrlResource -> GraphQL: validate http(s) URL, create KBResource (type URL, ADDED) + — no SAS/PUT; ingestion fetches sourceUrl itself later + 4. MIngestKbResource -> GraphQL: status=QUEUED, ctx.tasks.ingestKBResource.runNoWait + -> Hatchet task (dispatch end; handler = stub -> real ingestion later) + (future) external ingestion service reads blob via workload identity + (future) POST /api/webhooks/kb-ingestion (HMAC signed) + 5. poll QGetKb (2s while active) <- backend-docker route -> knowledgeWebhooks -> status update +``` + +Status lifecycle: `ADDED -> QUEUED -> PROCESSING -> READY | FAILED` (same for both kinds). POC verifies the webhook half by a manually-signed request (stands in for the not-yet-built ingestion service). + +## Data Model (slice 1 deliverable — copy this) + +New file `packages/prisma/src/prisma/schema/knowledge.prisma`: + +```prisma +enum KBResourceType { + BLOB // uploaded file stored in Azure Blob Storage + URL // external web resource (e.g., website, Kaltura MediaSpace video) fetched during ingestion +} + +enum KBResourceStatus { + ADDED // resource registered (blob uploaded or URL added), not yet queued for ingestion + QUEUED + PROCESSING + READY + FAILED +} + +model KB { + id String @id @default(uuid()) @db.Uuid + name String + description String? + + owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade, onUpdate: Cascade) + ownerId String @db.Uuid + + resources KBResource[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([ownerId]) +} + +model KBResource { + id String @id @default(uuid()) @db.Uuid + + type KBResourceType + title String + + // URL resources: location of the external web resource + sourceUrl String? + + // BLOB resources: upload metadata and blob location (resolver-enforced as required for type BLOB) + originalFilename String? + mimeType String? + sizeBytes Int? + blobName String? + blobHref String? + + status KBResourceStatus @default(ADDED) + statusMessage String? + ingestedAt DateTime? + + kb KB @relation(fields: [kbId], references: [id], onDelete: Cascade, onUpdate: Cascade) + kbId String @db.Uuid + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([kbId, status]) +} +``` + +Plus `user.prisma`: add `kbs KB[]` relation on `User`. + +Per-kind field contract (resolver-enforced, D13): `type BLOB` requires `originalFilename`, `mimeType`, `sizeBytes`, `blobName`, `blobHref`, no `sourceUrl`; `type URL` requires `sourceUrl` (valid http/https), blob fields null. + +Deliberately absent vs 5078 (add later): snippet kind, graph fields, metadata profiles, counts, URL refresh scheduling, ingestion-run table, webhook inbox, course/chatbot link tables, soft delete. + +## Env Vars (new) + +| Var | Where | Value (dev) | +|---|---|---| +| `KB_WEBHOOK_SECRET` | backend-docker + hatchet-worker-general | dev-only literal in `.devcontainer/devcontainer.env`; Infisical elsewhere. Webhook returns 503 (no detail) when unset. | +| `KB_WEBHOOK_URL` | hatchet-worker-general | `http://localhost:3000/api/webhooks/kb-ingestion` — valid because dev worker runs alongside backend-docker. Containerized worker would use the backend service hostname. | + +All new names -> `turbo.json` `globalEnv`. Reuse existing `BLOB_STORAGE_ACCOUNT_NAME`/`BLOB_STORAGE_ACCESS_KEY`. No `APP_KB_*`/`NEXT_PUBLIC_KB_*` (no new app). + +## Skill Routing + +- Browser verification: `agent-browser` skill, delegated login (`lecturer`/`abcd`). Mandatory for every UI slice. KB route lives under manage. +- Review: per-slice review subagent + simplification subagent. +- Finish: `$rs-mr-description-writer`, `$security-review`, `$thermo-nuclear-code-quality-review`. Future ingestion read path: `$rs-workload-identity`. + +## Slices + +Rules per slice: implement -> verify -> review subagent -> simplify subagent -> update `Progress` -> commit ONLY that slice's files (conventional message). i18n: user-visible strings -> `packages/i18n/messages/en.ts` + `de.ts` (`kb.*`). Interactive elements get `data-cy`. + +Apollo rule (every UI mutation): pass `refetchQueries: [{ query: }]` to `useMutation` (pattern: `cockpit.tsx:106,115`, `MediaLibrary.tsx:81`). Without it the UI never reflects the change and every browser Check fails while the backend is fine. + +Prisma accessor gotcha: model `KB` generates client accessor `ctx.prisma.kB` (lowercased first letter). Not a typo. + +### S1 — Data model + +- Do: add `knowledge.prisma` (verbatim above) + `User.kbs` relation. `pnpm run prisma:migrate` (name `kb_poc_schema`), `pnpm run prisma:sync`, regenerate client. +- Check: `pnpm run prisma:migrate` applies cleanly; new tables in `pnpm run prisma:studio`; `pnpm --filter @klicker-uzh/prisma build` green; `apps/analytics` mirror updated. (`pnpm run prisma:setup` WIPES + reseeds — only if acceptable.) +- Commit: `feat(packages/prisma): add slim KB and KBResource models for KB POC` +- STOP after commit: this is the "review the data model" gate. Push, request review on the draft PR before continuing. + +### S2 — Package scaffold + mount in frontend-manage (end-to-end render tracer) + +- Do: + - Scaffold `packages/kb-management` mirroring `packages/shared-components`: `package.json` (`"name": "@klicker-uzh/kb-management"`, `"main": "src/index.ts"`, `"types": "src/index.ts"`, deps `@klicker-uzh/graphql`, `@klicker-uzh/i18n`, `@uzh-bf/design-system`, react peer), `tsconfig.json`. NO rollup, NO Storybook, NO dist. + - `src/index.ts` barrel + `src/KnowledgeBaseManager.tsx` (top-level component, placeholder "Knowledge Bases" heading + current user email for now). + - Add `@klicker-uzh/kb-management` to `apps/frontend-manage/next.config.mjs` `transpilePackages` and to manage `package.json` deps (`workspace:*`). `pnpm install`. + - Route in manage: `apps/frontend-manage/src/pages/knowledgeBases.tsx` (or under `resources/`) wrapping `` in the manage `Layout`. Add nav entry (mirror an existing Header menu item). +- Check: `pnpm install` clean; `pnpm --filter @klicker-uzh/frontend-manage check` green; agent-browser: delegated login `lecturer`/`abcd` -> navigate to KB route -> heading + email render inside manage chrome. Screenshot. +- Commit: `feat(packages/kb-management): scaffold KB management package mounted in frontend-manage` + +### S3 — KB CRUD (end-to-end) + +- Do: + - Service `packages/graphql/src/services/knowledge.ts`: `getUserKbs`, `getKb`, `createKb(name, description?)`, `deleteKb(id)`. Ownership helper used everywhere: + ```ts + async function getOwnedKbOrThrow(ctx, id: string) { + const kb = await ctx.prisma.kB.findUnique({ where: { id } }) + if (!kb || kb.ownerId !== ctx.user.sub) throw new GraphQLError('KB not found') + return kb + } + ``` + - Pothos types `packages/graphql/src/schema/knowledge.ts` (`KB`, `KBResource`). DO expose `type` and `sourceUrl` (S4 UI renders both — D13). Do NOT expose `blobName`/`blobHref` on the GraphQL `KBResource` type (internal; no download in POC). + - Ops: `QGetUserKbs`, `QGetKb`, `MCreateKb`, `MDeleteKb`. `pnpm --filter @klicker-uzh/graphql generate`. + - Package UI: KB list (cards/table), each links to detail route (detail built S4 — placeholder here), create dialog (name + description), delete with confirm dialog (`@uzh-bf/design-system` Modal — no `window.confirm`), empty state WITH create CTA (5078 flaw: empty state hid create button). +- Check: `pnpm --filter @klicker-uzh/graphql test` (vitest: create/list/delete + foreign-owner denial); agent-browser: create KB -> listed, delete -> gone, empty state shows CTA. Screenshots. +- Commit: `feat(kb): knowledge base CRUD across graphql and kb-management package` + +### S4 — File upload to blob storage (end-to-end) + +- Do: + - Service `requestKbFileUpload(kbId, fileName, contentType, sizeBytes)` — assert KB ownership; validate contentType against allowlist (pdf, txt, md, docx, pptx — D12) and `sizeBytes <= 25*1024*1024`; ensure PRIVATE container `kb-` (`createIfNotExists()` with NO access option); blob name `.`; blob-scoped SAS `BlobSASPermissions.parse('cw')`, 15 min. NO DB row. + - Service `confirmKbFileUpload(kbId, blobName, title, originalFilename, mimeType, sizeBytes)` — validate blobName shape (uuid.ext); server-side `blobClient.exists()`, reject if absent; then create `KBResource` (`type: BLOB`, status `ADDED`). + - Service `createKbUrlResource(kbId, url, title)` — assert KB ownership; validate `url` parses as http/https (`new URL()`, protocol allowlist); create `KBResource` (`type: URL`, `sourceUrl`, status `ADDED`). No SAS, no blob fields (D13). + - Service `deleteKbResource(id)` — for `type BLOB`: `blobClient.deleteIfExists()` THEN row delete; for `type URL`: row delete only. `deleteKb`: delete all BLOB-resource blobs then KB row. Server blob clients built with `StorageSharedKeyCredential` + `BlobServiceClient` from env (same as `elements.ts:1113-1125`), NOT the upload SAS. + - Ops: `MRequestKbFileUpload`, `MConfirmKbFileUpload`, `MCreateKbUrlResource`, `MDeleteKbResource`; regenerate. + - UI: KB detail route — dropzone (pattern `MediaLibrary.tsx:67-124`) PLUS "Add link" form (URL + title inputs) for web resources (Kaltura MediaSpace etc.); resource table (type icon, title, size or sourceUrl host, status badge, updated), per-row delete. Upload errors surfaced (toast), not swallowed. +- Check: vitest (validation rejects bad mime/size/foreign KB; bad/non-http URL rejected); agent-browser: upload PDF -> ADDED; upload .exe -> clean error; upload .pptx -> accepted; add Kaltura URL -> listed as URL kind, ADDED; delete both kinds -> gone. Verify blob exists then gone (Azurite/storage explorer; if unavailable locally, document manual check in PR). Screenshots. +- Commit: `feat(kb): blob upload and web-resource registration for KB resources` + +### S5 — Ingestion dispatch via Hatchet (trigger end only) + +- Do: + - Task `ingest-kb-resource` in `packages/hatchet/src/index.ts` (copy `create-audit-log-entry` block `:41-58`; add to return dict `:291-307` + `packages/types/src/hatchet.ts`). Input `{ resourceId, kbId, type, title, blobName?, containerName?, sourceUrl? }` — self-contained; blob fields set for BLOB, `sourceUrl` for URL. + - Handler = documented STUB: logs the dispatch, is the seam where the real ingestion service call goes. NO fake status flip, NO sleep. (Real ingestion built separately per Round 2 ruling 2.) + - Service `ingestKbResource(id)` — ownership; allowed from `ADDED`/`READY`/`FAILED`; set `QUEUED`; `ctx.tasks.ingestKBResource.runNoWait(...)`. If dispatch fails: revert status + surface error (5078 flaw: dispatch failures silently dropped). + - Op `MIngestKbResource`; `pnpm --filter @klicker-uzh/graphql generate`. UI: "Ingest" button per row (disabled while QUEUED/PROCESSING). + - Env: none new for dispatch (task uses existing Hatchet client). +- Check: worker registers task (start WITHOUT `tsx --watch`); click Ingest -> status QUEUED in UI; Hatchet dashboard shows task enqueued; worker log shows stub dispatch line. Status stays QUEUED (no ingestion yet — expected, verified fully in S6). +- Commit: `feat(kb): hatchet ingestion dispatch task and ingest trigger mutation` + +### S6 — Webhook status receiver (contract's other end) + +- Do: + - `packages/graphql/src/services/knowledgeWebhooks.ts`: export `handleKBIngestionWebhook({ prisma, rawBody, headers })` -> `{statusCode, body}`. Re-export from barrel `packages/graphql/src/index.ts` (else `app.ts` import fails at build). Verify: secret configured (503 generic if not); timestamp within 300s; HMAC via `crypto.timingSafeEqual` + length guard (do NOT copy `discourse.ts:37` `!==`). Parse `{ resourceId, status: PROCESSING|READY|FAILED, statusMessage? }`, allow-list status, transition guard (PROCESSING from QUEUED/PROCESSING; READY/FAILED from QUEUED/PROCESSING; stale/dupe -> 200 no-op). Set `ingestedAt` on READY. + - Route `apps/backend-docker/src/app.ts`: `app.post('/api/webhooks/kb-ingestion', express.raw({ type: 'application/json', limit: '1mb' }), ...)` — mirror 5078 shape. + - Env: `KB_WEBHOOK_SECRET` (backend + worker), `KB_WEBHOOK_URL` (worker) -> env files, `.devcontainer/devcontainer.env`, `turbo.json` globalEnv. Signing helper (for the real service + for test): HMAC-SHA256 hex over `` `${timestamp}.${rawBody}` ``, headers `x-kb-timestamp` + `x-kb-signature`. +- Check: vitest on `handleKBIngestionWebhook`: valid sig OK; bad sig 401; stale timestamp 401; bad status 400; illegal transition no-op; missing secret 503. Live: signed curl PROCESSING then READY -> watch ADDED->QUEUED->PROCESSING->READY in `prisma:studio` and via query; signed FAILED -> FAILED with message. (This manual signed request stands in for the future ingestion service.) +- Commit: `feat(kb): signed ingestion webhook receiver updating resource status` + +### S7 — UX polish + auto-refresh + +- Do: + - Polling: KB detail query `useQuery(GetKbDocument, { variables, pollInterval: anyActive ? 2000 : 0 })` where `anyActive` = any resource QUEUED/PROCESSING. Single approach; do NOT also call `startPolling`/`stopPolling`. + - Status badges (ADDED gray, QUEUED amber, PROCESSING amber+spinner, READY green, FAILED red + statusMessage tooltip), loading skeletons, empty states with CTA, toasts, disabled states during mutations. + - Responsive pass usable at 375px. i18n keys complete (en + de). `data-cy` on all interactive elements. +- Check: agent-browser full walkthrough inside manage: login -> KB route -> create -> upload -> ingest (QUEUED) -> signed curl PROCESSING/READY -> badge flips live without manual reload -> fail path -> delete. Screenshots desktop + mobile, EN + DE -> PR body. +- Commit: `enhance(kb): status badges, live polling, and responsive polish` + +### S8 — Finish + +- Do: root `pnpm run check:all`; fix strays. `docs/getting-started.md` KB route note if useful. (No CI app-list edits: package, not app; v3-ai check-types is change-detection.) +- Check: full CI green on PR. +- Commit: `chore(kb): final checks and dev docs for KB POC` (only if changes) +- Then finish gate. + +## Finish Gate (after S8) + +1. `$security-review` subagent (webhook, SAS, authz). Handle/defer explicitly. +2. `$thermo-nuclear-code-quality-review` maintainability pass. +3. Independent final branch review (external agent). +4. PR body via `$rs-mr-description-writer`: whole-branch summary, verification, screenshots (S7), manual-verify checklist (Azure CORS on storage account is out-of-repo — document; workload-identity read path is future work). +5. Mark PR ready only after user approval. Never merge without explicit user approval. + +## Verification Loop (once, before S2) + +- Stack: devcontainer or local docker compose (Postgres, Redis, hatchet-lite, Traefik) + `pnpm run dev:raw` scoped to manage + backend + worker. Multistack DNS gotcha: shared devnet `postgres` alias collides across worktree stacks — pin /etc/hosts (project memory). +- Blob: real dev storage account (`BLOB_STORAGE_*` via Infisical) or Azurite. Decide in S4, document in Progress. +- Browser: agent-browser, delegated login `lecturer`/`abcd`, KB route under manage. + +## Independent Plan Review + +- Reviewer: `droid exec --model glm-5.2` (pre-approved), 2026-07-15, Round 1 plan. 12 findings (2 MAJOR, 10 MINOR), all accepted and integrated (Apollo refetch rule, webhook-URL caveat, S5 codegen, polling-API single approach, server-side blob-delete client, non-destructive S1 check, dropped unused task input, inlined ownership helper, hid blobName/blobHref, barrel export, list->detail nav, confirm-upload exists() check). +- Round 2: 6 user rulings (see Grill Round 2) reshaped architecture (package not app, v3-ai base, no sim, docx/pptx). Plan rewritten accordingly. + +## Progress + +- [x] S1 data model — DONE 2026-07-15. Evidence: migration `20260715211610_kb_poc_schema` created + applied on fresh isolated postgres:15 (full history + new migration); `KB`/`KBResource` tables + indexes + cascade FKs verified via psql; `pnpm --filter @klicker-uzh/prisma build` green; `prisma:sync` mirrored to `apps/analytics` (gitignored artifact, not committed). Review subagent: 0 findings (byte-for-byte plan match, conventions cross-checked vs chat/course/resources.prisma, `prisma validate` clean). Simplify subagent: 2 proposals rejected — keep `updatedAt` (repo-wide model convention, edit mutation plausible) and keep `@@index([kbId, status])` (covers kbId-prefix queries, matches 5078 shape). AMENDED at S1 gate (user ruling 7 -> D13): added `KBResourceType { BLOB, URL }` + `sourceUrl`, blob fields nullable, initial status `UPLOADED` -> `ADDED`; migration regenerated in place as `20260715213657_kb_poc_schema` (old one never left this branch), re-applied + re-verified on fresh isolated postgres:15. NEXT: S1 review gate — data model review on PR before S2. +- [x] S2 package scaffold + mount — DONE 2026-07-17. Evidence: added the buildless `@klicker-uzh/kb-management` package (raw `src` entrypoints; no build/dist/Storybook), mounted it at `/resources/knowledgeBases` inside the existing manage `Layout`, wired Resources navigation, host transpilation, Tailwind source discovery, and paired `kb.*` EN/DE messages. `pnpm install --lockfile-only --frozen-lockfile` passed including supply-chain policy verification; package and frontend-manage typechecks passed; targeted Prettier and `git diff --check` passed. Browser: seeded the confirmed-empty local test DB, authenticated through real delegated access as `lecturer`, rendered the heading and signed-in email against this branch's backend on temporary port 3100 (port 3000 was already owned by another worktree), verified the Resources menu entry, and captured `/tmp/kb-s2-real-desktop.png`. Review subagent: one P2 accepted — removed unrelated pnpm peer-resolution churn so the lockfile contains only the manage dependency and package importer. Simplify subagent: exact-pin design system accepted; removing package/mount/i18n/Tailwind wiring and prop-plumbing the profile were rejected as either required or more complex. NEXT: S3 KB CRUD. +- [x] S3 KB CRUD — DONE 2026-07-17. Evidence: added owner-scoped `getUserKbs`/`getKb`/`createKb`/`deleteKb` services using the plan's non-disclosing `KB not found` ownership helper; exposed `KB`/safe `KBResource` fields (including `type` and `sourceUrl`, excluding blob location fields); added and regenerated all four operations; built the create/list/detail-placeholder/delete UI with paired EN/DE strings, stable `data-cy` hooks, Apollo query refetches, and design-system confirmation modal. Verification: GraphQL, `kb-management`, and `frontend-manage` typechecks green; full GraphQL suite 25 files / 487 tests green, then focused CRUD suite 5/5 green after review fixes; real delegated browser walkthrough created and listed `S3 Browser KB`, opened its detail placeholder, confirmed deletion, and returned to the empty state with both create CTAs; screenshots `/tmp/kb-s3-detail.png`, `/tmp/kb-s3-delete-modal.png`, `/tmp/kb-s3-empty.png`. Review subagent: accepted P2 server-side trim/reject for empty names with regression coverage and P3 close-button `data-cy` hooks; no remaining findings. Simplify subagent: removed dead S2 email string, reused the create label for the modal title, and trimmed unused operation selections; component boundaries, resource query seam, generated artifacts, and exact ownership helper retained. NEXT: S4 file upload and URL-resource registration. +- [x] S4 upload + blob — DONE 2026-07-17. Evidence: added owner-scoped upload-ticket, confirmation, URL-registration, resource-deletion, and KB blob-cleanup services; upload tickets create private `kb-` containers with no access option, generate UUID filenames and blob-scoped `cw` SAS credentials for 15 minutes, and create no row until the server verifies the blob. Confirmation retains the planned `exists()` gate, additionally checks the uploaded content length/type, deletes mismatches, and uses the blob UUID as the resource ID for retry-safe idempotency. Added and regenerated all four operations; built the PDF/TXT/MD/DOCX/PPTX dropzone, link form, localized resource list/statuses, and destructive delete modal with Apollo refetches, paired EN/DE strings, accessible labels/focus states, and `data-cy` hooks. Verification: frozen lockfile install plus GraphQL, `kb-management`, and `frontend-manage` typechecks green; focused integration suite 16/16 green, covering validation/foreign ownership, private container creation, exact SAS scope and expiry, no pre-confirm row, absent/mismatched blobs, sequential/concurrent retry idempotency, cross-KB contention, UUID non-disclosure, URL lifecycle, and blob-first resource/KB deletion. Real delegated browser walkthrough registered the Kaltura URL, showed it as URL/ADDED, rejected `.exe` with a visible error, deleted the URL, and returned to the empty state; screenshots `/tmp/kb-s4-url-resource-final.png`, `/tmp/kb-s4-invalid-file.png`, and `/tmp/kb-s4-url-deleted.png`. Blob choice: real dev Azure storage through the existing `BLOB_STORAGE_*` contract (no new env or Azurite endpoint deviation); PDF/PPTX happy-path upload and physical blob existence/deletion remain a manual PR check because local Infisical login and the Azure refresh token were expired. Review subagent: accepted retry-idempotency, actual blob-metadata validation, and storage-critical coverage; two follow-up concurrency/non-disclosure issues were fixed and the final narrow re-review found no remaining findings. UI review: accepted dropzone semantics/focus/busy/live state, label association, locale-aware sizes, and hostname wrapping; no remaining S4 findings (inline validation remains S7 polish). Simplify subagent: removed ineffective callback/block sizing/max-file duplication, made resource ownership a filtered Prisma query, reused the link label, and reduced the lockfile to importer-only changes; retained trust-boundary validation duplication, cohesive components, exhaustive statuses, direct dependencies, and generated artifacts. NEXT: S5 exact in-repo Hatchet logging-only dispatch stub and trigger mutation. +- [x] S5 hatchet dispatch — DONE 2026-07-17. Evidence: added the typed `ingest-kb-resource` Hatchet task as the plan's in-repo logging-only stub, with no HTTP request, FalkorDB call, sleep, or resource-status mutation; the awaited log records only resource/KB identifiers and type. Added the owner-scoped `ingestKbResource` service and generated mutation, using an atomic allowed-status claim to `QUEUED`, single-input `runNoWait`, and a conditional rollback that cannot overwrite a resource already advanced by a downstream callback. The discriminated Hatchet input requires a self-contained private-container/blob location for BLOB resources or `sourceUrl` for URL resources. Added the per-row localized Ingest action, loading/disabled states, exact Apollo detail refetch, toasts, and `data-cy` hook. Verification: focused KB integration suite 22/22 green, including ADDED/READY payloads, foreign/active rejection, concurrent single-claim dispatch, failed-dispatch rollback, and preservation of an advanced PROCESSING status; `types`, `hatchet`, `graphql`, `hatchet-worker-general`, `kb-management`, and `frontend-manage` typechecks green; Prettier and `git diff --check` green. The non-watch worker registered all 15 workflows including `ingestKBResource`; a real delegated browser click left both resources `QUEUED` in manage, Hatchet showed the new task succeeded, and its Logs tab showed `KB ingestion dispatch stub`; screenshots `/tmp/kb-s5-queued-final.png`, `/tmp/kb-s5-hatchet-run.png`, and `/tmp/kb-s5-hatchet-log.png`. Review subagents: accepted atomic claim/conditional rollback, discriminated payload typing, single-input dispatch, awaited privacy-safe logging, and race/rollback coverage; correctness, UI-pattern, and final simplification re-reviews found no remaining S5 issues. NEXT: S6 signed webhook receiver exactly as planned. +- [x] S6 webhook receiver — DONE 2026-07-17. Evidence: added the exported `handleKBIngestionWebhook` service and signing helper with the exact raw-byte HMAC-SHA256 contract, strict timestamp/signature validation, generic missing-secret response, UUID/payload allow-listing, and a single atomic transition update that makes stale, duplicate, missing-resource, and racing callbacks successful no-ops without allowing PROCESSING to regress a terminal state. Mounted the route-local 1 MB raw JSON receiver before Yoga and declared `KB_WEBHOOK_SECRET`/`KB_WEBHOOK_URL` in the applicable backend/worker env templates and Turbo global environment. The plan's `.devcontainer/devcontainer.env` target is non-applicable on this branch because no `.devcontainer` tree exists; no orphan structure was invented. Verification: real-PostgreSQL handler suite 10/10 green (PROCESSING, READY/`ingestedAt`, FAILED/message, bad signature, stale timestamp, bad status, illegal transition, concurrent PROCESSING/READY, malformed UUID, and missing secret); GraphQL, backend, and worker typechecks green; GraphQL/backend builds completed (with the repository's existing Rollup warnings only); Prettier and `git diff --check` green. Live signed callbacks against browser-created resources persisted QUEUED -> PROCESSING -> READY with `ingestedAt` and QUEUED -> FAILED with a message; direct PostgreSQL queries and a real authenticated manage-page reload confirmed Ready/Failed. Contract/security and simplification review subagents found no S6 code issues. S5 remains the plan's in-repo logging-only Hatchet stub. NEXT: S7 live polling and UX polish. +- [x] S7 UX polish — DONE 2026-07-17. Evidence: added a single dynamic Apollo `pollInterval` (`2000` ms only while any resource is QUEUED/PROCESSING; `0` for terminal-only sets, with no `startPolling`/`stopPolling` path), exact gray/amber/green/red status presentations, a reduced-motion-safe PROCESSING spinner, FAILED-message tooltip, accessible/live loading and status states, resource-empty upload/link CTAs, inline URL validation, mutation disabled states, long-text containment, locale-preserving internal links, and responsive two-column mobile actions. EN/DE `kb.*` key parity is 46/46 and all new interactive elements have stable `data-cy` hooks. Browser: in the real delegated manage session, Ingest changed READY -> QUEUED, manually signed S6 callbacks changed QUEUED -> PROCESSING -> READY live within the polling interval without a reload, the terminal state stopped polling, and a separate signed FAILED callback exposed its message through the tooltip; create-link, invalid-link, delete, empty-state CTA, German list/detail/back navigation, and 375 px layouts were also exercised. Committed PR screenshots: `project/screenshots/kb-poc-s7-en-processing.png`, `kb-poc-s7-en-mobile.png`, `kb-poc-s7-de-desktop.png`, and `kb-poc-s7-de-mobile.png`. Actual Azure PDF/PPTX upload remains the S4 manual PR check because the local Azure refresh token is expired; the visible invalid-upload path was already verified. Package and frontend-manage typechecks, targeted Prettier, EN/DE parity, and `git diff --check` are green. Review subagents: accepted consolidated status presentation logic; retained the small state/effect bridge required to feed current query results back into the next render's `pollInterval`, and retained Next `Link` after real browser evidence showed native anchors reset the active locale. Final correctness review found no blocking issues, and both final reviews confirmed S5/S6 are untouched; S5 remains the plan's in-repo logging-only stub. NEXT: S8 full checks and finish gate. +- [x] S8 finish — DONE locally 2026-07-17; PR CI pending after push. Evidence: root `pnpm run check:all` is green across the monorepo (typechecks, formatting, lint, and syncpack). The two KB integration suites were rerun against a disposable PostgreSQL 15 database after all 176 migrations applied cleanly: 2 files / 33 tests green, covering CRUD/upload/URL/dispatch and the signed webhook contract; the disposable container was stopped and removed afterward. Repository-requested `opengrep scan --config auto` completed; its whole-repo baseline contains 609 pre-existing findings outside this branch, while a focused scan of all 19 hand-written KB application/test files ran 210 rules with 0 findings. `docs/getting-started.md` is absent on the `kb-poc` target, so the plan's optional route note was correctly skipped instead of inventing a new documentation tree; this plan remains the implementation/verification record. Finish-gate review found and fixed one cross-KB blob-safety issue before publication: confirmation now detects an existing same-owner resource claim across all owned KBs before any metadata-mismatch cleanup, preserving foreign-owner non-disclosure and preventing deletion of a blob referenced by another KB; the new regression is included in the 33-test run. Final re-reviews passed: security found no blocker and explicitly deferred attempt correlation, upload-ticket/orphan hardening, future fetch SSRF controls, and Hatchet retention to the production ingestion follow-up; thermonuclear maintainability approved; independent whole-branch review approved after the blob fix. S5 remains the plan's in-repo logging-only stub. NEXT: push and draft PR targeting `kb-poc`; verify PR CI without marking ready. +- [x] External Hatchet follow-on bridge — DONE locally 2026-07-20; real cluster smoke remains an explicit deployment prerequisite. S5's existing awaited `KB ingestion dispatch stub` log was retained unchanged and still precedes dispatch; the separately approved follow-on now calls a dedicated external Hatchet service after that log. Each click claims only the selected resource with a fresh attempt UUID and the selected `balanced`/`quality`/`fast` mode (Balanced default), sends exactly one source, uses the stable KB/resource UUIDs for course/source/graph naming, and stores only the latest attempt, external run ID, and external start time. Private blob inputs receive a blob-scoped, read-only, HTTPS-only SAS valid for one hour with clock-skew tolerance; the code includes the approved note that the validity may need adjustment for future larger files. A singleton, non-overlapping database sweep checks all active runs once per minute, posts correlated signed status callbacks, handles external failure/cancellation with sanitized messages, and applies the environment-configured timeout (`KB_INGESTION_TIMEOUT_SECONDS`, default `3600`). Active resources and their parent KBs cannot be deleted while ingestion is active; database row locks serialize deletion against new ingestion claims, Azure deletion is bounded to 30 seconds, and its Prisma transaction has a 60-second budget so storage retries cannot outlive the database operation. Deterministic verification is green: shared signer 2/2, split GraphQL knowledge/ingestion/webhook suites 47/47, Hatchet bridge/SAS/retry/monitor 44/44, all eight focused typechecks, root `pnpm run check:all`, `git diff --check`, and focused Opengrep (210 rules / 0 findings). Real delegated-browser verification sent only the clicked row's `FAST` selection, then exercised a local external Hatchet run through `QUEUED -> PROCESSING -> READY` via the one-minute sweep and Apollo polling without reload; an external failure mapped to `FAILED`, a stale correctly signed callback returned 200 without mutating the latest attempt, and the Hatchet log contained the preserved stub exactly once with no URL-like text. Evidence screenshots are `project/screenshots/kb-external-ingestion-speed-en-desktop.png`, `project/screenshots/kb-external-ingestion-speed-de-mobile.png`, and `project/screenshots/kb-external-ingestion-ready-en-desktop.png`. Final security, thermo-nuclear maintainability, remediation, and whole-branch reviews approved the branch after resolving deletion/orphan races, failure-proof logging, status-index coverage, SAS-safe upload logging, bounded storage I/O, and the integration-test file split. Cluster verification cannot run until the external Hatchet host/API/TLS/workflow values and the out-of-repo `KB_INGESTION_HATCHET_CLIENT_TOKEN`, `KB_WEBHOOK_SECRET`, and `BLOB_STORAGE_ACCESS_KEY` are installed in the general-worker deployment; the external workflow itself does not call Klicker's webhook. +- [x] W1 scope correction — DONE locally 2026-07-26; final PR checks pending. Knowledge-graph visualization and model-selection work was removed from the core branch and preserved on draft PR #5206 for W9. The resource-level Ingest UI and its EN/DE labels are again present at HEAD. A repeated signed `PROCESSING` callback now refreshes the current message without regressing terminal states. Both Hatchet workers run without `tsx --watch`, so the shared client no longer imports or patches the SDK-internal logger. Final security review added a shared URL guard that rejects credentials and local/private/reserved literal destinations at resource registration and again before dispatch; its literal/numeric bypass regressions pass, while DNS/redirect egress remains an external deployment gate that the reviewer explicitly classified as outside #5182's Klicker code boundary. W5 owns abandoned-upload retention. Delegated browser verification created a URL resource, selected `FAST`, queued it through the live local worker, and confirmed the privacy-safe stub log; current evidence is `project/screenshots/kb-w1-resource-ingest-en-desktop.png`, `kb-w1-resource-ingest-queued-en-desktop.png`, and `kb-w1-resource-ingest-queued-de-mobile.png`. Root `check:all`, focused utility/Hatchet/GraphQL suites, and production builds of every W1-affected package pass; fresh PR CI is the remaining publication gate. The target `v3-ai` line predates the engineering wiki under `docs/`; its existing `project/CODEBASE_NOTES.md` and the worker plan/spec were updated instead of importing an unrelated documentation tree. Reconcile these facts into the wiki when the integration line next absorbs the current `v3` documentation structure. +- [x] W1 integration — DONE 2026-07-26. Fresh PR #5182 CI passed and its approved squash merge landed on `kb-poc` as `b66ae0107d05c455af4165b276a052b0740088c2`. The branch then absorbed current `v3-ai`; KB and verification schema/API conflicts were reconciled, the KB package was aligned with the line's Next 16/React 19.2 stack, and W1 behavior was incorporated into `docs/domain-model.md`, `docs/async-and-workers.md`, `docs/frontend-conventions.md`, and `docs/log.md`. Root `check:all`, frozen-lockfile installation, utility tests (73), Hatchet tests (48), and the full production build (22/22 tasks) pass on the merged line. The production build also verified the integration-only i18n correction: explicit locale loaders replace the Turbopack-incompatible computed import, and `next-intl` now resolves against one Next 16 peer graph. The next package is the hard-ordered W2 canonical ingestion-contract alignment; no UX expansion lands first. + +Worktree: `trees/kb-poc` (repo `klicker-uzh`, branch `kb-poc`, base `v3-ai`). Cleanup after merge: `git worktree remove trees/kb-poc && git worktree prune` (needs approval). + +## Next Steps After POC (out of scope) + +Real ingestion service (reads blobs via workload identity, fetches URL resources with DNS/redirect egress enforcement, calls webhook), webhook inbox/outbox hardening, expired unconfirmed-upload cleanup, snippet resource kind, URL refresh scheduling, chat-runtime KB consumption, deploy config, E2E tests, KB sharing/roles. diff --git a/project/2026-07-24-kb-production-v1-roadmap-plan.md b/project/2026-07-24-kb-production-v1-roadmap-plan.md new file mode 100644 index 0000000000..c698bfa8c6 --- /dev/null +++ b/project/2026-07-24-kb-production-v1-roadmap-plan.md @@ -0,0 +1,314 @@ +# KB Management — Production v1 Roadmap Plan + +## Plan Identity + +- Date: 2026-07-24; grill rulings recorded 2026-07-25; branch topology corrected and W1 integrated 2026-07-26. Implementing branch: `kb-poc`, carried by [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) into `v3-ai`. The roadmap was reviewed on `claude/admiring-nightingale-c30fa4`, then moved onto the implementing branch before W1 execution. +- Inputs reviewed: [PR #5182](https://github.com/uzh-bf/klicker-uzh/pull/5182) (junior implementation of the [POC plan](2026-07-15-pr-5174-kb-poc-plan.md), branch `feat/kb-poc-management-ui`, base `kb-poc`, 149 files / +23k), [PR #5078](https://github.com/uzh-bf/klicker-uzh/pull/5078) (older full-scale KB prototype, branch `codex/kb-management-ui`, ruled REPLACE by the 2026-07-23/24 program review), the R5.0 scope-grill handoff and the program's final roadmap + A6 Klicker lens report (external `_local` review artifacts, not in this repo). +- Method: three independent Opus review subagents (5182 core-plan conformance; 5182 beyond-plan scope vs the canonical ingestion-platform contract; 5078 salvage inventory vs 5182), synthesized here. `trees/kb-review-5182` was attached to `feat/kb-poc-management-ui` on 2026-07-26 and is now the W1 implementation worktree. +- Purpose: (1) reviewable record of the [PR #5182](https://github.com/uzh-bf/klicker-uzh/pull/5182) findings, (2) the ruled decision record of the product-scope grill (program gate R5.0), (3) the finalized production v1 work-package roadmap. Rulings recorded and roadmap finalized 2026-07-25. + +## Fixed Program Constraints (user-ruled 2026-07-23/24 — do not re-litigate) + +- Ingestion is an external producer-neutral service: Klicker calls a synchronous HTTP API (`POST /v1/resources` family), never fire-and-forget dispatch. Status returns via durable HMAC-signed webhooks (retry + dead-letter) PLUS reconciliation polling; webhooks are never the sole source of truth. +- Canonical status event: `OperationStatusEvent` with `X-Ingestion-*` headers, `operation_id` correlation, `resource_version` int, nested `serving: {active_resource_version, active_sha256}`; receiver schemas are `extra="forbid"`-strict — adopt verbatim, field-for-field. +- Resource identity is Klicker-owned: `external_resource_id == KBResource.id`; ingestion never assigns or overwrites Klicker-side ids. +- Per-resource monotonic versions; no whole-KB revision model; two-axis status (operation vs serving: "update failed, still serving v(N)"); candidate content invisible until `resource_active=true`. +- Feature gating via GrowthBook (external dependency, integration in progress); no ad hoc flag infra. +- `kb_id` server-side validation is deferred ONLY until lecturer self-service KB creation exists (D-8) — self-service in v1 triggers that gate. +- Platform-side gates outside this repo: Klicker tenant mount on shared `mcp-doc-query.stg-doc-query` (R4.3, needs blast-radius fixes R1.3/R1.4), producer-enable + 10-scenario synthetic journey (R4.4), `resource_active` retrieval gating (R1.1) and Milvus partition wiring (R1.2) block ANY retrieval canary. + +## [PR #5182](https://github.com/uzh-bf/klicker-uzh/pull/5182) Review — Verdict and Findings + +**Verdict: needs-fixes (1 blocker) before merging into `kb-poc`.** The core S2-S8 implementation conforms to the POC plan and is hardened beyond its minimum in several places. The problems concentrate where later beyond-plan work landed on the same branch. + +### Blocker + +| # | File | Problem | Fix | +| --- | --- | --- | --- | +| B1 | `packages/kb-management/src/components/KnowledgeBaseResourceList.tsx:193-207` | The per-resource "Ingest" button (POC Goal #4, slices S5/S7) was removed by later KG commit `61987498c`. The full `ingestKbResource` path (service `packages/graphql/src/services/knowledge.ts:484-571`, resolver, Hatchet task, ~1200 LOC + ~1900 LOC tests) is live and reachable via direct GraphQL but has zero UI callers — a zombie surface with doubled maintenance and attack surface. Plan `Progress` (S5/S7) and the committed screenshots are stale — they show a UI that no longer exists at HEAD. | Ruled by grill Q1: either restore a per-resource ingest entry point, or bless graph-level-only ingestion and delete the resource-level path. Update Progress + screenshots to match HEAD either way. | + +### Major + +| # | File | Problem | Fix | +| --- | --- | --- | --- | +| M1 | `packages/hatchet/src/kbIngestion.ts:442-519`, `kbGraphIngestion.ts:128-254` | Singleton monitor cron (`* * * * *`, `maxRuns:1/CANCEL_NEWEST`) polls every active resource/graph sequentially with synchronous external `runs.get_status()` calls — no batching, no per-call timeout, no cap. At dozens of concurrent ingestions one tick overruns 60s and `CANCEL_NEWEST` silently drops the next tick, stalling all status updates. | Batch status lookups, add per-call timeouts, shard the sweep. (Moot for the resource path if Q1/Q2 replace the bridge — still applies to the graph path.) | +| M2 | Two parallel ingestion state machines (`KBResource`-level and `ChatbotKnowledgeGraph`-level) are both live, monitored every minute, with two different internal boundaries (signed webhook vs direct Prisma writes from the worker) for the structurally identical problem. | One product owner per state machine, or collapse. Ruled by grill Q1/Q5. | +| M3 | `packages/i18n/messages/en.ts:1170-1172` + `de.ts` | Three dead `kb.ingestResource*` key pairs (fallout of B1). | Remove or restore with B1. | + +### Minor + +- `knowledgeWebhooks.ts:121-124` — `PROCESSING` transition allow-list is `[QUEUED]` only; plan says `[QUEUED, PROCESSING]`. Behaviorally equivalent (repeat PROCESSING no-ops); align code or plan text. +- `packages/hatchet/src/client.ts:9-19` — dev-only monkeypatch (`logger.undefined = ...`) works around the Hatchet 1.9.4 + `tsx --watch` bug instead of the repo's documented fix (run workers without `--watch`); breaks silently on SDK bump. Replace or fence with a version assertion. +- `packages/types/src/hatchet.ts` — model allow-list is a hardcoded TS array spanning three apps; production needs a config/DB-driven list. +- `packages/graphql/src/services/knowledge.ts:148-185` — core KB queries include the `knowledgeGraph` relation, coupling core-KB and KG code review-wise. + +### Strengths to build on (verified) + +- SAS/upload path exceeds the plan: server verifies uploaded blob `contentLength`/`contentType` against claims, idempotent `upsert` keyed by blob UUID, tested concurrent/cross-KB confirmation races. +- Deletion serialized with `SELECT ... FOR UPDATE` + bounded timeouts against concurrent ingestion claims. +- Webhook crypto textbook: HMAC over exact `${timestamp}.${rawBody}` raw bytes, `timingSafeEqual` with length guard, 300s window, 503-no-detail without secret; worker failure path goes through the signed webhook, not direct DB writes (D5 preserved). +- FalkorDB reader: parameterized Cypher, hard server-side LIMITs, read-only queries, node-id validation; graph DTO normalization denylists embeddings/secrets/SAS params before reaching the browser. +- i18n 51/51 keys en+de parallel; `data-cy` on every interactive element; `refetchQueries` uniform. +- Test suite (795-line `knowledge.test.ts`, 314-line webhook suite, KG + ingestion suites) exceeds 5078's coverage for the same surface. + +### Contract fit — 5182's external Hatchet bridge vs canonical platform + +| Element | Fit | Note | +| --- | --- | --- | +| Transport: `runNoWait()` onto a second Hatchet tenant (client token/host/TLS in Klicker worker) | **Opposed** | Exactly the Hatchet-to-Hatchet coupling the platform contract retires. Replace with sync `POST /v1/resources`. | +| Durable signed webhook + reconciliation sweep as complementary sources | **Aligned** | Correct shape; keep the skeleton (verification, correlation, DB-driven sweep discovery). | +| Event schema `{resourceId, ingestionAttemptId, status, statusMessage}` over `x-kb-*` headers | **Opposed** | Replace with `OperationStatusEvent` + `X-Ingestion-*` verbatim; HMAC-over-raw-body mechanics reusable. | +| Identity: `resourceId == KBResource.id` end-to-end | **Aligned** | Already correct. | +| Versions / two-axis status | **Adaptable** | Absent at resource level; KG's `selectionRevision`/`builtRevision` equality gate is the best existing analog of "candidate invisible until active". | +| Feature gating | **Absent** | No flags anywhere; GrowthBook per D-5. | + +## [PR #5078](https://github.com/uzh-bf/klicker-uzh/pull/5078) Salvage — What to Take, When + +5182 already independently rebuilt the valuable 5078 primitives (HMAC sign/verify, ownership + row-lock patterns, larger test suite) — **nothing to port there**. Genuine take-later items, mapped to v1 work packages: + +| 5078 item | 5182 state | Recommendation | Target package | +| --- | --- | --- | --- | +| Soft-delete (`deletedAt`/`deletedById`) + delete-guard while QUEUED/PROCESSING | Absent (hard delete) | Take-later (columns + guard only) | W5 | +| `KBIngestionRun` per-resource run history | Absent (current-status only; retries lose history) | Take-later | W3 | +| `KBCourse`/`KBChatbot` M:N bindings + one-enabled-per-chatbot invariant | Absent; 5182 instead has `ChatbotKnowledgeGraph` (curated resource subset, revisioned) — **fundamentally different model** | Design reconciliation, not a port (grill Q5) | W4 | +| Typed metadata Zod-per-profile validation | Absent | Take-later (pattern, not values) if lecturer tagging prioritized | backlog | +| KB aggregate counts (`resourceCount`/`sizeBytes`) | Absent | Take-later | W7 | +| `KBWebhookInbox` event log | Absent; 5182's `updateMany` transition guard is immune to 5078's TOCTOU bug | Optional pure audit log later (insert-on-conflict-ignore), never as the dedupe gate | backlog | +| UX: ResourceInspector panel, bulk select/delete (ADD the missing confirm dialog), search/filter bar + server-side filter input, per-row progress bar, linked-consumers panel, metrics header, retry affordance | All absent | Re-implement (not copy) | W7 | +| KLICKER_OBJECT resource kind (reference live quizzes/elements as KB sources) | Absent | Take-later, needs enum translation onto 5182's `type` | v2 | +| Tests | 5182's suite is larger and matches its own API | Drop | — | + +**Danger patterns from 5078 — never copy:** phantom upload flow (DB row, bytes never leave browser); non-transactional DB-write-then-webhook-dispatch mutations; unguarded `BigInt()` on webhook payload; `?? undefined` null-collapse making nullable fields unclearable; `window.prompt`/`window.confirm` for identifiers and destructive actions. (Plus the three program-ruled items: fire-and-forget dispatch, payload-sourced `externalResourceId`, pre-canonical event vocabulary.) + +## Decisions — Grill Rulings (R5.0, ruled 2026-07-25) + +All Q1-Q11 ruled by the user in the 2026-07-25 grill session. Ruling column is binding; consequences are folded into the roadmap below. + +| Q | Decision | Ruling | Rationale / consequence | +| --- | --- | --- | --- | +| Q1 | Zombie ingest path (finding B1): v1 ingestion primitive | (a) Restore the per-resource Ingest entry point; resource-level ingestion is the primary primitive, KG build is one consumer of it | Matches the canonical platform contract (per-resource operations + versions). W1 restores the button and the `kb.ingestResource*` i18n keys (M3); Progress/screenshots refreshed to HEAD. | +| Q2 | Bridge replacement timing | Hybrid: merge 5182 into `kb-poc` as-is now (bridge intact); W2 contract alignment is the immediate next package and nothing else lands on the line before W2 completes; no STG lecturer exposure until aligned | Preserves working POC value without stalling the merge on the external API. M1 resource-path monitor fix is deferred into W2 (the sweep is replaced there by operations-API polling); revisit only if W2 slips. | +| Q3 | KG visualization + model selection | (b) Split into a separate parked PR during W1; core 5182 merges without it; re-lands as W9 after W2 | Reviewer-verified separable (clean migration split; coupling = `kbGraphIngestion` imports from `kbIngestion` + shared cron). Graph-path M1 monitor fix travels with the parked PR (W9). | +| Q4 | Self-service KB creation in v1 | (a) Keep self-service | Triggers the D-8 gate: server-side `kb_id` validation must ship in v1 (W6). | +| Q5 | KB↔chatbot↔course binding model | (a) KB-level attach: one enabled KB per chatbot (5078 `KBChatbot`-style one-enabled invariant); `KBCourse` deferred; KG resource-curation stays a KG-feature concern | Matches the scope-token design (one `kb_id` claim per chat request); cheapest correct v1; multi-KB (`kb_ids` array) stays v2 per prior ruling. Resolves M2 direction: one state machine per concern, resource level owns ingestion. | +| Q6 | Versioning depth in v1 | (a) Schema columns now (`resourceVersion`, `activeResourceVersion`, `activeContentSha256`, `errorCode`), replace-on-re-ingest semantics, minimal two-axis status UI | The canonical event schema requires the fields anyway; columns are cheap now, painful later (W3). | +| Q7 | Delete semantics for lecturers | (a) Soft-delete fence (`deletedAt`/`deletedById` + delete-guard while QUEUED/PROCESSING), async hard cleanup, tombstone-compatible ingestion delete | Aligns with the platform tombstone contract; port the 5078 column pair + guard (W5). | +| Q8 | Per-KB quotas for the ingestion registry allowlist | 100 resources / 500 MB per KB (tighter than the proposed 200 / 1 GB) | Conservative pilot posture; raisable later without migration (W6). Per-file 25 MB stands. Contract verification in W6 narrows the production-v1 MIME set to PDF/TXT/MD because the ingestion platform accepts only PDF and plain text; Markdown is sent as `text/plain`. DOCX/PPTX remain a future platform-contract expansion and must not be offered as a broken flow. | +| Q9 | GrowthBook cohort shape | Broader pilot: courses opt in via an external Microsoft Forms form (faculty, course, use-case description, AI-Buddy-pilot participation — if participating, AI Buddy pays the AI cost, otherwise the course pays itself); the user then enables the course on GrowthBook. The gate covers **tutor chatbots only**; the other AI features are public beta for anyone with Catalyst at UZH. User-only flag administration; soft kill-switch = disable ingestion dispatch + hide attach UI, already-ingested content keeps serving | W8 rewritten to this shape. The Forms front door lives outside Klicker; Klicker-side work is the GrowthBook gate + kill-switch semantics only. | +| Q10 | Branch topology | Superseded 2026-07-26: `kb-poc` / [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) is the v1 integration line into `v3-ai`. [PR #5182](https://github.com/uzh-bf/klicker-uzh/pull/5182) merges into `kb-poc`; W2+ continues there. The roadmap branch remains review history and is not rebased or force-pushed. | Keeps the plan and implementation in one PR. The older full-scale [PR #5078](https://github.com/uzh-bf/klicker-uzh/pull/5078) remains a read-only source for selective reimplementation of useful UI and behavior; it is never merged wholesale. | +| Q11 | Legacy static course chatbots (informational confirm) | Confirmed: untouched throughout v1 until the gated `chatbot_id`→`kb_id` migration (program R10.4); lecturers keep current behavior | Informational; no roadmap change. | + +### Amendments (2026-07-29) + +- Decision: GrowthBook is not yet available; the W8 GrowthBook gate is deferred. Interim gating uses the existing per-lecturer `User.privatePreview` flag (already gating the chatbots nav on this branch) plus a deployment env kill-switch on ingestion dispatch. This supersedes the gating mechanism of ruling Q9 and the "Feature gating via GrowthBook" fixed-constraint line for the pilot window only; the Q9 course-cohort model (Forms opt-in → course enablement) resumes when GrowthBook lands and replaces the interim gate. +- Consequence: interim gate granularity is per-lecturer-account, not per-course. Accepted for the private-preview phase; course-level cohorting is explicitly deferred, not dropped. +- Evidence: `packages/prisma/src/prisma/schema/user.prisma:111` (`privatePreview`), `apps/frontend-manage/src/components/common/Header.tsx:59-69` (chatbots nav already gated), `packages/graphql/src/schema/mutation.ts:1639-1735` (KB mutations currently `asUserFullAccess` only, no preview check), `packages/graphql/src/lib/context.ts:38-47` (`ctx.user` carries JWT claims only — no `privatePreview`). +- Decision: the 2026-07-29 senior review findings ([review record](2026-07-29-kb-w1-w7-senior-review.md)) define the pre-pilot remediation set: P1-1 stranded UPSERT dispatch recovery, P1-2 URL-creation quota placeholder, P1-3 refresh-failure/mutation-success isolation. Secondary items (P2-1 poll bound, P2-2/P2-3 test hardening, P2-4 SelectField, P2-5 cascade documentation, P3-1 log fix) batch into the same package; P3-2 rate limiting stays backlog (pre-existing app-wide gap). + +## Production v1 Roadmap (Klicker side — finalized 2026-07-25) + +Work packages, dependency-ordered. Each lands as its own slice set with per-slice review per `$rs-sliced-development-workflow`; merges user-gated; everything behind the Q9 gating mechanism until platform gates pass. + +| Pkg | What | Depends on | Notes | +| --- | --- | --- | --- | +| W1 | **5182 finish + merge into `kb-poc`**: keep this roadmap on the `kb-poc` integration line (Q10, done); extract KG visualization + model selection into a parked PR (Q3); restore the per-resource Ingest button + `kb.ingestResource*` i18n keys (B1/M3, Q1); align webhook transition table (minor); fence/remove the `tsx --watch` monkeypatch; refresh stale Progress/screenshots and affected engineering-wiki pages to HEAD; then merge [PR #5182](https://github.com/uzh-bf/klicker-uzh/pull/5182) into `kb-poc` with the bridge intact (Q2 hybrid) | Rulings Q1/Q2/Q3/Q10 (done) | CI on 5182 was green before W1 changes and must run fresh afterward. M1 is NOT fixed here: resource-path fix deferred to W2, graph-path travels with the parked PR (W9). Nothing else lands on the line until W2 completes. | +| W2 | **Contract alignment** (immediately after W1, hard ordering per Q2): replace bridge transport with sync ingestion API client (`POST /v1/resources` family), including the authenticated blob Source Gateway; receiver speaks `OperationStatusEvent` + `X-Ingestion-*` verbatim (`extra="forbid"`, replay-window check); keep HMAC/correlation/reconciliation-sweep skeleton; drop second-Hatchet client + `runs.list` recovery; reconciliation cron polls the operations API instead (absorbs the M1 resource-path fix). Persist `resourceVersion` and `contentSha256` here because both are required inputs to the canonical create contract and safe re-ingestion cannot use a hard-coded version. | W1; platform contract stable | Program packages R5.1/R5.2 effectively collapse into W1+W2: 5182 supersedes the "re-author 5078" framing. The W2/W3 boundary correction was user-approved on 2026-07-26 after comparison with the canonical fixtures. | +| W3 | **Two-axis status + history**: add `activeResourceVersion`, `activeContentSha256`, and `errorCode`; complete replace-on-re-ingest behavior; add `KBIngestionRun`-style attempt history, minimal status UI (operation vs serving axes), and retry affordance | W2 | `resourceVersion` and `contentSha256` moved to W2 as contract prerequisites; salvage 5078 run-history shape here | +| W4 | **Chatbot binding + retrieval seam**: KB-level attach with one-enabled-KB-per-chatbot invariant (Q5), ES256 scope-token minting per chat request (`kb_id` claim), `ChatbotMCPServer`/`ChatbotMCPConfig` wiring (no schema change needed), citation-card fix (`KB.doc_query` → `KB_doc_query`), "no enabled KB" warning | W2; platform R4.3 tenant mount (external) | The actual lecturer-value moment: chatbots answer from KBs. `KBCourse` deferred per Q5. | +| W5 | **Delete/tombstone + retention cleanup** (Q7): soft-delete fence (`deletedAt`/`deletedById` from 5078), ingestion `DELETE /v1/resources/{id}` + tombstone handling, async hard cleanup including expired unconfirmed upload blobs/tickets, delete-guard while active ops | W2 | W1 intentionally creates the DB row only after upload confirmation; W5 owns abandoned-upload retention rather than expanding the bridge schema. | +| W6 | **Quotas + `kb_id` validation**: per-KB caps 100 resources / 500 MB (Q8) enforced at mutation layer + ingestion registry numbers; server-side `kb_id` validation (D-8, triggered by Q4=self-service) | W2 | Per-file 25MB + MIME allowlist stand as ruled | +| W7 | **Scale + UX pack**: pagination/cursor on KB + resource lists, aggregate counts, search/filter (server-side filter input), inspector panel, bulk actions with confirm dialog, per-row progress, async-wait messaging, linked-consumers panel | W1 (parallel to W2-W6) | Re-implement 5078 nuggets in 5182's package | +| W8 | **GrowthBook gating + pilot** (Q9): gate covers tutor chatbots only (other AI features public beta for Catalyst users); opt-in via external Microsoft Forms (faculty, course, use case, AI-Buddy participation → cost ownership), user enables courses on GrowthBook; user-only flag admin; soft kill-switch (stop dispatch + hide attach UI, existing content keeps serving); default-off; STG canary with the platform's 10-scenario synthetic journey evidence before any real lecturer traffic | W2-W6; GrowthBook availability (external); platform R4.4 | Forms front door lives outside Klicker | +| W9 | **KB-owned knowledge graph** (Q3=split; redesigned 2026-07-31, supersedes the chatbot-owned framing): re-home Patrick's parked KG work from `ChatbotKnowledgeGraph` to the KB, replace the direct Hatchet generation bridge with an external graph-generation service on a pinned manifest, and re-own both viewers. Five-layer stack: L1 platform-initiated refresh event; L2 KB-owned graph model + digest + reader package (carries [ADR 0009](../docs/adr/0009-kb-owns-two-derived-projections.md)); L3 build lifecycle + GraphQL API; L4 manage UI + lecturer viewer; L5 chat student viewer | W2 | Design grill complete 2026-07-31 (13 rulings, see [ADR 0009](../docs/adr/0009-kb-owns-two-derived-projections.md) and `docs/domain-model.md`). Parked [PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206) is Patrick's work and the base to evolve, not to rebuild — its 25 commits squash to ~5 layer-aligned commits, rebased onto `kb-poc`. [PR #5116](https://github.com/uzh-bf/klicker-uzh/pull/5116) stays open as a React Flow / package-pattern reference. No scheduled builds: each spends the lecturer's AI budget. Model allow-list becomes a config-mapped quality tier. | + +External (platform-track) dependencies to watch, not owned here: R1.1 `resource_active` gating, R1.2 partition wiring, R1.3/R1.4 blast-radius fixes, R4.3 Klicker tenant mount, R4.4 producer enable + Gap-D proof, D-2 Langfuse per-tenant project for Klicker. + +## Progress + +- [x] 2026-07-24: Three-agent review complete (5182 core, 5182 extras/contract, 5078 salvage); findings synthesized; grill agenda drafted; roadmap drafted pending rulings. Review worktree `trees/kb-review-5182` still present (removal needs approval). +- [x] 2026-07-25: Grill rulings Q1-Q11 recorded (see Decisions section) +- [x] 2026-07-25: Roadmap finalized from rulings +- [x] 2026-07-26: Roadmap moved onto `kb-poc`; Q10 corrected so [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) is the integration line into `v3-ai` +- [x] 2026-07-26: PR #5182 worktree attached to `feat/kb-poc-management-ui`; full pre-split head `9b5fc7af2` preserved on local branch `feat/kb-knowledge-graph-parked`; latest `kb-poc` roadmap merged into the implementation branch +- [x] 2026-07-26: W1 extraction complete — full KG visualization/model-selection head preserved on `feat/kb-knowledge-graph-parked` and published as draft [PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206); core extraction committed as `f4984229a` +- [x] 2026-07-26: W1 core fixes complete — repeated `PROCESSING` callbacks accepted (`dab6aa63e`); unsupported Hatchet internal-logger patch removed and both workers verified live without `tsx --watch` (`720fe43cd`) +- [x] 2026-07-26: W1 UI re-verified on `https://manage.klicker.feat-kb-poc-management-ui.localhost` — delegated login, URL resource creation, `FAST` selection, Ingest -> `QUEUED`, local `ingest-kb-resource` receipt, privacy-safe stub log, EN desktop and DE 375 px screenshots. The isolated environment has no real external ingestion endpoint, so the expected sanitized external-dispatch failure remains the already-documented deployment smoke prerequisite. +- [x] 2026-07-26: W1 security remediation complete — URL resources reject embedded credentials and local/private/reserved literal destinations at registration and again before external dispatch; shared utility, GraphQL, and Hatchet regression suites pass. External DNS/redirect egress enforcement remains a required W2/deployment gate before lecturer exposure. Abandoned-upload retention remains explicitly owned by W5. +- [x] 2026-07-26: W1 local finish gate complete — root `check:all` passed (24 typecheck tasks plus lint, formatting, and syncpack); focused utility, Hatchet, and real-PostgreSQL GraphQL suites passed (73, 48, and 48 tests); the pre-push production build passed all 22 tasks. Contract and maintainability re-reviews found no W1 blocker. Security re-review confirmed literal/numeric destination bypasses are closed and classified DNS/redirect enforcement as the documented external deployment gate, not a #5182 merge blocker. Opengrep was unavailable in the environment. NEXT: read back fresh #5182 CI, then stop for merge approval. +- [x] 2026-07-26: Fresh PR #5182 CI passed and the approved squash merge landed on `kb-poc` as `b66ae0107d05c455af4165b276a052b0740088c2`. The integration line then absorbed current `v3-ai`; KB and verifiable-credential Prisma/GraphQL surfaces were reconciled, KB package peers were aligned with Next 16/React 19.2, generated artifacts were regenerated, and W1 behavior was added to the current engineering wiki. The merge also exposed and fixed a Next 16 integration fault: shared locale loading now uses statically analyzable imports and the i18n package resolves `next-intl` against the line's single Next 16 peer context. The merged line passes root `check:all`, the utility suite (73 tests), the Hatchet suite (48 tests), frozen-lockfile installation, and the full production build (22/22 tasks). The local GraphQL Docker harness cannot run inside the repository-mandated devcontainer because that image has no Docker CLI; W1's fresh pre-merge real-PostgreSQL GraphQL run and PR CI remain the database-backed evidence for this unchanged integration. +- [x] 2026-07-26: W2 contract alignment complete at `0d2853929`. Canonical dispatch, authenticated source streaming, operations-API reconciliation, strict signed status ingestion, terminal-race handling, generated artifacts, deployment configuration, wiki/skill updates, focused suites, real-PostgreSQL GraphQL tests, Helm rendering, EN/DE desktop/mobile browser verification, root `check:all`, production build, final contract/maintainability/security reviews, and fresh PR CI all pass; Playwright passed build plus all eight shards. GitGuardian incident `1509424` remains a separate administrative classification for local-only base-branch test credentials (`Skip: test credential`); no secret remediation or history rewrite is required. +- [x] 2026-07-27: W3 complete at `89f2a4eb9` and `07bd7f21a`. Contextual Ingest/Retry/Re-ingest controls, separate latest-operation and active-serving status, an owner-checked five-attempt history outside the two-second detail poll, localized stable-code failure detail, and an SSR detail route for arbitrary KB ids are live on `kb-poc`. The PostgreSQL GraphQL suite passes 25 tests, focused Hatchet suites pass 28 tests, root `check:all` and the 22-task production build pass, and the generated API plus wiki validator are current. Delegated-login browser verification covers EN/DE desktop/mobile and replacement success/failure; retrying while history remains expanded advances the failed candidate to version 4, retains active version 1, refreshes the history immediately, and preserves the existing failure-message selector. Final contract, maintainability, security, and committed-range crosschecks are clean. Fresh PR CI passed in runs `30253081057`, `30253081156`, `30253081201`, `30253081237`, `30253081275`, `30253081309`, and `30253082158`, including GraphQL and all eight Playwright shards. GitGuardian incident `1509424` remains the separate known base-branch test-credential signal. Browser target: `https://manage.klicker.kb-poc.localhost`. +- [x] 2026-07-27: W4 started from clean head `519aa2a14`. Current schema/runtime and the read-only #5078 implementation were re-audited against the approved doc-query contract. W4 will reimplement only the useful KB-level binding shape, enforce the one-enabled-KB invariant in PostgreSQL, derive retrieval scope exclusively from owner-checked database state, and mint a separate five-minute ES256 token per chat request. The existing participant HS256 token, legacy static course chatbots, `KBCourse`, KG resource curation, and the external retrieval service remain outside this change. Local stack verified at `https://manage.klicker.kb-poc.localhost`; the platform R4.3 tenant mount remains the explicit blocker for a live end-to-end retrieval canary, not for local implementation and contract proof. +- [x] 2026-07-27: W4 lecturer binding slice complete locally. `KBChatbot` now carries the typed KB/chatbot relationship, pair uniqueness, and a PostgreSQL partial unique index for one enabled KB per chatbot; attach/replace/detach services lock owned rows, provision only `doc_query` for tutor and explainer modes, and fail closed when the global KB server is not scope-token configured. The focused real-PostgreSQL GraphQL suite passes 31 tests, Prisma schemas are synchronized, GraphQL and both affected UI packages typecheck, and generated operations are current. Browser proof covers attach, replace with an explicit warning, detach, the chatbot no-KB warning, the linked-KB view, English desktop, and German mobile at `https://manage.klicker.kb-poc.localhost`. The independent slice review correctly found that enabling the scoped MCP configuration before the chat runtime understands `scope_token` would leave an unauthenticated path, so this slice is intentionally held for an atomic commit with the retrieval seam. +- [x] 2026-07-27: W4 retrieval seam complete locally. Every KB chat request now derives its binding from owner-checked database state, mints a separate five-minute ES256 scope token with `kb_id`, `chatbot_id`, opaque `sub`, and `jti`, and passes it only to the reserved KB MCP server; the participant token remains unchanged. The runtime rejects reserved KB configurations that do not use `scope_token`, and the seed reconciles tutor and explainer modes symmetrically for both existing and new configurations. Citation UI registration now matches the emitted `KB_doc_query` tool name. Verification passes 31 real-PostgreSQL GraphQL tests, 24 chat/MCP tests, `check:all`, the 22-package production build, Helm lint/template checks, documentation validation, an Opengrep diff scan, a real-database enabled/disabled seed probe, and independent contract/security/maintainability reviews. The live retrieval canary remains deferred solely to the external R4.3 Klicker tenant mount; no staging tenant, key, or platform state was changed. +- [x] 2026-07-27: W4 publication complete at `69c2a499a`. Draft [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) now documents the full W1-W4 branch and current browser evidence. Fresh CI passed formatting, lint, syncpack, types, GraphQL, build/compile, all eight Playwright shards and both status gates, and both fallback image builds. GitGuardian remains failed only on existing unrelated incident `1509424`. +- [x] 2026-07-27: W5 research complete. The canonical delete contract is `DELETE /v1/resources/{external_resource_id}` with bearer auth, a stable idempotency key, and the strict JSON body `{project_id, producer, resource_version, scope:{kb_id}}`; success is `202 {operation_id}`, delete operations reconcile with `expected_sha256=null`, and a completed tombstone serves no resource version or digest. The current hard-delete path loses that reconciliation state and performs blob I/O while holding row locks. The selective #5078 salvage is only its `deletedAt`/`deletedById` fence and default query filtering. W5 will preserve whole-KB deletion by extending the same pending-delete marker to `KB`, hide tombstoned records immediately, and keep local rows until external and blob cleanup succeed. +- [x] 2026-07-27: W5 plan committed at `8a061fcc5` and independently reviewed. The accepted review corrections serialize upload-ticket issue, upload confirmation, and URL-resource creation against parent-KB deletion; preserve hidden tombstones when post-commit queueing fails and let scheduled maintenance re-enqueue the same stable delete attempt; and atomically consume upload tickets after resource creation while returning an already-confirmed matching resource before ticket validation on retry. No ADR conflict was found. +- [x] 2026-07-27: W5 tombstone tracer bullet committed at `3b8f743e1`. Resource and whole-KB actions now retain hidden owner-attributed tombstones, use KB-first row locking, disable linked chatbot retrieval, create explicit `DELETE` runs, send the canonical idempotent external delete request, and reconcile only delete-fenced operations whose serving state is empty. Queue failures keep the same hidden attempt retryable. Focused verification passes 34 Hatchet/API tests, 50 real-PostgreSQL GraphQL/webhook tests, schema sync, and root pre-commit checks. Independent review found the deletion contract clean and identified the planned upload-ticket seam as the sole remaining threshold concern. +- [x] 2026-07-28: W5 is complete locally through `4cd1a836b`. Exact-expiry upload tickets, atomic confirmation, hidden owner-attributed tombstones, canonical external delete dispatch, correlated reconciliation, and bounded hard cleanup are committed. Review remediation now starts a fresh fenced attempt for both failed and superseded external deletes, isolates dispatch configuration failures from independent cleanup, rotates 32-row windows so retained failures cannot starve later work, serializes whole-KB deletion against chatbot replacement, and verifies the persisted KB scope before external dispatch. The focused Hatchet/API suite passes 46 tests; the real-PostgreSQL GraphQL, webhook, ingestion, and source-gateway suite passes 67 tests; root `check:all` passes 25/25 tasks; and the production build passes 22/22 tasks. Delegated-login browser proof covers the English desktop resource warning and background-cleanup toast plus the German 390x844 whole-KB warning and toast, using un-ingested local fixtures only; the four final images are under `project/screenshots/kb-w5-*.png`. The local stack had no external-ingestion API configuration, and its Hatchet log confirmed the local delete dispatch was retained rather than accepted externally; no external ingestion state was mutated. Contract, security, and maintainability re-reviews are clean after all findings were fixed. Opengrep 1.25.0 remains unavailable for this slice because its `auto` rules require a third-party network fetch that was not authorized. NEXT: commit the final plan and screenshot evidence, run the committed full-branch crosscheck, push, update draft PR #5174, and read back fresh CI without merging. +- [x] 2026-07-28: W5 publication complete at `281cac9c8`. The final documentation and screenshot evidence were committed, draft [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) was refreshed, and fresh CI passed build, static checks, GraphQL, and all eight Playwright shards. GitGuardian remains the known unrelated incident `1509424`. +- [x] 2026-07-28: W6/W7 research complete. Klicker currently enforces the 25 MiB file boundary and its deployed ingestion bridge supports PDF and plain text (`.txt`/`.md` map to `text/plain`); the current external registry does not support a registered per-project `kb_id` set, per-KB resource counts, or per-KB byte totals. Therefore Klicker can close its mutation, ownership, reservation, pagination, and UX contracts in this branch, while platform-side D-8 validation and registry quota enforcement remain an explicit external gate. The older #5078 branch is useful only as an information-architecture reference (metrics, filters, resource table, inspector); its demo settings, graph surfaces, client-only filtering, inaccessible modal behavior, and unconfirmed bulk deletion will not be copied. +- [x] 2026-07-28: W6 Klicker-owned quota enforcement is complete locally at `5e8b02f27` plus the W7 review-remediation range. Upload requests reserve exact bytes and one resource under the parent-KB lock; confirmation converts a matching reservation without double counting; URL resources claim observed bytes with `retained usage - previous claim + observed size`; retained tombstones and unconsumed tickets continue to consume quota until W5 cleanup removes them. The independent review closed legacy gaps: every unknown-size retained row now conservatively claims 25 MiB, legacy zero-byte tickets are revalidated under the KB lock, idempotent confirmation requires matching normalized metadata, and a quota failure transaction cannot commit without its exact correlated failed-run transition. Stable resource/storage-limit and upload-mismatch codes are localized. The supported self-service source set is PDF/TXT/MD, with Markdown sent as `text/plain`; DOCX/PPTX are not advertised. The focused real-PostgreSQL GraphQL suite passes 47 tests and the focused Hatchet/API suite passes 51 tests. Platform-side registered `kb_id` validation and registry quota enforcement remain the explicit external D-8 deployment gate; no external platform state was changed. +- [x] 2026-07-28: W7 API and lecturer workspace are implemented locally; final full-branch review and publication gates remain. Owner/filter-bound keyset connections page KBs by `(updatedAt, id)` and resources by immutable `(createdAt, id)`, with bounded page sizes, exact totals, server search/type/status filters, malformed/mismatched cursor rejection, and tombstone hiding. Exact catalog/detail metrics use six grouped queries rather than per-KB fan-out and distinguish visible, quota, reservation, unknown-size, cleanup, and linked-consumer state. Bounded bulk deletion locks one owned KB then sorted child ids, rejects unsafe selections atomically, and creates independently dispatched W5 delete runs after commit. The EN/DE UI now has catalog search/load-more, a metrics header, resource search/filters, accessible selection and confirmed bulk deletion, an inspector with lazy history and contextual actions, loaded-window active polling, indeterminate operation progress, and safe-to-leave/quota-release messaging. Delegated-login browser proof passed server-backed catalog and resource pagination/search, lazy inspector history, active-operation polling, selection persistence through polling, explicit two-resource confirmation, successful deletion with visible/pending-cleanup metrics changing from `23/1` to `20/4`, and German `390x844` rendering. Evidence is under `project/screenshots/kb-w7-*.png`. The current web-interface-guideline pass added autocomplete behavior and focus-visible native-select treatment within the design system's controlled-field contract. Focused verification passes 47 real-PostgreSQL GraphQL tests, 51 Hatchet/API/maintenance tests, and GraphQL plus KB UI package typechecks. Root `check:all` passes 25/25 tasks and the production build passes 22/22 tasks. NEXT: independent final reviews, committed slices, push, draft-PR refresh, and fresh CI read-back. +- [x] 2026-07-28: W7 final-review remediation is complete locally. The unbounded legacy `getUserKbs` field and misleading always-empty `KB.resources` field are removed; resource filtering now targets the actual latest ingestion run; equal-time runs use an id tie-breaker; and post-commit delete dispatch bookkeeping cannot turn a committed tombstone into a mutation error. The UI caps normal selection at 50, reports ambiguous deletion failures safely, distinguishes previous serving versions in both row and inspector views, and uses full-width indeterminate progress. Active polling now fetches the first page with `no-cache` before explicitly merging it into the accumulated connection, preventing the network response from overwriting loaded pages. A disposable 60-row delegated-login fixture remained at 60 across three polling cycles and a select-50/clear cycle; its third-page active row remained visible, the Processing filter returned exactly that row, and the inspector reported `Version 1 remains available`. The focused real-PostgreSQL suite passes 47/47, root `check:all` passes 25/25, and the production build passes 22/22. Updated browser evidence is `project/screenshots/kb-w7-review-fixes-en-desktop.png`. NEXT: commit this remediation, rerun the independent committed-range review, push, refresh draft PR #5174, and read back fresh CI. +- [x] 2026-07-28: The committed-range re-review of `24f1e2f3c..bef1b1f8b` found that retaining later pages was insufficient: later-page active rows did not refresh, action refreshes could still collapse the loaded connection, a stale poll could cross a filter change, and two 50-item selection edge cases remained. The follow-up now refetches the entire loaded cursor window page by page, generation-fences every refresh, uses the same path for uploads and mutations, replaces select-all deterministically with the first 50 available rows, and removes rows from selection when they become active. Resource cursors include the owner and every resource/count query reasserts the live owned parent relation. Browser proof transitioned a third-page row from Processing to Succeeded while all 60 rows remained loaded, replaced an out-of-window selection with exactly 50 rows, automatically cleared a row when polling observed it become active, and retained 60 loaded rows after adding a disposable 61st URL resource. The disposable row was removed. The focused real-PostgreSQL suite passes 47/47, GraphQL plus KB UI package checks pass, root `check:all` passes 25/25 tasks, and the production build passes 22/22 tasks. NEXT: commit, rerun the committed-range review, push, refresh draft PR #5174, and read back fresh CI. +- [x] 2026-07-28: The final standards pass found only two non-blocking P3 edges. The bulk-confirmation modal now closes automatically if polling removes its last selected row, and creation-triggered list-refresh failures are caught, logged, and shown through the localized resource-load error instead of becoming unhandled promises. Each creation refresh key is consumed once per KB/key pair, so later loaded-count changes cannot retrigger it. A disposable two-row polling fixture verified that the open modal disappeared, the selected row cleared, and its checkbox disabled when that row became active; the fixture was removed. Focused KB package and lecturer-app checks pass. NEXT: amend the final fix commit, obtain the clean exact-range ruling, push, refresh draft PR #5174, and read back fresh CI. +- [ ] Program roadmap §3a amended (R5.0 satisfied) — external `_local` artifact, done outside this repo on user request +- [x] W1 executed (KG split, 5182 fixes, wiki/screenshots refreshed, merge into `kb-poc`) +- [x] 2026-07-28: W6/W7 publication complete — this entry closes the stale `NEXT:` markers in the four 2026-07-28 entries above. Head `925eea6a8` (`fix(kb): preserve loaded resource state`) is committed and pushed, draft [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) is refreshed, and fresh current-head CI passes every check (formatting, lint, syncpack, types, GraphQL, builds, CodeQL, all eight Playwright shards and both status gates) except the known pre-existing GitGuardian incident `1509424`. +- [x] 2026-07-29: Read-only W1–W7 senior architecture/engineering review complete over `20a953251..925eea6a8` (six lenses, findings line-verified). No P0. Three P1s (stranded UPSERT dispatch after a crash between commit and enqueue; URL-creation quota missing the 25 MiB unknown-size placeholder; refresh failures masking mutation success in 7 of 8 UI handlers), five P2s, four P3s. Verdicts: architecture sound, engineering good, security clean with no new secret/PII exposure (GitGuardian red = pre-existing `1509424` only), safe to continue W8/W9 while draft. Full findings: [2026-07-29-kb-w1-w7-senior-review.md](2026-07-29-kb-w1-w7-senior-review.md). +- [x] 2026-07-29: User ruling — GrowthBook deferred (not yet available); W8 gating replaced by the interim per-lecturer `privatePreview` gate plus env kill-switch (see Amendments). W8-Interim slice plan added; execution pending approval. +- [x] W8-Interim package executed (slices 1-8: review record, P1-1/P1-2/P1-3 remediation, privatePreview gate + kill switch, workspace polish, test hardening, wiki + finish gate) +- [x] 2026-07-29: W8-Interim slice 1 committed (`891ae4120`, plan + review record). Slice 2 (P1-2) implemented: `createKbUrlResource` now pre-charges `sizeBytes: MAX_KB_FILE_SIZE_BYTES`; new real-PostgreSQL byte-boundary test (21st unknown-size URL rejected with `KB_STORAGE_LIMIT_REACHED`, 20th succeeds) and concurrent byte-quota race test (one placeholder of headroom, exactly one of two concurrent creations succeeds). Focused suite 49/49, GraphQL package typecheck clean (in-container). +- [x] 2026-07-29: W8-Interim slice 4 (P1-3 + P3-1) implemented: the seven affected handlers now use the two-block pattern (mutation try/catch with error toast + return; refresh best-effort with console.error only; success toast/close/reset on mutation success), with nested try/catch preserving the `finally` loading-state resets in the dropzone and `handleIngest`; the dropzone `console.error` now includes the caught error object. kb-management and frontend-manage typechecks clean (in-container). Browser proof deferred to the consolidated slice 4-6 pass. Open reviewer question: whether `awaitRefetchQueries: true` on the chatbot-binding mutations can reject an already-successful mutation. +- [x] 2026-07-29: Slice 2 review clean (no findings; reviewer independently confirmed boundary arithmetic and that both new tests fail without the fix); accepted simplification committed (`00bcc63cb`, shared `legacyUrlResources` seeding helper, suite re-green 49/49). Slice 4 review confirmed the `awaitRefetchQueries` question as a real P1-3 residue (Apollo 3.13.8 folds refetch failures into the mutate() rejection, verified in installed source); adjustment committed (`e766f4ed2`): dropped `awaitRefetchQueries` on attach/detach and extracted the shared `refreshAfterMutation` helper across all eight swallow sites. kb-management and frontend-manage typechecks re-green. +- [x] 2026-07-29: W8-Interim slice 3 (P1-1) implemented: `maintainKBResources` gains an UPSERT recovery branch mirroring `deletionRetryWhere` — selects `deletedAt: null, UPSERT, QUEUED, externalOperationId: null, ingestionAttemptId: not null, updatedAt <= now - KB_MAINTENANCE_INTERVAL_MS` (15 min, one maintenance interval; `updatedAt` is reset by every fresh dispatch claim), same batch offset/bounded-concurrency idiom, and re-drives `dispatchKBIngestion` with the SAME stored `ingestionAttemptId` (stable Idempotency-Key; `dispatchKBIngestion`'s internal CAS re-check no-ops on concurrent delete/re-ingest). Three new tests (retry-with-same-attempt-id, exclusion where-clause incl. young/in-flight/tombstoned/DELETE rows, double-sweep idempotency without run creation). Hatchet suites 43/43, package typecheck clean (in-container). +- [x] 2026-07-29: Slice 3 review clean (selection/CAS/payload-parity all verified against the claim transaction and `dispatchKBIngestion` guards; advisories only — no-backoff retry and mock-order fragility are precedented file-wide patterns); accepted simplification committed (`1a32d5c16`, shared `logInvalidRetryPayload`), suite re-green 14/14. +- [x] 2026-07-29: W8-Interim slice 5 (interim gate + kill switch) implemented: `knowledge-bases-item` nav gated behind `user?.privatePreview` (chatbots spread idiom); `assertKbPreviewAccess(ctx)` (PK `findUnique` on `privatePreview`, `KB_PREVIEW_ACCESS_REQUIRED`) as first statement of all 15 KB service entry points, always before transactions; `assertKbIngestionEnabled()` (`KB_INGESTION_DISABLED === 'true'`) refuses createKbUrlResource/requestKbFileUpload/ingestKbResource while reads/deletes/confirm/attach-detach stay live; env wired through turbo.json globalEnv, chart configmap conditional (default renders omit the var; helm template/lint verified), and commented-off stg/prd values lines; EN/DE `kb.previewAccessError`/`kb.ingestionDisabledError` with `KB_INGESTION_DISABLED` mapped in dropzone/URL-form/handleIngest. Tests: 15-entry-point non-preview rejection loop + kill-switch enable/allow test via vi.stubEnv; suite 51/51, graphql/kb-management/frontend-manage typechecks clean. Seeded `lecturer` retains `privatePreview: true`; grep found zero existing KB e2e coverage to regress. +- [x] 2026-07-29: Slice 5 review — one accepted finding (75, gated_auto): `kb.previewAccessError` was defined but never consumed, so a non-preview lecturer hitting a bookmarked KB URL saw the generic load/notFound error; the workspace list and detail error branches now map `KB_PREVIEW_ACCESS_REQUIRED` to the purpose-built message. Accepted simplification: kill-switch test aligned to the file's blocked-calls loop convention. DEFERRED (50, manual, product call, recorded default: out of interim-gate scope): `getChatbotsInfo` surfaces a bound KB's id+name to its own owner without the preview gate — own-account metadata only, reachable only via direct API for bindings created while preview-enabled; revisit with the GrowthBook-based gate. Review otherwise verified all 15 entry points wired, guard-before-transaction everywhere, delegated logins share one User row (inheritance holds by construction), kill-switch env read at call time, ticket-bounded confirm posture, and configmap checksum pod-roll on toggle. Suite re-green 51/51. +- [x] 2026-07-29: W8-Interim slice 6 (P2-1 + P2-4) implemented: the 2s active poll now refetches only the first page plus pages known to contain active rows, using per-page keyset cursors captured by the last full walk (established by `refreshLoadedResources`, extended by `loadMore`, reset on kb/filter change), with self-healing fallback to a full walk on missing bookkeeping, absent connection, or unexpected non-tail page length (concurrent-delete signal). All five W7 invariants preserved (later-page active refresh, no loaded-window collapse, generation fencing after every await, selection invalidation via the shared updateQuery path, no-cache first-page merge). Both native select filters replaced with design-system `SelectField` (Radix forbids `''` item values → `FILTER_ALL_VALUE` sentinel at the field boundary only; state/query semantics unchanged; data-cy preserved via the forwarded `data` prop). kb-management + frontend-manage typechecks clean. Browser proof scenarios queued for the consolidated pass. +- [x] 2026-07-30: Slice 6 review remediation complete locally: bounded polls reject shifted cursor windows before merging, fall back on cursor drift or any loaded-page length mismatch, refresh `totalCount` from page 0 without corrupting tail `pageInfo`, and run a full loaded-window walk every tenth tick so untracked-page and connection-metadata staleness is bounded to 20 seconds. Tail-page splices use the observed page length. kb-management and frontend-manage typechecks plus the focused Prettier check pass; consolidated browser proof remains queued. +- [x] 2026-07-30: W8-Interim slice 7 implemented locally: the source-gateway suite now proves the real PostgreSQL filter for exact version, live tombstone state, BLOB type, digest presence, and QUEUED/PROCESSING status; the gateway-key test documents its intentional system-to-system trust across owner containers rather than claiming caller-owner authorization. Review discovery added `deletedAt: null`, so a tombstoned resource returns 404 before Blob Storage access. Direct `resolvePublicIPv4` coverage exercises loopback/private/link-local/IPv6 rejection and public-IPv4 acceptance without external network, and catalog metrics are verified across three differently composed KBs. GraphQL and Hatchet typechecks, focused Prettier, 67/67 targeted real-PostgreSQL GraphQL tests, and the complete 61/61 Hatchet suite pass. Independent slice review and simplification remain queued. +- [x] 2026-07-30: Slice 7 independent review clean with no threshold findings and no ADR conflicts. The accepted simplification removed a duplicate nested mock reset, localized one-use Hatchet/EventEmitter fixtures to setup, and inlined three invariant resource fields without weakening the clause-specific database proof. GraphQL typecheck, focused Prettier, and the targeted 67/67 real-PostgreSQL GraphQL tests pass again. The required Opus reviewer was unavailable (`Not logged in · Please run /login`); one native read-only reviewer and one separate native simplification agent provided the fallback gates. +- [x] 2026-07-30: Consolidated W8 browser proof passed the interim preview gate (enabled lecturer navigation; non-preview navigation hidden and direct-route denial), EN desktop and DE `375x812` workspace/denial copy, KB create/delete, URL-resource creation, chatbot attach/detach, and design-system resource filtering by pointer and keyboard. A continuous 24-second in-page sampler kept all 40 loaded rows while the page-two active row remained visible and both page-zero and active-page GraphQL polls ran, crossing the 20-second full-walk bound. This pass exposed and fixed two polling races: background promise-only reads now use `ApolloClient.query` instead of a render-driving lazy-query hook, and refreshes read the latest committed loaded count. Focused Prettier plus KB-management and lecturer-app typechecks pass. The safe TXT fixture upload could not be claimed: agent-browser reported upload success but never fired the application input handler, and no upload ticket or BLOB resource was created. Separate-process status/delete transitions also invalidated the inspected target (`CDP error (Runtime.evaluate): Inspected target navigated or closed`), so this pass does not claim those timing-sensitive transitions; the earlier committed W7 proof remains their evidence. All disposable KB/resource fixtures were removed and both test-account locales restored to English. `project/screenshots/kb-w8-*.png` covers the access, locale, and mutation-toast states; the filter interaction and 24-second polling sampler are live browser observations without a committed screenshot or trace. +- [x] 2026-07-30: W8 local finish-gate follow-up completed through `d9e3e1b36`. Focused GraphQL real-PostgreSQL suites pass 93/93, Hatchet passes 61/61, root `check:all` passes 25/25, and the production build passes 22/22. The final maintainability review found a load-more race with in-flight polling; `00ab85bc7` fences stale refresh generations and `d9e3e1b36` queues one full reconciliation so mutation refreshes are not dropped during load-more. The committed-range crosscheck also corrected slice 7's stale foreign-owner wording to the accepted tenant-wide gateway-key contract. A fresh upload attempt did fire the real dropzone handler, but the sanitized GraphQL response stopped at `Blob storage is not configured`; no upload ticket/resource was created, and the disposable KB was deleted. The full upload happy-path proof therefore remains blocked by local environment configuration, W8's completion checkbox stays open, and publication/CI are still pending. The user explicitly skipped the final security check for this run. +- [x] 2026-07-30: W8 publication read-back completed for pushed head `885622a2b`: [draft PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174) remains draft into `v3-ai`, its title/body/screenshots now cover W1–W8-Interim and the explicit upload/security boundaries, and CodeQL passes for Java/Kotlin, JavaScript/TypeScript, and Python. GitGuardian is the only failing check; its live metadata identifies the same pre-existing incident `1509424` in commit `d898fadb0` (`.devcontainer/docker-compose.yml`), outside the W8 range. This publication evidence does not close the upload blocker or authorize un-drafting, merge, or deployment. +- [x] 2026-07-30: W8's local upload blocker is closed with the managed DevPod's Blob-only Azurite 3.36.0 service. Browser-facing SAS URLs use the routed workspace origin while backend and worker SDK traffic uses the container-internal endpoint; production and staging retain the existing Azure URL fallback when no internal override is set. Post-start configures exact-origin CORS and `devrouter ensure` now proves 11 routes, including Blob Storage. A real delegated-login TXT upload through the hidden file input created a 105-byte BLOB resource with no remaining reservation; the disposable resource and KB were then deleted. Evidence is `project/screenshots/kb-w8-azurite-upload-success-en-desktop.png`. The utility suite passes 90 tests, focused real-PostgreSQL GraphQL suites pass 67 tests, Hatchet passes 61 tests, root `check:all` passes 25/25 typecheck tasks plus all six static gates, and the production build passes 22/22 tasks. The restored route's exact-origin PUT preflight returns HTTP 200 from `Azurite-Blob/3.36.0`; wiki validation passes with only 20 pre-existing hygiene warnings. No external ingestion endpoint, deployment, un-draft, or merge was exercised. The final security check remains explicitly skipped by user instruction. NEXT: commit and push this closure, refresh draft PR #5174, and read back fresh CI. + +### W4 Slice Plan + +1. **Lecturer binding tracer bullet** + - Add `KBChatbot` as the minimal KB-level join with an enabled flag, unique KB/chatbot pair, and a PostgreSQL partial unique index that permits at most one enabled KB per chatbot. + - Add owner-checked query, attach/replace, and detach services. Serialize replacement on the chatbot row, never accept a retrieval `kb_id` from the client, and provision the existing global `KB` MCP server configuration with exactly `doc_query` when a binding is enabled. + - Add a narrow owned-chatbot selector and linked-consumer controls to the KB detail page, including an explicit replacement warning, detach action, EN/DE copy, stable selectors, and a chatbot-details warning when no enabled KB exists. + - Verify migration shape, concurrent invariant behavior, GraphQL authorization, generated operations, UI type/format checks, and delegated-login EN/DE desktop/mobile browser flows. +2. **Retrieval seam tracer bullet** + - Add a dedicated ES256 scope-token minting path using the deployment-provided private key. Claims are `iss`, `aud`, opaque per-session `sub`, `iat`, `exp`, `jti`, `kb_id`, and audit-only `chatbot_id`; TTL is five minutes and the protected header carries the active `kid`. + - Resolve the enabled KB from database state inside the authenticated chat request. Add the KB MCP client only when that binding exists, send the scope token only to the `KB` server, preserve all existing participant-token behavior for other servers, and fail closed without logging token material when scope configuration is absent or invalid. + - Reconcile the dev seed to `scope_token`, disable participant/chatbot header passthrough for the KB server, expose nonsecret issuer/audience/key-id configuration through Turbo and Helm, and consume the private key only through the existing externally managed chat secret. + - Fix the citation-card tool name from `KB.doc_query` to the runtime name `KB_doc_query`; prove claims, key id, TTL, isolation, no-binding behavior, existing MCP auth compatibility, and citation rendering with focused tests. +3. **Finish gates** + - Update the affected data-model, GraphQL, chat-platform, frontend, and change-log wiki pages. + - Run per-slice review and simplification, focused PostgreSQL/GraphQL/chat suites, generated artifacts, root `check:all`, the production build, delegated-login browser verification with screenshots, final maintainability/security/branch crosschecks, and fresh draft-PR CI. + - Record the external R4.3 tenant mount as the only deferred live retrieval proof. Do not expose staging, create keys, mutate the platform tenant, merge PR #5174, or claim the external canary passed. + +### W5 Slice Plan + +1. **Tombstone tracer bullet** + - Add explicit delete-operation state, `deletedAt`/`deletedById` fences on resources, and a matching pending-delete marker on the parent KB so the existing whole-KB delete action remains safe across asynchronous child cleanup. Mirror the schema into analytics and add one migration. + - Make owner queries exclude tombstoned KBs/resources. Ticket issue, upload confirmation, and URL-resource creation lock the parent KB row and require it to remain live before writing; resource deletion locks the parent KB before the resource, and whole-KB deletion uses the same KB-first order before locking its resources and tickets. This ordering serializes every child-creation path against parent deletion. Deletion rejects any active create/update operation, disables affected chatbot bindings for a deleted KB, advances the resource version, creates a correlated delete run, and enqueues one bounded Hatchet delete batch only after the database transaction commits. + - Extend the ingestion API client with the exact canonical `DELETE /v1/resources/{id}` request. Dispatch with the stable delete-run idempotency key, persist the returned operation id, and reconcile only matching `delete` operations with `expected_sha256=null`; success requires an empty serving state. Keep failed tombstones hidden and retryable by the maintenance path without exposing external error text. + - If post-commit enqueueing fails, retain the hidden tombstone and opaque local failure state; scheduled maintenance re-enqueues the same delete run and stable idempotency key. Never roll back or expose deletion intent because queueing failed. + - Verify owner isolation, active-operation guards, concurrent delete claims, ticket/confirm/URL-create races against whole-KB deletion, exact request shape, idempotent dispatch/retry, callback and polling correlation, empty-serving cutover, queue-failure retention and maintenance retry, KB binding shutdown, and default query filtering against real PostgreSQL where applicable. + - Commit boundary: tombstone schema, API, worker, GraphQL, generated artifacts, and focused tests. +2. **Retention and hard-cleanup tracer bullet** + - Persist each upload ticket with its KB, blob name, and the same 15-minute expiry as its SAS credential. Confirmation first returns an already-created matching resource for retry safety; otherwise it requires a matching unexpired ticket and atomically consumes that ticket after resource creation. + - Add one single-run scheduled maintenance task. It removes expired unconfirmed blobs and ticket rows after a conservative 24-hour grace; removes confirmed BLOB storage only after a correlated external delete has succeeded; then hard-deletes the resource row and finally a pending-delete KB once no resource, ticket, or enabled chatbot relation remains. + - Keep cleanup idempotent and bounded. Blob/API failures retain the tombstone or ticket for the next run, and logs contain opaque ids only. Never scan or delete outside the exact owner container and ticket/blob name. + - Verify expiry boundaries, confirm/cleanup races, foreign-owner isolation, blob-before-row ordering, retry after storage failure, URL-resource cleanup, parent-KB finalization, and no cleanup before external tombstone success. + - Commit boundary: upload-ticket lifecycle, scheduled maintenance, storage cleanup, tests, and operator documentation. +3. **Lecturer proof and finish gates** + - Update EN/DE delete copy to explain immediate removal plus background cleanup without claiming external completion. Keep the existing resource and KB confirmation flows and stable selectors. + - Update the data-model, GraphQL, async-worker, frontend, and change-log wiki pages plus affected skills. + - Run per-slice review and simplification, focused real-PostgreSQL GraphQL and Hatchet suites, generated artifacts, root `check:all`, production build, delegated-login browser verification with EN/DE desktop/mobile screenshots, final maintainability/security/branch crosschecks, and fresh draft-PR CI. + - Do not mutate the external ingestion platform, create credentials, expose staging, merge [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174), or claim a live platform delete. The exact external request/callback fixtures and local state transitions are the W5 contract proof. + +### W6 Slice Plan + +1. **Concurrency-safe quota foundation** + - Add the ruled constants: 100 retained or reserved resources, 500 MiB retained or reserved bytes per KB, and the existing 25 MiB source limit. Store the requested byte reservation on `KBUploadTicket`, add one migration, synchronize the analytics schema, and make the upload/API contract match the verified ingestion bridge: accept PDF/TXT/MD, submit Markdown as `text/plain`, and remove DOCX/PPTX until the platform accepts those formats. + - Define quota usage as every retained `KBResource`, including hidden tombstones awaiting cleanup, plus every unconsumed upload ticket. Count known resource bytes plus reserved ticket bytes. This deliberately releases quota only when W5 hard cleanup removes the retained row or ticket. + - Serialize quota claims using the existing parent-KB row lock. File-upload requests reserve one resource and their exact declared bytes; confirmation atomically converts the matching reservation into a resource without double counting and rejects metadata mismatches. URL creation reserves one resource with unknown bytes; source preparation atomically applies `retained usage - previous resource size + observed size` under the same KB lock, correlated to the exact resource/version/attempt. A rejected URL size records the stable `KB_STORAGE_LIMIT_REACHED` failed-run code before dispatch. + - Return stable GraphQL error codes for resource-count exhaustion, storage exhaustion, and upload-ticket mismatch. Verify exact boundaries, retry idempotency, owner isolation, concurrent claims, tombstone retention, cleanup release, smaller/larger URL re-ingestion near the cap, URL-size races, and PDF/TXT/MD upload-through-source-preparation acceptance plus DOCX/PPTX rejection against real PostgreSQL where applicable. + - Commit boundary: migration, quota accounting and enforcement, ingestion source-size persistence, tests, generated artifacts, and affected wiki/skill contracts. +2. **`kb_id` validation boundary** + - Keep every Klicker-produced `kb_id` derived from an owner-checked persisted KB or resource relation; never accept it as free text from a lecturer-facing ingestion mutation. Add focused negative tests for mismatched KB/resource scope at source preparation, dispatch, status reconciliation, deletion, and source-gateway boundaries. + - Document the external D-8 requirement precisely: the ingestion service must validate `scope.kb_id` against a per-project registered set before self-service is exposed. Its current producer registry has no such field and cannot enforce the per-KB 100/500 MiB caps. Do not add a duplicate Klicker-side registry, mutate the external repositories, or claim the end-to-end D-8 gate is closed. +3. **Finish gate** + - Run focused API/Hatchet and real-PostgreSQL GraphQL tests, migration/schema sync, generation, root `check:all`, production build, independent contract/security/maintainability review, and committed-range crosscheck. W6 may be published as Klicker-complete with the external D-8 deployment gate still open; it must not be described as platform-complete. + +### W7 Slice Plan + +1. **Paginated query contracts and exact metrics** + - Add opaque keyset cursors ordered by `(updatedAt DESC, id DESC)` for owner-scoped KB connections and immutable `(createdAt DESC, id DESC)` for resource connections so status polling cannot move rows across page boundaries. Bind every cursor to the normalized search/filter set, reset pagination when filters change, bound page size, reject malformed or mismatched cursors, and return `items`, `pageInfo`, and exact `totalCount`. + - Add server-side normalized text search for KB name/description and resource title/filename/URL plus resource-type and operation-status filters. Keep full ingestion history in its existing bounded lazy query rather than nesting it into the polled list. + - Expose exact live and quota usage metrics without denormalized counters: visible resource count/known bytes, retained-or-reserved quota count/bytes, configured limits, and pending-cleanup/reservation differences. The bounded 100-resource domain keeps these aggregates simple and drift-free. + - Verify stable traversal across ties, no duplicates, malformed/foreign/filter-mismatched cursors, combined filters, owner isolation, tombstone visibility, a status update between page fetches, exact aggregates, and empty/final pages. + - Commit boundary: services, GraphQL connection/filter types, operations, generated artifacts, and focused tests. +2. **Resource operations at list scale** + - Add a bounded bulk-delete mutation for at most 50 selected resources. Lock the live parent KB before resources, reject foreign or active resources, create one independently retryable W5 delete attempt per accepted resource, and enqueue only after commit. + - Keep each row's real operation status visible. Use indeterminate progress for `QUEUED`/`PROCESSING`; do not invent numeric progress. Poll every loaded cursor page while it contains active operations, and preserve that loaded window across action refreshes. + - Verify all-or-nothing authorization and active-operation guards, deterministic locking, duplicate ids, bounded input, independent post-commit queue behavior, and unchanged single-resource actions. +3. **Lecturer scale and UX pack** + - Rework the KB catalog around server search and cursor-driven loading, with aggregate metrics and existing create/delete behavior preserved. + - Rework KB detail into an accessible resource workspace: metrics header; server-backed search/type/status filters; selectable resource rows; explicit bulk-delete confirmation; a resource inspector that exposes source metadata, operation-versus-serving state, contextual ingest/retry/re-ingest/delete actions, and lazy five-attempt history; per-row indeterminate progress; and clear background-ingestion/deletion/quota-release messaging. + - Preserve `KnowledgeBaseChatbotBindings` as the linked-consumers surface while making its count and empty state visible alongside the workspace. Provide all new copy in English and German, stable `data-cy` selectors, keyboard/focus-visible behavior, and usable desktop and 390 px mobile layouts. + - Reimplement only the useful #5078 interaction model. Do not add its fake settings, graph view, upload-zone claims, client-only filtering, or phantom actions. + - Browser target: delegated login at `https://manage.klicker.kb-poc.localhost`. Capture EN/DE desktop/mobile proof for search/filter pagination, inspector, selection and confirmed bulk deletion, active-row progress, quota messaging, and linked consumers. +4. **Finish gate** + - Update the domain, data, GraphQL, async-worker, frontend, testing, and change-log wiki pages plus affected task skills. + - Run focused real-PostgreSQL/API/Hatchet suites, generated-artifact cleanliness, package checks, root `check:all`, production build, current web-interface-guideline review, independent maintainability/security/spec reviews, committed-range crosscheck, delegated-login browser proof, push/read-back, draft-PR refresh, and fresh CI. Do not merge or deploy. + +### W8-Interim Slice Plan (2026-07-29 — review remediation + privatePreview gating) + +Scope: the 2026-07-29 [senior-review](2026-07-29-kb-w1-w7-senior-review.md) remediation set plus the Amendments-ruled interim gate. Non-goals: GrowthBook integration, Forms course-cohort, W9 KG re-landing (parked [PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206)), external D-8/R4.3/R4.4 gates, rate limiting (backlog), merging or un-drafting [PR #5174](https://github.com/uzh-bf/klicker-uzh/pull/5174). + +1. **Plan and review record** (this commit) + - Do: commit `project/2026-07-29-kb-w1-w7-senior-review.md` and this plan amendment; no code changes. + - Commit: `docs(kb): record senior review and plan W8-interim package`. +2. **Quota placeholder fix (P1-2)** + - Do: pass `sizeBytes: MAX_KB_FILE_SIZE_BYTES` in `createKbUrlResource`'s `assertKbQuotaAvailable` call (`knowledge.ts:1398`). + - Check: new real-PostgreSQL boundary test creating unknown-size URL resources through the service up to the byte cap (21st rejected with `KB_STORAGE_LIMIT_REACHED`); add the missing concurrent byte-quota race test mirroring the count-boundary pattern (`knowledge.test.ts:786`). + - Commit: `fix(kb): reserve unknown-size quota at URL creation`. +3. **Stranded UPSERT dispatch recovery (P1-1)** + - Decision (recommended): extend `maintainKBResources` with an UPSERT retry branch mirroring `deletionRetryWhere` — select `deletedAt: null, ingestionOperation: UPSERT, status: QUEUED, externalOperationId: null` rows older than a staleness threshold (≥ one maintenance interval) and re-dispatch the same attempt with its stable idempotency key. Rejected alternative: widening `monitorActiveKBIngestions`, which is correlation-based and would conflate "never dispatched" with "in flight". + - Check: `kbMaintenance.test.ts` cases for retry-after-crash-window, no interference with live QUEUED rows younger than the threshold, and idempotent double-dispatch via the stable key. + - Commit: `fix(kb): recover stranded ingestion dispatches`. +4. **Frontend refresh/success isolation (P1-3 + P3-1)** + - Do: apply the two-block pattern from `DeleteKnowledgeBaseResourcesModal.tsx:30-49` to the seven affected handlers; include the caught error object in the dropzone `console.error`. + - Check: KB package + manage typecheck; delegated-login browser proof of create/delete/upload/attach happy paths at `https://manage.klicker.kb-poc.localhost`. + - Commit: `fix(kb): keep mutation success distinct from refresh failures`. +5. **Interim gating + kill switch (Amendments ruling)** + - Do: gate the `knowledge-bases-item` nav entry behind `user?.privatePreview` (mirror chatbots, `Header.tsx:45-51`); add a shared `assertKbPreviewAccess(ctx)` guard (one PK-indexed `user.findUnique` selecting `privatePreview`) called at every KB mutation entry point and the KB queries; add env kill-switch (e.g. `KB_INGESTION_DISABLED`) refusing new dispatch (ingest/upload/URL-create) with a stable localized error while reads, deletion, and already-serving content stay live; wire the var through `turbo.json` `globalEnv` and Helm values. + - Decision (recommended): DB-read guard over JWT-claim extension — no token-shape change across apps, flag effective without re-login, trivially removable when GrowthBook replaces it. + - Check: negative real-PostgreSQL tests (non-preview user rejected on every KB mutation/query; kill-switch refuses dispatch, allows deletion); grep Playwright/Cypress specs for KB flows run by non-preview fixtures (seeded `lecturer` has `privatePreview: true`); EN/DE browser proof incl. hidden nav for a non-preview user. + - Commit: `enhance(kb): gate KB workspace behind private preview`. +6. **Workspace polish (P2-1 + P2-4)** + - Do: bound active polling to the pages containing active rows (fall back to first page) instead of the full loaded window; replace the two native `` filters + (`KnowledgeBaseResourceList.tsx:639-651`) vs the design system `SelectField` + used everywhere else, including this PR's `KnowledgeBaseChatbotBindings.tsx:128`. +- P2-5 (50, critical-class carve-out, manual/doc): `KB.owner … onDelete: Cascade` + (`knowledge.prisma:35`) — a future user hard-delete flow would cascade + KB→KBResource→KBIngestionRun and bypass tombstone cleanup, orphaning Azure + blobs + external state. Latent: no app code hard-deletes users today. + Document before any account-deletion/GDPR flow. + +### P3 findings + +- P3-1 (75, gated_auto): `KnowledgeBaseFileDropzone.tsx:82` + `console.error('Failed to upload KB file')` drops the caught error object. +- P3-2 (75, advisory): no rate limiting on the two secret-gated routes + (`app.ts:202,243`); pre-existing app-wide gap, relevant only on secret leak. +- P3-3 (50, advisory): single shared `KB_SOURCE_GATEWAY_KEY` authorizes any + owner's blob by resource id — all-tenants blast radius; likely the intended + system-to-system model (per-KB scoping enforced platform-side); confirm and + document. +- P3-4 (advisory, tests): no concurrent byte-quota boundary test (count + boundary has one at `knowledge.test.ts:786`); rotation "cannot starve" + proven only single-snapshot and `kbMaintenance` lacks `kbIngestion`'s + wraparound-fill; six-groupBy metrics never tested with multiple KBs per + page; webhook tests hardcode UUIDs (safe only while `singleFork: true`). + +## External platform gates (not branch defects) + +- D-8: platform cannot yet validate `scope.kb_id` per project nor enforce + registry quotas (documented in-repo, no duplicate registry added). +- R4.3/R4.4 gate any real retrieval canary; scope-token verification lives + outside this repo by design. +- Klicker retry-safety assumes the platform honors `Idempotency-Key` replays; + worth one staging fault-injection exercise. +- Correlation-persist failure after `acceptResource` + (`kbIngestion.ts:373-434`) can leave an accepted operation unrecorded + Klicker-side; platform delete-supersedes-upsert semantics decide if harmless. + +## Verified clean (highlights) + +Webhook CAS fencing (stale/replayed/foreign events cannot resurrect deleted +state); HMAC over raw bytes with `timingSafeEqual`, no auth oracle, byte-exact +canonical-JSON strict schema; SSRF checks at registration and dispatch with +per-hop DNS resolution + socket pinning (literal-encoding bypasses closed by +URL normalization); globally consistent lock ordering (KB→chatbot/resource, +sorted ids); no blob/HTTP I/O under row locks; deletion pipeline crash-safe +end-to-end; post-commit delete dispatch failures never surface as mutation +errors; one-enabled-KB invariant DB-enforced (partial unique index); cursors +strictly lexicographic, owner+filter-hash bound; generated artifacts +drift-free; analytics schema byte-identical; EN/DE 132/132 key parity; +enumeration-safe not-found behavior. GitGuardian: the branch adds no new +secret or personal-data exposure; the sole red check is pre-existing incident +`1509424` (base-branch local test credential). + +## Verdicts + +1. Architecture: sound; single blemish is the UPSERT/DELETE outbox asymmetry + (P1-1), closable within the existing pattern. +2. Engineering/maintainability: good; short fix list (P1-3, P2-1, P2-4). +3. Security: pass; advisories only, no new exposure. +4. Safe to continue into W8/W9 while draft: yes (W8 does not touch these + models; W9 anticipated via reserved webhook event types; land P1-2 before + W9 volume; KG may need an additive third `KBIngestionOperation` value). +5. Smallest remediation set: P1-2 one-liner + boundary test; P1-1 UPSERT + recovery; P1-3 seven-handler pattern fix. Everything else batches into the + next package. diff --git a/project/2026-07-31-kb-graph-kb-owned-stack-plan.md b/project/2026-07-31-kb-graph-kb-owned-stack-plan.md new file mode 100644 index 0000000000..a9f7da8606 --- /dev/null +++ b/project/2026-07-31-kb-graph-kb-owned-stack-plan.md @@ -0,0 +1,632 @@ +# W9 — KB-owned knowledge graph (stacked) + +Roadmap: [2026-07-24-kb-production-v1-roadmap-plan.md](2026-07-24-kb-production-v1-roadmap-plan.md), package W9. +Decision record: [ADR 0009](../docs/adr/0009-kb-owns-two-derived-projections.md). +Base: `kb-poc` @ `38625cbbf`. Worktree: `trees/kb-graph-stack`. Gate 1 approved 2026-07-31. + +## Provenance + +- Parked [PR #5206](https://github.com/uzh-bf/klicker-uzh/pull/5206) (`feat/kb-knowledge-graph-parked` @ `9b5fc7af2`) is **Patrick Louis Aldover's** work — all 25 commits in `b4a99893c..9b5fc7af2`, 2026-07-20 to 2026-07-22. It is the base to evolve, not to rebuild. Preserve the branch and PR untouched until the stack validates against it. +- [PR #5116](https://github.com/uzh-bf/klicker-uzh/pull/5116) (`codex/falkordb-chatbot-graphs`, `packages/falkordb`, React Flow drawer) stays open as a reference for its visuals and package patterns. Not merged, not closed. +- The branch forked at `6b86f9ec7`; `kb-poc` has since advanced 75 commits (W1–W8), rewriting the ingestion ledger, serving identity, deletion fencing, quota, and upload tickets underneath the KG code. + +## Rulings (2026-07-31 grill) + +- R1 One active build per KB, conditional-update claim. Repeat request for the building revision returns it. KB advancing mid-build: build finishes and publishes for its own revision; no cancel, no auto-follow-up, no blocked edits. +- R2 KB revision = digest over active serving set (`resourceId` + `activeContentSha256`), computed on demand, stamped per build. No column on `KB`. +- R3 New platform-initiated refresh event; handler creates a ledger row from the platform operation id and advances serving identity. Existing attempt-scoped guards untouched. +- R4 Build request pins `buildId`, `kbId`, digest, per-resource hashes. External service resolves to **processed documents** and fails on hash mismatch. +- R5 Webhook-primary + cron poll backstop. Configurable generous timeout fails a wedged build and releases the slot. Late success accepted only if digest still matches. +- R6 No scheduled rebuilds. Explicit lecturer request only — builds spend the lecturer's AI budget. +- R7 Named quality tier, mapped to models in server config, never in source. +- R8 Build controls in a dedicated card in `KnowledgeBaseDetail`. +- R9 Stale label is **lecturer-only**, on KB and graph views. Students get no staleness signal. (Narrows the 2026-07-30 "every graph-backed feature" ruling.) +- R10 One ADR, establishing `docs/adr/`. +- R11 Nothing lands in FalkorDB until complete; every build writes to its own recorded graph name and the published pointer moves only to a successful build. Versioning via GraphML export to Blob. +- R12 GraphML under a reserved prefix in the owner's KB container, excluded from resource quota. After a bounded grace period, `kbMaintenance` sweeps both GraphML and any graph name that is neither active nor published. +- R13 Lecturer viewer in KB workspace **and** student viewer in chat, both KB-owned. +- R14 Carry Patrick's implementation and attribution forward while preserving the parked branch and PR. Keep a buildable verbatim port as a provenance commit where possible; when the current base makes a raw port intentionally non-buildable, re-home it in the first buildable layer commit with the source refs recorded in the commit body and this plan. Never bypass hooks or mutate Patrick's branch. + +## Porting Patrick's 25 commits (R14) + +His commits interleave layers rather than arriving in layer order, so contiguous squashing cannot produce layer-aligned groups. Group by file path instead. The range splits into **56 added** files and **44 modified** files: + +- **Added files that remain buildable** are ported verbatim as Patrick-authored provenance commits — L2 is `068241088`. The adaptation to the KB-owned design lands as separate commits on top, so the two are never conflated. +- **Modified files are hand-merged inside each layer's refactor.** Taking his versions verbatim would revert W1–W8: `packages/prisma/src/prisma/schema/knowledge.prisma`, `packages/hatchet/src/kbIngestion.ts`, `packages/graphql/src/services/knowledge.ts`, and `packages/types/src/hatchet.ts` all evolved substantially after the fork. Generated artifacts (`ops.ts`, `ops.schema.json`, `public/*.json`, `public/schema.graphql`, `pnpm-lock.yaml`) are regenerated, never ported. + +| Layer | Added files ported | +| --- | --- | +| L2 | `packages/knowledge-graph/**` (15), `packages/types/src/knowledgeGraph.ts` | +| L3 | `packages/hatchet/src/kbGraphIngestion.ts` + test, `packages/graphql/src/schema/chatbotKnowledgeGraph.ts`, `services/chatbotKnowledgeGraphs.ts` + test, 6 `.graphql` ops | +| L4 | `packages/shared-components/src/knowledgeGraph/**` (6), `ChatbotKnowledgeGraphPanel.tsx`, `ChatbotKnowledgeGraphPreview.tsx` | +| L5 | `apps/chat/**` knowledge-graph route, page, components, server lib, 3 tests (10) | + +L3 provenance note: the L3 source port was first copied verbatim from Patrick's parked tip (`9b5fc7af2`), but the mandatory normal hook stopped at obsolete Chatbot types after the KB re-home. No `--no-verify` port commit was created. The first buildable L3 commit carries the KB-owned re-home and preserves the source lineage in its body; the parked branch remains untouched. + +Deliberately **not** ported (11 files), each explained at union validation: + +- `docs/superpowers/**` (6) — design docs for the rejected chatbot-owned architecture, superseded by ADR 0009 and preserved in PR #5206. +- `project/screenshots/*chatbot-knowledge-graph*` (4) — verification screenshots of the chatbot-owned UI, replaced by fresh ones at L4/L5. +- `packages/prisma/.../20260720150000_chatbot_knowledge_graph/migration.sql` — creates `ChatbotKnowledgeGraph`; the KB-owned migration is written fresh at L2 rather than created and then dropped. + +## Layers + +Each layer is independently functional, independently reviewable, green at its own tip. Drafts until actionable. No merge, un-draft, or deploy without explicit authority. + +**L1 `feat/kb-ingestion-refresh-event`** — new platform-initiated refresh event type in `packages/graphql/src/services/knowledgeWebhooks.ts`; ledger row from the platform operation id; serving-identity advance; attempt-scoped guards preserved for lecturer-initiated runs. Carries this plan, ADR 0009, and the roadmap W9 revision. No parked equivalent — new work. Ships value alone: keeps RAG current regardless of graph work. + +**L2 `feat/kb-graph-model`** — re-home `packages/knowledge-graph` and its schema from chatbot to KB. Patrick's 16 added files are ported verbatim as his commit (`068241088`); the re-homing lands on top: + +- **Schema.** Drop `ChatbotKnowledgeGraph`. Add `KBGraphBuild`, an append-only attempt ledger mirroring `KBIngestionRun` (client-supplied uuid id = idempotency key; `status`, `qualityTier`, `sourceContentDigest`, `graphName`, `graphmlBlobName`, `externalOperationId`, `externalStartedAt`, `statusMessage`, `errorCode`, `startedAt`, `finishedAt`, and retention claim timestamps). Each build also owns immutable source snapshots so later resource edits or deletions cannot change the manifest being reconciled. On `KB`, two plain uuid columns following the `ingestionAttemptId` precedent — `activeGraphBuildId` (the single-slot claim target for R1) and `publishedGraphBuildId` (what FalkorDB currently serves, R11). Enums `KBGraphBuildStatus` (QUEUED/PROCESSING/SUCCEEDED/FAILED/SUPERSEDED — timeouts are FAILED with an error code, keeping the enum aligned with `KBIngestionStatus`) and `KBGraphQualityTier`. Fresh migration; the parked `20260720150000_chatbot_knowledge_graph` is not ported. +- **Digest (R2).** Compute over the active serving set — `resourceId` + `activeContentSha256` of every non-deleted resource with an active hash — on demand, no column on `KB`. A pending replacement never suppresses the revision still serving RAG. +- **Identity.** `getKnowledgeGraphName(kbId, buildId)` → `klickeruzh:kb::`; each completed build records its own graph name, while `graphSession` and `getPublishedKnowledgeGraph` are re-scoped to `kbId`. These are the only three seams the reader exposes. +- **Publication rule (R11).** The parked rule returned `DIRTY` = not published. Invert it: a build whose digest no longer matches the KB still serves; staleness is a label, not an outage. + +`docs/domain-model.md` lands with this refactor, where its prose becomes true. Green at its own tip: `check`, `lint`, and the package's vitest run in-container. + +**L3 `feat/kb-graph-lifecycle`** — re-home Patrick's direct external-Hatchet graph workflow dispatch to the KB-owned pinned manifest (R4), direct status-poll reconciliation with timeout release (the external callback contract required to complete R5 is not available yet), GraphML export plus retention sweep of only unreferenced, non-active/non-published graph names in `kbMaintenance` (R12), quality-tier config mapping (R7); GraphQL status/rebuild/read ops re-pointed at the KB with KB-edit authorization. The companion LightRAG branch extends that existing external workflow to verify each pinned source hash after extraction and write the deterministic GraphML artifact; it does not introduce a new graph-generation service. + +**L4 `feat/kb-graph-manage-ui`** — move build controls from `ChatbotKnowledgeGraphPanel` to a dedicated card in `KnowledgeBaseDetail` (R8): status, stale label (R9), tier selector, rebuild with cost stated; lecturer viewer keeping Patrick's Cytoscape presentation and accessible DOM fallback. + +**L5 `feat/kb-graph-chat-viewer`** — Patrick's chat graph workspace and viewer, re-owned by the KB binding (R13). No staleness surface here (R9). + +## Verification + +- Per layer: `pnpm run check`, `pnpm run lint`, focused vitest, in-container per [klicker-verification-loop](../docs/index.md). Never run host-side `pnpm install`/`build`. +- L1/L3: webhook and reconciliation unit coverage including the refresh event, timeout release, and late-success digest mismatch. +- L4/L5: `agent-browser` against the stack's own devcontainer with before/after screenshots; delegated login (`lecturer`/`abcd`). +- Local FalkorDB: harvest the docker-compose service and env wiring from PR #5116 rather than inventing one. +- Union validation before any layer leaves draft: compare the stack against `b4a99893c..9b5fc7af2` and explain every deliberate difference. + +## Boundaries + +- Graph generation stays outside this repository. Do not reimplement it here. +- PRs #5174, #5206, #5116 stay untouched drafts. No merge, un-draft, deploy, or external-platform mutation. +- The W8 security review was skipped by explicit user choice; that is not authority for a broad W9 security assessment. Ask first. +- Public repo: no credentials, no real lecturer or student data. + +## M1 continuation — W2 package + +This section extends the existing W9 plan for the approved M1 package. It is +not a second plan or a new stack. The five W9 layers remain the topology; this +continuation adds the production-base reconciliation, graph-cost seams, and +current contract acceptance required by the roadmap. + +### Identity and current boundary + +- Branch: `feat/kb-graph-lifecycle` in `trees/kb-graph-stack`. +- Target: current GitHub `v3` ref `9a82e7fa63ba6b0f6b373470e3d6b77ae265d371` + (materialized in the read-only clone `/tmp/klicker-v3-review.THD0dw` on + 2026-08-16). +- Current implementation tip: `1245ba6102659f11d92d256dc4790a36daf5d779`. +- Current local comparison: 145 commits ahead and 81 behind the stale local + `origin/v3`. Shared Git metadata still rejects `FETCH_HEAD` writes, so the + exact target was reconciled through the read-only clone without changing + refs. +- Local `origin/v3` is stale at `2bcaddabe3bf3b39e23e71e7cf3eda7179f6291f`; + it is an ancestor of the exact target. The target range contains only + `9222929ad` (`chat`: structured video citations), and the scoped W2 paths + have no target delta. W2 therefore needs no rebase or merge for this target; + preserve the dirty primary checkout and this worktree's intentional + docs/ADR changes. +- W1 graph contract: provider-side schema and metered result identity come from + Catalyst W1. W2 owns Klicker consumer fixtures, reservation/settlement + behavior, and lecturer cost presentation. This is separate from Catalyst's + public chat-engine contract gate. +- Credential boundary: non-credential work may proceed. Credential-facing UI, + provider-bearing/model-backed runs, and beta activation remain blocked on the + generic credential architecture. + +### Goal and non-goals + +- Problem: W9 graph lifecycle and viewers exist locally, but current-base + compatibility, disposable migration proof, cost settlement, and complete + non-credential UX evidence remain open. +- Decision: Reconcile the current target before implementation, retain the + five-layer topology, and land quota state in the model/reader layer, + reservation/settlement in lifecycle, and cost presentation in the lecturer + layer. Do not create a sixth quota package. +- Non-goals: no credential UI or custody, paid model call, cluster/runtime + mutation, merge, push, PR update, production activation, or cleanup of + parked branches/dirty files. + +### Delegation map + +| Workstream | Owner | Dependency | Acceptance | +| --- | --- | --- | --- | +| W2-R active-plan and stack refresh | main | writable Git metadata | current target/base ledger, recovery ref, five-layer ownership and no duplicate plan | +| W2-A lifecycle and quota state | executor | W2-R and W1 graph fixtures | migrations, kill switch, opt-in, atomic reservation, idempotent settlement, focused tests | +| W2-B non-credential UI | executor | W2-A | cost/quota states and lecturer/student browser evidence without credential controls | +| W2-C five-layer integration | main | W1-B, W2-A, W2-B | contract conformance, union verification, and independently green layers | + +### Feature-wide test portfolio + +| Risk or behavior | Existing evidence | Test obligation | Primary seam | Distinct failure | Owner | +| --- | --- | --- | --- | --- | --- | +| Current base breaks W9 behavior | W9 focused tests and review evidence | extend existing | target-base revalidation and union diff | stale branch silently regresses current v3 behavior | W2-R | +| Migration or rollback is unsafe | Prisma migrations exist; application unverified | add new | disposable PostgreSQL migration and compatibility smoke | deployment migration fails or old app crashes on new schema | W2-A | +| Graph dispatch ignores kill switch or per-KB opt-in | lifecycle code and focused tests exist | extend existing | GraphQL mutation/dispatch boundary | disabled or non-opted-in KB starts a build | W2-A | +| Quota is oversubscribed or settled twice | no graph-cost ledger proof | add new | transactional reservation and terminal reconciliation | concurrent builds overrun quota or duplicate result double-charges | W2-A | +| Cost display misleads lecturer | partial UI cost label exists | add new | lecturer card with synthetic reservation/settlement states | estimate, balance, billing label, or actual cost is absent/wrong | W2-B | +| Unauthorized graph access | focused auth tests exist | extend existing | GraphQL and chat route auth seams | graph leaks across KB, owner, chatbot, or participant | W2-C | +| Patrick presentation regresses | component tests and parked provenance exist | extend existing | routed browser at mobile/desktop | Cytoscape or accessible fallback is unusable | W2-B/C | +| Graph archive is purged with serving cleanup | retention code exists | add new | maintenance policy over graph names and GraphML keys | durable GraphML is deleted while KB retention is active | W2-A/C | + +### Approved slices + +#### W2-R — Reconcile the current target and active W9 plan + +- Route: `main`. +- Do: Obtain writable shared Git metadata or an equivalent approved runtime; + create a recovery ref before any rebase/merge; compare current target to W9 + layers and parked provenance; classify new target changes touching W2 paths. + Keep this existing plan as the single W2 plan. +- Check: exact target ref, merge base, dirty-state ledger, no unrelated file + staged, and no topology mutation beyond the approved branch. +- Commit: plan/progress update first; no implementation before it. + +#### W2-A — Complete lifecycle, quota, and contract acceptance + +- Route: `executor` for bounded model/lifecycle changes after W2-R; main owns + schema and cross-repository seams. +- Do: Add/verify the graph-build kill switch and per-KB opt-in; apply the + W1-versioned fixtures; atomically reserve estimated maximum cost in integer + minor units; deny unaffordable dispatch; settle valid actual cost exactly + once by build ID; fail closed on duplicate, malformed, mismatched, or + over-reservation results; release unused reservation in the same transition. +- Check: disposable migration, rollback compatibility, concurrency and + duplicate/invalid terminal tests, graph publication/retention tests, and + focused GraphQL checks. +- Commit: `enhance(kb-graph): complete quota and lifecycle seams`. + +#### W2-B — Complete non-credential lecturer and student evidence + +- Route: `executor` for bounded UI changes; main owns browser evidence and + contract integration. +- Do: Render pre-dispatch estimate, remaining semester quota, worst-case + balance, billing label, post-settlement actual usage and cost; keep provider + credential controls absent; preserve stale lecturer-only state and student + graph availability semantics. +- Check: `agent-browser` against the current live branch at mobile and desktop + widths for empty, building, published, stale, failure, available, and + unavailable states; screenshots contain synthetic data only. +- Commit: `enhance(kb-graph): present quota and settlement state`. + +#### W2-C — Revalidate the five-layer integration + +- Route: `main`. +- Do: Regenerate affected Prisma/GraphQL artifacts, run union verification + against Patrick's parked range, and account for every deliberate difference. + Keep W5 parity/harness retirement outside W2. +- Check: focused GraphQL/knowledge-graph/chat/component suites, workspace + check/lint/build as available, migration evidence, browser screenshots, and + contract fixture readback. +- Commit: `test(kb-graph): verify integrated graph stack` for test-only deltas, + or the smallest accurate type for actual integration changes. + +### Review and finish gates + +- W2-A crosses data integrity and cross-system seams: run one simplifier and + one risk-selected slice reviewer in parallel on its exact committed range. +- W2-B is a substantive UI slice: run the simplifier and mandatory browser + verification; add the slice reviewer if it changes authorization or data + exposure. +- W2-C receives one integrated final reviewer after fresh verification. +- No PR/stack readiness claim, push, merge, deployment, or production proof is + made in this package until the pre-open gate is explicitly authorized. + +## Progress + +- 2026-07-31: Design grill complete (14 rulings). Gate 1 approved. Worktree `trees/kb-graph-stack` created from `kb-poc` @ `38625cbbf` on `feat/kb-ingestion-refresh-event`. ADR 0009 written, `docs/domain-model.md` KB section rewritten to the rulings, roadmap W9 row revised. +- 2026-07-31: `ed2cba55d` on L1 — this plan, ADR 0009, and the roadmap revision; later L1 implementation is tracked below. +- 2026-07-31: `068241088` on L2 — Patrick's `packages/knowledge-graph` (15 files) and `packages/types/src/knowledgeGraph.ts` ported verbatim, authored as him. Does not typecheck alone; identity still resolves through `ChatbotKnowledgeGraph`. +- 2026-07-31: The original L2 model refactor introduced `KBGraphBuild` + KB pointers, migration `20260731200443_kb_owned_knowledge_graph`, on-demand digest, per-build graph names, and the inverted publication rule. Its source tip passed workspace `check` (26/26), `syncpack:lint`, Prettier, and 67 package tests; revalidation after L1 propagation is pending. `apps/analytics` lint failed in-container on a `uv`/pandas source build — environmental, pre-existing, and untouched by this layer (its lint is `ruff`, which never reads `.prisma`). +- 2026-08-01: L1 implementation started after adopting the approved native stack (`kb-poc ← feat/kb-ingestion-refresh-event ← feat/kb-graph-model`). The focused real-PostgreSQL webhook baseline passes 17/17 in the managed DevPod. The broader `test:local` bootstrap remains unavailable there because its Docker-based harness cannot find a Docker client. +- 2026-08-01: L1 implementation complete locally. Signed `resource.content_refreshed` events append an operation-correlated terminal ledger row and advance only non-stale active serving state, leaving a concurrent lecturer attempt intact; repeat delivery is serialized and deduplicated. Focused real-PostgreSQL webhook coverage passes 20/20, `@klicker-uzh/graphql` typecheck passes, the full production build passes 22/22 packages, and the documentation bundle is OKF core-conformant. `pnpm run check:all` remains blocked only by the known unrelated analytics lint environment: `uv` cannot build `pandas==2.2.2` without a C compiler. +- 2026-08-01: Independent L1 review found that a platform-refresh ledger row could displace a concurrent lecturer attempt in the polled resource projection. The connection and its status filter now dereference `KBResource.ingestionAttemptId`; focused real-PostgreSQL `knowledge.test.ts` plus `knowledgeWebhooks.test.ts` pass 73/73 and GraphQL typecheck passes. +- 2026-08-01: Separate simplification review over `ed2cba55d..35988b8d2` found no actionable reduction. Gate 2 is approved and L2 is rebasing onto L1; revalidation is pending. +- 2026-08-01: L2 rebased cleanly onto L1. Its R2 digest now follows every non-deleted `activeContentSha256`, not the latest resource status, so a queued or processing replacement cannot omit its still-serving revision. The graph package passes 68 focused tests and typecheck; Prisma sync, workspace `check` (26/26), syncpack, Prettier, documentation validation, and the production build (23/23) pass. Workspace lint remains blocked only by the known analytics `pandas==2.2.2` C-compiler environment failure. Independent L2 review and simplification remain pending. +- 2026-08-01: Independent L2 review required the reader to reject a pointer to a foreign or non-successful build and Turbo to retain `KB_FALKORDB_*` configuration. The follow-up adds queued, failed, and foreign-pointer coverage (71 graph tests), records the pointer invariant in `klicker-data-model`, and aligns the per-build graph-name and bounded-retention contract. Focused tests, workspace `check` (26/26), documentation validation, and the production build (23/23) pass; separate L2 simplification remains pending. +- 2026-08-01: Separate L2 simplification review found no actionable reduction. The only unrun L2 proof is applying `20260731200443_kb_owned_knowledge_graph` to a disposable PostgreSQL database: the normal Docker-based local harness has no Docker client in this DevPod, so no shared development database was touched. +- 2026-08-01: L3 started. Patrick's existing direct external-Hatchet workflow is the dispatch seam being adopted, not replaced. Companion branch `feat/kb-graph-manifest-contract` in `/Users/rschlae/Git/klicker/lightrag/trees/feat-kb-graph-manifest-contract` will add the backward-compatible pinned-hash and deterministic-GraphML contract before the Klicker adapter is re-homed to KB ownership. +- 2026-08-01: L3 implementation is prepared on `feat/kb-graph-lifecycle`: KB-owned GraphQL rebuild/status/read operations, build-local source snapshots, pinned external Hatchet manifests, private-blob SAS URLs, timeout and late-success reconciliation, deterministic GraphML/FalkorDB retention, worker/chart configuration validation, and the LightRAG companion contract are in place. The adopted runtime seam remains direct Hatchet status polling because no authenticated inbound graph callback contract exists yet; the cron monitor is the reconciliation path. Focused Hatchet tests pass 73/73, knowledge-graph tests pass 72/72, workspace check/lint/build pass (26/26, 6/6, 23/23), Prisma sync, syncpack, AGENTS, and changed-file formatting pass; full-tree formatting still reports five unrelated generated `next-env.d.ts` files. Migration apply and live external integration remain unverified because this DevPod has no Docker-backed local harness and no live services were touched. +- 2026-08-01: L3 follow-up hardens the lifecycle boundary: timed-out and graph-retention windows rotate, retention claims the build before external deletion and uses explicit NULL-safe pointer guards, late success rechecks under a KB row lock and cannot race a cleanup claim, soft-deleting a KB clears its published graph pointer, and the v3 chart wires optional non-secret FalkorDB settings to both GraphQL and the general worker. The async-worker catalog now documents all six local workflows. Focused Hatchet coverage remains green after the hardening; migration application and live external integration are still intentionally unverified. +- 2026-08-01: Final L3 review found a digest race through resource-refresh webhooks. Late-success reconciliation now locks the KB and every live resource row before recomputing the pinned digest, while the worker reuses the locked KB projection and the shared GraphQL build select. Graph dispatch now fails Helm rendering without a FalkorDB host, and the general worker validates the complete FalkorDB config when graph integration is enabled. Hatchet tests pass 74/74; chart lint, configured/missing-host renders, changed-file formatting, and package typechecks pass. The webhook-primary R5 callback remains deferred because no authenticated external callback contract is available; direct status polling is the current seam. Migration application and live external integration remain intentionally unverified. +- 2026-08-01: The L3/L4/L5 takeover is reconciled on `feat/kb-graph-lifecycle`, preserving the parked branch as the merge parent. The pinned external builder contract at `f6cb38b` passes `uv run pytest` with 61 passed and 1 skipped. The target now has KB-owned graph config/rebuild/read GraphQL wiring, a dedicated lecturer card and Cytoscape viewer, and a student viewer resolved through the chatbot's enabled KB binding; the obsolete chatbot-owned graph GraphQL/UI/migration surfaces are not carried forward. Focused chat graph tests pass 41/41, the repository check passes 35/35, and affected lint passes with only five pre-existing chat warnings. Migration application, Docker-backed cross-repo stack/browser screenshots, live FalkorDB/Blob/Hatchet integration, deployment, and external publication remain unverified by design. +- 2026-08-02: Independent Git-level union verification confirms parked tip `9b5fc7af2` is an ancestor of `feat/kb-graph-lifecycle` through merge `bc6262f65`; both `b4a99893c..9b5fc7af2` and `b4a99893c..HEAD` pass `git diff --check`. The current DevRouter status reports the router and Docker unavailable, so disposable migration, cross-repository runtime, and browser screenshot evidence remain open; no local stack was started. +- 2026-08-15: The first M1 W2 correction pass completed through `c1358a2db`. The + committed W2 range includes quota/lifecycle seams (`567cad080`, + `a52d1cb18`, `a32648781`, `7ca0ff21f`, `18dce8eba`) and the final-review + correction (`c1358a2db`). The correction revalidates the kill switch, KB + opt-in, and complete reservation before external dispatch; fences + pre-accounting rows and cleanup-claimed late success; rejects zero-value cost + readiness; presents maximum cost and localized reservation status; isolates + opt-in refresh failures; and adds release/cleanup accounting coverage. + Prisma migration deployment to disposable databases, focused GraphQL + accounting (5/5), cost (6/6), Hatchet (16/16), GraphQL, Hatchet, and + kb-management checks, the root pre-commit suite (26/26), and staged secret + scanning are green. Browser evidence remains deliberately limited to the + synthetic empty/unconfigured EN/DE desktop/mobile state; enabled, active, + settled, held, published, stale, failure, available, and student-visibility + states still require a live-stack proof. Provider callback authentication + and live external graph execution remain outside this package. +- 2026-08-16: W2 final-review corrections are implemented in + `810ce4edb4768403a1326b0e400a1623136f2520`: valid metered non-success + results now settle actual usage without publication; the worker validates + every persisted reservation field and linked quota identity before the + external effect; W1 counters and aggregate usage are bounded to PostgreSQL + `INTEGER`; quota currency/limit drift is reported as unavailable while + historical build cost stays separate; and the generated GraphQL contract, + wiki, and task skills are synchronized. Real-PostgreSQL accounting passes + 7/7, pure contract/config passes 24/24, Hatchet passes 17/17, and the root + pre-commit suite passes 26/26. The implementation tip above is the review + base; this plan update is docs-only and does not change the package + boundary. Browser proof remains limited to the previously recorded empty, + unconfigured synthetic state, and live external execution remains outside + this package. +- 2026-08-16: The exact target `v3` ref was reconciled in a read-only clone: + stale local `origin/v3` `2bcaddabe3bf3b39e23e71e7cf3eda7179f6291f` is an + ancestor of target `9a82e7fa63ba6b0f6b373470e3d6b77ae265d371`; its only + intervening commit is the chat-only `9222929ad`, with no W2-scoped path + delta. W2 correction commit `22a13576e` adds the durable dispatch claim and + ambiguous-acceptance hold, locked matching/stale/superseded timeout + reconciliation, current-quota-currency presentation, and the disposable + migration plus real-PostgreSQL seam tests. The local hook passes 26/26, + focused Hatchet passes 80/80 across its four files, focused accounting + passes 10/10 on PostgreSQL, and the package checks pass. Browser evidence + remains limited to the previously recorded synthetic empty/unconfigured + state; no provider run, cluster access, merge, push, or deployment occurred. +- 2026-08-16: W2 final-review follow-up is committed in `045c01c11` and + `1245ba610`. The first commit wires the external Hatchet terminal-result + fetch and GraphQL settlement adapters into the backend, general worker, and + scheduled script compositions, closes dispatch-claim compensation races, and + validates late-success accounting before claiming publication. The second + retains the active KB build slot for an accepted-but-uncorrelated provider + run and refuses a second rebuild until recovery, cancellation, settlement, or + manual resolution; the regression is covered in the GraphQL integration and + Hatchet tests. Hatchet passes 83/83, the PostgreSQL-backed knowledge and graph + accounting tests pass 66/66, all four affected package checks pass, the root + hook passes 26/26, and staged secret scanning is clean. W3's corrected + transfer ledger is committed at `48ba5ff093439b61f5d5165f42ddd8287089c436`; + its focused suite passes 34/34 with Ruff, Pyrefly, catalog-sync, ShellCheck, + and shell syntax checks. The W3 repository's four unrelated pre-existing + dirty paths remain unstaged. Independent final review of W2 range + `eb4c0fd546b94a73068a4ae2e3226682f1103c85..e851e1deb` and W3 range + `06d55a4cc7b86bdda86adeab2238d5de56ad16c2..48ba5ff093439b61f5d5165f42ddd8287089c436` + passed with no findings. The reviews confirmed the terminal-result wiring, + dispatch and ambiguous-acceptance fences, late-success accounting, newest + attempt visibility, active-content digest contract, stable named evidence, + and mutation/retry semantics. Residual verification limits remain browser + proof, live provider execution, worker-process startup, migration + application, cluster access, merge, push, and deployment. +- 2026-08-16: W2-B/W2-C runtime verification was attempted without changing + code or external state. `devrouter status --json` reported the router, TLS, + shared network, and eleven-route repository configuration healthy, but the + exact `devrouter ensure .` retry reached the named worktree and failed while + Docker Compose tried to attach a referenced missing network. The managed + route returned `502`; no browser session or screenshot evidence was produced. + The failed exact runtime was stopped with `devrouter stop .`, and the second + ensure attempt reproduced the same environment blocker. Migration application, + worker startup, provider execution, cluster access, merge, push, and deployment + remain unverified; repair the exact DevPod/network and rerun W2-B/W2-C before + treating M1 as complete. +- 2026-08-16: The requested runtime retry found the real root cause and caused + a cross-workspace dev-database incident. A Docker daemon restart about six + hours earlier left this workspace's `postgres`, `redis_*`, and `mailhog` + containers attached to no Docker network, while the shared `devnet` resolves + the bare name `postgres` to four sibling workspaces' Postgres containers. + Repeated `devrouter ensure .` retries recreated the app container, and each + recreation re-ran `.devcontainer/post-create.sh`, whose `prisma migrate reset + --skip-seed --force` plus `db push` therefore executed against sibling + databases. Read-only inspection confirms the `klicker-prod` databases of + `trees/pr5134-b2-ui`, `trees/fix-chat-recovery-e2e-selector`, + `trees/chat-history-rail`, and the + `.claude/worktrees/klicker-uzh-ux-accessibility-32b832` worktree now carry + this branch's `20260816120000_kb_graph_dispatch_claim` migration and were + reset to this branch's schema; this workspace's own database was untouched + (its newest migration remains `20260815190114_kb_graph_cost_accounting`). + The exact runtime was halted with `devrouter stop .`; DevPod reports + `Stopped`, every project container is exited, and zero routes remain for + `feat-kb-ingestion-refresh-event`. One stale hatchet container (created + against the deleted compose network) was removed and recreated during the + retries; its named config/token volumes were preserved. No browser evidence + was produced; W2-B/W2-C verification stays blocked pending (1) explicit + approval to recreate this workspace's network-detached `postgres`, `redis_*`, + and `mailhog` containers before any further `ensure`, and (2) a ruling on + remediating the four sibling dev databases (each can rebuild via its own + worktree's post-create reset/reseed, which is itself destructive). +- 2026-08-16: Both approved repairs are complete. Each affected sibling + database was rebuilt from its own worktree code with `DATABASE_URL` pinned + to its own postgres container name (no DNS ambiguity): + `fix-chat-recovery-e2e-selector` and `chat-history-rail` at + `20260721193705_chat_message_rating`, the ux-accessibility worktree at + `20260706151837_add_verifiable_credentials`, and `pr5134-b2-ui` at + `20260815180000_live_quiz_pending_response_generation`; all four verify + with 5 users and 52 participants, and `pr5134-b2-ui`'s containers were + returned to their prior stopped state. Recorded contamination events + (UTC): rs-917c1 20:02, rs-0f6d6 20:05, rs-497d8 20:07, cl-7d302 20:18 and + 20:32. The 22:32 re-attempt proved `devrouter ensure` keeps appending the + localhost overlay on re-up (postgres `127.0.0.1:5432` conflicts with + devrouter Traefik's `0.0.0.0:5432`), which recreated the app and re-ran + post-create against a sibling once more before the runtime was halted. +- 2026-08-16: This workspace's stack was recovered under direct compose + control and is fully verified. Six stale-network containers were removed + and recreated with `docker compose -p default-fe-625ea` over only + `docker-compose.yml` + `docker-compose.devrouter.yml` (workspace env set, + no localhost overlay); the app's `DATABASE_URL`, `SHADOW_DATABASE_URL`, and + `LTI_DB_HOST` are pinned to `default-fe-625ea-postgres-1` via + `/tmp/kb-graph-pin-db.yml`. Our database was reset and seeded from this + branch (latest migration `20260816120000_kb_graph_dispatch_claim`, 5 users, + 52 participants); in-app `postgres` DNS resolves to exactly this stack's + default-network IP. Hatchet migrated and minted its client token; dev + processes run through the canonical `post-start.sh` with the devrouter + process helper copied from a healthy sibling container; all 11 routes were + reconciled via `devrouter app run`. Host proof: manage returns 200, the + API returns its expected CSRF 403 for an unauthenticated curl POST, and the + general worker executed `monitor-kb-graph-builds` successfully. Do not run + `devrouter ensure` for this worktree until the overlay bug and the shared + devnet bare-`postgres` alias hazard are fixed upstream; the runtime stays up + under an explicit lease for the approved W2-B/W2-C browser verification, + whose screenshots are the next open evidence. +- 2026-08-17: W2-C student-view evidence captured. Root cause of the earlier + hydration hang is fixed and recorded in the roadmap: the worktree shipped a + stale `allowedDevOrigins: ['**.klicker.localhost']` in + `packages/next-config/index.js` (predates upstream fix in #5248), which + blocked the dev HMR WebSocket for the four-label worktree host; Next's + app-router hydration decoder waits on the HMR debug channel, so the page + hung pre-hydration. Applied the upstream pattern + (`allowedDevOrigins: ['**.localhost']` in development) to the worktree + file only (uncommitted; commit needs approval). Stack restarted, chat + hydrates, all instrumentation restored. W2-C: minted a chat-guest JWT + (HS256, `CHAT_GUEST` scope, 14d) for testuser1 in the container from the + chat dev process env, set as host-only `chat_participant_token` cookie, + and captured + `w2c-kb-graph-student-graph-unavailable-en-desktop.png` (Benibot → KB + graph, guest session, desktop 1440×900): graph workspace renders the + graceful "Knowledge graph temporarily unavailable" state with Retry, + search, and zoom controls — expected with FalkorDB absent (partial + evidence; graph unavailability, not a healthy graph). Contract readback: + with the published binding enabled, plain-Node probe of + `getPublishedKnowledgeGraphForChatbot` resolves to + `10000000-...-0004` build `20000000-...-0004` (isStale false) and the + overview read fails with `KB_FALKORDB_HOST must be a non-empty value` + (503 source). Unbinding (isEnabled=false) makes the same probe throw + `KnowledgeGraphNotPublishedError` code EMPTY (409 source), and the + binding was restored to enabled afterward (verified). Caveat: the live API + degrades every graph error to 503 in this dev runtime because Turbopack + cannot load `@klicker-uzh/knowledge-graph` via `createRequire` + ("Cannot find module as expression is too dynamic"), so + `isKnowledgeGraphNotPublishedError` never matches in dev; the 409 branch + is unreachable in this environment but proven correct by direct probe. + +- 2026-08-17 (merge + relaunch): Integrated v3 through #5420 (tip + `3fd5259ad`, base `3872caee7`) into this worktree and merged into + `feat/kb-graph-lifecycle`. v3 does NOT contain the KB schema or + chat-graph UI — the feature remains branch-only; every conflict was v3-ai + reintegration lineage. Took v3 side for ~20 pure-lineage files + (shared-components questions, schema/resource.ts, seedChatbots, codegen + outputs, chatStore reset, tool-fallback, credits routes, assistant reset + base); unioned turbo.json, chat/hatchet/prisma-data package.json, + devcontainer.env, post-start.sh, app-sidebar.tsx (Guest badge + graph + switch + v3 header), chatStore.ts (v3 participation refactor + re-added + `setParticipationRequired` for KG), assistant.tsx (rebuilt from v3 base + + re-added `useChatGuestTokenBootstrap`, authedFetch disclaimers, + graphMode + ChatGraphModeSwitch + ChatKnowledgeGraphWorkspace), + docs/log/ -> v3 per-batch convention. Commit 1 `7882bccc3`. Kept our KB + docs sections and v3's course-duplication. During codegen, the merge + dropped our `enabledKnowledgeBase` field on Chatbot (I took v3's + resource.ts); orphan op `QGetChatbotsInfo` referenced it. Re-added the + Pothos type + field to packages/graphql/src/schema/resource.ts (resolve + `chatbot.enabledKnowledgeBase ?? null`; service already maps + `knowledgeBases[0]?.kb`) and regenerated — commit 2 `919d22f54`; + graphql build EXIT=0 (only non-fatal circular-dep warnings). NOTE: the + service returns enabledKnowledgeBase only when knowledgeBases is loaded — + verify chat/manage resolvers include .knowledgeBases. Combined pnpm + install kept OOM-killing the app container (its cgroup, not host RAM); + containers rebuilt via `docker start` and install succeeded with + `--child-concurrency=1 --network-concurrency=4`. Relaunched pipeline via + explicit post-start.sh (NEVER `devrouter ensure` — overlay bug). All + apps Ready (3001-3004, 3010), graphql build green, hatchet workers up. + Runtime left RUNNING under the current lease for user self-test (user + asked to keep it up). Expected dev-only errors: worker + `KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY must be configured` (no external + graph builder wired — no .local-kb-services.env, no UPSTREAM_OPENAI_API_KEY, + no FalkorDB), Langfuse no-op exporter warnings, LTI + MISSING_PLATFORM_URL_OR_CLIENTID. +- 2026-08-17 (KG data seeding): Confirmed there is NO prebuilt graph data to + seed. No *.graphml fixtures anywhere in the repo; local Azurite blob store + (default-fe-625ea-azurite-1) is EMPTY — the two SUCCEEDED KBGraphBuild rows + (kb-graph/synthetic-0004/0005.graphml) are record metadata only, with no + blob artifact. No FalkorDB container is running; graph data is written + directly to FalkorDB by the external Catalyst `kg-content-generation/ + lightrag_research` stack (bridge contract export_to/n_graph_name), not by + Klicker from GraphML. Klicker only reads (client.ts FalkorDB.connect) and + removes graphs/artifacts (kbMaintenance cleanup). ADR 0010's "GraphML + archive recovers FalkorDB" is an operational recovery concept (re-run the + external builder from archived GraphML), NOT an in-repo import API. So + seeding a real student-visible graph requires running the external + lightrag/FalkorDB stack (util/configure-local-kb-graph-builder.sh), which + is not set up here. The 5 synthetic KBs + SUCCEEDED builds exist in the DB + (verified) purely to render the empty/active/failed/published/stale UI + states; without FalkorDB the student graph view shows the graceful + "temporarily unavailable" state (W2-C evidence). + +## 2026-08-23 finalization amendment — single PR #5424 landing + +The user superseded the earlier five-PR delivery topology for this package. +[PR #5424](https://github.com/uzh-bf/klicker-uzh/pull/5424) is now the sole +integration line and the first KB PR to land on `v3-ai`. PR #5174 is its +ancestor, PR #5078 remains selective reference material, and the question +generation PRs remain under their existing owner. Question generation consumes +the canonical `KBGraphBuild` ledger after a build is succeeded and published; +it does not introduce a separate `KBGraphVersion` lifecycle or webhook. + +### Frozen integration refs + +- Clean worktree: `trees/rs/kb-v3-ai-finalization` on + `rs/kb-v3-ai-finalization`. +- Target: `origin/v3-ai` at + `3425cebb41c6f92a0c6be64e4325382205e9619c`. +- PR head: `origin/feat/kb-graph-lifecycle` at + `599ffcd155377f9a24e8688e16af685674d29682`. +- The frozen head is 158 commits ahead and one commit behind the target. Recheck + both refs before publication and stop for review if either changes. + +### Integration and corrective slices + +1. **I1 — integrate current `v3-ai`.** Merge the frozen target into the clean + finalization branch. Seven files changed on both sides. Four require manual + conflict resolution: preserve the local KB/Azurite setup alongside current + MCP and LiteLLM documentation in `.devcontainer/README.md`; keep the generic + dependency build plus the knowledge-graph changed-path trigger in + `.github/workflows/test-chat.yml`; retain auth, client-auth, and public-URL + entries in `packages/util/rollup.config.js`; and keep the current + `suggestions` naming in `playwright/tests/Y-manage-assistant.spec.ts`. + Inspect the three automatic merges in `.devcontainer/devcontainer.env`, + `.devcontainer/post-start.sh`, and `docs/chat-platform.md` before committing. +2. **B1 — restore backend ingestion routes.** Mount the authenticated GET source + gateway and raw-body POST ingestion webhook before end-user JWT middleware. + Add service-free route tests and CI execution for successful forwarding, + invalid input/content type, and generic failure responses. +3. **B2 — keep new-thread token scope stable.** Preallocate one UUID for a new + chat thread, mint the MCP scope token with that UUID, and persist the same ID + only after required-MCP availability succeeds. Preserve no-thread-on-MCP- + failure behavior and cover first-turn and existing-thread ownership cases. +4. **B3 — bound graph reconciliation.** Reuse the rotating monitor window for at + most 32 active builds, process at concurrency eight, and apply a ten-second + timeout to every provider operation awaited by the sweep. A timed-out call + leaves correlated state fenced for a later retry; independent builds continue. +5. **B4 — complete safe local-runtime wiring.** Selectively reimplement only the + relevant semantics from `8fa7ea50d` and `09ac131d8`: overridable loopback- + bound graph and ingestion ports; host-reachable Azurite source URLs through + `KB_GRAPH_BLOB_ACCOUNT_URL`; HTTP SAS URLs only for loopback or `.localhost`; + optional explicitly configured shared PostgreSQL ingestion state while + keeping SQLite as the default; and matching tests, docs, env, post-start, + ignore, and Turbo wiring. Do not copy model IDs or unrelated dirty runtime/UI + work. Preserve the user-owned dirty `trees/kb-graph-stack` unchanged. +6. **B5 — add graph interaction selectors.** Add stable `data-cy` hooks to the + graph search submit, search results, loaded nodes, relationships, and close + action without changing interaction behavior. +7. **B6 — align owner-only authorization guidance.** Clarify the GraphQL wiki + and matching API skill: KB aggregates are owner-only and enforce ownership + inside the service; `withPermission` remains required for shareable + aggregates supported by `PermissionCheck`. Do not widen KB sharing. Record + the behavioral/documentation changes in the dated wiki log. + +### Verification and publication boundary + +- Run each focused package suite, route tests, type checks, generation where + needed, root `check:all`, the production build, and `git diff --check`. +- Start the exact finalization worktree through DevRouter. Verify the affected + lecturer graph interactions and first-turn chat scope path with delegated + local login, capture required screenshots, then stop that exact runtime and + prove zero remaining routes and a stopped provider workspace. +- Run simplifier and risk-selected slice review on substantive B1-B4 commits, + then one final reviewer over the integrated verified branch. +- Publication authority covers normal push of the exact finalization commits to + `feat/kb-graph-lifecycle`, PR #5424 body/readback updates, reviewed Sonar thread + disposition, and exact-head CI monitoring. Force-push, merge, close, deploy, + external ingestion/platform mutation, and sibling PR changes remain withheld. +- GitGuardian incidents are classified separately using names/statuses only. + Stop if classification requires incident values, credentials, or new + administrative authority. The hardcoded private CIDRs in `publicUrl.ts` are + intentional SSRF denylist entries and are documented as Sonar false positives. + +### Progress + +- 2026-08-23: Ref freeze and independent plan review completed. The review + confirmed three merge blockers at the frozen head: the source and webhook + handlers are not mounted, a first-turn MCP token uses the request ID rather + than the eventual thread ID, and active graph reconciliation is unbounded and + sequential. The seven-changed-path/four-conflict distinction is recorded + above. No merge, push, PR mutation, deployment, or sibling worktree mutation + occurred during this planning checkpoint. +- 2026-08-23: Corrective commits `4319bb4b3` and `7c2763742` close the two + code findings from the final-package review. Graph and maintenance rotation + now uses page-aligned starts, so non-multiple totals do not lose the wrapped + portion of a 32-row window; graph-monitor and maintenance tests cover totals + 33 and 65. The development graph loader now uses a literal package + specifier, and a direct Node 24 development import reaches the expected + `EMPTY` publication error instead of failing module resolution. Exact Node 24 + focused Hatchet tests pass 99/99, chat typecheck passes, and the full Node 24 + production build passes 25/25 tasks. The host pre-push hook remains unusable + for this branch because the host runs Node 26 while the repository pins Node + 24; the equivalent container build is the supported evidence. Exact + authenticated browser screenshots remain open: the branch-local namespaced + route could not be reconciled because the shared route ledger/TLS probe + reports `curl (60) SSL certificate problem: out of memory`, so no browser + success or screenshot is claimed. Runtime teardown remains required after + the final runtime-dependent checks. +- 2026-08-23: Publication landed at head `30c3b0ee7` on + `feat/kb-graph-lifecycle` (fast-forward through `0387560a8`); PR #5424 + readback matches this exact head and frozen target `v3-ai` is unchanged at + `3425cebb4`. A formatting-only follow-up commit fixed the one drift CI found. + Browser evidence remains open for a different reason than first recorded: + the route itself serves correctly (curl reaches Traefik over HTTPS), but + macOS curl cannot validate the oversized multi-SAN certificate, and both the + in-app automation browser and sandboxed Chrome refuse every `*.localhost` + origin with `ERR_BLOCKED_BY_CLIENT`, so no authenticated screenshot could be + captured from this environment. Runtime teardown follows as the last step; + merge/close/deploy authority stays withheld per plan. +- 2026-08-23: Publication completed. Normal push landed + `feat/kb-graph-lifecycle` at `30c3b0ee7` (fast-forward from the frozen head; + target `v3-ai` unchanged at `3425cebb4`). The first push failed CI file + formatting on a biome drift in `kbHttpRoutes.test.ts`; fixed in commit + `30c3b0ee7` and re-pushed. Exact-head CI now passes 26/26 workflows with zero + failures (codebase check, types, graphql/chat/hatchet/MCP tests, Playwright + E2E, CodeQL, SonarCloud, gitleaks, stg image builds). Remaining open: + authenticated browser screenshots (TLS/curl route-reconciliation blocker, + documented above) and runtime teardown. +- 2026-08-23: Runtime lifecycle closed. The exact checkout + `/Users/rschlae/Git/klicker/klicker-uzh/trees/rs/kb-v3-ai-finalization` + (DevPod workspace `rs-kb-v3-ai-finalization`, provider docker) is + `stopped`: `devpod status` reports Stopped, all nine task containers + (app, hatchet, postgres, litellm, three redis, azurite, mailhog) exited, + and the devrouter route ledger carries zero routes for this workspace. + No deletion was performed. Merge/close/deploy authority remains withheld + per plan; the plan and log record the browser-evidence limitation instead + of claiming screenshots. +- 2026-08-24: Sol recommended merging the current target into the published + PR head instead of rebasing its 174-commit history. The merge reported 29 + manual conflict paths (the earlier inventory's “30” was a counting error), + and the resolutions preserve both the KB graph/MCP lifecycle and current + `v3-ai` feature-flag, assessment, and API changes. Generated GraphQL + artifacts and `pnpm-lock.yaml` were regenerated. Local merge commit + `703d026ac` has parents `8eee6f084` and `8e23a7bd1`; it is not pushed, so + PR #5424 remains at its published head. Root `pnpm run check` passed 29/29 + tasks; focused KB GraphQL, knowledge-graph, Hatchet, util, backend, and chat + suites passed (95, 72, 99, 5, 6, and 590 tests respectively), and + `pnpm run format:check` passed. The GraphQL `test:local` wrapper still needs + a Docker CLI unavailable inside the DevPod; direct focused suites pass. + `pnpm run check:all` remains blocked only by the analytics Python + environment needing a compiler to build pandas, and the full build reached + 24/26 tasks before the manage frontend was killed with exit 137 during page + data collection. The host commit hook also remains unusable under Node 26; + the equivalent Node 24 container checks passed. Browser screenshots remain + unavailable under the documented TLS/browser restriction. No push, PR + merge, close, deploy, or sibling worktree mutation occurred. +- 2026-08-24: Sol's final review identified four GrowthBook/ADR findings as + target-owned baseline concerns: they are present on `origin/v3-ai` but are + outside the `origin/v3-ai..HEAD` PR diff, so this reconciliation made no + sibling or target-branch changes for them. The review also found one actual + conflict-loss omission; the Prisma seed-reconciliation test row is restored + in the verification skill. PR-owned graph corrections are committed in + `c92d89cb9`: timed-out provider operations remain counted until their + underlying promises settle, and `127.example.com` is rejected as a fake + loopback host while real loopback literals remain available for local use. + The Hatchet package check and complete package suite pass (101/101), the + focused regression suite passes, and repository formatting plus diff checks + pass. The restarted runtime hit the existing TLS/route-readiness + `curl (60) ... out of memory` limitation and the shared lifecycle lock + prevented `devrouter exec`; equivalent checks ran directly through the + already-running task DevPod. No push, PR merge, close, deploy, or sibling + worktree mutation occurred. +- 2026-08-24: Sol's final integrated review passed cleanly at + `ce9001d448ead64ec6f7d16e1ec9601260f23c54`. It confirmed the merge parents, + all 29 conflict paths, the restored verification row, both PR-owned graph + regressions, generated-artifact/hash consistency, and the lockfile against + the current Next 16 package definition. The package is ready for normal + publication; the target-owned GrowthBook/ADR baseline findings and the + documented full-build and browser-proof limitations remain explicitly + outside this PR's local reconciliation. Merge, close, deploy, and sibling + worktree changes remain withheld. diff --git a/project/2026-08-10-kb-graph-production-roadmap.md b/project/2026-08-10-kb-graph-production-roadmap.md new file mode 100644 index 0000000000..deb84733e4 --- /dev/null +++ b/project/2026-08-10-kb-graph-production-roadmap.md @@ -0,0 +1,1104 @@ +# KB knowledge graph production roadmap + +Status: M1 execution approved and in progress. W3 is closed at the merged +delivery layer in both provider repositories. W1 and W2 remain at locally +reviewed evidence, and cross-package compatibility and external delivery gates +remain. + +Date: 2026-08-10 + +Roadmap owners: KlickerUZH KB product stack and Catalyst KG runtime + +Current roadmap branch: `feat/kb-graph-lifecycle` + +Parent plans: + +- [KB-owned graph stack](2026-07-31-kb-graph-kb-owned-stack-plan.md) +- [KB production v1 roadmap](2026-07-24-kb-production-v1-roadmap-plan.md) +- Temporary provider-side harness plan, used only as a coverage-transfer source: + `project/2026-08-10-kb-pgvector-graph-e2e-plan.md` in the + `data-ingestion` repository +- [KB-owned projection decision](../docs/adr/0009-kb-owns-two-derived-projections.md) + +Audience: an agent or engineer starting with no session context. Read the +parent plans and ADRs before starting a work item. Each work item becomes its +own full-path `$rs-sliced-development-workflow` plan, branch or existing draft, +verification loop, and MR/PR finish gate. + +## Goal + +Bring the KB-owned knowledge-graph capability from locally reviewed code to a +reliable, observable, reversible public lecturer beta. Preserve Patrick Louis +Aldover's authored work and visualizations, prove the full +Klicker → data-ingestion → pgvector → graph-worker → FalkorDB/GraphML path, and +make that proof repeatable in CI and deployed environments. + +## Out of scope + +- General availability beyond the first lecturer beta, including unrestricted + student visibility. +- Scheduled lecturer graph rebuilds; rebuilds remain explicit because they + consume AI budget. +- Replacing Patrick's graph implementation or rebuilding its visualizations. +- Replacing Hatchet, pgvector, FalkorDB, Blob storage, or doc-processing. +- Introducing the deferred authenticated graph callback; direct Hatchet status + polling remains the initial reconciliation seam. +- Multi-region graph serving, automatic graph-worker scale-to-zero, or a second + semantic-quality tier before the standard tier is production-proven. +- Any merge, push, deployment, secret mutation, cluster connection, or + production mutation merely because this roadmap is approved. + +## How to work on this roadmap + +### Rules for every work item + +1. Fetch the relevant remote and recheck the recorded base, branch, worktree, + CI, and dirty state before editing. +2. Use the existing branch or worktree named below when it still exists. Create + a repo-local `trees/` worktree only when the item explicitly + calls for one and after confirming `trees/` is ignored. +3. Write and review the item's sliced-development plan before implementation. + Keep that plan in the item's repository and ship it with the implementation. +4. Run Git on the host. Run Klicker pnpm, Prisma, build, and tests through the + existing DevRouter container. +5. Preserve unrelated dirty or untracked files. Stage explicit paths only. +6. Treat cluster access, deployment, paid model runs, production mutations, + merge, and publication as separate approval boundaries. + +### Repository verification loops + +| Repository | Working context | Normal verification | +| --- | --- | --- | +| KlickerUZH | `/Users/rschlae/Git/klicker/klicker-uzh/trees/kb-graph-stack`; existing `feat/kb-graph-lifecycle`; stack base `kb-poc`; eventual feature branches follow the existing W9 stack | `devrouter ensure .`; focused Vitest; `pnpm run check`; affected lint; production build; disposable Prisma migration; browser verification for lecturer and student viewers | +| data-ingestion | `/Users/rschlae/Git/ai/data-ingestion`; AI infrastructure owns this provider and its pgvector projection; existing `rs/kb-pgvector-graph-e2e` contains a temporary cross-system harness that must not make AI infrastructure the KG owner | Provider-contract and ingestion tests; `uv run poe check`; non-E2E ingestion suite; applicable ingestion E2E; `git diff --check` | +| kg-content-generation | `/Users/rschlae/Git/ai/kg-content-generation`; existing `feat/kb-graph-manifest-contract`; target `main` | graph workflow tests; full repository test/check command established by W1; worker image build; local Hatchet smoke | +| KB graph runtime | `/Users/rschlae/Git/klicker/klicker-uzh-catalyst`; private GitHub repository `uzh-bf/klicker-uzh-catalyst`; fetched `origin/main` was `37657ad2d` on 2026-08-15 and contains merged PR #2, merged PR #3, and later merged work; W1 starts from the latest `main` at execution time | Repository-native checks established by W1; manifest render and policy validation; server-side dry-run only when cluster access is separately authorized | + +## Current state + +Live refs reconciled on 2026-08-15. Recheck before implementation because +branch and remote state can advance. + +| Area | State | Evidence | +| --- | --- | --- | +| Klicker graph ownership and lifecycle | Implemented locally on `feat/kb-graph-lifecycle` at `b348adeea`; KB-owned schema, lifecycle, GraphQL operations, lecturer viewer, and student viewer exist; the branch was 133 commits ahead and 60 behind the current GitHub `v3` ref `3c64c9726` on 2026-08-15 (the local `origin/v3` ref is stale at `2bcaddabe`) | [W9 progress](2026-07-31-kb-graph-kb-owned-stack-plan.md#progress) and live ref comparison | +| Patrick's authorship and visualizations | Preserved. Parked tip `9b5fc7af2` is an ancestor through merge `bc6262f65`; Patrick-authored commits remain in history | W9 progress and `git log --format='%h %an %s'` | +| Klicker migration and live proof | Not complete | Disposable migration application, cross-repository runtime, current screenshots, and live FalkorDB/Blob/Hatchet evidence remain open | +| Graph builder contract | Implemented on `feat/kb-graph-manifest-contract` at `c4a4236d`; seven commits ahead of current graph `origin/main`; GitLab MR !3 is open, mergeable, and has a successful current pipeline, although its body cites stale commit and pipeline evidence | 38 focused graph workflow tests passed on 2026-08-10; live MR and ref readback on 2026-08-15 | +| Graph builder production status | Not ready | `lightrag_research/README.md` classifies the worker as research/non-production; GitLab CI prepares and builds images but has no blocking test job and publishes mutable `latest` on main | +| Temporary cross-system harness | Implemented in data-ingestion on `rs/kb-pgvector-graph-e2e` at `a0fcd4f2`; its safety and assertion work is reusable, but KG-system ownership belongs in Catalyst rather than AI infrastructure | 33 focused tests and `uv run poe check` passed on 2026-08-10; prior capable review passed `a2c72a7..c499c80` with no verified P1/P2 findings | +| Data-ingestion base | Needs semantic reconciliation | Branch was 16 commits ahead and 16 behind current `origin/main` on 2026-08-15; both sides touched workflow timeout/default files and tests; unrelated local changes remain preserved | +| Full local system proof | Not run successfully | No completed upload → ingestion → pgvector → graph build → GraphQL/FalkorDB/GraphML → cleanup run exists; the prior proof containers and port `18081` listener were absent on 2026-08-15 | +| Runtime ownership | Resolved; W1 start base selected | The Klicker team owns the runtime in private repository `uzh-bf/klicker-uzh-catalyst`. PR #2 and PR #3 are merged, and later work is also on `main`. W1 starts from the latest fetched `main`, integrates the complete `kg-content-generation` history after a sensitive-history audit, preserves both repositories' ancestry and Patrick's authorship, and refactors only in later Catalyst commits; see ADR 0016. AI infrastructure supplies services such as doc-processing but does not own this runtime | +| Production review | Not started | Full-path security, maintainability, and exact-final-outcome gates remain required per work item before publication | + +## Non-negotiables + +- **Do not re-litigate:** Klicker KB owns graph orchestration, published build + selection, and the two derived projections. The rationale lives in ADR 0009. +- **Do not re-litigate:** Klicker owns KB product state, authorization, + lifecycle, quota enforcement, and the lecturer/student experience. Catalyst + owns graph generation, FalkorDB operation, the GraphML archive, KG quality, + and KG-system E2E. AI infrastructure owns data-ingestion, doc-processing, and + pgvector; Catalyst and Klicker consume those services through contracts. See + ADR 0011. +- **Do not re-litigate:** Patrick's branches, authored commits, Cytoscape + presentation, and parked PR remain source history. Adaptations are separate + commits; squashing must never erase his authorship. +- **Do not re-litigate:** every graph build writes its own graph name and + GraphML artifact. Publication moves the Klicker pointer only after a complete + successful build. +- **Do not re-litigate:** direct Hatchet polling is sufficient for the first + production release. An authenticated callback is a later capability. +- A graph can continue serving after its KB digest becomes stale. Lecturer + surfaces show staleness; student surfaces do not expose it. +- Production images are pinned by immutable digest. Mutable tags can exist as + convenience aliases but are never deployment evidence. +- **Do not re-litigate:** completed GraphML files are the durable graph archive + and FalkorDB is a reconstructible serving projection. The first beta does not + require FalkorDB high availability or database backup as the recovery source; + see ADR 0010. +- Model-quality evaluation never substitutes for deterministic system checks, + and a non-empty graph never counts as model-quality evidence. Existing + testing and the production canary are sufficient to open the explicitly + labeled beta; curated quality evaluation gathers evidence during beta and + gates widening, general availability, and quality claims. See ADR 0014. +- The first deployment is disabled by default. New graph builds require a + dedicated kill switch independent of ordinary KB ingestion. +- The lecturer beta is publicly described as beta and self-service. Enabling it + requires explicit cost disclosure, an approved model credential path, a + verified external billing association where applicable, and an enforced + spending cap. +- Sensitive lecturer billing information stays outside the Klicker database. + For UZH-issued keys, the beta keeps the cost-account association in a + manually maintained spreadsheet. BYOK lecturers are billed by their own + provider. Klicker stores non-sensitive quota state and enforces graph-cost + limits for both paths. +- The public-beta feature flag grants a lecturer permission to enable knowledge + graphs; each knowledge base remains opted out until the lecturer explicitly + enables its graph capability. +- AI-provider credentials are a platform-wide capability shared by tutoring, + content generation, grading feedback, knowledge graphs, and future AI + features. KG/Catalyst is one consumer, not the definition of the problem. + Consumer applications retain only opaque credential handles and safe status; + custody ownership, submission, resolution, rotation, revocation, and runtime + use are isolated in the generic AI credential-management handoff. +- Klicker enforces a per-lecturer, per-semester monetary quota and a per-build + maximum. It reserves estimated cost before dispatch and idempotently settles + actual metered cost reported by Catalyst; see ADR 0013. +- Secrets enter workers and test jobs through the approved secret store or + protected runtime variables. Tokens, SAS query strings, DSNs, raw source + content, and credentials never enter roadmap, logs, reports, or commits. +- Production mutations and cluster changes need explicit approval at the time + of execution. This roadmap grants neither. + +## Known traps + +- **A data-ingestion rebase appears mechanical.** Cause: `origin/main` and the + harness branch both changed workflow timeout/default files and tests. Remedy: + compare intended semantics commit by commit, drop duplicate timeout work, + resolve against current shared defaults, and run the complete relevant suite. +- **The local stacks authenticate but the E2E still cannot run.** Cause: + Hatchet API health does not prove an ACTIVE worker has the required workflow, + and doc-processing has its own lifecycle. Remedy: rely on the harness's + workflow-specific preflight and exact doc-processing `status: ok` check. +- **Patrick's graph stack is accidentally stopped by ingestion bootstrap.** + Cause: both stacks traditionally use 7077/8888/10000. Remedy: keep graph on + its established ports and ingestion on 17077/18888/15433/20000; preserve the + harness's atomic namespace and token separation checks. +- **A failed automated run leaves fixtures.** Cause: the harness intentionally + cleans up only after all assertions pass so a failure cannot trigger an + unsafe delete. Remedy: retain failure IDs for diagnosis and add an + environment-scoped TTL sweeper that can delete only labeled synthetic + resources after a grace period. +- **A green graph image build hides broken code.** Cause: the current graph CI + has no test stage. Remedy: W1 makes tests and checks predecessors of every + image build. +- **A deterministic smoke is mistaken for real-model evidence.** Cause: a + CI-only graph stub can prove contracts while bypassing LightRAG/model quality. + Remedy: label evidence by layer and require the separate W6 real-model eval. +- **Klicker schema changes appear safe because generated code compiles.** Cause: + the new migrations have not been applied to disposable PostgreSQL. Remedy: + apply the migration chain from the actual target base, exercise rollback + compatibility, and retain the database output as evidence. +- **Old screenshots imply the current UI was tested.** Cause: parked screenshots + cover the chatbot-owned version. Remedy: capture fresh lecturer and student + states from the KB-owned branch at mobile and desktop widths. +- **The deployment primary checkout looks reusable.** Cause: it is on an + unrelated branch, far behind `origin/main`, with unrelated untracked files. + Remedy: leave it untouched; W4 belongs in the Catalyst repository. +- **A stale Catalyst checkout suggests W1 should start from an obsolete stack + tip.** Cause: local `main` can lag merged PRs and later work. Remedy: fetch + current `main` at W1 start, verify the merged ancestry, and integrate the + complete graph-runtime history on top without replacing either ancestry; see + ADR 0016. +- **Harness output looks machine-readable because the final line is regular.** + Cause: it is still human text and does not record stage timing or failure + structure. Remedy: W3 adds a schema-validated JSON report and JUnit export + while keeping human output. + +## Delivery topology + +This roadmap is a release train, not one cross-repository branch. + +| Milestone | Work packages | Parallelism | Exit gate | +| --- | --- | --- | --- | +| M1 — code readiness | W1 Catalyst graph worker, W2 Klicker stack, W3 AI-ingestion reconciliation and transfer contract | W1, W2, and W3 may proceed in parallel after their own base checks; one writer per repo | W1 and W2 are reviewed, green, and contract-compatible; W3's generic provider changes and transfer ledger are accepted. Catalyst parity and temporary-harness retirement remain W5/M2 work; no deployment yet | +| M2 — staging platform | W4 staging GitOps, W5 deployed synthetic E2E | W5 waits for W1–W4 | Two consecutive clean staging system runs, cleanup verified, alerts and rollback exercised | +| M3 — quality and lecturer beta | W6 real-model quality learning, W7 production rollout | W6 starts after W5 has stable artifacts and continues during beta; W7 does not wait for W6 to open the initial labeled beta | Explicit production authorization, disabled deployment healthy, internal canary accepted, lecturer beta enabled; W6 evidence gates later widening | + +### Stack Gate 1 — Catalyst W1 topology + +Closed on 2026-08-15. In GitHub, [Catalyst PR #2](https://github.com/uzh-bf/klicker-uzh-catalyst/pull/2) +merged as `8f46d58b8` and +[Catalyst PR #3](https://github.com/uzh-bf/klicker-uzh-catalyst/pull/3) +merged as `98a69f7a1`. Fetched `origin/main` was `37657ad2d`, later merged +work is present, and the native stacks API reports no current stack objects. +W1 therefore starts from the latest fetched Catalyst `main` and is delivered as +an ordinary PR/package. It does not pin the former PR #3 commit or create a +replacement stack. W1 inventories the imported history first; whether its own +changes need an internal split is decided from that inventory and the resulting +review size before implementation is packaged. + +Klicker W2 retains the existing five-package GitHub stack shape: + +1. platform refresh ingestion event; +2. KB-owned graph model and reader; +3. graph lifecycle, external manifest contract, and retention; +4. lecturer controls and viewer; +5. student chat viewer. + +Before publishing or changing that topology, verify native GitHub stack support +and run `$rs-stacked-change` migration planning. If native stack support is not +available, stop and ask for approval to use ordinary PRs; never hand-roll a +stack. Each layer must be independently green, reviewable, and safe to land. + +## Feature-wide test portfolio + +| Consequential risk | Existing protection | Test obligation | Primary stable seam | Distinct failure caught | Owner | +| --- | --- | --- | --- | --- | --- | +| Klicker and graph worker disagree on payload or result shape | TypeScript and Pydantic tests exist independently | Add new | Versioned JSON Schema plus representative producer payload and consumer result fixture | One repository deploys a contract the other rejects | W1, W2 | +| A graph build reads content different from the pinned digest | Unit tests cover pinned hashes | Extend existing | External manifest extraction and source hash verification | Late source refresh silently changes the graph being published | W1, W2 | +| Klicker migrations fail or make rollback impossible | Prisma schema and migration files exist | Add new | Disposable PostgreSQL migration from target base plus old-app/new-schema compatibility smoke | Production migration fails, or rolling back the app crashes on the new schema | W2 | +| Harness mutates an unready or wrong Hatchet namespace | Harness preflight and namespace tests exist | Extend existing | Harness mode/preflight contract | Upload happens before the correct ingestion and graph workers are eligible | W3 | +| pgvector contains the wrong resource/version or malformed embeddings | Harness pgvector assertions exist | Extend existing | PostgreSQL query over fixture resource metadata and vector dimension | Graph rebuild uses stale or unrelated indexed content | W3, W5 | +| Published graph is unrelated to the fixture | Resource references, digest, counts, and optional marker exist | Extend existing | GraphQL published build plus FalkorDB and GraphML provenance | An old non-empty graph makes the run look green | W3, W5 | +| Failed automation leaks resources or graphs | Success-only cleanup is verified | Add new | Synthetic-run labels and bounded TTL sweeper | Repeated failures consume storage, quota, or graph names | W3, W5 | +| Lecturer or student can read an unauthorized graph | Focused auth tests exist | Extend existing | GraphQL authorization and chat route integration tests | A graph leaks across KB, chatbot, owner, or participant scope | W2 | +| Lecturer controls or Patrick's visualizations regress | Focused component and route tests exist | Extend existing | Real browser against the integrated local stack | Build action, stale state, Cytoscape rendering, or accessible fallback is unusable | W2 | +| Graph worker cannot be operated safely | Hatchet task logs and local spans exist | Add new | Worker health, shutdown, metrics, and deployment smoke | Worker receives jobs before ready, drops an active job, or fails silently | W1, W4 | +| FalkorDB state cannot recover after pod/data loss | GraphML export exists | Add new | Restore one published build from its GraphML artifact and repoint only after verification | A serving outage becomes permanent despite retained artifacts | W4, W5 | +| AI graph is non-empty but educationally wrong | No production-grade quality suite | Add new | Committed DeepEval dataset plus deterministic graph metrics | Unsupported concepts, wrong relations, missing citations, or source leakage pass system E2E | W6 | +| Rollback starts new builds or drops the last good graph | Build pointers and polling tests exist | Add new | Staging kill-switch and rollback drill | Failed release keeps dispatching or removes the last published graph | W5, W7 | +| FalkorDB cleanup deletes durable graph history | Retention intent exists in ADR 0015 | Add new | Maintenance policy over retired graph names and GraphML archive keys | Operational graph retirement incorrectly purges an archive still retained by its KB | W2, W4, W5 | + +## Work items + +### W1 — Make the graph worker a production adopter + +**Problem:** `kg-content-generation` implements the pinned-source and GraphML +contract but still declares the worker research-only. Its pipeline builds an +image without first proving the code or publishing an immutable release input. + +**Working context:** source work currently lives in +`/Users/rschlae/Git/ai/kg-content-generation` on +`feat/kb-graph-manifest-contract`. The destination is the planned internal +`/Users/rschlae/Git/klicker/klicker-uzh-catalyst` checkout of private GitHub +repository `uzh-bf/klicker-uzh-catalyst`. PR #2 and PR #3 are merged, and +fetched `origin/main` was `37657ad2d` on 2026-08-15 with later merged work. +Start from the latest `main` at execution time. Import the complete +`kg-content-generation` history after auditing it for secrets and private data, +preserving both commit graphs, then refactor in Catalyst. Preserve the source +repository and branch until the destination validates its complete history and +file coverage; see ADR 0016 and Stack Gate 1. + +**Do:** + +1. Create and review the W1 sliced-development plan. Fetch current Catalyst + `main`, verify that the merged PR #2/PR #3 ancestry and later mainline work + remain reachable, and record the exact start tip. Audit the complete source + history for secrets and private data before import. Produce a coverage ledger + for commits, authorship, dates, graph assets, and expected paths; integrate + the history without squashing or rewriting either ancestry; and prove the + source tip, selected Catalyst base, and Patrick-authored commits remain + reachable. Reconcile the imported graph history with current graph `main` + before refactoring, without changing the Klicker payload contract. +2. Add blocking check and test jobs using the repository's pinned uv + environment. Every MR and protected-main image build must depend on them. +3. Define the separate KB graph input/result contract from the provider-side + Pydantic model and emit one versioned JSON Schema. W2 owns the Klicker + consumer acceptance fixtures and settlement interpretation; W1 owns the + provider-side metered-cost semantics keyed by the Klicker-generated + graph-build ID, including one unambiguous monetary unit and enough provider + usage detail for audit without exposing credentials. This is distinct from + Catalyst's existing public chat-engine contract-generation gate. Add a + `contract_version` only if backward compatibility cannot be enforced from + the existing shape without it. +4. Replace deployment reliance on mutable `latest` with commit and digest + output. Keep MR image builds no-push and prove the protected-main digest is + pullable before it becomes deployable. +5. Add an explicit production worker entrypoint contract: startup config + validation, Hatchet registration readiness, graceful termination, bounded + in-flight behavior, and secret-safe structured logging carrying build ID, + KB ID, and Hatchet run ID. +6. Expose the smallest useful health and metrics surface: worker readiness, + build counts by terminal class, stage duration, active jobs, and last + successful registration. Avoid a new telemetry stack; use the platform's + existing Prometheus/OpenTelemetry conventions. +7. Run one local Hatchet workflow through pinned source verification, graph + creation, FalkorDB export, GraphML upload, and terminal summary. The smoke may + use synthetic inputs but must run the production worker entrypoint. +8. Update the README classification and operator documentation only when the + production-adopter checks are true. + +**Check:** + +- Full graph test/check loop passes from a clean checkout. +- MR pipeline fails if the graph workflow tests are intentionally broken. +- MR build is no-push; protected-main build emits a valid immutable digest. +- Malformed, version-incompatible, or hash-mismatched payloads fail before + graph mutation. +- History and file-coverage evidence accounts for every expected source commit, + author, date, graph asset, and path; the pre-import source and Catalyst tips + remain reachable. +- A representative terminal result binds metered cost to the expected graph + build ID and rejects missing, malformed, or mismatched cost identity. +- SIGTERM readiness/shutdown smoke retains a clear terminal or retryable job + state. +- Local workflow produces one non-empty graph and GraphML artifact with the + expected build ID and no credentials in logs. + +**Depends on:** the latest fetched Catalyst `main` at W1 start, with its merged +history preserved. It has no W2, W3, or credential-architecture dependency. + +**Priority:** P1. + +### W2 — Finish and package the Klicker KB graph stack + +**Problem:** the KB-owned model, lifecycle, and Patrick-derived viewers are +present, but the migrations and full integrated UX have not run against a +production-like local stack. The branch also needs a safe reviewer-facing stack +topology before publication. + +**Working context:** reuse +`/Users/rschlae/Git/klicker/klicker-uzh/trees/kb-graph-stack` and the existing +`feat/kb-graph-lifecycle` lineage. Base remains the current approved `kb-poc` +stack until its owner changes it. Preserve parked branches and PRs. + +**Do:** + +1. Create and review the W2 execution plan. Fetch and compare `kb-poc`, the + parked graph branch, and every existing W9 layer before any rebase or stack + migration. Create recovery refs before cascading changes. +2. Verify or establish the five-package stack in the delivery-topology section. + Keep Patrick-authored commits and merge ancestry intact. Generated GraphQL, + Prisma, and lockfile changes belong with the layer that requires them. +3. Apply both graph migrations to disposable PostgreSQL created from the real + W9 base. Verify current-app/new-schema and previous-app/new-schema behavior; + rollback never drops the new graph tables or columns. +4. Add a dedicated graph-build kill switch. Disabled means no new graph + dispatch from GraphQL or workers while existing published graphs remain + readable. Keep it separate from `KB_INGESTION_DISABLED`. +5. Add the lecturer beta eligibility flag as a capability gate and a separate + per-KB graph opt-in. Eligibility alone must not enable graphs on any KB. +6. Validate graph-worker configuration in the general worker and graph-reader + configuration in GraphQL. Partial configuration must fail startup or Helm + rendering; completely absent configuration keeps the feature disabled. +7. Keep credential-facing behavior outside this package until the generic AI + credential architecture and consumer contract are approved. Non-credential + product state, quota state, disabled configuration, and contract checks may + proceed; do not add provider credential submission, selection, resolution, + or status UI by inventing a KG-specific path. +8. Validate the versioned graph payload/result fixtures from W1 in Klicker CI + and lock the consumer-side acceptance and settlement interpretation to that + schema. +9. Implement the graph-cost ledger at the dispatch and reconciliation seams. + Reserve the estimated maximum atomically in integer minor units before + dispatch, reject requests above the remaining semester quota or per-build + maximum, and settle the Catalyst-reported actual cost idempotently by graph + build ID. Duplicate terminal results must not charge twice; invalid, + mismatched, or over-reservation results fail closed for review; unused + reservation is released only through the same settlement transition. Show + estimated maximum cost, remaining quota, and worst-case balance before + dispatch, then actual usage and cost after settlement. +10. Run the actual local integration with the current graph worker, FalkorDB, + Blob/Azurite, doc-processing, and Hatchet. Exercise success, stale-digest late + success, timeout, failure, and retention of one unreferenced graph. +11. Use `agent-browser` on the live branch. Capture lecturer empty/building/ + published/stale/failure states and student available/unavailable states at + mobile and desktop widths. Confirm Patrick's Cytoscape presentation and the + accessible DOM fallback. +12. Run union verification against Patrick's parked range and explain every + deliberate difference. Obtain per-layer review and CI before Gate 3. + +**Check:** + +- Disposable migration chain applies cleanly from the target base. +- Focused Hatchet, knowledge-graph, GraphQL, chat, and component suites pass. +- Workspace check, affected lint, generated-code checks, chart lint/render, and + production build pass at every stack layer. +- Kill-switch test proves dispatch is rejected while the last published graph + remains readable. +- Concurrent reservation tests prove the semester and per-build limits cannot + be oversubscribed. Duplicate, late, malformed, mismatched-build, and + over-reservation terminal results prove settlement is idempotent and fails + closed without double charging or silently releasing quota. +- Pre-dispatch and settled lecturer states show the required estimate, balance, + billing label, actual usage, and cost without exposing credential or billing + account data. +- Browser screenshots show all named lecturer and student states and contain no + real user or course data. +- `git log` still attributes Patrick-authored work to Patrick and the parked tip + remains reachable. + +**Depends on:** W1 contract and metered-cost result semantics for final contract +and settlement checks. Migration, kill-switch, quota-state, local component +checks, and stack preparation may proceed in parallel with W1. Credential-facing +UI and runtime binding remain blocked on the separately approved generic AI +credential architecture and consumer contract. + +**Priority:** P1. + +### W3 — Restore the AI ingestion boundary and hand off KG E2E + +**Problem:** the temporary harness in data-ingestion proved valuable KG safety +and provenance behavior, but data-ingestion is an AI infrastructure provider, +not a Klicker or Catalyst component. The branch must preserve generic ingestion +improvements while transferring graph-system ownership to Catalyst. + +**Working context:** reuse `/Users/rschlae/Git/ai/data-ingestion` and +`rs/kb-pgvector-graph-e2e`. Preserve the unrelated untracked Vorkurs files and +Mensa roadmap. Target `main`. + +**Do:** + +1. Create and review the W3 execution plan. Fetch, record a recovery ref, and + reconcile with current `origin/main`. Resolve overlapping workflow timeouts + semantically and drop duplicate work already present upstream. +2. Inventory every branch change as one of: generic AI ingestion improvement, + Catalyst KG E2E behavior, or unrelated/duplicate upstream work. Record a + source-to-destination coverage ledger before moving code. For every + graph-specific behavior, name the observable guarantee, stable assertion + seam, source path, and intended Catalyst owner; numeric test-count parity is + not the transfer contract. +3. Keep only independently justified generic ingestion improvements in an AI + infrastructure MR. Data-ingestion may expose test-safe provider contracts, + status, and observability needed by consumers, but it does not import + GraphQL, FalkorDB, GraphML, viewer, or Catalyst lifecycle concepts. +4. Produce the accepted transfer contract for graph-specific harness behavior, + tests, and evidence. W5 owns the Catalyst port and retirement decision. + Preserve authorship and history where practical; never keep two active + copies after Catalyst reaches ledger-defined parity. +5. Define the provider-facing E2E contract Catalyst can use without owning the + service: source mutation, terminal ingestion identity, active vector/version + evidence, readiness, and cleanup evidence. Prefer supported API or read-only + observability over direct production database coupling. +6. Keep ingestion and graph Hatchet namespaces atomic and separate. Catalyst + may observe the ingestion workflow but never starts, stops, deploys, or + reconfigures AI infrastructure in deployed environments. +7. Run the full repository verification loop. Diagnose the two prior CI-release + rehearsal timeouts; either fix a proven branch interaction or record a + reproducible unrelated baseline with current-main evidence. + +**Check:** + +- A coverage ledger accounts for every harness and launcher change and names its + Catalyst destination or AI-infrastructure disposition. +- `uv run poe check`, shell syntax, non-E2E ingestion tests, and applicable E2E + tests pass on the reconciled branch. +- The AI infrastructure MR contains no Catalyst, FalkorDB, GraphML, or Klicker + product orchestration. +- The transfer ledger gives W5 a stable assertion seam and acceptance evidence + for every graph-specific behavior before the temporary data-ingestion copy is + retired; raw test counts are recorded only as historical evidence. +- Logs and artifacts in both repositories contain no injected test secret, + signed URL, DSN password, source-gateway key, or fixture source text. + +**Depends on:** none for generic base reconciliation, provider improvements, +and the transfer ledger. The temporary source remains until W5 accepts the +ledger-defined Catalyst parity, but that later retirement does not block W3 or +M1 completion. + +**Priority:** P1. + +### W4 — Deploy the staging graph serving projection through GitOps + +**Problem:** no declarative staging owner exists for the graph worker or +FalkorDB. Application code cannot be promoted until runtime ownership, +networking, storage, secrets, resources, and rollback are explicit. + +**Working context:** use the Klicker-owned internal KB graph runtime repository +selected in the decision gates. Do not add the runtime to the AI infrastructure +deployment repository. The exact path, existing-versus-new repository choice, +base, branch, and target must be recorded before W4 starts. + +**Do:** + +1. Apply the closed runtime-repository and durability rulings. Create and + review the W4 execution plan. State the + exact namespaces, Kubernetes ownership, secret syncs, storage class, + recovery target, and blast radius before manifests are written. +2. Add a staging-only FalkorDB package to the internal runtime repository with + no public ingress, resource requests/limits, health checks, + disruption/update behavior, and NetworkPolicies allowing only the graph + worker and authorized Klicker readers. FalkorDB storage may be ephemeral or + opportunistically persistent, but recovery must not depend on its volume. +3. Add a staging graph-worker package using W1's immutable digest. Wire the + Klicker-owned runtime to AI infrastructure services such as Hatchet, + doc-processing, Blob, and the model gateway through explicit service + contracts and runtime secrets. Use a dedicated service account and least + privilege. +4. Wire Klicker staging GraphQL and the general worker to FalkorDB and the graph + Hatchet namespace. Keep the graph-build kill switch enabled. +5. Add Prometheus scraping and alerts for no eligible worker, sustained build + failure, build timeout, retention failure, FalkorDB unavailability, and + storage pressure. Alerts must carry environment and service, never source + URLs or credentials. +6. Add a GraphML-to-FalkorDB recovery job or documented one-shot command that + validates graph identity and counts before changing the published pointer. + Archive every completed GraphML artifact while its KB exists, retain it for + 30 days after KB deletion, and then purge it under ADR 0015. +7. Render and validate manifests locally. Deployment and cluster verification + happen only after a separate explicit approval. + +**Check:** + +- Kustomize build and repository validators pass from a clean checkout. +- Rendered objects contain pinned image digests, non-root/least-privilege + settings where images support them, resource bounds, probes, and restrictive + NetworkPolicies. +- No secret values or external credentials appear in rendered or committed + manifests. +- With separately authorized cluster access, pods become ready, the graph + workflow has an ACTIVE eligible worker, FalkorDB accepts only authorized + connections, and alerts are healthy. +- Recovery drill restores one synthetic GraphML build into a new graph name and + verifies it before publication. + +**Depends on:** W1 image and W2 configuration contract. Deployment remains +separately gated on explicit cluster authorization. + +**Credential gate:** manifest authoring, rendering, and a separately authorized +disabled deployment do not wait for the generic credential architecture. +Binding provider-bearing runtime configuration or enabling model-backed work +does. + +**Priority:** P1. + +### W5 — Establish the deployed staging E2E release gate + +**Problem:** local component checks cannot prove deployed DNS, identities, +secrets, network policy, worker registration, storage, application migration, +or cleanup. A single repeatable staging journey must become release evidence. + +**Working context:** Catalyst owns this KG-system gate and the temporary-harness +retirement decision. Port the reusable safety, provenance, correlation, timeout, +and cleanup behavior from the accepted W3 transfer ledger. Keep only +provider-contract and internal ingestion tests in AI infrastructure. +Configuration and run history remain outside public repositories. + +**Do:** + +1. Implement every graph-specific behavior in the accepted W3 transfer ledger + at a stable Catalyst seam, preserving authorship and history where practical. + Record behavior-level parity evidence. Retire the temporary data-ingestion + copy only after that evidence is accepted and no active consumer depends on + it; do not use equal test counts as proof of parity. +2. Provision the selected dedicated non-human staging identity and synthetic KB + through supported application/admin paths. Give it only the permissions + needed to own and mutate that KB. +3. Configure protected runtime inputs for the GraphQL token/token file, KB ID, + the AI-ingestion consumer credentials and read-only evidence contract, graph + Hatchet read credentials, FalkorDB assertion credentials, and GraphML + artifact access. Catalyst treats AI infrastructure as a deployed provider; + report input names and status, never values. +4. Run read-only preflight first. Enable the graph-build kill switch only for + the synthetic owner/KB if the product supports scoped rollout; otherwise use + the environment switch during a controlled window. +5. Run the exact journey: upload deterministic text → confirm → external + ingestion → active pgvector rows → graph rebuild → published digest and + source reference → GraphQL viewer read → FalkorDB counts → GraphML counts → + resource cleanup → zero active vector rows. +6. Run it twice from clean synthetic state. A fix invalidates earlier evidence; + rerun twice after the final change. No blanket CI retry counts as a second + run. +7. Exercise failure evidence once by disabling or isolating the CI-only graph + worker in a controlled staging drill. Prove the job fails before publication, + writes usable IDs, and the TTL sweeper later removes only the synthetic + residue. +8. Exercise rollback: disable new builds, roll back the graph worker or app to + the previous compatible version, and prove the last published graph remains + readable. Restore forward without deleting graph schema. +9. Make post-deploy and nightly staging jobs emit JSON, JUnit, selected Hatchet + log links or IDs, and an artifact manifest with every tested image digest. + +**Check:** + +- Two consecutive successful staging reports exist for the same exact + component digests, each with verified cleanup. +- Every graph-specific W3 ledger entry has behavior-level Catalyst evidence, + and the retired source leaves no second active KG-system harness in AI + infrastructure. +- Resource ID, ingestion attempt, graph build, graph name, GraphML key, and both + Hatchet runs can be correlated from the report without a secret. +- Controlled dependency failure stops publication and produces a stable failure + class. +- Rollback drill preserves the last published graph and stops new dispatch. +- Alerts fire in the controlled failure and resolve after recovery. + +**Depends on:** W1–W4 and the accepted W3 transfer ledger. Provisioning the +already-selected synthetic identity is an execution prerequisite. + +**Credential gate:** Catalyst code/contract parity and separately authorized +read-only preflight may proceed before the generic AI credential design closes. +Provider-bearing configuration and any model-backed or paid staging mutation +wait for the approved generic credential architecture and consumer contract, +plus their normal deployment and spend authorization. + +**Priority:** P1. + +### W6 — Add a separate real-model graph quality gate + +**Problem:** deterministic E2E proves system behavior but only checks graph +presence and provenance. Production needs repeatable evidence that real model +outputs are source-grounded and useful without making non-deterministic paid +tests block ordinary MRs. + +**Working context:** Catalyst owns graph-quality evaluation because it owns the +KG output and release claim. Reuse generic DeepEval conventions where useful, +but do not place KG metrics, goldens, or release gates in `data-ingestion`. +AI-infrastructure evaluations remain limited to provider quality. Start the +Catalyst work after W5 produces stable staging artifacts. + +**Do:** + +1. Run the DeepEval intake before writing application or eval code: judge model, + dataset source, tracing, and iteration count. Keep reports local at first; + hosted reporting requires a separate data-boundary approval. +2. Create a committed, non-personal dataset of approximately 30–50 reviewed + goldens from approved source documents. Reuse an existing dataset if one + meets the contract; otherwise generate through `deepeval generate` and have + a domain reviewer validate it before thresholds become release gates. +3. Add deterministic metrics first: required source references, no references + outside the manifest, expected anchor concepts/relations for stable fixtures, + connectedness bounds, duplicate rate, and empty/degenerate graph rejection. +4. Add three to five DeepEval metrics in a separate metrics module. Start with + custom `GEval` criteria for concept correctness, relation correctness, and + source-grounded claims; add citation coverage or hallucination-sensitive + measures only when the dataset supplies their required fields. +5. Trace the real graph build at useful stages when approved. Keep the system + run and judge run identifiers separate so a judge failure cannot look like a + graph-worker failure. +6. Run `deepeval test run` through the repository's eval task. Store local CI + artifacts by default; use hosted reporting only after its data boundary is + approved. +7. Establish thresholds from baseline runs and domain review. Never lower a + threshold or delete a failing golden merely to obtain green CI. +8. Run the quality gate nightly once stable during beta and before beta widening + or general availability. Ordinary MRs run deterministic contract tests only. + +**Check:** + +- Dataset and metrics are inspectable, versioned, contain no student or real + lecturer data, and can be rerun without an agent. +- At least one deliberately unsupported concept or relation fails the grounding + criteria. +- Repeated baseline runs report score variance and model/judge versions. +- The release report distinguishes deterministic failures, graph-model quality + failures, and judge/eval infrastructure failures. +- Approved thresholds pass for the exact model configuration proposed for the + beta before that beta widens or becomes generally available. + +**Depends on:** W5 stable staging artifacts. It does not gate the initial +labeled beta; it gates beta widening, general availability, and quality claims. + +**Priority:** P2 during the initial beta; P1 before widening it, general +availability, or claiming graph quality. + +### W7 — Roll out production disabled, then open the public lecturer beta + +**Problem:** production activation combines schema, application, worker, +FalkorDB, identity, model cost, and user-facing behavior. It needs a reversible +sequence that preserves the last published graph at every step. + +**Working context:** after W4 merges, create the production package in the +Klicker-owned internal KB graph runtime repository. Klicker application +configuration travels through its normal v3 deployment flow. Every cluster +action and production mutation requires explicit authorization. + +**Do:** + +1. Apply the closed canary ruling when creating and reviewing the W7 rollout + plan and rollback runbook. Pin the exact W1 graph image, W2 Klicker + revision/chart, W5 gate revision, W4 manifest lineage, and currently tested + model configuration. +2. Deploy production FalkorDB and graph worker with the graph-build kill switch + enabled. Verify infrastructure health and worker eligibility without a graph + mutation. +3. Apply Klicker migrations through the normal migration path. Deploy the + compatible app and general worker with graph dispatch disabled. Verify + existing KB ingestion and non-graph chat behavior. +4. Run the W5 read-only production preflight. Do not reuse staging identities, + KBs, tokens, graph names, or storage paths. +5. Enable graph builds for two allow-listed internal lecturers, one using BYOK + and one using a UZH-issued key, with one opted-in KB each. Require two clean + builds per KB, exercise rollback and GraphML restore once, and observe the + exact deployment for 72 hours under a fixed canary cost cap. After the + canary passes, expose self-service beta activation through the product. The + global kill switch remains available for immediate rollback. +6. Trigger the minimum approved real builds through the lecturer product flow. + Verify status, publication, viewer behavior, costs, retention, logs, and + alerts. The canary succeeds only after the agreed number of clean builds and + no orphaned active build, resource, vector, graph, or artifact state. +7. Roll back immediately on authorization leakage, digest/provenance mismatch, + repeated terminal failure, unbounded cost, retention failure, or inability + to disable new dispatch. Rollback disables dispatch first and preserves the + last published graph and new schema. +8. Open the public beta after the internal canary. A lecturer who activates the + feature becomes eligible to opt individual KBs into graph generation. An + opted-in KB can build only after the lecturer accepts the cost disclosure, + supplies the approved credential, and remains under both the semester quota + and per-build cap. Before dispatch, the UI shows estimated maximum cost, + remaining quota, and worst-case resulting balance; after settlement it shows + actual usage and cost. BYOK is labeled provider-billed. A UZH-issued + credential additionally requires the manual cost-account association and is + labeled semester-billed. Klicker stores no sensitive billing information. + Students bound to that opted-in KB can use the viewer after a graph publishes; + other KBs expose no student graph. +9. Decide separately whether to authorize an ongoing production synthetic + mutation. Until then, production automation remains read-only and real + canary builds are explicit human operations. + +**Check:** + +- Disabled production deployment is healthy and causes no graph mutation. +- Migrations and rollback-compatibility smoke pass. +- Read-only production preflight passes with exact deployed component versions. +- Approved internal builds publish the expected digest and remain readable + after dispatch is disabled. +- Rollback procedure is executed in a controlled canary drill and leaves no + active build slot or broken published pointer. +- General availability occurs only after W5 and W6 evidence, all mandatory + review gates, and explicit user authorization. + +**GATED on:** explicit production authorization and an approved generic AI +credential architecture plus consumer contract before any provider-bearing +canary mutation or beta activation. Disabled infrastructure and a separately +authorized read-only production preflight may proceed earlier. Depends on +W1–W5. W6 runs during the beta and gates widening or general availability. + +**Priority:** P1 for production release. + +## Decision gates + +These decisions do not block writing or reviewing the roadmap. They are hard +stops for the named work items. Record each ruling here with its date and mark +it closed; later agents must not reopen closed rulings. + +| Decision | Options and effect | Recommendation | Gates | +| --- | --- | --- | --- | +| First production promise — **closed 2026-08-10** | Internal pilot; lecturer beta; or general availability | **Ruling:** public lecturer beta after an internal canary. It is self-service with explicit cost disclosure and a hard spending cap; student access is limited to beta KBs with a successfully published graph | W7 | +| FalkorDB durability — **closed 2026-08-10** | Ephemeral and reconstruct on loss; persistent database plus GraphML recovery; or high availability | **Ruling:** FalkorDB is reconstructible on operational issues. A clean archive of completed GraphML artifacts is the durable recovery source from the first release; see ADR 0010 | W4, W5 | +| System ownership boundary — **closed 2026-08-10** | Keep KG orchestration in AI infrastructure; split it across service repositories; or put the KG system in Catalyst | **Ruling:** Klicker owns KB product state, authorization, quota enforcement, and lecturer/student UX. Catalyst owns graph generation, FalkorDB, GraphML archive, KG quality evaluation, and KG-system E2E. AI infrastructure owns data-ingestion, doc-processing, and pgvector; Catalyst consumes their contracts without importing their code or operational lifecycle. See ADR 0011 | W1, W3–W7 | +| Runtime repository and history — **closed 2026-08-10; Stack Gate 1 closed 2026-08-15** | Import all `kg-content-generation` history; import a filtered production subtree with preserved authors; or take a clean snapshot | **Ruling:** the Klicker team owns private repository `uzh-bf/klicker-uzh-catalyst`. PR #2 and PR #3 are merged and later work is on `main`; W1 starts from the latest fetched `main` and uses an ordinary PR/package. Audit and integrate the entire `kg-content-generation` history before refactoring, preserving both repositories' ancestry plus Patrick's authorship, dates, and visualizations. Decide any internal W1 split only after the history and file-coverage inventory. AI infrastructure supplies selected services such as doc-processing but no AI-infrastructure provider code moves into Catalyst. See ADR 0016 and Stack Gate 1 | W1, W4, W7 | +| Synthetic staging identity — **closed 2026-08-10** | Static human lecturer credentials; dedicated non-human test owner using supported auth; or an application-specific service-account API | **Ruling:** dedicated non-human test owner created through supported auth/admin paths, scoped to one synthetic KB, with short-lived or regularly rotated credentials from the secret store. No E2E-specific auth bypass | W5 | +| Quality-eval timing, data, and reporting — **closed 2026-08-10** | Gate initial beta or learn during beta; existing or generated goldens; local or hosted reporting | **Ruling:** existing testing and the internal canary are sufficient to open the explicitly labeled beta. During beta, Catalyst versions 30–50 reviewed, non-personal goldens from approved or synthetic source documents and emits local DeepEval/CI artifacts. Quality evidence gates beta widening, general availability, and quality claims. Hosted reporting requires separate data-boundary approval. See ADR 0014 | W6, W7 | +| Beta activation — **closed 2026-08-10** | Lecturer flag enables every KB; per-KB opt-in only; or lecturer eligibility plus per-KB opt-in | **Ruling:** the public-beta feature flag grants the lecturer permission to enable knowledge graphs. Each KB remains opted out until that lecturer explicitly enables it. Students can use a graph only for an opted-in KB after publication | W2, W7 | +| Generic AI credential management — **open; delivery boundary closed 2026-08-15** | Per-feature custody; consumer-owned storage; or one reusable credential capability | **Ruling:** provider credentials are a generic concern for every AI capability, not a KG-specific Catalyst feature. KG, tutoring, content generation, grading feedback, and future AI services consume the same safe abstraction. Consumer applications keep only opaque handles and status; exact custody ownership and contracts are delegated to `~/.handoffs/klicker-uzh/2026-08-10-ai-credential-management-security-design-handoff.md`. While that architecture is open, W1, W3, non-credential W2, disabled W4 infrastructure, and separately authorized read-only preflight may proceed. Credential-facing UI, provider-bearing/model-backed or paid runs, canary mutations, and beta activation remain blocked | Generic platform design; credential-facing W2, provider-bearing W5/W7, canary mutation, and public beta | +| Graph-cost quota — **closed 2026-08-10; executable seams assigned 2026-08-15** | Build-count cap; token cap; monetary cap; or combined controls | **Ruling:** W1 defines actual metered-cost result semantics keyed by graph-build ID. W2 atomically reserves a per-build maximum against the lecturer's semester quota before dispatch, denies unaffordable work, settles valid terminal results idempotently, fails closed on invalid or mismatched results, and presents estimates and actuals. W7 validates those seams through the approved credential and billing paths. Quota data is non-sensitive and contains no billing account details. See ADR 0013 | W1, W2, W7 | +| Beta billing association — **closed 2026-08-10** | Klicker database; dedicated billing service; Catalyst registry; or manual external record | **Ruling:** for UZH-issued keys, keep the sensitive lecturer-to-cost-account association in a manually maintained spreadsheet. BYOK lecturers are billed by their own provider and need no internal billing association. Klicker stores no billing details and applies quota controls to both paths; later institutional integration is a separate decision | W7 | +| Production canary scope — **closed 2026-08-10** | Environment-wide switch; owner/KB allow-list; or separate canary deployment | **Ruling:** allow-list two internal lecturers, one BYOK and one UZH-issued, with one opted-in KB each. Require two clean builds per KB, one rollback and GraphML restore drill, 72 hours of observation, and a fixed canary cost cap before opening self-service beta behind the global kill switch | W7 | +| GraphML archive retention — **closed 2026-08-10** | Indefinite; while the KB exists; or fixed duration | **Ruling:** retain every successful GraphML version while its KB exists. After KB deletion, retain it through a 30-day recovery grace period and then purge it. Revisit long-term archival before general availability. See ADR 0015 | W4, W5, W7 | +| Lecturer cost display — **closed 2026-08-10** | Disclosure only; estimate before build; or estimate plus actual usage | **Ruling:** before dispatch show estimated maximum cost, remaining semester quota, and worst-case resulting balance. After settlement show actual usage and cost. Label BYOK as provider-billed and UZH-issued usage as semester-billed | W2, W7 | + +## External dependencies to watch + +| Dependency | Required outcome | Blocks | +| --- | --- | --- | +| AI-infrastructure ingestion contract | Stable source mutation, terminal ingestion identity, vector/version evidence, readiness, and cleanup contract without Catalyst controlling the provider runtime | W3, W5 | +| Generic AI credential architecture and consumer contract | Approved reusable custody, opaque-handle, safe-status, runtime-resolution, rotation, and revocation contract | Credential-facing W2; provider-bearing/model-backed W5 and W7; paid runs; canary mutation; beta activation. Does not block W1, W3, non-credential W2, disabled W4, or separately authorized read-only preflight | +| Current graph `main` and GitLab runners | Branch compatibility, test-capable runner, registry digest publication | W1 finish | +| Klicker `kb-poc` stack and GitHub stack support | Current base, preserved branch topology, per-layer CI | W2 publication | +| Doc-processing staging service | Exact health contract and successful extraction of synthetic source | W5 | +| Shared Hatchet environments | Dedicated eligible ingestion and graph workflows with read credentials for preflight | W4, W5 | +| Secret-store ownership | Runtime names for the Klicker-owned graph worker, FalkorDB, and E2E principal without exposing values | W4, W5 | +| Model gateway and budget owner | Approved model mapping, quota, and cost envelope | W6, W7 | +| Domain reviewer | Curated graph-quality goldens and threshold approval | W6 | + +## E2E automation contract + +| Layer | What runs | Cadence | Blocking policy | Durable evidence | +| --- | --- | --- | --- | --- | +| Repository contract | Harness tests; graph input/result schema; Klicker lifecycle, auth, migration, chart, and viewer tests | Every relevant MR/PR | Blocking for the owning package | Test/JUnit output, exact commit, schema version | +| Hermetic system smoke | Real GraphQL, Hatchet, pgvector, FalkorDB, Blob/Azurite, doc-processing interface, and CI-only deterministic graph workflow | Relevant main builds and release candidates | Blocking for release candidate; no paid model | JSON/JUnit report, component digests, run/resource/build IDs, cleanup | +| Deployed staging E2E | Real deployed services and standard-tier model, dedicated synthetic identity and KB | Post-deploy and nightly | Blocks production promotion; two clean runs after the final change | Redacted JSON/JUnit, Hatchet IDs, graph counts, digest, screenshots, alert/rollback drill | +| Model-quality eval | Real graph worker and model over curated goldens | During beta, nightly once stable, and pre-promotion | Does not block the initial labeled beta; blocks beta widening, quality claims, and general availability after thresholds are approved; not ordinary MRs | DeepEval report, dataset revision, model/judge versions, variance, failure samples | +| Production canary and beta | Read-only preflight on deploy; explicitly authorized internal product build; then self-service beta activation | Every graph-affecting deploy; mutation during approved canary and opted-in beta use | Stops beta opening or widening and triggers rollback | Deployed digests, migration result, build/pointer evidence, cost, alerts, rollback state | + +### Reliability rules + +- One run ID connects every report and supported log while product IDs retain + their own meaning. +- Every stage has a bounded timeout inside one absolute run deadline. +- Assertions and mutations are never retried as a whole. A documented + idempotent read may retry transient transport failures with a small bound. +- A scheduled job runs against one explicit environment; absence of that + environment is a failure or skip with a named reason, never a fallback. +- Component images and application revisions are immutable inputs recorded in + the report. +- Failure reports retain safe IDs and failure class. They exclude secrets, + source content, raw GraphML, and signed URLs. +- Cleanup is an assertion. Success requires resource invisibility and zero + active pgvector rows; TTL cleanup handles failure residues separately. +- Flakes are tracked by stable signature, owner, and expiry. A CI retry is not + passing evidence, and the same failure twice requires diagnosis. +- System correctness and semantic quality remain separate gates with separate + failure ownership. + +## Deployment and rollback contract + +### Staging order + +1. Deploy FalkorDB and graph worker with graph dispatch disabled. +2. Verify storage, networking, secrets, worker eligibility, metrics, and alerts. +3. Apply Klicker migrations and deploy compatible app/worker configuration. +4. Run read-only preflight. +5. Run the two clean W5 synthetic journeys and failure/rollback drill. +6. Run W6 quality evaluation for the proposed model mapping. + +### Production order + +1. Deploy serving infrastructure disabled. +2. Apply additive migrations; deploy app and workers with dispatch disabled. +3. Verify existing non-graph paths and read-only graph preflight. +4. Enable only the approved canary scope. +5. Verify product-triggered builds, viewers, cost, retention, and alerts. +6. Disable dispatch first on any rollback trigger; preserve schema and last + published graph. +7. Widen lecturer access, then student visibility, only through explicit gates. + +### Rollback triggers + +- Cross-owner or cross-KB authorization failure. +- Source digest, resource reference, graph name, or artifact provenance + mismatch. +- Repeated terminal workflow failure or unbounded active build. +- Inability to stop new dispatch with the kill switch. +- FalkorDB data loss without successful GraphML recovery. +- Retention deleting an active/published graph or failing without an alert. +- Model cost exceeds the approved envelope or quality falls below the approved + threshold. + +### Rollback behavior + +- Enable the global graph-build kill switch. +- Keep GraphQL and student reads on the last verified published graph when + safe; disable viewer entry points if reader correctness is in doubt. +- Roll back worker/app images only to versions compatible with the additive + schema. Do not drop graph migrations during incident rollback. +- Preserve failed build rows and redacted identifiers for diagnosis. +- Restore FalkorDB from the retained GraphML artifact into a new graph name, + verify it, then repoint through the normal publication invariant. + +## Cluster-level changes + +The roadmap proposes, but does not authorize, these cluster changes: + +- New Catalyst-owned staging and production FalkorDB workloads, storage, services, and + NetworkPolicies. +- New Catalyst-owned staging and production graph-worker workloads, service accounts, secrets, + metrics, and alerts. +- Klicker GraphQL/general-worker network access and configuration for graph + Hatchet and FalkorDB. +- Optional recovery jobs and protected deployed-E2E runners. + +Before implementation takes ownership, the W4/W7 plans must name existing +owners, namespaces, secret syncs, resource quotas, lifecycle, and blast radius, +and receive explicit approval. The agent never establishes cluster connectivity +or applies these resources without a separate instruction. + +## Review and evidence expectations + +At every W-item boundary, provide: + +1. The reviewed item plan, repo/worktree/branch/target, and MR/PR link when one + exists. +2. Exact commit or range, substantive human-authored size excluding generated + files/lockfiles/project docs, and a generated-delta summary. +3. Fresh verification commands and results, including the negative check named + by the W-item. +4. Required review reports under the repository's ignored `_local/reviews/` + directory: risk-selected intermediate review when applicable, full-path + security review, maintainability review, and exact-final-outcome capable + review. +5. Test delta (`added / changed / removed`) with each test tied to a distinct + consequential failure. +6. For UI: local URLs, mobile/desktop screenshots, auth route used, and manual + browser findings. +7. For deployments: rendered manifests, exact image digests, policy validation, + approved cluster action, rollout state, alerts, and rollback evidence. +8. An append-only Progress entry below. Never rewrite earlier evidence; append a + correction that supersedes it. + +No W-item is complete because its branch is clean, a component suite is green, +or a handoff exists. Completion requires its Check section and mandatory review +gates. No stack layer is ready for human review until its own CI is green and it +is independently safe to land. + +## Progress + +- 2026-08-10: Repository review completed across KlickerUZH, data-ingestion, + kg-content-generation, and deployment. Current blockers are the absent full + execute proof, graph worker's research-only deployment status, unreconciled + data-ingestion base, unverified Klicker migrations/browser path, and missing + graph GitOps ownership. +- 2026-08-10: Fresh local evidence: data-ingestion graph harness tests 33/33; + `uv run poe check` green; graph workflow tests 38/38; relevant diff checks + clean. The latest recorded complete data-ingestion run remains 1,283 passed + and 54 skipped; a later full rerun had two CI-release rehearsal timeouts while + the seven-case rehearsal subset passed alone. +- 2026-08-10: Draft roadmap written after the user explicitly deferred the + planning-stage specialist review. This status is not plan approval and grants + no implementation, merge, push, deployment, paid-run, cluster, or production + authority. Next step is human and capable-model review of this draft, followed + by rulings on the decision-gate table. +- 2026-08-10: Grill round 1 set the release boundary to a lecturer beta, assigned + the separate internal graph runtime to the Klicker team while retaining AI + infrastructure services such as doc-processing, and selected a dedicated + non-human staging E2E owner. The exact existing-versus-new runtime repository + remains open for round 2. +- 2026-08-10: Grill round 2 made the lecturer beta public and self-service with + cost disclosure, an API-key path, an external semester billing association + where applicable, and a hard cap. Students can use graphs only for + beta-enabled KBs after publication. The planned runtime home is the internal + `klicker-uzh-catalyst` GitHub repository, possibly moving to GitLab later. + FalkorDB is reconstructible; the GraphML archive is the durable recovery + source from the first release. +- 2026-08-10: Live repository lookup confirmed + `uzh-bf/klicker-uzh-catalyst` exists as a private GitHub repository with + default branch `main`; its local checkout and remote contain no commits. No + GitLab repository exists under that path. History migration must be decided + before the first Catalyst commit. +- 2026-08-10: Grill correction closed the system boundary. Data-ingestion, + doc-processing, and pgvector remain AI infrastructure; all knowledge-graph + behavior, quality evaluation, and KG-system E2E belong to Catalyst, while + Klicker owns the product state, authorization, quota enforcement, and user + experience. Sensitive lecturer billing information stays outside the Klicker + database; Klicker still owns non-sensitive graph quota state and enforcement. +- 2026-08-10: Grill round 3 selected a complete-history import into Catalyst, + with refactoring only after authorship-preserving migration. The lecturer + feature flag grants eligibility and each KB needs an explicit graph opt-in. + Catalyst owns API-key custody at a high level; its detailed security design + moved to a separate handoff. Klicker reserves and settles per-lecturer, + per-semester monetary quota with a per-build maximum. The beta billing + association for UZH-issued keys remains a manual spreadsheet outside Klicker; + BYOK lecturers are billed by their own provider. +- 2026-08-10: Grill round 4 accepted the proposed quality dataset, production + canary, GraphML retention, and lecturer cost display. Existing testing plus + the canary is sufficient to open the explicitly labeled beta; the curated + quality program gathers evidence during beta and gates later widening, + general availability, and quality claims. +- 2026-08-10: The production-roadmap grill is complete. The only intentionally + unresolved architecture decision is the detailed API-key security design, + isolated in its indexed handoff. The roadmap remains an unapproved draft + until its deferred planning-stage review and user approval. +- 2026-08-10: A fresh remote readback corrected the earlier empty-Catalyst + assumption. Remote `main` has five scaffold commits, and clean stacked drafts + PR #2 and PR #3 add operational verification and a stateless tutoring runtime. + W1 must preserve that history and select a current stack base before + integrating the complete graph-runtime history; ADR 0016 supersedes ADR 0012. +- 2026-08-10: The credential-security work was reframed from KG-specific + Catalyst custody to generic AI credential management for every AI consumer. + KG remains one consumer. The generic design handoff supersedes the narrower + framing and is the only intentionally open architecture decision. +- 2026-08-10: Live delivery readback found no PR for local Klicker branch + `feat/kb-graph-lifecycle`; parked PR #5206 remains an old draft with a + GitGuardian failure. Graph-builder MR !3 remains open and mergeable at + `c4a4236d` without an assigned reviewer, while its local branch is 11 commits + ahead and 8 behind its remote branch. The data-ingestion harness remains a + local-only branch at `a0fcd4f2`. Catalyst PR #2 and stacked PR #3 are clean, + draft, and green. These states must be reconciled before any implementation + branch is extended or published. +- 2026-08-15: Live repository reconciliation superseded the earlier Catalyst + delivery snapshot. Fetched Catalyst `origin/main` is `37657ad2d`; PR #2 and + PR #3 are merged, later merged work is present, and the native stack API + reports no current stack objects. Stack Gate 1 therefore starts W1 from the + latest `main` in an ordinary PR/package, with any internal split deferred to + the W1 history inventory. +- 2026-08-15: The graph-builder branch and remote feature ref agree at + `c4a4236d`; MR !3 is open and mergeable with a successful current pipeline, + but its body cites stale commit and pipeline evidence. The data-ingestion + harness is 16 commits ahead and 16 behind current `origin/main` with unrelated + local changes preserved. The Klicker roadmap branch remains at `b348adeea`, + 133 commits ahead and 81 behind current `origin/v3`. No PR, MR, branch, + runtime, or deployment state was mutated during this reconciliation. +- 2026-08-15: The previous proof containers and port `18081` listener are + absent. `devrouter ls` still fails with `could not determine process identity + for host route update lock`; no proof environment was recreated or repaired. +- 2026-08-15: The user accepted the planning-review corrections. Catalyst + parity and temporary-harness retirement now belong to W5/M2; W1, W2, and W7 + own graph-cost result semantics, reservation/settlement, and rollout + validation respectively; and the generic credential design explicitly blocks + credential-facing or provider-bearing work while allowing W1, W3, + non-credential W2, disabled W4, and separately authorized read-only preflight. + The user then approved M1 execution. This authorizes local implementation + work and its normal verification gates, but not merge, push, deployment, paid + runs, cluster access, or production mutation. +- 2026-08-15: M1 base readback refreshed Catalyst `origin/main` to + `37657ad2d5bfcfcd93a9f7f19c70470695944977`; the new repo-local W1 worktree + `trees/kb-graph-production-adopter` is based exactly on that tip. The graph + source fetch completed at `c4a4236d`, while the data-ingestion branch remains + `a0fcd4f2` at 16 commits ahead and 16 behind current `origin/main`. + Klicker linked-worktree and primary-checkout fetches are blocked by a shared + Git metadata `FETCH_HEAD` permission error; current GitHub `v3` was verified + read-only with `git ls-remote`. No existing dirty files were modified. +- 2026-08-16: Phase-5 boundary reconciliation records Catalyst W1 as reviewed + at its authorized local delivery boundary. The Catalyst execution plan is + `docs/project/2026-08-15-kb-graph-production-adopter-plan.md` on + `rs/kb-graph-production-adopter` at `b1c7edd4e`; its graph checks pass with + 86 tests passing and one skipped, the formatting gate passes, and the exact + integrated final review is clean. This is `reviewed` local evidence only — + not `pr_ready`, merged, released, or live-proven delivery. Hosted CI, W2 + consumer compatibility, image build/run and registry-digest pullability, + live Hatchet/dependency proof, and separately authorized publication, push, + PR, deployment, and paid smoke remain open. +- 2026-08-16: The M1 sibling evidence is refreshed without changing the + roadmap's existing W1/W2/W3 parallelism. Klicker W2 has a clean local + final-review record through its current implementation tip, but disposable + migration, required lecturer/student browser states, live-stack, and provider + execution checks remain open. Data-ingestion W3's transfer ledger is at + `48ba5ff093439b61f5d5165f42ddd8287089c436`; the later W2 record contains its + exact final-review evidence, while the W3 plan still has stale final-review + close-out bookkeeping. Catalyst parity and source retirement remain W5/M2 + work. M1 therefore remains in progress and no W4 staging action is implied. +- 2026-08-16: No hosted CI, image-registry publication, live service or + cluster access, secret access, push, PR/MR mutation, merge, deployment, or + paid model run was performed during this reconciliation. +- 2026-08-17: W2-C student-view browser evidence captured against the local + stack; root cause of the earlier hydration hang was a stale + `allowedDevOrigins: ['**.klicker.localhost']` in the worktree's + `packages/next-config/index.js` (predates #5248), which blocked the HMR + WebSocket for the four-label worktree host and left the app-router hydration + decoder waiting on its debug channel forever. The worktree file now matches + upstream (`['**.localhost']` in development, uncommitted). Evidence: + `w2c-kb-graph-student-graph-unavailable-en-desktop.png` showing the + graceful "temporarily unavailable" graph state (FalkorDB absent — partial + evidence), plus contract readback proving the published binding resolves + (build `20000000-...-0004`, isStale false) and fails on + `KB_FALKORDB_HOST must be a non-empty value` (503), while an unbound + chatbot throws `KnowledgeGraphNotPublishedError` code EMPTY (409). Dev + caveat: Turbopack cannot load `@klicker-uzh/knowledge-graph` via + `createRequire`, so the live dev API degrades all graph errors to 503; + the 409 branch is proven by direct probe only. + +- 2026-08-20: **W3 is closed at the merged delivery layer.** This supersedes the + 2026-08-16 W3 statements above, which placed the transfer ledger at + `48ba5ff0` in data-ingestion and called the W3 plan's close-out bookkeeping + stale; neither describes the delivered shape. + - The original W3 carrier package was **withdrawn** on 2026-08-19. MR !119 was + closed and its remote branch deleted because, once current `origin/main` was + merged in, the one genuinely generic change collapsed to zero delta and the + package became entirely Klicker KG content inside the provider repository — + contradicting W3's own boundary. Its review gates checked the package against + the ledger and never checked the plan against the boundary. + - W3's Do steps 3 and 4 were then satisfied by a different shape, recorded as + ADR 0018 `docs/adr/0018-providers-ship-launchers-consumers-run-e2e.md` + (`1a80d7833`, inside PR #5424 on this branch): the provider ships a supported + launcher for its own path, and the consumer's E2E runner invokes it rather + than reassembling the provider. + - Provider deliverable: data-ingestion + [!121](https://gitlab.uzh.ch/ai-infrastructure/services/data-ingestion/-/merge_requests/121) + merged as `39df49e` on `origin/main`. 29 commits, substantive 1347 added / + 39 removed. `scripts/start_ingestion_workers.sh --with-resource-path` starts + the resource API, `resource_dispatcher`, and `resource_fetch_worker`, with + per-session port and Compose-project isolation, an + `INGESTION_PRODUCER_REGISTRY_DIR` seam so a consumer supplies its own producer + registry, process-scoped status and stop, and a synthetic default fixture that + carries a secret reference and no credential value. + - Transfer ledger: moved to Catalyst as kg-content-generation + [!8](https://gitlab.uzh.ch/uzh-bf/tc/kg-content-generation/-/merge_requests/8), + now at `lightrag_research/project/2026-08-15-kb-graph-ingestion-transfer-plan.md` + on Catalyst `main`. No withdrawal note was owed in data-ingestion: the file sat + on zero remote branches there, so the local copy was simply removed. + - W3 Check criteria: the ledger accounts for every harness and launcher change + and names its destination. The provider MR contains no Catalyst, FalkorDB, + GraphML, or Klicker orchestration — `git diff origin/main...HEAD` excluding + `project/` has 0 matches for `klicker|catalyst|lightrag|falkor|kg-content` and + 0 paths under `src/ingestion/project_configs/`. Pipelines 647854 (`0ca20d7`) + and 647856 (`9134dd3`) each passed all 7 jobs, including `check`, `unit`, and + `e2e`. `gitleaks` is clean on all nine changed files. + - **Evidence limit carried forward:** no `ingestion.workers.*` process has ever + started on the development host, because `uv sync` for `modules/ingestion` + returns 401 for `uzh-doc-processing-client==0.1.1` and + `uzh-web-scraping-client==0.3.2` from the private index. Every worker-level + check used a stub or stand-in, and the launcher's live `--with-resource-path` + path is proven only by CI's suite, not by a running local fleet. W5 must not + treat the launcher as locally exercised. + - M1 remaining: **W1** (Catalyst graph worker, at `reviewed` local evidence per + 2026-08-16, not published) and **W2** (Klicker stack, PR #5424 open and + non-draft, mergeable but unstable with GitGuardian Security Checks failing; + the user reports it not ready). Both are P1 and the delivery-topology table + declares them parallel, so this entry orders nothing new. The M1 exit + criterion is unchanged, and no W4 or M2 staging action is implied. + - This reconciliation performed no merge, deployment, cluster access, paid run, + or production mutation. diff --git a/project/screenshots/kb-external-ingestion-ready-en-desktop.png b/project/screenshots/kb-external-ingestion-ready-en-desktop.png new file mode 100644 index 0000000000..19b986b795 Binary files /dev/null and b/project/screenshots/kb-external-ingestion-ready-en-desktop.png differ diff --git a/project/screenshots/kb-external-ingestion-speed-de-mobile.png b/project/screenshots/kb-external-ingestion-speed-de-mobile.png new file mode 100644 index 0000000000..a77c5f20ce Binary files /dev/null and b/project/screenshots/kb-external-ingestion-speed-de-mobile.png differ diff --git a/project/screenshots/kb-external-ingestion-speed-en-desktop.png b/project/screenshots/kb-external-ingestion-speed-en-desktop.png new file mode 100644 index 0000000000..90fc953bb4 Binary files /dev/null and b/project/screenshots/kb-external-ingestion-speed-en-desktop.png differ diff --git a/project/screenshots/kb-poc-s7-de-desktop.png b/project/screenshots/kb-poc-s7-de-desktop.png new file mode 100644 index 0000000000..6e7f7bf398 Binary files /dev/null and b/project/screenshots/kb-poc-s7-de-desktop.png differ diff --git a/project/screenshots/kb-poc-s7-de-mobile.png b/project/screenshots/kb-poc-s7-de-mobile.png new file mode 100644 index 0000000000..18f8c89b72 Binary files /dev/null and b/project/screenshots/kb-poc-s7-de-mobile.png differ diff --git a/project/screenshots/kb-poc-s7-en-mobile.png b/project/screenshots/kb-poc-s7-en-mobile.png new file mode 100644 index 0000000000..a4e36887e0 Binary files /dev/null and b/project/screenshots/kb-poc-s7-en-mobile.png differ diff --git a/project/screenshots/kb-poc-s7-en-processing.png b/project/screenshots/kb-poc-s7-en-processing.png new file mode 100644 index 0000000000..665e449f52 Binary files /dev/null and b/project/screenshots/kb-poc-s7-en-processing.png differ diff --git a/project/screenshots/kb-w1-resource-ingest-en-desktop.png b/project/screenshots/kb-w1-resource-ingest-en-desktop.png new file mode 100644 index 0000000000..fb3321c3e9 Binary files /dev/null and b/project/screenshots/kb-w1-resource-ingest-en-desktop.png differ diff --git a/project/screenshots/kb-w1-resource-ingest-queued-de-mobile.png b/project/screenshots/kb-w1-resource-ingest-queued-de-mobile.png new file mode 100644 index 0000000000..629b6161a8 Binary files /dev/null and b/project/screenshots/kb-w1-resource-ingest-queued-de-mobile.png differ diff --git a/project/screenshots/kb-w1-resource-ingest-queued-en-desktop.png b/project/screenshots/kb-w1-resource-ingest-queued-en-desktop.png new file mode 100644 index 0000000000..b65d6f260e Binary files /dev/null and b/project/screenshots/kb-w1-resource-ingest-queued-en-desktop.png differ diff --git a/project/screenshots/kb-w2-resource-ingest-de-mobile.png b/project/screenshots/kb-w2-resource-ingest-de-mobile.png new file mode 100644 index 0000000000..290fcba186 Binary files /dev/null and b/project/screenshots/kb-w2-resource-ingest-de-mobile.png differ diff --git a/project/screenshots/kb-w2-resource-ingest-en-desktop.png b/project/screenshots/kb-w2-resource-ingest-en-desktop.png new file mode 100644 index 0000000000..fec6cbcdf3 Binary files /dev/null and b/project/screenshots/kb-w2-resource-ingest-en-desktop.png differ diff --git a/project/screenshots/kb-w3-status-cutover-de-mobile.png b/project/screenshots/kb-w3-status-cutover-de-mobile.png new file mode 100644 index 0000000000..fc4ffbaf57 Binary files /dev/null and b/project/screenshots/kb-w3-status-cutover-de-mobile.png differ diff --git a/project/screenshots/kb-w3-status-cutover-en-desktop.png b/project/screenshots/kb-w3-status-cutover-en-desktop.png new file mode 100644 index 0000000000..6d591fedb2 Binary files /dev/null and b/project/screenshots/kb-w3-status-cutover-en-desktop.png differ diff --git a/project/screenshots/kb-w3-status-history-de-mobile.png b/project/screenshots/kb-w3-status-history-de-mobile.png new file mode 100644 index 0000000000..b75a8683b5 Binary files /dev/null and b/project/screenshots/kb-w3-status-history-de-mobile.png differ diff --git a/project/screenshots/kb-w3-status-history-en-desktop.png b/project/screenshots/kb-w3-status-history-en-desktop.png new file mode 100644 index 0000000000..caa9bfe464 Binary files /dev/null and b/project/screenshots/kb-w3-status-history-en-desktop.png differ diff --git a/project/screenshots/kb-w4-binding-de-mobile.png b/project/screenshots/kb-w4-binding-de-mobile.png new file mode 100644 index 0000000000..6d6e9dafec Binary files /dev/null and b/project/screenshots/kb-w4-binding-de-mobile.png differ diff --git a/project/screenshots/kb-w4-binding-en-desktop.png b/project/screenshots/kb-w4-binding-en-desktop.png new file mode 100644 index 0000000000..0de251a39f Binary files /dev/null and b/project/screenshots/kb-w4-binding-en-desktop.png differ diff --git a/project/screenshots/kb-w4-chatbot-linked-en-desktop.png b/project/screenshots/kb-w4-chatbot-linked-en-desktop.png new file mode 100644 index 0000000000..8b318a01e0 Binary files /dev/null and b/project/screenshots/kb-w4-chatbot-linked-en-desktop.png differ diff --git a/project/screenshots/kb-w4-chatbot-no-kb-en-desktop.png b/project/screenshots/kb-w4-chatbot-no-kb-en-desktop.png new file mode 100644 index 0000000000..30abc5342d Binary files /dev/null and b/project/screenshots/kb-w4-chatbot-no-kb-en-desktop.png differ diff --git a/project/screenshots/kb-w4-replacement-warning-en-desktop.png b/project/screenshots/kb-w4-replacement-warning-en-desktop.png new file mode 100644 index 0000000000..75a01c5fc1 Binary files /dev/null and b/project/screenshots/kb-w4-replacement-warning-en-desktop.png differ diff --git a/project/screenshots/kb-w5-kb-delete-dialog-de-mobile.png b/project/screenshots/kb-w5-kb-delete-dialog-de-mobile.png new file mode 100644 index 0000000000..c851ec234f Binary files /dev/null and b/project/screenshots/kb-w5-kb-delete-dialog-de-mobile.png differ diff --git a/project/screenshots/kb-w5-kb-delete-toast-de-mobile.png b/project/screenshots/kb-w5-kb-delete-toast-de-mobile.png new file mode 100644 index 0000000000..2a4d89283f Binary files /dev/null and b/project/screenshots/kb-w5-kb-delete-toast-de-mobile.png differ diff --git a/project/screenshots/kb-w5-resource-delete-dialog-en-desktop.png b/project/screenshots/kb-w5-resource-delete-dialog-en-desktop.png new file mode 100644 index 0000000000..ff773c5062 Binary files /dev/null and b/project/screenshots/kb-w5-resource-delete-dialog-en-desktop.png differ diff --git a/project/screenshots/kb-w5-resource-delete-toast-en-desktop.png b/project/screenshots/kb-w5-resource-delete-toast-en-desktop.png new file mode 100644 index 0000000000..859c450f84 Binary files /dev/null and b/project/screenshots/kb-w5-resource-delete-toast-en-desktop.png differ diff --git a/project/screenshots/kb-w7-bulk-delete-en-desktop.png b/project/screenshots/kb-w7-bulk-delete-en-desktop.png new file mode 100644 index 0000000000..12a3a8f379 Binary files /dev/null and b/project/screenshots/kb-w7-bulk-delete-en-desktop.png differ diff --git a/project/screenshots/kb-w7-bulk-selection-en-desktop.png b/project/screenshots/kb-w7-bulk-selection-en-desktop.png new file mode 100644 index 0000000000..a752705444 Binary files /dev/null and b/project/screenshots/kb-w7-bulk-selection-en-desktop.png differ diff --git a/project/screenshots/kb-w7-catalog-en-desktop.png b/project/screenshots/kb-w7-catalog-en-desktop.png new file mode 100644 index 0000000000..5cd409a939 Binary files /dev/null and b/project/screenshots/kb-w7-catalog-en-desktop.png differ diff --git a/project/screenshots/kb-w7-inspector-en-desktop.png b/project/screenshots/kb-w7-inspector-en-desktop.png new file mode 100644 index 0000000000..ecbdb41e6c Binary files /dev/null and b/project/screenshots/kb-w7-inspector-en-desktop.png differ diff --git a/project/screenshots/kb-w7-resources-en-desktop.png b/project/screenshots/kb-w7-resources-en-desktop.png new file mode 100644 index 0000000000..1c86a06267 Binary files /dev/null and b/project/screenshots/kb-w7-resources-en-desktop.png differ diff --git a/project/screenshots/kb-w7-review-fixes-en-desktop.png b/project/screenshots/kb-w7-review-fixes-en-desktop.png new file mode 100644 index 0000000000..731ce01626 Binary files /dev/null and b/project/screenshots/kb-w7-review-fixes-en-desktop.png differ diff --git a/project/screenshots/kb-w7-workspace-de-mobile.png b/project/screenshots/kb-w7-workspace-de-mobile.png new file mode 100644 index 0000000000..deb72bc002 Binary files /dev/null and b/project/screenshots/kb-w7-workspace-de-mobile.png differ diff --git a/project/screenshots/kb-w7-workspace-en-desktop.png b/project/screenshots/kb-w7-workspace-en-desktop.png new file mode 100644 index 0000000000..f43e6bd755 Binary files /dev/null and b/project/screenshots/kb-w7-workspace-en-desktop.png differ diff --git a/project/screenshots/kb-w8-azurite-upload-success-en-desktop.png b/project/screenshots/kb-w8-azurite-upload-success-en-desktop.png new file mode 100644 index 0000000000..698f383e25 Binary files /dev/null and b/project/screenshots/kb-w8-azurite-upload-success-en-desktop.png differ diff --git a/project/screenshots/kb-w8-chatbot-attach-success-en-desktop.png b/project/screenshots/kb-w8-chatbot-attach-success-en-desktop.png new file mode 100644 index 0000000000..baeda7f17f Binary files /dev/null and b/project/screenshots/kb-w8-chatbot-attach-success-en-desktop.png differ diff --git a/project/screenshots/kb-w8-chatbot-detach-success-en-desktop.png b/project/screenshots/kb-w8-chatbot-detach-success-en-desktop.png new file mode 100644 index 0000000000..32f3e7bd24 Binary files /dev/null and b/project/screenshots/kb-w8-chatbot-detach-success-en-desktop.png differ diff --git a/project/screenshots/kb-w8-create-toast-en-desktop.png b/project/screenshots/kb-w8-create-toast-en-desktop.png new file mode 100644 index 0000000000..eaf52f417a Binary files /dev/null and b/project/screenshots/kb-w8-create-toast-en-desktop.png differ diff --git a/project/screenshots/kb-w8-delete-toast-en-desktop.png b/project/screenshots/kb-w8-delete-toast-en-desktop.png new file mode 100644 index 0000000000..df91398b76 Binary files /dev/null and b/project/screenshots/kb-w8-delete-toast-en-desktop.png differ diff --git a/project/screenshots/kb-w8-non-preview-direct-denied-de-mobile.png b/project/screenshots/kb-w8-non-preview-direct-denied-de-mobile.png new file mode 100644 index 0000000000..20f3f0e67f Binary files /dev/null and b/project/screenshots/kb-w8-non-preview-direct-denied-de-mobile.png differ diff --git a/project/screenshots/kb-w8-non-preview-direct-denied-en-desktop.png b/project/screenshots/kb-w8-non-preview-direct-denied-en-desktop.png new file mode 100644 index 0000000000..c6f77a01fb Binary files /dev/null and b/project/screenshots/kb-w8-non-preview-direct-denied-en-desktop.png differ diff --git a/project/screenshots/kb-w8-non-preview-nav-hidden-en-desktop.png b/project/screenshots/kb-w8-non-preview-nav-hidden-en-desktop.png new file mode 100644 index 0000000000..eaf4b9bb8f Binary files /dev/null and b/project/screenshots/kb-w8-non-preview-nav-hidden-en-desktop.png differ diff --git a/project/screenshots/kb-w8-preview-nav-en-desktop.png b/project/screenshots/kb-w8-preview-nav-en-desktop.png new file mode 100644 index 0000000000..ea7c458b7a Binary files /dev/null and b/project/screenshots/kb-w8-preview-nav-en-desktop.png differ diff --git a/project/screenshots/kb-w8-url-create-success-en-desktop.png b/project/screenshots/kb-w8-url-create-success-en-desktop.png new file mode 100644 index 0000000000..81ac77f70f Binary files /dev/null and b/project/screenshots/kb-w8-url-create-success-en-desktop.png differ diff --git a/project/screenshots/kb-w8-workspace-de-mobile.png b/project/screenshots/kb-w8-workspace-de-mobile.png new file mode 100644 index 0000000000..c47c868609 Binary files /dev/null and b/project/screenshots/kb-w8-workspace-de-mobile.png differ diff --git a/turbo.json b/turbo.json index 4a416f2842..a551865103 100644 --- a/turbo.json +++ b/turbo.json @@ -20,6 +20,8 @@ "AZUREWEBJOBSSTORAGE", "BLOB_STORAGE_ACCESS_KEY", "BLOB_STORAGE_ACCOUNT_NAME", + "BLOB_STORAGE_ACCOUNT_URL", + "BLOB_STORAGE_INTERNAL_ACCOUNT_URL", "COOKIE_DOMAIN", "CORS_ALLOWED_ORIGINS", "CRON_TOKEN", @@ -51,6 +53,44 @@ "HATCHET_API_URL", "HATCHET_TENANT_ID", "HATCHET_LOG_LEVEL", + "KB_INGESTION_API_URL", + "KB_INGESTION_API_KEY", + "KB_INGESTION_PROJECT_ID", + "KB_INGESTION_DISABLED", + "KB_GRAPH_HATCHET_CLIENT_TOKEN", + "KB_GRAPH_HATCHET_CLIENT_HOST_PORT", + "KB_GRAPH_HATCHET_API_URL", + "KB_GRAPH_HATCHET_CLIENT_TLS_STRATEGY", + "KB_GRAPH_HATCHET_WORKFLOW_NAME", + "KB_GRAPH_BLOB_ACCOUNT_URL", + "KB_GRAPH_DISABLED", + "KB_GRAPH_COST_CURRENCY", + "KB_GRAPH_BILLING_MODE", + "KB_GRAPH_COST_PRICING_VERSION", + "KB_GRAPH_STANDARD_ESTIMATE_MINOR_UNITS", + "KB_GRAPH_HIGH_ESTIMATE_MINOR_UNITS", + "KB_GRAPH_MAX_COST_MINOR_UNITS", + "KB_GRAPH_SEMESTER_QUOTA_MINOR_UNITS", + "KB_GRAPH_SEMESTER_KEY", + "KB_GRAPH_TIMEOUT_SECONDS", + "KB_GRAPH_STANDARD_GENERATION_MODEL", + "KB_GRAPH_STANDARD_CLEANING_MODEL", + "KB_GRAPH_HIGH_GENERATION_MODEL", + "KB_GRAPH_HIGH_CLEANING_MODEL", + "KB_FALKORDB_HOST", + "KB_FALKORDB_PORT", + "KB_FALKORDB_USERNAME", + "KB_FALKORDB_PASSWORD", + "KB_FALKORDB_TLS", + "KB_FALKORDB_QUERY_TIMEOUT_MS", + "KB_SOURCE_GATEWAY_URL", + "KB_SOURCE_GATEWAY_KEY", + "KB_WEBHOOK_SECRET", + "KB_WEBHOOK_PREVIOUS_SECRET", + "DOC_QUERY_SCOPE_PRIVATE_KEY", + "DOC_QUERY_SCOPE_KID", + "DOC_QUERY_SCOPE_ISSUER", + "DOC_QUERY_SCOPE_AUDIENCE", "LTI_DB_TYPE", "LTI_DB_HOST", "LTI_DB_PORT", @@ -124,6 +164,7 @@ "LANGFUSE_PUBLIC_KEY", "LANGFUSE_SECRET_KEY", "LANGFUSE_BASE_URL", + "LANGFUSE_HOST", "NEXT_PUBLIC_CHAT_URL", "MCP_STUDENT_GRAPHQL_ENDPOINT", "MCP_STUDENT_HOST", @@ -166,6 +207,7 @@ "@klicker-uzh/util#build", "@klicker-uzh/prisma#build", "@klicker-uzh/graphql#build", + "@klicker-uzh/hatchet#build", "@klicker-uzh/markdown#build", "@klicker-uzh/word-cloud#build", "@klicker-uzh/transactional#build" @@ -178,6 +220,7 @@ "@klicker-uzh/util#build", "@klicker-uzh/prisma#build", "@klicker-uzh/graphql#build", + "@klicker-uzh/hatchet#build", "@klicker-uzh/markdown#build", "@klicker-uzh/word-cloud#build", "@klicker-uzh/transactional#build" @@ -190,6 +233,7 @@ "@klicker-uzh/util#build", "@klicker-uzh/prisma#build", "@klicker-uzh/graphql#build", + "@klicker-uzh/hatchet#build", "@klicker-uzh/markdown#build", "@klicker-uzh/word-cloud#build", "@klicker-uzh/transactional#build" @@ -202,6 +246,7 @@ "@klicker-uzh/util#build", "@klicker-uzh/prisma#build", "@klicker-uzh/graphql#build", + "@klicker-uzh/hatchet#build", "@klicker-uzh/markdown#build", "@klicker-uzh/word-cloud#build", "@klicker-uzh/transactional#build" diff --git a/util/configure-local-kb-graph-builder.sh b/util/configure-local-kb-graph-builder.sh new file mode 100755 index 0000000000..8fab90cab7 --- /dev/null +++ b/util/configure-local-kb-graph-builder.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +set -euo pipefail + +umask 077 + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +graph_repo="${KG_CONTENT_GENERATION_REPO:-}" +token_file="${KG_CONTENT_GENERATION_HATCHET_ENV:-}" +output_file="${KLICKER_LOCAL_KB_SERVICES_ENV:-$repo_root/.devcontainer/.local-kb-services.env}" +falkordb_host_port="${KB_GRAPH_FALKORDB_HOST_PORT:-${FALKORDB_HOST_PORT:-6379}}" + +if [[ -z "$graph_repo" ]]; then + echo "KG_CONTENT_GENERATION_REPO must point to the kg-content-generation checkout." >&2 + exit 2 +fi +if [[ -z "$token_file" ]]; then + token_file="$graph_repo/lightrag_research/scripts/hatchet/.env.local" +fi +if [[ ! -f "$token_file" ]]; then + echo "Local graph Hatchet env not found: $token_file" >&2 + echo "Start lightrag_research/scripts/hatchet/start_local_stack.sh first." >&2 + exit 2 +fi + +token="$(sed -nE 's/^export HATCHET_CLIENT_TOKEN="?([^" ]+)"?.*$/\1/p' "$token_file" | tail -n 1)" +if [[ -z "$token" ]]; then + echo "No HATCHET_CLIENT_TOKEN found in $token_file" >&2 + exit 1 +fi + +mkdir -p "$(dirname "$output_file")" +tmp_file="$(mktemp "${output_file}.tmp.XXXXXX")" +trap 'rm -f "$tmp_file"' EXIT + +cat >"$tmp_file" <&2 + exit 2 +fi + +if [[ -z "$data_ingestion_repo" ]]; then + echo "DATA_INGESTION_REPO must point to the data-ingestion checkout." >&2 + exit 2 +fi +if [[ ! -f "$data_ingestion_repo/modules/ingestion-api/pyproject.toml" ]]; then + echo "data-ingestion API project not found under: $data_ingestion_repo" >&2 + exit 2 +fi +if [[ -z "$app_origin" ]]; then + if [[ -n "${WORKSPACE:-}" ]]; then + app_origin="https://api.klicker.${WORKSPACE}.localhost" + else + app_origin="http://localhost:3000" + fi +fi + +state_root="$data_ingestion_repo/.ingestion-local/klicker-resource-api" +registry_dir="$state_root/producer-registry" +state_db="$state_root/state.db" +pid_file="$state_root/api.pid" +log_file="$state_root/api.log" +mkdir -p "$registry_dir" + +if [[ -f "$pid_file" ]]; then + existing_pid="$(<"$pid_file")" + if [[ "$existing_pid" =~ ^[0-9]+$ ]] && kill -0 "$existing_pid" 2>/dev/null; then + if curl -fsS "http://127.0.0.1:${api_port}/ready" >/dev/null 2>&1; then + echo "Local KB ingestion API already running on http://127.0.0.1:${api_port}." + exit 0 + fi + echo "Existing local KB ingestion API process is not ready: ${existing_pid}." >&2 + exit 1 + fi + rm -f "$pid_file" +fi + +cat >"$registry_dir/klicker.yaml" <&2 + exit 2 + fi + export INGESTION_STATE_BACKEND=postgres + export INGESTION_STATE_DSN="$state_dsn" + export INGESTION_STATE_SCHEMA="$state_schema" + ;; + *) + echo "KB_INGESTION_STATE_BACKEND must be sqlite or postgres." >&2 + exit 2 + ;; +esac + +uv run --project "$data_ingestion_repo/modules/ingestion-api" python -m ingestion_api.migrations + +if [[ "$foreground" == "1" ]]; then + echo "Starting local KB ingestion API in the foreground on http://127.0.0.1:${api_port}." + cd "$data_ingestion_repo" + exec uv run --project modules/ingestion-api uvicorn \ + ingestion_api.app:create_app --factory \ + --host 127.0.0.1 --port "$api_port" +fi + +( + cd "$data_ingestion_repo" + exec nohup uv run --project modules/ingestion-api uvicorn \ + ingestion_api.app:create_app --factory \ + --host 127.0.0.1 --port "$api_port" +) >"$log_file" 2>&1 < /dev/null & +api_pid=$! +echo "$api_pid" >"$pid_file" + +cleanup_stale_pid() { + if ! kill -0 "$api_pid" 2>/dev/null; then + rm -f "$pid_file" + return 1 + fi + return 0 +} + +for _attempt in $(seq 1 30); do + if curl -fsS "http://127.0.0.1:${api_port}/ready" >/dev/null 2>&1; then + echo "Local KB ingestion API is ready: http://127.0.0.1:${api_port}" + echo "State: $state_db" + exit 0 + fi + cleanup_stale_pid || { + echo "Local KB ingestion API exited; inspect $log_file" >&2 + exit 1 + } + sleep 1 +done + +echo "Timed out waiting for the local KB ingestion API; inspect $log_file" >&2 +exit 1