Skip to content

Store and serve versioned HTML and Markdown artifacts - #1401

Open
innoavator wants to merge 4 commits into
mainfrom
feat/artifacts
Open

innoavator wants to merge 4 commits into
mainfrom
feat/artifacts

Conversation

@innoavator

@innoavator innoavator commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Publish a sandbox HTML or Markdown page as an immutable, versioned artifact. Identical republishes keep the current version.
  • Store each version on local disk or S3 and serve files through a short-lived signed URL with a sandboxed content policy.
  • Peered installs must set a shared artifact directory or S3, plus a content signing key. The chart mounts that directory for a single server replica.

Test plan

  • pnpm --filter @truefoundry/trueforge test -- tests/unit/artifacts
  • pnpm --filter @truefoundry/trueforge test:store:sqlite -- tests/db/sqlite/artifact-store
  • Publish a bundle through POST /api/v1/artifacts, then open /artifact-content/<token>/<path> and confirm referenced files load
  • Publish the same manifest again and confirm the response reuses the existing version
  • Boot with STANDALONE=false and no ARTIFACT_STORAGE_LOCAL_DIR and confirm startup fails
  • Render the chart with more than one server replica and local artifact storage and confirm it fails

The sidebar, publish dialog, and viewer are in #1402, stacked on this branch.


Note

Medium Risk
Adds new storage (local/S3), signed public content URLs, and sandbox file reads with publish limits; misconfigured multi-replica local storage or signing keys can break availability or access control.

Overview
This PR adds versioned artifacts: users can publish an HTML or Markdown entry file from a sandbox turn (plus referenced assets) into durable storage and manage it through a new /api/v1/artifacts API (list/create/update/delete, bundle preview, versions, downloads, and short-lived content tokens).

Publishing walks sandbox references, enforces size/file limits, hashes manifests for deduplicated republishes, and persists metadata via a new artifact store while blobs land on local disk or S3. Published pages are served outside OpenAPI at /artifact-content/:token, with HTML/CSS URL rewriting, a sandboxed CSP, and signed tokens (ARTIFACT_CONTENT_SIGNING_KEY).

Deploy/config changes wire this through .env.example, docker-compose artifact volumes, and the Helm chart (PVC for single-replica local storage, auto signing Secret, env validation blocking multi-replica + local backend). trueforge-core exports listSandboxFiles for POSIX-safe directory listing during bundle collection.

Reviewed by Cursor Bugbot for commit 37b549d. Bugbot is set up for automated code reviews on this repo. Configure here.

Part of #1403

@changeset-bot

changeset-bot Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 37b549d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@truefoundry/trueforge Patch
@truefoundry/trueforge-core Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@veria-ai

veria-ai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

PR overview

The PR adds storage and serving of versioned HTML and Markdown artifacts, including publishing new artifacts and additional versions.

Six issues have been addressed, with one cleanup issue still open. Failed publishes can leave partially uploaded files without a version record or an API deletion path; repeated failures triggered by a valid asset filename can accumulate orphaned files and exhaust local storage.

Open issues (1)

Fixed/addressed: 6 · PR risk: 6/10

Reviews

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread charts/trueforge/values.yaml Resolved
Comment thread docker-compose.yml Resolved
Publish a sandbox page as an immutable version on local disk or S3 and open it through a signed content URL.

Co-authored-by: Cursor <cursoragent@cursor.com>
@innoavator innoavator changed the title Publish HTML and Markdown artifacts Store and serve versioned HTML and Markdown artifacts Nov 16, 2026
The design discussion lives in the pull request, so the working draft is no longer needed in the repo.

Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Abhishek Choudhary and others added 2 commits November 16, 2026 23:05
Bundle collection now lists the entry's folder once, only follows references inside it, and skips hidden files, so mounted credentials can't be published. Markdown and CSS reference scanning is linear-time, truncated listings are rejected, and publishing is capped by a per-tenant storage quota that is re-checked inside the publish transaction.

Content tokens are redacted from access and error logs, the compose file no longer ships a signing key, and only artifact errors expose a machine-readable code. Artifact responses use created_by_subject and a files-free version summary, and listing loads latest versions in one query.

CI: move the migration after main's, narrow the exec result type, and validate the local artifact directory at startup instead of config import so non-standalone store tests load.
Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 37b549d. Configure here.

{{- include "trueforge.labels" . | nindent 4 }}
spec:
accessModes:
- ReadWriteOnce

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local PVC breaks rolling server updates

High Severity

The artifacts claim is ReadWriteOnce and is mounted on the server deployment, which still uses RollingUpdate. A one-replica rollout surges a second pod, and RWO cannot attach to both, so upgrades stick in ContainerCreating.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 37b549d. Configure here.

storageFailures.push(error);
}
}
return storageFailures;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete races concurrent version publish

Medium Severity

delete snapshots storage prefixes first, then deletes the artifact row in a later transaction with no row lock. A concurrent publishVersion can insert a new version after that snapshot; the cascade removes the new row while its blobs are never deleted.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 37b549d. Configure here.

): Promise<{ prefix: string; backend: ArtifactStorageBackendName }> {
const prefix = this.#storage.write.resolvePrefix({ key: `${tenantId}/${artifactId}/${versionId}` });
for (const file of bundle.files) {
await this.#storage.write.put({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Partial uploads are never cleaned up

Both publishing methods call #upload() before entering their cleanup blocks, so a later put() failure leaves earlier files behind without a version row or an API deletion path; for example, a valid 255-byte asset basename makes local storage fail with ENAMETOOLONG when it appends .<pid>.tmp, allowing repeated failed publishes to fill the volume. Delete the allocated prefix inside this helper whenever an upload fails, covering both publishNew() and publishVersion().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant