`, or common content selectors if no rich text container is found -- Parse JSON-LD structured data for metadata -- Download every image -- Extract products as WooCommerce-compatible CSV (if e-commerce detected) -- Preserve for each piece of content: title, URL slug, publish date, categories/tags, SEO title and description, featured image - -**If extraction gets interrupted:** - -```bash -npm run liberate -- [WEBFLOW URL] --output ./output --resume -``` - -## Step 3: Verify the extraction - -Check the output before importing: - -```bash -npm run verify -- ./output/[site-directory] -``` - -This reports: -- How many pages, posts, and media files were extracted -- Any stale Webflow CDN URLs still in content (assets-global.website-files.com) -- Failed pages or media downloads -- Quality score breakdown -- Items needing manual attention - -Show me the verification report. If there are failures, offer to investigate with `--resume` or the `/diagnose` workflow. - -## Step 4: Set up WordPress - -I need to create/have a WordPress site. Help me: -- Recommend a theme that matches my current site's visual style -- Create all categories and tags from my Webflow site -- Configure basic settings: site title, tagline, permalink structure - -Then validate the WordPress connection: - -```bash -npm run setup -- --site [MY-WORDPRESS-SITE] --username [MY-USERNAME] --token [APP-PASSWORD] -``` - -This checks site reachability, REST API availability, and authentication. If anything fails, it shows step-by-step guidance (how to create an Application Password at WordPress Admin > Users > Profile > Application Passwords). Note: on WordPress.com and wpcomstaging.com sites, the password must be generated from the site's own wp-admin, not from wordpress.com/me/security/application-passwords (account-level passwords only work for the WordPress.com public API). - -## Step 5: Import everything - -```bash -npm run liberate -- import ./output/[site-directory]/output.wxr \ - --site [MY-WORDPRESS-SITE] --username [MY-USERNAME] --token [APP-PASSWORD] -``` - -This imports in order: -1. Media files to the WordPress media library -2. Categories and tags -3. Pages with correct parent/child relationships -4. Blog posts with correct dates, categories, tags, and featured images -5. Navigation menus -6. If products were extracted: import `products.csv` via WooCommerce > Products > Import in WP admin - -All content is imported as **drafts** — you review and publish manually. - -## Step 6: Verify the import - -After import: -- Show me a URL mapping table: old Webflow URL -> new WordPress URL (from `redirect-map.json`) -- Flag any posts/pages that are missing or had import errors -- Run verify again to check for any images still pointing to Webflow CDN URLs -- List everything that needs manual attention (forms, interactions, animations) with your recommendation for what WordPress plugin to use - -Work methodically — do one step at a time, show me progress, and wait for my go-ahead before moving to the next step. If you hit something unexpected, tell me what you found rather than guessing. diff --git a/packages/data-liberation-agent/prompts/wix.md b/packages/data-liberation-agent/prompts/wix.md deleted file mode 100644 index a793a3af67..0000000000 --- a/packages/data-liberation-agent/prompts/wix.md +++ /dev/null @@ -1,108 +0,0 @@ -# Wix to WordPress Migration Prompt - -Copy everything below this line and paste it into your AI assistant (Claude, ChatGPT, Gemini, etc.). - ---- - -I want to migrate my website from Wix to WordPress. My Wix site URL is: **[PASTE YOUR WIX URL HERE]** - -I have (or will create) a WordPress site. Please help me migrate using the playbook at https://github.com/Automattic/studio/tree/trunk/packages/data-liberation-agent — read AGENTS.md first for full instructions. - -Here's what I need you to do: - -## Step 1: Inspect my site - -Run the inspection to see what we're working with: - -```bash -npm run inspect -- [WIX URL] -``` - -This will: -- Detect the platform and confirm it's Wix -- Scan the sitemap and categorize every URL (pages, blog posts, products, galleries, events, etc.) -- Detect platform-specific features (Wix Stores, Bookings, Forms, Members Area, Events, Forum) and flag which ones transfer automatically vs which need a WordPress plugin -- Probe sample pages to test extractability - -Show me the full inspection results — especially the feature flags — and wait for my approval before proceeding. - -## Step 2: Extract all content - -Run the full extraction: - -```bash -npm run liberate -- [WIX URL] --output ./output --verbose -``` - -This uses a headless browser to: -- Load every page and intercept Wix's internal API calls (the `/_api/` and `wixapis.com` responses contain the real content) -- Extract window globals (`__WIX_DATA__`, `__SITE_DATA__`, etc.) and JSON-LD structured data -- Fall back to DOM extraction and accessibility tree if API data is sparse -- Download every image from Wix's CDN (wixstatic.com/wixmp.com) -- Extract products as WooCommerce-compatible CSV (if store detected) -- Preserve for each piece of content: title, URL slug, publish date, categories/tags, SEO title and description, featured image - -**For large sites:** Extraction may take several minutes. If it gets interrupted, resume with: - -```bash -npm run liberate -- [WIX URL] --output ./output --resume -``` - -## Step 3: Verify the extraction - -Check the output before importing: - -```bash -npm run verify -- ./output/[site-directory] -``` - -This reports: -- How many pages, posts, and media files were extracted -- Any stale Wix CDN URLs still in content (these will break if Wix changes anything) -- Failed pages or media downloads -- Quality score breakdown -- Items needing manual attention - -Show me the verification report. If there are failures, offer to investigate with `--resume` or the `/diagnose` workflow. - -## Step 4: Set up WordPress - -I need to create/have a WordPress site. Help me: -- Recommend a theme that matches my current site's visual style -- Create all categories and tags from my Wix site -- Configure basic settings: site title, tagline, permalink structure - -Then validate the WordPress connection: - -```bash -npm run setup -- --site [MY-WORDPRESS-SITE] --username [MY-USERNAME] --token [APP-PASSWORD] -``` - -This checks site reachability, REST API availability, and authentication. If anything fails, it shows step-by-step guidance (how to create an Application Password at WordPress Admin > Users > Profile > Application Passwords). Note: on WordPress.com and wpcomstaging.com sites, the password must be generated from the site's own wp-admin, not from wordpress.com/me/security/application-passwords (account-level passwords only work for the WordPress.com public API). - -## Step 5: Import everything - -```bash -npm run liberate -- import ./output/[site-directory]/output.wxr \ - --site [MY-WORDPRESS-SITE] --username [MY-USERNAME] --token [APP-PASSWORD] -``` - -This imports in order: -1. Media files to the WordPress media library (Wix CDN URLs are rewritten to WordPress URLs) -2. Categories and tags -3. Pages with correct parent/child relationships -4. Blog posts with correct dates, categories, tags, and featured images -5. Navigation menus -6. If products were extracted: import `products.csv` via WooCommerce > Products > Import in WP admin - -All content is imported as **drafts** — you review and publish manually. - -## Step 6: Verify the import - -After import: -- Show me a URL mapping table: old Wix URL → new WordPress URL (from `redirect-map.json`) -- Flag any posts/pages that are missing or had import errors -- Run verify again to check for any images still pointing to Wix CDN URLs -- List everything that needs manual attention (bookings, forms, members area, events) with your recommendation for what WordPress plugin to use - -Work methodically — do one step at a time, show me progress, and wait for my go-ahead before moving to the next step. If you hit something unexpected, tell me what you found rather than guessing. diff --git a/packages/data-liberation-agent/scripts/_install.ts b/packages/data-liberation-agent/scripts/_install.ts deleted file mode 100644 index 71c4b6f13e..0000000000 --- a/packages/data-liberation-agent/scripts/_install.ts +++ /dev/null @@ -1,30 +0,0 @@ -// Throwaway: assemble the on-disk theme tree (text files) into themeFiles[] and -// launch the replica preview (creates/reuses a site, imports WXR + media, -// activates the theme). Binary assets (woff2 fonts, logo.png) are bridged -// separately after install since themeFiles content is string-only. The theme -// slug is derived from the output dir, so this works for any extracted site. -import { readdirSync, readFileSync } from 'node:fs'; -import { join, relative } from 'node:path'; -import { startPreview } from '../src/lib/preview/studio.js'; -import { requireOutputDir, installThemeSlug } from './_site-meta.js'; - -const outputDir = requireOutputDir(); -const themeDir = join(outputDir, 'theme'); -const themeSlug = installThemeSlug(outputDir); -const TEXT = /\.(css|json|php|html|svg)$/i; -const files: { relativePath: string; content: string }[] = []; -(function walk(dir: string) { - for (const e of readdirSync(dir, { withFileTypes: true })) { - const p = join(dir, e.name); - if (e.isDirectory()) walk(p); - else if (TEXT.test(e.name)) files.push({ relativePath: relative(themeDir, p), content: readFileSync(p, 'utf8') }); - } -})(themeDir); -console.error(`themeFiles (${files.length}): ${files.map((f) => f.relativePath).join(', ')}`); - -const res = await startPreview({ - outputDir, - themeFiles: files, - themeSlug, -}); -console.log(JSON.stringify(res, null, 2)); diff --git a/packages/data-liberation-agent/scripts/_pw.ts b/packages/data-liberation-agent/scripts/_pw.ts deleted file mode 100644 index e85f84639e..0000000000 --- a/packages/data-liberation-agent/scripts/_pw.ts +++ /dev/null @@ -1,34 +0,0 @@ -// Shared Playwright helpers for the dev scripts in this directory. -// -// THE __name FIX -// tsx transpiles with esbuild's `keepNames`, which rewrites named functions and -// classes to `__name(fn, "name")` so `fn.name` survives. That's fine in Node, but -// when such a function is handed to `page.evaluate` / `page.addInitScript`, -// Playwright serializes it and runs it IN THE BROWSER — where `__name` does not -// exist → `ReferenceError: __name is not defined`. (vitest's transform does NOT -// keepNames, so vitest-run code is unaffected; this only bites `tsx scripts/*.ts` -// that pass a named function — or a closure containing one — into page.evaluate.) -// -// Fix: define a `__name` identity shim in every page context BEFORE any evaluate. -// The init script is passed as a STRING on purpose — a string is not transpiled by -// esbuild, so it can't itself be rewritten to reference the not-yet-defined shim -// (which a compiled closure would, re-introducing the bootstrap failure). -import type { Browser, Page } from 'playwright'; - -const NAME_SHIM = 'window.__name = window.__name || function (f) { return f; };'; - -/** A new page with the `__name` shim installed — use this instead of - * `browser.newPage()` in any tsx script that calls page.evaluate. */ -export async function newShimmedPage( - browser: Browser, - viewport: { width: number; height: number } = { width: 1440, height: 900 }, -): Promise { - const page = await browser.newPage({ viewport }); - await page.addInitScript(NAME_SHIM); - return page; -} - -/** Install the shim on a page created elsewhere (call before the first evaluate). */ -export async function shimNames(page: Page): Promise { - await page.addInitScript(NAME_SHIM); -} diff --git a/packages/data-liberation-agent/scripts/_qa-shot.ts b/packages/data-liberation-agent/scripts/_qa-shot.ts deleted file mode 100644 index 93e6ac61e9..0000000000 --- a/packages/data-liberation-agent/scripts/_qa-shot.ts +++ /dev/null @@ -1,25 +0,0 @@ -/** - * Throwaway QA screenshot helper: full-page shot of a URL at a given viewport. - * npx tsx scripts/_qa-shot.ts [width=1440] [height=900] - */ -import { chromium } from 'playwright'; -import { shimNames } from './_pw.js'; - -const [url, out, w = '1440', h = '900'] = process.argv.slice(2); -if (!url || !out) { console.error('usage: _qa-shot.ts [w] [h]'); process.exit(2); } - -const browser = await chromium.launch(); -const ctx = await browser.newContext({ viewport: { width: +w, height: +h }, deviceScaleFactor: 1 }); -const page = await ctx.newPage(); -await shimNames(page); -await page.goto(url, { waitUntil: 'load', timeout: 45000 }); -await page.evaluate(async () => { - for (let y = 0; y < document.body.scrollHeight; y += 600) { window.scrollTo(0, y); await new Promise((r) => setTimeout(r, 50)); } - window.scrollTo(0, 0); - const imgs = Array.from(document.images); - await Promise.race([Promise.all(imgs.map((im) => (im.complete ? 0 : im.decode().catch(() => 0)))), new Promise((r) => setTimeout(r, 5000))]); -}); -await page.waitForTimeout(1200); -await page.screenshot({ path: out, fullPage: true }); -await browser.close(); -console.log('shot:', out); diff --git a/packages/data-liberation-agent/scripts/_shot.ts b/packages/data-liberation-agent/scripts/_shot.ts deleted file mode 100644 index d2b1db3cb4..0000000000 --- a/packages/data-liberation-agent/scripts/_shot.ts +++ /dev/null @@ -1,17 +0,0 @@ -// Full-page screenshot of a URL at a given width, no 5s MCP cap. Used by the -// match-section and match-page skills to capture source-vs-built crops. -// Usage: node scripts/run.mjs _shot -import { chromium } from 'playwright'; -const [, , url, out, widthArg] = process.argv; -const width = Number(widthArg || 1008); -const b = await chromium.launch(); -const page = await b.newPage({ viewport: { width, height: 900 } }); -await page.goto(url, { waitUntil: 'networkidle', timeout: 60_000 }).catch(() => {}); -// Force below-fold lazy images to load by scrolling through (mouse.wheel — no -// page.evaluate, so no tsx __name footgun), then back to top. -for (let y = 0; y < 6000; y += 700) { await page.mouse.wheel(0, 700); await page.waitForTimeout(120); } -await page.keyboard.press('Home').catch(() => {}); -await page.waitForTimeout(2000); -await page.screenshot({ path: out, fullPage: true, timeout: 60_000 }); -await b.close(); -console.log('wrote', out); diff --git a/packages/data-liberation-agent/scripts/_site-meta.ts b/packages/data-liberation-agent/scripts/_site-meta.ts deleted file mode 100644 index e9fe0a60d3..0000000000 --- a/packages/data-liberation-agent/scripts/_site-meta.ts +++ /dev/null @@ -1,63 +0,0 @@ -// Shared helpers for the replica-reconstruction driver scripts. Site identity is -// always derived from the run's OWN output dir (its WXR + dir name) so the -// scripts work for any extracted site — nothing is hardcoded to one source. -import { existsSync, readFileSync } from 'node:fs'; -import { join } from 'node:path'; - -// Single source of truth for the install theme slug (e.g. -// `output/www.example.com` → `www-example-com-replica`). -export { deriveInstallThemeSlug as installThemeSlug } from '../src/mcp-server/handlers/install-theme.js'; - -/** - * Require an output dir as the first positional CLI arg (e.g. `output/www.example.com`). - * Exits with a usage hint rather than silently defaulting to one site. - */ -export function requireOutputDir(scriptUsage = ' [args...]'): string { - const dir = process.argv[2]; - if (!dir) { - console.error(`usage: tsx scripts/${scriptName()} ${scriptUsage} (e.g. output/www.example.com)`); - process.exit(1); - } - return dir; -} - -function scriptName(): string { - return process.argv[1]?.split('/').pop() ?? ' -
first (for sites that have them) - * 2. Collect content blocks, skipping chrome - * 3. Fall back to with chrome elements stripped - */ -export const HOSTINGER_CHROME_CLASS = /\b(block-sticky-bar|block-header|block--footer|block-header-cart|block-header-item|block-blog-header)\b/; -export const CHROME_SECTION_STRIP = /]*\bclass=["'][^"']*\b(block-sticky-bar|block-header|block--footer|block-blog-header)\b[^"']*["'][^>]*>[\s\S]*?<\/section>/gi; -export const NAV_HEADER_FOOTER_STRIP = [ - /]*>[\s\S]*?<\/nav>/gi, - /]*>[\s\S]*?<\/header>/gi, - /