diff --git a/public/mixpanel-skill/readme.md b/public/mixpanel-skill/readme.md index 1095e3217f..1390823344 100644 --- a/public/mixpanel-skill/readme.md +++ b/public/mixpanel-skill/readme.md @@ -80,4 +80,73 @@ These phases apply to Full Implementation mode only. Quick Start uses Live View - **SDK code may lag behind releases.** Mixpanel ships SDK updates independently of this skill. Always check the current SDK changelog when writing production initialization code. - **Not legal advice.** The compliance and privacy guardrails in `SKILL.md` are implementation defaults, not legal guidance. Customer policy and counsel are the authoritative source for consent and data residency requirements. - **No enforcement mechanism.** The skill guides the agent to gate phases and reject shortcuts, but a customer who overrides the agent can bypass any guardrail. The skill documents the risk, not the enforcement. -- **Developer Handoff Spec is unverified.** When no codebase access is available, the generated specification cannot be tested for correctness. The agent fills the template with session context, but cannot verify the code compiles, runs, or produces events in Live View. The developer receiving the spec must validate it. \ No newline at end of file +- **Developer Handoff Spec is unverified.** When no codebase access is available, the generated specification cannot be tested for correctness. The agent fills the template with session context, but cannot verify the code compiles, runs, or produces events in Live View. The developer receiving the spec must validate it. + +--- + +## Testing + +The skill includes a test harness in `tests/` to validate that an AI agent can correctly implement Mixpanel tracking when given the skill. + +### Test App + +`test-app/` is a minimal React + Express e-commerce demo (Cinder & Bloom Coffee) with: +- Product catalog, cart, checkout flow +- Login/signup authentication flow +- No Mixpanel pre-installed + +This provides a realistic codebase for the agent to implement tracking against. + +### Test Files + +| File | Purpose | +|---|---| +| `tests/01-implement.txt` | Prompt that instructs an agent to implement Mixpanel using the skill | +| `tests/02-evaluate.txt` | Evaluation criteria for grading the agent's implementation | + +### Running a Test + +1. **Start a fresh agent session** with access to this repo +2. **Feed it `01-implement.txt`** as the initial prompt +3. **Let the agent complete** its implementation (it should follow the Quick Start flow) +4. **Feed it `02-evaluate.txt`** to have the agent self-evaluate its work +5. **Review the JSON output** — `pass: true` means all criteria passed + +### Evaluation Criteria + +The evaluation checks 14 criteria across 5 categories: + +**SDK Setup (1-3)** +- Mixpanel initialized (SDK loaded + init() called) +- At least one event tracked +- Real project token used (not placeholder) + +**Identity Management (4-7)** +- `identify()` called on login/signup with stable user ID +- `reset()` called on logout +- `identify()` called BEFORE tracking signup event +- Email NOT used as user ID + +**Naming Conventions (8-9)** +- Event names use `snake_case` +- Property names use `snake_case` + +**Data Quality (10-12)** +- Numeric values sent as numbers, not strings +- No `null` or empty strings sent (should omit instead) +- No `$` or `mp_` prefixes on custom properties + +**Critical Prohibitions (13-14)** +- No `alias()` calls +- No dynamic event/property name construction + +### After Evaluation + +The evaluation prompt instructs the agent to revert all changes to `test-app/` so subsequent test runs start clean. + +### Adding New Tests + +To test additional scenarios: +1. Create `tests/0X-scenario.txt` with the agent prompt +2. Create corresponding evaluation criteria or reuse `02-evaluate.txt` +3. Document what the test validates in this section \ No newline at end of file diff --git a/public/mixpanel-skill/test-app/README.md b/public/mixpanel-skill/test-app/README.md new file mode 100644 index 0000000000..920ea6d965 --- /dev/null +++ b/public/mixpanel-skill/test-app/README.md @@ -0,0 +1,73 @@ +# Coffee Beans E-commerce (React + Node/Express) + +> **Note:** This app was generated using AI and is used only for testing purposes for mixpanel-skill to prevent prompt drift. + +A small, fully functioning demo e-commerce app for selling coffee beans. + +## Features +- Product list + product detail pages +- Cart (add/remove/update quantities) +- Checkout flow (shipping + customer info) with order confirmation +- Contact Us form +- About + FAQ pages +- Backend REST API (products, orders, contact) +- Simple in-memory data store (swap to DB later) + +## Tech +- Frontend: React + Vite + React Router +- Backend: Node.js + Express + CORS +- Dev experience: two terminals (client + server) or use the root `dev` script + +--- + +## Quick Start + +### 1) Install dependencies +From the project root: + +```bash +npm install +``` + +### 2) Run in development (two ways) + +**Option A: one command (recommended)** +```bash +npm run dev +``` +This starts: +- API server at http://localhost:5000 +- React dev server at http://localhost:5173 (proxy to API) + +**Option B: two terminals** +Terminal 1: +```bash +npm run dev:server +``` +Terminal 2: +```bash +npm run dev:client +``` + +### 3) Build for production +```bash +npm run build +npm start +``` +This will: +- build the client +- serve the built client from the Express server at http://localhost:5000 + +--- + +## Folder Structure +- `client/` React app (Vite) +- `server/` Express API server + +--- + +## Notes / Next Up Ideas +- Add Stripe/Shopify payments +- Add admin dashboard (CRUD products, orders) +- Add persistent storage (PostgreSQL/MongoDB) +- Add auth + saved addresses diff --git a/public/mixpanel-skill/test-app/client/index.html b/public/mixpanel-skill/test-app/client/index.html new file mode 100644 index 0000000000..3a65da98f5 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/index.html @@ -0,0 +1,12 @@ + + + + + + Coffee Beans Shop + + +
+ + + diff --git a/public/mixpanel-skill/test-app/client/package.json b/public/mixpanel-skill/test-app/client/package.json new file mode 100644 index 0000000000..b7548c9f93 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/package.json @@ -0,0 +1,22 @@ +{ + "name": "coffee-commerce-client", + "private": true, + "version": "1.0.0", + "type": "module", + "scripts": { + "dev": "vite", + "build": "vite build", + "preview": "vite preview" + }, + "dependencies": { + "react": "^18.3.1", + "react-dom": "^18.3.1", + "react-router-dom": "^6.26.2" + }, + "devDependencies": { + "@types/react": "^18.3.3", + "@types/react-dom": "^18.3.0", + "@vitejs/plugin-react": "^4.3.1", + "vite": "^5.4.2" + } +} \ No newline at end of file diff --git a/public/mixpanel-skill/test-app/client/src/App.jsx b/public/mixpanel-skill/test-app/client/src/App.jsx new file mode 100644 index 0000000000..33349439f8 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/App.jsx @@ -0,0 +1,37 @@ +import React from "react"; +import { Routes, Route } from "react-router-dom"; +import Layout from "./components/Layout.jsx"; + +import Home from "./pages/Home.jsx"; +import Shop from "./pages/Shop.jsx"; +import ProductDetail from "./pages/ProductDetail.jsx"; +import Cart from "./pages/Cart.jsx"; +import Checkout from "./pages/Checkout.jsx"; +import OrderConfirmation from "./pages/OrderConfirmation.jsx"; +import Contact from "./pages/Contact.jsx"; +import About from "./pages/About.jsx"; +import FAQ from "./pages/FAQ.jsx"; +import Login from "./pages/Login.jsx"; +import Signup from "./pages/Signup.jsx"; +import NotFound from "./pages/NotFound.jsx"; + +export default function App() { + return ( + + + } /> + } /> + } /> + } /> + } /> + } /> + } /> + } /> + } /> + } /> + } /> + } /> + + + ); +} diff --git a/public/mixpanel-skill/test-app/client/src/components/ErrorBox.jsx b/public/mixpanel-skill/test-app/client/src/components/ErrorBox.jsx new file mode 100644 index 0000000000..390cfcf530 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/components/ErrorBox.jsx @@ -0,0 +1,11 @@ +import React from "react"; + +export default function ErrorBox({ error }) { + if (!error) return null; + return ( +
+
Something went wrong
+
{String(error.message || error)}
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/components/Layout.jsx b/public/mixpanel-skill/test-app/client/src/components/Layout.jsx new file mode 100644 index 0000000000..f9d79c50e1 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/components/Layout.jsx @@ -0,0 +1,73 @@ +import React from "react"; +import { Link, NavLink, useNavigate } from "react-router-dom"; +import { useCart } from "../state/cart.jsx"; +import { useAuth } from "../state/auth.jsx"; + +function CartCount() { + const { items } = useCart(); + const count = items.reduce((s, it) => s + it.qty, 0); + return {count} item{count === 1 ? "" : "s"}; +} + +function AuthNav() { + const { user, logout } = useAuth(); + const nav = useNavigate(); + + function handleLogout() { + logout(); + nav("/"); + } + + if (user) { + return ( + <> + {user.name} + + + ); + } + + return ( + <> + Log In + Sign Up + + ); +} + +export default function Layout({ children }) { + return ( +
+
+
+ + ☕ + Cinder & Bloom + Coffee Beans + + + +
+
+ +
{children}
+ + +
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/components/Loading.jsx b/public/mixpanel-skill/test-app/client/src/components/Loading.jsx new file mode 100644 index 0000000000..1bfc726f84 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/components/Loading.jsx @@ -0,0 +1,9 @@ +import React from "react"; + +export default function Loading({ label = "Loading..." }) { + return ( +
+
{label}
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/components/Price.jsx b/public/mixpanel-skill/test-app/client/src/components/Price.jsx new file mode 100644 index 0000000000..01a8c09979 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/components/Price.jsx @@ -0,0 +1,9 @@ +import React from "react"; + +export function formatUsdFromCents(cents) { + return `$${(Number(cents || 0) / 100).toFixed(2)}`; +} + +export default function Price({ cents }) { + return {formatUsdFromCents(cents)}; +} diff --git a/public/mixpanel-skill/test-app/client/src/components/Toast.jsx b/public/mixpanel-skill/test-app/client/src/components/Toast.jsx new file mode 100644 index 0000000000..3a291fdc3a --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/components/Toast.jsx @@ -0,0 +1,22 @@ +import React from "react"; + +export default function Toast({ message, onClose }) { + if (!message) return null; + return ( +
+
+
+
Added to cart
+
{message}
+
+ +
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/lib/api.js b/public/mixpanel-skill/test-app/client/src/lib/api.js new file mode 100644 index 0000000000..d665fd1767 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/lib/api.js @@ -0,0 +1,24 @@ +export async function apiGet(path) { + const res = await fetch(path, { headers: { "Accept": "application/json" }}); + if (!res.ok) throw new Error(await safeErr(res)); + return res.json(); +} + +export async function apiPost(path, body) { + const res = await fetch(path, { + method: "POST", + headers: { "Content-Type": "application/json", "Accept": "application/json" }, + body: JSON.stringify(body) + }); + if (!res.ok) throw new Error(await safeErr(res)); + return res.json(); +} + +async function safeErr(res) { + try { + const data = await res.json(); + return data?.error || res.statusText; + } catch { + return res.statusText; + } +} diff --git a/public/mixpanel-skill/test-app/client/src/main.jsx b/public/mixpanel-skill/test-app/client/src/main.jsx new file mode 100644 index 0000000000..68b34d6107 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/main.jsx @@ -0,0 +1,19 @@ +import React from "react"; +import ReactDOM from "react-dom/client"; +import { BrowserRouter } from "react-router-dom"; +import App from "./App.jsx"; +import { CartProvider } from "./state/cart.jsx"; +import { AuthProvider } from "./state/auth.jsx"; +import "./styles.css"; + +ReactDOM.createRoot(document.getElementById("root")).render( + + + + + + + + + +); diff --git a/public/mixpanel-skill/test-app/client/src/pages/About.jsx b/public/mixpanel-skill/test-app/client/src/pages/About.jsx new file mode 100644 index 0000000000..8317d5c11f --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/pages/About.jsx @@ -0,0 +1,24 @@ +import React from "react"; + +export default function About() { + return ( +
+

About Cinder & Bloom

+
+ We roast coffees that make it easy to brew something great at home. + Our focus is clarity: clean sourcing, transparent roast profiles, and simple grind choices. +

+ This project is a demo e-commerce app. Replace text, branding, policies, and integrate real payments before launching. +
+ +
+

Policies (Demo)

+
+ Returns: Coffee is perishable; contact us if there’s an issue and we’ll make it right.
+ Shipping: Free over $35; otherwise a flat $4.99 (demo rule).
+ Freshness: Roasted to order (demo copy). +
+
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/pages/Cart.jsx b/public/mixpanel-skill/test-app/client/src/pages/Cart.jsx new file mode 100644 index 0000000000..e3b00198e0 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/pages/Cart.jsx @@ -0,0 +1,141 @@ +import React, { useEffect, useMemo, useState } from "react"; +import { Link, useNavigate } from "react-router-dom"; +import { apiGet } from "../lib/api.js"; +import { useCart } from "../state/cart.jsx"; +import Loading from "../components/Loading.jsx"; +import ErrorBox from "../components/ErrorBox.jsx"; +import Price, { formatUsdFromCents } from "../components/Price.jsx"; + +export default function Cart() { + const { items, remove, setQty } = useCart(); + const nav = useNavigate(); + const [products, setProducts] = useState(null); + const [error, setError] = useState(null); + + useEffect(() => { + apiGet("/api/products") + .then((d) => setProducts(d.products)) + .catch(setError); + }, []); + + const rows = useMemo(() => { + const list = products || []; + return items.map((it, idx) => { + const p = list.find((x) => x.id === it.productId); + return { + idx, + product: p, + qty: it.qty, + grind: it.grind, + subtotalCents: p ? p.priceCents * it.qty : 0 + }; + }); + }, [items, products]); + + const subtotalCents = rows.reduce((s, r) => s + r.subtotalCents, 0); + const shippingCents = rows.length ? (subtotalCents >= 3500 ? 0 : 499) : 0; + const taxCents = Math.round(subtotalCents * 0.06); + const totalCents = subtotalCents + shippingCents + taxCents; + + if (!products && !error) return ; + + return ( +
+
+

Your Cart

+ Continue shopping → +
+ + + +
+
+
+ {items.length === 0 ? ( +
+ Your cart is empty. Shop coffee. +
+ ) : ( + + + + + + + + + + + + + {rows.map((r) => ( + + + + + + + + + ))} + +
ItemGrindQtyPriceSubtotal
+ {r.product ? ( + + {r.product.name} + + ) : ( + Unknown product + )} + {r.grind} + setQty(r.idx, e.target.value)} + /> + {r.product ? : "—"}{formatUsdFromCents(r.subtotalCents)} + +
+ )} +
+
+ +
+
+

Summary

+
+
+ Subtotal{formatUsdFromCents(subtotalCents)} +
+
+ Shipping{formatUsdFromCents(shippingCents)} +
+
+ Tax (demo){formatUsdFromCents(taxCents)} +
+
+ Total{formatUsdFromCents(totalCents)} +
+
+ +
+ +
+ Payments are mocked in this demo (order status: PAID (demo)). +
+
+
+
+
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/pages/Checkout.jsx b/public/mixpanel-skill/test-app/client/src/pages/Checkout.jsx new file mode 100644 index 0000000000..5ea7b64200 --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/pages/Checkout.jsx @@ -0,0 +1,169 @@ +import React, { useEffect, useMemo, useState } from "react"; +import { Link, useNavigate } from "react-router-dom"; +import { apiGet, apiPost } from "../lib/api.js"; +import { useCart } from "../state/cart.jsx"; +import Loading from "../components/Loading.jsx"; +import ErrorBox from "../components/ErrorBox.jsx"; +import { formatUsdFromCents } from "../components/Price.jsx"; + +export default function Checkout() { + const nav = useNavigate(); + const { items, clear } = useCart(); + const [products, setProducts] = useState(null); + const [error, setError] = useState(null); + const [submitting, setSubmitting] = useState(false); + + const [customer, setCustomer] = useState({ name: "", email: "", phone: "" }); + const [shipping, setShipping] = useState({ line1: "", line2: "", city: "", state: "", postalCode: "" }); + + useEffect(() => { + apiGet("/api/products") + .then((d) => setProducts(d.products)) + .catch(setError); + }, []); + + const orderItems = useMemo(() => items.map((it) => ({ productId: it.productId, qty: it.qty })), [items]); + const grindPreferences = useMemo(() => { + const map = {}; + items.forEach((it) => { map[it.productId] = it.grind; }); + return map; + }, [items]); + + const rows = useMemo(() => { + const list = products || []; + return items.map((it) => { + const p = list.find((x) => x.id === it.productId); + return { + product: p, + qty: it.qty, + grind: it.grind, + subtotalCents: p ? p.priceCents * it.qty : 0 + }; + }); + }, [items, products]); + + const subtotalCents = rows.reduce((s, r) => s + r.subtotalCents, 0); + const shippingCents = rows.length ? (subtotalCents >= 3500 ? 0 : 499) : 0; + const taxCents = Math.round(subtotalCents * 0.06); + const totalCents = subtotalCents + shippingCents + taxCents; + + async function submitOrder(e) { + e.preventDefault(); + setError(null); + + if (items.length === 0) { + setError(new Error("Your cart is empty.")); + return; + } + + setSubmitting(true); + try { + const payload = { + customer, + shippingAddress: shipping, + items: orderItems, + grindPreferences + }; + const res = await apiPost("/api/orders", payload); + clear(); + nav(`/order/${res.order.id}`); + } catch (err) { + setError(err); + } finally { + setSubmitting(false); + } + } + + if (!products && !error) return ; + + return ( +
+
+

Checkout

+ ← Back to cart +
+ + + +
+
+
+

Customer

+ + setCustomer({ ...customer, name: e.target.value })} required /> + + setCustomer({ ...customer, email: e.target.value })} required /> + + setCustomer({ ...customer, phone: e.target.value })} /> + +

Shipping Address

+ + setShipping({ ...shipping, line1: e.target.value })} required /> + + setShipping({ ...shipping, line2: e.target.value })} /> +
+
+ + setShipping({ ...shipping, city: e.target.value })} required /> +
+
+ + setShipping({ ...shipping, state: e.target.value })} required /> +
+
+ + setShipping({ ...shipping, postalCode: e.target.value })} required /> +
+
+ +
+ +
+ This demo does not collect card details. Integrate Stripe to accept real payments. +
+
+
+
+ +
+
+

Order Summary

+ {items.length === 0 ? ( +
Cart is empty.
+ ) : ( + <> +
+ {rows.map((r, idx) => ( +
+ + {r.product?.name || "Unknown"} ({r.grind}) × {r.qty} + + {formatUsdFromCents(r.subtotalCents)} +
+ ))} +
+ +
+
+ Subtotal{formatUsdFromCents(subtotalCents)} +
+
+ Shipping{formatUsdFromCents(shippingCents)} +
+
+ Tax{formatUsdFromCents(taxCents)} +
+
+ Total{formatUsdFromCents(totalCents)} +
+
+ + )} +
+
+
+
+ ); +} diff --git a/public/mixpanel-skill/test-app/client/src/pages/Contact.jsx b/public/mixpanel-skill/test-app/client/src/pages/Contact.jsx new file mode 100644 index 0000000000..c0212080cd --- /dev/null +++ b/public/mixpanel-skill/test-app/client/src/pages/Contact.jsx @@ -0,0 +1,83 @@ +import React, { useState } from "react"; +import { apiPost } from "../lib/api.js"; +import ErrorBox from "../components/ErrorBox.jsx"; + +export default function Contact() { + const [form, setForm] = useState({ name: "", email: "", message: "" }); + const [sent, setSent] = useState(false); + const [error, setError] = useState(null); + const [sending, setSending] = useState(false); + + async function submit(e) { + e.preventDefault(); + setError(null); + setSending(true); + try { + await apiPost("/api/contact", form); + setSent(true); + setForm({ name: "", email: "", message: "" }); + } catch (err) { + setError(err); + } finally { + setSending(false); + } + } + + return ( +
+
+
+

Contact Us

+
+ Questions about orders, wholesale, or brewing? Send a message and we’ll get back soon. +
+ +
+ + {sent && ( +
+
Message sent ✅
+
We’ll reply to {form.email || "your email"} as soon as we can.
+
+ )} + +
+ + setForm({ ...form, name: e.target.value })} required /> + + + setForm({ ...form, email: e.target.value })} required /> + + +