Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Copy to .env.local. All values are intentionally blank; basic dev/build needs none.
# Database: runtime pooler URL, and optional direct/session URL for operations.
# Database: Supabase transaction pooler (6543) for runtime.
# A separate direct/session URL (5432) is required for migrations and administration.
# Copy both URLs from Supabase Connect; credentials and hosts can differ.
DATABASE_URL=
DATABASE_URL_UNPOOLED=

Expand Down
69 changes: 67 additions & 2 deletions bun.lock

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions docs/codex-stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Supabase cron calls a protected Vercel route every 15 minutes.
Use a Supabase database with Vault, `pg_cron`, and `pg_net` support. Configure
these in `.env.local` for local administration and in Vercel for the deployment:

- `DATABASE_URL`: runtime database connection.
- `DATABASE_URL_UNPOOLED`: direct or session-pooler connection for migrations,
account registration, and cron setup when runtime uses a transaction pooler.
- `DATABASE_URL`: runtime transaction-pooler connection (port `6543`).
- `DATABASE_URL_UNPOOLED`: required direct or session-pooler connection (port
`5432`) for migrations, account registration, and cron setup.
- `CRON_SECRET`: a random secret of at least 32 characters, shared by the route
and scheduled requests.
- `VERCEL_PROJECT_PRODUCTION_URL`: production hostname, supplied by Vercel when
Expand Down
38 changes: 34 additions & 4 deletions docs/site-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,40 @@ values in Vercel environment settings.

## Deployment

For database-backed deployments, set `DATABASE_URL` for runtime queries and
`DATABASE_URL_UNPOOLED` to a direct or session-pooler connection for migrations
and cron setup. Each installation needs its own database; choose a Vercel region
near it in [vercel.json](../vercel.json).
For database-backed deployments, copy two URLs from Supabase's **Connect** panel:

- `DATABASE_URL`: transaction pooler, port `6543`, for runtime queries.
- `DATABASE_URL_UNPOOLED`: direct connection, port `5432`, for migrations, cron
setup, and account registration. Use the session pooler on `5432` when the
build host cannot reach the direct endpoint over IPv6.

Both must point to the same database. Copy the complete URLs; direct and pooled
connections use different usernames and hosts. Preserve the SSL parameters from
your configured connection. Admin scripts require `DATABASE_URL_UNPOOLED` and
never fall back to the runtime URL or rewrite ports. Local PostgreSQL URLs work
for development and tests. Each installation needs its own database; choose a
Vercel region near it in [vercel.json](../vercel.json).

The app uses Drizzle's node-postgres adapter with one shared connection per
process, a five-second idle timeout, and Vercel's `attachDatabasePool` lifecycle
integration. Queries are queued by the driver; no Postgres.js pipelining or
session-pooler workaround is needed. The remaining Postgres.js admin scripts
use only the explicit direct/session URL and close their clients in `finally`.

The migration runner uses Drizzle's official migrator and journal. Its advisory
lock and migrations share one dedicated connection; concurrent deployments wait
up to 60 seconds for locks. Closing that connection releases the migration lock
on success or failure. Local `db:migrate` uses the same runner.

An `EMAXCONNSESSION` error means the session pool is full. Check that runtime
traffic really uses `6543`; `max: 1` limits each app process, not the deployment.
Earlier versions rewrote runtime URLs to `5432`. After deploying the fix, old
instances may still hold sessions until they retire. A direct migration URL
bypasses that session pool when the build host can reach it.

References: [Supabase connection modes](https://supabase.com/docs/guides/database/connecting-to-postgres),
[Postgres.js pipelining limitations](https://supabase.com/docs/guides/database/postgres-js),
and [Vercel pool lifecycle](https://vercel.com/docs/functions/functions-api-reference/vercel-functions-package#attachdatabasepool).

Set Vercel's **Build Command** to:

Expand Down
6 changes: 5 additions & 1 deletion drizzle.config.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import { defineConfig } from "drizzle-kit";

import { administrationDatabaseUrl } from "./src/db/connection";
import { env } from "./src/env";

const databaseUrl = env.DATABASE_URL_UNPOOLED ?? env.DATABASE_URL;
const databaseUrl =
env.DATABASE_URL_UNPOOLED === undefined
? undefined
: administrationDatabaseUrl(env);

export default defineConfig({
...(databaseUrl === undefined || databaseUrl.length === 0
Expand Down
5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"format:typ": "typstyle -i career",
"format:typ:check": "typstyle --check career",
"db:generate": "drizzle-kit generate",
"db:migrate": "drizzle-kit migrate",
"db:migrate": "bun scripts/migrate-production-database.ts --local",
"db:studio": "drizzle-kit studio",
"codex:backfill": "bun scripts/backfill-codex.ts",
"codex:account": "bun scripts/configure-codex-account.ts",
Expand All @@ -37,6 +37,7 @@
"@remark-embedder/core": "3.0.3",
"@t3-oss/env-nextjs": "0.13.11",
"@tanstack/react-query": "5.102.8",
"@vercel/functions": "3.9.9",
"ai": "7.0.63",
"class-variance-authority": "0.7.1",
"cn": "0.4.0",
Expand All @@ -48,6 +49,7 @@
"motion": "13.2.0",
"next": "16.3.4",
"next-themes": "0.4.6",
"pg": "8.23.0",
"postgres": "3.4.9",
"posthog-js": "1.428.7",
"react": "19.2.8",
Expand All @@ -73,6 +75,7 @@
"@types/bun": "1.4.1",
"@types/mdx": "2.0.14",
"@types/node": "24.13.3",
"@types/pg": "8.23.1",
"@types/react": "19.2.17",
"@types/react-dom": "19.2.3",
"@typescript/native-preview": "7.0.0-dev.20260624.1",
Expand Down
18 changes: 10 additions & 8 deletions scripts/configure-codex-account.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import postgres from "postgres";

import { validateCodexAuthJson } from "@/lib/codex/stats";

import { administrationDatabaseUrl } from "../src/db/connection";

const [id, codexHome] = process.argv.slice(2);
if (id === undefined || !/^[a-z0-9][a-z0-9_-]{0,63}$/u.test(id)) {
throw new Error("Usage: bun run codex:account <id> <codex-home>");
Expand All @@ -13,18 +15,18 @@ if (codexHome === undefined) {
throw new Error("A dedicated Codex home is required.");
}

const databaseUrl = [
process.env.DATABASE_URL_UNPOOLED?.trim(),
process.env.DATABASE_URL?.trim(),
].find((value): value is string => value !== undefined && value.length > 0);
if (databaseUrl === undefined || databaseUrl.length === 0) {
throw new Error("DATABASE_URL_UNPOOLED or DATABASE_URL is required.");
}
const databaseUrl = administrationDatabaseUrl({
DATABASE_URL_UNPOOLED: process.env.DATABASE_URL_UNPOOLED,
});

const authJson = await readFile(path.resolve(codexHome, "auth.json"), "utf-8");
const providerAccountId = validateCodexAuthJson(authJson).tokens.account_id;
const secretName = `codex_auth_${id}`;
const sql = postgres(databaseUrl, { max: 1, prepare: false });
const sql = postgres(databaseUrl, {
connect_timeout: 10,
max: 1,
prepare: false,
});

try {
await sql`create schema if not exists vault`;
Expand Down
8 changes: 2 additions & 6 deletions scripts/configure-supabase-cron.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import postgres from "postgres";

import { administrationDatabaseUrl } from "../src/db/connection";
import { env } from "../src/env";
import { githubTokensFrom } from "../src/lib/github-accounts";
import {
Expand Down Expand Up @@ -56,12 +57,7 @@ const requiredEnvironmentValue = (

export const supabaseCronDatabaseUrlFrom = (
environment: SupabaseCronEnvironment
) => {
const unpooledDatabaseUrl = environment.DATABASE_URL_UNPOOLED?.trim();
return unpooledDatabaseUrl === undefined || unpooledDatabaseUrl.length === 0
? requiredEnvironmentValue("DATABASE_URL", environment.DATABASE_URL)
: unpooledDatabaseUrl;
};
) => administrationDatabaseUrl(environment);

export const supabaseCronUrlsFrom = (configuredSiteUrl: string) => {
const siteUrl = new URL(configuredSiteUrl);
Expand Down
116 changes: 34 additions & 82 deletions scripts/migrate-production-database.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
import { spawn } from "node:child_process";
import { once } from "node:events";
import { fileURLToPath } from "node:url";

import postgres from "postgres";
import { sql } from "drizzle-orm";
import { drizzle } from "drizzle-orm/node-postgres";
import { migrate } from "drizzle-orm/node-postgres/migrator";
import { Client } from "pg";

import { administrationDatabaseUrl } from "../src/db/connection";
import { env } from "../src/env";
import { reportOperationalError } from "../src/lib/operational-error";

// Keep the historical lock key so overlapping old/new deployments still coordinate.
const MIGRATION_LOCK_NAME = "f0rr0.dev:drizzle-migrations";
Expand All @@ -20,11 +24,6 @@ export class ProductionMigrationConfigurationError extends Error {
}
}

const configuredValue = (value: string | undefined) => {
const trimmed = value?.trim();
return trimmed === undefined || trimmed.length === 0 ? null : trimmed;
};

export const shouldApplyProductionMigrations = (environment: Environment) => {
if (environment.VERCEL !== "1") {
return false;
Expand All @@ -43,60 +42,31 @@ export const shouldApplyProductionMigrations = (environment: Environment) => {
return true;
};

export const productionMigrationDatabaseUrl = (environment: Environment) => {
const configured =
configuredValue(environment.DATABASE_URL_UNPOOLED) ??
configuredValue(environment.DATABASE_URL);
if (configured === null) {
throw new ProductionMigrationConfigurationError(
"A production database URL is not configured in Vercel."
);
}
export const productionMigrationDatabaseUrl = (environment: Environment) =>
administrationDatabaseUrl(environment);

let url: URL;
const migrateDatabase = async (databaseUrl: string) => {
// The lock and every migration use this same session. Closing it releases the lock.
const client = new Client({
connectionString: databaseUrl,
application_name: "f0rr0.dev:migrations",
connectionTimeoutMillis: 10_000,
});
client.on("error", (error) => {
reportOperationalError("database-migration", error);
});
try {
url = new URL(configured);
} catch {
throw new ProductionMigrationConfigurationError(
"The production database URL is invalid."
);
}
if (url.protocol !== "postgres:" && url.protocol !== "postgresql:") {
throw new ProductionMigrationConfigurationError(
"The production database URL must use PostgreSQL."
await client.connect();
const database = drizzle(client);
await database.execute(sql`set lock_timeout = '60s'`);
await database.execute(
sql`select pg_advisory_lock(hashtextextended(${MIGRATION_LOCK_NAME}, 0))`
);
}

if (url.port === "6543") {
if (!url.hostname.endsWith(".pooler.supabase.com")) {
throw new ProductionMigrationConfigurationError(
"A transaction-pooler database URL cannot run migrations."
);
}
url.port = "5432";
}
return url.toString();
};

const runDrizzleMigrations = async (databaseUrl: string) => {
const migrationProcess = spawn(process.execPath, ["run", "db:migrate"], {
env: {
...process.env,
DATABASE_URL: databaseUrl,
DATABASE_URL_UNPOOLED: databaseUrl,
},
stdio: "inherit",
});
const [code, signal] = (await once(migrationProcess, "exit")) as [
number | null,
NodeJS.Signals | null,
];
if (signal !== null) {
throw new Error(`Database migration received signal ${signal}.`);
}
const exitCode = code ?? 1;
if (exitCode !== 0) {
throw new Error(`Database migration exited with code ${String(exitCode)}.`);
await migrate(database, {
migrationsFolder: fileURLToPath(new URL("../drizzle", import.meta.url)),
});
} finally {
await client.end();
}
};

Expand All @@ -108,30 +78,12 @@ export const applyProductionMigrations = async (
return;
}

const databaseUrl = productionMigrationDatabaseUrl(environment);
const lockConnection = postgres(databaseUrl, {
connect_timeout: 10,
max: 1,
prepare: false,
});
let locked = false;
try {
process.stdout.write("Applying production database migrations.\n");
await lockConnection`
select pg_advisory_lock(hashtextextended(${MIGRATION_LOCK_NAME}, 0))
`;
locked = true;
await runDrizzleMigrations(databaseUrl);
} finally {
if (locked) {
await lockConnection`
select pg_advisory_unlock(hashtextextended(${MIGRATION_LOCK_NAME}, 0))
`;
}
await lockConnection.end({ timeout: 5 });
}
process.stdout.write("Applying production database migrations.\n");
await migrateDatabase(productionMigrationDatabaseUrl(environment));
};

if (import.meta.main) {
await applyProductionMigrations();
await (process.argv.includes("--local")
? migrateDatabase(administrationDatabaseUrl(env))
: applyProductionMigrations());
}
Loading
Loading