Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b618ffe
Local Supabase + Blorg milestone: schema, sample importer, Podman loc…
hatton Jul 17, 2026
c515c59
Track supabase/snippets/ (Podman requires bind-mount sources to exist)
hatton Jul 17, 2026
f346cf3
Add generated books.tags_text column for wildcard tag matching
hatton Jul 18, 2026
4319597
Merge remote-tracking branch 'origin/develop' into parse-to-supabase-…
hatton Jul 18, 2026
41a9c9d
docs: point db README at renamed FUNCTIONS-MIGRATION-PLAN.md
hatton Jul 18, 2026
9901bf2
Apply preflight review decisions for PR #9
hatton Jul 18, 2026
667563a
Gate book_languages and related_books reads on book visibility
hatton Jul 18, 2026
7990f7b
Dedupe per-book junction rows so a duplicated langPointer can't abort…
hatton Jul 18, 2026
ac679c1
Add copyright/license fields to the shared Book type
hatton Jul 18, 2026
a9e8a66
Add send-concern-email edge function
hatton Jul 18, 2026
c2d9827
Add tests for send-concern-email
hatton Jul 18, 2026
790633a
Document send-concern-email in the migration plan and api-spec-next
hatton Jul 18, 2026
8492977
sync-tool: make SAMPLE_TARGET actually scale the sample size
hatton Jul 18, 2026
ce5ed22
Merge send-concern-email edge function into the foundation PR
hatton Jul 18, 2026
8fa208a
Add match_topic_tags RPC for non-canonical topic filters
hatton Jul 18, 2026
be6db75
Wire send-concern-email's Mailgun secrets into the local edge runtime
hatton Jul 18, 2026
f6aa757
Document the 200k-books performance design goal and query-pattern design
hatton Jul 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,9 @@
BLOOM_PARSE_APP_ID_PROD=
BLOOM_PARSE_APP_ID_DEV=
BLOOM_PARSE_APP_ID_UNIT_TEST=

# send-concern-email: Mailgun ("report-a-book" template). Optional - if either is
# unset, the function logs and succeeds as a no-op (matches the legacy Parse cloud
# function's test-environment behavior).
MAILGUN_API_KEY=
EMAIL_REPORT_BOOK_RECIPIENT=
15 changes: 14 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,17 @@ jobs:
env:
BLOOM_PARSE_APP_ID_PROD: ${{ secrets.BLOOM_PARSE_APP_ID_PROD }}
BLOOM_PARSE_APP_ID_DEV: ${{ secrets.BLOOM_PARSE_APP_ID_DEV }}
BLOOM_PARSE_APP_ID_UNIT_TEST: ${{ secrets.BLOOM_PARSE_APP_ID_UNIT_TEST }}
BLOOM_PARSE_APP_ID_UNIT_TEST: ${{ secrets.BLOOM_PARSE_APP_ID_UNIT_TEST }}

# Keep in sync with the supabase version pinned in package.json.
- uses: supabase/setup-cli@v1
with:
version: 2.109.1

# Prove the SQL migrations apply cleanly from scratch (catches broken
# or order-dependent migrations before they reach staging).
- name: Validate migrations apply from scratch
run: |
supabase start -x logflare,vector,studio,imgproxy,realtime,mailpit,edge-runtime,storage-api
Comment thread
hatton marked this conversation as resolved.
supabase db reset
supabase stop --no-backup
15 changes: 15 additions & 0 deletions FUNCTIONS-MIGRATION-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,21 @@ All clients call `https://api.bloomlibrary.org/v1/<function>`. Known consumers:
- CI/CD: GitHub Actions. PRs run tests; push to `develop` deploys to the **staging** Supabase
project; push to `main` deploys to **production** (`.github/workflows/`).
- Secrets so far: `BLOOM_PARSE_APP_ID_{PROD,DEV,UNIT_TEST}` plus deploy credentials.
- **`send-concern-email` (`supabase/functions/send-concern-email/`)** — implemented on the
`send-concern-email` branch, **not yet merged or deployed**. This isn't an Azure Function; it
replaces the Parse Cloud Code function `sendConcernEmail` (bloom-parse-server, `cloud/emails.js`),
called by blorg's "report a concern about this book" form. Same shape as `fs`: `verify_jwt =
false` (blorg still authenticates via Parse, not Supabase Auth, so there's no Supabase JWT to
check yet — see the TODO in `config.toml`), book data comes from `BloomParseServer` (there is no
Supabase `books` table on `develop` yet — see the note below), and it sends via the Mailgun REST
API directly (no `mailgun-js`). New secrets: `MAILGUN_API_KEY`, `EMAIL_REPORT_BOOK_RECIPIENT`
(both optional — unset means log-and-no-op, matching the legacy cloud function's behavior on
the unit-test Parse server).

Note on book data: a Postgres `books` table (with `title`/`copyright`/`license` etc.) exists on
the separate `parse-to-supabase-db-foundation` branch but has not merged to `develop` as of this
writing. Once it lands, this function's Parse lookup is a candidate to switch to a direct
Postgres query, consistent with wherever the rest of this repo's book-reading functions land.

---

Expand Down
18 changes: 18 additions & 0 deletions PAPERCUTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Papercuts for bloom-core-supabase — small dev/agent/tooling friction points, captured now and
fixed later. See the "papercut" skill for the procedure.

Note: when resolving a git merge conflict here, keep both sides' entries unless they merge cleanly.

---

## 2026-07-18 — `pnpm test` hard-fails when `.env.local` doesn't exist

- **Cut:** `pnpm test` and `pnpm test:secrets-optional` both pass `--env-file=.env.local` to
deno, and deno hard-fails ("node: .env.local: not found", exit 126) when the file is
missing — the repo ships only `.env.example`. On a fresh checkout the only test script that
runs at all is `pnpm test:ci`.
- **Idea:** Make `.env.local` optional (deno supports `--env-file` gracefully via a wrapper
that checks existence, or document `cp .env.example .env.local` as a required setup step in
the README / setup hook).
- **Context:** Hit running unit tests on branch parse-to-supabase-db-foundation; agent had to
discover `test:ci` as the fallback by trial and error.
87 changes: 72 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pnpm install
cp .env.example .env.local
# Edit .env.local with your Parse Server credentials

# Start local development (requires Docker)
# Start local development (requires a container runtime — see Prerequisites)
pnpm dev

# Run tests
Expand All @@ -21,17 +21,54 @@ pnpm test

Test the fs function:
```bash
curl "http://127.0.0.1:54321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub" -o test.bloompub
curl "http://127.0.0.1:44321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub" -o test.bloompub
```

## 📋 Prerequisites

- [pnpm](https://pnpm.io/) v11+ (pins its own version via `packageManager` and downloads
Node.js v22.20.0 via `devEngines.runtime` in `package.json` — no Volta or manual Node install needed)
- [Deno](https://deno.com/) v2.9+ (runs and tests the edge functions)
- [Docker Desktop](https://docs.docker.com/desktop/) (local development only)
- A container runtime (local development only): [Podman](https://podman.io/) (see
[Windows + Podman setup](#-windows--podman-setup) below) or
[Docker Desktop](https://docs.docker.com/desktop/)
- [Supabase account](https://supabase.com/)

## 🪟 Windows + Podman setup

Podman is the supported non-Docker-Desktop way to run the local stack (verified 2026-07):

```powershell
winget install RedHat.Podman
podman machine init
podman machine set --rootful # required: rootless port forwarding doesn't reach the Windows host
podman machine start
```

Then start the stack. If Docker Desktop is also installed, point the CLI at Podman's pipe
explicitly, and exclude the analytics services (on Windows they require a TCP-exposed
Docker daemon, which Podman doesn't provide):

```powershell
$env:DOCKER_HOST = "npipe:////./pipe/podman-machine-default"
pnpm exec supabase start -x logflare,vector
```

Gotchas we hit so you don't have to:

- **Local ports are 443xx, not Supabase's default 543xx** (API `44321`, DB `44322`,
Studio `44323`, Mailpit `44324`). Windows reserves semi-random "excluded port ranges"
for Hyper-V/WSL inside the dynamic range (49152+), and Supabase's defaults landed inside
one — every service unreachable from the host, with no error anywhere. Ports below 49152
can't be dynamically excluded. Check yours with
`netsh interface ipv4 show excludedportrange protocol=tcp`.
- Podman (unlike Docker) doesn't auto-create missing bind-mount sources; the repo now
commits `supabase/snippets/` and `supabase/seed.sql` so `supabase start` has everything
it needs.
- `supabase stop`'s volume prune trips over a Podman/Docker API difference
("all" is an invalid volume filter) — harmless; use `supabase db reset` to get a truly
fresh database.

### Dependency policy

Both package resolvers enforce a **7-day cooldown**: a version published less than 7 days
Expand All @@ -48,17 +85,37 @@ lockfile changes.
## 📁 Project Structure

```
supabase/functions/
├── _shared/ # Shared utilities
│ ├── BloomParseServer.ts
│ └── utils.ts
├── fs/ # S3 proxy function (streams book files)
│ ├── index.ts
│ ├── BookData.ts
│ └── README.md
└── tests/ # Deno tests
supabase/
├── migrations/ # SQL migrations (the database schema: books, languages, tags, ...)
├── seed.sql # Local-dev seed (real data comes from packages/sync-tool)
└── functions/
├── _shared/ # Shared utilities
│ ├── BloomParseServer.ts
│ └── utils.ts
├── fs/ # S3 proxy function (streams book files)
│ ├── index.ts
│ ├── BookData.ts
│ └── README.md
└── tests/ # Deno tests
packages/
└── sync-tool/ # Parse -> Supabase data import (v0: sample importer)
docs/
└── db/ # Database migration docs (field mapping, plan review, roadmap)
```

## 📚 Importing sample data (local dev)

With the local stack running, pull ~100 real books (plus their languages, tags, uploaders,
and relatedBooks) from the production Parse server into your local database:

```bash
pnpm --filter @bloom/sync-tool import-sample
```

Idempotent — re-run any time to refresh. It refuses to write to a non-localhost Supabase
unless you set `SYNC_ALLOW_REMOTE=1` (env vars are `SYNC_*`-prefixed on purpose; see
`packages/sync-tool/src/import-sample.mjs`).

## 🔧 Scripts

```bash
Expand All @@ -78,13 +135,13 @@ Streams book files from S3 without exposing bucket details.
**Example**:
```bash
# Get thumbnail
curl "http://localhost:54321/functions/v1/fs/dev-harvest/ZWI7FUQnDd/thumbnails/thumbnail-256.png"
curl "http://localhost:44321/functions/v1/fs/dev-harvest/ZWI7FUQnDd/thumbnails/thumbnail-256.png"

# Download book (streams, no buffering)
curl "http://localhost:54321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub" -o book.bloompub
curl "http://localhost:44321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub" -o book.bloompub

# Range request
curl -H "Range: bytes=0-1023" "http://localhost:54321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub"
curl -H "Range: bytes=0-1023" "http://localhost:44321/functions/v1/fs/harvest/VuebFgcL0R/Ososi.bloompub"
```

See [`supabase/functions/fs/README.md`](supabase/functions/fs/README.md) for details.
Expand Down
46 changes: 46 additions & 0 deletions api-spec-next.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,52 @@ components:
type: integer
required: [tag]
paths:
# Implemented (supabase/functions/send-concern-email/) on the send-concern-email branch, but not
# yet merged/deployed, so it stays in api-spec-next.yml until it goes live. Replaces the Parse
# Cloud Code function `sendConcernEmail` used by blorg's "report a concern about this book" form.
/send-concern-email:
post:
description:
Emails the Bloom team a concern reported about a book, via a Mailgun
template. No auth (blorg gates this behind login client-side; there is
no Supabase JWT yet since blorg still authenticates via Parse).
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
fromAddress:
description: the reporter's email address; used as the outgoing email's "from"
type: string
format: email
content:
description: the reporter's free-text description of the concern (max 5000 chars)
type: string
maxLength: 5000
bookId:
description: the database ID of the book being reported
type: string
required: [fromAddress, content, bookId]
responses:
200:
description: OK
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
required: [success]
400:
description: Bad Request - missing/invalid fromAddress, content, or bookId
404:
description: Not Found - no book with that ID
500:
description: Internal Server Error - the book lookup or the Mailgun request failed

# not needed yet (since we have the expanded form from /books)
/languages/{tag}:
get:
Expand Down
Loading