Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/typecheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@ jobs:
- name: Slim-entry convention audit
run: node scripts/slim-entries-audit.mjs --fail

# This repo is PUBLIC and infra/origin holds production vhosts, so the audit keeps
# thresholds, source addresses, inline allowlists and secret markers out of them, and
# rejects a wildcard include on a file carrying a protective directive (nginx accepts
# one matching no files and reloads clean, i.e. it fails OPEN). --self-test runs first
# so a rule edited into uselessness fails here rather than passing silently.
- name: Audit committed origin nginx config
run: |
node scripts/origin-config-audit.mjs --self-test
Expand Down
22 changes: 22 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,22 @@ pnpm lint
pnpm typecheck
```

**Three script audits also gate `.github/workflows/typecheck.yml`**, and `pnpm test` runs none
of them:

```bash
node scripts/icon-tsx-audit.mjs --fail # icon sizing, see docs/icons.md
node scripts/icon-scss-audit.mjs # also fails if a retired SCSS rule reappears
node scripts/slim-entries-audit.mjs # feed payload invariants

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Run the slim-entry audit in failing mode

When the slim-entry audit finds a violation, this documented command still exits successfully because scripts/slim-entries-audit.mjs only returns status 1 when --fail is present. Anyone following this new audit checklist can therefore miss the feed-payload violations that CI rejects; add --fail here to match the workflow command.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both correct, fixed in fa62b35.

slim-entries-audit.mjs only exits 1 with --fail, so the command as I wrote it would report violations and exit 0 — the worst shape for a checklist, since it passes locally and fails in CI. And the heading said "Three" while listing four.

Now matched to typecheck.yml line for line, with the flags called out rather than left to be noticed:

node scripts/icon-scss-audit.mjs                 # also fails if a retired SCSS rule reappears
node scripts/icon-tsx-audit.mjs --fail           # icon sizing, see docs/icons.md
node scripts/slim-entries-audit.mjs --fail       # feed payload invariants
node scripts/origin-config-audit.mjs --self-test  # prove the rules still fire, THEN enforce
node scripts/origin-config-audit.mjs --fail

with "⛔ Copy the flags exactly: without --fail these REPORT and exit 0, so a violation passes locally and fails in CI."

Ordered as CI runs them, and icon-scss-audit deliberately has no flag because that is how the workflow invokes it.

node scripts/origin-config-audit.mjs --self-test && node scripts/origin-config-audit.mjs --fail
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Outdated
```

`origin-config-audit` keeps thresholds, source addresses, inline allowlists, secret markers and
fail-open wildcard includes out of `infra/`. It reads **comments** too — an earlier version
stripped them and reported a clean run while every rate sat in prose two lines above. Run
`--self-test` first: it proves each rule still fires, so a regex edited into uselessness is
caught in the same step that relies on it.

### Running Single Tests

```bash
Expand Down Expand Up @@ -99,6 +115,12 @@ pnpm publish:ui
- **packages/wallets** - Multi-chain wallet management (`@ecency/wallets`)
- **packages/render-helper** - Markdown rendering utilities (`@ecency/render-helper`)
- **packages/ui** - Shared UI component library (`@ecency/ui`)
- **infra/origin** - The web origin nginx vhosts (`eu`/`us.ecency.com.conf`), tracked since
2026-08-20. ⛔ **This repo is public**: structure is committed, thresholds and addresses are
not — they live in host-only includes. Enforced by `scripts/origin-config-audit.mjs` in CI.
⛔ CI does **not** deploy these; they are applied by hand to both boxes, and the rule is
apply-and-reload BEFORE committing so the tracked copy matches what is running. See
`infra/origin/README.md`.

All packages use `workspace:*` protocol for local dependencies. The main app transpiles workspace packages during build (configured in `next.config.js`).

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,9 @@ CF-Cache-Status: HIT

### Infra configuration

Nginx and CF worker configs live in the infra repo; their behaviour is
documented here in `docs/cache/`. The rules are simple: **respect origin
The web origin nginx vhosts live in this repo at `infra/origin/` (tracked since
2026-08-20), guarded by `scripts/origin-config-audit.mjs` in CI; the CF worker config
lives in the infra repo. Behaviour is documented here in `docs/cache/`. The rules are simple: **respect origin
`Cache-Control`**, **bypass on `active_user` cookie** and **preserve
`x-cache-tier`** in the response headers.

Expand Down
14 changes: 10 additions & 4 deletions apps/self-hosted/hosting/nginx-multi-tenant.conf
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,11 @@ server {
# THIS BOX (host.docker.internal, an extra_hosts entry in the compose file),
# deliberately not through public ecency.com: a Cloudflare hop would make
# every tenant's signup look like one datacenter client (one IP for consent
# records, one bucket for rate limits, and a bot challenge would break the
# form outright). The reader's address travels as CF-Connecting-IP, set from
# records, one bucket for rate limits, and an EDGE bot challenge would break
# the form outright). Note the form DOES carry a Turnstile widget of its own
# since 2026-08-20, solved in-page before submit and verified by the relay;
# that is a different thing from an edge interstitial, which the reader could
# not clear. The reader's address travels as CF-Connecting-IP, set from
# X-Real-IP, which the HOST's edge vhost asserted from its own remote_addr;
# the origin's /api/ location forwards exactly that header pair to the app.
# Managed instances only, by construction: only they sit behind this nginx.
Expand Down Expand Up @@ -247,8 +250,11 @@ server {
# THIS BOX (host.docker.internal, an extra_hosts entry in the compose file),
# deliberately not through public ecency.com: a Cloudflare hop would make
# every tenant's signup look like one datacenter client (one IP for consent
# records, one bucket for rate limits, and a bot challenge would break the
# form outright). The reader's address travels as CF-Connecting-IP, set from
# records, one bucket for rate limits, and an EDGE bot challenge would break
# the form outright). Note the form DOES carry a Turnstile widget of its own
# since 2026-08-20, solved in-page before submit and verified by the relay;
# that is a different thing from an edge interstitial, which the reader could
# not clear. The reader's address travels as CF-Connecting-IP, set from
# X-Real-IP, which the HOST's edge vhost asserted from its own remote_addr;
# the origin's /api/ location forwards exactly that header pair to the app.
# Managed instances only, by construction: only they sit behind this nginx.
Expand Down
4 changes: 4 additions & 0 deletions apps/web/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -56,5 +56,9 @@ NEWSLETTER_SERVICE_TOKEN=
# deployment that has not configured the newsletter at all never reaches that code, since
# the route already 503s without NEWSLETTER_API_URL and NEWSLETTER_SERVICE_TOKEN.
# specs/deploy/newsletter-wiring pins it on the web service in both compose files.
# ⛔ The SITEKEY is inlined at BUILD time, not read at runtime, so it must reach the image
# build: apps/web/Dockerfile takes it as an ARG and both deploy workflows pass it as a build
# arg. Setting it only in the runtime environment has no effect, and the client silently falls
# back to the literal in features/shared/turnstile.tsx.
NEXT_PUBLIC_TURNSTILE_SITEKEY=
TURNSTILE_SECRET=
7 changes: 6 additions & 1 deletion docs/cache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,9 +150,14 @@ Implemented in this repo:
- Entry-page TTL refined by post age via L1 Map + per-host Redis L2
- `scripts/purge-cache.sh` supports manual DMCA / moderation invalidation

Tracked here, applied by hand (CI does NOT deploy them):

- Nginx (`ssrcache` zone) on each origin host — the vhosts are in `infra/origin/`
since 2026-08-20; see [nginx.md](./nginx.md) and `infra/origin/README.md` for the
public-repo contract that keeps thresholds and addresses off the record

Operated outside this repo:

- Nginx (`ssrcache` zone) on each origin host — see [nginx.md](./nginx.md)
- Cloudflare worker `ecency-geo-router` — see [cloudflare-worker.md](./cloudflare-worker.md)

Before production rollout of changes here, verify `x-cache-tier` values
Expand Down
64 changes: 26 additions & 38 deletions docs/cache/nginx.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Nginx Cache Alignment

Nginx (`ssrcache` zone) sits between the CF worker and vision_web. It runs
on each origin server (eu/us/asia.ecency.com). Because the worker has
Nginx (`ssrcache` zone) sits between the CF worker and vision_web. It runs on each
origin server (EU and US; the asia origin was decommissioned). Because the worker has
already keyed on auth-class and only forwards cacheable requests, **nginx
does NOT need to gate on `active_user` cookie itself** — origin's
`Cache-Control` is the source of truth.
Expand All @@ -25,45 +25,33 @@ proxy_cache_path /var/cache/nginx/ssr levels=1:2

## Per-host config

⛔ **The vhosts are tracked: read `infra/origin/eu.ecency.com.conf` and
`us.ecency.com.conf` rather than a snippet here.** This section used to carry a
hand-copied `server { … }` block, which drifted and began contradicting the real config

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the bot map consumed by the parity spec

Removing the $html_limited_bot { ... } block makes apps/web/src/specs/features/next-middleware/social-bot-metadata.spec.ts deterministically fail: lines 114–116 read this document and require that exact map before comparing it with htmlLimitedBots. Consequently every workflow running pnpm -r test, including PR-branch.yml, fails with “the $html_limited_bot map is missing”; retain a machine-readable copy or update the parity test to read another canonical source.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in fa62b35. This was a real break and my judgement was wrong.

I read the whole ## Per-host config block as a stale hand-copy. Half of it was: the server { … } stanza had drifted into contradicting the tracked vhost, showing add_header X-Cache-Tier … always which infra/origin/eu.ecency.com.conf explicitly forbids because the upstream already sets it and adding it emitted the header twice on every response.

The map had not drifted, and is duplicated because it is enforced. social-bot-metadata.spec.ts:114-116 parses it out of this file and asserts term-for-term parity with htmlLimitedBots in next.config.js. So the block is not documentation of the config — it is a checked copy.

The part that should have stopped me: I wrote the invariant into my replacement text ("must stay in step with htmlLimitedBots") in the same edit that deleted the thing enforcing it. Documenting a rule while removing its enforcement is the exact failure the spec exists to prevent.

Restored, under a heading that says so rather than leaving the next reader to make the same call:

The bot map is CANONICAL here — do not remove it

⛔ This block is not documentation of the config, it IS a copy the test suite reads … Keep the fenced nginx block and the "~*( … )" shape — the test matches on them.

The stale server { } stanza stays removed, since that half of the finding was correct.

Verified: social-bot-metadata.spec.ts 5/5 (it was 4 passed / 1 failed, matching the repro), the whole next-middleware suite 165/165, and the full web suite 3090/3090.

Comment thread
greptile-apps[bot] marked this conversation as resolved.
Outdated
— it showed `add_header X-Cache-Tier $upstream_http_x_cache_tier always;`, which the
tracked file explicitly forbids because the upstream already sets that header and nginx
proxies it through, so adding it emitted the header **twice on every response**. It also
showed `always` on `X-Cache-Status`, which the real config deliberately omits: that value
is a cache diagnostic, not something we owe an error response.

What matters conceptually, and is stable:

```nginx
# See "Why the bot UA class is in the cache key" below.
map $http_user_agent $html_limited_bot {
default "";
"~*(Googlebot|[\w-]+-Google|Google-[\w-]+|googleweblight|Chrome-Lighthouse|Slurp|DuckDuckBot|baiduspider|yandex|sogou|bitlybot|tumblr|vkShare|quora link preview|redditbot|ia_archiver|Bingbot|BingPreview|applebot|facebookexternalhit|facebookcatalog|Twitterbot|LinkedInBot|Slackbot|Discordbot|TelegramBot|WhatsApp|SkypeUriPreview|Yeti)" "|htmlbot";
}

server {
listen 80;
server_name eu.ecency.com; # or us per host

location / {
proxy_cache ssrcache;
proxy_cache_key "$request_uri$html_limited_bot";
# Defer to origin Cache-Control. Fallback for responses without it.
proxy_cache_valid 200 0;
proxy_cache_valid any 30s;
# Stale-while-revalidate semantics
proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;
proxy_cache_lock_timeout 5s;

# Observability
add_header X-Cache-Status $upstream_cache_status always;
add_header X-Cache-Tier $upstream_http_x_cache_tier always;

proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5s;
proxy_send_timeout 20s;
proxy_read_timeout 20s;
}
}
proxy_cache ssrcache;
proxy_cache_key "$request_uri$html_limited_bot"; # see the next section
proxy_cache_valid 200 0; # defer to origin Cache-Control
proxy_cache_valid any 30s;
proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;
```

The `$html_limited_bot` map lives at `http` level in `/etc/nginx/nginx.conf`, which is
**not** tracked (see `infra/origin/README.md`, "What these files depend on"). Its pattern
list must stay in step with `htmlLimitedBots` in `next.config.js`: if the app serves a
blocking render to an agent the map does not classify, that response and a browser's
share one cache entry.

## Why the bot UA class is in the cache key

`htmlLimitedBots` in `apps/web/next.config.js` makes Next.js render metadata
Expand Down
24 changes: 22 additions & 2 deletions infra/origin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,28 @@ What IS committed and deliberately so:

**Comments count as published.** A threshold quoted in a comment is as disclosed as one in
a directive, so the vhost comments name the include rather than the number. The audit
reads comments for exactly this reason: an earlier version stripped them and reported a
clean run while every rate sat in prose two lines above.
(`scripts/origin-config-audit.mjs`, run from `.github/workflows/typecheck.yml` as
`--self-test` then `--fail`) reads comments for exactly this reason: an earlier version
stripped them and reported a clean run while every rate sat in prose two lines above.

## Why `location /` does not answer HEAD itself

It used to: `if ($request_method = HEAD) { add_header Cache-Control no-store; return 200; }`.
That made every HEAD report a live page whatever the truth was — `GET /@good-karma/points`
answered 307 while `HEAD` answered 200 — so link checkers, uptime probes and crawlers were all
told the wrong thing. Removed 2026-08-20 (#1575).

The `always` on the CORS and security `add_header`s is the **other half of the same fix**, not
a tidy-up. While every HEAD was a fabricated 200 those headers applied, because 200 is in
`add_header`'s default status list; deleting the block alone would have made HEAD reach real
404/429/5xx responses with the headers gone. `always` alone fixes nothing either, since an
`add_header` nested inside the `if` suppresses inheritance regardless. `X-Cache-Status`
deliberately keeps its default list: it is a cache diagnostic, not something we owe an error
response.

⛔ Verify this against the origin, never the public hostname. Cloudflare normalises the request
and already answered 307 with headers, so `curl -I https://ecency.com/...` reports the bug as
already fixed.

## What these files depend on

Expand Down
Loading