Skip to content

Harden the telemetry event system (back-port authup's event-log improvements) #1745

Description

@tada5hi

Context

authup's security event log (authup/authup#3229) was designed from this repo's telemetry event system (@privateaim/telemetry-kit Event, apps/server-telemetry, EntityEventHandler) and deliberately keeps its shape — (scope, name) pair, ref_type/ref_id, denormalized actor_* snapshot, request_* group, expiring + expires_at retention, entity-CRUD bridge with scalar data.diff. While porting, a number of weaknesses in hub's implementation surfaced; authup fixed them and is now the reference implementation. This issue tracks adopting those fixes back. Claims pinned to hub 0.12.1; the full mapping lives in authup's .agents/references/privateaim-hub.md.

Back-port checklist

  • Closed event vocabulary. ref_type/scope/name are free-form strings (min 3 is the only constraint) — typo-able, no compile-time safety. Introduce enums for the known vocabulary (entity bridge: entity scope + created/updated/deleted; component events per emitter), keeping free-form only where genuinely open-ended. authup: EventName/EventScope enums (camelCase values — hub's own convention, e.g. creationStarted).
  • Append-only events. Events are mutable: EventEntity has updated_at, and EventAPI.update() POSTs to events/:id — a route that does not exist in apps/server-telemetry (dead client code), with no EVENT_UPDATE permission ever checked. Remove the update surface; audit-shaped records should be immutable.
  • PII/credential write boundary. Event data (incl. the EntityEventHandler diff) is stored unfiltered — a diff over an entity with secret-bearing scalar columns persists them. authup: secret-denylist on diff keys (/(password|secret|hash|token|credential)/i, fail-closed) + a sanitizer at the write boundary re-validating the diff shape (one-level { next, previous } scalar pairs only).
  • serialize(null) persists the string 'null'. The text + serialize/deserialize transformer pattern (Event data and other entities using it) writes literal 'null' text instead of SQL NULL for null values (deserialize round-trips it, but the column value is wrong and NULL-predicates break). Guard the to side: value === null || typeof value === 'undefined' ? null : serialize(value).
  • IPv6. events.request_ip_address is varchar(15) with a zod.ipv4() validator — IPv6 requests cannot be recorded correctly (the ::1 → 127.0.0.1 rewrite in EntityEventHandler is a symptom, not a fix). Widen to varchar(45), validate v4+v6, drop the rewrite.
  • Index realm_id; prune the index set. realm_id — the primary read filter (UI queries realm_id ∈ [realm, null]) — is the one unindexed column, while ~14 single-column indexes sit on a write-mostly table. Add the realm index; reconsider the rest against actual query shapes.
  • Validator/column width mismatch. EventValidator allows 128 chars for scope/name/ref_type against varchar(64) columns → driver-dependent truncation or insert errors. Single-source the widths; truncate client-controlled strings at the write boundary (authup truncates actor_name/request_* to their column widths in the service).
  • Configurable retention. The entity-bridge TTL is a hard-coded WEEK_IN_MS in EntityEventHandler (and 24h in the master-image-builder call-site), the cleaner cron (0 1 * * *) equally fixed. Make them config keys (authup: eventLogRetentionDays / eventLogEntityRetentionDays / enable flags, with fail-loud cross-checks for dependent features).
  • dataPrevious transport audit. The subscriber payload carries the full pre-mutation entity (dataPrevious) to compute diffs. Today its only consumer is the internal WORK queue — verify nothing ever re-broadcasts that payload (realtime/socket re-publishers), and consider moving dataPrevious out of the shared payload shape onto a side-channel the way authup keeps it on the publish context, never the wire content.
  • (Optional) AsyncLocalStorage request context. authup captures actor/request attribution via an ALS store around the request (mounted after the auth middleware) instead of threading TypeORM SaveOptions.data → queryRunner.data through a base repository — no ORM coupling, and non-HTTP writes attribute as system automatically. Worth evaluating as a simplification of RequestRepositoryAdapter.extendOptionsData.

Out of scope (shared follow-up)

Aggregation/statistics endpoints (counts over time, per action) are missing in both ecosystems — worth designing once and implementing in both; separate issue when tackled.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions