feat: allow access tokens to enumerate groups

AGENTS.md

@@ -37,10 +37,10 @@ Upload bearer tokens are long-lived, reusable credentials, not one-time codes:
 ## Personal access token lifecycle
 
 Personal access tokens (`PersonalAccessToken`, raw prefix `gpat_`) are the
-**read-only** counterpart to upload tokens. They authenticate reads — currently the
-streaming group export (`GET /api/v1/groups/<slug>/export/`) — never uploads. They
-are a distinct model and credential from `UploadToken`; the two are never
-interchangeable.
+**read-only** counterpart to upload tokens. They authenticate the group index and
+streaming group export (`GET /api/v1/groups/` and
+`GET /api/v1/groups/<slug>/export/`) — never uploads. They are a distinct model and
+credential from `UploadToken`; the two are never interchangeable.
 
 - A user mints and revokes their own from the profile page; the raw token is shown
   exactly once and only its hash is stored.

docs/api-v1.md

@@ -7,24 +7,30 @@ JSONL evidence.
 
 ## Authentication
 
-All read APIs require a logged-in Goggles user. Upload APIs use reusable bearer
-tokens and are documented separately in the app workflow notes.
-
-The streaming group export additionally accepts a **personal access token** — a
-read-only bearer credential a user mints from their profile page (or an operator
-mints for a service account with `manage.py create_access_token "<name>" --user
-<username>`). Send it as `Authorization: Bearer gpat_…`. A personal access token
-authorizes only the streaming group export (below) — not uploads, and not the
-session-authenticated read APIs; it is revoked by the owner, by an admin, by expiry,
-or by deactivating the owning user. Upload tokens (`goggles_…`) and personal access
-tokens (`gpat_…`) are distinct credentials and never interchangeable.
+Most read APIs require a logged-in Goggles user session. The group index and
+streaming group export additionally accept a **personal access token** bearer
+credential. Send it as `Authorization: Bearer gpat_…`. On those bearer-enabled
+endpoints, missing, malformed, invalid, inactive, expired, or owner-deactivated
+credentials return `401` JSON — never an HTML login redirect.
+
+Upload APIs use reusable upload bearer tokens (`goggles_…`) and are documented
+separately in the app workflow notes. Upload tokens never authorize read APIs.
+
+Personal access tokens are read-only credentials a user mints from their profile
+page (or an operator mints for a service account with
+`manage.py create_access_token "<name>" --user <username>`). They authorize the
+group list (`GET /api/v1/groups/`) and the streaming group export
+(`GET /api/v1/groups/{slug}/export/`) — not uploads or the other
+session-authenticated projection APIs. A token is revoked by the owner, by an
+admin, by expiry, or by deactivating the owning user. Upload tokens and
+personal access tokens are distinct credentials and never interchangeable.
 
 The current internal deployment treats authenticated Goggles users as one shared
-internal tenant. Even so, endpoint implementations should route through
-object-level scope checks before returning group, account, engine, message,
-report, or evidence data. If Goggles later adds tenant or account isolation,
-unauthorized resources should use the same not-found style behavior as unknown
-resources so callers cannot enumerate data outside their scope.
+internal tenant. Even so, endpoint implementations route through a shared
+object-level readable-group scope before returning group data. If Goggles later
+adds tenant or account isolation, unauthorized groups are omitted from the list
+and exports for unknown slugs return `404`, indistinguishable from a missing
+group, so callers cannot enumerate data outside their scope.
 
 Responses must not expose bearer tokens, upload secrets, source IPs, or user
 agents. Derived projection responses carry pointer-only evidence refs. Raw event
@@ -109,6 +115,25 @@ metadata is the field-level hook for enforcing stricter access.
 - `GET /api/v1/accounts/{account_ref}/groups/`
 - `GET /api/v1/engines/{engine_id}/groups/`
 
+`GET /api/v1/groups/` accepts a logged-in session or a personal access token.
+It returns metadata for every group the reader may export. Newly-created groups
+appear on the next poll without manual slug configuration.
+
+Query parameters:
+
+- `limit`: defaults to `100`, capped at `500`
+- `offset`: defaults to `0`
+- `updated_since`: optional ISO-8601 timestamp; when set, only groups with
+  `updated_at` strictly after this value are returned. The response echoes the
+  applied filter as `updated_since`.
+
+Results are ordered by `(updated_at desc, slug asc)` so polling with
+`updated_since` is stable. Paginated responses include the common pagination
+object (see [Common Query Parameters](#common-query-parameters)). Follow
+`next_offset` until `has_more` is false before advancing the poller's watermark to
+the greatest returned `updated_at`. On the next poll, an empty `groups` array means
+no group metadata changed and no full exports need to be downloaded.
+
 Group responses include `schema_version`, group summary fields, tab counts, and
 classification metadata indicating whether full-data audit content may be
 present.

forensics/analysis.py

@@ -239,8 +239,11 @@ def file_rows_for_group(audit_files, group):
 # ---------------------------------------------------------------------------
 
 
-def group_list_rows():
-    groups = list(AuditGroup.objects.all())
+def group_list_rows(groups=None):
+    if groups is None:
+        groups = list(AuditGroup.objects.all())
+    else:
+        groups = list(groups)
     group_file_counts = audit_file_counts_for_groups(groups)
     group_event_stats, fork_group_ids = event_stats_for_groups(groups)
     for group in groups:

forensics/migrations/0014_auditgroup_updated_at_slug_index.py

@@ -0,0 +1,17 @@
+# Generated by Django 6.0.7 on 2026-07-29 08:43
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('forensics', '0013_personalaccesstoken'),
+    ]
+
+    operations = [
+        migrations.AddIndex(
+            model_name='auditgroup',
+            index=models.Index(fields=['-updated_at', 'slug'], name='forensics_a_updated_408b26_idx'),
+        ),
+    ]

forensics/models.py

@@ -21,6 +21,9 @@ class AuditGroup(models.Model):
 
     class Meta:
         ordering = ["-updated_at", "-created_at"]
+        indexes = [
+            models.Index(fields=["-updated_at", "slug"]),
+        ]
 
     def __str__(self) -> str:
         return self.name

forensics/tests.py

@@ -5774,6 +5774,204 @@ def test_mid_stream_error_yields_error_line_and_no_eof_within_a_200(self):
         self.assertNotIn("eof", {record["t"] for record in records})
 
 
+class GroupListApiTests(TestCase):
+    """GET /api/v1/groups/ — readable group enumeration for sessions and PATs."""
+
+    def setUp(self):
+        ingest_audit_log_bytes(dump_bytes=representative_audit_log().encode("utf-8"))
+        self.group = AuditGroup.objects.get(slug=GROUP_REF)
+        self.user = User.objects.create_user(
+            username="reader", password="correct horse battery staple"
+        )
+        self.url = reverse("api-group-list")
+
+    def test_lists_readable_groups_with_personal_access_token(self):
+        raw_token, token = PersonalAccessToken.issue("watchdog", user=self.user)
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(response.status_code, 200)
+        payload = response.json()
+        self.assertEqual(payload["schema_version"], "goggles-groups/v1")
+        slugs = [group["slug"] for group in payload["groups"]]
+        self.assertIn(self.group.slug, slugs)
+        token.refresh_from_db()
+        self.assertIsNotNone(token.last_used_at)
+
+    def test_requires_authentication_returns_401_json_not_redirect(self):
+        response = self.client.get(self.url)
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_rejects_invalid_bearer_token(self):
+        response = self.client.get(self.url, HTTP_AUTHORIZATION="Bearer gpat_deadbeef_nope")
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_rejects_expired_personal_access_token(self):
+        raw_token, _token = PersonalAccessToken.issue(
+            "stale", user=self.user, expires_at=timezone.now() - timedelta(seconds=1)
+        )
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_rejects_malformed_authorization_scheme(self):
+        response = self.client.get(self.url, HTTP_AUTHORIZATION="Token gpat_not_bearer")
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_rejects_inactive_personal_access_token(self):
+        raw_token, token = PersonalAccessToken.issue("revoked", user=self.user)
+        token.is_active = False
+        token.save(update_fields=["is_active"])
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_rejects_personal_access_token_of_deactivated_owner(self):
+        raw_token, _token = PersonalAccessToken.issue("orphaned", user=self.user)
+        self.user.is_active = False
+        self.user.save(update_fields=["is_active"])
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(response.status_code, 401)
+        self.assertEqual(response.json(), {"error": "authentication required"})
+
+    def test_upload_token_cannot_list_groups(self):
+        raw_token, _token = UploadToken.issue("device")
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(response.status_code, 401)
+
+    def test_lists_with_logged_in_session(self):
+        self.client.login(username="reader", password="correct horse battery staple")
+
+        response = self.client.get(self.url)
+
+        self.assertEqual(response.status_code, 200)
+        slugs = [group["slug"] for group in response.json()["groups"]]
+        self.assertIn(self.group.slug, slugs)
+
+    def test_denied_group_is_omitted_from_list_and_export_returns_404(self):
+        other = AuditGroup.objects.create(
+            name="Out of scope",
+            slug=OTHER_GROUP_REF,
+            group_ref=OTHER_GROUP_REF,
+        )
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+        export_url = reverse("api-group-export-stream", kwargs={"slug": other.slug})
+
+        def scoped_queryset(reader):
+            self.assertEqual(reader.user, self.user)
+            return AuditGroup.objects.filter(slug=self.group.slug)
+
+        with mock.patch("forensics.views.readable_groups_queryset", side_effect=scoped_queryset):
+            list_response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+            export_response = self.client.get(export_url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        self.assertEqual(list_response.status_code, 200)
+        slugs = [group["slug"] for group in list_response.json()["groups"]]
+        self.assertEqual(slugs, [self.group.slug])
+        self.assertEqual(export_response.status_code, 404)
+
+    def test_newly_created_group_appears_without_manual_slug_configuration(self):
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+        new_group = AuditGroup.objects.create(
+            name="Fresh group",
+            slug=OTHER_GROUP_REF,
+            group_ref=OTHER_GROUP_REF,
+        )
+
+        response = self.client.get(self.url, HTTP_AUTHORIZATION=f"Bearer {raw_token}")
+
+        slugs = [group["slug"] for group in response.json()["groups"]]
+        self.assertIn(new_group.slug, slugs)
+
+    def test_response_includes_bounded_pagination_metadata(self):
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+
+        response = self.client.get(
+            self.url,
+            HTTP_AUTHORIZATION=f"Bearer {raw_token}",
+            data={"limit": "1"},
+        )
+
+        payload = response.json()
+        self.assertEqual(payload["pagination"]["limit"], 1)
+        self.assertEqual(payload["pagination"]["offset"], 0)
+        self.assertEqual(payload["pagination"]["returned"], 1)
+        self.assertIn("has_more", payload["pagination"])
+        self.assertIn("next_offset", payload["pagination"])
+
+    def test_updated_since_filters_groups_for_polling(self):
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+        stale = AuditGroup.objects.create(
+            name="Stale",
+            slug="aa" * 32,
+            group_ref="aa" * 32,
+        )
+        AuditGroup.objects.filter(pk=stale.pk).update(updated_at=timezone.now() - timedelta(days=2))
+        stale.refresh_from_db()
+        fresh = AuditGroup.objects.create(
+            name="Fresh",
+            slug=OTHER_GROUP_REF,
+            group_ref=OTHER_GROUP_REF,
+        )
+        since = (timezone.now() - timedelta(hours=1)).isoformat()
+
+        response = self.client.get(
+            self.url,
+            HTTP_AUTHORIZATION=f"Bearer {raw_token}",
+            data={"updated_since": since},
+        )
+
+        slugs = [group["slug"] for group in response.json()["groups"]]
+        self.assertIn(fresh.slug, slugs)
+        self.assertNotIn(stale.slug, slugs)
+        self.assertIn(self.group.slug, slugs)
+
+    def test_updated_since_returns_empty_index_when_nothing_changed(self):
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+        after_all_updates = (timezone.now() + timedelta(seconds=1)).isoformat()
+
+        response = self.client.get(
+            self.url,
+            HTTP_AUTHORIZATION=f"Bearer {raw_token}",
+            data={"updated_since": after_all_updates},
+        )
+
+        self.assertEqual(response.status_code, 200)
+        self.assertEqual(response.json()["groups"], [])
+        self.assertFalse(response.json()["pagination"]["has_more"])
+
+    def test_group_list_enrichment_is_bounded_to_the_requested_page(self):
+        raw_token, _token = PersonalAccessToken.issue("watchdog", user=self.user)
+        for index in range(3):
+            AuditGroup.objects.create(
+                name=f"Extra {index}",
+                slug=f"{index:02d}" * 32,
+                group_ref=f"{index:02d}" * 32,
+            )
+        with mock.patch("forensics.views.group_list_rows", wraps=group_list_rows) as wrapped_rows:
+            self.client.get(
+                self.url,
+                HTTP_AUTHORIZATION=f"Bearer {raw_token}",
+                data={"limit": "2", "offset": "0"},
+            )
+
+        self.assertEqual(len(wrapped_rows.call_args.args[0]), 2)
+
+
 class ProfileAccessTokenTests(TestCase):
     """Self-service personal access tokens on the profile page — strictly
     owner-scoped: a user can only see, mint, and revoke their own."""