Skip to content

Commit 4d32fe7

Browse files
Merge pull request #6873 from mozilla/mntor-5344
feat(fx-api): add GET /user/breaches for the Firefox API (MNTOR-5344)
2 parents 3982cac + d29323b commit 4d32fe7

16 files changed

Lines changed: 1426 additions & 44 deletions

File tree

‎docs/architecture/flows/firefox-api-auth.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,11 @@ A person can be signed in to FxA in two separate places, possibly as different a
2222

2323
Diagrams show `GET /user/breaches`. `POST /user/breaches/resolutions` authenticates the same way.
2424

25-
After the scope check, [`checkRateLimit`](../../../src/app/api/utils/withFxaUser.ts#L110) allows each FxA uid `FIREFOX_API_RATE_LIMIT_PER_HOUR` requests an hour (default 60) across both endpoints, counted in Redis. Past that it returns `429` with `Retry-After`, the seconds until the window resets. `0` disables the limit. If Redis fails, the request goes through and Sentry gets an event.
25+
After the scope check, [`checkRateLimit`](../../../src/app/api/utils/withFxaUser.ts#L89) allows each FxA uid `FIREFOX_API_RATE_LIMIT_PER_HOUR` requests an hour (default 60) across both endpoints, counted in Redis. Past that it returns `429` with `Retry-After`, the seconds until the window resets. `0` disables the limit. If Redis fails, the request goes through and Sentry gets an event.
26+
27+
After the subscriber lookup, the [`FirefoxApi` feature flag](../../../src/app/api/utils/withFxaUser.ts#L204) gates both endpoints until launch. It's on for a subscriber when the flag is enabled, or when their primary email is in its allow list, both set at `/admin/feature-flags`. When it's off, Monitor returns `503` without `Retry-After`.
28+
29+
If FxA fails to introspect the token, or HIBP fails while answering ([`searchHibpRange`](../../../src/utils/hibp.ts#L388) throws), `withFxaUser` [returns `503`](../../../src/app/api/utils/withFxaUser.ts#L227) with `Retry-After: 300`. A failed lookup never returns a partial or empty `200`, so Firefox keeps the breaches it already holds.
2630

2731
| | "Legacy" auth | "New" auth |
2832
| ------------------------------ | ----------------------- | ----------------------- |
@@ -53,11 +57,11 @@ sequenceDiagram
5357
FE->>BE: GET /user/breaches + access token [3]
5458
BE->>FxA: Is this token valid? [4]
5559
FxA-->>BE: Yes, FxA uid + scopes
56-
Note over BE: Has Monitor scope?<br/>Under rate limit?<br/>Find subscriber by FxA uid
57-
BE-->>FE: 200, user's breaches,<br/>or 403, no-monitor-account [5],<br/>or 429, rate-limited [6]
60+
Note over BE: Has Monitor scope?<br/>Under rate limit?<br/>Find subscriber by FxA uid<br/>FirefoxApi flag on?
61+
BE-->>FE: 200, user's breaches,<br/>or 403, no-monitor-account [5],<br/>or 429, rate-limited [6],<br/>or 503, unavailable [7]
5862
end
5963
60-
Note over FE,BE: [1] GET /v1/account/attached_clients, which Desktop already fetches, or the newer /attached_oauth_clients. Includes Monitor's client_id only while it holds a refresh token. On FxA error, Firefox treats Monitor as not connected<br/>[2] POST /v1/oauth/token, scope apps/monitor. Firefox caches the token<br/>[3] Bearer header. On 401 Firefox silently gets a new token, retries once<br/>[4] POST /v1/introspect<br/>[5] Rare, no subscriber for this FxA uid. Firefox shows sign up, no retry<br/>[6] Firefox waits Retry-After seconds
64+
Note over FE,BE: [1] GET /v1/account/attached_clients, which Desktop already fetches, or the newer /attached_oauth_clients. Includes Monitor's client_id only while it holds a refresh token. On FxA error, Firefox treats Monitor as not connected<br/>[2] POST /v1/oauth/token, scope apps/monitor. Firefox caches the token<br/>[3] Bearer header. On 401 Firefox silently gets a new token, retries once<br/>[4] POST /v1/introspect<br/>[5] Rare, no subscriber for this FxA uid. Firefox shows sign up, no retry<br/>[6] Firefox waits Retry-After seconds<br/>[7] Flag off, FxA down, or HIBP down. Firefox keeps its breaches, waits Retry-After if sent, else its next poll
6165
```
6266

6367
### 1c. Not signed in

‎openapi.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,8 @@ paths:
208208
209209
**Cadence.** Poll no more than hourly, and once a day is enough. Breaches reach HIBP weeks or months after the incident, so polling harder gains nothing. Each account gets an hourly request budget, shared by both endpoints, and we answer the rest with a `429`.
210210
211+
**Unknown parameters.** We ignore query parameters we do not recognise, so adding one never breaks a request.
212+
211213
parameters:
212214
- name: domain
213215
in: query
@@ -253,7 +255,7 @@ paths:
253255
summary: No breaches found
254256
value: []
255257
"400":
256-
description: A query parameter failed validation.
258+
description: A query parameter failed validation, for example an empty or repeated `domain`.
257259
content:
258260
application/json:
259261
schema:
Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
/* This Source Code Form is subject to the terms of the Mozilla Public
2+
* License, v. 2.0. If a copy of the MPL was not distributed with this
3+
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
4+
5+
import {
6+
describe,
7+
it,
8+
expect,
9+
vi,
10+
beforeEach,
11+
afterEach,
12+
afterAll,
13+
} from "vitest";
14+
import { NextRequest } from "next/server";
15+
import MockRedis from "ioredis-mock";
16+
import createDbConnection from "../../../../../../db/connect";
17+
import { seeds } from "../../../../../../test/db";
18+
import { FXA_MONITOR_SCOPE } from "../../../../../../constants";
19+
20+
// Only FxA, HIBP and Redis are fake; Postgres is real
21+
vi.mock("../../../../../../utils/fxa", async (importOriginal) => ({
22+
...(await importOriginal<typeof import("../../../../../../utils/fxa")>()),
23+
introspectToken: vi.fn(),
24+
}));
25+
26+
vi.mock("../../../../../../utils/hibp", async (importOriginal) => ({
27+
...(await importOriginal<typeof import("../../../../../../utils/hibp")>()),
28+
searchHibpRange: vi.fn(),
29+
}));
30+
31+
const redis = new MockRedis();
32+
vi.mock("../../../../../../db/redis/client", () => ({
33+
redisClient: () => redis,
34+
REDIS_ALL_BREACHES_KEY: "breaches",
35+
BREACHES_EXPIRY_SECONDS: 43200,
36+
}));
37+
38+
vi.mock("../../../../../functions/server/logging", () => ({
39+
logger: { error: vi.fn(), info: vi.fn(), debug: vi.fn(), warn: vi.fn() },
40+
}));
41+
42+
import { introspectToken, getSha1 } from "../../../../../../utils/fxa";
43+
import { searchHibpRange } from "../../../../../../utils/hibp";
44+
import { GET } from "./route";
45+
46+
const conn = createDbConnection();
47+
48+
const FXA_UID = "integration-test-fxa-uid";
49+
const PRIMARY = "Primary@Example.com";
50+
const SECONDARY = "secondary@example.com";
51+
const UNVERIFIED = "unverified@example.com";
52+
53+
/** A breach every filter lets through */
54+
function breachSeed(name: string, addedDate: string) {
55+
return seeds.breaches({
56+
name,
57+
domain: `${name.toLowerCase()}.com`,
58+
added_date: new Date(addedDate),
59+
data_classes: ["passwords", "email-addresses"],
60+
is_verified: true,
61+
is_fabricated: false,
62+
is_sensitive: false,
63+
is_retired: false,
64+
is_spam_list: false,
65+
});
66+
}
67+
68+
/**
69+
* Seeds a subscriber, their emails, breaches and the
70+
* FirefoxApi flag. Returns the breach ids by name.
71+
*/
72+
async function seedUser(flagAllowList: string[]) {
73+
const breaches = await conn("breaches")
74+
.insert([
75+
breachSeed("Adobe", "2020-01-01"),
76+
breachSeed("LinkedIn", "2026-01-01"),
77+
{ ...breachSeed("Sensitive", "2025-01-01"), is_sensitive: true },
78+
])
79+
.returning(["id", "name"]);
80+
const ids = Object.fromEntries(breaches.map((b) => [b.name, b.id]));
81+
82+
const [subscriber] = await conn("subscribers")
83+
.insert(
84+
seeds.subscribers({
85+
fxa_uid: FXA_UID,
86+
primary_email: PRIMARY,
87+
primary_verified: true,
88+
// Stored as the website's resolve route writes it
89+
breach_resolution: {
90+
useBreachId: true,
91+
[PRIMARY]: { [ids.Adobe]: { resolutionsChecked: ["passwords"] } },
92+
},
93+
}),
94+
)
95+
.returning("*");
96+
97+
await conn("email_addresses").insert([
98+
seeds.emails(subscriber.id, { email: SECONDARY, verified: true }),
99+
seeds.emails(subscriber.id, { email: UNVERIFIED, verified: false }),
100+
]);
101+
102+
await conn("feature_flag_events").insert({
103+
name: "FirefoxApi",
104+
is_enabled: false,
105+
allow_list: flagAllowList,
106+
created_by_subscriber_id: subscriber.id,
107+
});
108+
}
109+
110+
/** HIBP lists these breach names, per email */
111+
function mockHibp(namesByEmail: Record<string, string[]>) {
112+
vi.mocked(searchHibpRange).mockImplementation(async (sha1) => {
113+
for (const [email, names] of Object.entries(namesByEmail)) {
114+
if (getSha1(email.toLowerCase()) === sha1) return names;
115+
}
116+
return [];
117+
});
118+
}
119+
120+
function authedRequest() {
121+
return new NextRequest("http://localhost/api/firefox/v1/user/breaches", {
122+
headers: { authorization: "Bearer fxa-access-token" },
123+
});
124+
}
125+
126+
beforeEach(async () => {
127+
// E2E "local" reads headers(), needing a request
128+
vi.stubEnv("E2E_TEST_ENV", "ci");
129+
await redis.flushall();
130+
vi.mocked(introspectToken).mockResolvedValue({
131+
active: true,
132+
sub: FXA_UID,
133+
scope: FXA_MONITOR_SCOPE,
134+
client_id: "integration-client",
135+
token_type: "access_token",
136+
});
137+
});
138+
139+
afterEach(async () => {
140+
vi.clearAllMocks();
141+
vi.unstubAllEnvs();
142+
await conn.raw("TRUNCATE TABLE subscribers, breaches CASCADE");
143+
});
144+
145+
afterAll(async () => {
146+
await conn.destroy();
147+
});
148+
149+
describe("GET /api/firefox/v1/user/breaches (integration)", () => {
150+
it("returns each email's breaches and resolutions from Postgres", async () => {
151+
await seedUser([PRIMARY]);
152+
mockHibp({
153+
[PRIMARY]: ["Adobe", "Sensitive"],
154+
[SECONDARY]: ["Adobe", "LinkedIn"],
155+
[UNVERIFIED]: ["LinkedIn"],
156+
});
157+
158+
const res = await GET(authedRequest());
159+
160+
expect(res.status).toBe(200);
161+
expect(await res.json()).toEqual([
162+
{
163+
id: "LinkedIn",
164+
breachedAccounts: [{ email: SECONDARY, resolvedDataClasses: [] }],
165+
},
166+
{
167+
id: "Adobe",
168+
breachedAccounts: [
169+
{ email: PRIMARY, resolvedDataClasses: ["passwords"] },
170+
{ email: SECONDARY, resolvedDataClasses: [] },
171+
],
172+
},
173+
]);
174+
// The unverified email is never looked up
175+
expect(searchHibpRange).toHaveBeenCalledTimes(2);
176+
});
177+
178+
it("returns 503 when the user is not on the flag's allow list", async () => {
179+
await seedUser(["someone-else@example.com"]);
180+
181+
const res = await GET(authedRequest());
182+
183+
expect(res.status).toBe(503);
184+
expect(searchHibpRange).not.toHaveBeenCalled();
185+
});
186+
});
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
/* This Source Code Form is subject to the terms of the Mozilla Public
2+
* License, v. 2.0. If a copy of the MPL was not distributed with this
3+
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
4+
5+
// @vitest-environment node
6+
7+
import { describe, it, expect, vi, beforeEach } from "vitest";
8+
import { NextRequest } from "next/server";
9+
import type { FxaSubscriber } from "../../../../utils/withFxaUser";
10+
11+
const subscriber = { id: 7 } as FxaSubscriber;
12+
const CLIENT_ID = "a1b2c3d4e5f6a7b8";
13+
14+
// Skip auth; withFxaUser has its own tests
15+
vi.mock("../../../../utils/withFxaUser", () => ({
16+
withFxaUser:
17+
(
18+
_endpoint: string,
19+
handler: (
20+
req: NextRequest,
21+
ctx: { subscriber: FxaSubscriber; clientId?: string },
22+
) => Promise<Response>,
23+
) =>
24+
(req: NextRequest) =>
25+
handler(req, { subscriber, clientId: CLIENT_ID }),
26+
}));
27+
28+
vi.mock("../../../../../../utils/metrics", () => ({
29+
incFirefoxApiUnknownParam: vi.fn(),
30+
}));
31+
32+
vi.mock("../../../../../functions/server/logging", () => ({
33+
logger: { info: vi.fn() },
34+
}));
35+
36+
vi.mock("../../../../../functions/server/getFirefoxUserBreaches", () => ({
37+
getFirefoxUserBreaches: vi.fn(),
38+
}));
39+
40+
import { getFirefoxUserBreaches } from "../../../../../functions/server/getFirefoxUserBreaches";
41+
import { UpstreamUnavailableError } from "../../../../../../utils/error";
42+
import { incFirefoxApiUnknownParam } from "../../../../../../utils/metrics";
43+
import { logger } from "../../../../../functions/server/logging";
44+
import { GET } from "./route";
45+
46+
const BREACHES = [
47+
{
48+
id: "Adobe",
49+
breachedAccounts: [{ email: "a@example.com", resolvedDataClasses: [] }],
50+
},
51+
];
52+
53+
/** GET with this query string */
54+
function get(query = "") {
55+
return GET(
56+
new NextRequest(`http://localhost/api/firefox/v1/user/breaches${query}`),
57+
);
58+
}
59+
60+
beforeEach(() => {
61+
vi.clearAllMocks();
62+
vi.mocked(getFirefoxUserBreaches).mockResolvedValue(BREACHES);
63+
});
64+
65+
describe("GET /user/breaches", () => {
66+
it("returns this user's breaches", async () => {
67+
const res = await get();
68+
69+
expect(res.status).toBe(200);
70+
expect(await res.json()).toEqual(BREACHES);
71+
expect(getFirefoxUserBreaches).toHaveBeenCalledWith(subscriber, undefined);
72+
});
73+
74+
it("is never cached", async () => {
75+
const res = await get();
76+
77+
expect(res.headers.get("Cache-Control")).toBe("private, no-store");
78+
});
79+
80+
it("passes the domain filter through", async () => {
81+
await get("?domain=adobe.com");
82+
83+
expect(getFirefoxUserBreaches).toHaveBeenCalledWith(
84+
subscriber,
85+
"adobe.com",
86+
);
87+
});
88+
89+
it("ignores unknown params, counting and logging their names", async () => {
90+
const res = await get("?domain=adobe.com&foo=secret&bar=1&foo=2");
91+
92+
expect(res.status).toBe(200);
93+
expect(getFirefoxUserBreaches).toHaveBeenCalledWith(
94+
subscriber,
95+
"adobe.com",
96+
);
97+
expect(incFirefoxApiUnknownParam).toHaveBeenCalledTimes(1);
98+
expect(incFirefoxApiUnknownParam).toHaveBeenCalledWith(
99+
"/user/breaches",
100+
CLIENT_ID,
101+
);
102+
expect(logger.info).toHaveBeenCalledWith("firefox_api_unknown_params", {
103+
endpoint: "/user/breaches",
104+
clientId: CLIENT_ID,
105+
unknownParams: ["foo", "bar"],
106+
});
107+
});
108+
109+
it("records nothing without unknown params", async () => {
110+
await get("?domain=adobe.com");
111+
112+
expect(incFirefoxApiUnknownParam).not.toHaveBeenCalled();
113+
expect(logger.info).not.toHaveBeenCalled();
114+
});
115+
116+
it("lets a HIBP failure reach withFxaUser", async () => {
117+
vi.mocked(getFirefoxUserBreaches).mockRejectedValue(
118+
new UpstreamUnavailableError("hibp", "timeout"),
119+
);
120+
121+
await expect(get()).rejects.toThrow(UpstreamUnavailableError);
122+
});
123+
124+
describe("returns 400 invalid-request", () => {
125+
it.each([
126+
["an empty domain", "?domain=", "domain must not be empty."],
127+
[
128+
"a repeated domain",
129+
"?domain=a.com&domain=b.com",
130+
"domain must appear only once.",
131+
],
132+
])("for %s", async (_label, query, message) => {
133+
const res = await get(query);
134+
135+
expect(res.status).toBe(400);
136+
expect(await res.json()).toEqual({ code: "invalid-request", message });
137+
expect(getFirefoxUserBreaches).not.toHaveBeenCalled();
138+
});
139+
});
140+
});

0 commit comments

Comments
 (0)