Skip to content

Commit a93977f

Browse files
bloveclaude
andauthored
Report visible engaged time, and finish the PostHog cutover (#1112)
* feat(website): report visible engaged time so readers stop counting as bounces posthog-js fires $pageleave only from its pagehide/unload handler, never on visibilitychange, and PostHog derives session_duration from the span of a session's events. A visitor who lands, reads, clicks nothing and leaves the tab open therefore emits one $pageview, records zero seconds, and is scored as a bounce. Measured over 30 days: 13.6% of desktop and 17.8% of mobile homepage sessions. The tab-close lands in a rotated session - 67 orphan pageleave-only sessions, median gap 221 minutes - which is what rules out an ad blocker. The gap concentrated where nothing passive could fire. marketing:stage_progress is gated to 1024x720 and fired on 0.0% of mobile sessions; 91% of mobile sessions emitted nothing but the pageview, so mobile engagement was not measurable at all. marketing:engaged_time has no viewport gate. It counts visible time only, pausing while the tab is hidden, so it corrects the recorded duration rather than inflating it. That distinction is the whole design: the hidden-time tests are load-bearing, not incidental. 10s matches PostHog's own bounce cutoff so a genuine ten-second read stops being recorded as zero. All 14 assertions mutation-verified: hidden time counted, threshold dedupe removed, visible-run restarted, cleanup dropped, listener unregistered, and interval never installed each turn exactly the expected tests red. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(growth): record the cutover and the established $pageleave cause The bounce series broke at 2026-09-19T02:05Z: the pageview fix deployed and autocapture_opt_out was flipped, verified by $autocapture events arriving and a soft navigation producing a $pageview. The prediction wording is now past tense, and the ~2 hour sliver between the pinned table window and the cutover is called out rather than left to be discovered. Replaces the "missing across every browser" framing with the actual mechanism. The uniformity across browsers was the symptom of a behavioural cause - tabs left open past the 30 minute idle timeout - not evidence of a transport problem, and the earlier draft read it the wrong way round. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(website): assert the never-seen-tab property where it is load-bearing Code review found that `startVisible: document.visibilityState === 'visible'` could be mutated to `true` with all 14 tests still green. That single argument IS the production implementation of "a tab nobody looked at reports nothing" - the exact inflation this design exists to prevent - and it was asserted only in the tracker unit test, the layer that cannot break it in production. A regression would let a background tab's throttled interval fire both thresholds for a page never seen. Three more surviving mutations closed: a leaked visibilitychange listener (which would emit engaged_time for every earlier page in the session, breaking the two-per-pageview bound), the [pathname] effect dependency, and `accumulatedMs +=` collapsing to `=` (invisible with one hide/show cycle, wrong from the second onward). All four now fail exactly their own test. Also stops the 1Hz interval once both thresholds have fired, rather than waking every second for the life of every page on the site. Docs: pin the orphan-session definition and window so the figure is re-derivable, and note that stage_progress is suppressed by prefers-reduced-motion as well as by viewport. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 23120d1 commit a93977f

10 files changed

Lines changed: 532 additions & 30 deletions

File tree

‎apps/website/src/app/layout.tsx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ import {
2020
} from '../lib/site-metadata';
2121
import { getFormPolicy } from '../lib/growth/form-policy';
2222
import { WebsiteSignals } from '../components/shared/WebsiteSignals';
23+
import { EngagedTimeSignal } from '../components/shared/EngagedTimeSignal';
2324
import { websiteContentCatalog } from '../lib/growth/website-content';
2425

2526
const display = Archivo_Black({
@@ -97,6 +98,7 @@ export default function RootLayout({
9798
*/}
9899
<JsonLd data={rootJsonLd()} />
99100
<WebsiteSignals catalog={websiteContentCatalog()} />
101+
<EngagedTimeSignal />
100102
<Nav />
101103
<div id="site-content">
102104
<main>
Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
import { render, act, cleanup } from '@testing-library/react';
2+
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
3+
import { EngagedTimeSignal } from './EngagedTimeSignal';
4+
5+
vi.mock('../../lib/analytics/client', () => ({
6+
trackEngagedTime: vi.fn(),
7+
}));
8+
9+
let pathname = '/';
10+
vi.mock('next/navigation', () => ({
11+
usePathname: () => pathname,
12+
}));
13+
14+
import { trackEngagedTime } from '../../lib/analytics/client';
15+
16+
const tracked = vi.mocked(trackEngagedTime);
17+
18+
/**
19+
* `engaged-time.spec.ts` covers the accounting. These cover the wiring, which
20+
* is where this can silently do nothing: an effect that never runs, an
21+
* interval that is never installed, or a listener that outlives the component.
22+
* The unit tests would stay green through all three.
23+
*/
24+
function setVisibility(state: 'visible' | 'hidden') {
25+
Object.defineProperty(document, 'visibilityState', {
26+
configurable: true,
27+
get: () => state,
28+
});
29+
document.dispatchEvent(new Event('visibilitychange'));
30+
}
31+
32+
describe('EngagedTimeSignal', () => {
33+
beforeEach(() => {
34+
vi.useFakeTimers();
35+
tracked.mockClear();
36+
pathname = '/';
37+
setVisibility('visible');
38+
});
39+
40+
afterEach(() => {
41+
cleanup();
42+
vi.useRealTimers();
43+
});
44+
45+
it('reports ten engaged seconds after ten visible seconds', () => {
46+
render(<EngagedTimeSignal />);
47+
act(() => {
48+
vi.advanceTimersByTime(10_000);
49+
});
50+
expect(tracked).toHaveBeenCalledWith(10);
51+
});
52+
53+
it('reports nothing before the first threshold', () => {
54+
render(<EngagedTimeSignal />);
55+
act(() => {
56+
vi.advanceTimersByTime(9_000);
57+
});
58+
expect(tracked).not.toHaveBeenCalled();
59+
});
60+
61+
it('reports thirty seconds as a second event, not a replacement', () => {
62+
render(<EngagedTimeSignal />);
63+
act(() => {
64+
vi.advanceTimersByTime(30_000);
65+
});
66+
expect(tracked).toHaveBeenNthCalledWith(1, 10);
67+
expect(tracked).toHaveBeenNthCalledWith(2, 30);
68+
});
69+
70+
it('does not accrue engagement while the tab is hidden', () => {
71+
render(<EngagedTimeSignal />);
72+
act(() => {
73+
vi.advanceTimersByTime(3_000);
74+
setVisibility('hidden');
75+
vi.advanceTimersByTime(120_000);
76+
});
77+
expect(tracked).not.toHaveBeenCalled();
78+
});
79+
80+
it('reports nothing for a tab that was never visible at mount', () => {
81+
// THE honesty property, asserted where it is load-bearing. The production
82+
// implementation of "a never-seen tab reports nothing" is the single
83+
// `startVisible: document.visibilityState === 'visible'` argument; a
84+
// regression to `true` would let a background tab's throttled interval
85+
// fire both thresholds for a page nobody ever looked at.
86+
setVisibility('hidden');
87+
render(<EngagedTimeSignal />);
88+
act(() => {
89+
vi.advanceTimersByTime(120_000);
90+
});
91+
expect(tracked).not.toHaveBeenCalled();
92+
});
93+
94+
it('ignores visibility changes after unmount', () => {
95+
// A leaked listener would let one tab-away emit engaged_time for every
96+
// page visited earlier in the session, each stamped with the CURRENT
97+
// source_page, breaking the two-events-per-pageview bound.
98+
const { unmount } = render(<EngagedTimeSignal />);
99+
act(() => {
100+
vi.advanceTimersByTime(30_000);
101+
});
102+
tracked.mockClear();
103+
unmount();
104+
act(() => {
105+
setVisibility('hidden');
106+
setVisibility('visible');
107+
vi.advanceTimersByTime(60_000);
108+
});
109+
expect(tracked).not.toHaveBeenCalled();
110+
});
111+
112+
it('starts a fresh budget on each pathname, not once per document', () => {
113+
const { rerender } = render(<EngagedTimeSignal />);
114+
act(() => {
115+
vi.advanceTimersByTime(30_000);
116+
});
117+
expect(tracked).toHaveBeenCalledTimes(2);
118+
tracked.mockClear();
119+
pathname = '/pricing';
120+
rerender(<EngagedTimeSignal />);
121+
act(() => {
122+
vi.advanceTimersByTime(10_000);
123+
});
124+
expect(tracked).toHaveBeenCalledExactlyOnceWith(10);
125+
});
126+
127+
it('stops reporting once unmounted', () => {
128+
// A leaked interval would keep attributing engagement to a page the
129+
// visitor has already navigated away from.
130+
const { unmount } = render(<EngagedTimeSignal />);
131+
unmount();
132+
act(() => {
133+
vi.advanceTimersByTime(60_000);
134+
});
135+
expect(tracked).not.toHaveBeenCalled();
136+
});
137+
});
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
'use client';
2+
3+
import { useEffect } from 'react';
4+
import { usePathname } from 'next/navigation';
5+
import { createEngagedTimeTracker } from '../../lib/analytics/engaged-time';
6+
import { trackEngagedTime } from '../../lib/analytics/client';
7+
8+
/**
9+
* How often to re-evaluate. The tracker derives elapsed time from a clock
10+
* rather than counting ticks, so a browser throttling this interval in a
11+
* background tab delays the event but never miscounts it.
12+
*/
13+
const TICK_MS = 1_000;
14+
15+
/**
16+
* Emits `marketing:engaged_time` once the visitor has genuinely been looking
17+
* at the page for 10s, and again at 30s.
18+
*
19+
* Without this the site has no passive engagement signal on most viewports:
20+
* `marketing:stage_progress` is gated to at least 1024x720 and fired on 0.0%
21+
* of mobile sessions in the 30 days to 2026-09-18. A visitor who reads and
22+
* clicks nothing emitted one `$pageview` and nothing else, which PostHog
23+
* scores as a zero-second session and therefore a bounce.
24+
*
25+
* Re-created per pathname so engaged time is measured per page rather than
26+
* per document, matching `capture_pageview: 'history_change'`.
27+
*/
28+
export function EngagedTimeSignal() {
29+
const pathname = usePathname();
30+
31+
useEffect(() => {
32+
if (typeof document === 'undefined' || typeof window === 'undefined') return;
33+
34+
const tracker = createEngagedTimeTracker({
35+
now: () => Date.now(),
36+
startVisible: document.visibilityState === 'visible',
37+
onThreshold: (thresholdMs) => trackEngagedTime(Math.round(thresholdMs / 1000)),
38+
});
39+
40+
const onVisibilityChange = () => {
41+
tracker.setVisible(document.visibilityState === 'visible');
42+
// Re-evaluate immediately: returning to a tab that already passed a
43+
// threshold should not wait out another interval.
44+
tracker.tick();
45+
};
46+
47+
document.addEventListener('visibilitychange', onVisibilityChange);
48+
// Stop waking up once both thresholds have fired. Without this the
49+
// interval runs at 1Hz for the life of every page on the site, doing
50+
// nothing.
51+
const intervalId = window.setInterval(() => {
52+
tracker.tick();
53+
if (tracker.isComplete()) window.clearInterval(intervalId);
54+
}, TICK_MS);
55+
56+
return () => {
57+
document.removeEventListener('visibilitychange', onVisibilityChange);
58+
window.clearInterval(intervalId);
59+
};
60+
}, [pathname]);
61+
62+
return null;
63+
}

‎apps/website/src/lib/analytics/client.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,11 @@ export function trackStageProgress(
5050
...(beat ? { beat } : {}),
5151
});
5252
}
53+
54+
/**
55+
* Passive engagement. `engaged_seconds` is visible time only, so it can be
56+
* read as real attention rather than a tab left open.
57+
*/
58+
export function trackEngagedTime(engaged_seconds: number) {
59+
track(analyticsEvents.marketingEngagedTime, { engaged_seconds });
60+
}
Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
import { describe, expect, it, vi } from 'vitest';
2+
import {
3+
createEngagedTimeTracker,
4+
ENGAGED_TIME_THRESHOLDS_MS,
5+
} from './engaged-time';
6+
7+
/**
8+
* Why this exists.
9+
*
10+
* PostHog derives `session_duration` from `max(timestamp) - min(timestamp)`
11+
* over a session's events, and `$pageleave` only fires on `pagehide` — never
12+
* on `visibilitychange`. A visitor who lands, reads, clicks nothing and leaves
13+
* the tab open therefore emits exactly one event, records a duration of zero,
14+
* and is classified as a bounce. Measured 2026-09-18: that is 13.6% of desktop
15+
* and 17.8% of mobile homepage sessions.
16+
*
17+
* This tracker exists to make the recorded duration reflect the real one. That
18+
* only holds if it counts *visible* time and nothing else — time in a hidden
19+
* background tab is not engagement, and counting it would inflate the metric
20+
* rather than correct it. The hidden-time tests below are the ones that keep
21+
* this honest, so treat them as load-bearing rather than incidental.
22+
*/
23+
24+
function setup(startVisible = true) {
25+
let clock = 1_000;
26+
const onThreshold = vi.fn();
27+
const tracker = createEngagedTimeTracker({
28+
onThreshold,
29+
now: () => clock,
30+
startVisible,
31+
});
32+
return {
33+
onThreshold,
34+
tracker,
35+
advance(ms: number) {
36+
clock += ms;
37+
tracker.tick();
38+
},
39+
advanceWithoutTick(ms: number) {
40+
clock += ms;
41+
},
42+
};
43+
}
44+
45+
describe('engaged time thresholds', () => {
46+
it('exports 10s and 30s, the first matching PostHog bounce cutoff', () => {
47+
// 10s is not arbitrary: PostHog's bounce test is `session_duration >= 10s`.
48+
// An event at a genuine 10 visible seconds makes the recorded duration
49+
// match reality for a reader who never clicks.
50+
expect(ENGAGED_TIME_THRESHOLDS_MS).toEqual([10_000, 30_000]);
51+
});
52+
53+
it('fires the first threshold only once ten visible seconds have passed', () => {
54+
const { advance, onThreshold } = setup();
55+
advance(9_999);
56+
expect(onThreshold).not.toHaveBeenCalled();
57+
advance(1);
58+
expect(onThreshold).toHaveBeenCalledExactlyOnceWith(10_000);
59+
});
60+
61+
it('fires each threshold exactly once, however many times it is ticked', () => {
62+
const { advance, onThreshold } = setup();
63+
advance(10_000);
64+
advance(1_000);
65+
advance(1_000);
66+
expect(onThreshold).toHaveBeenCalledExactlyOnceWith(10_000);
67+
});
68+
69+
it('fires the second threshold at thirty visible seconds', () => {
70+
const { advance, onThreshold } = setup();
71+
advance(30_000);
72+
expect(onThreshold).toHaveBeenNthCalledWith(1, 10_000);
73+
expect(onThreshold).toHaveBeenNthCalledWith(2, 30_000);
74+
expect(onThreshold).toHaveBeenCalledTimes(2);
75+
});
76+
77+
it('does not count time while the page is hidden', () => {
78+
// The honesty test. A backgrounded tab accrues wall-clock time but no
79+
// engagement; counting it would inflate session duration rather than
80+
// correct it.
81+
const { advance, tracker, onThreshold } = setup();
82+
advance(5_000);
83+
tracker.setVisible(false);
84+
advance(60_000);
85+
expect(onThreshold).not.toHaveBeenCalled();
86+
expect(tracker.visibleMs()).toBe(5_000);
87+
});
88+
89+
it('resumes accumulating when the page becomes visible again', () => {
90+
const { advance, tracker, onThreshold } = setup();
91+
advance(6_000);
92+
tracker.setVisible(false);
93+
advance(60_000);
94+
tracker.setVisible(true);
95+
advance(4_000);
96+
expect(onThreshold).toHaveBeenCalledExactlyOnceWith(10_000);
97+
expect(tracker.visibleMs()).toBe(10_000);
98+
});
99+
100+
it('keeps time from earlier visible runs across repeated hide/show cycles', () => {
101+
// One cycle cannot distinguish `accumulatedMs +=` from `accumulatedMs =`,
102+
// because the accumulator is still zero. Two can.
103+
const { advance, tracker, onThreshold } = setup();
104+
advance(4_000);
105+
tracker.setVisible(false);
106+
advance(60_000);
107+
tracker.setVisible(true);
108+
advance(4_000);
109+
tracker.setVisible(false);
110+
advance(60_000);
111+
tracker.setVisible(true);
112+
advance(2_000);
113+
expect(tracker.visibleMs()).toBe(10_000);
114+
expect(onThreshold).toHaveBeenCalledExactlyOnceWith(10_000);
115+
});
116+
117+
it('reports completion only once every threshold has fired', () => {
118+
const { advance, tracker } = setup();
119+
advance(10_000);
120+
expect(tracker.isComplete()).toBe(false);
121+
advance(20_000);
122+
expect(tracker.isComplete()).toBe(true);
123+
});
124+
125+
it('never fires for a page that starts hidden and is never seen', () => {
126+
// A prerendered or background-opened tab must not report engagement.
127+
const { advance, onThreshold, tracker } = setup(false);
128+
advance(120_000);
129+
expect(onThreshold).not.toHaveBeenCalled();
130+
expect(tracker.visibleMs()).toBe(0);
131+
});
132+
133+
it('counts time that elapsed between ticks, not tick count', () => {
134+
// The production caller ticks on an interval; a throttled background
135+
// timer must not under-report time the page was genuinely visible.
136+
const { advanceWithoutTick, tracker, onThreshold } = setup();
137+
advanceWithoutTick(30_000);
138+
tracker.tick();
139+
expect(onThreshold).toHaveBeenCalledTimes(2);
140+
});
141+
142+
it('does not double count when told it is visible twice', () => {
143+
const { advance, tracker, onThreshold } = setup();
144+
advance(5_000);
145+
tracker.setVisible(true);
146+
advance(4_999);
147+
expect(onThreshold).not.toHaveBeenCalled();
148+
expect(tracker.visibleMs()).toBe(9_999);
149+
});
150+
});

0 commit comments

Comments
 (0)