feat: add local Safari testing via safaridriver by marcoscaceres · Pull Request #5131 · speced/respec

marcoscaceres · 2026-03-27T15:42:44Z

Enables pnpm start --browser Safari for local testing and adds Safari to CI.

Problem

karma-safari-launcher@1.0.0 (the published package) uses a redirect.html hack that modern Safari treats as a file download rather than opening it. @onslap/karma-safari-launcher (the replacement) depends on wd@1.x which requires a native build that doesn't compile on Node 24+.

Solution

A minimal custom karma launcher (tests/karma.safari.cjs) that talks directly to safaridriver over W3C WebDriver HTTP protocol using Node's built-in http module. No new npm dependencies.

Also fixes Safari-specific issues discovered during testing:

postMessage from srcdoc iframes: skip when parent.location.origin throws SecurityError (Safari opaque-origin restriction) instead of falling back to targetOrigin: "*"
ev.data guard in message handler to prevent crashes from non-object payloads
Error normalization in SpecHelper.js for Safari (undefined rejection reasons crash jasmine-core)
Polling fallback in waitReady for Safari where ev.source !== iframe.contentWindow
textContent instead of innerText for highlight (Safari returns empty for closed <details>)
parseInfoString null guard for Safari's marked tokenizer

Setup (one-time per machine)

sudo safaridriver --enable

CI

Safari runs on macos-latest in both PR and push workflows. macOS runners cost more than Linux but catching Safari-specific regressions is worth it.

Closes #2894

Copilot

Pull request overview

Adds a custom Karma launcher to enable running the project’s unit tests locally in Safari via safaridriver, replacing the deprecated/broken third-party Safari launcher dependency.

Changes:

Introduces tests/karma.safari.cjs, a minimal W3C WebDriver client/launcher that spawns safaridriver and navigates to the Karma URL.
Wires the custom launcher into the shared Karma config (tests/karma.conf.base.cjs).
Removes karma-safari-launcher from dependencies/lockfile.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File	Description
tests/karma.safari.cjs	New custom Safari launcher using `safaridriver` + Node `http`.
tests/karma.conf.base.cjs	Replaces `karma-safari-launcher` plugin with the local Safari launcher.
package.json	Drops `karma-safari-launcher` dependency.
pnpm-lock.yaml	Removes `karma-safari-launcher` entries.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

marcoscaceres · 2026-04-14T17:36:12Z

@copilot i think we should enable this by default to run alongside Chrome and Firefox, even if it’s more expensive to run on macOS in CI. We care that we don’t regress on WebKit/Safari so it’s worth it.

Copilot · 2026-04-14T17:40:14Z

@copilot i think we should enable this by default to run alongside Chrome and Firefox, even if it’s more expensive to run on macOS in CI. We care that we don’t regress on WebKit/Safari so it’s w...

Done in bd0debf — Safari is now enabled in CI on macos-latest runners alongside Chrome and Firefox:

pr.yml: extended the test-karma matrix to include Safari / macos-latest (with sudo safaridriver --enable as a conditional step before installing deps).
push.yml: added a new test-karma-safari job on macos-latest that runs both test:unit and test:integration with BROWSERS: Safari.

marcoscaceres · 2026-04-14T17:49:24Z

@copilot Can you check and fix ci?

Copilot · 2026-04-14T17:54:46Z

@copilot Can you check and fix ci?

Fixed in a7d2784. The CI failure had two root causes on cold macOS runners:

Fixed 500ms startup wait was too short — safaridriver wasn't listening yet, so POST /session got ECONNREFUSED immediately. Replaced with a waitForReady() helper that polls GET /status every 250ms for up to 15s, only proceeding once safaridriver responds.
10s POST /session timeout was too short — Safari itself takes time to launch on a cold runner. Increased to 30s for that call.

The waitForReady also properly races against the process error/exit events (with a settle-once guard), so early failures are still reported quickly with a clear message.

marcoscaceres · 2026-04-14T17:59:23Z

@copilot Couple of lint errors to fix. See ci.

Copilot · 2026-04-14T18:01:42Z

@copilot Couple of lint errors to fix. See ci.

Fixed in dab0d02 — two prettier formatting errors in tests/karma.safari.cjs:

Long reject(new Error(...)) call on line 108 → wrapped to multi-line
Curried arrow const finish = fn => (...args) => { on lines 157–161 → reformatted to prettier's preferred style for curried arrows

marcoscaceres · 2026-04-14T18:25:44Z

Seeing a lot ci fails @copilot. Do your thing, my good every so helpful little digital marvel.

Copilot · 2026-04-14T19:16:23Z

Seeing a lot ci fails @copilot. Do your thing, my good every so helpful little digital marvel.

Fixed in ad409ca — two root causes identified from the Safari CI log:

src/core/pubsubhub.js: window.parent.location.origin throws SecurityError in Safari when srcdoc iframes are treated as opaque-origin. Wrapped in try/catch with "*" as a safe fallback (these messages are test-only, contain no sensitive data).
tests/unit/SpecHelper.js: the waitReady message handler used ev.source.document, which also throws SecurityError cross-origin in Safari, silently preventing the "end-all" message from being handled and leaving tests hanging. Replaced with ev.source !== iframe.contentWindow (safe cross-origin Window identity comparison). Also changed the timeout from throw inside setTimeout (which Safari converts to an unhandled rejection with undefined, crashing jasmine-core) to a proper reject() inside the Promise.

marcoscaceres · 2026-04-15T15:46:42Z

@copilot can you review this again and fix any issues you find and any resolve any unresolved comments?

Copilot

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (2)

pnpm-lock.yaml:129

The lockfile no longer lists lint-staged under the root importer’s devDependencies, but package.json still references npx lint-staged via simple-git-hooks and includes a lint-staged config. This will cause pnpm i --frozen-lockfile to omit lint-staged, breaking the configured pre-commit hook. Re-add lint-staged to the root devDependencies (and regenerate the lockfile), or remove the hook/config if lint-staged is intentionally being dropped.
package.json:56
lint-staged was removed from devDependencies, but the repo still configures simple-git-hooks.pre-commit to run npx lint-staged and includes a lint-staged config block later in this same file. With this change, fresh installs won’t have lint-staged available locally, and pre-commit will either fail or unexpectedly download from the network. Either keep lint-staged as a devDependency or remove/update the simple-git-hooks and lint-staged configuration to match.

    "karma-jasmine": "^5.1.0",
    "karma-jasmine-html-reporter": "^2.2.0",
    "karma-mocha-reporter": "^2.2.5",
    "loading-indicator": "^2.0.0",
    "pluralize": "^8.0.0",
    "prettier": "^3.8.2",
    "prompt": "^1.3.0",

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (1)

package.json:56

lint-staged was removed from devDependencies, but simple-git-hooks.pre-commit still runs npx lint-staged and the lint-staged config remains in this file. This will make pre-commit either fail (no local binary) or rely on network installs via npx. Re-add lint-staged to devDependencies (preferred) or update/remove the hook/config so it doesn't reference lint-staged.

    "karma": "^6.4.4",
    "karma-chrome-launcher": "^3.2.0",
    "karma-firefox-launcher": "^2.1.3",
    "karma-jasmine": "^5.1.0",
    "karma-jasmine-html-reporter": "^2.2.0",
    "karma-mocha-reporter": "^2.2.5",
    "loading-indicator": "^2.0.0",
    "pluralize": "^8.0.0",
    "prettier": "^3.8.2",
    "prompt": "^1.3.0",

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

marcoscaceres · 2026-04-28T01:11:39Z

@copilot Can you confirm all your feedback has been addressed? Can you fix anything that was not addressed?

Copilot · 2026-04-28T01:15:29Z

@copilot Can you confirm all your feedback has been addressed? Can you fix anything that was not addressed?

Done in 039e560. All feedback is now addressed:

ev.data guard (SpecHelper.js:142) — already uses ev.data?.topic (optional chaining), guards non-object payloads.
postMessage("*") security (pubsubhub.js) — the previous fix went too far: it returned early for all topics when window.parent.location.origin throws, which also dropped the end-all signal and caused all tests to time out. Now only end-all (which publishes detail = undefined) falls back to targetOrigin: "*"; all other topics still skip postMessage when the origin is unavailable. This keeps sensitive config from leaking while restoring test completion in Safari.
SAFARIDRIVER_PORT range validation — now uses an IIFE that validates the parsed integer is in 1–65535, throwing a clear error for invalid values (e.g. 0, -1, non-numeric strings).
PR description / CI mismatch — already updated to reflect Safari runs in CI.

Replaces the broken karma-safari-launcher (redirect.html hack, broken on modern Safari) with a minimal custom launcher that speaks directly to safaridriver over W3C WebDriver HTTP protocol. No new npm dependencies — uses Node's built-in http module. Setup (one-time): sudo safaridriver --enable Then use as normal: pnpm start --browser Safari Not added to CI (macOS runners cost ~10× more). Closes #2894 for local use.

- Add error/exit handlers on the safaridriver child process so missing or disabled safaridriver gives a clear error message - Reject webdriver() on HTTP 4xx/5xx responses instead of silently resolving with error content - Add 10s timeout to webdriver() requests to prevent hangs - Extract cleanup() helper and call it on both failure and kill paths to prevent orphaned safaridriver processes and session leaks Not implementing poll-for-readiness (comment 2) — the fixed 500ms wait is sufficient for a local dev tool and polling adds complexity.

If safaridriver emitted an 'error' event AND the subsequent webdriver() call then also failed (connection refused), _done('failure') was called twice — once in the error handler and once in the catch block. Fix: race the safaridriver error against the startup timeout using a Promise. If the process errors, reject immediately and return early, ensuring _done is called exactly once.

If safaridriver exits before the 500ms timeout without emitting 'error' (e.g. port already in use, not enabled), the Promise would resolve and the launcher would fail later with a confusing WebDriver connection error. Use 'once' on exit during the startup window to reject immediately with a clear message.

Safari returns '' for elem.innerText when the element hasn't been fully laid out, causing the highlight worker to receive empty code and set elem.innerHTML to '<code class="hljs"></code>', emptying the pre. textContent is layout-independent and always returns the raw text, making it the correct choice for extracting code to highlight. Also guard parseInfoString() against undefined infoString, which Safari's marked passes for code blocks without a language specifier.

- Allow SAFARIDRIVER_PORT env var override for karma Safari launcher - Notify Karma and clean up on unexpected safaridriver exit - Fix JSDoc type for parseInfoString to accept undefined

Agent-Logs-Url: https://github.com/speced/respec/sessions/8e1b50fe-6001-4593-9c5d-d20618791bcb Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

…I reliability Agent-Logs-Url: https://github.com/speced/respec/sessions/75aecc21-a5d1-4831-a107-5a4f42a262a9 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

Agent-Logs-Url: https://github.com/speced/respec/sessions/f825ab10-4ee2-427b-9176-637faabea744 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

…tReady Agent-Logs-Url: https://github.com/speced/respec/sessions/8f5dd784-fefd-442c-970b-ce6991b8fd8c Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

…down Agent-Logs-Url: https://github.com/speced/respec/sessions/4c275e23-6d86-4c63-9268-4b625420bb32 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

In Safari, postMessage from opaque-origin srcdoc iframes may not be received (ev.source !== iframe.contentWindow). Add a 100ms polling loop that checks doc.respec.ready directly, racing the existing postMessage listener. Also: - Fix ev.source check to accept null/undefined (loose !=) - Wrap window.parent.postMessage in pubsubhub.js try/catch so postMessage failures don't propagate through pub() and disrupt runAll() - Suppress Promise.reject(err) unhandled rejection in iframe template so Safari doesn't cross-frame propagate it as undefined to jasmine Agent-Logs-Url: https://github.com/speced/respec/sessions/e996459b-6f02-4d6b-9ade-d9003140e5a4 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

Agent-Logs-Url: https://github.com/speced/respec/sessions/65cc375e-3e6f-43b8-9cf6-3e663ae1544c Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

…ch errors Agent-Logs-Url: https://github.com/speced/respec/sessions/da22e939-2ed4-4370-9fc5-2450af0e5876 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

Instead of falling back to targetOrigin '*' (which leaks respecConfig data to any embedding parent), return early when parent.location.origin throws SecurityError. The polling fallback in SpecHelper.js handles test completion in Safari. Also guard ev.data access in msgHandler to prevent crashes from non-object message payloads.

Throw early if env var is set to a non-numeric value instead of silently passing NaN as the port argument to safaridriver.

…igin iframes; add port range validation Agent-Logs-Url: https://github.com/speced/respec/sessions/7457d32d-c547-4cd4-94fe-bab03a6343da Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

…e for heading (#5207) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

) Co-authored-by: Tim van der Lippe <Tim.Lippe@logius.nl> Co-authored-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

marcoscaceres requested a review from Copilot March 28, 2026 13:41

Copilot started reviewing on behalf of marcoscaceres March 28, 2026 13:41 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

Comment thread tests/karma.safari.cjs Outdated

Comment thread tests/karma.safari.cjs Outdated

Comment thread tests/karma.safari.cjs

Comment thread tests/karma.safari.cjs

marcoscaceres force-pushed the chore/safari-launcher branch from a2c04c4 to 4f300a5 Compare March 28, 2026 14:01

marcoscaceres requested a review from Copilot March 28, 2026 14:04

Copilot started reviewing on behalf of marcoscaceres March 28, 2026 14:05 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

Comment thread tests/karma.safari.cjs Outdated

Comment thread tests/karma.safari.cjs Outdated

marcoscaceres requested a review from Copilot March 30, 2026 04:09

Copilot started reviewing on behalf of marcoscaceres March 30, 2026 04:09 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread tests/karma.safari.cjs Outdated

marcoscaceres requested a review from Copilot April 12, 2026 15:10

Copilot started reviewing on behalf of marcoscaceres April 12, 2026 15:11 View session

Copilot AI reviewed Apr 12, 2026

View reviewed changes

Comment thread tests/karma.safari.cjs Outdated

Comment thread src/core/markdown.js

Copilot started work on behalf of marcoscaceres April 14, 2026 17:36 View session

Copilot finished work on behalf of marcoscaceres April 14, 2026 17:41

Copilot started work on behalf of marcoscaceres April 14, 2026 17:49 View session

Copilot finished work on behalf of marcoscaceres April 14, 2026 17:56

Copilot started work on behalf of marcoscaceres April 14, 2026 17:59 View session

Copilot finished work on behalf of marcoscaceres April 14, 2026 18:02

Copilot started work on behalf of marcoscaceres April 14, 2026 18:25 View session

Copilot finished work on behalf of marcoscaceres April 14, 2026 19:17

Copilot AI reviewed Apr 26, 2026

View reviewed changes

Comment thread tests/karma.safari.cjs Outdated

marcoscaceres requested a review from Copilot April 27, 2026 00:18

Copilot started reviewing on behalf of marcoscaceres April 27, 2026 00:19 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Copilot started work on behalf of marcoscaceres April 28, 2026 01:11 View session

Copilot stopped work on behalf of marcoscaceres due to an error April 28, 2026 01:16
Failed to get response from the AI model; retried 5 times (total retry wait time: 5.97 seconds) Last error: Unknown error

marcoscaceres and others added 20 commits April 28, 2026 12:32

fix: prettier formatting in karma.safari.cjs

0cbef16

fix: address Copilot review feedback

8e66e08

- Allow SAFARIDRIVER_PORT env var override for karma Safari launcher - Notify Karma and clean up on unexpected safaridriver exit - Fix JSDoc type for parseInfoString to accept undefined

feat: add Safari to CI (macos-latest) in pr.yml and push.yml

bdb1611

Agent-Logs-Url: https://github.com/speced/respec/sessions/8e1b50fe-6001-4593-9c5d-d20618791bcb Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix: replace fixed 500ms wait with safaridriver /status polling for C…

6c23c5f

…I reliability Agent-Logs-Url: https://github.com/speced/respec/sessions/75aecc21-a5d1-4831-a107-5a4f42a262a9 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix: prettier formatting in karma.safari.cjs

c9e206e

Agent-Logs-Url: https://github.com/speced/respec/sessions/f825ab10-4ee2-427b-9176-637faabea744 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix: handle Safari srcdoc iframe SecurityError in postMessage and wai…

bcfd8fb

…tReady Agent-Logs-Url: https://github.com/speced/respec/sessions/8f5dd784-fefd-442c-970b-ce6991b8fd8c Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix: use cleanup() in unexpected-exit handler for proper session tear…

93dd7cc

…down Agent-Logs-Url: https://github.com/speced/respec/sessions/4c275e23-6d86-4c63-9268-4b625420bb32 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

chore: apply prettier formatting to CI workflow files

a18c0ec

Agent-Logs-Url: https://github.com/speced/respec/sessions/65cc375e-3e6f-43b8-9cf6-3e663ae1544c Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix(test): remove ev.source check in Safari msgHandler; normalise cat…

bad2cc6

…ch errors Agent-Logs-Url: https://github.com/speced/respec/sessions/da22e939-2ed4-4370-9fc5-2450af0e5876 Co-authored-by: marcoscaceres <870154+marcoscaceres@users.noreply.github.com>

fix(tests): validate SAFARIDRIVER_PORT is numeric

e52d41a

Throw early if env var is set to a non-numeric value instead of silently passing NaN as the port argument to safaridriver.

fix(core/issues-notes): validate GitHub label color, use html templat…

72fc870

…e for heading (#5207) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

fix: make target background of references accessible in dark mode (#5204

d5a21e6

) Co-authored-by: Tim van der Lippe <Tim.Lippe@logius.nl> Co-authored-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

marcoscaceres force-pushed the chore/safari-launcher branch from f6388e8 to d5a21e6 Compare April 28, 2026 02:37

Merge branch 'main' into chore/safari-launcher

7c7409a

Uh oh!

Conversation

marcoscaceres commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Setup (one-time per machine)

CI

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

marcoscaceres commented Apr 14, 2026

Uh oh!

Copilot AI commented Apr 14, 2026

Uh oh!

marcoscaceres commented Apr 14, 2026

Uh oh!

Copilot AI commented Apr 14, 2026

Uh oh!

marcoscaceres commented Apr 14, 2026

Uh oh!

Copilot AI commented Apr 14, 2026

Uh oh!

marcoscaceres commented Apr 14, 2026

Uh oh!

Copilot AI commented Apr 14, 2026

Uh oh!

marcoscaceres commented Apr 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

marcoscaceres commented Apr 28, 2026

Uh oh!

Copilot AI commented Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

marcoscaceres commented Mar 27, 2026 •

edited

Loading