Skip to content

fix: Prevent context attributes from influencing judge template parsing#1258

Open
jsonbailey wants to merge 6 commits intomainfrom
devin/1775752763-fix-judge-template-injection
Open

fix: Prevent context attributes from influencing judge template parsing#1258
jsonbailey wants to merge 6 commits intomainfrom
devin/1775752763-fix-judge-template-injection

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Apr 9, 2026

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Addresses SEC-8020. Mirrors the fix applied in the Go server SDK (go-server-sdk@3317871). A matching Python SDK PR has also been opened in launchdarkly/python-server-sdk-ai.

Describe the solution you've provided

The Judge's _interpolateMessage previously used Mustache.render() to substitute {{message_history}} and {{response_to_evaluate}} placeholders (pass 2). Because pass 1 (LDAIClientImpl) already resolves user/context attributes via Mustache, an attacker who controls a context attribute could inject Mustache control sequences (e.g. {{=[ ]=}} delimiter-change tags) that corrupt pass 2, effectively blinding the judge to the actual conversation content.

This PR replaces the Mustache call in _interpolateMessage with a literal Object.entries().reduce() using split().join() for string replacement. Since the judge only ever substitutes two known placeholders, a full template engine is unnecessary. The mustache dependency remains because LDAIClientImpl still uses it for general config interpolation (pass 1).

Key changes:

  • Judge.ts: Remove Mustache import; replace Mustache.render() with reduce + split().join() over the known placeholder keys.
  • Judge.test.ts: Add template injection prevention describe block with regression tests covering delimiter changes, partials, comments, triple-stache, sections, inverted sections, injection via response, multiple placeholder occurrences, and preservation of Mustache-like syntax inside substituted values.

Describe alternatives you've considered

  • Using String.prototype.replaceAll() — not available with the es2020 target; split().join() is the standard workaround.
  • Using a for...of loop — disallowed by the repo's ESLint no-restricted-syntax rule; reduce is the idiomatic alternative.
  • Escaping Mustache special characters in the input before rendering — more complex and fragile than removing the template engine entirely for this use case.

Additional context

Reviewers should verify:

  • The split().join() pattern correctly handles all occurrences of each placeholder (it does — split with a string literal is equivalent to a global literal replace).
  • The mustache package remains in dependencies since LDAIClientImpl.ts still imports it for pass 1.
  • Tests access _constructEvaluationMessages via (judge as any) cast — this is the simplest way to exercise the interpolation logic end-to-end without mocking the full provider invocation.

Link to Devin session: https://app.devin.ai/sessions/651e799b906748a4834bafefb4a3e3e5
Requested by: @jsonbailey


Note

Medium Risk
Touches judge prompt construction logic to mitigate a template-injection vulnerability; while localized, it changes how placeholders are detected/replaced and could affect existing judge message templates that relied on Mustache behavior.

Overview
Hardens Judge prompt construction against template injection. Judge._interpolateMessage no longer uses Mustache.render; it now performs a literal {{key}} string replacement for known variables so attacker-controlled strings like {{=[ ]=}} cannot alter parsing in the second interpolation pass.

Adds a new template injection prevention test suite covering delimiter changes, sections/partials/comments, injection via response/history, multiple placeholder occurrences, and ensuring Mustache-like syntax in substituted values is preserved as literal text.

Reviewed by Cursor Bugbot for commit f24a371. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration bot added the devin-pr PRs created by Devin AI label Apr 9, 2026
@devin-ai-integration
Copy link
Copy Markdown
Contributor

@cursor review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 25623 bytes
Compressed size limit: 29000
Uncompressed size: 125843 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 31649 bytes
Compressed size limit: 34000
Uncompressed size: 112792 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 179372 bytes
Compressed size limit: 200000
Uncompressed size: 829982 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 37169 bytes
Compressed size limit: 38000
Uncompressed size: 204305 bytes

…t rule

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
Comment thread packages/sdk/server-ai/src/api/judge/Judge.ts Outdated
Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
@jsonbailey jsonbailey marked this pull request as ready for review April 13, 2026 21:48
@jsonbailey jsonbailey requested a review from a team as a code owner April 13, 2026 21:48
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fcc72ae. Configure here.

Comment thread packages/sdk/server-ai/src/api/judge/Judge.ts
devin-ai-integration bot and others added 2 commits April 13, 2026 21:53
… injection

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
*/
private _interpolateMessage(content: string, variables: Record<string, string>): string {
return Mustache.render(content, variables, undefined, { escape: (item: any) => item });
return content.replace(/\{\{(\w+)\}\}/g, (match, key) =>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to make sure that we are okay with this matching since \w+ will only match alphanumerics and _ (?). So whitespace and some common special characters will not pass.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct — \w+ matches [a-zA-Z0-9_]. The only two placeholders used in pass 2 are message_history and response_to_evaluate, which are purely alphanumeric + underscores. Anything that doesn't match a key in the variables map is left as-is (the callback returns match for unknown keys), so non-matching patterns like {{> partial}} or {{=[ ]=}} pass through untouched.

Comment thread packages/sdk/server-ai/src/api/judge/Judge.ts Outdated
Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr PRs created by Devin AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants