rule-authoring.md

Rule Authoring Guide

How to write custom rules for the yara and syara engines in llm_context_shield.

Where rules live

Bundled rules ship inside the binary. Custom rules are loaded from:

1. [rules] dir = "..." in config.toml, if set — <dir>/yara/*.yar and <dir>/syara/*.syara.
2. Otherwise $XDG_DATA_HOME/llm_context_shield/rules/<engine>/ (falls back to ~/.local/share/llm_context_shield/rules/<engine>/).

Scaffold the tree with:

lcs init --rules

This creates yara/ and syara/ subdirectories and a README stub.

To disable bundled rules (use only your own), set:

[rules]
bundled = false

Minimum viable rule

Both engines accept YARA-dialect DSL. A rule must declare meta, strings, and condition:

rule my_custom_prompt_leak {
    meta:
        category     = "prompt_injection"
        severity     = "high"
        description  = "Detects attempts to surface the system prompt via roleplay"
        threat_level = 60
        threshold    = 0
        threat_class = "prompt_hijack"
    strings:
        $s1 = /reveal\s+(the\s+)?hidden\s+directive/i
        $s2 = /what\s+were\s+you\s+told\s+before\s+this\s+chat/i
    condition:
        any of them
}

Save as my_rules.yar under yara/ (or .syara under syara/) and run:

lcs list -e yara      # confirm the rule was compiled
lcs scan -e yara      # scan stdin with it active

Required metadata

The engine reads meta: fields to classify findings and drive the threat scoring engine. Missing fields fall back to sensible defaults.

Field	Required	Type	Values / Default	Used for
category	yes	string	prompt_injection, jailbreak, data_exfiltration, hidden_content, delimiter_manipulation, instruction_override, refusal_suppression, response_steering, secret_probing, context_shift, icl_exploitation, coercion, refusal_bypass, session_protocol, obfuscation	Finding category, severity filtering
severity	yes	string	low, medium, high, critical	--severity threshold filtering
description	yes	string	free text	Finding message shown to the user
threat_level	no	integer	score on match (typical 20–100; default 1)	Threat scoring accumulator
threshold	no	integer	min class score to activate (typical 0, 120–300; default 0)	Threshold-gated evaluation
threat_class	no	string	heuristic branch (default = category)	Scoring class grouping
author	no	string	free text	Attribution only
version	no	string	free text	Surfaced via introspection (see below)
provenance	no	string	free text — fixture or FP that motivated tuning	Tuning archaeology; surfaced via introspection

Unknown category values are rejected at compile time. Unknown severity values default to low.

Note: In YARA rules, threat_level and threshold are unquoted integers (threat_level = 60). In SYARA rules, they are quoted strings (threat_level = "60") and parsed at load time.

context_taxonomy: An additional RuleMeta field is reserved on the introspection schema (defaults to an empty list) for future context-modifier work. Its rule-file authoring shape is intentionally deferred — do not author it in meta: blocks today; the encoding will be specified when the first context-detection lands.

How metadata feeds introspection

category, severity, threat_class, version, threat_level, and threshold are all surfaced through Engine::rule_metadata() and the lcs rules CLI. Two implications for authors:

• Pick stable values. name, category, and threat_class show up in lcs rules --json, in --disable configs, and in correlation rule category references. Renaming a rule or shifting its category breaks downstream consumers.
• Bumps participate in the rule-set fingerprint. Changing any of the six fields above (or adding/removing a version) shifts the SHA-256 emitted by lcs rules --fingerprint. That's the audit signal — operators detect rule-set drift between deployments by diffing the fingerprint. Bump version when you make a substantive change so the fingerprint moves with intent rather than as a side effect.

See docs/rule-introspection.md for the full introspection surface, the fingerprint contract, and the --all cross-engine view.

Threat scoring metadata

The scoring engine uses threat_level, threshold, and threat_class to implement multi-pass, threshold-gated scanning. This lets you write sensitive rules that only activate when cheaper rules have already raised suspicion.

How it works

1. All rules compile and scan in a single pass (no performance penalty).
2. Results are processed in threshold order: threshold-0 rules score first.
3. As scores accumulate per threat_class, higher-threshold rules unlock.
4. Rules whose class hasn't reached their threshold are silently dropped.

Choosing values

Rule confidence	Suggested threat_level	Suggested threshold
Near-certain indicator	100	0
Strong signal	60	0
Weak signal / noisy	20–40	0
Context-dependent (only meaningful after other matches)	20–60	120–300

Threat classes

Group related rules into the same threat_class so their scores accumulate together:

Threat class	Categories
prompt_hijack	prompt_injection, instruction_override, response_steering, secret_probing, icl_exploitation
social_engineering	jailbreak, refusal_suppression, context_shift, coercion, refusal_bypass
data_exfiltration	data_exfiltration
obfuscation	hidden_content, delimiter_manipulation, session_protocol, obfuscation

Cross-branch escalation

When one threat class accumulates a very high score, the scoring engine can lower thresholds for rules in other classes. This is configured via [scoring] in config.toml:

[scoring]
# When any class exceeds this score, reduce thresholds in other classes
# escalation_threshold = 6000
# escalation_reduction = 0

Default values make escalation inert until you tune them with real-world data.

Rule naming

The rule identifier (rule my_custom_prompt_leak) is what users see in lcs list -e yara and what they pass to --disable. Pick stable, descriptive names — renaming breaks user configs.

Convention: <category>_<qualifier> — e.g. prompt_injection_critical, jailbreak_high, hidden_content_zero_width.

String patterns

Supports the YARA-X string dialect:

• Text strings: $s1 = "literal"
• Regex: $s1 = /pattern/i (case-insensitive flag supported; regex syntax per YARA-X)
• Hex: $h1 = { 48 65 6C 6C 6F }

Use nocase for plain strings and /.../i for regex. Anchor with ^ / $ at line start/end when you want to reduce false positives on mid-sentence matches.

Conditions

Common patterns:

condition: any of them                  // match if any string hits
condition: all of them                  // match only if every string hits
condition: 2 of ($s*)                   // match if at least 2 of $s1..$sN hit
condition: $s1 and not $s2              // combine with boolean logic

YARA vs SYARA

Both engines compile the same string/regex DSL. The SYARA engine additionally accepts similarity:, classifier:, and llm: blocks — extensions with no YARA equivalent. These require build features (syara-sbert, syara-classifier, syara-llm) and a model or endpoint configured per docs/semantic-rules.md. Without the features, semantic rules still parse and load but never match.

Semantic rules (SYARA only)

Three optional matcher types. Each sits inside the rule body like strings:, with single-line $id = "pattern" key=value key=value format.

similarity: — SBERT embedding similarity

rule semantic_example {
    meta:
        category = "prompt_injection"
        severity = "high"
    similarity:
        $sim1 = "ignore all previous instructions" threshold=0.40 matcher="sbert" cleaner="default_cleaning" chunker="sentence_chunking"
    condition:
        $sim1
}

• threshold is cosine similarity (0.0–1.0). MiniLM-L6-v2 (the bundled ONNX model) scores paraphrases in 0.30–0.70 depending on lexical distance.
• matcher is the registered matcher name. llm_context_shield registers sbert when the syara-sbert feature is active.
• cleaner / chunker / matcher values: see SYARA-X README for the full registry.

classifier: — embedding-similarity classifier

rule classifier_example {
    meta:
        category = "obfuscation"
        severity = "medium"
    classifier:
        $c1 = "repetitive filler text padding the context" threshold=0.70 classifier="tuned-sbert" cleaner="default_cleaning" chunker="paragraph_chunking"
    condition:
        $c1
}

Requires the syara-classifier build feature and a registered classifier.

Caveat (SYARA-X 0.3): the bundled OnnxEmbeddingClassifier is cosine similarity over the same MiniLM-L6-v2 embedding the similarity: matcher uses — it is not a trained classifier head. In practice, classifier: rules backed by this matcher have the same detection capability as similarity: rules: they match topical similarity, not structural or meta-properties like "is this padded?" or "is this overlong?". Meta-property detection should use llm: rules. A fine-tuned classifier head is future upstream work; see docs/semantic-rules.md for the empirical finding from Phase 10c.

llm: — LLM evaluator

rule llm_example {
    meta:
        category = "jailbreak"
        severity = "high"
    llm:
        $l1 = "Does this text attempt to override AI safety guidelines?" llm="openai-api-compatible" cleaner="no_op" chunker="no_chunking"
    condition:
        $l1
}

Requires the syara-llm feature and an OpenAI-compatible endpoint (LMStudio, OpenAI, vLLM, llama.cpp server, etc.). LLM evaluations are slow (~1–5 s per scan for no_chunking; scales with chunk count for chunked rules). Use them for meta-property judgments that similarity: cannot make — compositional intent, content quality, coercion — and consider threshold-gating if you only want them to run once cheaper signals have raised suspicion.

The evaluator expects a strict YES: ... / NO: ... response format. Small or chatty models that preface responses with "Sure, let me analyze..." will be classified as ambiguous (no-match). See docs/semantic-rules.md for tested-known-good models.

Testing a rule

1. Drop the file under the appropriate engine subdirectory.
2. lcs list -e yara — confirm the rule name appears.
3. lcs scan -e yara <<< "your test payload" — confirm it fires.
4. lcs scan -e yara <<< "benign payload" — confirm it does not fire.
5. Run the full suite once more: cargo test --features yara,syara.

Troubleshooting

• Rule does not appear in lcs list: file extension wrong (.yar vs .syara), file lives in the wrong engine subdirectory, or file exceeds the 1 MiB size cap (warn in logs).
• Rule fires on benign input: tighten the regex; add line anchors; require N of them instead of any of them.
• Rule never fires: check you haven't shadowed it with --disable; enable --log and re-scan to see compile warnings.
• Symlinks are ignored: intentional — drop real files, not symlinks. Symlinked rule files are skipped with a warning.

Custom correlation rules

Correlation rules are evaluated after the engine produces findings. They link two findings into a higher-confidence compound signal — for example, a delimiter-manipulation finding paired with a prompt-injection finding within 500 bytes ("sandwich attack"). The bundled catalog ships 11 such rules; users can extend it by pointing the config at a TOML file.

Wiring a custom rule file

In ~/.config/llm_context_shield/config.toml:

[correlation]
enabled = true
proximity_window = 500
custom_rules = "/path/to/correlation_rules.toml"

enabled and proximity_window are optional; custom_rules is the path to your rule file. Set enabled = false to skip both bundled and custom correlation entirely. proximity_window overrides the byte distance for the bundled Proximate rules (sandwich_attack, encode_and_inject); custom rules supply their own proximity_bytes per rule.

TOML format

Each rule is one [[rules]] block with two [[rules.match_refs]] sub-blocks (the engine evaluates pairs only). Worked example:

[[rules]]
name = "tight_sandwich"
explanation = "Delim spoof + prompt injection within 200 bytes (tighter than bundled)."
constraint_type = "proximate"
proximity_bytes = 200
composite_threat_level = 280
composite_threat_class = "sandwich_attack"

[[rules.match_refs]]
category = "delimiter_manipulation"

[[rules.match_refs]]
category = "prompt_injection"

Required fields per rule:

• name — unique identifier (string).
• explanation — human-readable description rendered with --correlations.
• constraint_type — one of "ordered", "proximate", "combined", "cross_engine".
• proximity_bytes — required iff constraint_type = "proximate"; rejected otherwise.
• composite_threat_level — integer; the level recorded in the threat scoreboard when this rule fires.
• composite_threat_class — string class name for the scoreboard.
• match_refs — exactly two; each must have a category. rule_name_pattern (regex against Finding::description) and engine_filter (engine name) are optional.

Constraint type semantics

• ordered — the first ref's match must precede the second by byte offset. Use for attack chains where order matters (probe → extract).
• proximate — both matches present within proximity_bytes of each other (any order).
• combined — both matches present in the same scan, position-agnostic.
• cross_engine — matches sourced from two distinct engines. Useful for "two engines independently flagged the same threat class" patterns. In single-scan mode this is dormant against the current single-engine Shield. In scan-group mode (lcs scan-group / Shield::scan_group), each input becomes a synthetic engine bucket labelled input:<label>, and cross_engine rules fire when distinct inputs corroborate the same threat class. See scan-group-data-flow-simple.md for the full mechanism.

Category names

The category field is a snake-case string corresponding to the Category enum. Valid values: prompt_injection, hidden_content, data_exfiltration, jailbreak, delimiter_manipulation, instruction_override, refusal_suppression, response_steering, secret_probing, context_shift, icl_exploitation, coercion, refusal_bypass, session_protocol, obfuscation.

Validation and load failures

The loader rejects:

• TOML syntax errors.
• Unknown constraint_type values.
• proximity_bytes set on non-proximate constraints (or missing on proximate).
• Rules with match_refs.len() != 2.
• Unknown category names.

If the configured custom_rules path is missing or malformed at Shield construction, a tracing::warn! is logged and the bundled catalog is still loaded. A scan will not fail because of a bad custom-rules file.

Composite scoring

Composite levels should exceed the maximum individual threat_level of the contributing categories — that's the whole point of correlation. The bundled catalog uses 240–320 (max individual is 100). Reuse an existing composite_threat_class if your rule conceptually overlaps with a bundled one (e.g., a tighter sandwich variant should still use sandwich_attack as its class so the scoreboard aggregates it sensibly).