Skip to content

fix(rag): include heading_path in the prompt context blocks - #175

Merged
n0nuser merged 1 commit into
mainfrom
fix/prompt-includes-heading-path
Aug 9, 2026
Merged

fix(rag): include heading_path in the prompt context blocks#175
n0nuser merged 1 commit into
mainfrom
fix/prompt-includes-heading-path

Conversation

@n0nuser

@n0nuser n0nuser commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Closes #172

Problem

build_prompt read only source and chunk_index off each context and never touched context["metadata"], so heading_path reached the API sources list (engine.py:214) but never reached the model.

The structural chunker computes heading_path precisely so a chunk carries its position in the document hierarchy. Discarding it at the prompt boundary threw away the one signal distinguishing a passage about long-term epidemiological risk from one about immediate effects — which is how a query about a single short night got answered with chronic-exposure statistics.

The metadata was already there: the compressor copies the whole dict (compressor.py:105, popping only expanded_text), so this is a missing read, not missing plumbing.

Change

Context block headers gain a section= segment when heading_path is populated:

[1] source=book.md chunk=473 section=Cancer, Heart Attacks > SLEEP LOSS AND THE CARDIOVASCULAR SYSTEM

The segment is omitted entirely when heading_path is empty, missing, or metadata is absent, so text_block/code_block chunks and the ollama.py call site (which passes no metadata) keep the exact previous header.

On the truncation bound

The issue flagged token cost as worth checking. It was: compression budgets measure only chunk body text and reserve nothing for block scaffolding, so an unbounded heading path grows the prompt past what any budget counted.

reserved_prompt_tokens (512) is the reservation covering everything outside chunk bodies. Measured worst case — max_contexts headings of single-character words, one whitespace token per two characters — comes to 518 tokens at a 200-char bound and 262 at 120. MAX_SECTION_CHARS is therefore 120, and a test pins the invariant rather than leaving it as a comment that drifts.

Verification

  • Unit suite: 486 passed, 0 failed (6 new tests).
  • Lint, format, mypy: clean.
  • Integration, against a rebuilt image: 13 passed.
  • End-to-end against real Chroma (4,215 ingested chunks): section= present on all 4,043 chunks carrying heading_path, absent on the 172 without.

Notes for the reviewer

Two pre-existing issues surfaced during verification, both out of scope here:

  • Taskfile.yml:127LOCALRAG_BUILD_SHA="$$(git rev-parse HEAD)" does not expand outside Task's own escaping, so /build-info reports the literal string 99848(git rev-parse HEAD) and task docker-check can never pass.
  • The unit suite reads a developer's local .env, so 26 tests fail on a machine that has one. Moving .env aside takes the suite from 26 failed to 0.

https://claude.ai/code/session_01PhFVTFLaxhmP8icwzmcm5Q

build_prompt read only source and chunk_index off each context and never
touched context["metadata"], so heading_path reached the API sources list
but never the model. The structural chunker computes heading_path precisely
so a chunk carries its position in the document hierarchy; discarding it at
the prompt boundary threw away the signal that distinguishes a passage about
long-term risk from one about immediate effects.

Each context block header now carries a section= segment when heading_path
is populated, and is unchanged when it is not, so text_block and code_block
chunks keep the previous format.

Heading paths are bounded by MAX_SECTION_CHARS. The bound is derived rather
than picked: compression budgets measure only chunk body text and reserve
nothing for the block scaffolding, so worst-case headers have to fit inside
reserved_prompt_tokens. A test pins that invariant, since a looser bound
silently pushes the prompt past what any budget counted.

Refs #172

Claude-Session: https://claude.ai/code/session_01PhFVTFLaxhmP8icwzmcm5Q
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RAG prompt drops heading_path, so the model cannot see where a chunk came from

1 participant