-
Notifications
You must be signed in to change notification settings - Fork 74
docs(adr): accept ADR 0071 harness snapshot for agent traces #5524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
HofniGartner
wants to merge
5
commits into
fullsend-ai:main
Choose a base branch
from
HofniGartner:feat/adr-0071-harness-snapshot
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+107
−8
Open
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
ed6b146
docs(adr): propose ADR 0071 harness snapshot for agent traces
HofniGartner 0702f6a
docs(adr): address ADR 0071 review feedback
HofniGartner 6a6ec9b
docs(adr): clarify ADR 0071 disabled-mode traces and architecture sync
HofniGartner 120e756
docs(architecture): record ADR 0071 harness snapshot decision
HofniGartner fcdf055
docs(architecture): mark ADR 0071 snapshot as pending implementation
HofniGartner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
93 changes: 93 additions & 0 deletions
93
docs/ADRs/0071-forge-pointer-correlation-for-agent-traces.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| --- | ||
| title: "71. Harness snapshot and forge pointer correlation for agent traces" | ||
| status: Accepted | ||
| relates_to: | ||
| - operational-observability | ||
| topics: | ||
| - observability | ||
| - telemetry | ||
| - opentelemetry | ||
| - forge | ||
| --- | ||
|
|
||
| # 71. Harness snapshot and forge pointer correlation for agent traces | ||
|
|
||
| Date: 2026-07-22 | ||
|
|
||
| ## Status | ||
|
|
||
| Accepted | ||
|
|
||
| ## Context | ||
|
|
||
| [ADR 0050](0050-distributed-tracing-instrumentation.md) defines how fullsend | ||
| generates traces (OTel Go SDK; Level 1 = `run-telemetry.jsonl`; Level 2 = | ||
| OTLP). [Operational observability](../problems/operational-observability.md) | ||
| requires forge **pointers** (repo + SHA, change id, pipeline run) rather than | ||
| duplicating diffs, and a stable answer to: “what harness/config produced this | ||
| run?” Forge-neutral hosting concepts remain those in | ||
| [ADR 0005](0005-forge-abstraction-layer.md); this ADR only decides how join | ||
| pointers are recorded on a run. | ||
|
|
||
| After the OTel SDK migration, `run-summary.json` was removed; the sole Level 1 | ||
| telemetry file today is `run-telemetry.jsonl`. Eval wrappers that `export` | ||
| ambient CI variables are the wrong layer: harness identity is a **run-time** | ||
| concern of `fullsend run`, and downstream tools must not scrape CI env as the | ||
| source of truth. Cross-project join field names live in a shared join contract; | ||
| this ADR decides only what **fullsend** writes. | ||
|
|
||
| ## Options | ||
|
|
||
| **A. Root-span attributes only (no snapshot file).** Join keys exist in OTLP | ||
| backends, but offline/forensic consumers must parse spans, and keys disappear | ||
| when span export is disabled. | ||
|
|
||
| **B. Embed a snapshot event in `run-telemetry.jsonl`.** One Level 1 file, but | ||
| couples the harness/config contract to telemetry format and exporter semantics. | ||
|
|
||
| **C. Eval-script / ambient CI env exports.** Easy to prototype; unstable across | ||
| local runs, child dispatches, and forges; encourages scraping rather than an | ||
| explicit artifact. | ||
|
|
||
| **D. Dedicated `harness-snapshot.json` plus mirrored root-span attributes | ||
| (chosen).** Stable local run-start contract for offline consumers, plus | ||
| backend-friendly correlation for OTLP. | ||
|
|
||
| ## Decision | ||
|
|
||
| Every `fullsend run` records run-start join keys by writing | ||
| **`harness-snapshot.json`** next to `run-telemetry.jsonl` **and**, when tracing | ||
| is enabled, setting the same keys on the root `run` span (`vcs.*`, `cicd.*`, | ||
| harness content SHA, forge platform). The file holds pointers and a config | ||
| fingerprint only (harness identity, content hash, forge/CI pointers when known) | ||
| — no diffs, prompts, or skill bodies. Forge/CI fields are filled at write time | ||
| (`FULLSEND_*` overrides first, then standard CI env). Consumers read the JSON | ||
| (or stores that ingest it), not ambient CI env. | ||
|
|
||
| When tracing is enabled, write the snapshot after the root span exists and | ||
| include `trace_id` / `traceparent` so the file joins to `run-telemetry.jsonl` | ||
| and OTLP. When `OTEL_SDK_DISABLED=true`, still write the snapshot (config/forge | ||
| contract, not span export) but **omit** `trace_id` / `traceparent` and skip | ||
| root-span attribute mirroring — there is no usable trace context. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - Operators and downstream loggers can join a run to forge/CI via a single | ||
| decided artifact; when tracing is on, `trace_id` joins to `run-telemetry.jsonl`. | ||
| - OTLP backends receive join keys without parsing the JSON when tracing is on; | ||
| local JSON remains the forensic / handoff contract. | ||
| - Dispatched child runs must inherit forge context via env so child snapshots | ||
| are complete. | ||
| - GitLab/Bitbucket coverage depends on CI vars or `FULLSEND_*` overrides. | ||
| - Implementation is tracked separately ([#5449](https://github.com/fullsend-ai/fullsend/issues/5449)); | ||
| detailed field lists for cross-project consumers stay in the shared join | ||
| contract, not duplicated here. | ||
|
|
||
| ## Related | ||
|
|
||
| - [ADR 0050](0050-distributed-tracing-instrumentation.md) | ||
| - [ADR 0005](0005-forge-abstraction-layer.md) | ||
| - Shared join contract: [provenance_forge_pointers.md](https://github.com/RHEcosystemAppEng/ABEvalFlow/blob/main/Docs/provenance_forge_pointers.md) | ||
| - Implementation: [#5449](https://github.com/fullsend-ai/fullsend/issues/5449) | ||
| - [#2368](https://github.com/fullsend-ai/fullsend/issues/2368) | ||
| - [#294](https://github.com/fullsend-ai/fullsend/issues/294) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.