Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Clarified immutability guidance for emitted CTRF report artifacts ([#55](https://github.com/ctrf-io/ctrf/pull/55)).
- Clarified `tags` as simple keyless classifications and `labels` as structured key-value test metadata ([#54](https://github.com/ctrf-io/ctrf/pull/54)).
- Allowed non-empty arrays of strings, numbers, and booleans as label values ([#54](https://github.com/ctrf-io/ctrf/pull/54)).
26 changes: 26 additions & 0 deletions spec/ctrf.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,26 @@ As long as the document structure and semantics are respected, CTRF does not con

---

### 2.11. Immutable Report Artifacts

CTRF treats an emitted report as an immutable artifact.

A CTRF document represents a snapshot of test results at the time it is produced. Implementations are free to build or transform documents internally before emission, but once a CTRF document has been written, published, or otherwise made available as an artifact, it is treated as a fixed record.

Immutability supports:

- auditability of test results
- reproducibility of analysis
- stable caching and artifact storage
- compatibility with hashing, signing, and deduplication
- clear separation between raw results and derived results
Comment thread
Ma11hewThomas marked this conversation as resolved.

When a tool or system performs additional processing after initial report generation — such as merging shard reports, adding metadata, or computing insights — the design intent is that this produces a new CTRF document and, when `reportId` is used, assigns a different `reportId` value, rather than modifying the original.

This distinction between the act of constructing a report and the resulting artifact as an immutable record is central to how CTRF is designed to be used in pipelines, artifact stores, and analysis systems.

---

## 3. Terminology

The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
Expand Down Expand Up @@ -405,6 +425,8 @@ A unique identifier for this report instance.
`reportId` is OPTIONAL.
If present, it MUST be a valid UUID as defined in [RFC4122].

Each emitted CTRF document represents a distinct report instance. If a new CTRF document is produced through post-processing, it SHOULD use a different `reportId` value than the source document when `reportId` is used.

---

### 5.4. `timestamp`
Expand Down Expand Up @@ -1764,6 +1786,8 @@ Producers:
- MUST NOT introduce fields outside `extra` objects
- MUST NOT emit invalid enum values (status, etc.)
- MAY include `insights` if historical or aggregate data is available
- SHOULD treat emitted CTRF documents as immutable artifacts
- SHOULD emit a new CTRF document when performing post-processing after initial report generation and, when `reportId` is used, assign a different `reportId` value rather than modifying an emitted CTRF document

Producers SHOULD:

Expand All @@ -1784,6 +1808,8 @@ Consumers:
- MUST NOT require optional fields to be present
- MUST NOT assume the presence of `insights`
- SHOULD compute insights when sufficient historical data is available
- SHOULD assume that CTRF documents are immutable once emitted
- MUST NOT rely on in-place modification of existing reports

---

Expand Down
Loading