introducing: log.type

[hilmarf]

Fixes #3429

Changes

Audit log recording security-relevant activities for compliance and forensics. When log.type='audit', the log record MUST not get lost and body not changed.

Important

Pull requests acceptance are subject to the triage process as described in Issue and PR Triage Management.
PRs that do not follow the guidance above, may be automatically rejected and closed.

Merge requirement checklist

• CONTRIBUTING.md guidelines followed.
• Change log entry added, according to the guidelines in When to add a changelog entry.
  • If your PR does not need a change log, start the PR title with [chore]
• Links to the prototypes or existing instrumentations (when adding or changing conventions)

[trask]

OpenTelemetry hasn't expanded into audit logging yet, may be blocked on open-telemetry/community#2409

[hilmarf]

OpenTelemetry hasn't expanded into audit logging yet

True — but if we don’t start taking small steps soon, it will most likely never become possible to support such scenarios, which would be a pity.

[jsuereth]

Do you have a prototype of this in practice?

I'm nervous that since this is so low in the hierarchy, otel will naturally batch data by resource/scope and that means audit logs and regular logs are all mixed together.

Should we try to make sure we batch/send this data in a separate channel or some how do not automatically bundle app + audit logs in the same channel?

[hilmarf]

Do you have a prototype of this in practice?

Not yet. If you could point me to an example of what you would expect, I can try to create a prototype accordingly.

I'm nervous that since this is so low in the hierarchy, otel will naturally batch data by resource/scope and that means audit logs and regular logs are all mixed together.

Yes, unfortunately that’s correct.

Should we try to make sure we batch/send this data in a separate channel or some how do not automatically bundle app + audit logs in the same channel?

That would actually be great! My impression so far, however, is that introducing a new channel might be considered a rather large request and likely won't get accepted by the OTel code owners. Their last responses were along the lines of: "We have no time/interest to work on audit logging and are currently focusing solely on stabilization."

Given that audit logging is not currently on the roadmap, I'm investigating other mechanisms to clearly differentiate between non-essential and security-relevant log messages.

[kamphaus]

the log record MUST not get lost and body not changed

How would we enforce this? Should this be a spec change?
If you wanted to enforce this, it would have many consequences for the otel-collector and observability backends.
In my opinion the wording should use SHOULD. The audit data would be sent via the same channels as regular data and be a hint for later stages in the observability pipeline. You could then add routing / conditional processing to direct these logs to a separate destination.

[hilmarf]

the log record MUST not get lost and body not changed

How would we enforce this? Should this be a spec change? If you wanted to enforce this, it would have many consequences for the otel-collector and observability backends. In my opinion the wording should use SHOULD. The audit data would be sent via the same channels as regular data and be a hint for later stages in the observability pipeline. You could then add routing / conditional processing to direct these logs to a separate destination.

I've updated the wording as you suggested.

Nevertheless, the intention is indeed to implement some of the consequences you have in mind within the otel-collector itself — potentially as a dedicated audit-processor or a specialized exporter.

We're also evaluating whether we can add certain integrity‑validation mechanisms, so that audit logs can be verified downstream without requiring the collector to enforce immutability strictly.

[github-actions]

This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days.

[hilmarf]

bumping this up, thanks!

[github-actions]

This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days.

[hilmarf]

bumping this up, thanks!

[github-actions]

This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days.