Skip to content

ops: Emit opentelemetry traces via Unix Domain Socket#9078

Merged
cdecker merged 3 commits intomasterfrom
2026w11-uds-tracing
Apr 28, 2026
Merged

ops: Emit opentelemetry traces via Unix Domain Socket#9078
cdecker merged 3 commits intomasterfrom
2026w11-uds-tracing

Conversation

@cdecker
Copy link
Copy Markdown
Member

@cdecker cdecker commented Apr 23, 2026

common/trace.c has the ability to emit the traces it collects via a User-Defined Tracepoint which hands off the serialized trace into a ring buffer in the kernel if there is a listener active. This works, but is not available in some environments, specifically Kubernetes tends to not like exposing eBPF endpoints.

The challenge in finding a new sink is that it must be capable of supporting concurrent writes from unsynchronized processes (main daemon, subprocesses and plugins), preventing corruption of individual writes. The solution is to use a UDS with datagram mode (UDP) enabled. This means that writes up to the MTU of the medium are guaranteed to be atomic.

The tracing is enabled by setting the CLN_TRACE_SOCKET environment variable, binding a listener to the indicated path and read packets from it. The CLN processes will attach to the socket and use it to deliver its traces.

@cdecker cdecker force-pushed the 2026w11-uds-tracing branch from 72636e1 to 2233d09 Compare April 23, 2026 09:06
@cdecker
Copy link
Copy Markdown
Member Author

cdecker commented Apr 23, 2026

I had a test for this too, but ruff insisted on reformatting the entire file, which I found rather disruptive.

@madelinevibes madelinevibes added this to the 26.06 milestone Apr 24, 2026
@cdecker cdecker force-pushed the 2026w11-uds-tracing branch from b5b8b4d to 6e52e5f Compare April 24, 2026 13:59
cdecker added 3 commits April 24, 2026 16:09
Add a new tracing backend that sends completed spans as atomic
datagrams over a Unix Domain Socket (AF_UNIX + SOCK_DGRAM), enabled
via the CLN_TRACE_SOCKET envvar.

Refactor trace.c so span management logic compiles unconditionally
rather than being gated behind #if HAVE_USDT. DTRACE_PROBE macros
are defined as no-ops when USDT is unavailable. When no backend is
active (no USDT, no trace file, no socket), all tracing functions
return early via disable_trace.

Changelog-Added: tracing: Add a unix-domain socket sink for opentelemetry traces
Add Backend 2 section for the new CLN_TRACE_SOCKET UDS datagram
transport. Restructure the existing USDT docs as Backend 1. Add
an environment variable reference table and a minimal Python
collector example.
@cdecker cdecker force-pushed the 2026w11-uds-tracing branch from 6e52e5f to 64d806f Compare April 24, 2026 14:10
@cdecker cdecker merged commit 1fbf809 into master Apr 28, 2026
85 of 87 checks passed
@cdecker cdecker deleted the 2026w11-uds-tracing branch April 28, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants