Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/API_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ Block receives a `Langfuse::Config` object with these properties:
| `release` | String | No | `nil` (or `ENV["LANGFUSE_RELEASE"]` / common CI commit SHA env) | Default release identifier |
| `should_export_span` | `#call` | No | `nil` | Span export filter callback |
| `mask` | `#call` | No | `nil` | Mask callable for input/output/metadata (receives `data:` keyword) |
| `mask_otel_spans` | `#call` | No | `nil` | Export-stage span masking hook (receives `spans:` keyword; see [CONFIGURATION.md](CONFIGURATION.md#mask_otel_spans)) |

**Example:**

Expand Down
27 changes: 27 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,32 @@ config.mask = lambda { |data:|

See [TRACING.md](TRACING.md#masking) for usage patterns and behavior details.

#### `mask_otel_spans`

- **Type:** `#call` (Proc, Lambda, or any object responding to `call`) or `nil`
- **Default:** `nil` (export-stage masking disabled)
- **Description:** Export-stage masking hook applied to every span batch exported to Langfuse, including spans created by third-party instrumentations. Receives a `spans:` keyword argument — a frozen Hash mapping stable span identifiers (`"<hex trace_id>:<hex span_id>"`) to immutable `Langfuse::OtelSpanSnapshot` values (trace/span IDs, name, instrumentation scope name, frozen span attributes, frozen resource attributes). Return `nil` to export the batch unchanged, or a sparse Hash of patches keyed by the same identifiers. Each patch is a Hash with optional `:delete` (Array of attribute keys) and `:set` (Hash of replacement attributes); deletes run before sets, so a replacement wins when a key appears in both.

```ruby
config.mask_otel_spans = lambda { |spans:|
spans.filter_map { |id, snapshot|
next unless snapshot.attributes.key?("gen_ai.prompt")

[id, { set: { "gen_ai.prompt" => "[REDACTED]" } }]
}.to_h
}
```

Fail-closed behavior:

- A hook exception or an invalid top-level result drops the whole Langfuse export batch.
- A malformed per-span patch drops that span from the Langfuse export.
- An invalid replacement attribute value omits that attribute rather than exporting its original value.

Only the copy exported to Langfuse is transformed. Any other OpenTelemetry exporter receives the original, unmasked spans — Langfuse masking does not protect other telemetry backends. The hook is synchronous and runs on the batch export thread; it must not rely on request context, the current span, async work, or network calls.

See [TRACING.md](TRACING.md#masking) for how the two masking boundaries relate.

## Tracing Behavior and OpenTelemetry Ownership

There are three states worth documenting.
Expand Down Expand Up @@ -623,6 +649,7 @@ Validation rules:
- `prompt_cache_observer` must respond to `#call` (if set)
- `should_export_span` must respond to `#call` (if set)
- `mask` must respond to `#call` (if set)
- `mask_otel_spans` must respond to `#call` (if set)

## Accessing Current Configuration

Expand Down
29 changes: 29 additions & 0 deletions docs/TRACING.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,3 +316,32 @@ end
```

Masking applies to observation `input`, `output`, and `metadata`. The full configuration contract is in [CONFIGURATION.md](CONFIGURATION.md#mask).

### Export-stage masking with `mask_otel_spans`

`mask` runs while the SDK creates Langfuse-owned attributes; it never sees the raw
attributes of third-party spans (e.g. `gen_ai.*` attributes set by an OpenAI or
LangChain instrumentation). To transform those, configure the export-stage hook:

```ruby
Langfuse.configure do |config|
config.mask_otel_spans = lambda { |spans:|
spans.filter_map { |id, snapshot|
next unless snapshot.attributes.key?("gen_ai.prompt")

[id, { delete: ["gen_ai.completion"], set: { "gen_ai.prompt" => "[REDACTED]" } }]
}.to_h
}
end
```

The hook runs after `should_export_span` selects spans and just before the batch
is handed to the Langfuse OTLP exporter, so it sees exactly the spans this client
will export — Langfuse-owned and third-party alike. Returning `nil` exports the
batch unchanged. Errors fail closed: the batch (or the offending span) is dropped
rather than exported unmasked.

Only the Langfuse export copy is transformed. If you also send telemetry to
another OpenTelemetry backend, that backend receives the original spans and needs
its own masking. The two hooks are independent; applications may configure either
or both. The full contract is in [CONFIGURATION.md](CONFIGURATION.md#mask_otel_spans).
1 change: 1 addition & 0 deletions lib/langfuse.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ class UnauthorizedError < ApiError; end
require_relative "langfuse/sampling"
require_relative "langfuse/otel_setup"
require_relative "langfuse/masking"
require_relative "langfuse/masking_exporter"
require_relative "langfuse/otel_attributes"
require_relative "langfuse/propagation"
require_relative "langfuse/span_processor"
Expand Down
44 changes: 26 additions & 18 deletions lib/langfuse/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,26 @@ class Config

# @return [#call, nil] Mask callable applied to input, output, and metadata before serialization.
# Receives `data:` keyword argument. nil disables masking.
# This is a creation-time hook for Langfuse-owned attributes; it never sees
# raw third-party span attributes. See {#mask_otel_spans} for those.
attr_accessor :mask

# @return [#call, nil] Export-stage masking hook for spans exported to Langfuse.
# Receives a `spans:` keyword argument — a frozen Hash mapping stable span
# identifiers (`"<hex trace_id>:<hex span_id>"`) to {OtelSpanSnapshot}
# values covering every span in one export batch, including spans created
# by third-party instrumentations. Returns nil to export the batch
# unchanged, or a sparse Hash of patches keyed by the same identifiers.
# Each patch is a Hash with optional +:delete+ (Array of attribute keys)
# and +:set+ (Hash of replacement attributes); deletes run before sets.
# Only the copy exported to Langfuse is transformed — any other
# OpenTelemetry exporter receives the original, unmasked spans, so
# Langfuse masking does not protect other telemetry backends.
# The hook is synchronous and must not rely on request context, the
# current span, async work, or network calls. Exceptions and invalid
# results fail closed by dropping the Langfuse export batch.
attr_accessor :mask_otel_spans

# @return [String] Default Langfuse API base URL
DEFAULT_BASE_URL = "https://cloud.langfuse.com"

Expand Down Expand Up @@ -193,10 +211,11 @@ def validate!
validate_swr_config!

validate_cache_backend!
validate_prompt_cache_observer!
validate_sample_rate!
validate_should_export_span!
validate_mask!
validate_callable!(prompt_cache_observer, "prompt_cache_observer")
validate_callable!(should_export_span, "should_export_span")
validate_callable!(mask, "mask")
validate_callable!(mask_otel_spans, "mask_otel_spans")
end
# rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity

Expand Down Expand Up @@ -242,6 +261,7 @@ def initialize_tracing_defaults
self.sample_rate = env_value("LANGFUSE_SAMPLE_RATE") || DEFAULT_SAMPLE_RATE
@should_export_span = nil
@mask = nil
@mask_otel_spans = nil
end

def validate_cache_backend!
Expand All @@ -252,10 +272,10 @@ def validate_cache_backend!
"cache_backend must be one of #{valid_backends.inspect}, got #{cache_backend.inspect}"
end

def validate_prompt_cache_observer!
return if prompt_cache_observer.nil? || prompt_cache_observer.respond_to?(:call)
def validate_callable!(value, name)
return if value.nil? || value.respond_to?(:call)

raise ConfigurationError, "prompt_cache_observer must respond to #call"
raise ConfigurationError, "#{name} must respond to #call"
end

def validate_swr_config!
Expand Down Expand Up @@ -296,18 +316,6 @@ def validate_sample_rate!
raise ConfigurationError, "sample_rate must be between 0.0 and 1.0"
end

def validate_mask!
return if mask.nil? || mask.respond_to?(:call)

raise ConfigurationError, "mask must respond to #call"
end

def validate_should_export_span!
return if should_export_span.nil? || should_export_span.respond_to?(:call)

raise ConfigurationError, "should_export_span must respond to #call"
end

def detect_release_from_ci_env
COMMON_RELEASE_ENV_KEYS.each do |key|
value = env_value(key)
Expand Down
Loading
Loading