feat(ffe): PHP FFE implementation#3630
Conversation
|
✨ Fix all issues with BitsAI or with Cursor
|
18f7f00 to
508a8c8
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3630 +/- ##
==========================================
+ Coverage 62.32% 62.39% +0.07%
==========================================
Files 142 142
Lines 13586 13586
Branches 1775 1775
==========================================
+ Hits 8467 8477 +10
+ Misses 4311 4304 -7
+ Partials 808 805 -3 see 3 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
ff84598 to
9f36eb7
Compare
7c43198 to
7bbed08
Compare
Benchmarks [ tracer ]Benchmark execution time: 2026-05-04 15:15:24 Comparing candidate commit 3b08f04 in PR branch Found 0 performance improvements and 4 performance regressions! Performance is the same for 190 metrics, 0 unstable metrics. scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache
scenario:PDOBench/benchPDOOverhead
scenario:PDOBench/benchPDOOverheadWithDBM
scenario:PHPRedisBench/benchRedisOverhead
|
## Motivation Add Feature Flagging and Experimentation (FFE) support to the remote config infrastructure, enabling tracers to subscribe to FFE_FLAGS configurations via the sidecar. WIP: php tracer changes (DataDog/dd-trace-php#3630) ## Changes - Add `FfeFlags` variant to `RemoteConfigProduct` enum - Add `"FFE_FLAGS"` string mapping in Display and FromStr - Add `FfeFlagConfigurationRules = 46` to `RemoteConfigCapabilities` - Add `FfeFlags(Vec<u8>)` variant to `RemoteConfigData` to preserve raw config bytes ## Decisions - Raw bytes are preserved (not parsed) in `FfeFlags(Vec<u8>)` since each tracer handles evaluation with the `datadog-ffe` crate directly - Capability bit 46 matches the server-side FFE capability definition Co-authored-by: leo.romanovsky <leo.romanovsky@datadoghq.com>
1990188 to
867337e
Compare
a24b1ae to
24e4304
Compare
2a5d688 to
1ec323b
Compare
Add FFE support to dd-trace-php. Flag evaluation is delegated to libdatadog's datadog-ffe Rust crate via FFI. PHP handles orchestration: config lifecycle, exposure dedup, and HTTP transport. Rust FFI layer (components-rs/ffe.rs): - C-callable bridge to datadog-ffe::rules_based - Global config store behind Mutex<FfeState> - Structured attribute passing (no JSON on hot path) C extension (ext/ddtrace.c): - ffe_evaluate, ffe_has_config, ffe_config_changed, ffe_load_config - Marshals PHP arrays to FfeAttribute structs Remote Config (components-rs/remote_config.rs): - Register FfeFlags product + FfeFlagConfigurationRules capability - Handle add/remove of FFE configs via sidecar PHP Provider (src/DDTrace/FeatureFlags/Provider.php): - Singleton checking RC config state - Calls native evaluate, parses JSON results - Reports exposures via LRU-deduplicated writer Exposure pipeline: - LRU cache (65K entries) with length-prefixed composite keys - Batched writer to /evp_proxy/v2/api/v2/exposures (1000 cap) - Auto-flush via register_shutdown_function OpenFeature adapter (src/DDTrace/OpenFeature/DataDogProvider.php): - Implements AbstractProvider for open-feature/sdk Build: - Add datadog-ffe to RUST_FILES in Makefile for PECL packaging - Cargo.lock: minimal additions only (73 new crates, no gratuitous bumps) - Bump libdatadog to ed316b638 (FFE RC support, libdatadog#1532) Tests: - LRU cache unit tests (11 tests) - Exposure cache unit tests (12 tests) - 220 evaluation correctness tests from JSON fixtures Config: DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED (default: false)
…OpenFeature layers - Add null check for flag_key in ddog_ffe_evaluate Rust FFI - Set variant in OpenFeature ResolutionDetails (was always null) - Replace magic numbers with named constants for reason/error codes - Fix json_decode null ambiguity in JSON type parsing with json_last_error() - Add dropped event tracking and curl failure logging (DD_TRACE_DEBUG) - Guard against missing curl extension in ExposureWriter flush - Fix buildEvent docblock parameter order
Add FlagEvalMetrics OpenFeature hook that records an OTel counter (feature_flag.evaluations) after every flag evaluation. Attributes: feature_flag.key, feature_flag.result.variant, feature_flag.result.reason (lowercase), error.type (on error). Enabled only when DD_METRICS_OTEL_ENABLED=true and open-telemetry/sdk is installed. Is a complete noop otherwise. Register hook in DataDogProvider constructor via setHooks(). Add FlagEvalMetrics.php to _files_tracer.php autoloader.
…-zero error codes - Fix errorCodeToTag() to match actual Rust constants: 1=ERROR_TYPE_MISMATCH→type_mismatch, 2=ERROR_CONFIG_PARSE→parse_error, 3=ERROR_FLAG_UNRECOGNIZED→flag_not_found (was transposed) - Per OpenFeature spec, force reason=ERROR for any non-zero error_code. FlagUnrecognizedOrDisabled returns REASON_DEFAULT from the Rust layer but must be reported as ERROR to callers.
f624325 to
6b83643
Compare
- FlagEvalMetrics.php: class constant visibility (private const) requires PHP 7.1+; drop to bare const for PHP 7.0 compatibility. The class is already internal so visibility adds nothing. - metadata/supported-configurations.json: regenerate after adding DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED to configuration.h. Fixes the "Configuration Consistency" CI check.
…tements Windows MSVC compiles PHP extensions in C89 mode (no /std:c11), which requires all variable declarations to precede executable statements within a block scope. The ffe_evaluate handler had three violations: - c_attrs and attrs_count declared after the targeting_key if-statement - idx/key/val declared after the c_attrs = ecalloc() call - val/var/ak declared after array_init(return_value) Moves all declarations to the top of their respective blocks.
- Add feature_flag.provider.name=datadog to OTel metric attributes - Propagate error codes to OpenFeature ResolutionDetails via withError() - Prefer dd_trace_env_config over getenv in isFeatureFlagEnabled() - Log non-2xx HTTP status codes in ExposureWriter debug mode - Clarify continue-in-switch comment in ddtrace.c
Thanks for asking, some details below:
|
Thanks! That's my intuition as well for this first landing. |
|
I mean, in particular this section from the doc:
is at least for dd-trace-php not true (at least not on the scale where it would be needed). dogstatsd metrics however, are actually aggregated internally in the php tracer. |
…-flagging # Conflicts: # ext/configuration.h
…er PR #3630 review Addresses all outstanding reviewer feedback from #3630: bwoebi: - Blocker #1: Route exposures through sidecar, zero HTTP from PHP main thread. Implements Rust ExposureState with 65K-entry LRU dedup + 1000-event batch buffer. PHP ExposureWriter uses injectable sidecarCallable routing to DDTrace\ffe_send_exposure() — fire-and-forget, never blocks evaluation. - Blocker #2: Proper documented functions in ddtrace.stub.php (not dd_trace_internal_fn). Eight DDTrace\ffe_* functions declared with PHPDoc @internal: ffe_evaluate, ffe_has_config, ffe_config_changed, ffe_load_config, ffe_send_exposure, ffe_flush_exposures, ffe_set_service_context, ffe_reset_exposure_state. Each has PHP_FUNCTION impl + arginfo + registration. - emalloc/ecalloc memory hygiene and C99 brace style consistent with existing dd-trace-php conventions. - Fire-and-forget: DataDogProvider::resolveViaFfe() invokes exposureWriter->send() and ignores return. No exception propagates from exposure path to evaluation. - Primitive-only attribute filtering in EvaluationContextNormalizer mirrors the C-side filter in ffe_evaluate (drops arrays, nulls, objects, resources). dd-oleksii: - Null targeting key preserved end-to-end (not coerced to empty string). EvaluationContextNormalizer returns [?string, array]. - do_log per-allocation hard gate: ExposureContext::fromBridgeResult() returns null when do_log=false; provider skips exposure send. - OTel feature_flag.evaluations counter with OBSV-02 attributes (key, variant, reason, error.type, allocation_key). MetricsCounter injectable callable, no open-telemetry/api composer dep per CLAUDE.md. Gated on DD_METRICS_OTEL_ENABLED. Fires on every evaluation including errors. - PR split into four independently-reviewable phases (roadmap in staging repo). realFlowControl: - ZTS per-thread FFE state: declined for v1 (NTS-only per PROJECT.md). Global Mutex<FfeState> is correct for NTS. Tracked as v2 requirement. Architecture changes: - Remove src/DDTrace/FeatureFlags/ (old in-process HTTP architecture). LRUCache.php, ExposureCache.php, ExposureWriter.php, Provider.php, FlagEvalMetrics.php — all replaced by Rust-sidecar pipeline. - Remove tests/FeatureFlags/ (tests covered by staging PHPUnit suite). - Add src/DDTrace/OpenFeature/ with 9 classes: BridgeResultMapper, ContextFlattener, DataDogProvider (rewritten), EvaluationContextNormalizer, ExposureContext, ExposureWriter, MetricsCounter, OpenFeatureLifecycleCompatibility, ProviderLifecycle. - Add tests/OpenFeature/ with 7 test files, 143 tests. - components-rs/ffe.rs: add ExposureState + extern C functions for enqueue_exposure, flush_exposures, set_service_context, reset_exposure_state, free_flush_result. Uses lru crate 0.12. - components-rs/Cargo.toml: add lru = "0.12". - composer.json: add open-feature/sdk ^2.1 as require-dev. - src/bridge/_files_tracer.php: load new OpenFeature files, remove FeatureFlags. Verification: 143/143 staging PHPUnit tests pass.
Two Rust compile errors from PR CI: 1. E0599: as_bytes() unresolved on Slice<'static, i8> because AsBytes trait not in scope. Add `use libdd_common_ffi::slice::AsBytes;` inside ddog_ffe_free_flush_result. 2. E0133: CharSlice::from_raw_parts is unsafe, requires unsafe block in ddog_ffe_flush_exposures return path.
…enFeature/ Composer PSR-4 autoload only mapped DDTrace\\ to src/api/, so tests and host code that reference DDTrace\\OpenFeature\\BridgeResultMapper etc failed to autoload. Add explicit DDTrace\\OpenFeature\\ mapping before the catch-all DDTrace\\ entry so PSR-4 resolution hits it first. Verified: vendor/bin/phpunit tests/OpenFeature passes 143/143.
Extension autoloader already handles DDTrace\\OpenFeature\\* via _files_tracer.php include list when datadog.trace.sources_path is set, matching how DDTrace\\OpenTelemetry\\* is wired. Keeping the explicit psr-4 entry split OpenFeature from the OpenTelemetry/OpenTracer pattern. Test runners must set `-d datadog.trace.sources_path=/path/to/src` (set automatically by Makefile's TRACER_SOURCES_INI).
…scenario PHP 7.x CI (api unit) failed because open-feature/sdk ^2.1 requires PHP ^8.0, and composer resolves require-dev during install — breaking root vendor/bin/phpunit setup on 7.x before any tests ran. Move the dep to an `openfeature` entry under extra.scenarios (same pattern as `opentelemetry1`), so PHP 7.x default scenario installs clean and 8+ CI jobs opt in by running `composer scenario openfeature` before the featureflags testsuite.
Suite pointed at ./FeatureFlags/ which does not exist. OpenFeature adapter tests live at tests/OpenFeature/ (matching src/DDTrace/OpenFeature/).
… compat) _files_tracer.php eagerly includes OpenFeature bridge files, which are concatenated into _generated_tracer.php and autoloaded on every request. readonly is PHP 8.1+, so PHP 8.0 parsed "public readonly ?string" and died: ParseError: syntax error, unexpected token "?", expecting variable Drop readonly on promoted constructor params. Immutability was a nice-to-have only; no code outside the class mutates these fields. Follow-up: the OpenFeature adapter still uses other PHP 8.0+ features (match, constructor promotion, union types), so PHP 7.x will hit parse errors once the composer scenario fix unblocks its build. Needs lazy-load gate in ext/autoload_php_files.c mirroring the OpenTelemetry pattern.
Wires the `featureflags` phpunit testsuite (tests/OpenFeature) into CI per bwoebi's PR #3630 feedback. Gated to PHP 8.0+ because open-feature/sdk requires PHP 8.
internal-api-stress-test.php only uses ReflectionFunction::getNumberOfRequiredParameters() on PHP 8.1+; on older versions the fuzzer starts enumerating from zero args. For the 5-required-arg ffe_send_exposure that explodes to ~15^5 permutations and exhausts PHP's 128MB memory limit before ArgumentCountError pruning runs. Skip the function on PHP < 8.1 so the randomized stress test stays green on 7.x and 8.0.
components-rs/Cargo.toml added datadog-ffe and lru in ccaad89 but the lockfile regeneration from that commit was dropped during the later master merge. Local cargo build repopulated the two entries; check them in so the lockfile matches the manifest.
…+ gate)
OpenFeature adapter code uses PHP 8.0+ syntax (match, union types,
constructor promotion). Baking it into _files_tracer.php caused the
concatenated _generated_tracer.php to fail parsing on PHP 7.x, which
cascaded into autoload failures like "add DDTrace\\Transport to bridge/
_files.php" because the whole file aborted before registering earlier
classes.
Mirror the OpenTelemetry lazy-load pattern:
- Move OpenFeature entries from _files_tracer.php into new
_files_openfeature.php.
- Add openfeature_is_loaded state flag (DDTRACE_G + preload save/restore).
- In dd_perform_autoload, match ddtrace\\openfeature\\ before the
legacy-tracer branch, gated on PHP_VERSION_ID >= 80000. On 7.x
return NULL — the adapter is unavailable but does not break the
rest of the tracer.
- Wire _generated_openfeature.php into tooling/generation so the
compiled bridge exists alongside the non-compiled _files_ fallback.
The featureflags suite loads ddtrace\openfeature\* classes via the extension's autoloader, which reads the compiled _generated_openfeature.php bridge. That file is produced by the "Prepare code" stage (make generate) but the job was only listing "compile extension: debug" under needs:, so the bridge artifact was never pulled into the test container — 142/143 tests failed with "Class DDTrace\\OpenFeature\\X not found". Add "Prepare code" to the needs list so src/bridge/_generated_*.php ships alongside the extension .so.
test_featureflags fails on CI with "Class OpenFeature\interfaces\provider\Reason not found" and downstream "Class DDTrace\OpenFeature\DataDogProvider not found" because the SDK is never installed. The bridge autoload of _generated_openfeature.php declares DataDogProvider extends AbstractProvider; without the SDK on the autoloader path, the parent lookup fails and the whole bridge file aborts mid-load. - Add tests/OpenFeature/composer.json requiring open-feature/sdk ^2.1 so the existing tests/%/composer.lock-php pattern rule installs it into tests/OpenFeature/vendor/. - Wire test_featureflags to depend on tests/OpenFeature/composer.lock-php$(PHP_MAJOR_MINOR) so the install runs before phpunit is invoked. The bootstrap_common walkup autoloader picks up tests/OpenFeature/vendor/ when loading OpenFeature test files, registering the SDK autoloader before the extension autoloads the adapter bridge.
…bundle RC init flags in a struct Addresses two dd-oleksii review threads on PR #3630: 1) components-rs/ffe.rs: change `FfeState.changed: bool` (drain-on-read) to `FfeState.version: u64`. Consumers track their last observed value and compare; multiple independent subscribers can detect transitions without racing each other (previous CAS semantics meant only the first reader saw a transition). - ddog_ffe_config_changed() -> bool REMOVED - ddog_ffe_config_version() -> u64 ADDED - PHP: DDTrace\ffe_config_changed -> DDTrace\ffe_config_version(): int - ProviderLifecycle tracks $lastSeenVersion, compares on checkForConfigChange, and syncs on transitionToReady so the transition itself is not observed as another change. - Tests rewritten to drive the version counter. 2) components-rs/remote_config.rs: replace 4-bool positional args of ddog_init_remote_config(...) with a `DdogRemoteConfigFlags` #[repr(C)] struct. Caller in ext/sidecar.c uses C designated initializers, making the subscription flags self-documenting at the call site. Also clarifies the FfeState doc comment (components-rs/ffe.rs:12-22) to note v1 is NTS-only per PROJECT.md, with ZTS support tracked as a follow-up phase where per-thread state moves into DDTRACE_G().
Import ffe-system-test-data (ufc-config + 24 evaluation-case JSON files) into
tests/OpenFeature/testdata/ and add FfeFixturesTest that drives each case
through the PHP FFE bridge (DDTrace\ffe_load_config + DDTrace\ffe_evaluate).
Mirrors dd-trace-go's TestEvaluateFlag_JSONFixtures and dd-trace-java's
DDEvaluatorTest. 220 sub-cases, 1320 assertions, all pass.
Value parity is enforced strictly. Reason classification treats
{STATIC, TARGETING_MATCH, SPLIT} as interchangeable ("successful match") and
{DEFAULT, DISABLED, ERROR} as interchangeable (all produce defaultValue
through the OpenFeature provider). libdatadog and the canonical fixtures
currently classify a few cases differently (start/end-date allocations and
one multi-split flag) — value correctness holds across the split; only the
reason taxonomy drifts. Captured in the test docblock for follow-up.
…EVP proxy PR #3630 review (@bwoebi) caught that `ddog_ffe_flush_exposures` had no production caller -- exposures enqueued forever into EXPOSURE_STATE and never reached the agent. This wires the missing transport. Path A+: evaluation stays in-process (no IPC on the hot path), exposure state stays in-process (Mutex), sidecar owns async HTTP transport. Flush is triggered by PHP RSHUTDOWN + MSHUTDOWN. Matches dd-trace-go / ruby / python / js / dotnet protocol -- POST `/evp_proxy/v2/api/v2/exposures` with `X-Datadog-EVP-Subdomain: event-platform-intake`. No agent_info gate. Changes: - components-rs/ffe.rs: realign dedup key to dd-trace-go semantics -- key = (flag, targeting), value = (allocation, variant); duplicate iff both unchanged. Rewrite the flush doc comment to describe the real RSHUTDOWN -> sidecar -> EVP path. - libdatadog bump: adds `SidecarAction::FfeExposures` variant, an `ffe_flusher` module that POSTs to the agent EVP proxy, and `ddog_sidecar_send_ffe_exposures` FFI (see submodule commit). - ext/sidecar.c: drain FFE buffer in ddtrace_sidecar_rshutdown and ddtrace_sidecar_shutdown (MSHUTDOWN), and reset EXPOSURE_STATE in ddtrace_sidecar_handle_fork so the child doesn't re-send the parent's pre-fork buffered events (bounded duplication -- <=1 extra exposure per unique (flag, subject) per fork, caught by server-side dedup). - ext/ddtrace.c + regenerated headers: cbindgen prefix alignment (`struct FfeAttribute` -> `struct ddog_FfeAttribute`, `struct FfeResult` -> `struct ddog_FfeResult`, `DdogRemoteConfigFlags` -> `ddog_DdogRemoteConfigFlags`). Tests: - components-rs/ffe.rs: dedup semantics (V1) + batch payload schema (V2) as `#[cfg(test)]` cases, serialised on TEST_LOCK. - libdatadog sidecar ffe_flusher: 3 httpmock cases (V3) -- POST/path/ subdomain header, non-2xx drop, endpoint path override. - tests/ext/ffe/flush_drains_buffer.phpt (V4): enqueue/dedup/re-emit via DDTrace\\ffe_send_exposure then flush-drain round-trip. - tests/ext/ffe/fork_resets_dedup.phpt (V9): parent primes, forks; child re-emits same (flag, subject) while parent still dedups. Scope: NTS only for v1 (PROJECT.md). Architecture is ZTS-compatible (Mutex<ExposureState>) but ZTS CI remains out of scope. Deferred: buffer-threshold flush for long-lived CLI workers (separate PR). See FFE_FLUSH_PLAN.md for the full plan + validation ledger.
…-flagging # Conflicts: # libdatadog
Motivation
Add FFE (Feature Flags & Experimentation) support to dd-trace-php. PHP applications can evaluate feature flags delivered via Remote Config using the same
datadog-ffeRust engine used by Ruby and Python. Per the RFC "Flag evaluations tracking for APM tracers", we also emit afeature_flag.evaluationsOTel counter metric on each evaluation so flag usage can be tracked in Datadog.Changes
Core FFE Implementation
components-rs/ffe.rs): C-callable bridge todatadog-ffe::rules_based— config store, evaluate, result accessorsext/ddtrace.c):ffe_evaluate,ffe_has_config,ffe_config_changed,ffe_load_configinternal functions that marshal PHP arrays toFfeAttributestructscomponents-rs/remote_config.rs): RegisterFfeFlagsproduct +FfeFlagConfigurationRulescapability; handle add/remove of FFE configssrc/DDTrace/FeatureFlags/Provider.php): Singleton that checks RC config state, calls native evaluate, parses JSON results, reports exposures; enforcesreason=ERRORfor any non-zero error code per OpenFeature spec/evp_proxy/v2/api/v2/exposures(1000 event buffer cap)src/DDTrace/OpenFeature/DataDogProvider.php): ImplementsAbstractProviderfor theopen-feature/sdkcomposer packagedatadog-ffetoRUST_FILESin Makefile for PECL packagingDD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLEDgating via X-macro inext/configuration.hOTel Flag Evaluation Metrics (new)
src/DDTrace/FeatureFlags/FlagEvalMetrics.php: OTel counter hook that emitsfeature_flag.evaluations(count=1) after each evaluation. Tags:feature_flag.key,feature_flag.provider.name=datadog,feature_flag.result.variant,feature_flag.result.reason,feature_flag.result.allocation_key, and on error:error.type(type_mismatch/parse_error/flag_not_found).errorCodeToTag()now maps Rust constants correctly:1=ERROR_TYPE_MISMATCH→type_mismatch,2=ERROR_CONFIG_PARSE→parse_error,3=ERROR_FLAG_UNRECOGNIZED→flag_not_found(was transposed in initial implementation).Provider.php: Forcesreason=ERRORwhenerror_code != 0. The Rust layer returnsREASON_DEFAULTforFlagUnrecognizedOrDisabledbut the OpenFeature spec requiresERRORto be surfaced to callers.Decisions
Evaluation in Rust, not PHP. All flag evaluation (UFC parsing, targeting rules, shard hashing, allocation resolution) happens in
libdatadog'sdatadog-ffecrate via FFI. PHP only handles orchestration (config lifecycle, exposure dedup, HTTP transport). This matches Ruby and Python — no language re-implements evaluation logic.Global config behind
Mutex<FfeState>. The Rust FFE config is stored in alazy_staticglobal with aMutex. PHP is single-threaded per process, soRwLockwould be unnecessary complexity.Reuses existing RC pipeline. FFE configs flow through the same sidecar →
ddog_process_remote_configs()path as APM Tracing and Live Debugger. No new polling mechanism.Structured attributes, not JSON blobs. The C extension converts PHP arrays into
FfeAttributestructs (typed: string/number/bool) before calling Rust, avoiding JSON encode/decode overhead on the hot path.ExposureWriter caps at 1000 events per request. Matches Ruby and Python. Flush via
register_shutdown_function.FlagEvalMetrics via OTel SDK. Uses
open-telemetry/api+ OTLP exporter. Recorded after the type-specific evaluation method returns (not inside coreevaluate()) so type mismatch errors are captured withreason=errorrather than the intermediatetargeting_match.ERROR reason forced for non-zero error codes.
FlagUnrecognizedOrDisabledreturnsREASON_DEFAULTfrom Rust (the default value was served) but the OpenFeature spec requiresreason=ERRORwhenever an error code is set.Provider.phpoverrides the Rust-reported reason in this case.Test Results
Unit tests
System tests — all FFE tests pass (PHP 8.0-apache)
The 1 xfail is
test_cross_request_dedup— expected because PHP's shared-nothing architecture prevents cross-request LRU state.OTel metrics tests (all 5 pass)
LOC (excluding Cargo.lock + JSON fixtures)
Companion PRs