fix(observability-pipelines-worker): correct sgc_path in bootstrap template

[ckelner]

Summary

• Removes hardcoded sgc_path from bootstrap.yaml template, deferring to OPW's built-in default (/opt/.../bin/sgc since 2.13.0)
• Prevents duplicate secret: YAML block when both bootstrap.config and secretFileContents are set
• Uses hasKey instead of dig truthiness to correctly detect key presence regardless of value

Problem

The bootstrap.yaml template hardcoded sgc_path as /opt/datadog/observability-pipelines-worker/bin/datadog-secret-backend, but the binary was renamed to sgc in 2.15.1:

# Dockerfile:32
cp /opt/datadog-agent/embedded/bin/secret-generic-connector \
   /opt/datadog/observability-pipelines-worker/bin/sgc

OPW already defaults to the correct path when sgc_path is omitted (lib/config/src/bootstrap/mod.rs:35):

const DEFAULT_SGC_INSTALL_PATH: &str = "/opt/datadog/observability-pipelines-worker/bin/sgc";

When secretFileContents was used, OPW failed to load pipeline config:

ERROR: Failed to reload remote configuration.
  "Error while retrieving secret from backend \"exec_backend\": No such file or directory (os error 2)"

Additionally, providing bootstrap.config.secret alongside secretFileContents to override the path produced a duplicate secret: YAML key, which OPW rejects.

Fix

Three changes to templates/bootstrap.yaml:

1. Remove sgc_path line entirely, deferring to OPW's built-in default
2. Skip emitting the auto-generated secret: block when bootstrap.config already defines one
3. Use hasKey instead of dig to detect key presence (handles empty/null values correctly)

Test plan - helm template render results

All 6 permutations verified locally via helm template:

Case 1: secretFileContents only (most common path)

Expected: single secret: block with backend_type/backend_config, no sgc_path

bootstrap.yaml: |-
  secret:
    backend_type: json
    backend_config:
      file_path: /etc/observability-pipelines-secrets/secrets.json

Result: PASS - single block, no sgc_path, OPW uses default

Case 2: bootstrap.config.secret (vault) + secretFileContents

Expected: only user's vault config, no duplicate auto-generated block

bootstrap.yaml: |-
  secret:
    backend_type: vault
    sgc_path: /custom/sgc

Result: PASS - user config wins, no duplicate

Case 3: bootstrap.config.secret="" + secretFileContents (codex edge case)

Expected: user's secret: "" rendered, no duplicate auto block

bootstrap.yaml: |-
  secret: ""

Result: PASS - hasKey detects key presence despite empty value, no duplicate

Case 4: bootstrap.config (non-secret keys) + secretFileContents

Expected: user's non-secret config + auto secret: block (secret key absent from config)

bootstrap.yaml: |-
  api:
    enabled: true
  secret:
    backend_type: json
    backend_config:
      file_path: /etc/observability-pipelines-secrets/secrets.json

Result: PASS - auto block correctly added when user config has no secret key

Case 5: bootstrap.config.secret only (no secretFileContents)

Expected: user's config only, no auto block

bootstrap.yaml: |-
  secret:
    backend_type: aws

Result: PASS - unchanged behavior

Case 6: Neither bootstrap.config nor secretFileContents

Expected: no bootstrap configmap rendered at all

Result: PASS - 0 matches for "bootstrap" in rendered output

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccb904388e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[tessneau]

looks like OPW defaults secret.sgc_path when it's omitted -- see the default setting to /sgc in 2.21.1 & this test

That default matches the sgc binary since 2.13.0, whereas the change to /sgc happened in 2.15.1. Should we simply omit secret.sgc_path from the template and defer to the worker default?

(note i haven't seen the google doc yet)

[tessneau]

ok yeah I stand by this after looking at the doc. Changing the hardcoded path works too but omitting it fully to defer to the default feels a tad nicer

[tessneau]

could we add a changelog entry and bump the chart version for this ? this looks like a previous example #2814

[ckelner]

Addressed all review feedback in 1fe094c:

@tessneau - Good call on omitting sgc_path entirely. Updated to defer to OPW's built-in default (/opt/.../bin/sgc since 2.13.0). Also added CHANGELOG entry for 2.21.2 and bumped the chart version.

[tessneau]

thank you !