feat(midaz): add unified ledger config (CRM/Fees/KMS/tracer) and tracer templates

charts/midaz/README.md

@@ -3,7 +3,7 @@
 ## Chart Contract
 
 - Chart type: `multi-component`
-- Required secrets: `ledger.secrets.RABBITMQ_DEFAULT_PASS` and `ledger.secrets.RABBITMQ_CONSUMER_PASS` (operator-provided — see "Known limitation" below) plus `crm.secrets.LCRYPTO_HASH_SECRET_KEY` and `crm.secrets.LCRYPTO_ENCRYPT_SECRET_KEY` (app crypto material). The database, replica, and cache passwords (`DB_ONBOARDING_PASSWORD`, `DB_ONBOARDING_REPLICA_PASSWORD`, `MONGO_ONBOARDING_PASSWORD`, `DB_TRANSACTION_PASSWORD`, `DB_TRANSACTION_REPLICA_PASSWORD`, `MONGO_TRANSACTION_PASSWORD`, `REDIS_PASSWORD`, `crm.secrets.MONGO_PASSWORD`) are single-sourced from the bundled Bitnami subcharts and are only required when the matching backend is external.
+- Required secrets: `ledger.secrets.RABBITMQ_DEFAULT_PASS` and `ledger.secrets.RABBITMQ_CONSUMER_PASS` (operator-provided — see "Known limitation" below) plus `crm.secrets.LCRYPTO_HASH_SECRET_KEY` and `crm.secrets.LCRYPTO_ENCRYPT_SECRET_KEY` (app crypto material). With a 4.x `ledger.image.tag` and `KMS_VENDOR=none`, `ledger.secrets.LCRYPTO_HASH_SECRET_KEY` and `ledger.secrets.LCRYPTO_ENCRYPT_SECRET_KEY` are required too: the unified binary serves CRM in-process and initializes its cipher from them at boot. `LCRYPTO_ENCRYPT_SECRET_KEY` must be hex encoding an AES key of 16, 24, or 32 bytes (32, 48, or 64 hex characters); the chart rejects anything else at render time. The database, replica, and cache passwords (`DB_ONBOARDING_PASSWORD`, `DB_ONBOARDING_REPLICA_PASSWORD`, `MONGO_ONBOARDING_PASSWORD`, `DB_TRANSACTION_PASSWORD`, `DB_TRANSACTION_REPLICA_PASSWORD`, `MONGO_TRANSACTION_PASSWORD`, `REDIS_PASSWORD`, `crm.secrets.MONGO_PASSWORD`) are single-sourced from the bundled Bitnami subcharts and are only required when the matching backend is external.
 - Dependency notes: PostgreSQL, MongoDB, and Valkey passwords are single-sourced from the bundled Bitnami subchart Secrets (`<release>-postgresql` key `password`/`replication-password`, `<release>-mongodb` key `mongodb-root-password`, `<release>-valkey` key `valkey-password`) and injected into the ledger/crm workloads via `secretKeyRef`. RabbitMQ and optional OpenTelemetry are also bundled. When a backend is external (`<subchart>.enabled=false`/`.external=true`), supply its password through the component `secrets` block or point the subchart at an `auth.existingSecret`.
 - Production overrides: Provide RabbitMQ and CRM crypto credentials through the component `secrets` (or `useExistingSecret`); let the bundled Bitnami subcharts own the database/cache passwords (or set `<subchart>.auth.existingSecret`/`<subchart>.auth.password`). Override image tags, ingress, resources, namespace, and persistence as needed.
 - Source/license: Source is in `github.com/LerianStudio/helm`; license is Apache-2.0.
@@ -295,6 +295,31 @@ crm:
   #   MONGO_PASSWORD: "<your-mongo-password>"
 ```
 
+### Tracer
+
+The `tracer` service provides real-time transaction validation and fraud prevention. From midaz v4 it ships from the monorepo (`components/tracer`) and is published as `lerianstudio/midaz-tracer`; the standalone `lerianstudio/tracer` 1.x image predates this configuration contract. It is disabled by default (`tracer.enabled=false`) and existing releases render unchanged.
+
+- **Database.** Tracer uses the bundled PostgreSQL with the same `midaz` role that owns `onboarding` and `transaction`. `tracer.configmap.DB_HOST` is empty by default and resolves to this release's PostgreSQL primary Service (`<release>-postgresql-primary`, or `<release>-postgresql` in the standalone topology), so a release name other than `midaz` still connects; set it explicitly for an external PostgreSQL, where the render fails rather than guessing. Its database (`tracer.configmap.DB_NAME`, default `tracer`) is created by `files/midaz/init.sql` on a fresh internal cluster, and by the external bootstrap Job (`global.externalPostgresDefinitions.enabled=true`) otherwise; both paths use the configured name. On a pre-existing internal cluster, initdb scripts no longer run: create the configured database once by hand before enabling tracer (`CREATE DATABASE tracer;` with the default `tracer.configmap.DB_NAME`).
+- **Migrations.** Neither tracer nor the v4 ledger migrates at startup anymore. `tracer.migrations` and `ledger.migrations` render the dedicated `midaz-tracer-migrations` / `midaz-ledger-migrations` runner Jobs, whose tag defaults to the matching application tag. The tracer Job renders when both `tracer.enabled` and `tracer.migrations.enabled` (default `true`) are true; the ledger Job is on for 4.x ledger tags only (`ledger.migrations.enabled` pins it either way). Setting either `migrations.enabled` to `false` means the schema must be applied out of band, because the v4 services no longer migrate themselves. Both Jobs are plain Sync-phase resources because the bundled PostgreSQL Secret they read is only created during Sync; while a Job renders, its Deployment carries `argocd.argoproj.io/sync-wave: "1"` (`<component>.migrations.deploymentSyncWave`) so Argo CD applies the service only after the Job reports Complete. Against a pre-provisioned database, set `argocd.argoproj.io/hook: PreSync` under `<component>.migrations.annotations` instead.
+- **Ledger to tracer transport.** `ledger.configmap.TRACER_TRANSPORT` defaults to `rest`, because the bundled tracer keeps its gRPC reservation seam off (the app only starts it when `TRACER_GRPC_PORT` is set) and its Service exposes HTTP only. To use gRPC, the seam's production transport, set `tracer.configmap.TRACER_GRPC_PORT` (e.g. `":4021"`), which publishes `tracer.service.grpcPort` on the Service and container, and point `ledger.configmap.TRACER_BASE_URL` at it. Selecting `grpc` against a bundled tracer with the seam off fails the render.
+- **Memory.** `tracer.configmap.GOMEMLIMIT` must track `tracer.resources.limits.memory` (~90%). The image ships `GOMEMLIMIT=1800MiB`, sized for a 2Gi container, so without the override the Go heap outgrows a 512Mi cgroup and the pod is OOM-killed.
+- **Fail-fast configuration.** The chart mirrors the tracer boot validators, so misconfiguration fails `helm template` instead of crash-looping: `API_KEY_ENABLED=true` requires an API key and rejects wildcard CORS; `MULTI_TENANT_ENABLED=true` requires `PLUGIN_AUTH_ENABLED=true`, a service API key, and rejects `API_KEY_ENABLED_ONLY_VALIDATION=true`; `tracer.useExistingSecret=true` requires `tracer.existingSecretName`. Booleans are matched the way the runtime parses them (`strconv.ParseBool`), so `TRUE`, `True`, `1`, `t` and `T` are treated as enabled here too.
+
+```yaml
+tracer:
+  enabled: true
+  configmap:
+    API_KEY_ENABLED: "true"
+    CORS_ALLOWED_ORIGINS: "https://app.example.com"
+  secrets:
+    API_KEY: "<your-api-key>"
+  # DB_PASSWORD is single-sourced from the bundled Bitnami postgresql subchart
+  # (Secret `midaz-postgresql`, key `password`) — only set it here when using an
+  # EXTERNAL PostgreSQL (postgresql.enabled=false / postgresql.external=true).
+```
+
+> **Images.** Every image this chart references is expected to be pullable anonymously, so no `imagePullSecrets` are needed for a default install. Three v4 packages do not satisfy that yet: `lerianstudio/midaz-tracer`, `lerianstudio/midaz-tracer-migrations`, and `lerianstudio/midaz-ledger-migrations` still answer `denied/unauthorized` on Docker Hub, while `lerianstudio/midaz-ledger` resolves. The chart keeps the coordinates the midaz release pipeline publishes; making those repositories public is the fix.
+
 ## Observability
 
 Midaz uses [Grafana Docker OpenTelemetry LGTM](https://github.com/grafana/docker-otel-lgtm) for observability. This component collects, processes, and exports telemetry data such as traces and metrics.

charts/midaz/files/midaz/init.sql

@@ -8,4 +8,13 @@ SELECT pg_create_physical_replication_slot('replication_slot');
 SELECT * FROM pg_create_logical_replication_slot('logical_slot', 'pgoutput');
 
 CREATE DATABASE onboarding;
-CREATE DATABASE transaction;
+CREATE DATABASE transaction;
+
+-- Tracer shares this cluster and the `midaz` role that owns the databases above
+-- (its Deployment reads the same subchart Secret). Created unconditionally, and
+-- not gated on tracer.enabled, because initdb scripts run exactly once: gating
+-- would leave the database missing for anyone enabling tracer after install.
+-- An unused empty database costs nothing. The name tracks
+-- tracer.configmap.DB_NAME so the bundled cluster and the external bootstrap
+-- Job create the same database the tracer service and its migration Job read.
+CREATE DATABASE {{ dig "configmap" "DB_NAME" "" (.Values.tracer | default dict) | default "tracer" }};

charts/midaz/templates/_helpers.tpl

@@ -166,6 +166,51 @@ app.kubernetes.io/name: {{ include "midaz.name" .context }}-{{ .name }}
 app.kubernetes.io/instance: {{ .context.Release.Name }}
 {{- end }}
 
+{{/*
+Create a default fully qualified app name for Tracer.
+*/}}
+{{- define "midaz-tracer.fullname" -}}
+{{- if .Values.tracer.fullnameOverride }}
+{{- .Values.tracer.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" (include "midaz.name" .) (default .Values.tracer.name .Values.tracer.nameOverride) | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create Tracer app version
+*/}}
+{{- define "tracer.defaultTag" -}}
+{{- default .Chart.AppVersion .Values.tracer.image.tag }}
+{{- end -}}
+
+{{/*
+Return valid Tracer version label
+*/}}
+{{- define "tracer.versionLabelValue" -}}
+{{ regexReplaceAll "[^-A-Za-z0-9_.]" (include "tracer.defaultTag" .) "-" | trunc 63 | trimAll "-" | trimAll "_" | trimAll "." | quote }}
+{{- end -}}
+
+{{/*
+Tracer Common labels
+*/}}
+{{- define "midaz-tracer.labels" -}}
+helm.sh/chart: {{ include "midaz.chart" .context }}
+{{ include "midaz-tracer.selectorLabels" (dict "context" .context "name" .name) }}
+app.kubernetes.io/version: {{ include "tracer.versionLabelValue" .context }}
+app.kubernetes.io/managed-by: {{ .context.Release.Service }}
+{{- end }}
+
+{{/*
+Tracer Selector labels
+*/}}
+{{- define "midaz-tracer.selectorLabels" -}}
+{{- if .name -}}
+app.kubernetes.io/name: {{ include "midaz.name" .context }}-{{ .name }}
+{{- end }}
+app.kubernetes.io/instance: {{ .context.Release.Name }}
+{{- end }}
+
 {{/*
 Enable internal dependencies
 */}}
@@ -215,3 +260,186 @@ Secret/Service names render even when all bundled subcharts are disabled
 {{- end -}}
 {{- end -}}
 {{- end -}}
+
+{{/*
+midaz.tagIsV4 — reports "true" when the passed image tag is a semver >= 4.0.0
+(pre-releases included), "" otherwise. Non-semver tags ("latest", digests,
+branch builds) resolve to "" so they never trip a version-gated requirement.
+
+Midaz v4 changed two boot contracts that 3.x does not have: the unified ledger
+binary serves CRM in-process (so it initializes the CRM cipher at startup) and
+neither ledger nor tracer migrates its schema anymore. Both are gated on this
+helper so 3.x releases keep rendering exactly as before.
+*/}}
+{{- define "midaz.tagIsV4" -}}
+{{- $tag := . | toString | trimPrefix "v" -}}
+{{- /* Anchored, complete SemVer 2.0.0 match: semverCompare hard-fails rendering
+       on a malformed version, so a prefix-only test would let "4.0.0.1" through
+       and abort the render instead of resolving to "not v4". */ -}}
+{{- if regexMatch "^[0-9]+\\.[0-9]+\\.[0-9]+(-[0-9A-Za-z-]+(\\.[0-9A-Za-z-]+)*)?(\\+[0-9A-Za-z-]+(\\.[0-9A-Za-z-]+)*)?$" $tag -}}
+{{- if semverCompare ">=4.0.0-0" $tag -}}true{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+midaz.ledgerMigrationsEnabled — resolves the tri-state ledger.migrations.enabled.
+Unset (null) means "auto": on for 4.x ledger tags, off for 3.x, which still runs
+its migrations in-process. An explicit true/false always wins, so operators who
+apply the schema out-of-band (managed-Postgres S3 pipeline) can pin it off.
+*/}}
+{{- define "midaz.ledgerMigrationsEnabled" -}}
+{{- $enabled := dig "migrations" "enabled" nil .Values.ledger -}}
+{{- if kindIs "invalid" $enabled -}}
+{{- include "midaz.tagIsV4" (.Values.ledger.image.tag | default .Chart.AppVersion) -}}
+{{- else if eq (toString $enabled) "true" -}}
+true
+{{- end -}}
+{{- end -}}
+
+{{/*
+midaz-tracer.migrationsFullname — one Job name per migration image tag. A Job
+spec is immutable, so a stable name would silently skip the run on upgrade; the
+tag suffix makes every version bump create a new Job. Re-running is safe:
+golang-migrate tracks progress in the schema_migrations table.
+*/}}
+{{- define "midaz-tracer.migrationsFullname" -}}
+{{- include "midaz.migrationsJobName" (dict "base" (include "midaz-tracer.fullname" .) "tag" (include "midaz-tracer.migrationsTag" .)) -}}
+{{- end -}}
+
+{{- define "midaz-ledger.migrationsFullname" -}}
+{{- include "midaz.migrationsJobName" (dict "base" (include "midaz-ledger.fullname" .) "tag" (include "midaz-ledger.migrationsTag" .)) -}}
+{{- end -}}
+
+{{/*
+midaz.migrationsJobName — "<base>-migrations-<tag>" while it fits in the 63
+character limit. Beyond that, truncating the whole string would cut the tag off
+and make two releases collide on one immutable Job name, so the long form
+truncates the base and ends in a hash of the tag, which stays discriminating.
+*/}}
+{{- define "midaz.migrationsJobName" -}}
+{{- $tag := regexReplaceAll "[^a-z0-9.]+" (lower .tag) "-" -}}
+{{- $name := printf "%s-migrations-%s" .base $tag -}}
+{{- if le (len $name) 63 -}}
+{{- $name | trimSuffix "-" | trimSuffix "." -}}
+{{- else -}}
+{{- printf "%s-migrations-%s" (trunc 42 .base | trimSuffix "-" | trimSuffix ".") (sha256sum $tag | trunc 8) -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Migration-runner image tags default to the matching application image tag so the
+schema and the binary reading it can never drift when only one is bumped.
+*/}}
+{{- define "midaz-tracer.migrationsTag" -}}
+{{- dig "migrations" "image" "tag" "" .Values.tracer | default .Values.tracer.image.tag | default .Chart.AppVersion -}}
+{{- end -}}
+
+{{- define "midaz-ledger.migrationsTag" -}}
+{{- dig "migrations" "image" "tag" "" .Values.ledger | default .Values.ledger.image.tag | default .Chart.AppVersion -}}
+{{- end -}}
+
+{{/*
+midaz-tracer.validate — render-time mirror of the tracer bootstrap validators
+(components/tracer/internal/bootstrap). Every rule below is a configuration the
+v4 process rejects at boot, so failing the render turns a CrashLoopBackOff into
+a helm error the operator can read.
+*/}}
+{{- define "midaz-tracer.validate" -}}
+{{- $tracer := .Values.tracer -}}
+{{- $cm := $tracer.configmap | default dict -}}
+{{- $secrets := $tracer.secrets | default dict -}}
+{{- $extra := $tracer.extraEnvVars | default dict -}}
+{{- if and $tracer.useExistingSecret (not $tracer.existingSecretName) -}}
+{{- fail "tracer.useExistingSecret=true requires tracer.existingSecretName (an empty secretRef.name is rejected by the API server)" -}}
+{{- end -}}
+{{- if include "midaz.isTrue" ($cm.API_KEY_ENABLED | default "false") -}}
+{{- if and (not $secrets.API_KEY) (not $tracer.useExistingSecret) -}}
+{{- fail "tracer.secrets.API_KEY is required when API_KEY_ENABLED=true (ValidateAuthConfig rejects an empty key at boot); or set tracer.useExistingSecret" -}}
+{{- end -}}
+{{- if eq ($cm.CORS_ALLOWED_ORIGINS | default "*" | toString) "*" -}}
+{{- fail "tracer.configmap.CORS_ALLOWED_ORIGINS=\"*\" is rejected at boot when API_KEY_ENABLED=true: any site could drive authenticated calls once the key leaks. Set a concrete origin allow-list" -}}
+{{- end -}}
+{{- end -}}
+{{- if include "midaz.isTrue" ($cm.MULTI_TENANT_ENABLED | default "false") -}}
+{{- if not (include "midaz.isTrue" ($cm.PLUGIN_AUTH_ENABLED | default "false")) -}}
+{{- fail "tracer.configmap.PLUGIN_AUTH_ENABLED must be \"true\" when MULTI_TENANT_ENABLED=true: API-key-only auth cannot verify tenant JWT signatures, so any caller could forge a tenantId" -}}
+{{- end -}}
+{{- if or (include "midaz.isTrue" ($cm.API_KEY_ENABLED_ONLY_VALIDATION | default "false")) (include "midaz.isTrue" (dig "API_KEY_ENABLED_ONLY_VALIDATION" "false" $extra)) -}}
+{{- fail "API_KEY_ENABLED_ONLY_VALIDATION=true is incompatible with MULTI_TENANT_ENABLED=true: it lets /v1/validations bypass plugin auth, reopening cross-tenant forgery" -}}
+{{- end -}}
+{{- if and (not $secrets.MULTI_TENANT_SERVICE_API_KEY) (not $tracer.useExistingSecret) -}}
+{{- fail "tracer.secrets.MULTI_TENANT_SERVICE_API_KEY is required when MULTI_TENANT_ENABLED=true" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+midaz.isTrue — "true" when the value is one of the tokens strconv.ParseBool
+accepts as true, "" otherwise. Midaz loads boolean env vars through ParseBool,
+so `TRUE`, `True`, `1`, `t` and `T` all enable a feature at runtime; comparing
+against the literal string "true" would let those render past a validation the
+process then fails, and would reject a valid `PLUGIN_AUTH_ENABLED=TRUE`.
+Matching is exact (no trimming), like ParseBool: " true" is false at runtime.
+*/}}
+{{- define "midaz.isTrue" -}}
+{{- if has (. | toString) (list "1" "t" "T" "TRUE" "true" "True") -}}true{{- end -}}
+{{- end -}}
+
+{{/*
+midaz.postgresqlPrimaryHost — in-cluster hostname of the bundled PostgreSQL
+primary for THIS release, so a release named something other than `midaz` still
+resolves its database. Bitnami names the Service `<fullname>-primary` in the
+replication topology and `<fullname>` when standalone. Returns "" when
+PostgreSQL is external, where only the operator knows the address.
+*/}}
+{{- define "midaz.postgresqlPrimaryHost" -}}
+{{- $pg := .Values.postgresql | default dict -}}
+{{- if and (ne (toString $pg.enabled) "false") (not $pg.external) -}}
+{{- $fullname := include "common.names.dependency.fullname" (dict "chartName" "postgresql" "chartValues" $pg "context" .) -}}
+{{- if eq ($pg.architecture | default "standalone") "replication" -}}
+{{- printf "%s-primary" $fullname -}}
+{{- else -}}
+{{- $fullname -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+midaz-tracer.dbHost — explicit tracer.configmap.DB_HOST, else the bundled
+PostgreSQL primary of this release. Fails when neither is available, which is
+the external-PostgreSQL case where a wrong guess would silently point the
+Deployment and the migration Job at a host that does not exist.
+*/}}
+{{- define "midaz-tracer.dbHost" -}}
+{{- $explicit := (.Values.tracer.configmap | default dict).DB_HOST -}}
+{{- if $explicit -}}
+{{- $explicit -}}
+{{- else -}}
+{{- $bundled := include "midaz.postgresqlPrimaryHost" . -}}
+{{- if $bundled -}}
+{{- $bundled -}}
+{{- else -}}
+{{- fail "tracer.configmap.DB_HOST is required when PostgreSQL is external (postgresql.enabled=false / postgresql.external=true)" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+midaz.validateLcryptoKey — LCRYPTO_ENCRYPT_SECRET_KEY is hex-decoded and handed
+to aes.NewCipher (lib-commons Crypto.InitializeCipher), so it must be hex
+encoding 16, 24 or 32 bytes. A non-hex value fails at boot with
+"encoding/hex: invalid byte" and a wrong length with "crypto/aes: invalid key
+size", both of which this catches at render time instead. Only applies to keys
+supplied through values; an existing Secret is opaque to the chart.
+*/}}
+{{- define "midaz.validateLcryptoKey" -}}
+{{- $key := .key | toString -}}
+{{- if $key -}}
+{{- if not (regexMatch "^[0-9a-fA-F]+$" $key) -}}
+{{- fail (printf "%s must be a hex-encoded AES key (0-9a-f only): it is hex-decoded before aes.NewCipher, so a non-hex value fails at boot" .name) -}}
+{{- end -}}
+{{- if not (has (len $key) (list 32 48 64)) -}}
+{{- fail (printf "%s must be 32, 48 or 64 hex characters (AES-128/192/256); got %d" .name (len $key)) -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}