🔄 synced file(s) with PaddleHQ/go-library-template

.ghokin.yml

@@ -1,5 +1,3 @@
 indent: 2
 aliases:
   json: "jq ."
-exclude:
-  - "ghokin/fixtures/*.input.feature"

.github/actionlint.yaml

@@ -1,8 +1,28 @@
+# GENERATED FILE - DO NOT EDIT
+# Synced from the actionlint_runner_labels local in
+# PaddleHQ/terraform-github by the sync-actionlint-runners workflow.
 self-hosted-runner:
   labels:
+    - airflow-2-core
+    - airflow-4-core
+    - dbt-2-core
+    - dbt-4-core
+    - e2e-runner-16
+    - e2e-runner-2
+    - e2e-runner-32
+    - e2e-runner-4
+    - e2e-runner-8
+    - generic-2-core
+    - go-16-core
+    - go-2-core
+    - go-32-core
     - go-4-core
-    - go-4-core-latest
+    - go-4-core-amd64
+    - go-64-core
     - go-8-core
-    - go-8-core-latest
-    - go-16-core
-    - go-16-core-latest
+    - permifrost-2-core
+    - permifrost-4-core
+    - terraform-16-core
+    - terraform-2-core
+    - terraform-4-core
+    - terraform-8-core

.github/actions/README.md

@@ -0,0 +1,54 @@
+# Composite actions
+
+Shared workflow building blocks. Listed here with a one-line purpose and, for actions that own a cache, the contract that lets you avoid layering parallel caches on top.
+
+| Action | Purpose |
+|---|---|
+| [`setup-go`](./setup-go/action.yml) | Install Go (via `actions/setup-go`), authenticate to private PaddleHQ modules, run `go mod download all`. **Owns the Go deps cache** (see below). |
+| [`setup-databases`](./setup-databases/action.yml) | Start Postgres (+ MySQL on opt-in), create the `runner` role and `testdatabase`, export `TESTAMENT_POSTGRES_DSN`. **Restores the Postgres template cache** (see below). |
+| [`snapshot-postgres-templates`](./snapshot-postgres-templates/action.yml) | Tar the PG data dir at job end so `actions/cache`'s post-step uploads it. **Saves the Postgres template cache** (the other half of the lifecycle owned by `setup-databases`). |
+| [`lint-scope`](./lint-scope/action.yml) | Decide whether `golangci-lint` should run, and if so against which packages. Outputs `skip`, `pkgs`, `new-from-rev`. Used by `go-lint.yml` (main) and `go-lint-experimental.yml` (different `config-glob` input, same logic). |
+| [`otel-export`](./otel-export/action.yml) | Export the workflow trace to Honeycomb at job end. |
+| [`resolve-generated-paths`](./resolve-generated-paths/action.yml) | Expand the `generated-paths` patterns into a concrete list (used by coverage filtering). |
+| [`trigger-automerge`](./trigger-automerge/action.yml) | Kick the automerge workflow once required checks pass. |
+| [`automerge-skipped-comment`](./automerge-skipped-comment/action.yml) | Comment on a PR when automerge declined to run. |
+
+## Cache topology — one owner per artefact
+
+Two caches are managed by composite actions in this repo. **Do not add parallel `actions/cache` steps targeting these paths in any workflow** — they collide on tar extract (`Cannot open: File exists`), `actions/cache` marks the restore as failed, and the post-step saves a fresh ~1.2 GB cache on every run for nothing. This bug has been fixed four times already (lint #483, validate #485, test.yml's `Cache test results` (#487), and inside `actions/setup-go` itself via `cache: false` (#487)).
+
+### Go modules + build cache
+
+| | |
+|---|---|
+| Paths | `~/go/pkg/mod`, `~/.cache/go-build` |
+| Key | `setup-go-${{ runner.os }}-${{ runner.arch }}-go-${{ go-version }}[-<cache-suffix>]-${{ hashFiles('**/go.sum') }}` |
+| Owner | [`setup-go`](./setup-go/action.yml) — explicit `actions/cache@v5.0.5`; `actions/setup-go`'s built-in cache is disabled (it 409s on every primary-key hit) |
+| Used by | every workflow that needs Go — lint, validate, build, fuzz (default key); test (`cache-suffix: cover`); race and test-combined (`cache-suffix: race`) |
+
+`setup-go` takes a `cache-suffix` input. Jobs that compile with non-default flags pass a flavour so each writer owns its own key — race and test-combined pass `race`, test passes `cover`, everything else stays empty. Without per-flavour keys, parallel jobs collide on a single key and only one save wins per run, leaving the others permanently cold.
+
+**Rule:** call `setup-go` (with the right `cache-suffix` if you compile with non-default flags). Don't write a second `actions/cache` step for these paths.
+
+### Postgres template DB
+
+| | |
+|---|---|
+| Path | `/tmp/pg-template.tar.zst` |
+| Key | `${{ runner.os }}-pg${{ pg-major }}-template-testament${{ testament-version }}-${{ migration-hash }}` |
+| Restore owner | [`setup-databases`](./setup-databases/action.yml) |
+| Save owner | [`snapshot-postgres-templates`](./snapshot-postgres-templates/action.yml), called with `if: always()` at job end |
+| Used by | test, test-combined, race |
+
+**Why two composites for one cache?** GitHub composite actions can't declare post-steps. setup-databases restores the tar before PG starts; the snapshot composite tars again at job end so `actions/cache`'s auto-save uploads on cache-miss. Two composites — one for each end of the lifecycle.
+
+**Rule:** if you add a new job that uses testament, call `setup-databases` at the start *and* `snapshot-postgres-templates` at the end. Don't bypass either side.
+
+## Caches not owned here
+
+| Artefact | Owner |
+|---|---|
+| `~/.cache/golangci-lint` (lint analysis cache) | `golangci/golangci-lint-action`'s built-in cache (in `go-lint.yml`) |
+| `~/.cache/go-build/fuzz` (fuzz corpus) | inline `actions/cache` step in `fuzz.yml` (independent sub-path, no collision risk) |
+
+These are noted for completeness — they don't conflict with anything above.

.github/actions/install-go-tools/action.yml

@@ -0,0 +1,48 @@
+name: Install Go tools
+description: |
+  For each requested tool, prepend the baked /opt/runner-tools/<name>/<version>
+  dir to $GITHUB_PATH so consumers use the version baked into the custom runner
+  image. Falls back to `go install` if the baked path is missing (stock runner,
+  or pre-rollout). Versions come from tool-versions.json — synced across repos
+  by paddle-config.
+
+inputs:
+  tools:
+    description: |
+      Whitespace-separated list of tool names. Each must have an entry in
+      tool-versions.json. Example:
+        tools: |
+          govulncheck
+          modernize
+    required: true
+  tool-versions:
+    description: Path to tool-versions.json (relative to the workspace).
+    required: false
+    default: tool-versions.json
+
+runs:
+  using: composite
+  steps:
+    - shell: bash
+      env:
+        TOOLS: ${{ inputs.tools }}
+        TOOL_VERSIONS: ${{ inputs.tool-versions }}
+      run: |
+        set -euo pipefail
+        for name in $TOOLS; do
+          entry=$(jq -r --arg n "$name" '.tools[$n] // empty' "$TOOL_VERSIONS")
+          if [ -z "$entry" ]; then
+            echo "::error::tool '${name}' not found in ${TOOL_VERSIONS}"
+            exit 1
+          fi
+          version=$(jq -r '.version' <<< "$entry")
+          package=$(jq -r '.package' <<< "$entry")
+          baked="/opt/runner-tools/${name}/${version}"
+          if [ -x "${baked}/${name}" ]; then
+            echo "${baked}" >> "$GITHUB_PATH"
+            echo "  ${name}@${version} → baked"
+          else
+            echo "  ${name}@${version} not baked, running go install"
+            go install "${package}@${version}"
+          fi
+        done

.github/actions/lint-scope/action.yml

@@ -0,0 +1,74 @@
+# DO NOT EDIT: This file should only be modified in the `go-library-template` repo.
+
+name: Determine lint scope
+description: |
+  Compute which Go packages to lint and whether to lint at all, based on
+  the diff against the PR's base ref. Three outcomes:
+
+  - lint config changed → `skip=false`, `pkgs=./...`, `new_from_rev` empty
+    (full-codebase lint so any new rules are enforced repo-wide).
+  - one or more `*.go` files changed → `skip=false`, `pkgs` is the unique
+    list of touched directories, `new_from_rev=--new-from-rev=<base>` so
+    golangci-lint reports only issues new to the diff.
+  - neither → `skip=true`. The caller should gate the lint step on
+    `steps.<id>.outputs.skip != 'true'`; running golangci-lint here would
+    default to `./...` and analyse the whole codebase, but `--new-from-rev`
+    would filter every report out, so it'd be pure waste.
+
+  The caller passes a `config-glob` regex matching the lint-config files
+  that should trigger a full-repo lint. Main lint passes the default
+  matching only `.golangci.{yml,yaml}`; experimental lint passes a wider
+  pattern that also matches `.golangci.experimental.{yml,yaml}` because
+  experimental inherits from the main config.
+
+inputs:
+  base-sha:
+    description: SHA of the PR base ref to diff against.
+    required: true
+  config-glob:
+    description: Regex (extended) matching the lint-config file(s) whose change should trigger a full repo lint.
+    default: '(^|/)\.golangci\.(yml|yaml)$'
+    required: false
+
+outputs:
+  skip:
+    description: '"true" if nothing should be linted (no .go files and no config change). Caller should gate the lint step on this.'
+    value: ${{ steps.scope.outputs.skip }}
+  pkgs:
+    description: Space-separated list of packages (or `./...`) to pass to golangci-lint. Empty if `skip` is true.
+    value: ${{ steps.scope.outputs.pkgs }}
+  new-from-rev:
+    description: '`--new-from-rev=<sha>` argument or empty. Pass through to golangci-lint args.'
+    value: ${{ steps.scope.outputs.new_from_rev }}
+
+runs:
+  using: composite
+  steps:
+    - id: scope
+      shell: bash
+      env:
+        BASE_SHA: ${{ inputs.base-sha }}
+        CONFIG_GLOB: ${{ inputs.config-glob }}
+      run: |
+        if git diff --name-only "$BASE_SHA"..HEAD | grep -qE "$CONFIG_GLOB"; then
+          {
+            echo "skip=false"
+            echo "pkgs=./..."
+            echo "new_from_rev="
+          } >> "$GITHUB_OUTPUT"
+        else
+          pkgs=$(git diff --name-only --diff-filter=d "$BASE_SHA"..HEAD -- '*.go' \
+            | xargs -I{} dirname {} \
+            | sort -u \
+            | sed 's|^|./|' \
+            | tr '\n' ' ')
+          if [ -z "$pkgs" ]; then
+            echo "skip=true" >> "$GITHUB_OUTPUT"
+          else
+            {
+              echo "skip=false"
+              echo "pkgs=$pkgs"
+              echo "new_from_rev=--new-from-rev=$BASE_SHA"
+            } >> "$GITHUB_OUTPUT"
+          fi
+        fi

.github/actions/otel-export/action.yml

@@ -8,14 +8,24 @@ description: |
 
 inputs:
   honeycomb-team:
-    description: Honeycomb API key. Pass `${{ secrets.HONEYCOMB_GHA_SECRET }}` — secrets cannot be read directly inside composite actions.
+    description: "Honeycomb API key. Pass secrets.HONEYCOMB_GHA_SECRET — secrets cannot be read directly inside composite actions."
     required: true
+  honeycomb-dataset:
+    description: "Honeycomb dataset name. Pass vars.HONEYCOMB_GHA_DATASET — vars cannot be read directly inside composite actions."
+    required: true
+  github-token:
+    description: "GitHub token for API calls. Pass secrets.GITHUB_TOKEN."
+    required: true
+  extra-attributes:
+    description: "Optional extra OTel resource attributes to append, comma-separated key=value (e.g. service.name=my-workflow-e2e)."
+    required: false
+    default: ""
 
 runs:
   using: composite
   steps:
     - name: Checkout workflow file
-      uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
       with:
         sparse-checkout: .github/workflows
         sparse-checkout-cone-mode: false
@@ -40,6 +50,6 @@ runs:
       continue-on-error: true
       with:
         otlpEndpoint: grpc://api.honeycomb.io:443/
-        otlpHeaders: "x-honeycomb-team=${{ inputs.honeycomb-team }},x-honeycomb-dataset=${{ vars.HONEYCOMB_GHA_DATASET }}"
-        githubToken: ${{ github.token }}
-        customAttributes: '{"workflow.source.hash":"${{ steps.wfhash.outputs.hash }}","runner.image.version":"${{ steps.image.outputs.version }}"}'
+        otlpHeaders: "x-honeycomb-team=${{ inputs.honeycomb-team }},x-honeycomb-dataset=${{ inputs.honeycomb-dataset }}"
+        githubToken: ${{ inputs.github-token }}
+        extraAttributes: "workflow.source.hash=${{ steps.wfhash.outputs.hash }},runner.image.version=${{ steps.image.outputs.version }}${{ inputs.extra-attributes != '' && format(',{0}', inputs.extra-attributes) || '' }}"

.github/actions/setup-databases/action.yml

@@ -1,9 +1,18 @@
 # DO NOT EDIT: This file should only be modified in the `go-library-template` repo.
 
 name: Setup Databases
-description: |
-  This action starts and configures any databases required for testing, and exports the DSN for `go-testament` to use.
-  Use `enableMySQL: true` in your `build-config.yaml` to enable MySQL.
+description: "Starts databases and exports DSNs for `go-testament`. Set `enableMySQL: true` in `build-config.yaml` for MySQL."
+
+outputs:
+  testament-imported:
+    description: "'true' if the consumer's go.mod requires go-testament"
+    value: ${{ steps.go_testament.outputs.imported }}
+  postgres-template-cache-hit:
+    description: "'true' if a Postgres template tar was restored from actions/cache"
+    value: ${{ steps.pg-cache.outputs.cache-hit }}
+  postgres-major-version:
+    description: "PostgreSQL major version detected on the runner (e.g. 17)"
+    value: ${{ steps.pg-detect.outputs.version }}
 
 runs:
   using: "composite"
@@ -26,28 +35,89 @@ runs:
 
     - name: Read build config
       id: build_config
-      uses: step-security/action-read-yaml@08f859a2769067ea7fd26c1cd03a9b940c0ac01b # v1.0.0
+      uses: step-security/action-read-yaml@705ee0c6475c5a26a356e79ee380eb3527b3772d # v1.0.1
       with:
         config: ${{ github.workspace }}/build-config.yaml
 
+    - name: Detect PostgreSQL major version
+      id: pg-detect
+      if: steps.go_testament.outputs.imported == 'true'
+      shell: bash
+      run: |
+        version=$(ls /etc/postgresql 2>/dev/null | grep -E '^[0-9]+$' | sort -n | tail -1)
+        if [ -z "$version" ]; then
+          echo "Error: no PostgreSQL installation found under /etc/postgresql/" >&2
+          exit 1
+        fi
+        echo "version=${version}" >> "$GITHUB_OUTPUT"
+        echo "PostgreSQL major version: ${version}"
+
+    - name: Compute Postgres template cache key
+      id: pg-cache-key
+      if: steps.go_testament.outputs.imported == 'true'
+      shell: bash
+      run: |
+        if [ -d "${GITHUB_WORKSPACE}/database/migrations" ]; then
+          hash=$(find "${GITHUB_WORKSPACE}/database/migrations" -type f -print0 \
+            | sort -z | xargs -0 sha256sum | sha256sum | cut -c1-16)
+        elif [ -d "${GITHUB_WORKSPACE}/migrations" ]; then
+          hash=$(find "${GITHUB_WORKSPACE}/migrations" -type f -print0 \
+            | sort -z | xargs -0 sha256sum | sha256sum | cut -c1-16)
+        else
+          hash="no-migrations"
+        fi
+        # Pin testament version into the key: its template-name hash algorithm changes between versions.
+        testament_version=$(go list -m -f '{{.Version}}' \
+          github.com/PaddleHQ/go-testament/v3 2>/dev/null \
+          || go list -m -f '{{.Version}}' \
+            github.com/PaddleHQ/go-testament/v4 2>/dev/null \
+          || echo "unknown")
+        echo "hash=${hash}" >> "$GITHUB_OUTPUT"
+        echo "testament-version=${testament_version}" >> "$GITHUB_OUTPUT"
+        echo "Migrations hash: ${hash}, testament: ${testament_version}"
+
+    - name: Restore Postgres template cache
+      id: pg-cache
+      if: steps.go_testament.outputs.imported == 'true'
+      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+      with:
+        path: /tmp/pg-template.tar.zst
+        key: ${{ runner.os }}-pg${{ steps.pg-detect.outputs.version }}-template-testament${{ steps.pg-cache-key.outputs.testament-version }}-${{ steps.pg-cache-key.outputs.hash }}
+        restore-keys: |
+          ${{ runner.os }}-pg${{ steps.pg-detect.outputs.version }}-template-testament${{ steps.pg-cache-key.outputs.testament-version }}-
+
     - name: Start PostgreSQL
       if: steps.go_testament.outputs.imported == 'true'
       shell: bash
       run: |
+        if [ -f /tmp/pg-template.tar.zst ]; then
+          sudo systemctl stop postgresql.service || true
+          sudo rm -rf /var/lib/postgresql/${{ steps.pg-detect.outputs.version }}/main
+          sudo mkdir -p /var/lib/postgresql/${{ steps.pg-detect.outputs.version }}
+          sudo tar --zstd -xf /tmp/pg-template.tar.zst -C /var/lib/postgresql/${{ steps.pg-detect.outputs.version }}/
+          sudo chown -R postgres:postgres /var/lib/postgresql/${{ steps.pg-detect.outputs.version }}/main
+          sudo chmod 0700 /var/lib/postgresql/${{ steps.pg-detect.outputs.version }}/main
+        fi
+
         sudo systemctl start postgresql.service
         pg_isready
         sudo -u postgres psql -c "ALTER SYSTEM SET max_connections TO '2000';"
         sudo -u postgres psql -c "SELECT pg_reload_conf();"
         sudo systemctl restart postgresql.service
         pg_isready
 
-        sudo -u postgres psql -c "CREATE USER runner WITH SUPERUSER CREATEDB REPLICATION PASSWORD 'hunter2'"
-        sudo -u postgres psql -c "CREATE DATABASE testdatabase WITH OWNER runner"
+        # NOT EXISTS: cache restore brings the role back; recreating changes the OID.
+        sudo -u postgres psql -tAc "SELECT 1 FROM pg_roles WHERE rolname='runner'" | grep -q 1 \
+          || sudo -u postgres psql -c "CREATE USER runner WITH SUPERUSER CREATEDB REPLICATION PASSWORD 'hunter2'"
+        sudo -u postgres psql -tAc "SELECT 1 FROM pg_database WHERE datname='testdatabase'" | grep -q 1 \
+          || sudo -u postgres psql -c "CREATE DATABASE testdatabase WITH OWNER runner"
 
         echo "TESTAMENT_POSTGRES_DSN=host=127.0.0.1 port=5432 user=runner password=hunter2 dbname=testdatabase sslmode=require" >> "$GITHUB_ENV"
 
-        # deprecated, can be removed once all repos have updated to `go-testament` >= v3.3
-        echo "TESTAMENT_POSTGRES_16_DSN=host=127.0.0.1 port=5432 user=runner password=hunter2 dbname=testdatabase sslmode=require" >> "$GITHUB_ENV"
+        # Version-specific DSN keyed on the runner's detected major version: go-testament
+        # resolves TESTAMENT_POSTGRES_<major>_DSN when a test pins an explicit version, and
+        # its PostgresDefault shares a value with the latest version constant.
+        echo "TESTAMENT_POSTGRES_${{ steps.pg-detect.outputs.version }}_DSN=host=127.0.0.1 port=5432 user=runner password=hunter2 dbname=testdatabase sslmode=require" >> "$GITHUB_ENV"
 
     - name: Start MySQL
       if: steps.build_config.outputs['enableMySQL'] == 'true'