chore(pipe): add kind install + upgrade test for Helm charts #1833
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Charts with a KNOWN install/upgrade failure in a bare, credential-less kind | ||
| # cluster. The install-test gate reports these as a warning instead of failing, | ||
| # so the gate can protect the other charts and all future changes. Remove a chart | ||
| # from this list once its issue is fixed — a clean chart must stay clean. | ||
| # | ||
| # Format: <chart> # reason | ||
| # | ||
| tracer # PodDisruptionBudget sets BOTH minAvailable and maxUnavailable — the API server rejects it ("cannot be both set"). Real chart bug: the PDB template/values must emit only one. | ||
| reporter # References external KEDA custom resources (keda.sh/v1alpha1: ScaledObject / ScaledJob / TriggerAuthentication) whose CRDs are not present in a bare kind cluster. Needs the KEDA CRDs pre-installed (a future enhancement) or the chart guarded behind a toggle. | ||
| otel-collector-lerian # DaemonSet renders a DUPLICATE env var GOMEMLIMIT — server-side apply rejects it ("duplicate entries for key GOMEMLIMIT"). Real chart bug: drop the duplicate (likely set in both the chart values and the opentelemetry-collector subchart). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| #!/usr/bin/env bash | ||
| # install-test.sh — install + upgrade a chart into the current kube context. | ||
| # Core value over `helm template`: the API server validates every manifest | ||
| # server-side (rejects invalid/immutable/admission-failing objects), and the | ||
| # upgrade path catches immutable-field breaks. Does NOT wait for app pods to | ||
| # become Ready (images are private; readiness is a separate, creds-gated concern). | ||
| # | ||
| # Usage: install-test.sh <chart-dir> [values-file] | ||
| set -uo pipefail | ||
|
|
||
| CHART_DIR="$1" | ||
| CHART="$(basename "$CHART_DIR")" | ||
| # Values: explicit arg wins; else the render-gate's vetted sample values, if any. | ||
| VALUES_FILE="${2:-}" | ||
| [[ -z "$VALUES_FILE" && -f ".github/configs/helm-render-values/${CHART}.yaml" ]] \ | ||
| && VALUES_FILE=".github/configs/helm-render-values/${CHART}.yaml" | ||
| REL="$CHART" | ||
| NS="it-${CHART}" # release namespace (helm -n); also created | ||
| TARGETS="" # every distinct namespace the chart's manifests reference (created too) | ||
| TIMEOUT="${IT_TIMEOUT:-180s}" | ||
| VARGS=() | ||
| [[ -n "$VALUES_FILE" && -f "$VALUES_FILE" ]] && { VARGS=(-f "$VALUES_FILE"); echo " values: $VALUES_FILE"; } | ||
| # --no-hooks: hook Jobs (migrations/bootstrap) pull private images and need real | ||
| # backing services, so they never complete in a credential-less CI cluster. We | ||
| # validate the steady-state manifests server-side; hook Jobs are out of scope here. | ||
| HOOKS=(--no-hooks) | ||
|
|
||
| # Library charts are not installable. | ||
| if grep -qiE '^type:[[:space:]]*library([[:space:]]|$)' "$CHART_DIR/Chart.yaml"; then | ||
| echo "::notice::$CHART is a library chart — skipping install test." | ||
| exit 0 | ||
| fi | ||
|
|
||
| fail() { echo "::error::[$CHART] $1"; kubectl get events -n "$NS" --sort-by=.lastTimestamp 2>/dev/null | tail -15; cleanup; exit 1; } | ||
| cleanup() { | ||
| helm uninstall "$REL" -n "$NS" >/dev/null 2>&1 || true | ||
| helm uninstall "${REL}-base" -n "$NS" >/dev/null 2>&1 || true | ||
| for x in "$NS" $TARGETS; do kubectl delete ns "$x" --wait=false >/dev/null 2>&1 || true; done | ||
| } | ||
| # (Re)create the release namespace + every namespace the chart pins, then install. | ||
| do_install() { # <release> <chart-dir> | ||
| for x in "$NS" $TARGETS; do kubectl create ns "$x" >/dev/null 2>&1 || true; done | ||
| helm install "$1" "$2" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NS" --timeout "$TIMEOUT" 2>&1 | ||
| } | ||
| deployed() { [[ "$(helm status "$1" -n "$NS" -o json 2>/dev/null | tr -d ' \n' | grep -o '"status":"[a-z]*"' | head -1)" == '"status":"deployed"' ]]; } | ||
|
|
||
| echo "===== [$CHART] dependency build =====" | ||
| # `helm dependency build` needs every HTTP dependency repo registered first | ||
| # (oci:// and file:// deps don't). Mirror what the render-gate's Go tool does: | ||
| # register each `repository: https://…` from Chart.yaml before building. | ||
| depn=0 | ||
| while IFS= read -r repo_url; do | ||
| [[ -z "$repo_url" ]] && continue | ||
| helm repo add "dep${depn}" "$repo_url" >/dev/null 2>&1 || true | ||
| depn=$((depn + 1)) | ||
| done < <(grep -E 'repository:[[:space:]]*"?https?://' "$CHART_DIR/Chart.yaml" | grep -Eo 'https?://[^"[:space:]]+' | sort -u) | ||
|
Comment on lines
+52
to
+56
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
rg -n -C 2 "repository:[[:space:]]*'https?://" charts --glob 'Chart.yaml' || true
rg -n -C 2 'repository:' charts --glob 'Chart.yaml' || trueRepository: LerianStudio/helm Length of output: 10415 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- install-test.sh ---'
cat -n .github/scripts/install-test.sh | sed -n '1,90p'
printf '%s\n' '--- repository declarations ---'
rg -n "repository:" charts --glob 'Chart.yaml' | head -80
printf '%s\n' '--- regex behavior probe ---'
python3 - <<'PY'
import re
pattern = re.compile(r'repository:[\s]*"?https?://')
extract = re.compile(r'https?://[^"\s]+')
samples = [
'repository: https://repo.example',
'repository: "https://repo.example"',
"repository: 'https://repo.example'",
]
for sample in samples:
matched = bool(pattern.search(sample))
value = extract.search(sample)
print(f"{sample!r}: line_match={matched}, extracted={value.group(0) if value else None}")
PYRepository: LerianStudio/helm Length of output: 9614 Parse The pattern matches plain and double-quoted HTTP(S) URLs but skips valid single-quoted scalars such as 🤖 Prompt for AI AgentsSource: MCP tools |
||
| [[ "$depn" -gt 0 ]] && helm repo update >/dev/null 2>&1 | ||
|
Comment on lines
+51
to
+57
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
rg -n -C 4 \
'HELM_(CONFIG_HOME|REPOSITORY_CONFIG|REPOSITORY_CACHE)|helm repo (add|remove)|install-test\.sh|for c in \$CHARTS' \
.github/scripts/install-test.sh .github/workflows/helm-install-test.yml || trueRepository: LerianStudio/helm Length of output: 3739 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- install-test.sh ---'
cat -n .github/scripts/install-test.sh | sed -n '1,90p'
printf '%s\n' '--- workflow execution context ---'
cat -n .github/workflows/helm-install-test.yml | sed -n '80,120p'
printf '%s\n' '--- repository declarations ---'
rg -n -g 'Chart.yaml' '^[[:space:]]*repository:' charts .github || true
printf '%s\n' '--- Helm availability and relevant configuration ---'
command -v helm || true
rg -n 'HELM_(CONFIG_HOME|REPOSITORY_CONFIG|REPOSITORY_CACHE)|XDG_CONFIG_HOME|XDG_CACHE_HOME' .github charts || trueRepository: LerianStudio/helm Length of output: 11714 🌐 Web query:
💡 Result: In Helm 3, attempting to add a repository using a name that is already configured will result in an error [1][2][3]. This behavior was introduced as a security measure in Helm 3.3.2 to prevent accidental or unauthorized overwriting of existing repository configurations [1][2][4]. If you need to update the URL for an existing repository, you must explicitly use the --force-update flag [5][2][3]: helm repo add [NAME] [URL] --force-update When --force-update is used, Helm will overwrite the existing repository configuration for that name with the new URL provided [3][4]. If you attempt to add a repository with the same name and the exact same URL, recent versions of Helm treat this as a no-op (no operation) and exit with a success code, making the command idempotent [4]. Regarding dependency management: 1. The helm dependency build command requires that all repositories containing dependencies be manually added to your local Helm configuration beforehand [6][7][8]. 2. If a required repository is not in your local list, helm dependency build will fail with an error [8][9]. It does not automatically add repositories that are missing from your configuration [6][8]. 3. For this reason, in automated environments or CI/CD pipelines, you should ensure that the necessary repositories are added using helm repo add (with --force-update if you need to ensure the URL is correct) before running helm dependency build [3][4][8]. Citations:
🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- dependency names and HTTP repository order ---'
python3 - <<'PY'
from pathlib import Path
import re
for path in sorted(Path("charts").glob("*/Chart.yaml")):
text = path.read_text()
deps = re.findall(r'(?ms)^\s*-\s*name:\s*([^\n]+).*?^\s*repository:\s*["\']?([^"\'\s]+)', text)
http = sorted({url for _, url in deps if url.startswith(("http://", "https://"))})
if http:
print(f"{path}:")
print(" dependencies:", ", ".join(name.strip().strip('"\\'') for name, _ in deps))
print(" extracted HTTP URLs:", " | ".join(http))
printf '%s\n' '--- chart scope construction ---'
cat -n .github/workflows/helm-install-test.yml | sed -n '45,85p'Repository: LerianStudio/helm Length of output: 543 🌐 Web query:
💡 Result: In Helm, the Citations:
🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- dependency names and HTTP repository order ---'
python3 - <<'PY'
from pathlib import Path
import re
for path in sorted(Path("charts").glob("*/Chart.yaml")):
text = path.read_text()
deps = re.findall(
r'(?ms)^\s*-\s*name:\s*([^\n]+).*?^\s*repository:\s*["\']?([^"\'\s]+)',
text,
)
http = sorted({url for _, url in deps if url.startswith(("http://", "https://"))})
if http:
print(f"{path}:")
print(" dependencies:", ", ".join(name.strip().strip('"\'') for name, _ in deps))
print(" extracted HTTP URLs:", " | ".join(http))
printf '%s\n' '--- chart scope construction ---'
cat -n .github/workflows/helm-install-test.yml | sed -n '45,85p'Repository: LerianStudio/helm Length of output: 413 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- dependency names and HTTP repository order ---'
python3 - <<'PY'
from pathlib import Path
import re
for path in sorted(Path("charts").glob("*/Chart.yaml")):
text = path.read_text()
deps = re.findall(
r'(?ms)^\s*-\s*name:\s*([^\n]+).*?^\s*repository:\s*["\']?([^"\'\s]+)',
text,
)
http = sorted({url for _, url in deps if url.startswith(("http://", "https://"))})
if http:
names = [name.strip().strip("\"'") for name, _ in deps]
print(f"{path}:")
print(" dependencies:", ", ".join(names))
print(" extracted HTTP URLs:", " | ".join(http))
PY
printf '%s\n' '--- chart scope construction ---'
cat -n .github/workflows/helm-install-test.yml | sed -n '45,85p'
printf '%s\n' '--- Chart.lock repository fields ---'
rg -n -C 2 '^[[:space:]]*repository:' charts -g 'Chart.lock' || true
printf '%s\n' '--- Helm dependency manager source ---'
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/pkg/downloader/manager.go \
| rg -n -C 5 'Repository|repository|ResolveChartVersion|Chart.lock|No repository definition' \
| sed -n '1,240p'Repository: LerianStudio/helm Length of output: 23260 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- Helm build flow and URL matching ---'
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/pkg/downloader/manager.go \
| sed -n '86,240p;458,630p'
printf '%s\n' '--- deterministic sequential repository simulation ---'
python3 - <<'PY'
from pathlib import Path
import re
charts = []
for path in sorted(Path("charts").glob("*/Chart.yaml")):
urls = re.findall(r'repository:\s*"?(https?://[^"\s]+)', path.read_text())
urls = sorted(set(urls))
if urls:
charts.append((str(path), urls))
repo_by_name = {}
for path, urls in charts:
collisions = []
missing = []
for index, url in enumerate(urls):
name = f"dep{index}"
previous = repo_by_name.get(name)
if previous is not None and previous != url:
collisions.append((name, previous, url))
elif previous is None:
repo_by_name[name] = url
configured_urls = set(repo_by_name.values())
missing = [url for url in urls if url not in configured_urls]
print(path)
print(" URLs:", " | ".join(urls))
print(" duplicate-name URL changes:", collisions or "none")
print(" URLs absent after non-forcing adds:", missing or "none")
if missing:
print(" RESULT: dependency build cannot find every repository")
break
PYRepository: LerianStudio/helm Length of output: 12001 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- Helm build flow and URL matching ---'
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/pkg/downloader/manager.go \
| sed -n '86,240p;458,630p'
printf '%s\n' '--- deterministic sequential repository simulation ---'
python3 - <<'PY'
from pathlib import Path
import re
charts = []
for path in sorted(Path("charts").glob("*/Chart.yaml")):
urls = re.findall(r'repository:\s*"?(https?://[^"\s]+)', path.read_text())
urls = sorted(set(urls))
if urls:
charts.append((str(path), urls))
repo_by_name = {}
for path, urls in charts:
collisions = []
for index, url in enumerate(urls):
name = f"dep{index}"
previous = repo_by_name.get(name)
if previous is not None and previous != url:
collisions.append((name, previous, url))
elif previous is None:
repo_by_name[name] = url
configured_urls = set(repo_by_name.values())
missing = [url for url in urls if url not in configured_urls]
print(path)
print(" URLs:", " | ".join(urls))
print(" duplicate-name URL changes:", collisions or "none")
print(" URLs absent after non-forcing adds:", missing or "none")
if missing:
print(" RESULT: dependency build cannot find every repository")
break
PYRepository: LerianStudio/helm Length of output: 12001 Prevent repository-name collisions across chart runs. When multiple charts run in one job, Use an isolated Helm repository configuration per chart or add repositories with 🤖 Prompt for AI AgentsSource: MCP tools |
||
| # Build with one retry (dependency fetches are network-flaky) and surface the real | ||
| # error on final failure instead of a bare "build failed". | ||
| db_out="$(helm dependency build "$CHART_DIR" 2>&1)" \ | ||
| || db_out="$(helm dependency update "$CHART_DIR" 2>&1)" \ | ||
| || { echo "$db_out" | tail -8 | sed 's/^/ /'; fail "helm dependency build failed"; } | ||
|
|
||
| # Lerian charts pin their own namespaces (namespaceOverride / global.namespace), so | ||
| # resources land there regardless of `-n` — and some charts even span MORE than one | ||
| # (most in the release ns, a few in a fixed one). Create EVERY namespace the render | ||
| # references instead of fighting it; the release itself lives in $NS. | ||
| TARGETS="$(helm template "$REL" "$CHART_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NS" 2>/dev/null \ | ||
| | awk '/^ namespace:/{gsub(/"/,"",$2); print $2}' | awk 'NF' | sort -u | grep -vxF "$NS" | tr '\n' ' ')" | ||
| echo " namespaces: $NS${TARGETS:+ + $TARGETS}" | ||
|
|
||
| cleanup # idempotent: clear any stale release/namespace from a prior aborted run | ||
|
|
||
| # ---- 1. Fresh install of the PR chart (server-side manifest validation) ---- | ||
| echo "===== [$CHART] install (PR) =====" | ||
| do_install "$REL" "$CHART_DIR" || fail "helm install failed (invalid manifest / admission / hook)" | ||
| deployed "$REL" || fail "release not in deployed state" | ||
| n=0; for x in "$NS" $TARGETS; do n=$((n + $(kubectl get all -n "$x" --no-headers 2>/dev/null | wc -l))); done | ||
| echo " created $n objects" | ||
| [[ "$n" -gt 0 ]] || fail "install produced no objects" | ||
|
|
||
| # ---- 2. Upgrade the PR chart in place (upgrade code path) ---- | ||
| # Same values on purpose: a no-change upgrade still re-renders and re-applies | ||
| # (new revision, STATUS deployed). Forcing a value change is unsafe — a strict | ||
| # root-closed schema (e.g. br-sfn) rejects an injected podAnnotations key. | ||
| echo "===== [$CHART] upgrade (PR -> PR) =====" | ||
| helm upgrade "$REL" "$CHART_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NS" --timeout "$TIMEOUT" >/dev/null 2>&1 \ | ||
| && deployed "$REL" || fail "in-place upgrade failed" | ||
|
|
||
| # Free the PR release before the baseline so only ONE release is ever installed at | ||
| # a time — two full installs of a subchart-heavy chart exhaust a single-node kind | ||
| # cluster's memory. The baseline reuses the same (chart-pinned) namespace. | ||
| cleanup | ||
|
|
||
| # ---- 3. Real upgrade path: base (origin/main) -> PR, when the chart exists on main ---- | ||
| if git cat-file -e "origin/main:charts/${CHART}/Chart.yaml" 2>/dev/null; then | ||
| echo "===== [$CHART] upgrade (origin/main -> PR) =====" | ||
| BASE_DIR="$(mktemp -d)/$CHART"; mkdir -p "$BASE_DIR" | ||
| git archive "origin/main" "charts/${CHART}" | tar -x --strip-components=2 -C "$BASE_DIR" 2>/dev/null | ||
| helm dependency build "$BASE_DIR" >/dev/null 2>&1 || echo " (base dep build failed — skipping baseline)" | ||
| if do_install "${REL}-base" "$BASE_DIR" >/dev/null 2>&1 && deployed "${REL}-base"; then | ||
| helm upgrade "${REL}-base" "$CHART_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NS" --timeout "$TIMEOUT" >/dev/null 2>&1 \ | ||
| && deployed "${REL}-base" || fail "upgrade from origin/main failed (immutable-field break?)" | ||
| echo " origin/main -> PR upgrade OK" | ||
| else | ||
| echo " (baseline install failed — likely unrelated to this PR; skipping)" | ||
|
Comment on lines
+98
to
+106
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- .github/scripts/install-test.sh ---'
cat -n .github/scripts/install-test.sh | sed -n '1,110p'
printf '%s\n' '--- related references ---'
rg -n -C 3 'VARGS|origin/main|baseline|git archive|helm dependency build|helm install|helm upgrade' .github/scripts .github 2>/dev/null | head -n 240Repository: LerianStudio/helm Length of output: 24975 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- workflow invocation ---'
rg -n -C 5 'install-test\.sh|helm-render-values|values-file|values' .github/workflows .github/scripts 2>/dev/null | head -n 220
printf '%s\n' '--- chart and render-value files ---'
git ls-files 'charts/*/Chart.yaml' '.github/configs/helm-render-values/*' | head -n 160
printf '%s\n' '--- origin/main chart tree for a representative comparison ---'
git ls-tree --name-only origin/main:charts 2>/dev/null | head -n 80
printf '%s\n' '--- relevant branch/ref state ---'
git branch --show-current
git show-ref --verify --quiet refs/remotes/origin/main && git rev-parse --short refs/remotes/origin/main || trueRepository: LerianStudio/helm Length of output: 19700 🏁 Script executed: #!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
p = Path(".github/scripts/install-test.sh")
text = p.read_text()
section = text.split("# ---- 3. Real upgrade path", 1)[1].split('echo "===== [$CHART] OK ====="', 1)[0]
assert 'helm dependency build "$BASE_DIR" >/dev/null 2>&1 || echo' in section
assert 'if helm install "${REL}-base" "$BASE_DIR"' in section
assert ' else\n echo " (baseline install failed — likely unrelated to this PR; skipping)"' in section
assert ' echo " (new chart — not on origin/main; skipping baseline upgrade)"' in section
print("source checks: baseline dependency failure is non-fatal; baseline install failure is a skip; chart absence is the separate skip path")
PY
tmp_script="$(mktemp)"
cat >"$tmp_script" <<'BASH'
set -uo pipefail
CHART=demo
CHART_DIR=charts/demo
REL=demo
NSB=it-demo-base
TIMEOUT=180s
VARGS=(-f .github/configs/helm-render-values/demo.yaml)
HOOKS=(--no-hooks)
fail() { printf '%s\n' "FAIL: $1"; exit 1; }
git() {
if [[ "$1" == cat-file ]]; then return 0; fi
return 1
}
helm() {
case "$2" in
dependency) return 1 ;;
install) return 1 ;;
upgrade) printf '%s\n' 'STATUS: deployed'; return 0 ;;
esac
}
if git cat-file -e "origin/main:charts/${CHART}/Chart.yaml" 2>/dev/null; then
BASE_DIR="$(mktemp -d)/$CHART"; mkdir -p "$BASE_DIR"
helm dependency build "$BASE_DIR" >/dev/null 2>&1 || echo " (base dep build failed — skipping baseline upgrade)"
if helm install "${REL}-base" "$BASE_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NSB" --create-namespace --timeout "$TIMEOUT" >/dev/null 2>&1; then
helm upgrade "${REL}-base" "$CHART_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NSB" --timeout "$TIMEOUT" 2>&1 \
| grep -qE 'STATUS: deployed' || fail "upgrade from origin/main failed (immutable-field break?)"
echo " origin/main -> PR upgrade OK"
else
echo " (baseline install failed — likely unrelated to this PR; skipping)"
fi
else
echo " (new chart — not on origin/main; skipping baseline upgrade)"
fi
echo "===== [$CHART] OK ====="
BASH
output="$(bash "$tmp_script")"
rm -f "$tmp_script"
printf '%s\n' '--- mocked baseline-failure execution ---'
printf '%s\n' "$output"
grep -q 'baseline install failed' <<<"$output"
grep -q '===== \[demo\] OK =====' <<<"$output"
! grep -q 'origin/main -> PR upgrade OK' <<<"$output"
printf '%s\n' 'behavior check: baseline install failure reaches success output without running the upgrade'Repository: LerianStudio/helm Length of output: 288 🏁 Script executed: #!/bin/bash
set -e
tmp_script="$(mktemp)"
cat >"$tmp_script" <<'BASH'
set -uo pipefail
CHART=demo
CHART_DIR=charts/demo
REL=demo
NSB=it-demo-base
TIMEOUT=180s
VARGS=(-f .github/configs/helm-render-values/demo.yaml)
HOOKS=(--no-hooks)
fail() { printf '%s\n' "FAIL: $1"; exit 1; }
git() {
if [[ "$1" == cat-file ]]; then return 0; fi
return 1
}
helm() {
case "$1" in
dependency) return 1 ;;
install) return 1 ;;
upgrade) printf '%s\n' 'STATUS: deployed'; return 0 ;;
*) return 1 ;;
esac
}
if git cat-file -e "origin/main:charts/${CHART}/Chart.yaml" 2>/dev/null; then
BASE_DIR="$(mktemp -d)/$CHART"; mkdir -p "$BASE_DIR"
helm dependency build "$BASE_DIR" >/dev/null 2>&1 || echo " (base dep build failed — skipping baseline upgrade)"
if helm install "${REL}-base" "$BASE_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NSB" --create-namespace --timeout "$TIMEOUT" >/dev/null 2>&1; then
helm upgrade "${REL}-base" "$CHART_DIR" ${VARGS[@]+"${VARGS[@]}"} "${HOOKS[@]}" -n "$NSB" --timeout "$TIMEOUT" 2>&1 \
| grep -qE 'STATUS: deployed' || fail "upgrade from origin/main failed (immutable-field break?)"
echo " origin/main -> PR upgrade OK"
else
echo " (baseline install failed — likely unrelated to this PR; skipping)"
fi
else
echo " (new chart — not on origin/main; skipping baseline upgrade)"
fi
echo "===== [$CHART] OK ====="
BASH
output="$(bash "$tmp_script")"
status=$?
rm -f "$tmp_script"
printf '%s\n' '--- corrected mocked baseline-failure execution ---'
printf '%s\n' "$output"
test "$status" -eq 0
grep -q 'base dep build failed' <<<"$output"
grep -q 'baseline install failed' <<<"$output"
grep -q '===== \[demo\] OK =====' <<<"$output"
! grep -q 'origin/main -> PR upgrade OK' <<<"$output"
printf '%s\n' 'behavior check: baseline install failure reaches success output without running the upgrade'Repository: LerianStudio/helm Length of output: 443 Fail when the baseline upgrade does not run. Lines 74–80 ignore baseline dependency-build failures and treat every baseline-install failure as a successful skip. Use values from 🤖 Prompt for AI Agents |
||
| fi | ||
| else | ||
| echo " (new chart — not on origin/main; skipping baseline upgrade)" | ||
| fi | ||
|
|
||
| echo "===== [$CHART] OK =====" | ||
| cleanup | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| name: Helm Install Test | ||
|
|
||
| # Installs each changed chart into a throwaway kind cluster and exercises the | ||
| # upgrade path. The value over the render-gate (which only runs `helm template`) | ||
| # is that the real API server VALIDATES every manifest server-side — it rejects | ||
| # invalid field values, bad API versions, and immutable-field changes on upgrade | ||
| # that `helm template` renders happily. Hook Jobs (migrations/bootstrap) are | ||
| # skipped (--no-hooks in the script): they pull private images and need real | ||
| # backing services, so they cannot complete in a credential-less CI cluster. | ||
| # App pods are NOT waited on for the same reason — steady-state manifest | ||
| # validation + the upgrade path are what this gate proves. | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - "charts/**" | ||
| - ".github/configs/helm-render-values/**" | ||
| - ".github/scripts/install-test.sh" | ||
| - ".github/workflows/helm-install-test.yml" | ||
| workflow_dispatch: {} | ||
|
|
||
| concurrency: | ||
| group: helm-install-test-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| install-test: | ||
| name: Install + upgrade (kind) | ||
| runs-on: blacksmith-4vcpu-ubuntu-2404 | ||
| timeout-minutes: 30 # a full --all sweep (workflow/lib change) installs every app chart sequentially | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | ||
| with: | ||
| fetch-depth: 0 # full history so the script can `git archive origin/main` for the baseline upgrade | ||
|
|
||
| - name: Fetch base branch | ||
| run: git fetch --no-tags --depth=1 origin main | ||
|
|
||
| - name: Setup Helm | ||
| uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4 | ||
|
|
||
| - name: Create kind cluster | ||
| uses: helm/kind-action@a1b0e391336a6ee6713a0583f8c6240d70863de3 # v1.10.0 | ||
| with: | ||
| cluster_name: chart-install-test | ||
|
|
||
| - name: Determine chart scope | ||
| id: scope | ||
| shell: bash | ||
| env: | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| BASE_SHA: ${{ github.event.pull_request.base.sha }} | ||
| HEAD_SHA: ${{ github.sha }} | ||
| run: | | ||
| set -euo pipefail | ||
| all_app_charts() { | ||
| for d in charts/*/; do | ||
| c="$(basename "$d")" | ||
| grep -qiE '^type:[[:space:]]*library([[:space:]]|$)' "charts/$c/Chart.yaml" 2>/dev/null && continue | ||
| printf '%s ' "$c" | ||
| done | ||
| } | ||
| if [[ "$EVENT_NAME" != "pull_request" ]]; then | ||
| echo "charts=$(all_app_charts)" >> "$GITHUB_OUTPUT"; exit 0 | ||
| fi | ||
| changed="$(git diff --name-only "$BASE_SHA" "$HEAD_SHA")" | ||
| # An infra change (script/workflow/shared library/fixtures) can affect every | ||
| # chart, so test them all; otherwise only the charts whose files changed. | ||
| if grep -Eq '^(\.github/scripts/install-test\.sh|\.github/workflows/helm-install-test\.yml|charts/lerian-common/)' <<< "$changed"; then | ||
| echo "charts=$(all_app_charts)" >> "$GITHUB_OUTPUT"; exit 0 | ||
| fi | ||
| scope="" | ||
| for c in $(awk -F/ '/^charts\/[^/]+\// {print $2}' <<< "$changed" | sort -u); do | ||
| [[ -d "charts/$c" ]] || continue # skip deleted charts | ||
| grep -qiE '^type:[[:space:]]*library([[:space:]]|$)' "charts/$c/Chart.yaml" && continue | ||
| scope="${scope:+$scope }$c" | ||
| done | ||
| echo "charts=$scope" >> "$GITHUB_OUTPUT" | ||
|
Comment on lines
+77
to
+82
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win Select charts for render-values changes. A pull request that changes only Map render-values filenames to their chart, or include render-values changes in the full-chart scope. 🤖 Prompt for AI Agents |
||
|
|
||
| - name: Install + upgrade each chart | ||
| shell: bash | ||
| env: | ||
| CHARTS: ${{ steps.scope.outputs.charts }} | ||
| run: | | ||
| set -uo pipefail | ||
| if [[ -z "${CHARTS// }" ]]; then | ||
| echo "No installable charts in scope — nothing to test."; exit 0 | ||
| fi | ||
| # Charts with a KNOWN pre-existing install/upgrade defect (documented in the | ||
| # allow-failure file) are reported but don't fail the gate — same baseline | ||
| # pattern the chart-standard check uses. A PR that touches such a chart should | ||
| # remove it from the file once fixed. New/changed clean charts must still pass. | ||
| ALLOW="$(grep -vE '^\s*(#|$)' .github/configs/helm-install-test-allow-failure.txt 2>/dev/null | awk '{print $1}' || true)" | ||
| echo "Charts in scope: $CHARTS" | ||
| fails=""; allowed_fails="" | ||
| for c in $CHARTS; do | ||
| echo "::group::install-test $c" | ||
| if bash .github/scripts/install-test.sh "charts/$c"; then | ||
| echo "::endgroup::" | ||
| elif grep -qxF "$c" <<< "$ALLOW"; then | ||
| echo "::endgroup::"; echo "::warning::[$c] install test failed but is allow-listed (known pre-existing defect)" | ||
| allowed_fails="${allowed_fails:+$allowed_fails }$c" | ||
| else | ||
| echo "::endgroup::"; fails="${fails:+$fails }$c" | ||
| fi | ||
| done | ||
| [[ -n "$allowed_fails" ]] && echo "Allow-listed failures (not blocking):$allowed_fails" | ||
| if [[ -n "$fails" ]]; then | ||
| echo "::error::Install/upgrade test failed for:$fails" | ||
| exit 1 | ||
| fi | ||
| echo "All charts in scope installed and upgraded cleanly (allow-listed excepted)." | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: LerianStudio/helm
Length of output: 4049
🏁 Script executed:
Repository: LerianStudio/helm
Length of output: 21927
🏁 Script executed:
Repository: LerianStudio/helm
Length of output: 330
🌐 Web query:
kubectl delete --wait=false behavior Helm uninstall --wait default official documentation💡 Result:
The behavior of
kubectl deleteandhelm uninstallregarding the--waitflag differs in its default state and mechanism. kubectl delete The--waitflag defaults to true [1][2]. When enabled,kubectl deletewaits for all resources specified to be removed from the API (including waiting for finalizers to complete) before returning [1][2]. To prevent this blocking behavior and return immediately after the delete request is acknowledged by the API, you must explicitly pass--wait=false[3][2]. helm uninstall The--waitflag forhelm uninstallbehaves differently [4][5]. Unlikekubectl, the default behavior when the--waitflag is omitted is not to wait for full resource deletion; it instead relies on its internal "hookOnly" strategy [4][5]. When--waitis explicitly provided, it triggers a "watcher" strategy that monitors resources until they are ready or deleted, up to the defined--timeout[4][5]. Crucially, because Helm defaults to "background" cascade strategy, simply adding--waitmay not be sufficient to ensure all resources—especially those with finalizers—are fully cleaned up before the command returns [4][5]. Official Helm documentation recommends using the flag combination--cascade foreground --waitto ensure resources with finalizers are fully deleted before the command exits [4][5]. If a user encounters issues where resources persist after an uninstall, this combination is the standard workaround [6].Citations:
Wait for Helm and namespace deletion before the next install.
Add
--wait --cascade=foreground --timeout "$TIMEOUT"to bothhelm uninstallcommands, and remove--wait=falsefrom both namespace deletions. Otherwise, the baseline install and the next chart can run while previous workloads still consume kind-node capacity, causing flaky or out-of-memory test runs.🤖 Prompt for AI Agents