Skip to content

feat: hash-based manual release workflow#20

Merged
ValentaTomas merged 10 commits intomainfrom
feat/release-workflow
May 9, 2026
Merged

feat: hash-based manual release workflow#20
ValentaTomas merged 10 commits intomainfrom
feat/release-workflow

Conversation

@ValentaTomas
Copy link
Copy Markdown
Member

Mirrors the manual release pipeline from e2b-dev/fc-versions, but because the kernel inputs (configs + patches) live in this repo, each build is identified by a content hash of those inputs:

vmlinux-<kernel_version>_<sha256[:7]>

Changing a flag in configs/x86_64/<v>.config or dropping a patch in patches/<v>/ produces a new, traceable artifact and release; unchanged kernels reuse their existing artifact.

What's in here

  • .github/workflows/release.ymlworkflow_dispatch pipeline: validatebuild (matrix per missing arch) → publish (GH release with vmlinux-{amd64,arm64}.bin + legacy vmlinux.bin) → deploy (uploads to gs://$GCP_BUCKET_NAME/kernels/<version_name>/). Existing artifacts/objects are skipped.
  • scripts/validate.py — resolves version names, computes hashes, generates the build matrix, and queries the GitHub release to skip arches that are already published.
  • build.sh — now takes <kernel_version> [arch], computes the same version_name (or honors VERSION_NAME), and applies any patches/<version>/*.patch. make build / make build-arm64 still work.
  • Drops the old build-on-every-push workflow; releases are now explicit and manual.

Workflow inputs

  • kernel_versions (optional, comma-separated) — defaults to kernel_versions.txt.
  • build_amd64 / build_arm64 (default true) — arch toggles.

Notes

  • Required repo config is unchanged: GCP_WORKLOAD_IDENTITY_PROVIDER, GCP_SERVICE_ACCOUNT_EMAIL, vars.GCP_BUCKET_NAME.
  • Bash and Python hash implementations are kept in sync; verified locally that they produce the same hash for the existing 6.1.102 and 6.1.158 configs.

Mirror the manual build/release pipeline from e2b-dev/fc-versions, but
identify each kernel build by a content hash of its inputs (configs +
optional patches). Changing a flag or adding a patch yields a new
artifact named vmlinux-<version>_<sha256[:7]>.

- scripts/validate.py: resolves version names, computes hashes, builds
  the matrix and skips arches whose artifact is already in the GitHub
  release.
- build.sh: now accepts <kernel_version> [arch], computes the same
  version_name (or honors VERSION_NAME), supports patches/<version>/.
- .github/workflows/release.yml: workflow_dispatch with validate ->
  build -> publish (GH release) -> deploy (GCS), skipping work whose
  artifacts already exist.
- Drops the old build-on-every-push workflow; releases are explicit.
@cla-bot cla-bot Bot added the cla-signed label May 8, 2026
@cursor
Copy link
Copy Markdown

cursor Bot commented May 8, 2026

PR Summary

Medium Risk
Medium risk because it rewires the CI/release workflow (triggers, tagging, and GCS upload) and changes build script behavior, which can break publishing or produce wrong artifacts if assumptions are off.

Overview
The old push-triggered fc-kernels.yml workflow is removed and replaced with release.yml, which builds all (kernel version × arch) pairs on pull requests and only publishes (GitHub release + GCS upload) on workflow_dispatch using a calver tag.

build.sh is changed to accept an optional single version and arch, apply optional patches/<version>/*.patch, and write outputs under builds/vmlinux-<version>/{amd64,arm64}/vmlinux.bin with a legacy vmlinux.bin copy for x86_64; it also force-checks out the matching upstream tag and installs deps based on target arch.

The PR title/description mention hash-based versioning and skipping already-published artifacts, but the included workflow/script changes shown here don’t implement that (no content-hash naming or reuse/skip logic), so releases will always rebuild and tag by date.

Reviewed by Cursor Bugbot for commit fd2430a. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Artifact path loses version_name directory on download
    • Changed artifact upload path from ./builds/$version_name to ./builds to preserve the version_name directory structure in artifacts.
  • ✅ Fixed: Multi-version local builds break when patches are applied
    • Added -f flag to git checkout to force-overwrite uncommitted patch changes between multi-version build iterations.

Create PR

Or push these changes by commenting:

@cursor push a85f2957d0
Preview (a85f2957d0)
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -67,7 +67,7 @@
       - uses: actions/upload-artifact@v4
         with:
           name: ${{ matrix.version_name }}-${{ matrix.arch }}
-          path: ./builds/${{ matrix.version_name }}
+          path: ./builds
           retention-days: 7
 
   publish:

diff --git a/build.sh b/build.sh
--- a/build.sh
+++ b/build.sh
@@ -102,7 +102,7 @@
   cp "$SCRIPT_DIR/configs/${target_arch}/${version}.config" .config
 
   echo "Checking out repo for kernel at version: $version"
-  git checkout "$(get_tag "$version")"
+  git checkout -f "$(get_tag "$version")"
 
   apply_patches "$version"

You can send follow-ups to the cloud agent here.

Comment thread .github/workflows/release.yml Outdated
Comment thread build.sh
@ValentaTomas ValentaTomas marked this pull request as ready for review May 8, 2026 19:43
- upload-artifact: keep version_name in the artifact tree so the
  publish step's ./builds/<version_name>/<arch>/vmlinux.bin lookup
  resolves after merge-multiple download.
- build.sh: git checkout -f so a dirty tree from a prior iteration's
  apply_patches doesn't abort multi-version local builds.
Comment thread .github/workflows/release.yml Outdated
@ValentaTomas ValentaTomas marked this pull request as draft May 8, 2026 19:47
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Hash includes nested patches that apply_patches never applies
    • Modified hash computation in both build.sh and validate.py to only include top-level *.patch files, matching the behavior of apply_patches.

Create PR

Or push these changes by commenting:

@cursor push 3e7936a79e
Preview (3e7936a79e)
diff --git a/build.sh b/build.sh
--- a/build.sh
+++ b/build.sh
@@ -37,7 +37,7 @@
   if [ -d "$SCRIPT_DIR/patches/$version" ]; then
     while IFS= read -r -d '' f; do
       files+=("$f")
-    done < <(find "$SCRIPT_DIR/patches/$version" -type f -print0 | sort -z)
+    done < <(find "$SCRIPT_DIR/patches/$version" -maxdepth 1 -type f -name '*.patch' -print0 | sort -z)
   fi
   if [ "${#files[@]}" -eq 0 ]; then
     echo "Error: no configs found for kernel version $version" >&2

diff --git a/scripts/validate.py b/scripts/validate.py
--- a/scripts/validate.py
+++ b/scripts/validate.py
@@ -51,7 +51,7 @@
             paths.append(cfg)
     patches_dir = REPO_ROOT / "patches" / version
     if patches_dir.is_dir():
-        paths.extend(sorted(p for p in patches_dir.rglob("*") if p.is_file()))
+        paths.extend(sorted(p for p in patches_dir.glob("*.patch") if p.is_file()))
 
     if not paths:
         raise SystemExit(f"::error::No configs found for kernel version {version}")

You can send follow-ups to the cloud agent here.

Comment thread build.sh
apply_patches only processes patches/<v>/*.patch, so the hash inputs
must match. Otherwise unrelated/nested files in patches/<v>/ would
silently change the version_name without affecting the build.
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Inconsistent kernel_versions.txt parsing between Python and Bash
    • Updated build.sh to strip inline comments and trim whitespace, matching the Python parser's behavior to ensure consistent version hashing.

Create PR

Or push these changes by commenting:

@cursor push c23d53f5a8
Preview (c23d53f5a8)
diff --git a/build.sh b/build.sh
--- a/build.sh
+++ b/build.sh
@@ -161,7 +161,9 @@
     local version_name="${VERSION_NAME:-$(compute_version_name "$single_version")}"
     build_version "$single_version" "$target_arch" "$version_name"
   else
-    grep -v '^ *#' <"$SCRIPT_DIR/kernel_versions.txt" | while IFS= read -r version; do
+    grep -v '^ *#' <"$SCRIPT_DIR/kernel_versions.txt" | while IFS= read -r raw; do
+      version="${raw%%#*}"
+      version="$(echo "$version" | xargs)"
       [ -z "$version" ] && continue
       local version_name
       version_name="$(compute_version_name "$version")"

You can send follow-ups to the cloud agent here.

Comment thread scripts/validate.py Outdated
@ValentaTomas ValentaTomas marked this pull request as ready for review May 8, 2026 23:38
Strip inline comments and trim whitespace so build.sh and
scripts/validate.py produce identical version names for entries like
`6.1.158 # stable`.
Comment thread .github/workflows/release.yml Outdated
- workflow_dispatch with no inputs; pick the branch in the GitHub UI.
- Always build every entry in kernel_versions.txt for amd64 and arm64
  in parallel (one runner per arch).
- One release per run, tagged YYYY.MM.DD (with .N suffix on collision),
  with every binary uploaded as vmlinux-<version>-<arch>.bin (plus the
  legacy vmlinux-<version>.bin for amd64).
- Same binaries pushed to GCS under kernels/vmlinux-<version>/...

Drops the per-build content-hash version_name machinery and the
validate.py helper that fed it.
Comment thread .github/workflows/release.yml
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Calver tag suffix skips .1 due to off-by-one
    • Changed counter initialization from n=1 to n=0 so the first suffixed tag is .1 instead of .2, matching the documented behavior.

Create PR

Or push these changes by commenting:

@cursor push 73766c5030
Preview (73766c5030)
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -67,7 +67,7 @@
           set -euo pipefail
           base="$(date -u +%Y.%m.%d)"
           tag="$base"
-          n=1
+          n=0
           while gh release view "$tag" >/dev/null 2>&1; do
             n=$((n + 1))
             tag="${base}.${n}"

You can send follow-ups to the cloud agent here.

Comment thread .github/workflows/release.yml
Open PRs now run the build matrix and upload kernel binaries as
workflow artifacts so reviewers can grab and inspect them from the
PR's checks tab. Release/GCS publishing remains manual-only.
Comment thread .github/workflows/release.yml
Generate the build matrix dynamically from kernel_versions.txt so each
(version, arch) pair runs on its own runner. With N versions × 2 archs
that's 2N parallel jobs (e.g. 2 versions -> 4 runners).

build.sh already supports single-version mode, so no script changes;
the matrix job emits a JSON include list that the build job fans out
over.
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Comment thread .github/workflows/release.yml Outdated
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Calver tag check ignores existing git tags causing retry failures
    • Changed tag uniqueness check from 'gh release view' to 'git rev-parse' to detect existing git tags, making the workflow idempotent on partial failures.

Create PR

Or push these changes by commenting:

@cursor push 4a50f9f36e
Preview (4a50f9f36e)
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -86,7 +86,7 @@
           base="$(date -u +%Y.%m.%d)"
           tag="$base"
           n=0
-          while gh release view "$tag" >/dev/null 2>&1; do
+          while git rev-parse "$tag" >/dev/null 2>&1; do
             n=$((n + 1))
             tag="${base}.${n}"
           done

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit c167397. Configure here.

Comment thread .github/workflows/release.yml
…alver

- Drop workflow-level id-token: write; only the publish job (which
  needs OIDC for GCP auth) gets id-token: write. PR builds no longer
  receive a token they can exchange for cloud creds.
- Validate kernel versions against [0-9]+(\.[0-9]+)+ in the matrix job
  and pass matrix.version to build.sh via env vars instead of inline
  YAML expression interpolation, so a malicious kernel_versions.txt
  entry can't shell-inject into the runner.
- Calver tag picker now also checks local and remote git tags, not
  just GH releases, so retries after a partial publish no longer pick
  a tag that's already been pushed.
The 'Main' ruleset on this repo requires status checks named exactly
'Build kernels (x86_64)' and 'Build kernels (arm64)' (left over from
the previous workflow). The new per-(version, arch) jobs use different
names, so those required checks never reported and PRs couldn't merge.

Add a tiny aggregator job whose name interpolates to those exact two
strings and that gates on the build matrix's combined result, so
parallelism stays per-(version, arch) but branch protection is
satisfied.
@ValentaTomas ValentaTomas merged commit fbae1bf into main May 9, 2026
13 checks passed
@ValentaTomas ValentaTomas deleted the feat/release-workflow branch May 9, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants