Replace GoReleaser with Make + snapcraft pipeline (keep Snap publishing)#776
Replace GoReleaser with Make + snapcraft pipeline (keep Snap publishing)#776Copilot wants to merge 8 commits into
Conversation
Agent-Logs-Url: https://github.com/tg123/sshpiper/sessions/4f658271-17a0-43dc-803e-6453ef92de65 Co-authored-by: tg123 <170430+tg123@users.noreply.github.com>
Agent-Logs-Url: https://github.com/tg123/sshpiper/sessions/200f5796-e3a7-4c0e-837f-3f1d1f780eac Co-authored-by: tg123 <170430+tg123@users.noreply.github.com>
|
@copilot could you please make binaries exactly the same in docker image and gh release? e.g. cp from docker or other solution |
…naries Agent-Logs-Url: https://github.com/tg123/sshpiper/sessions/017bf54b-383c-48d3-ba0a-0ce63ed0da20 Co-authored-by: tg123 <170430+tg123@users.noreply.github.com>
Done in 51b0afb. Restored the |
Agent-Logs-Url: https://github.com/tg123/sshpiper/sessions/85954bf7-55b1-442e-9298-cea30c6ef8df Co-authored-by: tg123 <170430+tg123@users.noreply.github.com>
|
@copilot resolve the merge conflicts in this pull request |
Co-authored-by: tg123 <170430+tg123@users.noreply.github.com>
Resolved in beb0759. Conflicts were in |
| binary: sshpiperd | ||
| hooks: | ||
| post: | ||
| - &replace_with_docker_bin 'scripts/goreleaser-replace-with-docker-bin.sh "{{ .Path }}" {{ .Os }} {{ .Arch }} "{{ .Name }}"' |
There was a problem hiding this comment.
@copilot help me understand, is there any other way to avoid replace after build
There was a problem hiding this comment.
Yes — the cleanest alternative is GoReleaser's builder: prebuilt mode. make docker-bins would still produce the linux binaries via the Dockerfile's bin-export stage, but for the linux entries we'd set:
builds:
- id: sshpiperd-linux
builder: prebuilt
goos: [linux]
goarch: [amd64, arm64]
prebuilt:
path: .docker-bins/{{ .Os }}_{{ .Arch }}/sshpiperd
binary: sshpiperd
- id: sshpiperd-cross # windows/darwin only, still compiled by goreleaser
goos: [windows, darwin]
...No go build runs for linux, so no replace step is needed. Trade-offs vs the current post-hook:
- Pro: no "build then overwrite" — goreleaser just packages what docker produced.
- Con: every binary (sshpiperd, each plugin, webadmin, admin…) has to be split into two build entries (linux-prebuilt + cross-compiled), roughly doubling the
builds:section, and the same split repeats inarchives.ids.
Two alternatives I considered but didn't pick:
- Pure reproducible builds (drop docker-bins, trust
-trimpath -s -w): fragile in practice — host Go (whatever's on the runner) vs container Go (pinnedgolang:1.26-bookworm) embed different build info, so sha256s drift unless every runner pins the exact same Go version. - Docker
COPYfromdist/(the previousEXTERNAL=1path): you asked me to drop it in the earlier review.
Happy to switch to builder: prebuilt if you prefer the doubled-up builds: section over the one-line post-hook — let me know.
There was a problem hiding this comment.
Tried it locally and hit a blocker: builder: prebuilt is a GoReleaser Pro–only feature, not in the OSS version this repo uses.
Reproduction with v2.15.4 (latest OSS):
$ goreleaser check
⨯ command failed
error=
│ yaml: unmarshal errors:
│ line 29: field prebuilt not found in type config.Build
│ line 36: field prebuilt not found in type config.Build
│ ...
A grep through the v2.15.4 module source confirms there's no prebuilt field on config.Build — the builder only exists in goreleaser-pro.
So the options are:
- Stay on the current post-hook (
scripts/goreleaser-replace-with-docker-bin.sh) — works on OSS. - Adopt GoReleaser Pro and switch to
builder: prebuilt— needs a Pro license for the project. - Drop docker-bins, rely on
-trimpath -s -wreproducibility — would need every CI runner to pin the exact same Go toolchain version as the Dockerfile (golang:1.26-bookworm) for sha256s to match.
Reverted my prebuilt experiment; the current .goreleaser.yaml post-hook is unchanged. Let me know which direction you'd like to take.
There was a problem hiding this comment.
@copilot help me understand how hard to remove goreleaser ? snap?
There was a problem hiding this comment.
Quick scoping — these are separable, so I'll cover them independently.
What GoReleaser does for this repo today
Looking at .goreleaser.yaml + release.yaml:
- Cross-compiles ~12 binaries (sshpiperd, 9 plugins, webadmin, admin) × {linux, windows, darwin} × {amd64, arm64}. Linux is already produced by the Dockerfile —
make docker-binsextracts it and the post-hook swaps it in. - Packages per-OS-arch archives (
tar.gzfor linux/darwin,zipfor windows) withallow_different_binary_count. - Generates
dist/checksums.txt. - Uploads everything to the GH release (the workflow then re-uploads via
gh release upload). - Builds & publishes the Snap (
snapcrafts:block).
Removing GoReleaser only (keeping Snap)
Effort: medium (~1 day). Mechanical but touches several files. Roughly:
- New
Makefiletargetsrelease-bins/release-archives/release-checksumsthat loopGOOS/GOARCHand callgo build -trimpath -ldflags "-s -w -X main.mainver=…"+tar/zip+sha256sum. For linux they'd just package the binariesmake docker-binsalready produces — no replace hook needed, which incidentally answers the original "avoid replace after build" question. release.yaml: drop the goreleaser action, callmake release-archives+make docker-push+gh release upload.- Snap would have to be built directly with
snapcraft pack(andsnapcraft upload), which means re-creating asnap/snapcraft.yamlby hand from the currentsnapcrafts:block (~60 lines, mostly the parts/apps/plugs already in.goreleaser.yaml). Doable, but the Snap CI step gets longer than what goreleaser-action gives you for free.
Net trade-off: ~150 lines of shell/Make replacing ~280 lines of yaml + a third-party action; gain full control + the ability to drop the prebuilt/replace hack; lose the snapcraft integration GoReleaser ships out of the box.
Removing Snap (independent of GoReleaser)
Effort: low (~1 hour), but it's a user-facing breaking change.
In code/build it's just:
- Drop
snapcrafts:from.goreleaser.yaml(or its equivalent if goreleaser is gone). - Delete
cmd/sshpiperd/snap/(launcher/,configgen/,hooks/, README — thesnaplauncherbuild entry too). - Drop the
Setup Snapstep +SNAPCRAFT_STORE_CREDENTIALSfromrelease.yaml. - Update root README to remove the Snap install instructions.
The real cost is product: sudo snap install sshpiperd stops working for existing users on the stable/beta channels. The Snap currently advertises itself prominently (badge in cmd/sshpiperd/snap/README.md). If you're OK breaking that, the code change is trivial.
Combined cheapest path
Drop Snap first (1 hr), then dropping GoReleaser becomes much easier (no snapcrafts: block to replace, no snapcraft CLI in CI) — closer to half a day for the goreleaser removal.
Happy to do either or both in this PR (or a follow-up) if you give the go-ahead. My recommendation, given the scope of this PR is already "Makefile + image-binary parity", would be a follow-up PR per concern so it's easy to review and easy to revert if needed.
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Drops GoReleaser in favor of a plain Make + Bash release pipeline, while preserving the Snap publishing flow that was previously driven by goreleaser's
snapcrafts:block.Release pipeline (
scripts/build-release.sh)make release-bins— linux binaries come from the Dockerfile'sbin-exportstage viamake docker-bins(byte-identical to the runtime image); windows/darwin are cross-compiled with-trimpath -ldflags "-s -w".make release-archives— preserves the legacysshpiperd_with_plugins_<os>_<arch>.{tar.gz,zip}naming so existing download URLs keep working; archives include README/LICENSE for parity with goreleaser's defaults.make release-checksums— writesdist/checksums.txtcovering archives and.snapfiles.Snap (
snap/snapcraft.yaml.in,snap/hooks/configure,scripts/build-snap.sh)make snap— for each arch, builds a per-arch prime dir from the same docker-extracted linux binaries plus a cross-compiledlauncher(withgo generaterunningconfiggenforconfigentry.txt), renders the snapcraft.yaml template, and runssnapcraft pack --destructive-mode.make snap-push— uploads viasnapcraft upload --release=<channels>usingSNAPCRAFT_STORE_CREDENTIALS.configurehook now lives at the snapcraft-conventionalsnap/hooks/configure(snapcraft auto-wraps it with the declaredplugs); the old copy undercmd/sshpiperd/snap/hooks/is removed.Workflows
release.yamlrewritten: dropsgoreleaser/goreleaser-action, runsmake release→make snap→ (conditionally)make snap-push→make docker-push. The snap-push step is gated onSNAPCRAFT_STORE_CREDENTIALSso forks without store access still succeed.test.yml: replaces thegoreleaser release --snapshot --cleansmoke test withmake release+make snap.Removed / updated
.goreleaser.yaml,scripts/goreleaser-replace-with-docker-bin.sh, and the duplicatecmd/sshpiperd/snap/hooks/configure.Dockerfile,Makefile,.github/copilot-instructions.md; expanded.gitignorefor snap build state (snap/prime-*,snap/snapcraft.yaml,snap/parts/,snap/stage/).Linux binaries continue to be the single source of truth produced by the Dockerfile, so the bytes shipped in the runtime image, the GH release archives, and the
.snapfiles remain identical.