Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .changeset/turbo-android-native.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
'@gtbuchanan/pnpm-termux-shim': patch
'@gtbuchanan/cli': minor
---

Run the npm-distributed turbo on Termux instead of a Termux-packaged one.
turbo 2.10.8 publishes its `linux-arm64` binary under `os: ["android",
"linux"]`, so `node_modules/.bin/turbo` now starts on Android and the
`pkg install turbo` escape hatch in `gtb turbo` is gone.

In its place `gtb turbo` rewrites every PATH entry to an absolute path.
turbo resolves the package manager against PATH from the directory it was
invoked in and keeps the path that search produced, then runs each task
with that package's directory as the cwd — so a match from a relative
entry is re-interpreted against the child's directory and the spawn
fails. pnpm always prepends a relative `./node_modules/.bin`, which is
exactly where `@gtbuchanan/pnpm-termux-shim` installs its `pnpm`.
18 changes: 10 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ consumer repos. Authored in this repo; deployed locally via `gtb task
deploy:skills` for dogfooding.

- **`gtb-build-pipeline`** (`@gtbuchanan/cli`) — Turborepo task graph,
`gtb sync` / `gtb verify` / `gtb turbo` (with the Android/Termux
escape hatch), consumer script customization, test-bucket strategy,
aggregate semantics
`gtb sync` / `gtb verify` / `gtb turbo` (PATH normalization, plus
Android/Termux setup), consumer script customization, test-bucket
strategy, aggregate semantics
- **`gtb-eslint-config`** (`@gtbuchanan/eslint-config`) — `configure()`
API and options, pre-commit `createRequire` pattern, bundled plugin
set, suppression conventions, two-plugin Markdown lint split,
Expand Down Expand Up @@ -456,13 +456,15 @@ reasoning here so the question doesn't get re-litigated:
- Mise's only registered backend for turbo is `npm:turbo` (no aqua /
ubi / cargo fallback). Confirmed via `mise registry turbo`.
- Vercel ships turbo exclusively through npm. The latest release has
no binary assets on GitHub Releases;
no binary assets on GitHub Releases.
[vercel/turborepo#5616](https://github.com/vercel/turborepo/issues/5616)
(request for Android binaries) was closed as "not planned".
(request for Android binaries) was closed as "not planned", then
reversed by
[vercel/turborepo#12735](https://github.com/vercel/turborepo/pull/12735)
is the live PR re-litigating that decision — if it (or anything
like it) lands, upstream distribution shifts and this whole
trade-off changes.
in turbo 2.10.8 — but that shipped Android as an `os` widening of the
existing npm `@turbo/linux-arm64` package, not as standalone release
assets, so the distribution channel is unchanged and so is this
trade-off.
- mise's npm backend writes only `version` + `backend` to `mise.lock`
— no per-platform integrity. `pnpm-lock.yaml` records per-platform
integrity for `turbo` plus every `@turbo/<platform>-<arch>` optional
Expand Down
1 change: 0 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ disable_tools = ["node", "pnpm", "hk", "pkl", "actionlint"]
from source on Bionic, which doesn't compile. mise picks up system
node from `PATH`.
- **pnpm** — `npm i -g pnpm` (no `android/arm64` aqua asset).
- **turbo** — `pkg install turbo`.
- **hk** — download the static musl aarch64 release tarball
(`hk-aarch64-unknown-linux-musl.tar.gz`); it runs unmodified on
Bionic.
Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@
"valibot": "catalog:",
"vitest": "catalog:"
},
"optionalDependencies": {
"@gtbuchanan/pnpm-termux-shim": "catalog:"
},
"packageManager": "pnpm@11.17.0",
"engines": {
"node": ">=24.0.0",
Expand Down
60 changes: 36 additions & 24 deletions packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ This reconciles `turbo.json`, tsconfigs, `package.json` scripts, and
or changing capabilities.

Run the pipeline via the `gtb turbo` wrapper (generated root scripts
delegate to it so Android/Termux users get a transparent escape hatch):
delegate to it so its PATH normalization applies everywhere):

```sh
pnpm exec gtb turbo run check # compile → lint + test:fast (parallel)
Expand All @@ -37,30 +37,42 @@ single-package repo (where the root _is_ the package) gets none: a root
script named after an aggregate makes turbo re-enter itself, so the
`gtb turbo run` form above is the entry point there.

`gtb turbo` is a thin pass-through to `turbo` on every supported
platform. On Android, `process.platform === 'android'` causes the
node_modules launcher to refuse to start; the wrapper resolves the
native turbo from Termux's package registry and execs it directly.
Install it once per Termux environment:

```sh
pkg install turbo
`gtb turbo` runs `turbo` with one adjustment: every PATH entry is
rewritten to an absolute path first. Turbo resolves the package manager
binary against PATH from the directory it was invoked in, keeps the path
that search produced, then runs each task with that package's directory
as the cwd. A match from a relative entry is therefore a relative
program path, re-interpreted against the child's directory — so turbo
reports `unable to spawn child process` instead of falling through to a
later absolute entry. pnpm always prepends a relative
`./node_modules/.bin`, so any bin named after the package manager that
lives there is affected.

That is the layout on Termux/Android, where
[`@gtbuchanan/pnpm-termux-shim`](../pnpm-termux-shim) supplies a working
`pnpm`. Add the shim to your **workspace root** `package.json`
`optionalDependencies` (not inside any individual package — under pnpm
strict layout, only the root's `node_modules/.bin/` is on turbo's PATH
at spawn time):

```jsonc
{
"optionalDependencies": {
"@gtbuchanan/pnpm-termux-shim": "^0.1.1",
},
}
```

That puts a Bionic-built `turbo` at `$PREFIX/bin/turbo` (typically
`/data/data/com.termux/files/usr/bin/turbo`), which `gtb turbo`
resolves and execs.

The Termux-pkg turbo is Bionic-built, so its child-process spawns
honor Termux's `LD_PRELOAD` shebang rewriter and resolve
`#!/usr/bin/env <name>` correctly.
[`@gtbuchanan/pnpm-termux-shim`](../pnpm-termux-shim) is retained
defensively in case turbo reintroduces a glibc npm distribution, or
another glibc binary in the graph needs to spawn `pnpm`. Add it to
your **workspace root** `package.json` `optionalDependencies` (not
inside any individual package — under pnpm strict layout, only the
root's `node_modules/.bin/` is on turbo's PATH at spawn time). The
shim's `os: ["android"]` filter keeps it off non-Android hosts.
The shim's `os: ["android"]` filter keeps it off non-Android hosts,
where nothing named `pnpm` occupies `node_modules/.bin` and the
normalization is a no-op.

Turbo runs natively on Termux as of
[vercel/turborepo#12735](https://github.com/vercel/turborepo/pull/12735)
(turbo 2.10.8), which ships the `linux-arm64` binary under
`os: ["android", "linux"]`. Earlier versions require the Termux-packaged
turbo (`pkg install turbo`) instead — the npm launcher refuses to start
on Android and pnpm installs no platform binary.

The `prepare` script must be declared so pnpm runs it on install to
sync skills from installed packages:
Expand Down Expand Up @@ -92,7 +104,7 @@ generated `package.json` scripts (`"typecheck:ts": "gtb task typecheck:ts"`).
| --------- | ------------------------------------------------------- |
| `verify` | Validate generated config against workspace state |
| `sync` | Reconcile `turbo.json`, tsconfigs, scripts, codecov.yml |
| `turbo` | Run turbo (with an Android escape hatch) |
| `turbo` | Run turbo with cwd-independent PATH entries |
| `prepare` | Sync skills from installed packages |

### Task leaves (`gtb task <name>`)
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/e2e/turbo-generate.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { describe, it } from 'vitest';
* turbo.json snapshot: turbo aborts the whole run when a task in `dependsOn`
* resolves to no definition, and silently restores nothing when a cached
* task declares no outputs. `gtb turbo` is used rather than turbo directly
* so the Android/Termux escape hatch applies.
* so its PATH normalization applies.
*/

const jsonIndent = 2;
Expand Down
37 changes: 19 additions & 18 deletions packages/cli/skills/gtb-build-pipeline/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: gtb-build-pipeline
description: Build pipeline guidance for projects using @gtbuchanan/cli. Covers the Turborepo task graph, gtb sync and verify (including scoped runs), the gtb hk pre-commit runner, the gtb turbo wrapper (with the Android/Termux escape hatch), consumer script customization, and test-bucket strategy. Trigger keywords - @gtbuchanan/cli, @gtbuchanan/pnpm-termux-shim, turbo.json, gtb sync, gtb sync mise, gtb verify, gtb verify mise, gtb turbo, gtb task, gtb hk, hk:all, hk:base, mise.tasks.toml, compile:ts, pack:npm, deploy:skills, task graph.
description: Build pipeline guidance for projects using @gtbuchanan/cli. Covers the Turborepo task graph, gtb sync and verify (including scoped runs), the gtb hk pre-commit runner, the gtb turbo wrapper (PATH normalization, plus Android/Termux setup), consumer script customization, and test-bucket strategy. Trigger keywords - @gtbuchanan/cli, @gtbuchanan/pnpm-termux-shim, turbo.json, gtb sync, gtb sync mise, gtb verify, gtb verify mise, gtb turbo, gtb task, gtb hk, hk:all, hk:base, mise.tasks.toml, compile:ts, pack:npm, deploy:skills, task graph, unable to spawn child process, turbo on Termux.
---

# @gtbuchanan/cli build pipeline
Expand Down Expand Up @@ -46,11 +46,14 @@ scripts are unaffected: they call `gtb task <name>` and never turbo.
`gtb verify` reports any root script that shadows an aggregate, so a
repo synced before this rule landed is told which scripts to delete.

`gtb turbo` is a thin pass-through to `turbo` on every supported
platform. On Android (`process.platform === 'android'`) it resolves
the global turbo binary installed via Termux's package registry
(`$PREFIX/bin/turbo`) and execs it directly, bypassing the
node_modules launcher (which rejects the platform upfront). See
`gtb turbo` runs `turbo` with every PATH entry rewritten to an absolute
path first. Turbo resolves the package manager against PATH from the
directory it was invoked in and keeps the path that search produced,
then runs each task with that package's directory as the cwd — so a
match from a relative entry is re-interpreted against the child's
directory and the spawn fails. pnpm always prepends a relative
`./node_modules/.bin`, which matters wherever a bin named after the
package manager lives there. See
[Android-Termux setup](#android-termux-setup) below.

`pnpm verify`, `pnpm prepare`, and `pnpm run gtb <cmd>` invoke the CLI directly.
Expand Down Expand Up @@ -155,30 +158,28 @@ Invoked via mise (`mise run hk:base`) so hk and its tools resolve from mise. The

## Android-Termux setup

Two issues are caused by Termux's Node reporting `process.platform === 'android'`; a third (memory pressure) is unrelated and applies to any low-memory host. Native Android support upstream was declined in [vercel/turborepo#5616](https://github.com/vercel/turborepo/issues/5616), so `gtb turbo` ships the workaround instead.
Two issues are specific to Termux — getting a turbo binary that runs, and getting it to spawn `pnpm`; a third (memory pressure) is unrelated and applies to any low-memory host.

**1. Node_modules launcher rejects android.** The launcher in `node_modules/.bin/turbo` exits early when `process.platform === 'android'`, and pnpm filters `@turbo/<os>-<arch>` optional dependencies by host platform so none of the bundled platform binaries are installed either. Install the native turbo from Termux's package registry instead:
**1. Getting a turbo that runs.** Turbo 2.10.8 ships the `linux-arm64` binary under `os: ["android", "linux"]` ([vercel/turborepo#12735](https://github.com/vercel/turborepo/pull/12735), reversing the 2023 decline in [#5616](https://github.com/vercel/turborepo/issues/5616)), so pnpm installs a platform binary on Termux and the `node_modules/.bin/turbo` launcher starts normally. Nothing special is required beyond depending on turbo `^2.10.8`.

```sh
pkg install turbo
```

That puts a Bionic-built `turbo` at `$PREFIX/bin/turbo` (typically `/data/data/com.termux/files/usr/bin/turbo`). `gtb turbo` resolves it directly via `$PREFIX` (with the standard prefix as fallback) and execs it, bypassing the node_modules launcher entirely.
Below that version the launcher exits early on `process.platform === 'android'` and pnpm installs no `@turbo/<os>-<arch>` binary at all; the workaround was a Termux-packaged turbo (`pkg install turbo`) invoked directly. If you meet a repo pinned to an older turbo, bump it rather than reviving the escape hatch.

**2. Turbo child-process spawn ENOENT (historically).** The npm-distributed Linux turbo binary is glibc-built, but Termux is Bionic. Termux's `LD_PRELOAD=libtermux-exec-ld-preload.so` rewrites `/usr/bin/env` shebangs in `execve` syscalls — but the preload is Bionic-only, so it never loads into a glibc turbo. When such a turbo spawns `pnpm`, the kernel sees `#!/usr/bin/env node` and fails because Termux has no `/usr/bin/env`.
**2. Turbo child-process spawn ENOENT.** Termux has no `/usr/bin/env`, so a binary that `execve`s `pnpm` hits its literal `#!/usr/bin/env node` shebang and fails. Termux's `LD_PRELOAD=libtermux-exec-ld-preload.so` rewrites those shebangs, but `LD_PRELOAD` is a dynamic-loader feature and the npm-distributed turbo is statically linked (no `PT_INTERP`), so the rewriter never reaches it.

The Termux-pkg turbo is Bionic-built, so the preload loads correctly and child-process spawns resolve `pnpm` without issue. `@gtbuchanan/pnpm-termux-shim` is retained defensively in case turbo reintroduces a glibc npm distribution, or another glibc binary in the graph needs to spawn `pnpm`. The shim is an `os: ["android"]`-filtered package whose `bin: { pnpm: ... }` entry has an absolute-path shebang; pnpm symlinks it into `<rootDir>/node_modules/.bin/pnpm` ahead of the system `pnpm` in PATH. On non-Android hosts it's filtered out at install — zero footprint.

Add it as an `optionalDependencies` entry on the workspace root (so the bin lands in the root's `node_modules/.bin`, not nested under a transitive dep):
The fix is `@gtbuchanan/pnpm-termux-shim`, an `os: ["android"]`-filtered package whose `bin: { pnpm: ... }` entry has an absolute-path shebang. Add it to the **workspace root** `optionalDependencies` — under pnpm's strict layout only the root's `node_modules/.bin/` is on turbo's PATH at spawn time:

```jsonc
{
"optionalDependencies": {
"@gtbuchanan/pnpm-termux-shim": "^0.1.0",
"@gtbuchanan/pnpm-termux-shim": "^0.1.1",
},
}
```

On non-Android hosts it's filtered out at install — zero footprint.

Installing the shim is what makes `gtb turbo`'s PATH normalization load-bearing: it puts a `pnpm` inside the relative `./node_modules/.bin` entry pnpm prepends, and turbo carries that relative match into each task's package directory, where it no longer resolves. Symptom is `unable to spawn child process: No such file or directory (os error 2)` on every non-root task. Running turbo through `gtb turbo` (which every generated root script does) resolves it; invoking `turbo` directly from a `pnpm run` script reintroduces it.

**3. Memory-bound concurrency for heavy aggregates.** Unrelated to `process.platform`: phones typically have 2–4GB free RAM under load. Turbo's default `--concurrency=10` is fine for `check` (typecheck + lint + fast tests fan out narrowly under the dependency graph). It is **not** fine for `build`, `test:slow`, or `test:e2e`, which fork their own vitest worker pools per task — `--concurrency=2` already crashed the OS in measurement. Run heavy aggregates with `--concurrency=1` on memory-constrained devices:

```sh
Expand Down
17 changes: 9 additions & 8 deletions packages/cli/skills/gtb-build-pipeline/evals/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,22 @@
},
{
"expectations": [
"Recommends running `pkg install turbo` to install the native turbo from the Termux package registry",
"Explains that the node_modules launcher rejects `process.platform === 'android'` upfront, and pnpm filters `@turbo/<os>-<arch>` optional deps by host platform so none of the bundled platform binaries are installed either",
"Notes that `gtb turbo` resolves `$PREFIX/bin/turbo` (with `/data/data/com.termux/files/usr` as the standard fallback) and execs it directly, bypassing the launcher"
"Says `pkg install turbo` is no longer needed — turbo 2.10.8 publishes the linux-arm64 binary under `os: [\"android\", \"linux\"]`, so pnpm installs it and the node_modules launcher runs on Termux",
"Tells me to depend on turbo `^2.10.8` (bump rather than keep a Termux-packaged turbo)",
"Notes the npm turbo still needs `@gtbuchanan/pnpm-termux-shim` on the workspace root to spawn `pnpm`"
],
"expected_output": "Activates skill. Tells me to run `pkg install turbo` to install the native turbo from the Termux package registry. Explains that the node_modules launcher rejects `process.platform === 'android'` upfront, and pnpm filters `@turbo/<os>-<arch>` optional deps by host platform so none of the bundled binaries are installed either. After install the Bionic turbo lands at `$PREFIX/bin/turbo` and `gtb turbo` resolves it directly (honoring `$PREFIX`, with `/data/data/com.termux/files/usr` as fallback) and execs it, bypassing the launcher.",
"expected_output": "Activates skill. Explains that `pkg install turbo` is obsolete: turbo 2.10.8 (vercel/turborepo#12735) ships the `linux-arm64` binary under `os: [\"android\", \"linux\"]`, so pnpm installs a platform binary on Termux and `node_modules/.bin/turbo` starts normally — depend on turbo `^2.10.8`. Notes that the npm-distributed turbo still can't spawn Termux's `pnpm` on its own, so `@gtbuchanan/pnpm-termux-shim` belongs in the workspace root `optionalDependencies`.",
"files": [],
"id": 2,
"prompt": "When I run `gtb turbo run build` on Termux it says `the global turbo binary is not installed`. How do I fix that?"
"prompt": "I set up this repo on Termux a while back with `pkg install turbo`. Do I still need that, or can I just use the turbo from node_modules now?"
},
{
"expectations": [
"Identifies the cause: glibc turbo's `#!/usr/bin/env node` shebang fails because Termux's LD_PRELOAD shebang shim is Bionic-only and never loads into the glibc turbo, while /usr/bin/env doesn't exist in Termux",
"Recommends adding `@gtbuchanan/pnpm-termux-shim` as an `optionalDependencies` entry on the workspace root (not a sub-package)"
"Identifies the cause: Termux has no `/usr/bin/env`, and the `LD_PRELOAD` shebang rewriter never reaches the statically linked npm turbo (no dynamic loader involved), so `#!/usr/bin/env node` fails",
"Recommends adding `@gtbuchanan/pnpm-termux-shim` as an `optionalDependencies` entry on the workspace root (not a sub-package)",
"Notes that turbo must be invoked through `gtb turbo`, whose PATH normalization keeps the shim reachable once it occupies pnpm's relative `./node_modules/.bin` entry"
],
"expected_output": "Activates skill. Explains that the Linux turbo binary is glibc-built but Termux is Bionic; Termux's `LD_PRELOAD=libtermux-exec-ld-preload.so` rewrites `/usr/bin/env` shebangs in `execve` syscalls, but the preload is Bionic-only and doesn't load into glibc turbo, so the kernel sees `#!/usr/bin/env node` and fails because `/usr/bin/env` doesn't exist. Fix: add `@gtbuchanan/pnpm-termux-shim` as an `optionalDependencies` entry on the workspace root so its absolute-path `bin/pnpm` lands in `<rootDir>/node_modules/.bin/pnpm` ahead of the system pnpm.",
"expected_output": "Activates skill. Explains that Termux has no `/usr/bin/env`, so pnpm's `#!/usr/bin/env node` shebang fails on `execve`; Termux's `LD_PRELOAD=libtermux-exec-ld-preload.so` normally rewrites those, but it's a dynamic-loader feature and the npm turbo is statically linked, so it never loads. Fix: add `@gtbuchanan/pnpm-termux-shim` as an `optionalDependencies` entry on the workspace root so its absolute-path `bin/pnpm` lands in `<rootDir>/node_modules/.bin/pnpm`, and invoke turbo through `gtb turbo` so PATH entries are absolute — otherwise turbo matches the shim via pnpm's relative `./node_modules/.bin` and loses it when each task spawns from its own package directory.",
"files": [],
"id": 3,
"prompt": "On Termux turbo fails to spawn pnpm with ENOENT. What's going on and how do I fix it?"
Expand Down
Loading