Repository navigation
fix(stake): gasless deposit follow-ups before the flag goes on (stacked on #1558) #1539
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # HAL adversarial gate — prompt-injection / jailbreak probes against the real | |
| # `POST /api/v1/hal/evaluate` route, driven by promptfoo. | |
| # | |
| # THIS IS A NEW WORKFLOW, NOT AN EDIT TO ci.yml. Nothing here can change the verdict of the | |
| # existing type-check / unit / integration jobs. | |
| # | |
| # THREE OUTCOMES, NEVER TWO. `security/promptfoo/run-gate.mjs` exits: | |
| # 0 VERIFIED every tier ran and passed (and the failability canary failed as required) | |
| # 2 NOT_CHECKED the key-free tier passed; the live-quorum tier could not run | |
| # 1 FAILED a probe failed, the harness died, or the canary passed | |
| # The step below maps those back to the same three words and only fails the job on FAILED. | |
| # NOT_CHECKED is surfaced as a ::warning annotation and written to the job summary — a green | |
| # job here does NOT mean the live-quorum tier ran, and the summary always says which it was. | |
| # | |
| # WHY A NOT_CHECKED RUN IS STILL WORTH RUNNING: the key-free tier is not a no-op. It boots the | |
| # real route and asserts the injection screen, the 400 refusal path and the response contract, | |
| # so it fails for real on a fork with no secrets. And the run REFUSES to report a pass at all | |
| # until hal-failability-canary.yaml — a suite built to fail — has failed. | |
| # | |
| # TRIGGERS: `pull_request` is deliberately UNFILTERED, both by branch and by path. ci.yml | |
| # carries the same note for the same reason: a branch filter meant stacked PRs ran no tests | |
| # at all while looking checked, and a path filter would skip this gate on exactly the changes | |
| # most likely to move it (a refactor that stops calling scanForInjection touches neither the | |
| # guard nor the probes). | |
| # | |
| # NOT WIRED ON `push: main` YET — deliberate. The gate should accumulate a few PR runs before | |
| # it is allowed to redden the default branch. Adding `push: { branches: [main] }` here is the | |
| # intended follow-up, and is a one-line change. | |
| name: Security · HAL adversarial gate | |
| on: | |
| pull_request: | |
| workflow_dispatch: | |
| inputs: | |
| reason: | |
| description: 'Why this manual run — recorded on the run itself' | |
| required: false | |
| default: 'manual' | |
| schedule: | |
| # Weekly. Catches drift from merges that never touched a path a filter would have caught. | |
| - cron: '17 6 * * 1' | |
| permissions: | |
| contents: read | |
| jobs: | |
| hal-adversarial: | |
| name: HAL prompt-injection / jailbreak probes | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: actions/setup-node@v7 | |
| with: | |
| # 22, NOT 20, AND THIS IS THE WHOLE REASON THE GATE NEVER RAN. | |
| # promptfoo declares engines >=22.22.0 and refuses to start below it: | |
| # "promptfoo requires a supported Node.js runtime. Detected: v20.20.2 | |
| # Required: >=22.22.0" | |
| # On Node 20 every invocation is rejected before a single probe executes, so the gate | |
| # reported NOT_CHECKED honestly and stayed green while evaluating nothing. | |
| # | |
| # This is the ONLY job that needs 22. The app itself targets >=20.9.0 and Railway is | |
| # untouched by this line — it applies to this workflow's runner and nothing else. | |
| node-version: '22' | |
| - name: Install (legacy peer deps per nixpacks.toml) | |
| run: npm install --legacy-peer-deps | |
| # NO GLOBAL INSTALL — but NOT for the reason this comment used to give. | |
| # | |
| # It read: "the install succeeded, the binary was not on the PATH the child process | |
| # inherited, every invocation exited 127." The 127 was real; the explanation was a guess | |
| # made from the exit code alone, and it was wrong. The actual cause of BOTH that 127 and | |
| # the later npx exit 1 was the Node version pinned above — promptfoo refuses to run on | |
| # Node 20, so the global install had no working binary to place and npx downloaded the | |
| # package then bailed at the engine check. One cause, two symptoms, two wrong diagnoses. | |
| # | |
| # It took shipping instrumentation that prints the command's own stderr to find that, | |
| # after two attempts at guessing. The lesson is the exit code alone did not identify it. | |
| # | |
| # npx is still the right call: it needs no PATH and keeps the version pinned, so the | |
| # gate's verdict cannot drift without a commit here. | |
| # | |
| # run-gate.mjs falls back to `npx --yes promptfoo@<pinned>` when PROMPTFOO_BIN is unset, | |
| # which needs no PATH at all and keeps the version pinned, so the gate's verdict still | |
| # cannot drift without a commit here. Slower per run; a security gate that actually | |
| # executes is worth the download. | |
| - name: Run the gate | |
| id: gate | |
| env: | |
| PROMPTFOO_DISABLE_TELEMETRY: '1' | |
| PROMPTFOO_DISABLE_UPDATE: '1' | |
| # Absent on forks and on this repo until someone adds them — an empty value is | |
| # falsy to buildFactCheckProviders(), so the live-quorum tier simply does not run | |
| # and the gate reports NOT_CHECKED. These two are the providers the default | |
| # provider set enables without an extra HAL_S2_ENABLE_* flag. | |
| GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }} | |
| CEREBRAS_API_KEY: ${{ secrets.CEREBRAS_API_KEY }} | |
| run: | | |
| set +e | |
| node security/promptfoo/run-gate.mjs | |
| code=$? | |
| set -e | |
| echo "exit_code=$code" >> "$GITHUB_OUTPUT" | |
| case "$code" in | |
| 0) | |
| echo "::notice title=HAL adversarial gate::VERIFIED — every tier ran and passed" | |
| ;; | |
| 2) | |
| # DO NOT NAME THE REASON HERE. There are several ways to reach exit 2 — the | |
| # live-quorum tier lacking a key, promptfoo not being invocable, the canary | |
| # failing on infrastructure rather than an assertion — and this line cannot | |
| # tell them apart. It previously asserted "the key-free tier passed", and on | |
| # 022631b it printed exactly that directly beneath the runner's own | |
| # "promptfoo is not invocable — nothing was evaluated". A summary line that | |
| # contradicts the tool it is summarising is worse than no line at all. | |
| # The runner prints the real reason immediately above; point at it. | |
| echo "::warning title=HAL adversarial gate::NOT_CHECKED — this is not a pass. The gate output above this annotation names which tier did not run and why." | |
| ;; | |
| *) | |
| echo "::error title=HAL adversarial gate::FAILED (runner exit $code)" | |
| exit "$code" | |
| ;; | |
| esac | |
| - name: Upload promptfoo results | |
| if: always() | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: hal-adversarial-results | |
| path: security/promptfoo/.results/ | |
| if-no-files-found: warn | |
| retention-days: 14 |