Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/linux-kernel-publish.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# Lives in v9fs/test only (#15). Do not add a copy to v9fs/linux.
#
# Triggers: workflow_dispatch here, or `gh workflow run` from
# sync-torvalds-linux.yml. repository_dispatch (publish-kernel-image) is
# emergency/manual only — linux is mirror-only and must not fire it.
name: Publish Linux kernel (arm64 Image)

on:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ kernel/
tmp/
initrd.cpio
qemu.pid
.env
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ Tweak freely.

## Branching and change hygiene

- Do active work on `rework` unless told otherwise.
- Do active work on `main` unless told otherwise.
- Keep `README.md`, `CHANGES.md`, and `TODO.md` updated as changes land.
- Do not commit generated outputs (`logs/`, `kernel/`, `tmp/`, `initrd.cpio`, pid files).
- Do not commit generated outputs (`logs/`, `kernel/`, `tmp/`, `initrd.cpio`, pid files) or `.env` (local `GH_TOKEN`).

## Test philosophy
- Prefer **guest-direct execution** (Option A): run tests **inside the QEMU guest**.
Expand Down Expand Up @@ -38,8 +38,8 @@ Tweak freely.
- **Kernel publishing** workflow: builds `v9fs/linux` arm64 `Image` and publishes it.
- **Harness CI** workflows: download a published kernel `Image` and run tests.
- Publishing:
- Prefer a stable, `wget`-able GitHub Release asset `Image` tagged `kernel-main`, `kernel-nightly`, or `kernel-<version>`.
- GHCR is optional/secondary; keep it consistent if used.
- Prefer a stable, `wget`-able GitHub Release asset `Image` tagged `kernel-latest`, `kernel-main`, or `kernel-<version>`.
- Do not add workflows to `v9fs/linux`; sync and publish are owned by this repo.

## Logging and debuggability

Expand Down
2 changes: 2 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
## Unreleased

- Docs: README/TODO/AGENTS match live workflows (`ci.yml`, publish, sync); drop stale `demand.yml`/`nightly.yml` snapshot under `old/rework-take-1/` (#3). Header comment on `linux-kernel-publish.yml` records the test-repo-only trigger contract.
- Ignore local `.env` (`GH_TOKEN` for host-side `gh` / `act`); do not pass it into the guest.
- Instrument diod `t0010` first `access=<uid>` mount (stderr/dmesg/diod log), pin `version=9p2000.L`, pre-create export, `Defaults !requiretty`, and force a fresh patched build stamp so CI can triage residual non-root mount failures.
- Fix non-root `ACCESS_SINGLE` mounts across diod sharness: `scripts/v9fs-mount-9p` + trash on `/tmp`; t0010 runasuser now expected PASS including root-negatives.
- Run diod sharness as non-root user `v9fs` (so ACCESS_SINGLE / `--runas` negatives work); per-test `timeout` via automake `LOG_COMPILER` instead of one outer suite timeout.
Expand Down
171 changes: 39 additions & 132 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,168 +1,75 @@
# v9fs test harness (clean-slate)

This branch is being reworked from a clean slate.

The previous implementation snapshot is preserved at:

- Git tag: `rework-take-1`
- Directory: `old/rework-take-1/`
# v9fs test harness

## Current rebuild (take 2)
Test code and GitHub Actions for exercising the Linux **9p (v9fs)** client.

This repo now contains a **minimal 9p client smoke test harness**:
Kernel source under test is the git **mirror** [v9fs/linux](https://github.com/v9fs/linux). That repo is mirror-only: **no CI workflows there** ([#15](https://github.com/v9fs/test/issues/15)). All regression orchestration lives in **this** repository.

- QEMU runs an **arm64** Linux kernel `Image`
- QEMU exports a host directory via **virtio-9p**
- An initrd runs the smoke test **inside the guest** (no SSH)
- The test exercises basic filesystem operations on the 9p mount
Prior harness snapshot (not used by CI): tag `rework-take-1` / `old/rework-take-1/`.

### Local smoke run (Docker + QEMU)
## How tests run

1. Build the image:
Guest-direct: QEMU arm64 + virtio-9p export of the container root, initrd mounts `hostshare`, then a Debian chroot runs the suite **inside the guest** (no SSH).

```bash
docker pull ghcr.io/v9fs/docker:latest
```

1. Put a kernel `Image` at `./kernel/.build/arch/arm64/boot/Image`
2. Run the smoke test:

```bash
# Place arm64 Image at ./kernel/.build/arch/arm64/boot/Image
# or download with gh (optional gitignored .env with GH_TOKEN=...):
# set -a && source .env && set +a
# gh release download kernel-latest -p Image -D kernel/.build/arch/arm64/boot
make docker-smoke
```

Logs land under `./logs/<timestamp>/` (QEMU serial is `qemu.log`; guest writes `guest.log` + `guest.exitcode`).

# v9fs test harness

This repository contains **test code and scripts** for exercising the Linux **9p (v9fs)** filesystem.

The kernel source under test lives in the upstream repository:
Logs land under `./logs/<timestamp>/` (`qemu.log`, `guest.log`, `guest.exitcode`).

- `https://github.com/v9fs/linux`

## What lives here

- **CI workflows**: automation to build/run tests against a chosen kernel revision
- **Test code**: focused repros and regression tests for v9fs behavior
- **Scripts**: helpers to run locally and/or in CI

## How we work in this repo

- **Development branch**: all active work happens on `rework`
- **Change log**: keep `CHANGES.md` updated for every change set
- **Work tracking**: keep `TODO.md` updated as items are added/removed

## Quick start

This repo intentionally does *not* vendor the kernel source. Most workflows/scripts will:

1. Fetch `github.com/v9fs/linux` (or a fork/branch you specify)
2. Build the kernel (or use a provided artifact)
3. Run the tests in this repository against that kernel

### Local (macOS) via Docker + QEMU

Clone the kernel repo beside this repo:
Other suites (same Docker + Image):

```bash
git clone https://github.com/v9fs/linux ../linux
```

Build the test environment image:

```bash
docker build -t v9fs-test-env:local .
```

Build the kernel once (and export it as a reusable artifact):

```bash
mkdir -p ./tmp ./kernel
docker run --rm --privileged \
-v "$PWD:/home/v9fs-test/test" \
-v "$PWD/../linux:/workspaces/linux" \
-v "$PWD/kernel:/workspaces/kernel" \
-v "$PWD/tmp:/workspaces/tmp" \
-w /home/v9fs-test/test \
v9fs-test-env:local \
bash -lc "v9fs-build-kernel && v9fs-export-kernel /workspaces/linux /workspaces/linux/.build /workspaces/kernel"
```

Then run tests repeatedly without rebuilding the kernel:

```bash
mkdir -p ./tmp
docker run --rm --privileged \
docker run --rm --privileged --user 0:0 \
-e KERNELBUILD=/workspaces/kernel/.build \
-v "$PWD:/home/v9fs-test/test" \
-v "$PWD/kernel:/workspaces/kernel" \
-v "$PWD/tmp:/workspaces/tmp" \
-w /home/v9fs-test/test \
v9fs-test-env:local \
bash -lc "v9fs-run-tests short ci"
ghcr.io/v9fs/docker:latest \
bash -lc "./scripts/v9fs-run-tests smoke"
```

## How tests run (guest-direct)

`v9fs-run-tests` boots QEMU with an initrd that mounts the host-exported workspace
over **9p** and then runs the suite **inside the guest** (no SSH/port-forwarding).

The host-visible output is primarily the QEMU serial log:

- `logs/<timestamp>/qemu.log`

The guest mounts the repo at `/mnt/9/test` (see `scripts/v9fs-guest-run`).
Harness matrix suites: `smoke`, `fsx`, `postmark`, `dbench`, `diod-regression`, `qemu-9p2000.L`.

## GitHub Actions

CI uses the same Docker + QEMU flow as local development, but **kernel builds are
published separately** from the harness tests:

1. A dedicated workflow builds `v9fs/linux` and publishes the arm64 `Image` to
**GitHub Releases** (and GHCR).
2. The harness workflows download that published `Image` into `kernel/.build/arch/...`
and run `v9fs-run-tests ...` with `--privileged` so the harness can bind-mount a
stable 9p export root (`/workspaces/share`).
Kernel **build** and **test** are separate. Images are GitHub Release assets (`Image` on a `kernel-*` tag). GitHub-hosted runners (`ubuntu-24.04-arm` for build/test, `ubuntu-latest` for sync/report).

### Workflows
| Workflow | File | When |
| --- | --- | --- |
| **Sync torvalds/linux** | `.github/workflows/sync-torvalds-linux.yml` | Every 6h + manual. Fast-forwards `v9fs/linux` `upstream`, tracks new **v6+** tags, and can `gh workflow run` publish for the newest tag (`kernel-<tag>` + `kernel-latest`). |
| **Publish Linux kernel** | `.github/workflows/linux-kernel-publish.yml` | Manual `workflow_dispatch`, or `gh workflow run` from sync. Optional `repository_dispatch` (`publish-kernel-image`) for emergencies — **not** fired from `v9fs/linux`. Builds arm64 `Image` with 9p options enabled. |
| **Harness CI** | `.github/workflows/ci.yml` | Push, manual, or `workflow_run` after a successful publish. Downloads `kernel-latest` by default. |

There are no `demand.yml` or `nightly.yml` workflows. Those names were from the take-1 harness and only existed as a snapshot under `old/rework-take-1/` (removed).

| Workflow | File | When it runs |
| ------------------------ | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Publish Linux kernel** | `.github/workflows/linux-kernel-publish.yml` | `**repository_dispatch`** from `v9fs/linux` (recommended) or `**workflow_dispatch**` here. Builds arm64 `Image`, uploads it to a release tag you choose (`kernel-main`, `kernel-nightly`, `kernel-<version>`, …), and pushes a GHCR bundle tagged by the **linux commit SHA** plus your release tag. |
| **CI (push and manual)** | `.github/workflows/demand.yml` | On **every push** (all branches), **manual** dispatch, or `**workflow_call`**. Downloads `Image` from the `**kernel-main**` release by default (override via `kernel_release`). |
| **Mainline** | `.github/workflows/nightly.yml` | **Daily** schedule + **manual** dispatch. Downloads `Image` from `**kernel-nightly`** by default (override via `kernel_release`). |
Dashboard: wiki [Regression-Dashboard](https://github.com/v9fs/test/wiki/Regression-Dashboard). Per-run `results.json` / `diff-report.md` attach to the `kernel-latest` release when `V9FS_LINUX_SYNC_TOKEN` is set.

### Kernel Images

Because GitHub Actions cannot natively “watch” another repository’s pushes, the
`v9fs/linux` repo should call `repository_dispatch` on `v9fs/test` when branches change
(see the header comment in `linux-kernel-publish.yml` for an example payload).
| Release tag | Meaning |
| --- | --- |
| `kernel-latest` | Floating tip (newest published v6+ tag Image) |
| `kernel-v*` | Image for that kernel tag |
| `kernel-main` | `v9fs/linux` `upstream` (when published) |

### Kernel images as packages (GHCR)

Published kernels are also pushed to GitHub Packages (GHCR) as an OCI artifact:

- **Package**: `ghcr.io/v9fs/v9fs-test-kernel`
- **Tags**:
- `linux-<kernel_commit_sha>` (immutable)
- The **release tag** you passed (e.g. `kernel-main`, `kernel-nightly`, `kernel-6.12.0`)

```bash
oras pull ghcr.io/v9fs/v9fs-test-kernel:linux-<sha> -o .
tar -tzf kernel-image.tar.gz | head
```
Example: `https://github.com/v9fs/test/releases/download/kernel-latest/Image`

### Kernel images as direct downloads (GitHub Releases)
### Secrets

The publish workflow uploads the **arm64** kernel `Image` as a stable release asset:
- `V9FS_LINUX_SYNC_TOKEN` — Contents write on `v9fs/linux`, Actions + Contents on `v9fs/test` (sync, publish chaining, wiki, attach reports).

- **Rolling mainline**: `https://github.com/v9fs/test/releases/download/kernel-main/Image`
- **Rolling nightly**: `https://github.com/v9fs/test/releases/download/kernel-nightly/Image`
- **Versioned** (example): `https://github.com/v9fs/test/releases/download/kernel-6.12.0/Image`
Local host-side `gh` (release download, `act --secret-file .env`): put `GH_TOKEN=...` in a repo-root `.env` (gitignored). Do not pass that file into the QEMU/test container.

Log artifact names (avoid collisions when jobs run in parallel):
Do **not** add `.github` workflows to `v9fs/linux`.

- CI manual/push: `test-results-ci`, `test-results-latency`
- Nightly: `test-results-regression`, `test-results-latency`
## Repo hygiene

- Active work: `main` (via PRs). Keep `CHANGES.md` and `TODO.md` updated.
- Do not commit `logs/`, `kernel/`, `tmp/`, generated initrds, or `.env`.
- Guest-direct only. SSH/`cpu` helpers under `old/rework-take-1/` are unsupported.
22 changes: 10 additions & 12 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
## TODO

- Extend smoke test into a small suite (create/rename/unlink, fsync, directory traversal, large file IO).
- Add benchmark stages (fsx, postmark, dbench) to guest-direct CI.
- Add a diod regression suite stage (guest-direct) to exercise the kernel v9fs client.
- Decide whether to adopt a u-root/u-root+cpu initramfs for richer tooling distribution.
- Configure repo/org secret `V9FS_LINUX_SYNC_TOKEN` (Contents write on `v9fs/linux`, Actions + Contents on `v9fs/test` for publish chaining and wiki/`results.json` attach).
- After Image publish, harness `workflow_run` builds tip report + wiki summary table (`Regression-Dashboard`).
- Retarget “linux pushes trigger test” work (#6) to sync/orchestrate from this repo.

## TODO

- Decide whether to keep `ubuntu-latest` runners or switch back to self-hosted for KVM acceleration.
- Remove or clearly fence legacy SSH-based helpers (`test.bash`, `scripts/cpu`) if they are no longer part of the supported workflow.
- Configure/keep repo secret `V9FS_LINUX_SYNC_TOKEN` (Contents write on `v9fs/linux`, Actions + Contents on `v9fs/test` for publish chaining and wiki/`results.json` attach).
- Nightly/mainline/for-next/fixes Images and harness runs (#5, #6).
- QEMU serial / dmesg BUG+WARN scanning (#4).
- Legacy 9p2000 / 9p2000.u protocol cells (#10).
- pjdfstest / fstest (#7).
- Kconfig / cache-mode sweep (#8).
- Memory/ops metrics over time (#11).
- Dashboard history + optional bisection (#2).
- Shrink diod XFAIL: `t0011-v9fs-allsquash`, `t0013-v9fs-acl`.
- u-root/cpu for richer guest tooling (optional).
3 changes: 3 additions & 0 deletions old/rework-take-1/.github/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Take-1 GitHub Actions YAML used to live here (`demand.yml`, `nightly.yml`, `linux-kernel-publish.yml`). They are **not** active.

Live workflows: repository-root `.github/workflows/` (`ci.yml`, `linux-kernel-publish.yml`, `sync-torvalds-linux.yml`).
Loading
Loading