Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,11 @@ jobs:
shell: bash
run: |
mkdir -p build
python3 tools/test_boot_memory.py --case "${{ matrix.case }}" 2>&1 | tee build/boot-memory-run.log
if [ "${{ matrix.case }}" = loader-version ]; then
python3 tools/test_boot_memory.py --loader-admission 2>&1 | tee build/boot-memory-run.log

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Give the grouped loader run an aggregate timeout

When several loader cases hang, this command cannot fulfill the new guarantee that all later cases are attempted and their manifests retained: each of the eight cases can spend the default 120 seconds in boot() (and up to 600 seconds building), while the entire boot-memory-admission job at line 324 is capped at 15 minutes, including dependency installation. Eight boot timeouts alone require 16 minutes, so GitHub Actions will cancel the job before the loop and artifact-upload step finish. Increase the job timeout or assign bounded per-case timeouts that fit the aggregate budget.

Useful? React with 👍 / 👎.

else
python3 tools/test_boot_memory.py --case "${{ matrix.case }}" 2>&1 | tee build/boot-memory-run.log
fi
- uses: actions/upload-artifact@v7
if: always()
with:
Expand Down
15 changes: 14 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ Review unsafe source changes with `python3 tools/check_unsafe.py --diff`, then
explicitly regenerate the baseline with `--write-baseline` and check it with
`--check`. A baseline is retained context, not a safety approval. The full QEMU
suite and the exception, BSP and boot-memory harnesses require committed source
for exact evidence. `python3 tools/test_boot_memory.py` runs all six malformed
for exact evidence. `python3 tools/test_boot_memory.py` runs all thirteen loader/map and malformed
handoff cases; `--case loader-version` selects one. Its host parser/fixture tests
are included by `python3 -m unittest discover -s tools -p 'test_*.py'`. The local CI
configuration assigns each boot-memory case an independent matrix job and retains
Expand Down Expand Up @@ -302,3 +302,16 @@ independent real fault, NMI, machine-check and return-state cases; `--case irq-p
selects one. It retains QMP transcripts as well as source/image/fault evidence.
Nested termination, first kernel fatal handling and software process denial are
distinct proof outcomes and cannot substitute for one another.

The loader decoder group can be run independently with
`python3 tools/test_boot_memory.py --loader-admission`. It covers descriptor
version/stride/length, empty/capacity, range/alignment and overlap. Each case
requires its exact expected decoder error before kernel entry. Cases use bounded
stack-backed fixture bytes after the real firmware exit and never allocate or
call boot services there. The runner retains every case's manifest, including
failures, and fails the overall group after attempting remaining cases.
The existing `Boot memory admission (loader-version)` CI context now runs this
entire group; the five other contexts keep their separate handoff cases. Required
context names and repository protection policy are unchanged. These decoder
fixtures do not emulate firmware map growth, stale-key retries or allocation
failures; those remain F1.1 qualification work.
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ must not deepen a known ownership or isolation violation.
- [x] The [kernel ELF load plan](docs/KERNEL_ELF.md) validates the whole image before allocation/copy; six malformed-image VM cases prove rejection before kernel entry and explicit QMP guest shutdown.
- [x] Seven kernel stacks have fourteen inaccessible guards with boot-time mapping readback; four selected CPU faults prove exact guard addresses and deliberate containment.
- [ ] **F1.1:** validate complete BootInfo and firmware maps: descriptor count/stride/version, checked ranges, map growth, stale exit keys, overlap and reserved kernel/firmware/device memory. Capacity exhaustion must not silently truncate.
Map/handoff checks are implemented; the pinned UEFI helper supplies one stale-key retry. A [bounded kernel ELF load plan](docs/KERNEL_ELF.md) now validates the complete image before allocation/copy. Exhaustive map-growth/exit fault injection and firmware allocation-failure qualification remain open.
Map/handoff checks are implemented; the pinned UEFI helper supplies one stale-key retry. A [bounded kernel ELF load plan](docs/KERNEL_ELF.md) now validates the complete image before allocation/copy. Thirteen real loader/handoff cases now include exact stride, length, empty-map, capacity, range, alignment and overlap decoder errors, with no kernel entry for loader rejection. Exhaustive firmware map-growth/exit fault injection and firmware allocation-failure qualification remain open.
- [ ] **F1.2:** inventory all process-visible register state. Implement a bounded, CPUID-validated eager XSTATE save/restore policy, with a justified narrower fallback; define initial state and kernel FPU/SIMD use. Test every enabled component through preemption, syscall, fault and reuse.
The fixed 512-byte FXSAVE64 fallback now accompanies private DS/ES/FS/GS selectors, fixed zero bases, disabled debug/fast-entry policy and a [bounded register inventory](docs/CPU_STATE.md). Pending unmasked x87 state and user/kernel #MF have local proofs. Strict user/kernel #XM fixtures fail on pinned TCG: the instruction records MXCSR invalid status but does not trap. Exact results and retained failures are in [VERIFICATION.md](docs/VERIFICATION.md). Nonzero TLS/debugger APIs, larger CPU profiles, exhaustive FP/flag interactions and physical qualification remain open; no complete F1.2 claim.
- [x] **F1.3:** give boot, privilege, interrupt and emergency stacks inaccessible guards and usage measurements. Overflow must reach controlled containment instead of adjacent corruption or unexplained reset.
Expand Down
19 changes: 19 additions & 0 deletions docs/MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,22 @@ execution through its NX identity alias. Exact CR2/error bits, prior sealing,
deliberate containment, clean source and retained failed/successful artifacts are
required. The ordinary six CPU protection cases and full lifecycle suite remain
required; these new probes do not replace them.

## Loader map rejection qualification

The immutable-source boot harness now has eight loader decoder cases and five
kernel handoff cases. Run `python3 tools/test_boot_memory.py --loader-admission`
for the loader group, or `--case loader-capacity` for one case. The new fixtures
replace only the bounded input bytes/stride of `decode_uefi_map`, after the real
firmware exit. An exact error check precedes the case-tagged terminal record;
wrong errors, premature input rejection, missing/duplicate/wrong-case tags,
wrong rejection phase and kernel initialization fail admission.

The fixed stack fixtures allocate no memory or firmware resources after exit.
The over-capacity case supplies 257 descriptors to the production 256-entry
admission path; it must reject rather than truncate. Range, alignment and overlap
cases exercise the production map validator through the real loader. All thirteen
cases passed on clean `a2e810e`; source, fixture/image identities, serial logs and
manifests are in [verification](VERIFICATION.md#october-13-loader-map-rejection-qualification).
This is distribution VM portability evidence, not exhaustive firmware lifecycle
or frozen-reference qualification. F1.1 remains open.
46 changes: 46 additions & 0 deletions docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -2273,3 +2273,49 @@ original checklist to 53/100 (42/60 foundation entries). Full deferred syscall
completion/user-copy, device/DMA/reset interleavings, F6.4 fault schedules,
24/72-hour mixed pressure, independent review and the remaining roadmap gates
stay open. GenOS remains Experimental.


## October 13 loader map rejection qualification

Clean source `a2e810e6b5d93175891aae6aa82411852a688445` extends the immutable-source
boot-memory harness from six to thirteen cases. Seven new cases replace only
bounded decoder inputs after the real `ExitBootServices`: descriptor stride and
length, empty map, 257-descriptor capacity, page-count overflow, alignment and
overlap. The production decoder/validator and terminal rejection path remain
unchanged. Each fixture checks the exact `BootMemoryError` before its case-tagged
record and the normal loader rejection. An early firmware-buffer mismatch takes
a separate rejected path and cannot emit the expected proof tag.

All eight loader cases passed, without kernel entry or allocator/architecture
initialization, under `build/boot-memory-evidence/1791892177220255249/`. Each
retains its fixture patch, environment, image/serial identities and success
manifest. The five existing kernel handoff cases also passed on the same source:

| Case | Evidence run |
| --- | --- |
| count | `1791892374060422659` |
| overlap | `1791892388109776106` |
| initrd-usable | `1791892401578991067` |
| kernel-usable | `1791892415619555649` |
| cmdline | `1791892430301489234` |

The six harness tests reject missing/duplicated/wrong-case/error/phase evidence,
verify each fixture keeps the actual exit and decoder without post-exit allocation,
and inject a first-case timeout to require all later cases plus a failing overall
result. All 177 Python tests pass. CI's existing loader-version context now runs
the loader group, with separate retained per-case results; the other five context
names and required-check policy are unchanged.

This is QEMU 10.0.13/distribution OVMF portability qualification. Synthetic
post-exit decoder bytes are not firmware map-growth, stale-key retry or
allocation-failure injection. Those requirements remain open; F1.1 and the
53/100 checklist count do not advance because of this slice.

Separately, PR #11's source `9ca892072d85a25c24f0463f20a1740c4c02fa9c` passed all
83 required checks. Complete check-run API readback was matched against the
versioned contract using `audit_execution(..., require_success=True)`, rejecting
wrong head/app, missing/duplicate/unexecuted lanes. The report is retained at
`build/roadmap-evidence/pr11-check-runs.json`. All three workflow runs succeeded,
including hardware CPU and full network tests. This is remote execution evidence
for PR #11, not an independent reference-review sign-off or a CI result for the
later boot-map change.
82 changes: 73 additions & 9 deletions tools/test_boot_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,63 @@
from test_exception_entry import firmware_path, replace_once, require_clean_source
from reference_vm import environment, load_profile, qemu_args

CASES = ("loader-version", "count", "overlap", "initrd-usable", "kernel-usable", "cmdline")
LOADER_ERRORS = {
"loader-stride": "DescriptorStride",
"loader-length": "DescriptorLength",
"loader-empty": "Empty",
"loader-capacity": "Capacity",
"loader-range": "Range",
"loader-alignment": "Alignment",
"loader-overlap": "Overlap",
}
LOADER_CASES = ("loader-version", *LOADER_ERRORS)
CASES = (*LOADER_CASES, "count", "overlap", "initrd-usable", "kernel-usable", "cmdline")


def loader_error_record(case: str) -> str:
return f"BOOT_MEMORY_FIXTURE_ERROR case={case} error={LOADER_ERRORS[case]}"


def patch_fixture(root: Path, case: str) -> str:
path = root / "bootloader/src/main.rs"
before = path.read_text()
if case == "loader-version":
after = replace_once(before, "meta.desc_size, meta.desc_version", "meta.desc_size, 0xffff")
elif case in LOADER_ERRORS:
# Bounded stack-backed bytes replace only decoder inputs. No allocation,
# firmware call or boot service can occur after ExitBootServices.
length = {"loader-length": 39, "loader-empty": 0,
"loader-capacity": 40 * 257, "loader-overlap": 80}.get(case, 40)
setup = f" let mut fixture = [0u8; {length}];\n"
if case in ("loader-range", "loader-alignment", "loader-overlap"):
start = 1 if case == "loader-alignment" else 4096
pages = "u64::MAX" if case == "loader-range" else "1u64"
setup += (
" fixture[..4].copy_from_slice(&7u32.to_le_bytes());\n"
f" fixture[8..16].copy_from_slice(&{start}u64.to_le_bytes());\n"
f" fixture[24..32].copy_from_slice(&{pages}.to_le_bytes());\n"
)
if case == "loader-overlap":
setup += " let first = fixture[..40].try_into().unwrap();\n let first: [u8; 40] = first;\n fixture[40..].copy_from_slice(&first);\n"
else:
setup = setup.replace("let mut fixture", "let fixture")
stride = 39 if case == "loader-stride" else 40
expected = LOADER_ERRORS[case]
anchor = " boot_info.memory_map = boot_memory::decode_uefi_map(bytes, meta.desc_size, meta.desc_version)?;"
replacement = (
" let _ = bytes;\n" + setup +
f" let result = boot_memory::decode_uefi_map(&fixture, {stride}, 1);\n"
f" if result.as_ref().err() != Some(&boot_memory::BootMemoryError::{expected}) {{\n"
" halt_after_exit(b\"BOOT_MEMORY_FIXTURE_WRONG_ERROR\\r\\n\");\n"
" }\n boot_info.memory_map = result?;"
)
after = replace_once(before, anchor, replacement)
after = replace_once(after,
".ok_or(boot_memory::BootMemoryError::DescriptorLength)?;",
'.unwrap_or_else(|| halt_after_exit(b"BOOT_MEMORY_FIXTURE_INPUT_REJECTED\\r\\n"));')
rejection = 'halt_after_exit(b"BOOT_MEMORY_MAP_REJECTED\\r\\n");'
tagged = f'halt_after_exit(b"{loader_error_record(case)}\\r\\nBOOT_MEMORY_MAP_REJECTED\\r\\n");'
after = replace_once(after, rejection, tagged)
else:
mutations = {
"count": "boot_info.memory_map.region_count = 257;",
Expand All @@ -47,15 +96,22 @@ def patch_fixture(root: Path, case: str) -> str:


def validate_log(log: str, case: str) -> None:
if case not in CASES:
raise ValueError(f"unknown boot memory case: {case}")
lines = log.replace("\r", "").splitlines()
marker = "BOOT_MEMORY_MAP_REJECTED" if case == "loader-version" else "BOOT_INFO_REJECTED"
marker = "BOOT_MEMORY_MAP_REJECTED" if case in LOADER_CASES else "BOOT_INFO_REJECTED"
if lines.count(marker) != 1:
raise ValueError("missing or duplicated exact boot rejection")
if lines.count("GenOS kernel entered") != (0 if case == "loader-version" else 1):
if lines.count("GenOS kernel entered") != (0 if case in LOADER_CASES else 1):
raise ValueError("wrong rejection phase")
records = [line for line in lines if "BOOT_MEMORY_FIXTURE_ERROR" in line]
expected = [loader_error_record(case)] if case in LOADER_ERRORS else []
if records != expected or (expected and lines.index(expected[0]) >= lines.index(marker)):
raise ValueError("missing, duplicated, wrong-case or reordered decoder error")
for forbidden in ("BOOT_MEMORY_MAP_VALIDATED", "FRAME_ALLOCATOR_BITMAP_READY",
"GDT/TSS initialized", "GENOS_READY", "NORMAL_SHELL_READY",
"EXCEPTION_FRAME", "KERNEL PANIC", "BOOTLOADER_PANIC"):
"EXCEPTION_FRAME", "KERNEL PANIC", "BOOTLOADER_PANIC",
"BOOT_MEMORY_FIXTURE_WRONG_ERROR", "BOOT_MEMORY_FIXTURE_INPUT_REJECTED"):
if any(forbidden in line for line in lines):
raise ValueError(f"rejected handoff continued or faulted: {forbidden}")

Expand All @@ -68,7 +124,7 @@ def boot(root: Path, evidence: Path, case: str, timeout: int) -> None:
"-drive", load_profile(root)["boot_drive"] + ",file=build/genos.img", "-net", "none",
"-display", "none", "-monitor", "none", "-serial", f"file:{log_path}", "-no-reboot"]
(evidence / "qemu-command.json").write_text(json.dumps(args, indent=2) + "\n")
marker = "BOOT_MEMORY_MAP_REJECTED" if case == "loader-version" else "BOOT_INFO_REJECTED"
marker = "BOOT_MEMORY_MAP_REJECTED" if case in LOADER_CASES else "BOOT_INFO_REJECTED"
with (evidence / "qemu.log").open("w") as output:
process = subprocess.Popen(args, cwd=root, stdout=output, stderr=subprocess.STDOUT)
try:
Expand Down Expand Up @@ -96,7 +152,10 @@ def boot(root: Path, evidence: Path, case: str, timeout: int) -> None:

def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--case", choices=CASES)
selection = parser.add_mutually_exclusive_group()
selection.add_argument("--case", choices=CASES)
selection.add_argument("--loader-admission", action="store_true",
help="run every loader decoder case, retaining all failures")
parser.add_argument("--timeout", type=int, default=120)
options = parser.parse_args()
if sys.version_info < (3, 12) or options.timeout <= 0:
Expand All @@ -106,7 +165,9 @@ def main() -> None:
archive = subprocess.run(["git", "archive", "--format=tar", commit], cwd=source,
stdout=subprocess.PIPE, check=True, timeout=30).stdout
run = source / "build/boot-memory-evidence" / str(time.time_ns())
for case in ([options.case] if options.case else CASES):
cases = [options.case] if options.case else LOADER_CASES if options.loader_admission else CASES
failures = []
for case in cases:
evidence = run / case
evidence.mkdir(parents=True, exist_ok=False)
manifest = {"case": case, "status": "incomplete", "commit": commit, "source_clean": True}
Expand All @@ -130,13 +191,16 @@ def main() -> None:
except Exception as error:
manifest["status"] = "failed"
manifest["failure"] = str(error)
raise
failures.append((case, str(error)))
finally:
serial = evidence / "serial.log"
if serial.exists():
manifest["serial_sha256"] = hashlib.sha256(serial.read_bytes()).hexdigest()
(evidence / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
print(f"BOOT_MEMORY_PROBE_OK case={case} evidence={evidence}", flush=True)
outcome = "OK" if manifest["status"] == "passed" else "FAILED"
print(f"BOOT_MEMORY_PROBE_{outcome} case={case} evidence={evidence}", flush=True)
if failures:
raise RuntimeError(f"boot memory qualification failed: {failures}")


if __name__ == "__main__":
Expand Down
Loading
Loading