resize: EVE-patterned resume tests and SIGKILL chaos harness

[eriknordmark]

Mostly tests + instrumentation (a build-tagged, no-op-by-default backend hook
used only by the chaos test), plus one block-device production fix in run.go:
open the whole disk non-exclusively so child-partition e2fsck/resize2fs can
run, and accept e2fsck's "errors corrected" exit codes under fix-errors. That
fix is what introduces the go-diskfs dependency above.

What

Realistic, EVE-shaped resize tests plus a SIGKILL chaos harness that exercises
resume safety under crash injection.

• EVE-patterned layout tests (eve_resume_test.go): a scaled-down EVE disk
  (FAT32 ESP, squashfs IMGA/IMGB, CONFIG, ext4 P9 at index 9) driven through the
  resume/idempotency harness — a Case-1 in-place P9 shrink and a Case-2 grow of
  ESP/IMGA/IMGB into the freed space — plus byte-integrity, atomic-failure,
  shrink-below-used, and combined shrink+grow invariants, with per-stage GPT
  table logging.
• SIGKILL chaos (eve_stress_test.go): runs the full two-step EVE resize as
  a subprocess and group-SIGKILLs it at random points across all five pipeline
  steps, then re-runs to completion and asserts the result matches an
  uninterrupted run; records which step each kill hit.
• GPT-write-delay hook (delaybackend_*.go, -tags chaos; run.go
maybeWrapBackend): a no-op in normal builds. Under the chaos tag it delays
  around GPT-sector writes so kills can land inside the otherwise-instantaneous
  updatePartitions/createPartitions table writes.
• Post-kill GPT integrity (gpt_integrity_test.go): after each kill, checks
  primary/backup header self-CRCs, entry-array CRCs, and primary↔backup entry
  equality, so torn tables and table divergence are observed — and shown to be
  healed by the resume.
• Copy-dest state (copyprogress_test.go, opt-in): classifies a grow
  target's data region as empty/partial/complete after a kill.

Results

With the GPT-write delay, kills reach all five pipeline steps including
updatePartitions; the resume recovers from every observed torn-entry-array and
primary↔backup-mismatch state.

[eriknordmark]

@deitch ready for review when you have a chance — see the PR description for what it does.

[eriknordmark]

@deitch this is ready for review when you have a chance.

To back up the resume-safety claim, I ran an 8-hour SIGKILL chaos soak of TestEVEChaosKill with the GPT write-delay hook enabled (CHAOS_GPT_DELAY=5s), which widens the window around the otherwise-instantaneous GPT-table writes so kills can land inside them. Each iteration runs the full EVE-style two-step resize (shrink P9, then grow ESP/IMGA/IMGB) as a subprocess, SIGKILLs it at a random point, then re-runs to completion and asserts the result is byte-identical to an uninterrupted resize.

Result: 85 iterations, 0 failures. Every killed resize resumed cleanly.

Where the 584 kills landed:

Step	Kills
shrinkPartitions	186
preflight	167
updatePartitions	98
createPartitions	92
copyFilesystems	28
shrinkFilesystems	13

The delay did its job — 98 kills landed inside updatePartitions and 92 inside createPartitions, i.e. mid-GPT-write.

On-disk GPT integrity sampled at each kill (584 samples):

State	Count
Header self-CRC failure (primary or backup)	0
Torn primary entry array (CRC mismatch)	66
Torn backup entry array (CRC mismatch)	64
Primary↔backup entry mismatch	121
Clean	395

So both tear shapes occur in practice — a torn 32-sector entry array, and a primary/backup ordering skew (backup written first, primary not yet) — and the resume logic heals every one. Header self-CRCs never tear because the header is a single 512-byte sector and its write is atomic, confirmed across all 584 samples.

[deitch]

Some questions.

[deitch]

I am not particularly enamoured of calling these "EVE tests". I get that you are testing a layout similar (identical?) to what EVE has. But partitionresizer is no more EVE-specific than diskfs is linuxkit-specific. It was the original driver for the idea, but built to be more generic. Even if it is identical, calling it something else that addresses the general flow would be better.

[eriknordmark]

Agreed, the name overfits. What's actually under test is a representative mixed-filesystem layout — FAT32 ESP, raw squashfs read-only images, ext4 data — exercised through a shrink-the-tail / grow-the-head flow. EVE was just the layout that motivated it. I'll rename the files and symbols (eve_* → layout/resume, TestEVE* → TestMixedLayout* / TestChaos*, buildEVEFixture → buildMixedLayoutFixture) and reword the README so EVE appears only as the motivating example.

[eriknordmark]

I cleanup up the commits including removing intermediate references to old names (of scripts etc). No content changes in HEAD.

[deitch]

How does this look? Ready to go?

[eriknordmark]

I'm doing integration testing with EVE and that found an issue or two in go-diskfs fixed in #411, which is now merged. partitionresizer is re-pinned onto it in #20; once that lands we can run this PR through CI.

[deitch]

I guess bring in 411 now?

[eriknordmark]

Done — the re-pin is up as #20.