ptool, CI: deterministic GPT artifacts and pinned checksum check#111
Open
igoropaniuk wants to merge 2 commits intoqualcomm-linux:mainfrom
Open
ptool, CI: deterministic GPT artifacts and pinned checksum check#111igoropaniuk wants to merge 2 commits intoqualcomm-linux:mainfrom
igoropaniuk wants to merge 2 commits intoqualcomm-linux:mainfrom
Conversation
igoropaniuk
requested review from
lumag,
ndechesne,
sbanerjee-quic and
vkraleti
as code owners
Contributor
|
I'd prefer to let users specify the random seed (e.g. via the environment variable) and then use the fixed seed for all our CI builds (which would allow e.g. comparing by the results by the checksum). |
igoropaniuk
force-pushed
the
feat/deterministic_ptool_artifacts
branch
from
56aeddd to
f8d0399
Compare
igoropaniuk
changed the title
Contributor
Author
|
@lumag done |
lumag
previously approved these changes
Apr 28, 2026
Contributor
|
Thanks!. Maybe it would be nice (as a followup) to add regression testing in the form of the checksumming (some) of the generated files. |
Contributor
Author
I'm already in the middle of it, can push additional commits to this PR |
igoropaniuk
changed the title
Contributor
Author
|
@lumag done! |
igoropaniuk
commented
Apr 28, 2026
igoropaniuk
commented
Apr 28, 2026
lumag
requested changes
Apr 28, 2026
igoropaniuk
force-pushed
the
feat/deterministic_ptool_artifacts
branch
from
48cf3c8 to
1a1b6e8
Compare
The DiskGUID and per-partition UniquePartitionGUID values are produced by random.randint(0, 2**128) with an unseeded RNG. As a result, two consecutive runs of "make all" on the same commit yielded gpt_main*.bin, gpt_backup*.bin and gpt_both*.bin files that differed in ~250+ bytes (the random GUIDs themselves plus the header and partition-array CRC32 fields that cover them). This made it impossible to compare artifacts across CI runs or between developer trees on the same input by checksum. Honour PTOOL_SEED from the environment: when set and non-empty, seed random with its value before parsing the input, making the run fully reproducible. When unset, behaviour is unchanged and GUIDs stay random, matching pre-existing expectations for normal local use. Set PTOOL_SEED to a fixed value in the GitHub Actions build workflow so "make all integration" is reproducible in CI, enabling artifact comparison by checksum across branches and re-runs. The existing --sequentialguid flag and explicit per-partition uguid="..." overrides keep working unchanged; PTOOL_SEED only affects the random fallback path. GPT spec semantics are preserved: DiskGUID and UniquePartitionGUIDs are still full 128-bit values, each partition still gets a distinct GUID (including across LUNs on multi-LUN platforms like apq8096-db820c with 6 LUNs). Verified on apq8016-sbc/emmc/gpt_main0.bin: - Without PTOOL_SEED: two runs produce different DiskGUIDs (random). - PTOOL_SEED=42 twice: identical artifacts. - PTOOL_SEED=42 vs PTOOL_SEED=99: different DiskGUIDs. - PTOOL_SEED=qcom-ptool-ci + full "make all" twice: zero differing files under platforms/. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add tests/integration/checksums.sha256, a sha256sum -c manifest covering every generated *.bin and *.xml under platforms/, produced with PTOOL_SEED=qcom-ptool-ci (the seed CI uses). Wire two Makefile targets around it: - check-checksums: runs sha256sum -c against the manifest. Fails the build if any artifact has changed unexpectedly. - generate-checksums: regenerates the manifest from the current tree. Intended for use when partition configs or generator scripts are changed deliberately and the diff to checksums.sha256 then becomes part of the PR review (one line per affected artifact). The wrapper script lives at tests/integration/check-checksums and matches the style of the existing check-missing-files helper. The GitHub Actions build step now runs: $ make all integration check-checksums and a follow-up step regenerates the manifest and fails the build if the result does not match the checked-in file: $ make generate-checksums $ git diff --exit-code tests/integration/checksums.sha256 This catches contributors who add or modify a platform without also running make generate-checksums and committing the updated manifest; check-checksums alone would miss new files. Platform maintainer workflow: $ PTOOL_SEED=qcom-ptool-ci make clean generate-checksums $ git add tests/integration/checksums.sha256 $ git commit ... Verified locally: - Pristine seeded build passes (exit 0, no FAILED lines). - Tampering with a single artifact fails check-checksums (exit 1). - Building with a different seed fails check-checksums (288 GPT binaries diverge as expected). - Adding a stray file under platforms/ causes generate-checksums to produce a manifest that differs from the checked-in file, and git diff --exit-code reports it as expected. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
igoropaniuk
force-pushed
the
feat/deterministic_ptool_artifacts
branch
from
1a1b6e8 to
8d17375
Compare
Contributor
Author
|
@lumag all comments are addressed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Honour
PTOOL_SEEDfrom the environment to seed the RNG used for DiskGUID and UniquePartitionGUID, making "make all" reproducible when set and unchanged (random) when unset. The GitHub Actions build sets a fixed seed and now also runsmake check-checksumsagainst a pinnedtests/integration/checksums.sha256manifest covering every generated artifact, so unintended changes from a refactor or regression fail CI.A
make generate-checksumstarget is provided to regenerate the manifest when partition configs or generators change deliberately, with the diff becoming part of PR review.