fix: publish scratch artifacts to content-addressed storage - #574
harshitethic wants to merge 2 commits into
Conversation
kstonekuan
left a comment
There was a problem hiding this comment.
The fix itself is right, and both halves hold under mutation: forcing artifact_is_scratch_bound = False reddens 1, and dropping artifact_content_digest from the key reddens 1. publish does copy when source and destination differ, so the artifact really does leave scratch rather than just getting a nicer-looking URI.
Two things before this can land.
uv run ruff format --check fails on src/hflow/app.py, so this would turn main red:
- artifact_name_digest = hashlib.sha256(artifact_name.encode()).hexdigest()[:8]
+ artifact_name_digest = hashlib.sha256(artifact_name.encode()).hexdigest()[
+ :8
+ ]
And step_directory and artifact_name_digest are now computed twice, identically, in the two branches. Only the key layout differs between them. Hoist those two out above the if so the next person changing the naming scheme cannot change one copy and miss the other.
Worth a sentence in the code on why the non-scratch fallback stays non-content-addressed, since a reader hitting the second branch will wonder whether that is deliberate.
Fixes #545.
Summary
artifacts/storage instead of exposing scratch URIs;Why
Scratch is rewritten across reprocessing. A catalog URI that points into scratch can therefore start resolving to different bytes or disappear entirely. Content-addressed durable keys keep prior catalog URIs bound to the bytes that were published.
Testing