Skip to content

Fix unnecessary file updates when HTTP source lacks checksum headers - #612

Open
jorhett wants to merge 1 commit into
OpenVoxProject:mainfrom
jorhett:jorhett/idempotent-file-source-handling
Open

Fix unnecessary file updates when HTTP source lacks checksum headers#612
jorhett wants to merge 1 commit into
OpenVoxProject:mainfrom
jorhett:jorhett/idempotent-file-source-handling

Conversation

@jorhett

@jorhett jorhett commented Aug 16, 2026

Copy link
Copy Markdown

Short description

This is I believe a more minimal fix for #581 than what is proposed in #582.

Fixes #581
Replaces #582

Root Cause

The HttpMetadata class in http_metadata.rb was setting a default mtime checksum using Time.now when HTTP servers didn't provide checksum headers (Content-MD5, X-Checksum-*, ETag, or Last-Modified). This default now timestamp checksum changed on every run, triggering unnecessary updates.

Solution

  1. Removed the problematic Time.now default in HttpMetadata#initialize
  2. Modified the HTTP metadata retrieval to fetch file content and compute content's checksum when no checksum headers are available
  3. Updated the corresponding spec to reflect the new behavior

Impact

  • HTTP-sourced files without checksum headers now have stable checksums based on actual content
  • Files are only updated when their content actually changes
  • Slight performance impact: content is fetched once to compute the initial checksum, but this is offset by avoiding unnecessary file writes on subsequent runs

Checklist

I have:

Key considerations

I believe this to be a bug/regression and that this fix should be considered for a patch level update. I can find no documentation that says that files should be treated as updated NOW when a timestamp is unavailable, nor that files should always be replaced when timestamp or checksum metadata are unavailable.

I also believe that this would align with user's expectations, such that fixing it in a patch update would be acceptable.

Assisted-by: Cascade:SWE-1.6 (Devin Desktop)

Assisted-by: Cascade:SWE-1.6 (Devin Desktop)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: file resource with an http(s) source and no cache-validation headers is treated as changed on every run

1 participant