Skip to content

A-1414: support hosted cache#4028

Merged
zhming0 merged 1 commit into
mainfrom
ming/a-1414
Jun 25, 2026
Merged

A-1414: support hosted cache#4028
zhming0 merged 1 commit into
mainfrom
ming/a-1414

Conversation

@zhming0

@zhming0 zhming0 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Description

Support hosted cache via Namespace artifact storage, driven by an nsc://<namespace> cache store URL under the agent_managed store type.

What this does:

  • Scheme dispatch: NewBlobStore routes agent_managed URLs by scheme — nsc://NscStore, otherwise S3.
  • Namespace from URL: the namespace is parsed from nsc://<namespace> and passed as --namespace to the nsc CLI (errors if the namespace is missing).
  • Validation: validateCacheStore now accepts s3://, nsc://, and file:// for agent_managed, and rejects unknown schemes.
  • Default expiry: uploads pass --expires_in 24h (temporary stopgap; TTL refresh-on-access parity with S3 tracked in A-1437).
  • Context cancellation: the nsc CLI now runs via exec.CommandContext, so uploads/downloads abort on context cancel/timeout.
  • Cleanup: removed the unused local_hosted_agents store type (NSC is now reachable only via nsc://).

Tests are lean and don't require the nsc binary: URL parsing, namespace passthrough via an injected command-runner seam, validation short-circuiting, and nsc:// dispatch. The existing NSC_INTEGRATION_TEST-gated end-to-end test is retained.

Follow-ups: A-1437 (TTL refresh, pending upstream), A-1438 (cache busting / invalidation).

Context

A-1414

Testing

  • Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
  • Code is formatted (with go tool gofumpt -extra -w .)

Disclosures / Credits

LLM and me.

@zhming0 zhming0 requested review from a team as code owners June 25, 2026 00:11
@zhming0 zhming0 added the internal Non-user facing, internal change. label Jun 25, 2026
@zhming0 zhming0 requested a review from buildsworth-bk June 25, 2026 01:35
@buildsworth-bk-app

Copy link
Copy Markdown

On it! Follow along with my review here: https://buildkite.com/buildkite/buildsworth-review-pull-request/builds/1215

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read this as L2 rather than an approval because it changes cache-store dispatch/validation in the agent-managed path. I found one non-blocking mismatch around the local-testing URL path.

Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:

Download the buildsworth logs from build 1215, then answer my questions about the findings.

Comment thread internal/cache/save.go
Comment thread internal/cache/store/nsc.go Outdated
Comment thread internal/cache/store/nsc_test.go Outdated
Comment thread internal/cache/store/blob.go Outdated
@zhming0 zhming0 merged commit f057979 into main Jun 25, 2026
4 checks passed
@zhming0 zhming0 deleted the ming/a-1414 branch June 25, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Non-user facing, internal change.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants