Repository navigation
Force HTTP/1.1 in pubchem_get to dodge buggy h2 client at the compat floor - #83
Draft
ChrisRackauckas-Claude wants to merge 5 commits into
Draft
ChrisRackauckas-Claude wants to merge 5 commits into
ChrisRackauckas-Claude wants to merge 5 commits into
Conversation
…floor HTTP.jl before 2.12 never returns connection-level HTTP/2 receive-window credit for response bytes that arrive but are never read (abandoned bodies, resets, padding). On the downgrade lane's HTTP 2.3.1 floor, a single throttle-induced mid-body failure permanently drains the pooled connection's 64 KiB window, after which every later request stalls until the 120 s read-idle timeout fires — the deterministic `http timeout during request` error killing Core/glycolysis.jl (issue #82). Sequential REST GETs gain nothing from h2 multiplexing, so request h1 explicitly. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <noreply@cognition.ai> Agent-Harness: Devin CLI 3000.11.228 Agent-Model: swe-2-high Agent-Session: local job 82-devin (no session URL)
The worker pre-push hook requires an explicit formatter declaration when the repo does not call fredrikekre/runic-action directly (this repo uses the SciML shared runic workflow), matching sibling job checkouts. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <noreply@cognition.ai> Agent-Harness: Devin CLI 3000.11.228 Agent-Model: swe-2-high Agent-Session: local job 82-devin (no session URL)
Counted review flagged that HTTP 2.11.0 already returns connection-level h2 window credit for unread body bytes. Verified at source: the refund helper is absent through 2.6.7 and present by 2.7.1; the floor version 2.3.1 is demonstrably affected. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <noreply@cognition.ai> Agent-Harness: Devin CLI 3000.11.228 Agent-Model: swe-2-high Agent-Session: local job 82-devin (no session URL)
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Agent-Harness: Claude Code (claude-local) Agent-Model: qwen3.8-flash-next Agent-Session: local session e67c8f95-4e9d-4581-af70-717dfb1de194 (no session URL)
The branch already contains upstream/master (2c9fb53); merging it is a no-op. The failing 32-bit Core lane resolved HTTP 2.13.5, which does not load on 32-bit Julia. Current resolution picks HTTP 2.15.2, which loads on 32-bit (32-bit support restored in HTTP 2.15.1). No code change. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Muse <noreply@meta.com> Agent-Harness: Muse CLI Code Agent-Model: muse-wonder-full-1.0 Agent-Session: local fixer job SciML__PubChemReactions.jl__83 (no session URL)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ignore this PR until it has been reviewed by @ChrisRackauckas.
What changed and why
pubchem_getinsrc/data.jlnow passesprotocol = :h1toHTTP.get, so PubChem/Rhea requests use HTTP/1.1 instead of HTTP.jl's HTTP/2 client. At this package's compat floor (HTTP 2.3.1, what the Downgrade lane pins), the h2 client never returns connection-level receive-window credit for response bytes that arrive but are never read — so one throttle-induced mid-body failure permanently stalls every later request on the pooled connection, which is the deterministichttp timeout during request (elapsed ~120 s)intest/glycolysis.jl(upstream: JuliaWeb/HTTP.jl#1371, fixed in HTTP 2.12). These are sequential REST GETs, so h2 multiplexing gains nothing anyway.Closes #82
Also on the branch, as a separate commit:
.factory-pre-push.jsondeclaring Runic for.jlfiles — the worker pre-push hook requires an explicit declaration since this repo uses the SciML shared runic workflow rather thanfredrikekre/runic-actiondirectly (same file sibling checkouts carry).Diagnosis summary
julia-downgrade-compatindepsmode on Julialts(1.10.12), pinning HTTP 2.3.1._wait_h2_body_progress!→i/o timeout) while fetching the large PUG-View JSON atsrc/data.jl:89, reached fromglycolysis.jl:32→species_from_cid.StatusErrors, and ~20 s paced pauses occur on both HTTP 2.3.1 and 2.13.1.retrylands on the same dead pooled connection, which is why the existing backoff did not help. This also explains why only the oldest-deps lane fails: current deps run HTTP ≥ 2.12, where the credit is returned.Verification
Lane replica:
downgrade.jlproduced the floorManifest.toml(HTTP 2.3.1); installed and run onjulia +1.10= 1.10.12, same as the lane.Fail before (master
src/data.jl):Same error type, same frame stack, same ~120 s elapsed as CI (120151175323 ns there).
Pass after (same manifest, with
protocol = :h1):Mechanism reproducer on HTTP 2.3.1's shared pooled client: one response body abandoned mid-read, then every later request on the pool stalled at
read_idle_timeout(6 consecutiveHTTP.TimeoutErrors at 20.2 s each — the CI signature), whileprotocol = :h1returned the 3.4 MB record in 0.4 s on the same poisoned process state.Existing tests:
test/http_retry.jlpasses with the fix under both HTTP 2.3.1 and 2.13.1.runicmade no changes;typosis clean.Full lane group on the same downgraded manifest (
GROUP=Core julia +1.10 --project=. -e 'using Pkg; Pkg.test(allow_reresolve=false)'):Round-1 counted-review follow-up
:autoalready picks h1, cannot discriminate (same reasontest/http_retry.jlnever could); (b) local TLS server advertising["h2","http/1.1"]with server-sideRequest.versioncapture — on master,pubchem_get's:autoattempt still lands h1 (h2 attempt fails over before the handler), so the observed version is identical with and without the patch; (c)trace/RequestEventcapture of the attempted protocol — controlHTTP.getemits:h2but master'spubchem_getemitted only:h1events on this endpoint, so the test passes unfixed (reason for the h1 choice at that call site was not root-caused); (d) h2-only ALPN endpoint — ALPN mismatch is not fatal, the h1 request still completes. The behavior fix itself remains covered by the lane-faithful fail-before/pass-after above.Note on the x86 Core lane (pre-existing upstream breakage)
The
Core (julia 1, ubuntu-latest, x86)lane fails on this PR withLoadError: InexactError: convert(Int32, 0xffffffff). This is pre-existing on master and unrelated to this change: the error is raised inside HTTP.jl's own module include —const _HPACK_MAX_INTEGER = Int(typemax(UInt32))athpack.jl:747overflowsInt32while HTTP.jl precompiles, before any PubChemReactions code runs.protocol = :h1is a request-time keyword; it cannot affect HTTP.jl's own load. Reproducer (32-bit Julia, no package code involved):The constant is absent in HTTP 2.3.1, which is why the same PR's Downgrade lane (floor manifest) is green while the latest-deps x86 lane is red; master's x86 lane simply has not run since the HTTP release that introduced it. Fixing it requires an HTTP.jl-side change (e.g.
Int64/UInt32for_HPACK_MAX_INTEGER), which is out of scope here per the open x86-lane decision.Not verified
GROUP=QAnot run locally — the diff is one request kwarg plus a docstring paragraph.get_page'sDownloads.downloadpath (libcurl) is unaffected by this change and not covered by the evidence.What to push back on
HTTP = "2.12"is the alternative if a reviewer prefers it.Risk assessment
pubchem_getis the single entry point); current-dep users also switch to h1.Co-Authored-By: Chris Rackauckas accounts@chrisrackauckas.com
🤖 Posted by an AI agent — harness: Devin CLI 3000.11.228 · model: swe-2-high
Conversation: local job 82-devin (no session URL)
Fix by Muse CLI Code muse-wonder-full-1.0 (head f255f13)
No code change and no new commit: the failing Core lane is pre-existing upstream breakage, now fixed upstream, so this branch needs only a fresh CI run.
What was checked:
Core (julia 1, ubuntu-latest, x86), failing while HTTP.jl itself loads:LoadError: InexactError: convert(Int32, 0xffffffff)atHTTP/src/hpack.jl:747, before any PubChemReactions code runs. The lineconst _HPACK_MAX_INTEGER = Int(typemax(UInt32))overflowsInton 32-bit Julia. A request-time keyword inpubchem_getcannot affect HTTP.jl's own precompilation.InexactError, samehpack.jl:747. All non-x86 lanes passed there too. The default branch has no CI run since 2026-10-17, which is why its x86 lane still shows green from before the breaking HTTP.jl release.Int(min(typemax(UInt32), typemax(Int))). See the HTTP.jl v2.15.1 release notes and the closed upstream issue JuliaWeb/HTTP.jl#1558. This package's compatHTTP = "2.3.1"already admits v2.15.1 and later, so a fresh resolve selects a fixed HTTP.jl with no change here.[compat]was considered and rejected: compat entries are not architecture-specific, so it would block 64-bit users pinned to HTTP.jl 2.12–2.15.0 by other packages for a 32-bit-only upstream load bug that is already fixed.Validation commands and observed output:
Not verified: the 32-bit x86 lane itself was not run locally (this machine has no 32-bit runtime libraries), and
GROUP=QAwas not run because there is no code change. The x86 lane should pass on the next CI run, which resolves HTTP.jl v2.15.2.Next counted review must come from a smart model in a fresh session, not this fix session (risk.md § Independent reviews).
🤖 Posted by an AI agent — harness: Muse CLI Code · model: muse-wonder-full-1.0
Conversation: local fixer job SciML__PubChemReactions.jl__83 (no session URL)
Fix by Muse CLI Code muse-wonder-full-1.0 (head c8930da)
What changed:
upstream/masterand merged it intoagent/82-httptimeout. The merge was already up to date: the branch already containedupstream/masterat 2c9fb53, so there were no new default-branch commits to bring in and no code change resulted.Core (julia 1, ubuntu-latest, x86), which failed inside HTTP.jl itself while loading, before any package code ran.LoadError: InexactError: convert(Int32, 0xffffffff)athpack.jl:747. HTTP 2.15.2 (what a fresh resolution now picks; 32-bit support was restored in HTTP 2.15.1) loads cleanly on the same 32-bit Julia. The failure is therefore pre-existing upstream breakage in a dependency, not caused by this PR, and no package code was changed for it.Validation, run locally before pushing:
32-bit HTTP load check, run locally on Julia 1.12.7 (word size 32), fail before / pass after in dependency version only:
Not verified: the full Core group on a 32-bit Julia was not run locally; the 32-bit evidence above covers the exact load failure that made the lane red, and the full Core group passed locally on 64-bit Julia 1.12. CI on the new head is the check for the 32-bit lane end to end.
Next counted review must come from a smart model in a fresh session, not this fix session (risk.md § Independent reviews).
🤖 Posted by an AI agent — harness: Muse CLI Code · model: muse-wonder-full-1.0
Conversation: local fixer job SciML__PubChemReactions.jl__83 (no session URL)