python3Packages.{pyimg4,smart-meter-texas}: fix against asn1 3.x by adding asn1_2 - #555937
python3Packages.{pyimg4,smart-meter-texas}: fix against asn1 3.x by adding asn1_2#555937Bad3r wants to merge 3 commits into
Conversation
python-asn1 3.0.0 changed Decoder.read() to recursively decode constructed elements into lists instead of returning their raw content octets, Encoder.write() with Types.Constructed to re-encode each item of the value instead of emitting pre-encoded octets verbatim, and Decoder.eof() to be relative to the whole input instead of the entered container. Consumers written against the 2.x raw-octet API cannot be pointed at 3.x unchanged. pyimg4 is the only consumer in tree: it pins asn1<3.0.0 upstream (m1stadev/PyIMG4#59) and has been marked broken since asn1 was updated to 3.x. asn1 itself stays on 3.3.0 as the default. Validation: nix-build -A python3Packages.asn1_2 (upstream tests/test_asn1.py pass on 3.12, 3.13, 3.14). Assisted-by: Claude Code (claude-opus-5)
Marked broken in ef300f4 because asn1 is 3.3.0 and pyimg4 pins asn1<3.0.0. The pin is not stale metadata: with the constraint relaxed, 5 of 7 tests fail with "asn1.core.Error: Expecting bytes or a subclass of io.RawIOBase or BufferedIOBase. Get <class list> instead" and "UnexpectedTagError: Expected tag of type IA5String, got PrintableString", because Decoder.read() no longer returns the raw content octets that parser.py re-parses and re-emits. Upstream declined the port in m1stadev/PyIMG4#59 ("pyimg4 should be refactored around it first") and has not released since v0.8.8, so build against asn1_2 rather than patch. This unblocks ipsw-parser and pymobiledevice3, which fail to evaluate through pyimg4. A port is proposed in m1stadev/PyIMG4#63; once released, this can move back to asn1. Validation: nix-build -A python3Packages.pyimg4 (7 passed, 5 deselected), -A python3Packages.ipsw-parser, -A python3Packages.pymobiledevice3 (56 passed, 1 xfailed); nix-store -qR on the pymobiledevice3 output shows asn1 2.8.0 only, no collision with 3.3.0. Assisted-by: Claude Code (claude-opus-5)
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
ClientSSLContext._find_ca_issuers_uri() walks an X.509 authorityInfoAccess extension by recursing through Decoder.enter()/leave() while looping on Decoder.eof(). asn1 3.0.0 made eof() relative to the whole input rather than the entered container, so the loop runs past the end of a container, peek() returns None, and the next tag.typ access raises AttributeError. It only shows up when authorityInfoAccess holds more than one AccessDescription. That is the common real-world layout (OCSP first, then caIssuers), and it is why this was not caught: doCheck is false because upstream ships no tests, and get_ca_issuers_uri() wraps the call in "except Exception", so on asn1 3.x the failure is swallowed and logged as "Failed to lookup CA Issuers URI value" while the function silently returns None. Reproduced on a hand-built DER authorityInfoAccess with two AccessDescriptions: asn1 2.8.0 returns the caIssuers URI, asn1 3.3.0 raises "AttributeError: NoneType object has no attribute typ". Upstream requires asn1>=2.4.1, so 2.8.0 satisfies it without relaxing anything. Validation: nix-build -A python3Packages.smart-meter-texas. Assisted-by: Claude Code (claude-opus-5)
Follow-up:
|
This comment was marked as outdated.
This comment was marked as outdated.
✅
|
dotlambda
left a comment
There was a problem hiding this comment.
Versioned attributes are not allowed in dependencies (they must not be propagated by python3Packages). You can downgrade asn1 instead.
python3Packages.pyimg4has been marked broken since ef300f4, which setbroken = lib.versionAtLeast asn1.version "3". Sinceasn1is at 3.3.0, the guard always fires and takesipsw-parserandpymobiledevice3down with it:The pin is not stale metadata. With the constraint relaxed and
brokendropped, 5 of 7 tests fail against asn1 3.3.0:python-asn1 3.0 changed
Decoder.read()to recursively decode constructed elements into nested lists instead of returning their raw content octets, which is whatparser.pyre-parses and re-emits throughout. It also changedEncoder.write(..., Types.Constructed)to iterate the value rather than emit pre-encoded octets verbatim, so pre-encoded DER silently becomes one Integer per byte. Upstream declined the port in m1stadev/PyIMG4#59 ("Version 3.0.0 is technically much better, but pyimg4 should be refactored around it first to support it") and has not released since v0.8.8 in April 2025.So this adds
asn1_2at 2.8.0, the last 2.x, and buildspyimg4against it, following the existing convention for versioned Python attributes (cython_0,chardet_5,django_5).asn1stays at 3.3.0 as the default. NopythonRelaxDepsentry is needed, since 2.8.0 satisfies upstream'sasn1>=2.7.0,<3.0.0directly.smart-meter-texasis the only other Nixpkgs consumer ofpython3Packages.asn1, and it is broken on 3.x for the same root cause:_find_ca_issuers_uri()recurses throughDecoder.enter()/leave()while looping onDecoder.eof(), which 3.0 made relative to the whole input instead of the entered container. On a two-entryauthorityInfoAccess(the common OCSP-then-caIssuers layout) it raisesAttributeError: 'NoneType' object has no attribute 'typ'on 3.3.0 and returns the URI on 2.8.0. It goes unnoticed because upstream ships no tests andget_ca_issuers_uri()swallows the exception. The third commit moves it toasn1_2too; details and the reproduction are in a comment below.That leaves
asn1at 3.3.0 with no in-tree consumer, so downgrading it outright and droppingasn1_2would be a smaller diff and breaks nothing in the tree. I avoided that because it would silently hand 2.x to out-of-tree users who asked for the current release. Glad to switch if you prefer it.I have also proposed the actual port upstream in m1stadev/PyIMG4#63. Once that lands and is released,
pyimg4can move back toasn1andasn1_2can be dropped. Happy to go the other way instead and patchpyimg4with that diff here if maintainers prefer not to carry a secondasn1.Verification
nix-store -qRon thepymobiledevice3output containspython3.14-asn1-2.8.0and no 3.3.0, so there is no module collision in the closure.The affected set is closed and was built in full by hand:
asn1_2is new and onlypyimg4consumes it,pyimg4's only reverse dependency isipsw-parser, whose only reverse dependency ispymobiledevice3, which nothing else in tree depends on. All four are built above.nixpkgs-review pr 555937has since confirmed exactly that set: 8 packages built (asn1_2,pyimg4,ipsw-parser,pymobiledevice3on python313 and python314), 0 failures. Report posted as a comment below.Automation/AI disclosure
Per CONTRIBUTING.md: both commits carry an
Assisted-by: Claude Code (claude-opus-5)trailer, and this pull request summary was likewise drafted with that tooling. I have reviewed the diff and the reasoning, and verified the outcome independently by building every affected package and inspecting the runtime closure, as listed above.Things done
passthru.tests.nixpkgs-reviewon this PR. See nixpkgs-review usage../result/bin/.