test: patch Netty's lost-read-interest race in the test server via a javaagent - #4019
Closed
u9g wants to merge 1 commit into
Closed
test: patch Netty's lost-read-interest race in the test server via a javaagent#4019u9g wants to merge 1 commit into
u9g wants to merge 1 commit into
Conversation
Member
|
I'm not sure we should do this. That makes the server non vanilla and it removes a cause of failure that real users might see. |
…javaagent Vanilla servers up to 1.20.1 can stop reading a client's socket during login: Connection.sendPacket calls setAutoRead(false) on the main thread for every PLAY packet sent while the channel still says LOGIN and queues setAutoRead(true) onto the IO thread. Netty re-arms OP_READ only on a 0->1 flip of the flag but the queued clearReadPending0 drops it unconditionally, so when doSendPacket_i runs between the main thread's toggles for i+1 and i+2 the channel ends with autoRead=true and no read interest. Nothing re-arms it; ReadTimeoutHandler kills the connection 30s later and the whole version fails in CI. Mojang removed the toggle in 1.20.2 (MC-265209); Paper's archived branches carry no fix. The agent rewrites AbstractNioChannel.clearReadPending0 with the JDK ClassFile API to leave OP_READ alone when autoRead is already true. It is bytecode-level, so it applies to whatever Netty 4.1.x a server bundles (4.1.9 through 4.1.82 across 1.12-1.20.1) and is a no-op on 1.8.8's Netty 4.0.23, which predates the method. The jar is compiled from test/netty-agent at test time with the JDK's javac and jar and attached through JAVA_TOOL_OPTIONS, so minecraft-wrap is untouched; CI therefore needs a JDK rather than a JRE. Reproduced the stall at ~1% of logins under CPU load and ~10% with Connection DEBUG logging; with the agent, 0 stalls in 800 such logins.
u9g
force-pushed
the
test/netty-autoread-agent
branch
from
August 29, 2026 14:11
3e7be07 to
4d41973
Compare
Member
Author
You’re right, although it’s clientside that we should restart, done in #4018 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to #4018. That PR detects the login stall and reconnects; this one removes it on the test server.
The bug (vanilla ≤1.20.1, Paper included):
Connection.sendPacketcallssetAutoRead(false)on the main thread for each PLAY packet sent while the channel attr still says LOGIN, and queuessetAutoRead(true)on the netty IO thread. Netty re-armsOP_READonly on a 0→1 flip of the flag, but the queuedclearReadPending0drops it unconditionally — so whendoSendPacket_iruns between the main thread's toggles for i+1 and i+2, the channel ends upautoRead=truewith no read interest, nothing re-arms it, andReadTimeoutHandler(30)kills the connection. Reproduced deterministically on stock Netty 4.1.82 with latches, and on a real 1.19.3 server at ~1% of logins under CPU load (~10% withConnectionDEBUG logging). Mojang rewrote protocol switching in 1.20.2 (MC-265209); Paper's archived ≤1.20.1 branches have no fix, and there is no property to toggle.The fix: a
-javaagent(test/netty-agent/NettyAutoReadFixAgent.java, source only) that rewritesAbstractNioChannel.clearReadPending0with the JDKClassFileAPI to skipremoveReadOp()whenautoReadis already true. Bytecode-level, so it works on whatever Netty 4.1.x the server bundles (4.1.9–4.1.82 across 1.12–1.20.1) and is a logged no-op on 1.8.8's 4.0.23.test/common/nettyAgent.jscompiles and jars it at test time and the harness attaches it viaJAVA_TOOL_OPTIONS, so minecraft-wrap is untouched. If no JDK is present it logs and the suite runs unpatched.CI change:
java-package: jre→jdksojavac/jarexist on the runner (still Java 25).Validation
patched, 70/70patched, 69/69no clearReadPending0, 69/69