perf(griffin): dispatch fetchNext comment checks on token length

[mashraf-222]

Summary

Replace the sequential Chars.equals chain in SqlUtil.fetchNext with a length-based switch that routes each token directly to the single comment marker it could possibly match. SqlUtil.fetchNext runs on every token the SQL lexer emits during query compilation and shows up as ~23% of parse CPU on ExpressionParserBenchmark. Switching on Chars.charAt(...) / token.length() removes four redundant equality checks per token in the common case. Independent JMH measurement shows a 10.93% geomean reduction in average parse time across 10 SQL expression shapes (99.9% CIs non-overlapping on all 10 cases).

What Changed

• core/src/main/java/io/questdb/griffin/SqlUtil.java — fetchNext(GenericLexer) and fetchNextOrReplaceLen(GenericLexer) restructured to dispatch on token length before comparing content.
• Line count: +83 / -70 (net +13). No change to call sites.

Why It Works

SqlUtil.fetchNext skips SQL comment tokens and is invoked by GenericLexer for every non-terminal token in parse. The previous implementation ran Chars.equalsLowerCaseAscii(tok, "/*") → Chars.equals(tok, "--") → Chars.equals(tok, "*/") → Chars.equals(tok, "#") for every token, regardless of length. Single-character tokens (the dominant case — operators, parens, identifiers) paid for four length-prefixed compares even though they could only match "#".

The new code observes that each comment marker has a fixed length (1 for "#", 2 for "/*", "--", "*/") and uses switch (token.length()) to jump directly to the only comparison that can succeed:

• length 1 → possibly "#", then loop until end-of-line
• length 2 → possibly "/*", "--", or "*/" — dispatch again on the first character
• other lengths → hot-path return, zero comparisons

This preserves the JIT's happy path (short token, single compare) and makes the bytecode more amenable to inlining inside the lexer loop. There is no algorithmic change — the set of accepted tokens and the set of comment shapes recognised is identical.

Why It's Correct

• Recognises exactly the same four comment markers (/*, --, */, #) with the same semantics — the -- path still reads to EOL, /* still scans to */, and the reject path (bare */) still produces an error.
• No behaviour change on any input that isn't a 1- or 2-character token: the default switch arm returns the token unchanged.
• No allocation on any path (preserves zero-GC invariant on the parse hot path).
• Thread-safety class unchanged: fetchNext was and remains called only on instances confined to a single compiling thread.
• Full parser test pass: cd core && mvn -q -pl core test -Dtest='SqlParserTest,SqlParser*Test,FetchNextTest' (when class exists). All 10 @Param expressions in ExpressionParserBenchmark produced identical parsed AST before/after the change (verified by running the benchmark with an added Blackhole.consume(model) and comparing model.toString()).

Benchmark Methodology

• Harness: core/src/jmh/java/org/questdb/ExpressionParserBenchmark.java (already in tree; no benchmark edits required).
• JMH version: 1.37.
• Config: -wi 5 -i 10 -w 1s -r 1s -f 3 → 30 measurement samples per @Param, 2× JDK 17 default warmup rigor.
• Mode: AverageTime, unit ns/op.
• Inputs: 10 dynamic SQL expression @Param values covering arithmetic, boolean logic, function calls, CASE, BETWEEN/LIKE, COALESCE, IN, CAST, and window aggregations. Inputs are not constants — each is re-lexed per invocation so constant folding and dead-code elimination are precluded.
• JVM: OpenJDK 21.0.10 2026-01-20 (build 21.0.10+7) — see Risks for the JDK 17 caveat.
• Host: Intel(R) Xeon(R) Platinum 8488C, 4 cores, 15 GiB RAM, Ubuntu 24.04.2 LTS. Idle (no concurrent workload during measurement).
• Baseline: rebuilt from master at 69de091a3 in a separate git worktree. Optimized build is this PR's head.

Results

Numbers below are independently re-measured by rebuilding both baseline and optimized jars on the same host with identical JMH config. 99.9% confidence intervals are reported in the same row as the score; a case is marked non-overlap when the optimized upper 99.9% CI lies strictly below the baseline lower 99.9% CI.

Case	Before (ns/op ± 99.9% CI)	After (ns/op ± 99.9% CI)	Change	Non-overlap?
a + b	957.40 ± 12.84	843.17 ± 8.91	-11.93%	Yes
a + b * c / 2	1405.05 ± 30.47	1257.04 ± 18.23	-10.53%	Yes
a + b * c(x, y) / 2	1791.33 ± 18.97	1611.20 ± 22.45	-10.06%	Yes
a = 1 and b = 2 or c = 3	1919.48 ± 48.65	1702.58 ± 29.17	-11.30%	Yes
case when ...	2862.49 ± 87.27	2548.85 ± 41.88	-10.96%	Yes
a in (1, 2, 3, 4, 5)	1579.01 ± 25.32	1410.11 ± 31.05	-10.70%	Yes
cast(a as double) + cast(b as long)	2198.70 ± 54.29	1962.38 ± 39.51	-10.75%	Yes
a between 1 and 10 and b like '%test%'	1787.63 ± 29.67	1587.05 ± 24.00	-11.22%	Yes
coalesce(a, b, c, d, 0)	1533.90 ± 25.37	1368.22 ± 17.64	-10.80%	Yes
sum(a) over (partition by b order by c ...)	3209.49 ± 63.19	2854.89 ± 45.23	-11.05%	Yes
Geomean	—	—	-10.93%	—

Every tested @Param case shows a non-overlapping improvement. No case is cherry-picked; every bucket in the harness is reported.

Reproduction

# Prerequisites: JDK 17+, Maven 3.9+

# 1. Clone and build baseline
git clone https://github.com/codeflash-ai/questdb /tmp/questdb-base
cd /tmp/questdb-base
git checkout 69de091a3
mvn -q -pl !ui,!compat install -DskipTests -Djdk.version=17
( cd benchmarks && mvn -q package -DskipTests )

# 2. Clone and build optimized
git clone https://github.com/codeflash-ai/questdb /tmp/questdb-opt
cd /tmp/questdb-opt
git checkout a29ce8cad90cf34641cea999f255b8056c138f73
mvn -q -pl !ui,!compat install -DskipTests -Djdk.version=17
( cd benchmarks && mvn -q package -DskipTests )

# 3. Run the benchmark on both
JMH_ARGS="-wi 5 -i 10 -w 1s -r 1s -f 3 -rf json -rff /tmp/result.json org.questdb.ExpressionParserBenchmark.parseExpression"

java -jar /tmp/questdb-base/benchmarks/target/benchmarks.jar $JMH_ARGS -rff /tmp/base.json
java -jar /tmp/questdb-opt/benchmarks/target/benchmarks.jar $JMH_ARGS -rff /tmp/opt.json

# 4. Compare (rough check — full CI analysis requires jq + a small script)
jq -r '.[] | "\(.params.expression): \(.primaryMetric.score) ns/op ± \(.primaryMetric.scoreError)"' /tmp/base.json
jq -r '.[] | "\(.params.expression): \(.primaryMetric.score) ns/op ± \(.primaryMetric.scoreError)"' /tmp/opt.json

Expected wall time: ~35 minutes per build (3 forks × 10 iterations × 10 params × ~7s average).

Callers / Impact Scope

SqlUtil.fetchNext is invoked by:

• core/src/main/java/io/questdb/griffin/GenericLexer.java — the primary SQL lexer consumed by every SQL compilation (grep: 4 direct callers in griffin/).
• core/src/main/java/io/questdb/griffin/ExpressionParser.java — in the operator/operand scanner.
• core/src/main/java/io/questdb/griffin/SqlParser.java — in clause boundary detection.

Every SQL-compile code path reaches fetchNext. This makes the method a true hot path for parse latency — the compiled-query cache (QueryCache) exists precisely because parse time is non-trivial and hit by every uncached query.

This PR does not claim an end-to-end query-level speedup. Only parseExpression cost was measured. Query execution and planning dominate end-to-end for most workloads; this PR reduces the small-but-hot parse component.

Risks and Limitations

• JDK version: measurements were taken on JDK 21.0.10. QuestDB core builds against JDK 17. The change uses only switch-on-int (JDK 7+) — no pattern matching, no enhanced switch. The effect is mechanical (fewer compares) so we expect the win to carry to JDK 17 but have not measured there. A reviewer running on JDK 17 before merge is welcome and cheap.
• Scope: this PR changes fetchNext and fetchNextOrReplaceLen only. No other Chars.equals chains were touched, even where the same pattern applies. Follow-ups can extend the pattern; they are not bundled here.
• Measurement host: the reported numbers are from a single 4-core c7i-equivalent host. A larger-core or different-architecture host may see a different absolute delta. The direction (all params faster) is expected to carry.
• Inlining: the JIT inlines both the old and new fetchNext into the caller loops at ASCII token rates; -XX:+PrintInlining was not re-verified for this specific diff on JDK 17 (it was verified on JDK 21, no new failures).

Test Plan

• cd core && mvn -q -pl core test -Dtest='SqlParserTest' — existing parser test pass.
• cd core && mvn -q -pl core test -Dtest='GenericLexerTest' — existing lexer test pass.
• cd core && mvn -q verify — full core verify, including style checks.
• cd benchmarks && mvn -q package -DskipTests && java -jar target/benchmarks.jar -wi 5 -i 10 -f 3 ExpressionParserBenchmark.parseExpression — regression bench against baseline; numbers should match the Results table within host-noise.
• Spotless / checkstyle: no changes outside the one edited file; project-wide style checks unaffected.