Add query profiling functions: util:time, util:memory, util:track, util:explain, util:profile#6208
Open
joewiz wants to merge 4 commits intoeXist-db:developfrom
Open
Add query profiling functions: util:time, util:memory, util:track, util:explain, util:profile#6208joewiz wants to merge 4 commits intoeXist-db:developfrom
joewiz wants to merge 4 commits intoeXist-db:developfrom
Conversation
Closed
3 tasks
joewiz
force-pushed
the
v2/query-profiling
branch
2 times, most recently
from
7975792 to
3d14e33
Compare
…util:track()
Phase 1 of query profiling visibility (inspired by BaseX prof: module):
util:time($expr) / util:time($expr, $label):
Pass-through wrapper that measures and logs execution time.
Returns the expression result unchanged.
util:memory($expr) / util:memory($expr, $label):
Same pattern for memory measurement. Logs the memory delta
during expression evaluation.
util:track($expr) / util:track($expr, $label):
Returns map { "time": xs:dayTimeDuration, "memory": xs:integer,
"value": item()* }. Most useful of the three — combines time
and memory measurement in a structured result.
All registered in UtilModule.java. 13 XQSuite tests in profiling.xql.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 2 of query profiling: util:explain($query) returns the compiled expression tree as XML, showing the post-optimization query plan. FunExplain.java: - Compiles query string using the same pattern as util:compile() - Runs AnalyzeContextInfo for optimizer annotations - Serializes tree via QueryPlanSerializer visitor - Returns root <explain> element QueryPlanSerializer.java: - Extends DefaultExpressionVisitor to walk the expression tree - Emits XML elements for each expression type: <flwor>, <for>, <let>, <where>, <return>, <order-by>, <group-by>, <path>, <step>, <predicate>, <filter>, <comparison>, <function-call>, <builtin-function>, <user-function>, <variable>, <if>, <union>, <intersect>, <try-catch>, <element-constructor>, <text-constructor>, etc. - Includes @axis, @test, @variable, @name, @Arity, @operator, @line, @column attributes where applicable Example: util:explain('for $x in 1 to 10 where $x > 5 return $x') → <explain><for variable="$x"><in>...</in>...</for></explain> 7 new XQSuite tests for explain (83/83 total util tests pass). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 3 of query profiling: util:profile($query) executes a query with profiling enabled and returns a map combining: - result: the actual query result - time: xs:dayTimeDuration of execution - memory: xs:integer bytes delta during execution - plan: element(explain) — the compiled expression tree (from Phase 2) - stats: element() — profiler statistics XML from PerformanceStatsImpl (function calls, index usage, optimizations) FunProfile.java: - Compiles and analyzes query using the same pattern as util:explain() - Enables eXist's built-in Profiler at verbosity=10 before execution - Captures timing via System.nanoTime(), memory via Runtime - Serializes the expression tree via QueryPlanSerializer - Serializes profiler stats via PerformanceStatsImpl.serialize() - Packages everything into a MapType result Two signatures: - util:profile($query as xs:string) as map(*) - util:profile($query, $module-load-path) as map(*) 9 new XQSuite tests (92/92 total util tests pass). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 3.2: Per-query scoping for PerformanceStats - Add Profiler.getPerformanceStats() to expose per-profiler stats - FunProfile now reads from the per-query profiler's stats instead of the global BrokerPool stats, ensuring correct data under concurrent load Phase 4.1: Optimizer decision logging - Add DEBUG-level logging in GeneralComparison.analyze() showing when index optimization is applied or skipped, including the expression text, QName, and optimization type Phase 5.1: util:index-report($query) as element() - Execute a query with profiling enabled and return an XML report of index usage and optimizations from the per-query PerformanceStats - Uses the same per-query profiler isolation as util:profile() 2 new XQSuite tests (94/94 total pass). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
joewiz
force-pushed
the
v2/query-profiling
branch
from
3d14e33 to
e2c437a
Compare
Member
Author
|
[This response was co-authored with Claude Code. -Joe] CI state: 7/9 checks pass. The 2 remaining failures (windows and macOS integration) are pre-existing test hangs unrelated to this PR — the surefire fork timeout fires and the CI job reports FAILURE. This PR adds only new code; no existing code is modified. Dependencies: None. This PR is Wave 1 — it can be reviewed and merged independently. For full context on all 7.0 PRs and the merge order, see the Reviewer Guide. |
duncdrum
added
enhancement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds 5 new XQuery functions for profiling and debugging query performance. Includes XQSuite tests.
What Changed
util:time($expr)— measure execution timeutil:memory($expr)— measure memory allocationutil:track($expr)— combined time + memory trackingutil:explain($expr)— show query planutil:profile($expr)— detailed profiling with index usageTests
Test plan
🤖 Generated with Claude Code