fix(#6264): fourteen more algo.* iteration knobs get the minimum and the checkpoint #6216 gave three - #6288
Conversation
…the checkpoint #6216 gave three Follow-up to #6216, which established that an iteration-shaped knob needs a domain minimum rejected by name and a checkpoint inside the loop it drives, and gave both to algo.node2vec, algo.maxKCut and algo.influenceMaximization. Thirteen more procedures carried the identical defect, untouched: pageRank, personalizedPageRank, articleRank, eigenvector, hits, katz, louvain, leiden, labelPropagation, slpa, simRank, fastrp and hashgnn. Every one extracted its knob with a plain extractInt(n, "maxIterations"), and none contained a single guard call. Domain: CALL algo.pageRank({maxIterations: 0}) returned the uniform initial rank vector as though it were a PageRank result, algo.louvain returned every node in its own community, algo.fastrp the untouched random projection. The silent half is the more serious one - an un-iterated centrality is not obviously wrong to a caller, unlike an exception. All thirteen now reject a value below 1 naming the procedure, the parameter and the value. Time: no honest ceiling exists, so a large value stays legal and becomes abortable. Each iteration loop calls the shared WorkGuard, which observes a thread interrupt and the arcadedb.command.timeout deadline; a per-node checkpoint inside each pass bounds the abort latency below a whole graph sweep, throttled to once every 1024 nodes where a node's own work is small and unthrottled where it is already O(n). Five of the thirteen have no convergence test at all - the CSR PageRank kernel, simRank, fastrp, hashgnn and slpa - so the knob alone decided when they stopped. Two of them hand the work to GraphAlgorithms, which sits below the query layer. Rather than couple the OLAP kernels to the query engine, GraphAlgorithms.pageRank and labelPropagation gained an overload taking the new WorkCheckpoint, a one-method interface in com.arcadedb.graph.olap the procedures satisfy with a method reference to their own guard; existing callers are unchanged and get a checkpoint that never aborts. Heap: alone among the thirteen, slpa's iterations buys memory as well as time - one label-memory row of iterations + 1 ints per node, so {iterations: 1000000} on a 10k-node graph asks for 40 GB, and at Integer.MAX_VALUE the iterations + 1 wrapped to Integer.MIN_VALUE and died as a bare NegativeArraySizeException naming nothing. The footprint is now estimated in saturating long arithmetic and checked against the same arcadedb.cypher.algoMaxWalkMemory budget the walk buffers use, before the first row is allocated. checkWalkBudget() is the walk-shaped special case of a new checkBufferBudget(). Also: simRank's per-iteration reset of the n x n similarity matrix is an Arrays.fill plus the diagonal instead of an n^2 element-by-element write.
|
Tick the box to add this pull request to the merge queue (same as
|
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
ReviewRead through the full diff plus the current state of Correctness spot-checks (all held up)
One gap in scope
Minor
Style/processFollows the repo conventions well: Nothing here blocks merge; the GraphSAGE gap is the only actionable item. |
…ame defect
Found in review. AlgoGraphSAGE.layers has exactly the shape this PR fixes everywhere else - extracted with a
bare extractInt(n, "layers"), and its layer loop carries no checkpoint at all. {layers: 0} silently returned
the untouched random-Gaussian initial features as though they were trained embeddings, the same silent half
as fastrp and hashgnn, and {layers: 2000000000} had nothing to interrupt or time out. It sits in the same
package as the two that were fixed and the fix shape is identical, so it is folded in here rather than left
as a follow-up.
The Xavier initialisation inside the layer loop deliberately carries no checkpoint of its own: both its
bounds are embedding dimensions, capped at MAX_EMBEDDING_DIMENSION, so it is bounded work whatever the caller
asks for - unlike the per-node loop that follows it.
Also from review: the two GraphAlgorithms overloads say in their javadoc why their one-sweep abort latency is
coarser than the ~1024-node latency the inline procedure loops reach, and as fine as those kernels can be -
their per-node work runs on parallelForRange worker threads, which would not observe an interrupt aimed at
the calling thread, and throwing out of a chunk closure would leave its siblings running.
Test counts in the class javadoc corrected with it: fourteen procedures, six of which have no convergence
test at all. Four new parameterised cases (zero rejection, minimum accepted, interrupt abort, deadline
abort); the zero-rejection one is red against the unmodified AlgoGraphSAGE.
|
Review actioned. The GraphSAGE gap — valid, folded in (commit One judgement call inside it worth stating: the Xavier initialisation in the layer loop gets no checkpoint of its own. Both its bounds are embedding dimensions, capped at The javadoc callout on the granularity asymmetry — valid, added. Both Counts corrected throughout (title, release notes, test javadoc): fourteen procedures, six of them with no convergence test at all. The two minor points — agreed, no change. Your own reading is right on both: the SLPA array-capacity message is unreachable under a finite budget and is defence in depth, and the near-verbatim per-loop comments are intentional, since each is locally justified by the loop it sits above and a reader arriving at one file should not have to find another to know why the checkpoint is there. Tests after the change: 119 green across |
|
Review of #6288 I read through the full diff (20 files, ~780 additions) plus the current merged state of the affected sources (AbstractAlgoProcedure, WorkGuard, WorkCheckpoint, GraphAlgorithms, and all fourteen algo.* procedures). Overall this is a solid, disciplined follow-up to #6216/#6222 - it mechanically (and in two cases, non-mechanically and correctly so) extends an established pattern rather than inventing a new one per procedure. Code quality
Bugs
Performance
Security Test coverage
One minor gap: no test asserting that algo.simRank Arrays.fill refactor produces bit-identical results to the old element-by-element reset (it is mentioned only in the commit message, not the issue). pageRankStillConvergesToTheRightAnswerWithTheCheckpointInPlace covers the same "did the algorithm output change" concern for PageRank, but there is no equivalent test for SimRank buffer-reset change specifically - likely low risk since Arrays.fill(x, 0.0) followed by x[i]=1.0 is behaviorally identical to the loop it replaces, but worth a quick sanity check if not already covered elsewhere. Overall this closes a real gap (silent wrong answers from {maxIterations: 0}-shaped calls, plus genuinely unbounded CPU/heap) across the entire algo.* surface, and the two departures from the issue "mechanical application" framing (the WorkCheckpoint interface for the CSR kernels, and SLPA heap accounting) are both justified rather than convenient. |
…reserve The one place this PR changes what a kernel computes rather than only when it stops is SimRank's per-iteration reset of the n x n similarity matrix, which became an Arrays.fill plus the diagonal instead of an element-by-element write of `i == j ? 1.0 : 0.0`. Review asked for a test pinning the result, and it was right to: every other over-reach guard in the class covers a different procedure. The four-node cycle cannot carry the assertion - every SimRank of two distinct nodes there is 0, so a broken reset would go unnoticed. The test builds a hub pointing at two leaves instead, where the leaves share their only in-neighbour and sim(A, B) = decay x sim(hub, hub) = 0.8. It asserts at 1 AND 3 iterations, which is what makes it non-vacuous. The reset exists only for the diagonal - the u < v loop writes every off-diagonal cell itself - and the buffers swap each round, so dropping it leaves sim(hub, hub) at 0 from the second iteration on. Measured: with `newSim[i][i] = 1.0` removed, the run still returns 0.8 at 1 iteration and fails at 3.
|
Round 2 actioned — thanks, the SimRank gap was a fair catch. The one suggestion — added ( It asserts at 1 and 3 iterations, which is what makes it non-vacuous: the reset exists only for the diagonal (the The two "worth a second look" items — no change, and I think your own reading of both is right.
Suite after the change: 634 green across |
ReviewI read through the full diff (14 procedures + Code quality / correctness
Minor/non-blocking observations
Test coverageExcellent. The 62-case matrix (zero/negative rejection × 14, minimum-acceptance × 14, interrupt-abort × 14, deadline-abort × 12, plus targeted CSR-kernel and SLPA-heap tests) is exactly the right shape, and a few details stand out as careful rather than boilerplate:
No functional issues found. Nice work. |
…only its products From review. AlgoSLPA added WALK_ROW_OVERHEAD_BYTES to a saturatingProduct() with a plain +, so a product that saturated at Long.MAX_VALUE would wrap to a large negative number - and a negative estimate passes checkBufferBudget() unconditionally, because it satisfies `estimatedBytes <= budget` for any budget. The check would be silently disabled by exactly the input it exists to refuse. No current caller can reach it: every estimate here is bounded by an int-sized count, so the product never saturates in the first place. This closes the shape rather than an instance, which is the point - the helper is now general enough that the next caller's per-entry size may not be 4 bytes. New saturatingSum(), the companion to saturatingProduct(), used at both sites that mix the two. AlgoNode2Vec's bytesPerWalk had the same raw-arithmetic shape and the same proof that it cannot overflow today; it is written the same way so a reader does not have to redo that proof per call site to know which ones are safe.
|
Round 3 actioned ( The non-saturating addition — valid, fixed. You're right that I also applied it to The Suite: 634 green across |
ReviewI read through the full diff (14 procedure files, Code quality / correctness
Minor / stylistic
Test coverageVery thorough: zero/negative rejection, minimum-accepted over-reach guard, interrupt abort, deadline abort, and CSR-kernel-specific direct tests, all parameterized across the fourteen procedures. Two things stood out as good practice:
One coverage gap worth a sentence in the class javadoc if not already there: the negative-count test ( Security / performanceNo concerns. The added checks are cheap (an Nice follow-through on #6216's pattern, including catching the |
…oth, and the negative-count sampling is stated From review, both comment-only. AlgoPersonalizedPageRank and AlgoArticleRank are the only two whose OLTP path duplicates the CSR path's loop, and only their CSR copy carried the note saying why the checkpoint is where it is - so the OLTP copy read like an unexplained call in a PR where every other checkpoint explains itself. Both now say it, and so does the fallback branch inside the CSR path. The negative-count test samples 3 of the 14 procedures. That was deliberate and is now written down: what separates a negative value from zero lives entirely in the shared extractInt(value, name, minimum), and the per-procedure half - that the knob is extracted with a minimum at all - is what the zero case covers for every one of them. Fourteen rows there would re-test one shared comparison fourteen times. No behaviour change; 634 tests green across the algo and olap packages.
|
Round 4 actioned ( The uncommented checkpoints. You spotted the real cause of the inconsistency: The negative-count sampling. Written down rather than left to be re-derived: what separates a negative value from zero lives entirely in the shared 634 green across From my side this is complete: four review rounds, every actionable point either implemented (GraphSAGE as a fourteenth instance, the SimRank value-pinning test, |
Review of #6288I read through the full diff ( Correctness spot-checks that passed:
Minor, non-blocking observations:
Test coverage: the new No security or dependency concerns - no new third-party dependencies, and the changes only add validation/abort paths rather than removing any. Nice work - this closes a real gap (silent wrong answers plus unabortable OLAP kernels) and the test suite gives good confidence it does not regress the algorithms' actual output. |
|
Round 5 — nothing implemented, and here is the reasoning for each of the three. 1. Comment duplication — declining, and this one has history worth recording. Round 2 raised the same observation and concluded it read as intentional; round 4 then asked me to add the note to the two sites that lacked it ( 2. Release-notes conflict risk — checked, and thank you, this was the actionable one. Seven open PRs touch The one to watch is #6285 (#6263), which renames 3. Negative-count sampling — already documented in the Javadoc as deliberate, as you note. That closes every point raised across five rounds: implemented where actionable ( |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
🟢 Coverage 94.29% diff coverage · -6.57% coverage variation
Metric Results Coverage variation ✅ -6.57% coverage variation Diff coverage ✅ 94.29% diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (a03af88) 159043 125523 78.92% Head commit (c603ded) 191468 (+32425) 138531 (+13008) 72.35% (-6.57%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#6288) 105 99 94.29% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
ReviewI read through the full diff (all 14 procedure files, What looks right
Minor nit
One thing worth confirming (not a blocker)
I wasn't able to run |
#6288 (issue #6264) landed first and touched the same three places. Resolved toward one API rather than two. AbstractAlgoProcedure: #6288 generalised `checkWalkBudget` into `checkBufferBudget(db, bytes, what, detail)` so `algo.slpa` could price its label memory. That is the same idea this branch implements as `MemoryBudget`, which additionally accumulates over the call, so keeping both would leave two budget APIs on the same setting. `checkBufferBudget` and `checkWalkBudget` are gone; `algo.slpa`'s reservation goes through `MemoryBudget.reserve` and reads the same to a caller, since the message shape was already identical. Its estimate is now `matrixBytes(n, iterations + 1, INT_BYTES)` - the same figure its hand-rolled expression computed - and moved ahead of the adjacency build like the others. Both branches added `saturatingSum`; the merge kept both copies and the duplicate would not compile. #6288's version is kept, javadoc included: it documents why a footprint that mixes a saturated product with a plain addition must not wrap, which is the better explanation. Its `@link` now points at `MemoryBudget.reserve`, where the hazard it describes would land. GlobalConfiguration: one description covering both, so `algo.slpa`'s label memory is named alongside the matrices. #6288's own release-notes section referred to the setting by its old name, which this branch renames; it now says so and points forward. AlgoGraphSAGE: #6288 gave `layers` a minimum of 1, so the `if (layers > 0)` guard around the layer-matrix reservation is now unreachable-false. Reserved unconditionally. AlgoNode2Vec: this branch moved the reservation ahead of the adjacency build and #6288 made `bytesPerWalk` saturating in place. Both kept - the block moved, with the saturating arithmetic in it.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6288 +/- ##
==========================================
+ Coverage 70.12% 70.17% +0.04%
==========================================
Files 1839 1840 +1
Lines 158956 159115 +159
Branches 33534 33556 +22
==========================================
+ Hits 111463 111652 +189
+ Misses 33599 33596 -3
+ Partials 13894 13867 -27 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Closes #6264. Follow-up to #6216 (PR #6222).
#6216 established that an iteration-shaped knob needs two things - a domain minimum rejected by name, and a checkpoint inside the loop it drives - and gave both to the three procedures its parent review had named. Thirteen more carried the identical defect, untouched. Every one extracted its knob with a plain
extractInt(n, "maxIterations")and contained zeroguard.calls:algo.pageRank,algo.personalizedPageRank,algo.articleRank,algo.eigenvector,algo.hits,algo.katz,algo.louvain,algo.leiden,algo.labelPropagation,algo.slpa,algo.simRank,algo.fastrp,algo.hashgnn.Domain
CALL algo.pageRank({maxIterations: 0})returned the uniform initial rank vector as though it were a PageRank result;algo.louvainreturned every node in its own community;algo.fastrpthe untouched random projection. The silent half is the more serious one, as the issue says: an un-iterated centrality is not obviously wrong to a caller, unlike an exception. All thirteen now reject below 1, naming the procedure, the parameter and the value.maxIterationswith a minimum of 1 also matches Neo4j GDS, where it is declared@Configuration.IntegerRange(from = 1).Time
No honest ceiling exists for time, so a large value stays legal and becomes abortable instead. Each knob-driven loop calls the shared
WorkGuard(thread interrupt +arcadedb.command.timeout), and a per-node checkpoint inside each pass bounds the abort latency below a whole graph sweep. Following the rule #6222 arrived at over three review cycles - throttle only where one iteration can be cheaper than the check itself - the per-node checkpoint ischeckPeriodicallywhere a node's own work is O(deg) or O(dim), and unthrottled where it is already O(n) (Louvain'sgetCommunityDegreescan, SimRank's innervloop).Five of the thirteen have no convergence test at all - the CSR PageRank kernel,
simRank,fastrp,hashgnn,slpa- so the knob alone decided when they stopped.The two CSR delegates
algo.pageRankandalgo.labelPropagationhand a CSR-backed graph toGraphAlgorithms, which sits below the query layer and knew nothing about deadlines. Rather than couple the OLAP kernels to the query engine, both gained an overload taking the newWorkCheckpoint- a one-method interface incom.arcadedb.graph.olapthat the procedures satisfy withguard::check. Existing callers are unchanged and getWorkCheckpoint.NONE.Heap:
algo.slpaAlone among the thirteen, SLPA's
iterationsbuys memory as well as time: every node keeps a label-memory row ofiterations + 1ints, so{iterations: 1000000}on a 10k-node graph asks for 40 GB, and atInteger.MAX_VALUEtheiterations + 1wrapped toInteger.MIN_VALUEand died as a bareNegativeArraySizeExceptionnaming nothing. The footprint is now estimated in saturatinglongarithmetic and checked against the samearcadedb.cypher.algoMaxWalkMemorybudget the walk buffers use, before the first row is allocated.checkWalkBudget()becomes the walk-shaped special case of a newcheckBufferBudget().An alternative better than what the issue proposed
The issue framed this as "mechanical application" of #6216's mechanism. Two places it is not:
algo.pageRank's CSR path - the single worst offender, since that kernel never converges - unguarded, because the loop is in another package.WorkCheckpointfixes it without inverting the layering.iterationsa minimum and a time checkpoint and left{iterations: 1000000}as a 40 GB allocation. Unbounded CPU-shaped config knobs in the OpenCypher algo procedures (follow-up to #6065) #6216's actual principle is bound a knob by the resource it spends, and this knob spends two.Tests
New
Issue6264AlgoIterationKnobGuardTest, 62 cases:louvainandleiden, settle on the fixture before the clock is read - noted in the test rather than papered oversimRankHonoursTheCommandTimeoutInsideASingleIteration:maxIterations: 1on an 800-node degree-400 graph, so the outer checkpoint runs once before any work and only the per-node checkpoint can fire. Per fix(#6216): a graph-algorithm knob is bounded by the resource it spends, not by a guessed cap #6222's latency-test trap, "it throws" is non-vacuous here (the unguarded version returns a similarity rather than throwing) and the elapsed-time bound is measured from both sides: ~1.5 s guarded, ~29 s unguardedtoleranceat its default so that only the CSR path - which ignores tolerance - can still be running when the deadline hitsInteger.MAX_VALUEwith the budget disabled, and an under-budget over-reach guardSensitivity, measured against the unmodified sources (
git show HEAD:<file>for the 13 procedures only): 20 of 35 non-abort cases go red, and the 24 abort cases hang rather than fail, which is the defect itself.Full runs: 562 green across
com.arcadedb.query.opencypher.procedures.algo.*+com.arcadedb.graph.olap.*; 4963 green across allcom.arcadedb.query.opencypher.**+com.arcadedb.graph.**.Follow-ups spotted, not in scope
AlgoSLPA.mostFrequent()is O(len^2) in the listener's degree, so a supernode makes one round quadratic in a way no knob controls. Graph-driven rather than knob-driven, so it belongs with arcadedb.command.timeout is described as the timeout for commands but is honoured by only two code paths #6266's family rather than here.algo.simRankallocatesnew double[n][n]twice - node-count-driven and outside every budget, the same shape as Embedding matrices in the algo.* procedures are sized nodeCount x dimension and sit outside every budget (follow-up to #6216) #6263.🤖 Generated with Claude Code
https://claude.ai/code/session_016yjKJXMNMQeLPKdCno7cFX