Repository navigation
refactor: Kriging based models - #606
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is breaking. See Breaking changes below.
Rewrites the correctness-critical parts of
Kriging,GEK,GEKPLS,KPLSandKPLSK. Severalof these models were not computing what their documentation claimed; the sections below state each
defect, what it cost, and how it was verified.
What was wrong
GEKwas not a gradient-enhanced kriging model. Itsd-dimensional value–value block wasexp(-Σ θ_l Δ_l)— the difference neither squared nor absolute-valued, so the matrix was notsymmetric, not bounded by 1, and not positive definite. Its derivative–derivative block was missing
the
2θ_l δ_lmterm, leaving zeros on part of the diagonal where the true variance of∂_l fis2θ_l σ². And at prediction time it droppedthetafrom the kernel and summed only the firstnof
n(1+d)weights, discarding every gradient. The model missed its own training points by7.6e15.The predictive variance squared the wrong residual. Both
KrigingandGEKused(1 - rᵀR⁻¹r)²where the ordinary-kriging MSE (Jones 2001, eq. 5) has(1 - 𝟙ᵀR⁻¹r)². The thirdterm is the variance from estimating the trend, so its numerator is the trend residual, not the
prediction residual. This feeds straight into every acquisition function in
surrogate_optimize!.thetawas never fitted. It came from a fixed spread heuristic and stayed there. Thecorrelation scale is the parameter that determines everything about how a kriging model
interpolates.
Invalid input printed to stdout and returned
nothing. Duplicate samples, out-of-bounds points,mismatched
thetalengths, andn_comp > deither produced an uncatchable diagnostic and anothingsurrogate, or an opaqueDimensionMismatchseveral frames deep, or — forn_comp > d— asilently meaningless model whose correlation scales were fitted against all-zero PLS rotation
columns.
GEKPLSsilently answered a different question. Passing several points returned the predictionat the first one, with no error.
update!grew the caller's arrays.KPLS,KPLSKandGEKPLSallpush!ed onto the vectorsthe user had passed in.
What changed
GEKcovariance rebuilt from the analytic derivatives of the Gaussian kernel, all four blocktypes, verified exact to
1e-18againstForwardDiff.pis now validated to be2, since thederivative blocks are second derivatives that only exist for the Gaussian kernel.
max(·, 0)instead ofabs(·)so an ill-conditioned solvereports zero rather than a reflected error bar.
thetainKriging,GEKand (opt-in)GEKPLS, maximizing-n/2 log σ̂²(θ) - ½ log|R(θ)|with multi-start Nelder–Mead inlog₁₀ θover a ±5-decaderelative box. Verified against a brute-force grid.
inv(R)→ Cholesky throughout.inverse_of_Rsurvives as a deprecated property.κ = 1e8to1e12, following Mohammadi et al.(arXiv:1602.00853); one
eigvalscall instead of two full eigensolves;LAPACKExceptionhandledso an extreme scale visited by the search cannot abort the fit.
GEKPLSnugget starts at10·eps()and escalates only as far as the factorization needs,instead of a fixed
1e6·eps()on every problem — worth more than a factor of two in RMSE onwelded beam.
KPLS/KPLSKkeep the old jitter, so their fits are unchanged.Float32designs no longer promote toFloat64through unqualifiedliterals;
BigFloatexercises the generic path.ArgumentErrorfor input the model cannot accept,@warnplusa no-op for a duplicate in
update!(optimizers re-propose points near convergence, so erroringwould break every optimization loop).
vcatinstead ofpush!in everyupdate!.Tests
Kriging138 → 149,GEK116 → 119,GEKPLS35 → 40,KPLS5 → 47,KPLSK5 → 50. All pass.Tolerances tightened where the quantity is deterministic — interpolation at training points went
from
< 1e-6to< 1e-12,GEKvalues and derivatives fromatol = 1e-2to1e-8. Several ofthe bugs above survived because a tolerance of
1e-2on an interpolant evaluated at its owntraining points passes with almost anything.
New tests include independent-reference checks that share no code with the implementation:
KPLSand
KPLSKpredictions are compared against an ordinary kriging system rebuilt from the fittedfields and solved directly, and the
KrigingMSE against an augmented-Lagrangian solve of the BLUPsystem.
Breaking changes
inverse_of_Ris no longer a field. It is a deprecated property that still returnsR⁻¹with a
depwarn; the Cholesky factorization lives inR_fact. Code that reads it keeps working.Code that constructs
KrigingorGEKpositionally will break — the field list changed.thetais fitted by default. Predictions change whereverthetawas not suppliedexplicitly. They improved on every benchmark measured (by factors of 1.03 to 25.6 for
Kriging,and 140× to 1500× for
GEKon three of four problems), but they change.optimize_theta = falserestores the previous behaviour of using the heuristic as given. An explicitly supplied
thetais still honoured and is now also preserved across
update!.Invalid input throws instead of returning
nothing. Duplicate samples and out-of-boundstraining points now raise
ArgumentErrorfromKriging,GEK,KPLS,KPLSKandGEKPLS.Code that tested the return value for
nothingneeds atry/catch; code that did not wasalready broken further downstream.
GEKpredictions change completely, because the previous ones came from a covariance modelthat was not a covariance model.
GEKnow also requiresp == 2and rejects anything else,where the old default was
p = 1.p = 0is rejected forKriging. The documented interval was the closed[0, 2]; atp = 0every off-diagonal correlation is the constantexp(-θ), soRhas rank 2 and issingular for more than two samples.
n_compis range-checked inKPLSandKPLSK: it must satisfy1 ≤ n_comp ≤ d. Code thatpassed a larger value was building a model whose extra correlation scales were fitted against
all-zero rotation columns.
KPLSandKPLSKgained anoptimize_theta::Boolfield, so positional construction ofthose structs breaks. The new
optimize_thetakeyword defaults totrue, which is the previousbehaviour;
falseuses the suppliedthetaas given. UnlikeKrigingandGEKthe defaultcannot be derived from
theta— there it is an optional keyword, here a required positionalargument that is always supplied. For
KPLSK,falseskips both likelihood searches but stillexpands the reduced-space
thetato full dimension, since that expansion is the model KPLSK fitsrather than a search step. On a 300-point two-dimensional design the search costs 12.6s for
KPLSand 17.2s forKPLSK, against milliseconds without it.Made with claude code