apbase.cross_validate

Leave-one-out cross-validation for IDW, ordinary kriging, and cokriging.

This package exports the public functions used to evaluate IDW/kriging and collocated/ICM/LMC cokriging accuracy (MAE, RMSE, R2) on the same source dataset, plus select_best_model to automatically pick the statistically better method from either result.

class apbase.cross_validate.BestModelResult(method, metric, idw_score, kriging_score, margin, relative_margin, significant, confidence_level, ci_low, ci_high, n_common_valid)

Bases: object

Statistically-supported IDW-vs-kriging pick.

Variables:

  • method ({"idw", "kriging"}) – Selected method.

  • metric ({"rmse", "mae"}) – Metric used for comparison.

  • kriging_score (idw_score,) – Paired scores over common finite held-out samples.

  • relative_margin (margin,) – Absolute and relative improvement of the selected method.

  • significant (bool) – Whether the bootstrap confidence interval excludes zero.

  • confidence_level (float) – Confidence level used for the interval.

  • ci_high (ci_low,) – Bounds of the idw-minus-kriging score difference.

  • n_common_valid (int) – Number of paired finite samples used for comparison.

Parameters:

  • method (Literal['idw', 'kriging'])

  • metric (Literal['rmse', 'mae'])

  • idw_score (float)

  • kriging_score (float)

  • margin (float)

  • relative_margin (float)

  • significant (bool)

  • confidence_level (float)

  • ci_low (float)

  • ci_high (float)

  • n_common_valid (int)

method: Literal['idw', 'kriging']
metric: Literal['rmse', 'mae']
idw_score: float
kriging_score: float
margin: float
relative_margin: float
significant: bool
confidence_level: float
ci_low: float
ci_high: float
n_common_valid: int
class apbase.cross_validate.CokrigingCrossValidationResult(by_method, n_samples, radius, model_params)

Bases: object

Leave-one-out cross-validation results for collocated/ICM/LMC cokriging.

See cross_validate_cokriging().

Variables:

  • by_method (dict[str, MethodCrossValidation]) – Validation results keyed by method name. A method whose model could not be fitted is absent; see cross_validate_cokriging().

  • n_samples (int) – Number of held-out primary samples evaluated.

  • radius (float) – Shared local search radius.

  • model_params (dict) – Fitted cokriging parameters keyed by method name, with the same keys as by_method.

Parameters:

  • by_method (dict[str, MethodCrossValidation])

  • n_samples (int)

  • radius (float)

  • model_params (dict[str, dict[str, Any]])

by_method: dict[str, MethodCrossValidation]
n_samples: int
radius: float
model_params: dict[str, dict[str, Any]]
class apbase.cross_validate.CrossValidationResult(idw, kriging, n_samples, radius, model_params)

Bases: object

Leave-one-out cross-validation results for IDW and ordinary kriging.

Variables:

  • kriging (idw,) – Per-method validation results.

  • n_samples (int) – Number of held-out samples evaluated.

  • radius (float) – Shared local search radius.

  • model_params (dict) – Readable fitted variogram parameters.

Parameters:
idw: MethodCrossValidation
kriging: MethodCrossValidation
n_samples: int
radius: float
model_params: dict[str, float | int | str]
class apbase.cross_validate.MethodCrossValidation(mae, rmse, r2, n_valid, predicted, actual)

Bases: object

Leave-one-out results for a single interpolation method.

Variables:

  • r2 (mae, rmse,) – Aggregate validation metrics computed over finite predictions.

  • n_valid (int) – Number of finite predictions contributing to the metrics.

  • actual (predicted,) – Per-sample predicted values and held-out actual values.

Parameters:

  • mae (float)

  • rmse (float)

  • r2 (float)

  • n_valid (int)

  • predicted (ndarray)

  • actual (ndarray)

mae: float
rmse: float
r2: float
n_valid: int
predicted: ndarray
actual: ndarray
class apbase.cross_validate.MultiMethodBestModelResult(method, runner_up, metric, scores, margin, relative_margin, significant, confidence_level, ci_low, ci_high, n_common_valid)

Bases: object

Statistically-supported best-of-N pick from a named method mapping.

Generalizes BestModelResult to more than two methods (e.g. CokrigingCrossValidationResult.by_method) – see select_best_model().

Variables:

  • runner_up (method,) – Selected method and second-best method.

  • metric ({"rmse", "mae"}) – Metric used for comparison.

  • scores (dict[str, float]) – Paired scores over common finite held-out samples.

  • relative_margin (margin,) – Absolute and relative improvement of method over runner_up.

  • significant (bool) – Whether the bootstrap confidence interval excludes zero.

  • confidence_level (float) – Confidence level used for the interval.

  • ci_high (ci_low,) – Bounds of the runner-up-minus-winner score difference.

  • n_common_valid (int) – Number of paired finite samples used for comparison.

Parameters:

  • method (str)

  • runner_up (str)

  • metric (Literal['rmse', 'mae'])

  • scores (dict[str, float])

  • margin (float)

  • relative_margin (float)

  • significant (bool)

  • confidence_level (float)

  • ci_low (float)

  • ci_high (float)

  • n_common_valid (int)

method: str
runner_up: str
metric: Literal['rmse', 'mae']
scores: dict[str, float]
margin: float
relative_margin: float
significant: bool
confidence_level: float
ci_low: float
ci_high: float
n_common_valid: int
apbase.cross_validate.cross_validate(x, y, z, *, max_points=500, seed=None, radius=None, power=2.0, max_neighbors=40, min_neighbors=3, model_values=None)

Leave-one-out cross-validate local IDW and local ordinary kriging.

Each held-out point is removed one at a time and re-estimated from the remaining source points, reusing one variogram/radius and one spatial index for both methods. Evaluation is capped at max_points held-out samples; see max_points for the sampling behavior on larger datasets.

Parameters:

  • x (ArrayLike) – One-dimensional source coordinates and values. Rows where any of them are NaN or infinite are ignored.

  • y (ArrayLike) – One-dimensional source coordinates and values. Rows where any of them are NaN or infinite are ignored.

  • z (ArrayLike) – One-dimensional source coordinates and values. Rows where any of them are NaN or infinite are ignored.

  • max_points (int) – Maximum number of source points evaluated by leave-one-out. If the (finite) dataset has more points than this, a random subsample of this size is drawn instead of evaluating every point.

  • seed (int | None) – Seed for the subsample draw when x has more than max_points finite points. None uses fresh entropy.

  • radius (float | None) – Local search radius shared by both methods. If None, derived as range from the fitted (or supplied) variogram (range / 3 with 10 000 or more points), matching the default used by IDW and Kriging.

  • power (float) – IDW distance exponent.

  • max_neighbors (int) – Neighbor bounds shared by both methods. max_neighbors is capped at apbase.config["max_neighbors_ceiling"] (default 500).

  • min_neighbors (int) – Neighbor bounds shared by both methods. max_neighbors is capped at apbase.config["max_neighbors_ceiling"] (default 500).

  • model_values (ArrayLike | None) – Optional native variogram model vector (5 values). If omitted, a variogram is fit from x, y, z.

Returns:

MAE, RMSE, and R2 for both methods, plus per-sample predictions and the held-out actual values.

Return type:

CrossValidationResult

Raises:

  • ValueError – If input arrays have inconsistent size, too few finite rows, invalid neighbor bounds, invalid radius, or invalid bootstrap/sample configuration.

  • NativeExecutionError – If the native cross-validation kernel reports a failure status.

Examples

>>> result = cross_validate(x, y, z, max_points=300)
>>> result.idw.rmse, result.kriging.rmse
apbase.cross_validate.cross_validate_cokriging(x0, y0, z0, secondaries, *, max_points=500, seed=None, radius=None, max_neighbors=40, min_neighbors=3, n_structures=2, max_colocation_dist=None, max_secondary_neighbors=None)

Leave-one-out cross-validate collocated, ICM, and LMC cokriging together.

Each of the three coregionalization models is fit once from the full dataset (same as fit()); only the primary is then held out one point at a time and re-estimated, sharing one primary spatial index and one radius across all three methods – see cross_validate() for the analogous IDW/kriging function.

Parameters:

  • x0 (ArrayLike) – Primary variable source coordinates and values.

  • y0 (ArrayLike) – Primary variable source coordinates and values.

  • z0 (ArrayLike) – Primary variable source coordinates and values.

  • secondaries (SecondaryInput) – Same accepted forms as fit().

  • max_points (int) – Maximum number of source points evaluated by leave-one-out; see cross_validate().

  • seed (int | None) – Seed for the subsample draw when there are more than max_points finite points. None uses fresh entropy.

  • radius (float | None) – Local search radius shared by all three methods and by every variable’s neighbor search. If None, derived as range (range / 3 with 10 000 or more points) from a variogram fit on the primary alone (independent of any one method’s own joint range fit).

  • max_neighbors (int) – Neighbor bounds shared by all three methods.

  • min_neighbors (int) – Neighbor bounds shared by all three methods.

  • n_structures (int) – Number of nested LMC structures (S), see CoKriging.

  • max_colocation_dist (float | None) – Pairing distance for the cross-covariances and secondary neighbor limit (ICM and LMC), see CoKriging.

  • max_secondary_neighbors (int | None) – Pairing distance for the cross-covariances and secondary neighbor limit (ICM and LMC), see CoKriging.

Returns:

MAE, RMSE, and R2 per method under by_method ("collocated", "icm", "lmc"), plus per-sample predictions and the held-out actual values (identical across methods, evaluated on the same held-out samples). A method whose model cannot be fitted is left out instead of failing the call: any method when a secondary has too few samples paired with the primary (see max_colocation_dist), and ICM or LMC when the fit is rejected as non-positive-semidefinite or invalid. The error is raised only when no method can be fitted.

Return type:

CokrigingCrossValidationResult

Raises:

  • ValueError – If input arrays, neighbor bounds, secondary variables, radius, or sample configuration are invalid.

  • NativeExecutionError – If the cross-validation kernel reports a failure status, or if no method’s model can be fitted (the first fit error is raised).

Examples

>>> result = cross_validate_cokriging(x0, y0, z0, {"ndvi": (xu, yu, zu)})
>>> result.by_method["icm"].rmse
apbase.cross_validate.select_best_model(result: CrossValidationResult, *, metric: Literal['rmse', 'mae'] = 'rmse', n_bootstrap: int = 2000, confidence_level: float = 0.95, seed: int | None = None) → BestModelResult
apbase.cross_validate.select_best_model(result: Mapping[str, MethodCrossValidation], *, metric: Literal['rmse', 'mae'] = 'rmse', n_bootstrap: int = 2000, confidence_level: float = 0.95, seed: int | None = None) → MultiMethodBestModelResult

Pick the statistically better method from two or more evaluated methods.

Given a CrossValidationResult (IDW vs. kriging), this keeps its original two-method behavior and return type unchanged – see below. Given any other named mapping of MethodCrossValidation (e.g. CokrigingCrossValidationResult.by_method, or a hand-built {"a": ..., "b": ..., "c": ...}), it generalizes to a best-vs-runner-up comparison across all of them and returns a MultiMethodBestModelResult instead.

Pick the better of IDW/kriging from a CrossValidationResult.

Compares IDW against ordinary kriging on the held-out samples where both methods produced a finite leave-one-out prediction (some samples can be finite for one method and NaN for the other – e.g. a singular local kriging system – so the two error series stay paired point-for-point). The lower-metric method wins (ties go to "idw", matching create_map()’s selection rule). A paired bootstrap over that common set then estimates a confidence interval for the score difference (idw minus kriging); significant is True when that interval excludes zero, i.e. the win is unlikely to be an artifact of the particular held-out sample draw.

Parameters:

  • result (CrossValidationResult | Mapping[str, MethodCrossValidation]) – Output of cross_validate().

  • metric (Literal['rmse', 'mae']) – Error metric used both for the point estimate and for each bootstrap replicate. "rmse" matches the metric already used by create_map() to auto-select a method. R2 is intentionally not offered here: it is monotonic in RMSE for a fixed actual-value set, but its normalization (1 - SS_res/SS_tot) depends on the evaluated subset’s variance, so it does not compare as cleanly once the two methods’ NaN patterns diverge.

  • n_bootstrap (int) – Number of paired bootstrap resamples. Memory and time scale as O(n_bootstrap * n_common_valid); the default is fast (well under a second) up to cross_validate’s default max_points=500.

  • confidence_level (float) – Two-sided confidence level for the score-difference interval, e.g. 0.95 for a 95% CI.

  • seed (int | None) – Seed for the bootstrap resample draw. None uses fresh entropy.

Returns:

The selected method plus the paired scores, margin, and bootstrap confidence interval backing that choice. For IDW vs kriging, ci_low/ci_high bound the idw-minus-kriging score difference: a positive interval means kriging scored better, a negative interval means idw scored better.

Return type:

BestModelResult or MultiMethodBestModelResult

Raises:

ValueError – If the metric/bootstrap configuration is invalid, predicted/actual arrays are inconsistent, fewer than two methods are supplied, or no paired evidence exists for comparison.

Notes

This uses a percentile bootstrap on the paired per-sample errors instead of a parametric test (e.g. a paired t-test), so it needs no distributional assumption and no SciPy dependency: n_bootstrap index draws are taken once via numpy.random.Generator.integers(), and every replicate’s RMSE/MAE is computed in one vectorized reduction, not a Python loop. Resampling reuses the same draw for both methods on each replicate, which preserves the point-to-point pairing/correlation between IDW and kriging errors – essential for a meaningful difference interval.

Any held-out sample where actual itself is non-finite is also excluded (this cannot happen from cross_validate’s own output, which is filtered upfront, but protects hand-built CrossValidationResult inputs).

If one method has zero finite predictions on the common set while the other has at least one, that method wins outright (significant=True, margin=inf, no bootstrap – there is nothing to resample). If neither method has any finite prediction, or the two methods’ finite predictions never overlap on the same held-out sample, this raises ValueError: unlike create_map’s plain RMSE tie-break, this function refuses to guess when there is no shared, paired evidence to compare.

Examples

>>> result = cross_validate(x, y, z, max_points=300)
>>> best = select_best_model(result)
>>> best.method
'kriging'
>>> best.significant
True