# AOM Methods Gap Analysis

Date: 2026-06-05

This audit separates what already exists in `nirs4all-methods` from what still
had to be added from the AOM / moment exploration.

Hard invariant: deployable methods must not route by dataset name, source name,
database name, dataset id, or equivalent identity labels.

## Already In nirs4all-methods

Native/catalogued primitives:

- AOM preprocessing and AOM global PLS selection:
  - `aom_pop.aom_preprocessing`;
  - `aom_pop.aom_pls`;
  - `aom_pop.pop_pls`.
- Model heads or neighboring model families:
  - PLS, CPPLS, Continuum Regression, ECR, Ridge / Ridge-PLS, robust and
    weighted PLS, PCR via the existing method set.
  - ABI-close Python helper functions now expose reusable moment-model heads:
    `n4m.ridge`, `n4m.cppls`, `n4m.continuum_regression`, and `n4m.ecr`.
- Preprocessing families needed by the lab evidence:
  - Savitzky-Golay;
  - finite differences / derivatives;
  - Norris-Williams;
  - detrend;
  - Gaussian smoothing;
  - SNV, MSC, EMSC, local/weighted/piecewise variants;
  - baseline correction families;
  - FCK static transformer.

Python/reference Phase 7f objects already present before this pass:

- `AOMStructuralPolicy`;
- `AOMStructuralPolicyWithPgt1200Admissions`;
- `AOMStructuralPolicyWithP700ProtocolUnified`;
- `AOMStructuralPolicyWithP700BlockLocalAdmission`;
- `AOMControlSelector`;
- `AOMTrueBankEndpointPortfolio`;
- `AOMMidPEndpointStack`;
- `AOMEndpointMarginStabilityGate`;
- `AOMFallbackBlendGate`.
- `AOMRidgeBlender`;
- `AOMOperatorPLSStack`.

## Missing Before This Pass

The target repo had primitives and selector skeletons, but it did not yet have
the product-facing method layer needed by an end user:

- no built-in AOM preprocessing chain bank reproducing the useful diversity of
  the lab campaigns;
- `AOMControlSelector` was Ridge-only;
- no direct `fit/predict` robust-HPO estimator that builds the bank, screens
  Ridge/PLS heads, and reports the selected chain;
- no compact end-user preset;
- no wide/lab-style preset for controlled reproduction of larger preprocessing
  diversity.

## Added In This Pass

Python/reference methods:

- `AOMPreprocessingChain`:
  - fold-safe sequential preprocessing transformer;
  - deep-copies and fits each step inside CV;
  - used as the standard chain object for generated banks.
- `build_aom_control_chain_bank(profile=...)`:
  - `compact`: fast end-user bank;
  - `robust`: broader default bank;
  - `wide` / `lab` / `moment_lab`: larger lab-style bank;
  - includes the recurring families: raw, row center, SNV, MSC, EMSC,
    detrend, SavGol variants, Norris-Williams, derivatives, Gaussian,
    robust/area normalization, and selected short compositions.
- `AOMControlSelector` now supports:
  - Ridge heads;
  - PLS1 heads;
  - fold-local chain fitting;
  - mean/median/quantile/max criteria;
  - rank-mean or simple ensembles over top candidates.
- `AOMRobustHPORegressor`:
  - configurable end-user estimator;
  - builds or accepts a chain bank;
  - screens Ridge and/or PLS heads;
  - exposes `selection_report_`;
  - accepts metadata for audit only.
- `AOMRobustHPOCompact`:
  - fast compact preset for users.
- `AOMRobustHPOWide`:
  - larger lab-style preset for reproducing broad preprocessing diversity;
  - not recommended as the default until benchmarked from the integrated
    package.
- `NativeAOMRobustHPORegressor`:
  - sklearn-style wrapper over the native strict-linear robust-HPO ABI;
  - uses `input_coefficients` folded into the original feature space;
  - predicts on new spectra without replaying the selected preprocessing chain
    in Python.
- `AOMRidgeBlender`:
  - fold-safe OOF simplex blend ported from the AOM-Ridge headline family;
  - supports explicit estimator/factory candidates or a default chain+Ridge
    pool;
  - Python reference layer for explicit custom candidates; the strict-linear
    compact/wide Ridge pool is also available natively as
    `n4m.aom_ridge_blender` / `n4m_aom_ridge_blender_fit`.
- `AOMOperatorPLSStack`:
  - guarded fixed-operator PLS score stack with Ridge head;
  - optional OOF baseline gate for false-positive control;
  - default operator bank is restricted to strict-linear fixed matrices;
  - Python reference layer for custom operators and baseline gates; the
    compact/wide strict-linear PLS1 score stack is also available natively as
    `n4m.aom_operator_pls_stack` / `n4m_aom_operator_pls_stack_fit`.
- Native `n4m.aom_sweep_run` / `n4m_aom_sweep_run`:
  - configurable compact/wide strict-linear AOM chain sweep;
  - screens custom Ridge lambda grids and PLS component grids;
  - accepts explicit fold ids;
  - returns a candidate table with `candidate_id`, `chain_id`, `head_id`,
    `param`, `cv_rmse`, plus selected OOF/final predictions.
- Native `n4m.aom_chain_sweep_run` / `n4m_aom_chain_sweep_run`:
  - accepts caller-provided strict-linear preprocessing chains through flat
    chain/operator/parameter offsets;
  - supports identity, detrend, Savitzky-Golay, Norris-Williams, finite
    difference, Whittaker and FCK in this fold-safe path;
  - rejects stateful or non-strict operators such as SNV/MSC/EMSC/OSC/EPO.
- AOM route policy switch:
  - `moment_policy="auto"` uses guarded exact operator-moment routes;
  - `moment_policy="materialized"` or `"legacy"` forces the old
    materialized-chain screen for route timing comparisons and small-cell
    production guards;
  - `moment_policy="force_moments"` / `"moments_only"` rejects any
    candidate-screen route that would leave operator moments.
- AOM score-only output mode:
  - `score_only=True` keeps candidate scores, selected ids, route counters and
    fold ids while skipping selected-model outputs for broad ranking passes;
  - it avoids selected-model output refits and OOF/model buffers in
    operator-moment and materialized candidate-screen routes, but does not yet
    fuse the fold-local materialized scoring fits.
- ABI-close Python wrappers for the historical native AOM selectors:
  - `n4m.aom_pls` / `n4m.aom_global_select` call `n4m_aom_global_select`;
  - `n4m.pop_pls` / `n4m.aom_per_component_select` call
    `n4m_aom_per_component_select`;
  - both build the compact strict-linear operator bank by default, accept
    caller-provided strict operators, and build fold-safe validation plans from
    explicit `fold_ids` or contiguous CV folds.
- Reusable native AOM/POP selected models:
  - both historical selectors now export `input_coefficients` and `intercept`;
  - `n4m.sklearn.NativeAOMPLSRegressor` and
    `n4m.sklearn.NativePOPPLSRegressor` predict on new `X` from those
    input-space linear states.
- Native `n4m.aom_ridge_blender` / `n4m_aom_ridge_blender_fit`:
  - blends the native compact/wide strict-linear AOM Ridge candidate pool from
    fold-local OOF predictions;
  - returns candidate predictions and simplex weights for audit;
  - returns final `input_coefficients` and `intercept` for replay in original
    feature space;
  - is wrapped by `NativeAOMRidgeBlenderRegressor` for sklearn-style reuse;
  - requires strictly positive Ridge lambdas and does not cover custom Python
    estimator candidates.
- Native `n4m.aom_operator_pls_stack` / `n4m_aom_operator_pls_stack_fit`:
  - fits fold-local PLS1 score projectors over compact/wide strict-linear AOM
    operator views;
  - selects `(n_components, alpha)` by train-only CV criterion;
  - returns final stack features and Ridge head for audit;
  - returns `input_coefficients` and `input_intercept` for direct replay in
    original feature space;
  - is wrapped by `NativeAOMOperatorPLSStackRegressor` for sklearn-style reuse.

## Still Not In nirs4all-methods

Still intentionally absent or deferred:

- broad GPU/moment grinder:
  - remains a lab capacity-audit tool, not an end-user product method;
- source-route or dataset-name lookup:
  - explicitly forbidden;
- test-tuned property routes:
  - require held-out validation before becoming a deployable selector;
- native ABI endpoints for every historical AOM reference object:
  - the Python portfolio objects are now available for the main source-free
    route/policy/gate/blend/operator-stack patterns, but most are still
    pre-ABI and outside the strict native catalog; `n4m.aom_ridge_blender` is
    the native exception for the strict-linear AOM Ridge blend, and
    `n4m.aom_operator_pls_stack` is the native exception for the strict-linear
    PLS1 score stack.
- custom batched GPU screen for hundreds of thousands of preprocessing chains:
  - not shipped; the current product method builds in CUDA-enabled `libn4m`
    and uses the existing native model path, but it is not a fused GPU grinder.
- batched sweep engine ABI:
  - `n4m_moments_*` is now shipped as a raw/centered sufficient-statistics
    layer with exact row-subset subtraction and train-only recentering;
  - `n4m_sweep_run` now ships Ridge CV plus compatible single-target
    NIPALS/regression PLS1 component screening from train/held-out moments,
    with materialized prefix fallback for other PLS regimes;
  - `n4m_aom_sweep_run` now ships a configurable AOM strict-linear compact/wide
    sweep over those Ridge/PLS grids; Ridge candidate rows use exact
    operator-moment scoring when `p <= n_train`, including in mixed Ridge+PLS
    sweeps, compatible single-target NIPALS PLS1 rows score from moments, and
    identity/SavGol/Norris/finite/Gaussian/FCK chains now have a banded moment route
    outside the old dense `p <= 48` guard; `detrend_poly` chains use an exact
    structured low-rank projection moment route and Whittaker chains use an
    exact structured pentadiagonal solve route; both can compose with those
    banded local operators; CPU `auto` routes Ridge `p > n_train` rows through
    the exact materialized dual-Ridge scorer and PLS `min_train < 4p` rows
    through the exact materialized prefix scorer while CUDA `auto` keeps the
    moment route;
  - `n4m_aom_chain_sweep_run` now ships strict-linear user-defined chain
    descriptors with the same hybrid Ridge/PLS1-moment behavior and
    materialized fallbacks, or fail-fast strict behavior with
    `moment_policy="force_moments"`;
  - batched IKPLS and fused operator-moment updates are still design/backlog
    items, so the full 200k-chain AOM moment sweep remains outside the product
    ABI.

## Current Product Recommendation

Expose three levels to users:

- `n4m.aom_robust_hpo(..., profile="compact")` for the ABI-stable fast product
  screen; use `NativeAOMRobustHPORegressor` when a fitted sklearn-style
  predictor is needed from the same native result;
- `n4m.aom_sweep_run(...)` when the user wants to control Ridge lambdas, PLS
  component grids or explicit CV folds over the same native strict-linear bank;
- `n4m.aom_chain_sweep_run(...)` when the user wants to define the
  strict-linear preprocessing chains directly; use `moment_policy` to compare
  `auto` moment routes against the legacy materialized route, or
  `force_moments` to assert that the candidate screen did not leave moments;
  use `score_only=True` for first-pass chain ranking;
- `n4m.aom_pls(...)` and `n4m.pop_pls(...)` when the user wants the older
  global/per-component AOM-PLS selectors over a compact strict operator bank;
  use `NativeAOMPLSRegressor` or `NativePOPPLSRegressor` for sklearn-style
  reuse on new spectra;
- `n4m.aom_ridge_blender(...)` when the user wants a native Ridge-only OOF
  simplex blend over the compact/wide strict-linear AOM bank; use
  `NativeAOMRidgeBlenderRegressor` for fitted reuse;
- `n4m.aom_operator_pls_stack(...)` when the user wants a native PLS1
  operator-score stack with a Ridge head over the compact/wide strict-linear
  AOM bank; use `NativeAOMOperatorPLSStackRegressor` for fitted reuse;
- `n4m.aom_robust_hpo(..., profile="wide")` for a broader native strict-linear
  screen;
- `AOMRobustHPORegressor(profile="wide")` or `AOMRobustHPOWide` when fold-local
  stateful chains are required in Python/reference experiments.

The canonical catalog now contains `aom_pop.robust_hpo`,
`aom_pop.ridge_blender`, and `aom_pop.operator_pls_stack` through public native
ABI symbols and timing rows. The broader Python portfolio/gating objects remain
reference-layer APIs until each gets a native ABI contract and timing row.
`aom_pop.robust_hpo` also exposes `input_coefficients`, `intercept`,
dimension diagnostics, and the native sklearn replay wrapper.
`aom_pop.ridge_blender` now does the same for the weighted Ridge blend.
`aom_pop.operator_pls_stack` exposes `input_coefficients` and
`input_intercept` for the folded single-target stack.