missing_aware_nipals — Missing-aware NIPALS

Group: Missing data · Registry tolerance: 1e-08

Description

Missing-aware NIPALS PLS (§13)

Registry note — R softImpute + pls::plsr reference: matrix completion + SIMPLS. Different imputation algorithm than Nelson 1996 NIPALS-missing; same goal. Cell has no missing values so softImpute reduces to mean-fill; widened tolerance flags ref presence.

Parameters

Name	Type	Default	Notes
n_components	int	4	registry benchmark cell value

Explanations

Bibliographic source

Walczak, B. & Massart, D. L. (2001). Dealing with missing data: Part I and Part II. Chemometrics and Intelligent Laboratory Systems 58(1), 15–27 and 29–42. DOIs 10.1016/S0169-7439(01)00131-9 and 10.1016/S0169-7439(01)00132-0.

Mathematical principle

Standard PLS cannot ingest NaN values: the matrix operations propagate NaN. Missing-aware NIPALS treats missing entries as excluded from the inner products in the iterative loading-weight calculation. Concretely, the score \(t_i = \sum_{j \in \mathcal{O}_i} x_{ij} w_j\) and the loading \(p_j = \sum_{i \in \mathcal{O}_j} x_{ij} t_i / \sum_{i \in \mathcal{O}_j} t_i^2\) are computed only over the observed indices \(\mathcal{O}_{i}, \mathcal{O}_{j}\).

Convergence is slower than vanilla NIPALS and the result is sensitive to the missingness pattern, but the alternative (delete rows with any NaN or impute by column means) usually performs much worse on spectroscopic data. Acceptable up to ~10 % missing entries; beyond that low-rank matrix completion (softImpute) is a better front-end.

Note that missing-aware NIPALS produces a regression model that can still predict on complete new \(\mathbf{x}\); only the training step tolerates missing data.

Appropriate uses

Latent regression when predictor matrices contain scattered missing entries and complete-case deletion is undesirable.

Limits and validation

Pairwise updates can use different samples for different products, and convergence deteriorates with structured or extensive missingness.

Implementation

n4m_estimators_missing_aware_nipals_fit. Reference: R softImpute 1.4.3 for the imputation step; the PLS fitting itself is in-tree.

Sources and provenance

Current implementation: cpp/src/core/model.cpp.

API and bindings

C ABI (ABI 2): n4m_estimators_missing_aware_nipals_fit. Use the linked public header for the exact signature, configuration, and result handles.

Python: no current AST-verified public n4m re-export was found for this method. The linked C ABI above is the documented surface in this checkout.

R (source-verified): missing_aware_nipals(formula, data, ncomp = 2L, na.action = stats::na.pass).

The source signature has additional required inputs, so no example call is fabricated.

MATLAB / Octave (source-verified): missing_aware_nipals(X, Y, n_components).

The source signature has additional required inputs, so no example call is fabricated.

Registry parity references 📐

• 📐 ref.r_softimpute (R · r) — softImpute 1.4-1 · strict (rmse_rel ≤ 1e-08) — R softImpute::softImpute followed by pls::plsr on the completed (X, Y). Different imputation algorithm than Nelson 1996 NIPALS-missing; same goal.

Benchmarks

Archived measurement identity. Backend labels in this table are the raw IDs recorded when the benchmark ran (including historical pls4all.* IDs). They preserve measurement provenance and do not describe a current public Python, R, or MATLAB binding; use the source-verified API and bindings section above for current entry points.

Adaptive wall-clock per cell measured against full_matrix.csv. Only backends that implement this method are listed; libraries without the method are omitted.

Verdict  ·  ✓ ref / ≈ ref / ~ shape mark a reference-gate pass at strict / relaxed / qualitative tolerance  ·  ✓ bind = archived binding-harness result agrees with the C++ baseline  ·  ⇄ cross-check = documented by-design selector/RNG/model, noncanonical API/facade convention, or secondary oracle  ·  ✗ divergent  ·  ⚠ error  ·  — not run. The fastest backend per column is marked 🏆.

Reference gate: strict — numeric equivalence (rmse_rel_tol ≤ 1e-08).

Rows tagged with 📐 are the canonical parity references for this method (declared in parity_timing.registry). C++ and external rows show reference parity; archived language-harness rows show binding parity against the C++ backend. Hover the icon for role and tolerance band.

1 thread

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref 8e-15	1.27 ms
Python · archived pls4all benchmark
pls4all.python	✓ bind	1.25 ms🏆
pls4all.sklearn	✓ bind	1.39 ms
R · archived pls4all benchmark
pls4all.R	✓ bind	4.25 ms
pls4all.R.formula	✓ bind	4.88 ms
pls4all.R.mdatools	✓ bind	5.41 ms
pls4all.R.pls	✓ bind	4.85 ms
R · external
📐ref.r_softimpute	source	13.6 ms

3 threads

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref 8e-15	1.22 ms🏆
Python · archived pls4all benchmark
pls4all.python	✓ bind	1.36 ms
pls4all.sklearn	✓ bind	1.37 ms
R · archived pls4all benchmark
pls4all.R	✓ bind	4.02 ms
pls4all.R.formula	✓ bind	4.84 ms
pls4all.R.mdatools	✓ bind	4.88 ms
pls4all.R.pls	✓ bind	4.63 ms
R · external
📐ref.r_softimpute	source	14.9 ms

10 threads

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref 8e-15	1.34 ms🏆
Python · archived pls4all benchmark
pls4all.python	✓ bind	1.79 ms
pls4all.sklearn	✓ bind	3.36 ms
R · archived pls4all benchmark
pls4all.R	✓ bind	12.2 ms
pls4all.R.formula	✓ bind	16.6 ms
pls4all.R.mdatools	✓ bind	12.2 ms
pls4all.R.pls	✓ bind	18.2 ms
R · external
📐ref.r_softimpute	source	29.9 ms

---

See also: benchmark overview · methods index · interactive dashboard