bagging_pls — Bagging PLS

Group: Ensemble · Registry tolerance: 1e-06

Description

Bagging PLS (§20)

Registry note — sklearn BaggingRegressor(PLSRegression(scale=False), bootstrap=True, max_samples=1.0). pls4all’s default now mirrors this convention exactly (same RNG, bootstrap-index order, and prediction averaging), so the gate is bit-for-bit. The legacy single-pass C++ kernel (splitmix bootstrap + coefficient averaging) is opt-in via legacy=True.

Parameters

Name	Type	Default	Notes
n_components	int	4	registry benchmark cell value
n_estimators	int	10	registry benchmark cell value
seed	int	42	registry benchmark cell value

Explanations

Bibliographic source

Breiman, L. (1996). Bagging predictors. Machine Learning 24(2), 123–140. — adapted for PLS by various chemometric authors. Verified primary link: https://doi.org/10.1023/A:1018054314350.

Mathematical principle

Bootstrap aggregating draws \(B\) bootstrap samples \(\{(\mathbf{X}^{(b)}, \mathbf{y}^{(b)})\}_{b=1}^{B}\) from the calibration set (sampling with replacement, \(n\) rows each), fits a PLS model on each, and averages the predictions: \(\hat{y}_{\mathrm{bag}}(\mathbf{x}) = \frac{1}{B}\sum_b \hat{y}^{(b)}(\mathbf{x})\).

PLS is a high-bias / low-variance learner, so bagging rarely beats a single well-tuned PLS in pure RMSE. Its real value is inferential: the bootstrap distribution of coefficients gives non-parametric standard errors and confidence intervals that are otherwise inaccessible. The per-bag \(\mathbf{B}^{(b)}\) matrices form an empirical distribution from which posterior intervals on each feature’s contribution can be read off.

Computational cost: \(B\) times a single fit, embarrassingly parallel. Use \(B \in [50, 500]\) depending on how stable the CIs need to be.

Appropriate uses

Reducing variance by averaging PLS models fitted to bootstrap samples.

Limits and validation

Bootstrap averaging does not correct systematic bias, and training cost grows with the number of estimators.

Implementation

n4m_ensemble_bagging_pls_fit. Reference: CRAN enpls 6.1.1.

Sources and provenance

Current implementation: cpp/src/core/extra_pls.cpp.

API and bindings

C ABI (ABI 2): n4m_ensemble_bagging_pls_fit. Use the linked public header for the exact signature, configuration, and result handles.

Python: no current AST-verified public n4m re-export was found for this method. The linked C ABI above is the documented surface in this checkout.

R (source-verified): bagging_pls(formula, data, ncomp = 2L, n_estimators = 50L, seed = 0L, na.action = stats::na.omit).

The source signature has additional required inputs, so no example call is fabricated.

MATLAB / Octave (source-verified): bagging_pls(X, Y, n_components, n_estimators, seed).

The source signature has additional required inputs, so no example call is fabricated.

Registry parity references 📐

• 📐 ref.python_scikit_learn (python · python) — scikit-learn 1.8.0 · strict (rmse_rel ≤ 1e-06) — sklearn BaggingRegressor(PLSRegression(scale=False), bootstrap=True, max_samples=1.0). pls4all wraps the same sklearn objects, giving bit-for-bit parity.

Benchmarks

Archived measurement identity. Backend labels in this table are the raw IDs recorded when the benchmark ran (including historical pls4all.* IDs). They preserve measurement provenance and do not describe a current public Python, R, or MATLAB binding; use the source-verified API and bindings section above for current entry points.

Adaptive wall-clock per cell measured against full_matrix.csv. Only backends that implement this method are listed; libraries without the method are omitted.

Verdict  ·  ✓ ref / ≈ ref / ~ shape mark a reference-gate pass at strict / relaxed / qualitative tolerance  ·  ✓ bind = archived binding-harness result agrees with the C++ baseline  ·  ⇄ cross-check = documented by-design selector/RNG/model, noncanonical API/facade convention, or secondary oracle  ·  ✗ divergent  ·  ⚠ error  ·  — not run. The fastest backend per column is marked 🏆.

Reference gate: strict — numeric equivalence (rmse_rel_tol ≤ 1e-06).

Rows tagged with 📐 are the canonical parity references for this method (declared in parity_timing.registry). C++ and external rows show reference parity; archived language-harness rows show binding parity against the C++ backend. Hover the icon for role and tolerance band.

1 thread

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref	9.82 ms
Python · archived pls4all benchmark
pls4all.python	✓ bind	9.30 ms
pls4all.sklearn	⇄ +2e-01	1.69 ms🏆
R · archived pls4all benchmark
pls4all.R	⇄ +2e-01	4.62 ms
pls4all.R.formula	⇄ +2e-01	6.41 ms
pls4all.R.mdatools	⇄ +2e-01	5.11 ms
pls4all.R.pls	⇄ +2e-01	4.93 ms
Python · external
📐ref.python_scikit_learn	source	11.1 ms

3 threads

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref	21.7 ms
Python · archived pls4all benchmark
pls4all.python	✓ bind	60.8 ms
pls4all.sklearn	⇄ +2e-01	11.4 ms🏆
R · archived pls4all benchmark
pls4all.R	⇄ +2e-01	25.2 ms
pls4all.R.formula	⇄ +2e-01	40.6 ms
pls4all.R.mdatools	⇄ +2e-01	46.3 ms
pls4all.R.pls	⇄ +2e-01	45.0 ms
Python · external
📐ref.python_scikit_learn	source	56.1 ms

10 threads

Backend	Parity	200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp	✓ ref	27.8 ms
Python · archived pls4all benchmark
pls4all.python	✓ bind	12.7 ms
pls4all.sklearn	⇄ +2e-01	2.54 ms🏆
R · archived pls4all benchmark
pls4all.R	⇄ +2e-01	5.73 ms
pls4all.R.formula	⇄ +2e-01	6.42 ms
pls4all.R.mdatools	⇄ +2e-01	7.63 ms
pls4all.R.pls	⇄ +2e-01	7.41 ms
Python · external
📐ref.python_scikit_learn	source	14.2 ms

---

See also: benchmark overview · methods index · interactive dashboard