boosting_pls — Boosting PLS

Group: Ensemble · Registry tolerance: 1e-06

Description

Boosting PLS (§20)

From the pls4all.sklearn.BoostingPLSRegression docstring:

Boosted PLS regression.

Registry note — R mboost::glmboost(family=Gaussian()) — componentwise L2-Boost with a univariate linear base learner. pls4all’s default now mirrors this convention exactly (centred X, empirical Y-mean offset, greedy SSR-reduction feature selection), giving bit-for-bit parity. The original PLS-weak-learner boosting kernel is opt-in via legacy=True.

Parameters

Name

Type

Default

Notes

n_components

int

2

Number of latent components extracted (k).

n_estimators

int

50

Number of base PLS sub-models in the ensemble.

learning_rate

float

0.1

Boosting shrinkage applied to each successive base learner.

Explanations

Bibliographic source

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of Statistics 29(5), 1189–1232. — adapted for PLS as a base learner.

Mathematical principle

Gradient boosting builds an additive predictor \(F_M(\mathbf{x}) = \sum_{m=1}^M \eta\, h_m(\mathbf{x})\) where each weak learner \(h_m\) is fit on the negative gradient (the residuals, for squared-error loss) of the current ensemble. With PLS as the weak learner, each \(h_m\) is a small (\(k\)-component) PLS fitted on the pseudo-response \(r_i^{(m)} = y_i - F_{m-1}(\mathbf{x}_i)\).

The learning rate \(\eta\) (typically 0.05–0.1) and the number of boosting iterations \(M\) are the key hyperparameters; their product roughly controls the effective number of latent dimensions explored. Because boosting reduces bias, it can recover non-linear \(Y\)–\(X\) relationships even with linear PLS base learners — at the cost of much higher computational cost than a single PLS.

Implementation

n4m_boosting_pls_fit. Reference: CRAN mboost::glmboost with a PLS base learner (mboost 2.9.11).

Usage

Every pls4all binding tab dispatches into the same C kernel; the external libraries listed at the bottom of the page are the parity references registered in benchmarks.parity_timing.registry. Switch tabs to read the same fit in your language. The R package now ships drop-in-compatible facades for the CRAN pls package (plsr, pcr, mvr) and for the mdatools::pls(x, y, ...) matrix idiom — those tabs appear only on the methods that have a meaningful equivalence.

pls4all bindings

/* C ABI — libn4m */
n4m_context_t* ctx = n4m_context_create();
n4m_config_t*  cfg = n4m_config_create();
n4m_method_result_t* res = NULL;
n4m_boosting_pls_fit(ctx, cfg, &x_view, &y_view, /* hyperparams */, &res);
/* … read coefficients / mask / scores via */
/* n4m_method_result_get_double_matrix / vector / scalar … */
n4m_method_result_destroy(res);
n4m_config_destroy(cfg);
n4m_context_destroy(ctx);

import pls4all
from pls4all._methods import boosting_pls_fit
with pls4all.Context() as ctx, pls4all.Config() as cfg:
    res = boosting_pls_fit(ctx, cfg, X, y, n_components=4)
# then: res.matrix("predictions"), res.matrix("coefficients"),
# res.vector("mask"), res.scalar("intercept"), …

from pls4all.sklearn import BoostingPLSRegression
mdl = BoostingPLSRegression(n_components=2, n_estimators=50, learning_rate=0.1)
mdl.fit(X, y)
y_hat = mdl.predict(X_test)

library(pls4all)
# Unified low-level dispatcher (May 2026 R cleanup):
res <- pls4all_method("boosting_pls", X, y,
                      n_components = 4L, params = list(n_estimators = 10L, learning_rate = 0.1))
# res is a named list with MethodResult arrays/scalars.
# selected_indices / top_k_intervals are 1-based.

res = pls4all.boosting_pls(X, y, 4);
% see header of bindings/matlab/+pls4all/boosting_pls.m for full
% parameter surface:
%   res = boosting_pls(X, Y, n_components, n_estimators, learning_rate)
yhat = predict(res, Xtest);

mdl  = pls4all.fit("boosting_pls", X, y, "NumComponents", 4);
yhat = predict(mdl, Xtest);

Registry parity references 📐

  • 📐 ref.r_mboost (R · r) — mboost 2.9-11 · strict (rmse_rel ≤ 1e-06) — R mboost::glmboost(family=Gaussian()) — componentwise L2-Boost with univariate linear weak learners. pls4all’s default mirrors this exactly; bit-for-bit parity gate.

Benchmarks

Adaptive wall-clock per cell measured against full_matrix.csv. Only backends that implement this method are listed; libraries without the method are omitted.

Verdict  ·  ✓ ref / ≈ ref / ~ shape mark a reference-gate pass at strict / relaxed / qualitative tolerance  ·  ✓ bind = pls4all binding agrees with the C++ baseline  ·  ⇄ cross-check = documented by-design selector/RNG/model, noncanonical API/facade convention, or secondary oracle  ·  ✗ divergent  ·  ⚠ error  ·  — not run. The fastest backend per column is marked 🏆.

Reference gate: strict — numeric equivalence (rmse_rel_tol ≤ 1e-06).

Rows tagged with 📐 are the canonical parity references for this method (declared in parity_timing.registry). C++ and external rows show reference parity; pls4all language bindings show binding parity against the C++ backend. Hover the icon for role and tolerance band.

BackendParity200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 2e-151.23 ms🏆
Python · pls4all
pls4all.python✓ bind1.52 ms
pls4all.sklearn⇄ +2e+002.07 ms
R · pls4all
pls4all.R⇄ +2e+005.05 ms
pls4all.R.formula⇄ +2e+004.95 ms
pls4all.R.mdatools⇄ +2e+006.92 ms
pls4all.R.pls⇄ +2e+006.27 ms
R · external
📐ref.r_mboostsource18.5 ms
BackendParity200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 2e-151.25 ms🏆
Python · pls4all
pls4all.python✓ bind1.25 ms
pls4all.sklearn⇄ +2e+001.79 ms
R · pls4all
pls4all.R⇄ +2e+004.47 ms
pls4all.R.formula⇄ +2e+004.80 ms
pls4all.R.mdatools⇄ +2e+006.14 ms
pls4all.R.pls⇄ +2e+005.37 ms
R · external
📐ref.r_mboostsource16.4 ms
BackendParity200×30 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 2e-151.27 ms🏆
Python · pls4all
pls4all.python✓ bind1.70 ms
pls4all.sklearn⇄ +2e+002.85 ms
R · pls4all
pls4all.R⇄ +2e+009.14 ms
pls4all.R.formula⇄ +2e+008.90 ms
pls4all.R.mdatools⇄ +2e+0011.2 ms
pls4all.R.pls⇄ +2e+0014.2 ms
R · external
📐ref.r_mboostsource30.5 ms

See also: benchmark overview · methods index · interactive dashboard