ridge_pls — Ridge-augmented PLS

Group: Regularised · Registry tolerance: 0.1

Description

Ridge-augmented PLS

From the pls4all.sklearn.RidgePLSRegression docstring:

L2-augmented PLS regression.

Registry note — sklearn PLSRegression on the (X augmented with sqrt(λ)·I, Y augmented with zeros) is the standard data-augmentation trick for L2-penalized PLS. pls4all now defaults to NIPALS on the augmented matrix to match the reference bit-for-bit; SIMPLS on the same augmented matrix introduces a different FP reduction order and diverges by ~1e-3 on small sizes.

Parameters

Name

Type

Default

Notes

n_components

int

2

Number of latent components extracted (k).

ridge_lambda

float

1.0

L2 (ridge) penalty added to the SIMPLS augmented system.

Explanations

Bibliographic source

Hoerl, A. E. & Kennard, R. W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67. — combined with PLS via Tikhonov regularisation of the inner regression.

Mathematical principle

When the number of components \(k\) approaches the rank of \(\mathbf{X}\), the inner regression of \(\mathbf{Y}\) on the PLS scores becomes ill-conditioned. Ridge-augmented PLS adds an L2 penalty to that inner regression: \(\hat{\mathbf{Q}} = (\mathbf{T}^{\top}\mathbf{T} + \lambda \mathbf{I})^{-1}\mathbf{T}^{\top}\mathbf{Y}\), yielding a shrinkage-stabilised coefficient matrix.

Setting \(\lambda\) from cross-validation on a logarithmic grid is the standard procedure. The combined method is more forgiving than pure PLS to a slightly over-specified \(k\): pure PLS over-fits hard at \(k > k_{\mathrm{opt}}\) while ridge-augmented degrades smoothly. Conceptually it is a continuous interpolation between PLS (\(\lambda=0\)) and a heavily-regularised low-rank ridge regression in latent space.

When \(\lambda\) is set per component via the SVD spectrum of \(\mathbf{T}\), ridge PLS is closely related to Krylov-subspace PCR with shrinkage.

Implementation

n4m_ridge_pls_fit (in-sample only). No widely installable reference for this exact formulation; the test compares against an sklearn PLSRegression + manual Tikhonov inner regression.

MATLAB header (bindings/matlab/+pls4all/RidgePlsRegression.m):

pls4all.RidgePlsRegression  L2-augmented PLS regression.

Usage

Direct n4m Python helper:

import n4m

res = n4m.ridge_pls(
    X,
    y,
    n_components=4,
    ridge_lambda=0.5,
)
y_hat = res["predictions"]
coef = res["coefficients"]

The n4m.sklearn.NativeRidgePLSRegressor wrapper replays predictions from the returned coefficients plus reconstructed intercept.

Every pls4all binding tab dispatches into the same C kernel; the external libraries listed at the bottom of the page are the parity references registered in benchmarks.parity_timing.registry. Switch tabs to read the same fit in your language. The R package now ships drop-in-compatible facades for the CRAN pls package (plsr, pcr, mvr) and for the mdatools::pls(x, y, ...) matrix idiom — those tabs appear only on the methods that have a meaningful equivalence.

pls4all bindings

/* C ABI — libn4m */
n4m_context_t* ctx = n4m_context_create();
n4m_config_t*  cfg = n4m_config_create();
n4m_method_result_t* res = NULL;
n4m_ridge_pls_fit(ctx, cfg, &x_view, &y_view, /* hyperparams */, &res);
/* … read coefficients / mask / scores via */
/* n4m_method_result_get_double_matrix / vector / scalar … */
n4m_method_result_destroy(res);
n4m_config_destroy(cfg);
n4m_context_destroy(ctx);

import pls4all
from pls4all._methods import ridge_pls_fit
with pls4all.Context() as ctx, pls4all.Config() as cfg:
    res = ridge_pls_fit(ctx, cfg, X, y, n_components=4)
# then: res.matrix("predictions"), res.matrix("coefficients"),
# res.vector("mask"), res.scalar("intercept"), …

from pls4all.sklearn import RidgePLSRegression
mdl = RidgePLSRegression(n_components=2, ridge_lambda=1.0)
mdl.fit(X, y)
y_hat = mdl.predict(X_test)

library(pls4all)
# Unified low-level dispatcher (May 2026 R cleanup):
res <- pls4all_method("ridge_pls", X, y,
                      n_components = 4L, params = list(ridge_lambda = 0.5))
# res is a named list with MethodResult arrays/scalars.
# selected_indices / top_k_intervals are 1-based.

res = pls4all.ridge_pls(X, y, 4);
% see header of bindings/matlab/+pls4all/ridge_pls.m for full
% parameter surface:
%   res = ridge_pls(X, Y, n_components, ridge_lambda)
yhat = predict(res, Xtest);

mdl  = pls4all.fit("ridge_pls", X, y, "NumComponents", 4);
yhat = predict(mdl, Xtest);

Registry parity references 📐

  • 📐 ref.python_scikit_learn (python · python) — scikit-learn 1.4.2 · qualitative (rmse_rel ≤ 1e-01) — Ridge-augmented PLS via sklearn PLSRegression on the (X aug, Y aug) matrices — standard data-augmentation trick to fold an L2 penalty into a least-squares-style algorithm.

Benchmarks

Adaptive wall-clock per cell measured against full_matrix.csv. Only backends that implement this method are listed; libraries without the method are omitted.

Verdict  ·  ✓ ref / ≈ ref / ~ shape mark a reference-gate pass at strict / relaxed / qualitative tolerance  ·  ✓ bind = pls4all binding agrees with the C++ baseline  ·  ⇄ cross-check = documented by-design selector/RNG/model, noncanonical API/facade convention, or secondary oracle  ·  ✗ divergent  ·  ⚠ error  ·  — not run. The fastest backend per column is marked 🏆.

Reference gate: strict — numeric equivalence (rmse_rel_tol ≤ 1e-08).

Rows tagged with 📐 are the canonical parity references for this method (declared in parity_timing.registry). C++ and external rows show reference parity; pls4all language bindings show binding parity against the C++ backend. Hover the icon for role and tolerance band.

BackendParity200×50 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 6e-161.75 ms
Python · pls4all
pls4all.python✓ bind1.83 ms
pls4all.sklearn✓ bind1.75 ms🏆
R · pls4all
pls4all.R✓ 1e-144.62 ms
pls4all.R.formula✓ 1e-146.20 ms
pls4all.R.mdatools✓ 1e-145.56 ms
pls4all.R.pls✓ 1e-145.59 ms
Python · external
📐ref.python_scikit_learnsource2.19 ms
BackendParity200×50 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 6e-161.80 ms🏆
Python · pls4all
pls4all.python✓ bind1.80 ms
pls4all.sklearn✓ bind1.82 ms
R · pls4all
pls4all.R✓ 1e-144.67 ms
pls4all.R.formula✓ 1e-145.85 ms
pls4all.R.mdatools✓ 1e-145.48 ms
pls4all.R.pls✓ 1e-145.74 ms
Python · external
📐ref.python_scikit_learnsource2.20 ms
BackendParity200×50 (ms)
C++ native · libn4m
pls4all.cpp.blas+omp✓ ref 6e-161.83 ms🏆
Python · pls4all
pls4all.python✓ bind1.87 ms
pls4all.sklearn✓ bind1.85 ms
R · pls4all
pls4all.R✓ 1e-144.60 ms
pls4all.R.formula✓ 1e-145.61 ms
pls4all.R.mdatools✓ 1e-145.92 ms
pls4all.R.pls✓ 1e-146.40 ms
Python · external
📐ref.python_scikit_learnsource2.20 ms

See also: benchmark overview · methods index · interactive dashboard