Development — Release Process¶

How each binding of libn4m is versioned, gated, and published. Some paths are automated (PyPI, CRAN-tarball build); the JS / MATLAB / Octave bindings are published manually and are documented in full below.

Binding → registry → automation¶

Binding	Package	Registry	Automation	Trigger
Python (full)	`nirs4all-methods`	PyPI	Automated — `release-wheels.yml` (cibuildwheel matrix + Trusted Publishing) publishes the `nirs4all-methods` project	push tag `v*` (non-`-rc`) → PyPI; `workflow_dispatch` + `publish=true`
Python (slim)	`pls4all`	PyPI	Automated — `release-python.yml` (its own Trusted Publisher) publishes the `pls4all` project: sdist + cibuildwheel + retag-to-py3 + TestPyPI/PyPI + post-publish smoke. The two Python workflows are split one-per-PyPI-project (no collision), each with its own one-to-one Trusted Publisher.	push tag `v*` (non-`-rc`) → PyPI; `workflow_dispatch` + `publish=true`
R	`n4m`	CRAN	Semi-automated — `release-r.yml` vendors libn4m into `src/vendor/`, runs `R CMD check --as-cran` on the Linux/macOS/Windows + release/devel matrix, and (on tag push) attaches the tarball to the GitHub Release. Submission is the irreducible manual web form.	`workflow_dispatch`; tag push attaches the tarball
R	`pls4all` (slim)	CRAN	Semi-automated — same `release-r.yml`, the matrix has a `pkg: [n4m, pls4all]` leg.	`workflow_dispatch`; tag push attaches the tarball
JS / WASM	`@nirs4all/methods-wasm`	npm	Build CI-automated in `cross-binding-parity.yml` (emsdk pinned, `npm test` parity); publish manual (this doc)	—
MATLAB / Octave	`+pls4all`	GitHub Release	Automated — `release-matlab.yml` attaches `nirs4all-methods-matlab-octave-<version>.zip` to the Release. ONE binding serves both: users build the MEX with `build_mex.m` (MATLAB) or `mkoctfile` via `build_mex.m` (Octave); see `bindings/matlab/COMPAT.md`. A File Exchange / Octave Forge listing is optional + manual.	tag push attaches the zip

Exact release artifacts — what each binding ships, and where to upload it¶

Every artifact below is also attached to the GitHub Release for the tag — https://github.com/GBeurier/nirs4all-methods/releases/tag/v0.99.0 — so all of them are downloadable from one place.

Binding	Registry	Exact file(s)	Upload
Python `nirs4all-methods`	PyPI	`nirs4all_methods-0.99.0-*.whl` (cibuildwheel matrix: Linux x86_64/aarch64, macOS x86_64/arm64, Windows x86_64)	Automated — Trusted Publishing, no manual upload
Python `pls4all`	PyPI	`pls4all-0.99.0-py3-none-*.whl` + `pls4all-0.99.0.tar.gz` (sdist)	Automated — Trusted Publishing, no manual upload
R `n4m`	CRAN	`n4m_0.99.0.tar.gz` (source tarball)	Manual — web form (see R → CRAN below)
R `pls4all`	CRAN	`pls4all_0.99.0.tar.gz` (source tarball)	Manual — web form
R `n4m` + `pls4all`	R-universe	— (built from Git, no upload)	Automated — registry repo + app (see R → R-universe)
JS / WASM `@nirs4all/methods-wasm`	npm	the staged `dist/` package (via `npm publish`)	Automated — `release-npm.yml` (needs `NPM_TOKEN` — see JS → npm)
MATLAB / Octave `+pls4all`	GitHub Release	`nirs4all-methods-matlab-octave-<version>.zip` — the `bindings/matlab/` source (`+pls4all` package + `build_mex.m` + MEX sources + tests). ONE package for both; download + run `build_mex.m`.	Automated — `release-matlab.yml` attaches it (File Exchange / Octave Forge optional + manual)
Source + provenance	GitHub Release	`nirs4all-methods-0.99.0-src.tar.gz` · `…-src.zip` · `nirs4all-methods-0.99.0.cdx.json` (SBOM) · `SHA256SUMS`	Automated — `release-source.yml`

For R/CRAN, upload the source .tar.gz only — never a binary, the GitHub repo zip, or the Python artifacts. The PyPI and npm files publish from CI (no manual upload); they are listed here only so the GitHub Release carries every artifact.

Pre-release gates (release blockers)¶

These three CI checks block any release; run them before tagging or publishing anything (see CLAUDE.md → “Release / ABI gates”):

Version sync — scripts/bump_version.sh --check. The canonical version lives in cpp/include/n4m/n4m_version.h; the script syncs it into every active downstream manifest (bindings/python/pyproject.toml, parity/python_generator/pyproject.toml, bindings/r/n4m/DESCRIPTION, bindings/js/package.json + package-lock.json). MATLAB/Octave read the version from libn4m at runtime, so they have no manifest to sync; archived bindings under bindings/_archive/ are frozen and excluded. Bump with bump_version.sh --bump X.Y.Z.
ABI symbol surface — the exported n4m_* set must match cpp/abi/expected_symbols_{linux,macos,windows}.txt exactly.
SONAME / linkage — readelf -d sanity on the shared object.

Also confirm the cross-binding parity gate (cross-binding-parity.yml) and the self-consistency gate (make parity-paper-only) are green for the build you ship.

Automated paths (summary)¶

Each PyPI project gets its own workflow — one workflow per package, so PyPI Trusted Publishing stays one-to-one and the previous dual-publish collision on the pls4all name is structurally impossible:

PyPI (pls4all) — tag vX.Y.Z (non--rc) → release-python.yml builds the cibuildwheel matrix from bindings/python/ directly, retags wheels to py3-none-${platform} (pure-Python over ctypes-loaded libn4m; no per-cpython ABI), runs the sklearn parity gate + installed-wheel smoke, publishes via Trusted Publishing, then re-installs from PyPI to verify propagation. -rcN tags route to TestPyPI on workflow_dispatch.
PyPI (nirs4all-methods) — tag vX.Y.Z (non--rc) → release-wheels.yml builds the cibuildwheel matrix (Linux x86_64 + aarch64, macOS x86_64 + arm64, Windows x86_64 across cp310–cp313) from the generated bindings/python_nirs4all_methods/ dir (make_python_package.py --name nirs4all-methods writes the dir; prepare_wheel_packages.sh builds + stages libn4m into n4m/lib/ inside each cibuildwheel env so the bundled lib matches the wheel). Repairs use auditwheel / delocate / delvewheel --analyze-existing. Publishes via Trusted Publishing.
CRAN (both n4m and pls4all) — release-r.yml (workflow_dispatch, also attaches on tag push) vendors the full libn4m C/C++/Fortran core + static n4m_export.h into src/vendor/ via N4M_R_VENDOR=1 ./configure, then runs R CMD check --as-cran across the {pkg: n4m, pls4all} × {linux-release, linux-devel, macos-arm64-release, windows-release} matrix and produces a self-contained source tarball. CRAN submission itself is the irreducible manual step: download the tarball from the run (or the attached Release asset) and submit it at https://cran.r-project.org/submit.html.

Manual binding publication¶

The MATLAB/Octave package is attached to the GitHub Release automatically by release-matlab.yml (nirs4all-methods-matlab-octave-<version>.zip); the optional File Exchange / Octave Forge listing is still manual. The JS/WASM package is built + published to npm by release-npm.yml (@nirs4all/methods-wasm). Each binding’s artifact is built from the same libn4m source at the released version; always run the pre-release gates first.

JS → npm (`@nirs4all/methods-wasm`)¶

The npm package ships the Emscripten WASM module + the TypeScript wrappers; the npm run build step only runs tsc, so the WASM artifacts must be built first.

# 0. Gates: scripts/bump_version.sh --check   (syncs bindings/js/package.json)

# 1. Build the WASM module (requires the Emscripten SDK on PATH).
source /path/to/emsdk/emsdk_env.sh
cmake --preset emscripten
cmake --build --preset emscripten --target pls4all_wasm
#    → build/emscripten/bindings/js/{n4m.js,n4m.wasm}

cd bindings/js
npm run build           # tsc -p .  → dist/index.js + dist/index.d.ts (TS only)

# 2. STAGE the WASM artifacts into dist/ — `npm run build` runs only tsc, and
#    package.json ships `files: ["dist/"]`. The `stage:wasm` script copies the
#    Emscripten artifacts in (and `prepack` runs build + stage:wasm for you):
npm run stage:wasm      # → dist/n4m.js + dist/n4m.wasm

# 3. Verify the tarball actually contains index.js + n4m.wasm + n4m.js BEFORE
#    publishing, and run the smoke test:
npm pack --dry-run      # inspect the file list
npm test                # node test/run_smoke.mjs — must pass

# 4. Publish (scoped public package; needs npm login + 2FA OTP).
npm publish --access public

Notes: the version is already correct if bump_version.sh --check passed (it edits package.json + package-lock.json). The scope @nirs4all must exist on the npm org and the publisher must be a member.

One-time npm registration (required for the automated `release-npm.yml`)¶

release-npm.yml publishes @nirs4all/methods-wasm automatically on a non--rc v* tag — it builds the WASM (pinned setup-emsdk), then runs npm publish with NODE_AUTH_TOKEN=${{ secrets.NPM_TOKEN }} + provenance. Until the scope + token below exist, that one leg fails (the PyPI / R / source legs are independent and still succeed). To enable it:

Own the @nirs4all scope on npmjs.com — sign in as the maintainer, Add Organization → create the free org nirs4all (so the scope @nirs4all is yours). The publishing account must be a member, and the package name @nirs4all/methods-wasm must be free/owned.
Generate an automation token — npmjs.com → Access Tokens → Generate New Token → Granular Access (or Automation), granting Read and write on the @nirs4all scope / @nirs4all/methods-wasm package. Copy it.
Add it as a GitHub Actions secret — repo Settings → Secrets and variables → Actions → New repository secret, name NPM_TOKEN, value = the token.
Publish — either re-run release-npm.yml (Run workflow → publish=true) for the already-cut v0.99.0, or it publishes automatically on the next tag.

release-npm.yml requests id-token: write, so once the scope + token exist the package publishes with a verified npm provenance attestation. The WASM staging is handled by the package’s prepack script in CI — no manual step.

MATLAB → File Exchange (`+pls4all`)¶

MATLAB has no package registry with a CLI publish; distribution is a MATLAB File Exchange listing (typically linked to the GitHub repo) and/or a packaged .mltbx toolbox.

% 0. Build the MEX dispatcher against libn4m (see bindings/matlab/README.md).
cd bindings/matlab
build_mex            % build_mex.m — compiles the n4m_*_mex entry points

Then either link the GitHub repository from a File Exchange entry, or package a redistributable .mltbx. There is no committed Toolbox project file — to build a .mltbx you first create one (MATLAB Add-Ons → Package Toolbox, which writes a .prj), then matlab.addons.toolbox.packageToolbox('<that>.prj', 'pls4all.mltbx'). CI does not run MATLAB (no licensed runner); confirm MATLAB-specific behaviour against bindings/matlab/COMPAT.md before publishing. The +pls4all package reads its version from libn4m at runtime.

Octave (`pls4all`)¶

The Octave surface is the same bindings/matlab/ binding (built for the MATLAB ∩ Octave intersection); Octave builds the MEX via build_mex.m / mkoctfile — there is no dedicated CMake Octave target. The build is CI-tested on every push (the octave-mex job in cross-binding-parity.yml installs apt Octave, runs build_mex.m, and gates on test_parity with rmse_rel <= 1e-12; locally observed ~4e-16). What is still manual is publication (no Octave Forge submission is wired): ship the built binding with the GitHub Release, or have users build it locally against the released libn4m.

Why JS / MATLAB / Octave are manual today¶

Their builds are CI-automated in cross-binding-parity.yml (JS via emsdk + npm test, Octave via apt octave + build_mex.m + test_parity). What is still manual is publication: an npm publish job, a .mltbx build job, or an Octave Forge submission. None of those are required for the cross-binding promise (one libn4m → identical numbers in every binding) — they are distribution-channel ergonomics, tracked but not blocking. MATLAB itself stays out of CI entirely (no licensed runner); the Octave job validates the MATLAB ∩ Octave intersection per bindings/matlab/COMPAT.md.

R → R-universe (registration)¶

R-universe builds binaries (Windows/macOS/Linux) straight from Git — no review, no submission. Users then install.packages("n4m", repos = "https://gbeurier.r-universe.dev").

Registry repo (done): public https://github.com/GBeurier/GBeurier.r-universe.dev with packages.json listing both packages by subdir:

[
  { "package": "n4m",     "url": "https://github.com/GBeurier/nirs4all-methods", "subdir": "bindings/r/n4m" },
  { "package": "pls4all", "url": "https://github.com/GBeurier/nirs4all-methods", "subdir": "bindings/r/pls4all" }
]

No branch field → it tracks main (the R packages + .prepare hooks are on main post-merge).

GitHub App (one manual browser step — cannot be scripted): install https://github.com/apps/r-universe on the GBeurier account (grant All repositories, or at least GBeurier.r-universe.dev + nirs4all-methods). It is required — it lets R-universe build the universe and post status. Builds (re)trigger on any commit to the registry repo and auto-rebuild every 30 days.
Self-contained build (already wired): R-universe clones the whole monorepo and runs bindings/r/{n4m,pls4all}/.prepare before R CMD build (while the top-level cpp/ is on disk); the hook runs N4M_R_VENDOR=1 ./configure to vendor the core into src/vendor/ → a self-contained ~1 MB tarball (233 TUs, far under R-universe’s 100 MB cap).
Verify: watch https://gbeurier.r-universe.dev (it shows the R CMD check result but, unlike CRAN, does not block on a NOTE/WARNING).

R → CRAN (submission)¶

CRAN is the canonical R repo (install.packages("n4m") with default repos); submission is a manual web form with human review.

Build + check (automated)¶

release-r.yml (tag push, or workflow_dispatch) runs R CMD check --as-cran for both packages across {n4m, pls4all} × {linux-release, linux-devel, macos-arm64-release, windows-release}, re-vendors fresh via .prepare, and — on a non--rc v* tag — attaches n4m_0.99.0.tar.gz + pls4all_0.99.0.tar.gz to the GitHub Release. Build/attach are gated on --as-cran passing. Verified locally: both packages 0 ERROR / 0 WARNING (4 expected NOTEs).

Optional pre-submission smoke (recommended for a first submission):

devtools::check_win_devel("bindings/r/n4m")   # and bindings/r/pls4all
devtools::check_mac_release("bindings/r/n4m")
rhub::rhub_check("bindings/r/n4m")

Exactly what to upload¶

The R source tarball only — one submission per package:

n4m_0.99.0.tar.gz
pls4all_0.99.0.tar.gz

Get them from the v0.99.0 Release assets: the files are named exactly n4m_<version>.tar.gz and pls4all_<version>.tar.gz (attached by release-r.yml). There is no -cran-tarball suffix on the Release — download n4m_0.99.0.tar.gz and pls4all_0.99.0.tar.gz directly and upload those.

<pkg>-cran-tarball is only the GitHub Actions artifact name that wraps the same <pkg>_<version>.tar.gz on a release-r.yml Run workflow run — relevant only if you download from the Actions run instead of the Release.

You can also build the tarball locally (cd bindings/r/n4m && sh .prepare && R CMD build .).

The submission form — exact values¶

At https://cran.r-project.org/submit.html (submit n4m first, then pls4all):

Field	Value
Your name	`Gregory Beurier`
Your email	`gregory.beurier@cirad.fr` (matches the `Maintainer:` in DESCRIPTION)
Upload	the package’s `.tar.gz`
Optional comment	paste the matching block below

cran-comments.md is .Rbuildignored (not in the tarball), so the comment box is where these notes go:

— paste for n4m —

New submission.

n4m 0.99.0 — a portable Partial Least Squares (PLS) and Near-Infrared
Spectroscopy (NIRS) engine. The C++17/C/Fortran numerical core (233 vendored
translation units under src/vendor/) is compiled from source at install time;
no external system library is required. License: CeCILL-2.1 (a GPL-compatible
French free-software license, in R's license database). Imports: stats only.

SystemRequirements: GNU make — the Makevars use $(shell find ...) to enumerate
the vendored sources without hard-coding each filename.

Test environments:
- R CMD check --as-cran, R 4.3.3 (Ubuntu): 0 ERRORs, 0 WARNINGs (vignettes built,
  checkbashisms installed); 4 expected NOTEs only.
- CI matrix (R release + devel) via GitHub Actions: Ubuntu 22.04, macOS 14
  (arm64), Windows Server 2022.
- win-builder (devel + release) and R-hub v2 are run before submission.

Notes (all expected):
- New submission (first upload).
- GNU make is declared in SystemRequirements.
- Any 'compilation flags' NOTE on a local build comes from the host R's Makeconf
  (e.g. conda's -march=nocona), not from the package Makevars, which use no
  -O3 / -march=native / -Werror.
- The optional CUDA backend (cuda_dispatch.cpp) is intentionally excluded from
  the R build; the package exposes the portable scalar code path only.

Maintainer: Gregory Beurier <gregory.beurier@cirad.fr> (CIRAD).

— paste for pls4all —

New submission.

pls4all 0.99.0 — a portable Partial Least Squares engine for chemometrics: the
slim, PLS-focused distribution carved from the nirs4all-methods library. The
C++17/C/Fortran numerical core (233 vendored translation units under src/vendor/)
is compiled from source at install time; no external system library is required.
License: CeCILL-2.1 (a GPL-compatible French free-software license, in R's
license database). Imports: stats only.

SystemRequirements: GNU make — the Makevars use $(shell find ...) to enumerate
the vendored sources without hard-coding each filename.

Test environments:
- R CMD check --as-cran, R 4.3.3 (Ubuntu): 0 ERRORs, 0 WARNINGs (vignettes built,
  checkbashisms installed); 4 expected NOTEs only.
- CI matrix (R release + devel) via GitHub Actions: Ubuntu 22.04, macOS 14
  (arm64), Windows Server 2022.
- win-builder (devel + release) and R-hub v2 are run before submission.

Notes (all expected):
- New submission (first upload).
- GNU make is declared in SystemRequirements.
- Any 'compilation flags' NOTE on a local build comes from the host R's Makeconf
  (e.g. conda's -march=nocona), not from the package Makevars, which use no
  -O3 / -march=native / -Werror.
- The optional CUDA backend (cuda_dispatch.cpp) is intentionally excluded from
  the R build; the package exposes the portable scalar code path only.

Maintainer: Gregory Beurier <gregory.beurier@cirad.fr> (CIRAD).

After uploading, CRAN emails a confirmation link — click it to complete. For a new package, expect the automated incoming checks then a human reviewer.

Operational notes (lessons from the 0.99.0 multi-registry release)¶

npm needs an Automation token. A regular/Granular token without “bypass 2FA” fails npm publish with 403 — 2FA required (or E404 on the scoped PUT if it lacks @nirs4all org write). Use a classic Automation token (or a Granular token with read+write on the @nirs4all org) as the NPM_TOKEN secret.
release-npm.yml dispatch defaults to a dry run. Its publish input is default: false — pass -f publish=true to actually publish: gh workflow run release-npm.yml --ref main -f publish=true.
Re-run a publish on main, not by replaying an old tag. gh run rerun <id> on a tag-triggered run replays the tagged commit; re-running the v0.99.0 npm run failed with No such build preset: emscripten because that tag predates the preset. Dispatch the workflow on main instead. (gh run rerun also has no --failed flag in current gh.)
pls4all Trusted Publisher = repo nirs4all-methods, workflow release-python.yml, env pypi (NOT the historical GBeurier/pls4all repo).
vX.Y.Z tags publish only if non-pre-release (no -); the PyPI/Release jobs gate on !contains(ref, '-').