Restoring the method documentation

Scope

Every active entry in catalog/methods.yaml and every generated page under docs/methods/ must give a reader the method’s principle, appropriate uses, limits, scientific source (or an explicit statement that there is no canonical paper), implementation surface, and source provenance. Parity scores describe validation only; they do not substitute for scientific documentation.

This work changes documentation and documentation tooling only. It does not change numerical kernels, public ABI, fixtures, or benchmark results.

Source and generation model

catalog/methods.yaml remains the authoritative list of supported methods and their ABI-2 surface. The frozen docs/_extras/methods_bibliography.py is a lossless historical archive; the rendered, reviewed legacy corrections are in scientific_legacy.py. Current operator and ABI-2 records are divided by ownership across scientific_augmentation_filter_split.py, scientific_remaining.py, and scientific_aom.py. Every rendered record uses the same required fields: title, paper, principle, use_cases, limitations, implementation, and provenance.

docs/_extras/build_methods.py joins those sources to catalog entries and generates the pages and index reproducibly. public_api_docs.py AST-scans the current public Python, R, and MATLAB sources, so imports, signatures, and safe examples are derived rather than guessed. A method with no public Python wrapper receives its exact linked C-header surface. The older pls4all and n4m.sklearn paths are not presented as current APIs. Historical benchmark IDs remain visible only as archived measurement provenance. The card explorer uses the generator’s page mapping and presents a visible scientific-documentation link alongside its benchmark drill-down.

docs/_extras/method_truth_sources.json is the reviewed, host-independent snapshot of the parity-reference cards. It records the 73 registry methods, their 80 reference identifiers, versions, tolerances, and notes. The live registry resolves optional Python and R adapters, so it is not read while rendering these pages. Update this snapshot deliberately with a benchmark reference-contract change, from the benchmark-reference environment, using python3 docs/_extras/refresh_method_truth_sources.py --write; the source file records the canonical registry path and the generator validates its schema and complete method coverage.

The current reference index is generated by docs/_extras/generate_scientific_reference_index.py from the rendered records. It validates all current DOI/URL locators and source digests without inventing BibTeX. docs/bibliography.md links to that index first; its legacy section and BibTeX notes remain a clearly labelled unreviewed archive.

The generator owns catalog, registry, operator, and ABI-2 pages. The retained HPO/orchestration compatibility pages (asha, cmaes, ga_search, gp_ei, hyperband, lhs, median_pruner, optimization, pso_search, racing, random, sobol, ternary, tpe) and named legacy aliases remain manual sources under docs/methods/; the coverage test lists them explicitly so they cannot be silently removed or overwritten.

Maintenance and gates

After changing a catalog entry, binding surface, or scientific record, run:

python3 docs/_extras/build_methods.py --strict
python3 docs/_extras/generate_scientific_reference_index.py --check
python3 catalog/scripts/generate_bibliography.py --check
python3 -m pytest docs/_extras/test_methods_doc_lint.py docs/_extras/test_method_scientific_coverage.py -q
python3 -m sphinx -b html docs docs/_build/html --keep-going

The documentation checks fail for missing required scientific fields, generic placeholders, invisible Python escape characters, unbalanced inline math, duplicate record keys, stale ABI-1 or removed binding paths, catalog entries without a resolving page, generated pages without a record, invalid current references, stale snapshots, and invalid card/document links. A clean-output test generates twice and checks byte idempotence, all 209 catalog links, 178 current Python re-exports, 31 C-only surfaces, 70 R and 69 MATLAB mappings, and the 73 retained benchmark snapshots. The GitHub Pages workflow runs these checks before Sphinx and deploys only commits on main; Read the Docs consumes the committed generated pages.