Section Builders#

Decoding Sections#

coco_pipe.report.decoding provides nine section-adder functions and a one-shot factory that build report sections from ExperimentResult objects (or their serialized payloads). Every adder is also bound to Report as a fluent method.

—

1. Section Catalog#

Each adder reads from a specific ExperimentResult accessor and produces one section. Sections are silently skipped when the underlying data is absent.

Function / method

Backing accessor

add_decoding_overview()

result.meta

add_decoding_summary()

result.get_detailed_scores()

add_decoding_diagnostics()

result.get_detailed_scores() + result.get_splits()

add_decoding_performance()

result.get_roc_curve() / get_pr_curve() / get_calibration_curve()

add_decoding_temporal()

result.get_temporal_score_summary() / get_generalization_matrix()

add_decoding_statistical_assessment()

result.get_statistical_assessment()

add_decoding_neural_artifacts()

result.get_model_artifacts()

add_decoding_features()

result.get_feature_importances() / get_feature_stability()

add_decoding_topomaps()

result.get_feature_importances() + info / coords

Two private adders (_add_configuration, _add_provenance, _add_confusion_probability, _add_fit_diagnostics, _add_caveats, _add_export_inventory) handle administrative sections — they’re invoked by make_decoding_report() but aren’t exposed as part of the fluent API. See the source if you need to replicate their behavior in custom workflows.

—

2. Quickstart#

from coco_pipe.report import Report

report = (
    Report(title="Decoding")
    .add_decoding_overview(result)
    .add_decoding_summary(result)
    .add_decoding_performance(result)
    .add_decoding_temporal(result, metric="accuracy")
    .add_decoding_statistical_assessment(result)
    .add_decoding_features(result)
)
report.save("decoding.html")

Or all at once via the factory:

from coco_pipe.report import make_decoding_report

report = make_decoding_report(result, output_path="decoding.html")

—

3. Each Adder in Detail#

3.1 add_decoding_overview#

One-row context table summarising the experiment: task, sample count, feature count, observation level, inferential unit, schema version. Sourced from result.meta.

report.add_decoding_overview(result, name="Overview")

3.2 add_decoding_summary#

Per-model performance summary — mean ± std for every metric across folds. Includes a MetricsTableElement that highlights the best model per metric.

report.add_decoding_summary(result, name="Model Performance")

3.3 add_decoding_diagnostics#

Cross-validation diagnostics: per-fold scores, fold sizes, group counts (when group-based CV), and any fit/predict warnings (e.g., LinearAlgebra warnings).

report.add_decoding_diagnostics(result, model="SVM")   # filter to one model

3.4 add_decoding_performance#

ROC, precision-recall, calibration curves — interactive Plotly versions. Falls back gracefully if the result doesn’t carry probabilities.

report.add_decoding_performance(result, metric="roc_auc")

3.5 add_decoding_temporal#

For sliding / generalizing decoders: time-resolved score curves and, when present, the train-time × test-time generalization matrix heatmap.

report.add_decoding_temporal(result, metric="accuracy")

3.6 add_decoding_statistical_assessment#

The full assessment view: observed score, null distribution histogram, p-value, max-stat-corrected p-values for temporal decoders.

report.add_decoding_statistical_assessment(result, metric="accuracy")

3.7 add_decoding_neural_artifacts#

Loss / metric training curves, learning-rate schedules, and any artifacts stored by neural-network estimators (ExperimentResult.get_model_artifacts()).

report.add_decoding_neural_artifacts(result)

3.8 add_decoding_features#

Top-N feature importances, stability across folds, feature scores when a univariate selector was used. Accepts an optional feature_metadata DataFrame to annotate features with metadata (e.g., channel, frequency, descriptor family).

report.add_decoding_features(result, feature_metadata=meta_df)

3.9 add_decoding_topomaps#

Sensor-level topomaps of feature importance. Requires either an MNE Info or a coordinate table.

report.add_decoding_topomaps(
    result,
    feature_metadata=meta_df,
    info=raw.info,
)

—

4. make_decoding_report#

The factory builds a full report from an ExperimentResult in one call, calling every applicable adder in a stable order.

from coco_pipe.report import make_decoding_report

report = make_decoding_report(
    result,
    title="Decoding Report",
    feature_metadata=meta_df,        # optional, enables topomaps + annotation
    info=raw.info,                   # optional, enables topomaps
    sections="default",              # or a custom subset
    theme="paper",
    output_path="decoding.html",
)

4.1 Sections argument#

sections="default" enables every supported section. To narrow:

make_decoding_report(
    result,
    sections=["overview", "performance", "temporal"],
)

Valid section names (see DEFAULT_SECTIONS):

overview, configuration, provenance, model_summary, cv_summary,
performance, statistical, confusion_probability, temporal, features,
fit_diagnostics, caveats, export_inventory, topomaps

Unknown names raise ValueError at the call site.

4.2 Error tolerance#

Each section is wrapped in a try / except: if the underlying accessor returns None or raises, the section is logged and skipped, but the rest of the report still renders. This means you can pass a partial result without preflighting.

—

5. Loading from a Saved Result#

ExperimentResult supports JSON round-trips (save() / load()). Reports built from a loaded result are byte-identical to ones built from the in-memory version.

from coco_pipe.decoding.result import ExperimentResult
from coco_pipe.report import make_decoding_report

result = ExperimentResult.load("results/run01.json")
make_decoding_report(result, output_path="reports/run01.html")

—

6. Compared to the API Factory#

coco_pipe.report.from_experiment_result() is a thin wrapper over make_decoding_report() with a slightly different default surface (see Factories and Assembly). When in doubt, use make_decoding_report() — it accepts every argument the factory exposes.

Dimensionality-Reduction Sections#

coco_pipe.report.dim_reduction provides eleven section-adder functions and a one-shot factory for building report sections from DimReduction objects, embeddings, and MethodSelector comparison frames.

—

1. Section Catalog#

Function / method

Backing source

add_reduction_overview()

reduction.get_summary()

add_reduction_embedding()

Explicit X_emb (2-D or 3-D)

add_reduction_metrics()

reduction.get_metrics() / get_summary()["metric_records"]

add_reduction_diagnostics()

Shepard diagram (X_orig, X_emb)

add_reduction_interpretation()

get_summary()["interpretation"]

add_reduction_coranking()

diagnostics["coranking_matrix_"]

add_reduction_components()

reduction.get_components()

add_reduction_trajectory()

3-D trajectory tensor

add_reduction_trajectory_separation()

trajectory_separation output

add_reduction()

Single fully-assembled section

add_comparison()

Wide/tidy metric frame or MethodSelector

—

2. Two Levels of Granularity#

Two levels of granularity are supported depending on how much control the caller needs:

  • Coarse — Report.add_reduction() packages an overview, embedding/trajectory plot, scalar metrics, metric records, diagnostics (loss history, scree, co-ranking matrix, trajectory kinematics, separation timecourses), and interpretation into one section. One call per reduction.

  • Fine — call individual add_reduction_* adders for full layout control. Use this when assembling a custom report.

# Coarse
report.add_reduction(pca, X_emb=embedding, labels=y)

# Fine
(
    report
    .add_reduction_overview(pca, name="PCA Overview")
    .add_reduction_embedding(embedding, labels=y)
    .add_reduction_metrics(pca)
    .add_reduction_coranking(coranking_matrix)
    .add_reduction_components(pca.get_components())
)

—

3. Each Adder in Detail#

3.1 add_reduction_overview#

Key/value summary from DimReduction.get_summary — method name, n_components, random_state, capability flags, scalar quality metadata. Skips metrics / metric_records / diagnostics / interpretation blocks (they have dedicated adders).

report.add_reduction_overview(pca, name="PCA Summary")

3.2 add_reduction_embedding#

2-D scatter for X_emb.shape == (n, 2), 3-D scatter for (n, 3). Optional class labels color the points; optional metadata dict adds hover annotations.

report.add_reduction_embedding(
    X_emb, labels=class_ids,
    metadata={"group": groups, "score": scores},
    name="UMAP embedding",
)

3.3 add_reduction_metrics#

Quality metrics table (downloadable as CSV) plus a metric bar chart using coco_pipe.viz.dim_reduction.plot_metrics(). Accepts either the reducer (sourced from get_metrics() or the cached metric records) or a raw scores list.

report.add_reduction_metrics(pca, metric="trustworthiness")

3.4 add_reduction_diagnostics#

Shepard diagram: original pairwise distances vs. embedded pairwise distances on a sampled subset. Requires the original X_orig and the embedding X_emb.

report.add_reduction_diagnostics(X_orig, X_emb)

3.5 add_reduction_interpretation#

Renders correlation / perturbation / gradient results from DimReduction.interpret(). Accepts either the raw get_summary()["interpretation"] payload or a list of records.

summary = pca.get_summary()
report.add_reduction_interpretation(summary["interpretation"])

3.6 add_reduction_coranking#

Heatmap of a co-ranking matrix (typically DimReduction.diagnostics_["coranking_matrix_"]).

report.add_reduction_coranking(diagnostics["coranking_matrix_"])

3.7 add_reduction_components#

Linear reducer components heatmap (PCA, IncrementalPCA, DaskPCA, DaskTruncatedSVD, DMD, TRCA). Pass feature_names for axis labels.

report.add_reduction_components(
    pca.get_components(),
    feature_names=channel_names,
)

3.8 add_reduction_trajectory#

Renders a 3-D trajectory tensor of shape (n_trajectories, n_times, n_dims). Pair with times= for labelled x-axis, labels= for per-trajectory coloring.

report.add_reduction_trajectory(
    X_3d, times=time_axis, labels=condition_ids,
)

3.9 add_reduction_trajectory_separation#

Per-pair separation timecourses (e.g., distance between condition centroids over time) as a multi-line plot. Accepts the output of trajectory_separation() directly.

sep = trajectory_separation(traj, condition_ids, method="centroid")
report.add_reduction_trajectory_separation(sep, times=time_axis, top_n=5)

3.10 add_reduction#

Full section in one call: overview + embedding/trajectory plot + scalar metrics + metric records + every applicable diagnostic. Pass X_emb to enable embedding/trajectory plots; without it the section still renders scalar metrics, loss curves, scree, and co-ranking.

report.add_reduction(
    pca, name="PCA",
    X_emb=embedding,
    labels=condition_ids,
    metadata={"subject": subject_ids},
    times=time_axis,             # only when X_emb is 3-D
)

3.11 add_comparison#

Multi-method comparison: a metric × method table with best-value highlighting (MetricsTableElement), a metric-by-method heatmap, a bar chart of details, and — when applicable — a radar chart normalised across metrics.

from coco_pipe.dim_reduction.evaluation import MethodSelector

selector = MethodSelector(reducers).collect()
report.add_comparison(selector.to_frame(), name="Method Comparison")

Raises ValueError if the metrics frame is empty after normalisation.

—

4. make_reduction_report#

The factory builds a multi-reducer report by calling adders 1-9 for each reducer.

from coco_pipe.report import make_reduction_report

report = make_reduction_report(
    [pca, umap, phate],
    embeddings=[pca_emb, umap_emb, phate_emb],   # required for embedding sections
    labels=class_ids,
    metadata={"subject": subject_ids},
    times=time_axis,                              # required for 3-D embeddings
    sections="default",
    theme="paper",
    output_path="reduction.html",
)

4.1 Sections argument#

Valid section names (see DEFAULT_REDUCTION_SECTIONS):

overview, embedding, metrics, diagnostics, coranking,
interpretation, components, trajectory, trajectory_separation

Unknown names raise ValueError. Pass a subset to render only the sections you need:

make_reduction_report(
    [pca, umap],
    embeddings=[pca_emb, umap_emb],
    sections=["overview", "embedding", "metrics"],
)

4.2 Error tolerance#

Per-section try / except mirrors the decoding factory: unavailable sections are logged and skipped, the rest of the report still renders.

—

5. Combining With Comparison#

The natural full-comparison pattern:

from coco_pipe.report import make_reduction_report
from coco_pipe.dim_reduction.evaluation import MethodSelector

selector = MethodSelector(reducers).collect()
report = make_reduction_report(
    reducers,
    embeddings=embeddings,
    title="PCA vs UMAP vs PHATE",
)
report.add_comparison(selector.to_frame())
report.save("reduction_comparison.html")

—

6. Internal Helpers (Used by Adders)#

Two helpers in coco_pipe.report.dim_reduction are exposed for custom adders that need the same normalisation contract:

_get_reducer_summary(reducer)

Return a normalised summary dict (always populates method, metrics, metric_records, quality_metadata, diagnostics, interpretation, interpretation_records, capabilities). Raises if the reducer lacks get_summary().

_metrics_summary_table(metrics)

Pivot a tidy metrics frame into the method × metric wide table used by MetricsTableElement.

_trajectory_times(diagnostics, times)

Resolve the trajectory time axis, preferring an explicit times= argument over diagnostics["trajectory_times_"].

These are intentionally underscore-prefixed; their signatures may change. For custom workflows, prefer the public adders.

Data-Quality Checks and Findings#

coco_pipe.io.quality contains the dataclass-based quality checks that run automatically when a DataContainer is passed to Report.add_container, plus the CheckResult type that Section.add_finding consumes.

—

1. CheckResult#

Every check returns a CheckResult (or a list of them):

@dataclass
class CheckResult:
    check_name: str           # e.g. "Missingness"
    status: Literal["OK", "WARN", "FAIL"]
    message: str              # human-readable summary
    severity: int             # 0 (info) to 10 (critical)
    metric_name: str | None = None
    metric_value: float | None = None

    @property
    def is_issue(self) -> bool:
        return self.status in {"WARN", "FAIL"}

When a section accumulates findings, its overall status is the worst of any single finding’s status. FAIL is sticky: once a section reaches it, a later WARN does not downgrade.

—

2. Built-in Checks#

All checks accept either a pd.DataFrame or a NumPy array and operate on numeric columns only.

Check

What it does

check_missingness()

Fraction of NaN values. WARN ≥ threshold_warn (default 0.01), FAIL ≥ threshold_fail (default 0.20).

check_constant_columns()

Per-column near-zero variance. Returns one CheckResult per offending column; FAIL for any constant numeric column.

check_outliers_zscore()

Z-score outlier detection. WARN when outlier fraction exceeds threshold.

check_flatline()

Detect zero-variance signal arrays (e.g., flatlined EEG channels).

 
from coco_pipe.io.quality import (
    check_missingness, check_constant_columns,
    check_flatline, check_outliers_zscore,
)

miss = check_missingness(df)
constants = check_constant_columns(df)
outliers = check_outliers_zscore(df, threshold=3.0)
flat = check_flatline(signal_array, threshold=1e-10)

for result in [miss, *constants, outliers, flat]:
    print(result.status, result.check_name, result.message)

—

3. Automatic Integration via add_container#

The most common path: pass a DataContainer to a report and let the checks fire automatically.

from coco_pipe.report import Report
from coco_pipe.io import load_data

container = load_data("scores.csv", mode="tabular", target_col="label")

report = Report(title="Input QC")
report.add_container(container)
report.save("qc.html")

The added section will contain:

  • a metadata table (dims, coords, dtype),

  • a CalloutElement per data-quality finding,

  • a histogram or scatter preview of the values,

  • automatic section-status upgrades to WARN / FAIL when findings warrant it.

If the container itself can’t be inspected (e.g., wrong shape), the section is skipped with a UserWarning rather than raising.

—

4. Adding Custom Findings to Any Section#

A finding doesn’t have to come from a built-in check — you can attach your own at any point.

from coco_pipe.report import Section
from coco_pipe.io.quality import CheckResult

sec = Section(title="Manual QC")
sec.add_finding(CheckResult(
    check_name="Subject coverage",
    status="WARN",
    message="Only 14 of 20 subjects have all 3 conditions.",
    severity=5,
    metric_name="subjects_with_full_coverage",
    metric_value=14,
))

The section is rendered with a colored finding banner per attached result, the section header shows the status pill, and the sidebar TOC flags the section with a colored dot.

—

5. Custom Checks#

A custom check is any function returning a CheckResult (or a list of them):

def check_class_balance(y, *, threshold_warn=0.1):
    counts = pd.Series(y).value_counts(normalize=True)
    min_share = counts.min()
    status = "WARN" if min_share < threshold_warn else "OK"
    return CheckResult(
        check_name="Class balance",
        status=status,
        message=f"Smallest class share: {min_share:.2%}",
        severity=5 if status == "WARN" else 0,
        metric_name="min_class_share",
        metric_value=float(min_share),
    )

Then plug it into your own section adder or call it directly:

sec.add_finding(check_class_balance(y))

—

6. Severity Conventions#

The numeric severity field is informational (not used to compute the displayed status). Useful ranges:

severity

Meaning

0

Informational; no action.

1-3

Minor; worth noting in the report.

4-6

Warning; investigate before publishing.

7-9

Major; the analysis may be invalid.

10

Critical; downstream results should not be trusted.

Pair high severities with status="FAIL" for the visual loudness to match.