coco_pipe.decoding.run_statistical_assessment#

coco_pipe.decoding.run_statistical_assessment(observed_result, experiment_config, X, y, groups, sample_ids, sample_metadata, feature_names, time_axis, observation_level, inferential_unit)#

Orchestrate the statistical assessment of experiment results.

Resolves the chosen statistical method (binomial or permutation) and dispatches analysis for each model and metric.

Scientific Rationale#

Statistical significance in decoding is often non-trivial due to temporal autocorrelations and multiple comparisons. This orchestrator handles either analytical binomial tests (fast, theoretical chance) or full-pipeline permutation tests (rigorous, empirical null) to provide scientifically grounded inferential claims about model performance.

param observed_result:: The result of the actual experiment run.
type observed_result:: ExperimentResult
param experiment_config:: The full configuration of the experiment.
type experiment_config:: ExperimentConfig
param X:: The raw features and targets.
type X:: np.ndarray
param y:: The raw features and targets.
type y:: np.ndarray
param groups:: CV grouping vector.
type groups:: np.ndarray, optional
param sample_ids:: Unique identifiers for samples.
type sample_ids:: np.ndarray
param sample_metadata:: Metadata for unit resolution.
type sample_metadata:: pd.DataFrame, optional
param feature_names:: Names of input features.
type feature_names:: list of str, optional
param time_axis:: Time coordinates for temporal data.
type time_axis:: np.ndarray, optional
param observation_level:: Level of input rows (‘sample’ or ‘epoch’).
type observation_level:: str
param inferential_unit:: Definition of statistical independence (‘sample’ or ‘subject’).
type inferential_unit:: str
returns:: assessment_payload – Summary containing ‘rows’ (standardized results) and ‘nulls’.
rtype:: dict

Examples

>>> # Internal use within Experiment.run()
>>> # res = run_statistical_assessment(observed, config, X, y, ...)