PairedTTest

StatisticalTest

DashAI.back.statistical_tests.PairedTTest

---

Parametric test for comparing two related model evaluations.

This implementation uses SciPy's paired t-test on the differences between paired scores from two models evaluated on the same folds. It is appropriate when the differences are approximately normally distributed and the samples are paired, which is the usual case for cross-validated comparisons.

References

• https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_rel.html
• Student. (1908). The probable error of a mean. Biometrika, 6(1), 1-25.

Methods

get_metadata(cls) -> dict

Defined on PairedTTest

Return UI metadata describing the test capabilities and interpretation.

run(self, scores: dict[str, list[float]], alpha: float = 0.05, alternative: str = 'two-sided', correction_method: str = None, **kwargs) -> DashAI.back.statistical_tests.statistical_test_result.StatisticalTestResult

Defined on PairedTTest

Run a paired t-test over two or more model score collections.

Parameters

scores : dict[str, list[float]]

Mapping from model/run names to paired score vectors.

alpha : float, optional

Significance level, by default 0.05.

alternative : str, optional

Direction of the hypothesis test: two-sided, greater, or less.

correction_method : str or None, optional

Method used to adjust p-values when more than two models are being compared.

Returns

StatisticalTestResult

A result object with the test statistic, adjusted or unadjusted p-values, and the overall significance decision.