AnovaTest
DashAI.back.statistical_tests.AnovaTest
Parametric omnibus test for comparing three or more models on identical data.
This implementation wraps SciPy's f_oneway and is suitable when the
compared models are evaluated on the same folds and the assumptions of
normality and homoscedasticity are reasonable. It is typically followed by
a post-hoc test such as Tukey HSD to identify which pairs of models differ.
References
- https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.f_oneway.html
- Fisher, R. A. (1925). Statistical Methods for Research Workers.
Methods
get_metadata(cls) -> dict
AnovaTestReturn UI metadata describing the test capabilities and interpretation.
run(self, scores: dict[str, list[float]], alpha: float = 0.05, **kwargs) -> DashAI.back.statistical_tests.statistical_test_result.StatisticalTestResult
AnovaTestRun a one-way ANOVA over the provided score collections.
Parameters
- scores : dict[str, list[float]]
- Mapping from model/run names to lists of scores collected over the same folds or repeated evaluations.
- alpha : float, optional
- Significance level used to decide whether the omnibus null hypothesis is rejected, by default 0.05.
Returns
- StatisticalTestResult
- A result object with the ANOVA statistic, p-value, and a boolean flag indicating whether the differences are significant.