FriedmanTest
DashAI.back.statistical_tests.FriedmanTest
Non-parametric omnibus test for comparing three or more models.
This test is the rank-based alternative to ANOVA for repeated-measures or paired evaluations such as cross-validation results. It is commonly used when the assumptions of normality are not satisfied and is typically paired with a post-hoc test such as Nemenyi.
References
- https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.friedmanchisquare.html
- Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675-701.
Methods
get_metadata(cls) -> dict
FriedmanTestReturn UI metadata describing the test capabilities and interpretation.
run(self, scores: dict[str, list[float]], alpha: float = 0.05, **kwargs) -> DashAI.back.statistical_tests.statistical_test_result.StatisticalTestResult
FriedmanTestRun the Friedman test over the provided score collections.
Parameters
- scores : dict[str, list[float]]
- Mapping from model/run names to score vectors evaluated on the same folds or repeated evaluation blocks.
- alpha : float, optional
- Significance level used to decide whether the omnibus null hypothesis is rejected, by default 0.05.
Returns
- StatisticalTestResult
- A result object with the Friedman statistic, p-value, and the significance flag.