NemenyiTest

StatisticalTest

DashAI.back.statistical_tests.post_hoc_tests.NemenyiTest

---

Nemenyi post-hoc test for pairwise comparison after a significant Friedman test.

Uses rank-based pairwise comparisons with a critical difference threshold. Recommended by Demsar (2006) as the standard post-hoc test for comparing multiple classifiers evaluated with cross-validation.

Requires the scikit-posthocs package.

References

Demsar, J. (2006). Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 7, 1-30.

Methods

get_metadata(cls) -> dict

Defined on NemenyiTest

Metadata for Nemenyi Test.

run(self, scores: dict[str, list[float]], alpha: float = 0.05, statistic: float = None, p_value: float = None, **kwargs) -> DashAI.back.statistical_tests.statistical_test_result.StatisticalTestResult

Defined on NemenyiTest

Run the Nemenyi post-hoc test after a significant Friedman test.

Parameters

scores : dict[str, list[float]]

Mapping from model/run names to score vectors evaluated over the same folds.

alpha : float, optional

Significance level used to judge the pairwise p-values, by default 0.05.

statistic : float or None, optional

Precomputed Friedman statistic. If provided, it is reused instead of recomputing the omnibus statistic.

p_value : float or None, optional

Precomputed Friedman p-value. If provided, it is reused instead of recomputing the omnibus statistic.

Returns

StatisticalTestResult

A result object with the omnibus Friedman outcome and the pairwise post-hoc comparisons.