MatthewsCorrCoef
Correlation between predicted and true labels, robust to class imbalance.
The Matthews Correlation Coefficient (MCC) is a balanced measure of the quality of a classification that can be used even when the classes are of very different sizes. In essence it is the correlation coefficient between the observed and predicted classifications, computed from the confusion matrix.
::
MCC = (TP·TN − FP·FN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN))
Range: [-1, 1]. Interpretation: +1 a perfect prediction; 0 no better than random guessing; -1 total disagreement between prediction and observation. Because a high score requires the classifier to do well on every class (all four confusion-matrix quadrants), MCC is regarded as more informative than accuracy or F1 on imbalanced problems. It generalises to the multiclass setting via scikit-learn's implementation.
References
- [1] Matthews, B.W. (1975). "Comparison of the predicted and observed secondary structure of T4 phage lysozyme." Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2), 442-451.
- [2] Chicco, D. & Jurman, G. (2020). "The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation." BMC Genomics, 21(6).
- [3] https://scikit-learn.org/stable/modules/generated/ sklearn.metrics.matthews_corrcoef.html
Methods
score(true_labels: 'DashAIDataset', probs_pred_labels: 'np.ndarray', multiclass: Optional[bool] = None) -> float
MatthewsCorrCoefCalculate the Matthews Correlation Coefficient.
Parameters
- true_labels : DashAIDataset
- A DashAI dataset with labels.
- probs_pred_labels : np.ndarray
- A two-dimensional matrix in which each column represents a class and the row values represent the probability that an example belongs to the class associated with the column.
- multiclass : bool, optional
- Whether the task is a multiclass classification. If None, it will be determined automatically from the number of unique labels.
Returns
- float
- Matthews Correlation Coefficient between true labels and predicted labels.
get_metadata(cls: 'BaseMetric') -> Dict[str, Any]
BaseMetricGet metadata values for the current metric.
Returns
- Dict[str, Any]
- Dictionary with the metadata
is_multiclass(true_labels: 'np.ndarray') -> bool
ClassificationMetricDetermine if the classification problem is multiclass (more than 2 classes).
Parameters
- true_labels : np.ndarray
- Array of true labels.
Returns
- bool
- True if the problem has more than 2 unique classes, False otherwise.