IntervalCoverage
The share of actual values that fall inside the 95% prediction interval.
A 95% interval should hold about 95% of what actually happens. Coverage well below that says the model is more confident than it has reason to be; well above, that its intervals are wider than they need to be. It judges the intervals, not the point forecast, so it belongs beside an error metric rather than in place of one.
Range: [0, 1]. Higher means more of the actual values were bounded, which
is not the same as better: a model is well calibrated near 0.95.
MAXIMIZE says which way the axis runs, and no more. It cannot be
optimized (OPTIMIZABLE = False), since the widest intervals would win,
so a run that searches for it fails and no run is the best by it.
Methods
score_forecast(cls, forecast: DashAI.back.metrics.forecasting_metric.ScoredForecast) -> float
IntervalCoverageThe share of actual values within the forecast's 95% bounds.
get_metadata(cls: 'BaseMetric') -> Dict[str, Any]
BaseMetricGet metadata values for the current metric.
Returns
- Dict[str, Any]
- Dictionary with the metadata
score(cls, true_values, pred_values) -> float
ForecastingMetricA forecasting metric has no value on a run's rows alone.