EvaluateModelToArtifactUnit
Compute a trained model's metrics and publish them as data.
The sibling of EvaluateModelUnit, for a caller that has no Run row.
That unit writes Metric rows against a foreign key to run.id, so it
cannot work without one; this returns the same numbers instead of
persisting them, and whoever asked for them decides where they go.
Which metrics are computed is decided by the model, not by this unit, and
not configured here either: ModelFactory attaches the metric classes
and the data splits to the instance, so the metrics stay configured in one
place — where the model is built — rather than in two nodes that could
disagree.
Both this and EvaluateModelUnit score through
BaseModel.compute_metrics, which is what keeps them from drifting apart
into two answers for the same model and split.
Parameters
- splits : array, default=
['TRAIN', 'VALIDATION', 'TEST'] - Data splits the model is evaluated on.
Methods
execute(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
EvaluateModelToArtifactUnitDo the unit's work, reading from and writing to the context.
Parameters
- ctx : ExecutionContext
- The shared execution context.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
BaseUnitCheck preconditions without executing the unit.
Parameters
- ctx : ExecutionContext
- The shared execution context.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.