FitModelOverFoldsUnit
Fit a model once per fold, then once more on everything the folds used.
The sibling of FitModelUnit. It takes a list of partition sets rather
than one, which is a different REQUIRES and therefore a different unit;
everything around the fit is shared between them.
Three things happen here, in this order, and the order is the point:
- If the model declares optimizable parameters, the search runs first. Its objective is the whole fold loop, so one trial costs k fits. That is what cross-validation buys and what it costs, and it is why the objective had to become something the caller supplies rather than something the optimizer does.
- Every fold is fitted and scored, and a
FOLDmetric row is written for it. Those rows are what the fold charts and the statistical tests read, and they are indexed from zero without gaps -- repeated cross-validation buckets them by integer division, so a gap silently moves a fold into the wrong repetition. - The model that gets kept is fitted on every row the folds could use,
and left pointing at that partition set. Scoring it against the rows the
session reserved is not done here: that is a
LASTmetric like any other, so it isEvaluateModelUnit's, the same unit a holdout run uses -- and the caller is the one that knows whether anything was reserved to score against.
The per-fold scores are published rather than aggregated here. Turning
them into a mean and a deviation means writing Metric rows that carry a
std_value, and a unit may not write domain rows -- the one sanctioned
write in the domain layer, BaseModel._save_metrics, has nowhere to put
a deviation. Whoever asked for the fit aggregates and persists.
Nested cross-validation is not here either: its inner splitter is a required component field, and a component field cannot be made optional without leaving the user without a selector, so it is a further sibling rather than a flag on this one.
Parameters
- optimizer : object
- Optimizer used to search for hyperparameters, along with its own configuration. Only used when the model declares optimizable parameters.
- goal_metric : string, default=
Accuracy - Metric the hyperparameter search optimizes.
- scored_splits : array, default=
['TRAIN', 'VALIDATION'] - Partitions each trial of the search records a score for. A partition the model is not meant to be judged on belongs out of this list even when metrics are configured for it.
Methods
execute(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
FitModelOverFoldsUnitDo the unit's work, reading from and writing to the context.
Parameters
- ctx : ExecutionContext
- The shared execution context.
validate(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
FitModelOverFoldsUnitCheck preconditions without executing the unit.
Parameters
- ctx : ExecutionContext
- The shared execution context.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.