PrepareAndFoldUnit
Validate a dataset against a task and carve it into cross-validation folds.
The sibling of PrepareAndSplitUnit, sharing its whole body. The two
differ in which family of splitters they offer and, because of that, in the
shape of what they publish.
What comes back is a list, and that is why the keys are different. A
fold splitter returns one entry per fold plus a trailing entry that is not
a fold: its train partition is every row the folds could use and its test
partition holds the rows reserved for scoring the model that gets kept,
empty when the session reserved none. Publishing that list as x would
give one key two possible types, which a contract comparing key names
cannot express -- a graph would validate and then fail at run time, or
worse, quietly train on the wrong thing. So it is x_folds and
y_folds.
split_indexes keeps its name because it keeps its meaning -- the rows
of every partition this run produced -- even though a fold payload is
shaped differently from a holdout one. Which shape it is, is answered by
asking the splitter that produced it, not by inspecting the payload.
A session that ran a preprocessing sequence sets two optional fields
together, input_column_refs and preprocessing_artifacts_path, as
the holdout sibling does and for the same reason. Here each entry has a
fit of its own: fold i is transformed with fold_{i}.pkl and the
trailing entry with final.pkl, which is what keeps a fold from seeing
statistics fitted on the rows it is scored on.
Parameters
- task_name : string, default=
TabularClassificationTask - Name of the task the dataset is prepared for.
- input_columns : array, default=
[] - Names of the columns used as model input.
- output_columns : array, default=
[] - Names of the columns the model has to predict.
- splitter : object
- Splitter that decides how the dataset is partitioned, along with its own configuration.
- input_column_refs, default=
None - Column references the model reads when the session ran a preprocessing sequence: {'kind': 'raw', 'name': ...} for a column of the dataset, {'kind': 'group', 'step': n} for what converter step n produces. Set together with the artifacts path, or not at all.
- preprocessing_artifacts_path, default=
None - Directory holding the preprocessing fitted for this session, one artifact per split entry. Set together with the input column references, or not at all.
Methods
execute(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
PrepareAndFoldUnitDo the unit's work, reading from and writing to the context.
Parameters
- ctx : ExecutionContext
- The shared execution context.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
BaseUnitCheck preconditions without executing the unit.
Parameters
- ctx : ExecutionContext
- The shared execution context.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.