LoadDatafileDatasetUnit
Parse a dataset out of a download that already completed.
Separate from LoadUploadedDatasetUnit even though both end in the same
reader call, because finding what to read is the whole job here: the
download is a directory tree recorded in the database, not a file the caller
hands over. The unit re-reads that row in its own read-only session, the same
way LoadDatasetUnit does, and never writes to it.
Publishes only dataset, for the same reason as its sibling: an imported
file is not a stored dataset yet, so there is no id to correlate and no path
to save back to.
Parameters
- dataloader : object
- Reader used to parse the downloaded file, together with its own configuration.
- datafile_id : integer, default=
1 - Identifier of the completed download to read from. It has to have finished successfully; a download still running has no files yet.
- selected_file, default=
None - Which file inside the download to read, relative to its root. Leave empty to take the first one, ignoring hidden files.
Methods
execute(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
LoadDatafileDatasetUnitDo the unit's work, reading from and writing to the context.
Parameters
- ctx : ExecutionContext
- The shared execution context.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
BaseUnitCheck preconditions without executing the unit.
Parameters
- ctx : ExecutionContext
- The shared execution context.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.