ParseDocumentUnit
Parse source_path into a document with the chosen loader.
Reads the source from the context rather than its configuration, so it
composes after either source storing unit. Publishes only document:
where the result is stored is decided by whatever runs next.
validate checks the loader without reading anything, so a caller can
reject the request before it copies or writes a file.
Parameters
- dataloader : object
- Document loader used to parse the file, together with its own configuration.
Methods
execute(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
ParseDocumentUnitDo the unit's work, reading from and writing to the context.
Parameters
- ctx : ExecutionContext
- The shared execution context.
validate(self, ctx: DashAI.back.units.context.ExecutionContext) -> None
ParseDocumentUnitReject a loader that is missing or does not produce documents.
Parameters
- ctx : ExecutionContext
- The shared execution context.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.