HuggingFaceDatasetSource

DatasetSource

DashAI.back.dataset_sources.HuggingFaceDatasetSource

Dataset source that fetches public datasets from HuggingFace Hub.

Uses huggingface_hub.HfApi — no authentication required for public datasets. HfApi.list_datasets exposes an iterator rather than native cursors, so pagination is implemented by treating the cursor as a numeric offset and slicing the iterator.

Methods

download_dataset(self, dataset_id: str, temp_path: str) -> str

Defined on HuggingFaceDatasetSource

Download the raw dataset files from HuggingFace Hub.

Parameters

dataset_id : str: HuggingFace dataset identifier (e.g. "stanfordnlp/imdb").
temp_path : str: Local directory to download into.

Returns

str: Path to the directory containing the downloaded files.

get_info(self, dataset_id: str) -> 'DatasetEntry | None'

Defined on HuggingFaceDatasetSource

Return full metadata for a single HuggingFace dataset, including size.

Parameters

dataset_id : str: HuggingFace dataset identifier in "namespace/repo" form.

Returns

DatasetEntry or None: Full metadata entry, or None on error.

search(self, query: str, limit: int = 20, cursor: str | None = None, **filters: Any) -> DashAI.back.dataset_sources.base_dataset_source.SearchPage

Defined on HuggingFaceDatasetSource

Return public HuggingFace datasets matching a query.

Parameters

query : str: Search string passed to HfApi.list_datasets.
limit : int, optional: Maximum number of results, by default 20.
cursor : str or None, optional: Pagination cursor returned by the previous call (encoded numeric offset). None fetches the first page.
**filters : Any: Unused; reserved for future tag/task filters.

Returns

SearchPage: Matching datasets and cursor for the next page (or None).

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

dict: Dictionary representing the Json Schema of the component.

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict: A dictionary with the data provided by the user to initialize the model.

Returns

dict: A validated dictionary with the necessary objects.

Methods​

Methods