ImageDataLoader
Data loader for image datasets.
Expects a zip file containing images organized in subdirectories by class label (imagefolder format).
Methods
load_data(self, filepath_or_buffer: str, temp_path: str, params: Dict[str, Any], n_sample: int | None = None) -> DashAI.back.dataloaders.classes.dashai_dataset.DashAIDataset
ImageDataLoaderLoad an image dataset.
Parameters
- filepath_or_buffer : str
- An URL where the dataset is located or a FastAPI/Uvicorn uploaded file object.
- temp_path : str
- The temporary path where the files will be extracted and then uploaded.
- params : Dict[str, Any]
- Dict with the dataloader parameters.
- n_sample : int | None
- Indicates how many rows to load from the dataset, all rows if None.
Returns
- DashAIDataset
- A DashAI Dataset with the loaded image data.
extract_files(self, file_path: str, temp_path: str) -> str
BaseDataLoaderExtract a ZIP archive into a subdirectory of temp_path.
Parameters
- file_path : str
- Path to the ZIP archive to extract.
- temp_path : str
- Base temporary directory; extraction target is
<temp_path>/files/.
Returns
- str
- Path of the directory containing the extracted files (
<temp_path>/files/).
get_metadata(cls) -> Dict[str, Any]
BaseDataLoaderget_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
load_preview(self, filepath_or_buffer: str, params: Dict[str, Any], n_rows: int = 10) -> 'DataFrame'
BaseDataLoaderLoad a preview of the dataset using streaming.
Parameters
- filepath_or_buffer : str
- Path to the file or buffer to load.
- params : Dict[str, Any]
- Parameters for loading the data.
- n_rows : int, optional
- Number of rows to preview. Default is 10.
Returns
- DataFrame
- A pandas DataFrame with the preview data.
prepare_files(self, file_path: str, temp_path: str) -> str
BaseDataLoaderResolve a file path or URL into a local path suitable for loading.
Parameters
- file_path : str
- Path to a local file, a ZIP archive, or an HTTP(S) URL.
- temp_path : str
- Temporary directory used for extraction of ZIP or URL downloads.
Returns
- tuple of (str, str)
(path, type_path)wheretype_pathis"dir"for extracted archives/URLs or"file"for plain local files.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.