OpusMtEnESTransformer
Pre-trained transformer for english-spanish translation.
This model fine-tunes the pre-trained model opus-mt-en-es.
Parameters
- num_train_epochs : integer, default=
1 - Total number of training epochs to perform.
- batch_size : integer, default=
4 - The batch size per GPU/TPU core/CPU for training
- learning_rate : number, default=
2e-05 - The initial learning rate for AdamW optimizer
- device : string, default=
CPU - Hardware on which the training is run. If available, GPU is recommended for efficiency reasons. Otherwise, use CPU. If GPU is selected then it will use all gpus available.
- weight_decay : number, default=
0.01 - Weight decay is a regularization technique used in training neural networks to prevent overfitting. In the context of the AdamW optimizer, the 'weight_decay' parameter is the rate at which the weights of all layers are reduced during training, provided that this rate is not zero.
- log_train_every_n_epochs, default=
1 - Log metrics for train split every n epochs during training. If None, it won't log per epoch.
- log_train_every_n_steps, default=
None - Log metrics for train split every n steps during training. If None, it won't log per step.
- log_validation_every_n_epochs, default=
1 - Log metrics for validation split every n epochs during training. If None, it won't log per epoch.
- log_validation_every_n_steps, default=
None - Log metrics for validation split every n steps during training. If None, it won't log per step.
Methods
load(cls, filename: Union[str, ForwardRef('Path')])
OpusMtEnESTransformerRestore an OpusMtEnESTransformer instance from disk.
Parameters
- filename : str or Path
- Directory path from which the model files will be read.
Returns
- OpusMtEnESTransformer
- The restored model instance with
fittedset to the persisted value.
predict(self, x_pred: 'DashAIDataset') -> List
OpusMtEnESTransformerPredict with the fine-tuned model.
Parameters
- x_pred : Dataset
- Dataset with text data.
Returns
- List
- list of translations made by the model.
prepare_dataset(self, dataset: 'DashAIDataset', is_fit: bool = False) -> 'DashAIDataset'
OpusMtEnESTransformerApply the model transformations to the dataset.
Parameters
- dataset : DashAIDataset
- The dataset to be transformed.
Returns
- DashAIDataset
- The prepared dataset ready to be converted to an accepted format in the model.
save(self, filename: Union[str, ForwardRef('Path')]) -> None
OpusMtEnESTransformerStore the fine-tuned model and its configuration to disk.
Parameters
- filename : str or Path
- Directory path where the model files will be written.
tokenize_data(self, x: 'DashAIDataset', y: Optional[ForwardRef('DashAIDataset')] = None) -> 'DashAIDataset'
OpusMtEnESTransformerTokenize input and output.
Parameters
- x: DashAIDataset
- Dataset with the input data to preprocess.
- y: Optional DashAIDataset
- Dataset with the output data to preprocess.
Returns
- Dataset
- Dataset with the processed data.
train(self, x_train: 'DashAIDataset', y_train: 'DashAIDataset', x_validation: 'DashAIDataset' = None, y_validation: 'DashAIDataset' = None) -> 'OpusMtEnESTransformer'
OpusMtEnESTransformerFine-tune the opus-mt-en-es model on English-Spanish translation data.
Parameters
- x_train : DashAIDataset
- Input English text features for training.
- y_train : DashAIDataset
- Target Spanish translation labels for training.
- x_validation : DashAIDataset, optional
- Input English text features for validation. Defaults to None.
- y_validation : DashAIDataset, optional
- Target Spanish translation labels for validation. Defaults to None.
Returns
- OpusMtEnESTransformer
- The fine-tuned model instance.
calculate_metrics(self, split: DashAI.back.core.enums.metrics.SplitEnum = <SplitEnum.VALIDATION: 'validation'>, level: DashAI.back.core.enums.metrics.LevelEnum = <LevelEnum.LAST: 'last'>, log_index: int = None, x_data: 'DashAIDataset' = None, y_data: 'DashAIDataset' = None)
BaseModelCalculate and save metrics for a given data split and level.
Parameters
- split : SplitEnum
- The data split to evaluate (TRAIN, VALIDATION, or TEST). Defaults to SplitEnum.VALIDATION.
- level : LevelEnum
- The metric granularity level (LAST, TRIAL, STEP, or BATCH). Defaults to LevelEnum.LAST.
- log_index : int, optional
- Explicit step index for the metric entry. If None, the next step index is computed automatically. Defaults to None.
- x_data : DashAIDataset, optional
- Input features. If None, the dataset stored in the model for the given split is used. Defaults to None.
- y_data : DashAIDataset, optional
- Target labels. If None, the labels stored in the model for the given split are used. Defaults to None.
get_metadata(cls) -> Dict[str, Any]
BaseModelGet metadata values for the current model.
Returns
- Dict[str, Any]
- Dictionary containing UI metadata such as the model icon used in the DashAI frontend.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
prepare_output(self, dataset: 'DashAIDataset', is_fit: bool = False) -> 'DashAIDataset'
BaseModelHook for model-specific preprocessing of output targets.
Parameters
- dataset : DashAIDataset
- The output dataset (target labels) to preprocess.
- is_fit : bool
- Whether the call is part of a fitting phase. Defaults to False.
Returns
- DashAIDataset
- The preprocessed output dataset.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.