Skip to main content

OpusMtEnRoaTransformer

Model
DashAI.back.models.hugging_face.OpusMtEnRoaTransformer

Pretrained transformer for English to Romance translation.

Fine-tunes the Helsinki-NLP opus-mt-en-roa checkpoint, a multi-target MarianMT seq2seq model trained on parallel English to Romance corpora from the OPUS collection. The desired output language is chosen via the target_language parameter and injected as a sentence-initial >>id<< token on every source sentence (required by this checkpoint), covering English to Portuguese among the other Romance targets.

References​

Parameters​

num_train_epochs : integer, default=1
Total number of training epochs to perform.
batch_size : integer, default=4
The batch size per GPU/TPU core/CPU for training.
learning_rate : number, default=2e-05
The initial learning rate for AdamW optimizer.
device : string, default=CPU
Hardware on which training is run. GPU is recommended when available. If GPU is selected, all available GPUs are used.
weight_decay : number, default=0.01
L2 regularization coefficient applied via the AdamW optimizer to prevent overfitting.
log_train_every_n_epochs, default=1
Log train metrics every N epochs. None disables per-epoch logging.
log_train_every_n_steps, default=None
Log train metrics every N steps. None disables per-step logging.
log_validation_every_n_epochs, default=1
Log validation metrics every N epochs. None disables per-epoch logging.
log_validation_every_n_steps, default=None
Log validation metrics every N steps. None disables per-step logging.
target_language : string, default=French
Romance language to translate the English input into.

Methods​

load(cls, filename: Union[str, ForwardRef('Path')])

Defined on OpusMtEnRoaTransformer

Restore a model instance and its target language prefix.

save(self, filename: Union[str, ForwardRef('Path')]) -> None

Defined on OpusMtEnRoaTransformer

Persist the model, recording the chosen target language.

tokenize_data(self, x: 'DashAIDataset', y: Optional[ForwardRef('DashAIDataset')] = None) -> 'DashAIDataset'

Defined on OpusMtEnRoaTransformer

Tokenize like the base class but prepend the target language token.

calculate_metrics(self, split: DashAI.back.core.enums.metrics.SplitEnum = <SplitEnum.VALIDATION: 'validation'>, level: DashAI.back.core.enums.metrics.LevelEnum = <LevelEnum.LAST: 'last'>, log_index: int = None, x_data: 'DashAIDataset' = None, y_data: 'DashAIDataset' = None)

Defined on BaseModel

Calculate and save metrics for a given data split and level.

Parameters

split : SplitEnum
The data split to evaluate (TRAIN, VALIDATION, or TEST). Defaults to SplitEnum.VALIDATION.
level : LevelEnum
The metric granularity level (LAST, TRIAL, STEP, or BATCH). Defaults to LevelEnum.LAST.
log_index : int, optional
Explicit step index for the metric entry. If None, the next step index is computed automatically. Defaults to None.
x_data : DashAIDataset, optional
Input features. If None, the dataset stored in the model for the given split is used. Defaults to None.
y_data : DashAIDataset, optional
Target labels. If None, the labels stored in the model for the given split are used. Defaults to None.

get_metadata(cls) -> Dict[str, Any]

Defined on BaseModel

Get metadata values for the current model.

Returns

Dict[str, Any]
Dictionary containing UI metadata such as the model icon used in the DashAI frontend.

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

dict
Dictionary representing the Json Schema of the component.

predict(self, x_pred: 'DashAIDataset') -> List

Defined on OpusMtTransformerMixin

Translate source texts using the fine-tuned model.

prepare_dataset(self, dataset: 'DashAIDataset', is_fit: bool = False) -> 'DashAIDataset'

Defined on OpusMtTransformerMixin

Return the dataset unchanged (no preprocessing required).

prepare_output(self, dataset: 'DashAIDataset', is_fit: bool = False) -> 'DashAIDataset'

Defined on BaseModel

Hook for model-specific preprocessing of output targets.

Parameters

dataset : DashAIDataset
The output dataset (target labels) to preprocess.
is_fit : bool
Whether the call is part of a fitting phase. Defaults to False.

Returns

DashAIDataset
The preprocessed output dataset.

train(self, x_train: 'DashAIDataset', y_train: 'DashAIDataset', x_validation: 'DashAIDataset' = None, y_validation: 'DashAIDataset' = None)

Defined on OpusMtTransformerMixin

Fine-tune the Opus-MT model on translation data.

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict
A dictionary with the data provided by the user to initialize the model.

Returns

dict
A validated dictionary with the necessary objects.

Compatible with​