TFIDFVectorizerModel

Vectorizer

DashAI.back.models.RAG.retrievers.sparse.TFIDFVectorizerModel

Component that encapsulates a :class:TfidfVectorizer.

Validates parameters against :class:TFIDFVectorizerSchema and constructs the underlying scikit-learn vectorizer.

Parameters

strip_accents : string, default=None: Whether to strip accents from the text.
lowercase : boolean, default=True: Whether to convert all characters to lowercase.
analyzer : string, default=word: Whether the feature should be made of word or character n-grams.
stop_words : array, default=[]: List of stop words. Leave empty to use none.
ngram_range : array, default=[1, 1]: Lower and upper boundary of the n-gram range.
max_df : number, default=1.0: Ignore terms with document frequency above this threshold.
min_df : number, default=0.0: Ignore terms with document frequency below this threshold.
max_features : integer, default=1000: Maximum number of features. 0 means no limit.
norm : string, default=l2: Norm used to normalize term vectors.
use_idf : boolean, default=True: Enable inverse-document-frequency reweighting.
smooth_idf : boolean, default=True: Smooth IDF weights to prevent zero divisions.
sublinear_tf : boolean, default=False: Apply sublinear TF scaling (1 + log(tf)).

get_credential(self, name: str)

Defined on ConfigObject

Resolve a registered credential component by name.

Parameters

name : str: Credential component class name (e.g. "HuggingFaceCredential").

Returns

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict: A dictionary with the data provided by the user to initialize the model.

Returns