CharacterChunkModel

ChunkingModel

DashAI.back.models.RAG.chunking_models.CharacterChunkModel

---

Chunking model that splits documents into chunks based on character count. This model is useful for processing large text documents into manageable pieces.

Parameters

chunk_size : integer, default=400

Size of each chunk in characters.

chunk_overlap : integer, default=40

Number of characters to overlap between chunks. Must be less than chunk_size.

Methods

chunk_text(self, text: str) -> List[str]

Defined on CharacterChunkModel

Chunk the input text into smaller segments based on character count.

chunk_document(self, document: DashAI.back.models.RAG.documents.base_document.BaseDocument) -> Dict[int, DashAI.back.models.RAG.documents.chunk.Chunk]

Defined on BaseChunkingModel

Chunk a single document using the chunk_text method.

chunk_documents(self) -> Dict[int, Dict[int, DashAI.back.models.RAG.documents.chunk.Chunk]]

Defined on BaseChunkingModel

Chunk documents using the chunk_document method.

compute_chunks(self) -> None

Defined on BaseChunkingModel

Compute chunks if not already done.

get_chunks(self) -> Dict[int, Dict[int, DashAI.back.models.RAG.documents.chunk.Chunk]]

Defined on BaseChunkingModel

Get the chunks generated by the chunking model.

get_credential(self, name: str)

Defined on ConfigObject

Resolve a registered credential component by name.

Parameters

name : str

Credential component class name (e.g. "HuggingFaceCredential").

Returns

BaseCredential

An instance of the requested credential component.

get_id(self) -> int

Defined on BaseChunkingModel

Get the ID of the chunking model.

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

dict

Dictionary representing the Json Schema of the component.

set_id(self, id: int) -> None

Defined on BaseChunkingModel

Set the ID of the chunking model.

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict

A dictionary with the data provided by the user to initialize the model.

Returns

dict

A validated dictionary with the necessary objects.