BM25Retriever
Sparse retriever using BM25 (Okapi) ranking for document retrieval.
Computes BM25-weighted term-frequency vectors and retrieves via pairwise distance.
Parameters
- enable_retrieve_then_expand : boolean, default=
False - Widen each retrieved chunk with its neighbouring chunks before the chunks are returned as the retriever's context.
- expand_window : integer, default=
3 - Total number of chunks in the expanded window, the retrieved one included. Odd values are centred exactly on the retrieved chunk; even values keep the extra chunk after it. Windows never leave the document, and overlapping windows merge into one concatenated chunk.
- vectorizer : object
- Word counter whose counts BM25 weights.
- k1 : number, default=
1.5 - BM25 k1 parameter: term frequency saturation.
- b : number, default=
0.75 - BM25 b parameter: length normalization.
- delta : number, default=
0.0 - BM25 delta parameter for IDF smoothing.
- similarity_function : string, default=
cosine - Distance metric for comparing BM25-weighted vectors.
- top_k : integer, default=
5 - Number of chunks to select.
Methods
get_chunk_vectors(self, chunk_ids: List[int]) -> numpy.ndarray
BM25RetrieverReturn the BM25-weighted vectors for the given chunk IDs.
init_model(self) -> None
BM25RetrieverRestore saved state or fit BM25 from scratch.
load(self) -> bool
BM25RetrieverLoad a previously saved BM25 state from disk.
retrieve(self, query: str, top_k: int | None = None) -> Tuple[List[DashAI.back.models.RAG.documents.chunk.Chunk], Dict[str, Any]]
BM25RetrieverRetrieve the top-k chunks by BM25-weighted similarity.
save(self) -> None
BM25RetrieverPersist the vectorizer, matrices, and row to chunk id map to disk.
score_chunks(self, chunk_ids: List[int], query: str) -> List[Tuple[int, float]]
BM25RetrieverScore a set of chunk IDs against the query.
add(self, child: DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever) -> None
UnitRetrieverAdd a child retriever (not supported for unit retrievers).
close(self) -> None
BaseRetrieverRelease resources owned by this retriever.
Returns
- None
- Nothing.
expand_retrieved_chunks(self, chunks: List[DashAI.back.models.RAG.documents.chunk.Chunk], window_size: int) -> List[DashAI.back.models.RAG.documents.chunk.Chunk]
BaseRetrieverExpand chunks into centred windows and merge overlapping windows.
get_children(self) -> List[DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever]
UnitRetrieverReturn the (empty) list of child retrievers.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_id(self) -> int | None
BaseRetrieverReturn the database ID of this retriever, or None.
get_metadata(cls) -> Dict[str, Any]
BaseRetrieverReturn the UI metadata shared by every retriever.
Returns
- Dict[str, Any]
- Dictionary with the icon shown for the retriever in the frontend and, when declared, its representation template and the category the retriever pickers group it under.
get_representation(cls) -> DashAI.back.core.utils.MultilingualString | None
BaseRetrieverReturn the retriever's representation template, if any.
Returns
- MultilingualString | None
- The template with
{param}/{nested.param}placeholders, orNonewhen the frontend should fall back to the component description.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
get_schema_summary(self) -> DashAI.back.models.RAG.retrievers.retrieval_contracts.SchemaSummary
BaseRetrieverReturn a recursive, JSON-serializable summary of this retriever.
Returns
- SchemaSummary
{"class": ..., "params": ...}pluschildrenwhen present.
get_schema_summary_parameters(cls) -> Dict[str, str]
BaseRetrieverReturn a copy of the summary parameter mapping for the frontend.
inject_infra(self, env_RAG_path: str, chunks: Dict[int, Dict[int, DashAI.back.models.RAG.documents.chunk.Chunk]], persistence: Any, documents: Optional[Dict[int, DashAI.back.models.RAG.documents.source_document.SourceDocument]] = None) -> None
UnitRetrieverInject runtime infrastructure with type-checked persistence.
merge_expanded_chunks(self, chunks: List[DashAI.back.models.RAG.documents.chunk.Chunk]) -> List[DashAI.back.models.RAG.documents.chunk.Chunk]
BaseRetrieverMerge overlapping expanded units without widening plain chunks.
needs_save(self) -> bool
SparseRetrieverReturn whether the fitted state still has to be written to disk.
Returns
- bool
Truewhen the state was fitted rather than loaded.
remove(self, child: DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever) -> None
UnitRetrieverRemove a child retriever (not supported for unit retrievers).
row_map_for_corpus(self) -> Dict[int, int]
SparseRetrieverReturn the row -> chunk id map this corpus produces.
Returns
- Dict[int, int]
- The map the fitted matrices must be indexed with.
set_id(self, id: int) -> None
BaseRetrieverAssign a database ID to this retriever.
set_trace_params(self, params: Dict[str, Any]) -> None
BaseRetrieverStore a JSON-serializable snapshot of this retriever's configuration.
Parameters
- params : Dict[str, Any]
- Raw construction parameters captured by the factory before any nested component reference is consumed. A deep copy is stored so later mutations of the original mapping cannot affect trace rendering.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.