SentenceTransformerCrossEncoderRetriever
Cross-encoder re-ranker powered by SentenceTransformer models.
Retrieves candidates from a single child retriever and re-ranks them
using a SentenceTransformer CrossEncoder <https://sbert.net/docs/cross_encoder/pretrained_models.html>_
model that scores (query, chunk) pairs jointly.
The child retriever (the ranker) defines how many candidates are
fetched via its own top_k; this cross-encoder only selects
top_k of them after re-ranking.
Example usage::
Retrieve 15 candidates with BM25 (child's own top_k), keep top 5
after cross-encoding
config = { "model_name": "cross-encoder/ms-marco-MiniLM-L-6-v2", "top_k": 5, "children": [ { "component": "BM25Retriever", "params": {"...": "...", "top_k": 15}, } ], }
Parameters
- enable_retrieve_then_expand : boolean, default=
False - Widen each retrieved chunk with its neighbouring chunks before the chunks are returned as the retriever's context.
- expand_window : integer, default=
3 - Total number of chunks in the expanded window, the retrieved one included. Odd values are centred exactly on the retrieved chunk; even values keep the extra chunk after it. Windows never leave the document, and overlapping windows merge into one concatenated chunk.
- model_name : string, default=
cross-encoder/ms-marco-MiniLM-L-6-v2 - Pre-trained SentenceTransformer cross-encoder model to use.
- top_k : integer, default=
5 - Final number of chunks to return after re-ranking. The candidate set size is determined by the child retriever's own top_k.
- children : array, default=
[] - The child retriever whose candidates will be re-ranked.
Methods
inject_infra(self, env_RAG_path: str, chunks: Dict[int, Dict[int, DashAI.back.models.RAG.documents.chunk.Chunk]], persistence: Any, documents: Optional[Dict[int, DashAI.back.models.RAG.documents.source_document.SourceDocument]] = None) -> None
SentenceTransformerCrossEncoderRetrieverInject runtime infrastructure and load the cross-encoder model.
load(self, filename: str = '') -> None
SentenceTransformerCrossEncoderRetrieverDownload and load the cross-encoder model from HuggingFace Hub.
add(self, child: DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever) -> None
CompositeRetrieverAdd a child retriever.
close(self) -> None
CompositeRetrieverRelease every child's resources and this composite's own.
Returns
- None
- Nothing.
expand_retrieved_chunks(self, chunks: List[DashAI.back.models.RAG.documents.chunk.Chunk], window_size: int) -> List[DashAI.back.models.RAG.documents.chunk.Chunk]
BaseRetrieverExpand chunks into centred windows and merge overlapping windows.
get_children(self) -> List[DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever]
CompositeRetrieverReturn a copy of the children list.
get_chunk_vectors(self, chunk_ids: List[int]) -> numpy.ndarray
CrossEncoderRetrieverReturn chunk vectors from the child retriever.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_id(self) -> int | None
BaseRetrieverReturn the database ID of this retriever, or None.
get_metadata(cls) -> Dict[str, Any]
BaseRetrieverReturn the UI metadata shared by every retriever.
Returns
- Dict[str, Any]
- Dictionary with the icon shown for the retriever in the frontend and, when declared, its representation template and the category the retriever pickers group it under.
get_representation(cls) -> DashAI.back.core.utils.MultilingualString | None
BaseRetrieverReturn the retriever's representation template, if any.
Returns
- MultilingualString | None
- The template with
{param}/{nested.param}placeholders, orNonewhen the frontend should fall back to the component description.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
get_schema_summary(self) -> DashAI.back.models.RAG.retrievers.retrieval_contracts.SchemaSummary
BaseRetrieverReturn a recursive, JSON-serializable summary of this retriever.
Returns
- SchemaSummary
{"class": ..., "params": ...}pluschildrenwhen present.
get_schema_summary_parameters(cls) -> Dict[str, str]
BaseRetrieverReturn a copy of the summary parameter mapping for the frontend.
init_model(self) -> None
CrossEncoderRetrieverInitialize the model by loading it.
merge_expanded_chunks(self, chunks: List[DashAI.back.models.RAG.documents.chunk.Chunk]) -> List[DashAI.back.models.RAG.documents.chunk.Chunk]
BaseRetrieverMerge overlapping expanded units without widening plain chunks.
remove(self, child: DashAI.back.models.RAG.retrievers.base_retriever.BaseRetriever) -> None
CompositeRetrieverRemove a child retriever.
retrieve(self, query: str, top_k: int | None = None, **kwargs) -> Tuple[List[DashAI.back.models.RAG.documents.chunk.Chunk], Dict[str, Any]]
CrossEncoderRetrieverRetrieve and re-rank chunks using the cross-encoder.
save(self, filename: str = '') -> None
CrossEncoderRetrieverPersist the cross-encoder retriever's state.
score_chunks(self, chunk_ids: List[int], query: str) -> List[Tuple[int, float]]
CrossEncoderRetrieverScore a set of chunk IDs against the query with the cross-encoder.
set_id(self, id: int) -> None
BaseRetrieverAssign a database ID to this retriever.
set_trace_params(self, params: Dict[str, Any]) -> None
BaseRetrieverStore a JSON-serializable snapshot of this retriever's configuration.
Parameters
- params : Dict[str, Any]
- Raw construction parameters captured by the factory before any nested component reference is consumed. A deep copy is stored so later mutations of the original mapping cannot affect trace rendering.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.