RAGPipeline
Retrieval-Augmented Generation pipeline.
Receives dependencies injected — does not construct factories, repositories, or loaders. The caller (RAGJob) builds them from the current DB session and passes them in.
Orchestrates: document loading → chunk-set creation → chunking → retrieval → prompt formatting → LLM generation.
The stage LLMs (query transformation, retrieval evaluation and generation) are lazy proxies. Each one is constructed on first use inside its own stage and released when that stage ends, so the peak memory is the largest single model rather than the sum of all three.
Parameters
- documents : array, default=
None - List of document IDs to use in the RAG pipeline.
- prompt : object
- Prompt template used in the RAG pipeline.
- chunking_model : object
- Chunking model used to split documents into smaller pieces.
- retriever_model : object
- Retriever component used in the RAG pipeline.
- generation_model : object
- Text generation model used in the RAG pipeline.
- query_transformation, default=
None - Optional query transformation stage that turns the user message into one or more search queries before retrieval.
Methods
generate(self, input_data: 'Tuple[Dict[str, str], ...]', progress_callback: 'Optional[Callable[[Dict[str, Any]], None]]' = None) -> 'RAGGenerationOutput'
RAGPipelineRun the full RAG pipeline: retrieve, format, and generate.
multi_query_retrieval(self, queries: 'List[str]') -> 'Tuple[List[Chunk], List[Dict[str, Any]]]'
RAGPipelineRetrieve for every query and merge the results fairly.
Parameters
- queries : List[str]
- The search queries, in priority order. Always non-empty.
Returns
- Tuple[List[Chunk], List[Dict[str, Any]]]
- The round-robin merged, de-duplicated chunks and one trace entry per query —
{"query": <query>, "retriever": <trace node>}— in query order.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_metadata(cls) -> Dict[str, Any]
BaseGenerativeModelGet metadata values for the current generative model.
Returns
- Dict[str, Any]
- Dictionary indicating whether the model requires a download before use and the expected download size in bytes.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.