Phi4MiniInstructModel
Phi 4 Mini Instruct model for text generation using llama.cpp library.
Parameters
- model_name : string, default=
unsloth/Phi-4-mini-instruct-GGUF - Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
- quantization : string, default=
Phi-4-mini-instruct.Q8_0.gguf - The specific Phi 4 Mini Instruct model quantization to use. Options include various quantization sizes and the BF16 format. The choice of quantization can affect the model's performance and resource usage, with smaller quantizations typically requiring less memory but potentially sacrificing some accuracy.
- max_tokens : integer, default=
100 - Maximum number of new tokens the model will generate per response. Roughly 1 token ≈ 0.75 English words. Set to 100-200 for short answers, 500-1000 for detailed explanations or code.
- temperature : number, default=
0.7 - Sampling temperature controlling output randomness (range 0.0-1.0). At 0.0 the model picks the most likely token (deterministic). Around 0.7 balances quality and creativity. At 1.0 outputs are maximally varied.
- frequency_penalty : number, default=
0.1 - Penalizes tokens that have already appeared in the output based on frequency (range 0.0-2.0). Higher values discourage repetition.
- context_window : integer, default=
512 - Total token budget for a single forward pass, including prompt and response. Mistral-7B supports up to 32K tokens; Mistral-Nemo supports up to 128K tokens.
- device : string, default=
CPU - Hardware device for llama.cpp inference. 'CPU' runs the model fully in RAM. A GPU option offloads all layers for faster inference.
Methods
close(self) -> None
Phi4MiniInstructModelRelease the llama.cpp backend and reclaim its accelerator memory.
Returns
- None
- Nothing.
generate(self, prompt: list[dict[str, str]]) -> List[str]
Phi4MiniInstructModelGenerate output from the model given an input.
Parameters
- input : Any or Tuple[Any, Any]
- The input data or prompt. May be a single item or a tuple of (prompt, conditioning) depending on the model type.
Returns
- List[Any]
- A list of generated outputs (e.g. strings, images).
generate_function_call(self, messages: List[Dict[str, str]], functions: List[Dict[str, Any]]) -> DashAI.back.models.text_to_text_generation_model.FunctionCallResult
TextToTextGenerationTaskModelProduce a named function call for the given chat messages.
Parameters
- messages : List[Dict[str, str]]
- Chat messages (OpenAI format) as passed to
generate. - functions : List[Dict[str, Any]]
- OpenAI-style function specs (
name,description,parametersJSON schemas). A backend MAY force a specific call (the GGUF backend forcesfunctions[0]); callers must not rely on arbitrary selection among the specs.
Returns
- FunctionCallResult
- The called function name and its raw arguments payload. The
argumentsvalue is either a dict or a JSON-encoded string the caller is responsible for resolving.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_metadata(cls) -> Dict[str, Any]
BaseGenerativeModelGet metadata values for the current generative model.
Returns
- Dict[str, Any]
- Dictionary indicating whether the model requires a download before use and the expected download size in bytes.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
supports_function_calling(cls) -> bool
TextToTextGenerationTaskModelReport whether instances of this class support function calling.
Returns
- bool
Truewhen the concrete backend implementsgenerate_function_callwith schema-constrained sampling.
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.