Skip to main content

Phi4MiniInstructModel

GenerativeModel
DashAI.back.models.hugging_face.Phi4MiniInstructModel

Phi 4 Mini Instruct model for text generation using llama.cpp library.

Parameters​

model_name : string, default=unsloth/Phi-4-mini-instruct-GGUF
Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
quantization : string, default=Phi-4-mini-instruct.Q8_0.gguf
The specific Phi 4 Mini Instruct model quantization to use. Options include various quantization sizes and the BF16 format. The choice of quantization can affect the model's performance and resource usage, with smaller quantizations typically requiring less memory but potentially sacrificing some accuracy.
max_tokens : integer, default=100
Maximum number of new tokens the model will generate per response. Roughly 1 token ≈ 0.75 English words. Set to 100-200 for short answers, 500-1000 for detailed explanations or code.
temperature : number, default=0.7
Sampling temperature controlling output randomness (range 0.0-1.0). At 0.0 the model picks the most likely token (deterministic). Around 0.7 balances quality and creativity. At 1.0 outputs are maximally varied.
frequency_penalty : number, default=0.1
Penalizes tokens that have already appeared in the output based on frequency (range 0.0-2.0). Higher values discourage repetition.
context_window : integer, default=512
Total token budget for a single forward pass, including prompt and response. Mistral-7B supports up to 32K tokens; Mistral-Nemo supports up to 128K tokens.
device : string, default=CPU
Hardware device for llama.cpp inference. 'CPU' runs the model fully in RAM. A GPU option offloads all layers for faster inference.

Methods​

generate(self, prompt: list[dict[str, str]]) -> List[str]

Defined on Phi4MiniInstructModel

Generate output from the model given an input.

Parameters

input : Any or Tuple[Any, Any]
The input data or prompt. May be a single item or a tuple of (prompt, conditioning) depending on the model type.

Returns

List[Any]
A list of generated outputs (e.g. strings, images).

get_credential(self, name: str)

Defined on ConfigObject

Resolve a registered credential component by name.

Parameters

name : str
Credential component class name (e.g. "HuggingFaceCredential").

Returns

BaseCredential
An instance of the requested credential component.

get_metadata(cls) -> Dict[str, Any]

Defined on BaseGenerativeModel

Get metadata values for the current generative model.

Returns

Dict[str, Any]
Dictionary indicating whether the model requires a download before use and the expected download size in bytes.

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

dict
Dictionary representing the Json Schema of the component.

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict
A dictionary with the data provided by the user to initialize the model.

Returns

dict
A validated dictionary with the necessary objects.

Compatible with​