Saltar al contenido principal

Qwen25_15BInstruct

GenerativeModel
DashAI.back.models.hugging_face.Qwen25_15BInstruct

Qwen 2.5 1.5B Instruct GGUF checkpoint (Q8_0 quantization).

A 1.5B-parameter instruction-tuned model from Alibaba Cloud that offers higher response quality than the 0.5B variant while still running on CPU. Weights are stored locally after a one-time download from HuggingFace.

References​

Parameters​

max_tokens : integer, default=100
Maximum number of new tokens the model will generate per response. Roughly 1 token ≈ 0.75 English words. Set to 100-200 for short answers, 500-1000 for detailed explanations or code. Must not exceed the context window minus the prompt length.
temperature : number, default=0.7
Sampling temperature controlling output randomness (range 0.0-1.0). At 0.0 the model always picks the most likely token (greedy, fully deterministic). Around 0.7 is a good balance for conversational tasks. At 1.0 outputs are maximally varied and unpredictable.
frequency_penalty : number, default=0.1
Penalizes tokens that have already appeared in the output based on how often they occur (range 0.0-2.0). At 0.0 there is no penalty and the model may repeat itself. Values around 0.1-0.3 gently discourage repetition. High values (1.5+) strongly prevent reuse of any word, which may produce less coherent text.
context_window : integer, default=512
Total token budget for a single forward pass, including both the input prompt and the generated response. Larger values allow longer conversations but consume more RAM/VRAM. Llama 3.1 supports up to 128K tokens natively; Llama 3.2 models support up to 128K tokens.
device : string, default=CPU
Hardware device for llama.cpp inference. 'CPU' runs the model fully in RAM with no GPU requirement. Selecting a GPU option offloads all layers for faster inference, setting n_gpu_layers=-1 so every transformer layer is GPU-accelerated.

Methods​

component_dir(cls) -> pathlib.Path

Defined on DownloadableMixin

Return this component's own storage directory.

Returns

pathlib.Path
<COMPONENT_PATH>/<ClassName>.

delete(cls) -> None

Defined on DownloadableMixin

Remove the component's downloaded artifacts.

download(cls, report: Optional[Callable[[Optional[float], Optional[str]], NoneType]] = None) -> None

Defined on HFDownloadableMixin

Download all repos listed in hf_repos() into component_dir().

Parameters

report : ProgressReporter, optional
Callback invoked before each repo download with report(None, "Downloading <repo_id>"). None means no progress reporting.

generate(self, prompt: list) -> List[str]

Defined on GGUFTextGenerationModel

Generate a reply for the given chat prompt.

Parameters

prompt : list of dict
Conversation history in OpenAI chat format. Each dict must contain at least "role" ("system", "user", or "assistant") and "content" (the message text).

Returns

list of str
A single-element list containing the model's reply text, extracted from choices[0]["message"]["content"].

get_metadata(cls) -> Dict[str, Any]

Defined on BaseGenerativeModel

Get metadata values for the current generative model.

Returns

Dict[str, Any]
Dictionary indicating whether the model requires a download before use and the expected download size in bytes.

get_schema(cls) -> dict

Defined on ConfigObject

Generates the component related Json Schema.

Returns

dict
Dictionary representing the Json Schema of the component.

hf_repos(cls) -> List[tuple]

Defined on GGUFTextGenerationModel

Return the single HuggingFace repo entry for this checkpoint.

Returns

list of tuple
A list containing one 3-tuple (repo_id, "model", [gguf_pattern]) when REPO_ID is set, or an empty list otherwise.

is_downloaded(cls) -> bool

Defined on HFDownloadableMixin

Return whether all repo directories exist and are non-empty.

Returns

bool
True when every repo listed in hf_repos() has a non-empty local directory; False otherwise (including when the list is empty).

validate_and_transform(self, raw_data: dict) -> dict

Defined on ConfigObject

It takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.

Parameters

raw_data : dict
A dictionary with the data provided by the user to initialize the model.

Returns

dict
A validated dictionary with the necessary objects.

Compatible with​