PixArtSigma
Diffusion Transformer model for high efficiency text-to-image generation.
Wraps the PixArt-Sigma pipeline, which replaces the U-Net backbone used in Stable Diffusion with a scalable Diffusion Transformer (DiT) architecture. Text conditioning is provided by a T5-XXL encoder, enabling richer semantic understanding than CLIP-based models.
PixArt-Sigma achieves state of the art image quality with 14-25 denoising steps (compared to 20-50 for comparable U-Net models) and supports flexible multiscale resolutions up to 2048 px. Two checkpoint sizes are available: 512 px (lighter) and 1024 px (best quality).
References
- [1] Chen et al., "PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation", 2024. https://arxiv.org/abs/2403.04692
- [2] https://huggingface.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS
Parameters
- checkpoint : string, default=
1024 - Which PixArt-Sigma checkpoint to use: '1024' for best quality at 1024x1024 px, or '512' for a faster, lighter model at 512x512 px. Both checkpoints are downloaded together.
- negative_prompt, default=
None - num_inference_steps : integer, default=
20 - Number of denoising steps. PixArt-Sigma achieves good quality with 14-25 steps due to its efficient transformer architecture. More steps refine details but increase generation time.
- guidance_scale : number, default=
4.5 - Classifier-Free Guidance (CFG) scale. PixArt-Sigma works best with lower values (3.5-5.5) compared to U-Net models. Higher values enforce the prompt more strictly but may saturate colors. The default of 4.5 is recommended.
- device : string, default=
CPU - Hardware device for inference. GPU is strongly recommended. PixArt-Sigma uses a DiT (Diffusion Transformer) architecture with T5 text encoding, which is faster than U-Net on GPU.
- gpu_memory_mode : string, default=
auto - How much of the model is kept on the GPU. 'auto' measures free VRAM when the model loads and picks the fastest option that fits. 'full' keeps everything on the GPU (fastest, most VRAM). 'model' moves one component (text encoder, denoiser, VAE) to the GPU at a time. 'sequential' streams the weights layer by layer: slowest, but runs large models on small GPUs. Ignored on CPU.
- seed : integer, default=
-1 - Random seed for reproducible generation. A fixed positive integer always produces the same image. Use -1 for a random seed.
- width : integer, default=
1024 - Width of the output image in pixels. Must be a multiple of 8. PixArt-Sigma supports flexible resolutions up to 2048px.
- height : integer, default=
1024 - Height of the output image in pixels. Must be a multiple of 8. PixArt-Sigma supports flexible resolutions up to 2048px.
- num_images_per_prompt : integer, default=
1 - How many images to generate from a single prompt in one batch. Requires proportionally more GPU memory per additional image.
Methods
generate(self, input: str) -> List[Any]
PixArtSigmaGenerate images from a text prompt.
Parameters
- input : str
- Text prompt to generate an image from.
Returns
- List[Any]
- Generated output images in a list.
hf_repos(cls)
PixArtSigmaDownload both the 1024 (full pipeline) and 512 (transformer) repos.
Returns
- list of tuple of (str, str)
- The 1024 checkpoint (T5, VAE, scheduler, tokenizer, transformer) and the 512 checkpoint (transformer only), so either variant can be used after a single download.
component_dir(cls) -> pathlib.Path
DownloadableMixinReturn this component's own storage directory.
Returns
- pathlib.Path
<COMPONENT_PATH>/<ClassName>.
delete(cls) -> None
DownloadableMixinRemove the component's downloaded artifacts.
download(cls, report: Optional[Callable[[Optional[float], Optional[str]], NoneType]] = None) -> None
HFDownloadableMixinDownload all repos listed in hf_repos() into component_dir().
Parameters
- report : ProgressReporter, optional
- Callback invoked before each repo download with
report(None, "Downloading <repo_id>").Nonemeans no progress reporting.
get_credential(self, name: str)
ConfigObjectResolve a registered credential component by name.
Parameters
- name : str
- Credential component class name (e.g. "HuggingFaceCredential").
Returns
- BaseCredential
- An instance of the requested credential component.
get_metadata(cls) -> Dict[str, Any]
BaseGenerativeModelGet metadata values for the current generative model.
Returns
- Dict[str, Any]
- Dictionary indicating whether the model requires a download before use and the expected download size in bytes.
get_schema(cls) -> dict
ConfigObjectGenerates the component related Json Schema.
Returns
- dict
- Dictionary representing the Json Schema of the component.
is_downloaded(cls) -> bool
HFDownloadableMixinReturn whether all repo directories exist and are non-empty.
Returns
- bool
Truewhen every repo listed inhf_repos()has a non-empty local directory;Falseotherwise (including when the list is empty).
validate_and_transform(self, raw_data: dict) -> dict
ConfigObjectIt takes the data given by the user to initialize the model and returns it with all the objects that the model needs to work.
Parameters
- raw_data : dict
- A dictionary with the data provided by the user to initialize the model.
Returns
- dict
- A validated dictionary with the necessary objects.