Generate images with diffusion models
mistral.rs generates images from text prompts with FLUX.1, served at POST /v1/images/generations.
Running FLUX
Section titled “Running FLUX”mistralrs serve -m black-forest-labs/FLUX.1-schnellThere are two checkpoints:
| Model | Denoising steps | Prompt length | Access |
|---|---|---|---|
black-forest-labs/FLUX.1-schnell |
4, so it’s fast | Up to 256 tokens. Longer prompts return an error. | Open |
black-forest-labs/FLUX.1-dev |
50, with guidance scale 4.0 | Not limited to 256 tokens | Gated: accept the license on its Hugging Face page, then run hf auth login or set HF_TOKEN |
The step count is fixed per checkpoint; request fields like steps and guidance_scale are ignored.
curl http://localhost:1234/v1/images/generations \ -H 'Content-Type: application/json' \ -d '{ "model": "default", "prompt": "A golden retriever wearing a scarf in autumn leaves.", "height": 1024, "width": 1024, "response_format": "B64Json" }'from mistralrs import DiffusionArchitecture, ImageGenerationResponseFormat, Runner, Which
runner = Runner(which=Which.DiffusionPlain( model_id="black-forest-labs/FLUX.1-schnell", arch=DiffusionArchitecture.FluxOffloaded, # or Flux to keep it all on the GPU))response = runner.generate_image( "A vibrant sunset in the mountains.", ImageGenerationResponseFormat.Url, height=1024, width=1024, save_file="sunset.png",)use mistralrs::{DiffusionGenerationParams, DiffusionLoaderType, DiffusionModelBuilder, ImageGenerationResponseFormat};
let model = DiffusionModelBuilder::new( "black-forest-labs/FLUX.1-schnell", DiffusionLoaderType::FluxOffloaded,).build().await?;let response = model.generate_image( "A vibrant sunset in the mountains.".to_string(), ImageGenerationResponseFormat::Url, DiffusionGenerationParams { height: 1024, width: 1024 }, Some("sunset.png".into()),).await?;Request fields
Section titled “Request fields”| Field | Default | Meaning |
|---|---|---|
prompt |
Required | What to draw |
n |
1 | How many images to generate |
height, width |
720, 1280 | Output size in pixels. OpenAI’s size string (such as "1024x1024") is ignored, so set these instead. |
response_format |
Url |
Url saves the image on the server and returns its filename. B64Json returns the image inline. See Output handling. |
Memory
Section titled “Memory”FLUX can’t be quantized: --quant and --isq don’t apply to it. The FLUX transformer alone has about 12B parameters, roughly 24 GB at 16-bit precision, and the T5-XXL text encoder needs more memory on top while it reads the prompt.
If that doesn’t fit on your GPU, load the offloaded variant from Python or Rust (FluxOffloaded, as in the examples above). It keeps the model in system RAM and moves each layer onto the GPU only while that layer runs. GPU memory use drops sharply, but every step copies the model across, so images take longer. The CLI and server always load the full model onto the GPU.
Output handling
Section titled “Output handling”With B64Json, b64_json holds a complete data URL (data:image/png;base64,...). Strip the prefix before decoding:
import base64
payload = response.data[0].b64_json.split(",", 1)[1]with open("out.png", "wb") as output: output.write(base64.b64decode(payload))With Url, the server writes a PNG named like image-generation-<uuid>.png into its working directory and returns that filename in data[0].url. It’s a path on the server’s disk, not a download link, so use B64Json when your client runs on another machine. In the SDKs, pass save_file to choose where the PNG goes.