Skip to content

Supported models

Browse by family below, or choose a model by task. Expand an example for a command. For model-specific setup and limits, see model family notes.

Terminal window
mistralrs run -m <model> # interactive
mistralrs serve -m <model> # OpenAI-compatible server
Model families Example Architecture
DeepSeek-V2
deepseek-ai/DeepSeek-V2-Chatmistralrs run -m deepseek-ai/DeepSeek-V2-Chat
DeepseekV2ForCausalLM
DeepSeek-V3, DeepSeek-R1
deepseek-ai/DeepSeek-V3 (V3), deepseek-ai/DeepSeek-R1 (R1)mistralrs run -m deepseek-ai/DeepSeek-V3
mistralrs run -m deepseek-ai/DeepSeek-R1
DeepseekV3ForCausalLM
Flint
RecursiveSI/flintmistralrs run -m RecursiveSI/flint
FlintForCausalLM
Gemma
google/gemma-7b-itmistralrs run -m google/gemma-7b-it
GemmaForCausalLM
Gemma 2
google/gemma-2-9b-itmistralrs run -m google/gemma-2-9b-it
Gemma2ForCausalLM
GLM-4
zai-org/GLM-4-32B-0414mistralrs run -m zai-org/GLM-4-32B-0414
Glm4ForCausalLM
GLM-4.7
zai-org/GLM-4.7mistralrs run -m zai-org/GLM-4.7
Glm4MoeForCausalLM
GLM-4.7-Flash
zai-org/GLM-4.7-Flashmistralrs run -m zai-org/GLM-4.7-Flash
Glm4MoeLiteForCausalLM
GPT-OSS
openai/gpt-oss-20b (20b), openai/gpt-oss-120b (120b)mistralrs run -m openai/gpt-oss-20b
mistralrs run -m openai/gpt-oss-120b
GptOssForCausalLM
Granite 4.0
ibm-granite/granite-4.0-micromistralrs run -m ibm-granite/granite-4.0-micro
GraniteMoeHybridForCausalLM
HunYuan
tencent/Hunyuan-7B-Instructmistralrs run -m tencent/Hunyuan-7B-Instruct
HunYuanDenseV1ForCausalLM
HunYuan MoE
tencent/Hunyuan-A13B-Instructmistralrs run -m tencent/Hunyuan-A13B-Instruct
HunYuanMoEV1ForCausalLM
LFM2, LFM2.5
LiquidAI/LFM2.5-1.2B-Instruct (LFM2.5), LiquidAI/LFM2-1.2B (LFM2)mistralrs run -m LiquidAI/LFM2.5-1.2B-Instruct
mistralrs run -m LiquidAI/LFM2-1.2B
Lfm2ForCausalLM
LFM2 MoE, LFM2.5 MoE
LiquidAI/LFM2.5-8B-A1B (LFM2.5), LiquidAI/LFM2-8B-A1B (LFM2)mistralrs run -m LiquidAI/LFM2.5-8B-A1B
mistralrs run -m LiquidAI/LFM2-8B-A1B
Lfm2MoeForCausalLM
LFM3
LiquidAI/LFM3-1.7Bmistralrs run -m LiquidAI/LFM3-1.7B
Lfm3ForCausalLM
Llama 2, Llama 3.x
meta-llama/Llama-3.1-8B-Instructmistralrs run -m meta-llama/Llama-3.1-8B-Instruct
LlamaForCausalLM
Mistral
mistralai/Mistral-7B-Instruct-v0.3mistralrs run -m mistralai/Mistral-7B-Instruct-v0.3
MistralForCausalLM
Mixtral
mistralai/Mixtral-8x7B-Instruct-v0.1mistralrs run -m mistralai/Mixtral-8x7B-Instruct-v0.1
MixtralForCausalLM
Phi-2
microsoft/phi-2mistralrs run -m microsoft/phi-2
PhiForCausalLM
Phi-3, Phi-3.5
microsoft/Phi-3-medium-4k-instructmistralrs run -m microsoft/Phi-3-medium-4k-instruct
Phi3ForCausalLM
Phi-3.5-MoE
microsoft/Phi-3.5-MoE-instructmistralrs run -m microsoft/Phi-3.5-MoE-instruct
PhiMoEForCausalLM
Qwen2, Qwen2.5
Qwen/Qwen2.5-7B-Instruct (2.5), Qwen/Qwen2-7B-Instruct (2)mistralrs run -m Qwen/Qwen2.5-7B-Instruct
mistralrs run -m Qwen/Qwen2-7B-Instruct
Qwen2ForCausalLM
Qwen3
Qwen/Qwen3-4Bmistralrs run -m Qwen/Qwen3-4B
Qwen3ForCausalLM
Qwen3 MoE
Qwen/Qwen3-30B-A3Bmistralrs run -m Qwen/Qwen3-30B-A3B
Qwen3MoeForCausalLM
Qwen3-Next, Qwen3-Coder-Next
Qwen/Qwen3-Next-80B-A3B-Instructmistralrs run -m Qwen/Qwen3-Next-80B-A3B-Instruct
Qwen3NextForCausalLM
Qwen3.5 No published example Qwen3_5ForCausalLM
SmolLM3
HuggingFaceTB/SmolLM3-3Bmistralrs run -m HuggingFaceTB/SmolLM3-3B
SmolLM3ForCausalLM
Starcoder2
bigcode/starcoder2-7bmistralrs run -m bigcode/starcoder2-7b
Starcoder2ForCausalLM
Model families Example Architecture
DiffusionGemma
google/diffusiongemma-26B-A4B-itmistralrs run -m google/diffusiongemma-26B-A4B-it
DiffusionGemmaForBlockDiffusion
Gemma 3
google/gemma-3-12b-itmistralrs run -m google/gemma-3-12b-it
Gemma3ForConditionalGeneration
Gemma 3n
google/gemma-3n-E4B-itmistralrs run -m google/gemma-3n-E4B-it
Gemma3nForConditionalGeneration
Gemma 4
google/gemma-4-E4B-it (E4B), google/gemma-4-26B-A4B-it (26B-A4B MoE), google/gemma-4-31B-it (31B dense)mistralrs run -m google/gemma-4-E4B-it
mistralrs run -m google/gemma-4-26B-A4B-it
mistralrs run -m google/gemma-4-31B-it
Gemma4ForConditionalGeneration
Idefics2
HuggingFaceM4/idefics2-8bmistralrs run -m HuggingFaceM4/idefics2-8b
Idefics2ForConditionalGeneration
Idefics3, SmolVLM
HuggingFaceM4/Idefics3-8B-Llama3mistralrs run -m HuggingFaceM4/Idefics3-8B-Llama3
Idefics3ForConditionalGeneration
LFM2-VL, LFM2.5-VL
LiquidAI/LFM2.5-VL-1.6B (1.6B), LiquidAI/LFM2.5-VL-450M (450M)mistralrs run -m LiquidAI/LFM2.5-VL-1.6B
mistralrs run -m LiquidAI/LFM2.5-VL-450M
Lfm2VlForConditionalGeneration
LFM3-Audio
LiquidAI/LFM3-Audio-2Bmistralrs run -m LiquidAI/LFM3-Audio-2B
Lfm3AudioForConditionalGeneration
LFM3-Omni
LiquidAI/LFM3-Omni-2Bmistralrs run -m LiquidAI/LFM3-Omni-2B
Lfm3OmniForConditionalGeneration
LFM3-VL
LiquidAI/LFM3-VL-2Bmistralrs run -m LiquidAI/LFM3-VL-2B
Lfm3VlForConditionalGeneration
Llama 3.2 Vision
meta-llama/Llama-3.2-11B-Vision-Instructmistralrs run -m meta-llama/Llama-3.2-11B-Vision-Instruct
MllamaForConditionalGeneration
Llama 4
meta-llama/Llama-4-Scout-17B-16E-Instructmistralrs run -m meta-llama/Llama-4-Scout-17B-16E-Instruct
Llama4ForConditionalGeneration
LLaVA 1.5
llava-hf/llava-1.5-7b-hfmistralrs run -m llava-hf/llava-1.5-7b-hf
LlavaForConditionalGeneration
LLaVA-NeXT
llava-hf/llava-v1.6-mistral-7b-hfmistralrs run -m llava-hf/llava-v1.6-mistral-7b-hf
LlavaNextForConditionalGeneration
MiniCPM-o
openbmb/MiniCPM-o-2_6mistralrs run -m openbmb/MiniCPM-o-2_6
MiniCPMO
Mistral Small 3
mistralai/Mistral-Small-3.2-24B-Instruct-2506mistralrs run -m mistralai/Mistral-Small-3.2-24B-Instruct-2506
Mistral3ForConditionalGeneration
Muse Glimmer
meta-models/Muse-Glimmer-30Bmistralrs run -m meta-models/Muse-Glimmer-30B
MuseGlimmerForConditionalGeneration
Phi-3.5-Vision
microsoft/Phi-3.5-vision-instructmistralrs run -m microsoft/Phi-3.5-vision-instruct
Phi3VForCausalLM
Phi-4-multimodal
microsoft/Phi-4-multimodal-instructmistralrs run -m microsoft/Phi-4-multimodal-instruct
Phi4MMForCausalLM
Qwen 3.5, Qwen 3.6
Qwen/Qwen3.5-27B (3.5), Qwen/Qwen3.6-27B (3.6)mistralrs run -m Qwen/Qwen3.5-27B
mistralrs run -m Qwen/Qwen3.6-27B
Qwen3_5ForConditionalGeneration
Qwen 3.5 MoE, Qwen 3.6 MoE
Qwen/Qwen3.5-35B-A3B (3.5), Qwen/Qwen3.6-35B-A3B (3.6)mistralrs run -m Qwen/Qwen3.5-35B-A3B
mistralrs run -m Qwen/Qwen3.6-35B-A3B
Qwen3_5MoeForConditionalGeneration
Qwen2-VL
Qwen/Qwen2-VL-7B-Instructmistralrs run -m Qwen/Qwen2-VL-7B-Instruct
Qwen2VLForConditionalGeneration
Qwen2.5-VL
Qwen/Qwen2.5-VL-7B-Instructmistralrs run -m Qwen/Qwen2.5-VL-7B-Instruct
Qwen2_5_VLForConditionalGeneration
Qwen3-VL
Qwen/Qwen3-VL-4B-Instructmistralrs run -m Qwen/Qwen3-VL-4B-Instruct
Qwen3VLForConditionalGeneration
Qwen3-VL MoE
Qwen/Qwen3-VL-235B-A22B-Instructmistralrs run -m Qwen/Qwen3-VL-235B-A22B-Instruct
Qwen3VLMoeForConditionalGeneration
Qwen3.8-Flash-Next
Qwen/Qwen3.8-Flash-Nextmistralrs run -m Qwen/Qwen3.8-Flash-Next
Qwen4ExpForConditionalGeneration
Voxtral
mistralai/Voxtral-Mini-4B-Realtime-2602mistralrs run -m mistralai/Voxtral-Mini-4B-Realtime-2602
VoxtralRealtimeForConditionalGeneration

Use the decision API to classify, score, or extract values. These models do not generate chat responses.

Model families Example Architecture
d1, d1-Omni
LiquidAI/d1-2B, LiquidAI/d1-Omni-2Bmistralrs serve -m LiquidAI/d1-2B
mistralrs serve -m LiquidAI/d1-Omni-2B
D1Model
Model families Example Architecture
FLUX.1
black-forest-labs/FLUX.1-schnellmistralrs run -m black-forest-labs/FLUX.1-schnell
Flux
FLUX.1 (offloaded)
black-forest-labs/FLUX.1-schnellmistralrs run -m black-forest-labs/FLUX.1-schnell
FluxOffloaded
Model families Example Architecture
Dia
nari-labs/Dia-1.6Bmistralrs run -m nari-labs/Dia-1.6B
Dia
Model families Example Architecture
EmbeddingGemma
google/embeddinggemma-300mmistralrs run -m google/embeddinggemma-300m
Gemma3TextModel
Qwen3 Embedding
Qwen/Qwen3-Embedding-0.6Bmistralrs run -m Qwen/Qwen3-Embedding-0.6B
Qwen3ForCausalLM

For a checkpoint not shown here, match its config.json architectures value to the Architecture column. The examples are not an exhaustive list of sizes or fine-tunes. GGUF has a separate compatibility table. If neither lists your architecture, request model support.

Text, multimodal, speech, and embedding models support ISQ at load time. Diffusion models (FLUX) do not; they load at native precision. See GGUF support for GGUF compatibility; availability of UQFF, GPTQ, and AWQ artifacts varies by model on Hugging Face.

See speculative decoding for supported models and setup.