Decision requests
Decision models answer named questions about an input, such as a message, an image, or an audio clip, without generating text. Each answer is a typed value, such as a probability, a chosen option, a score, extracted text, or a bounding box. mistral.rs supports the d1 models LiquidAI/d1-2B and LiquidAI/d1-Omni-2B, and serves them at POST /v1/systemone. For a walkthrough, see Use decision models.
mistralrs serve -p 1234 -m LiquidAI/d1-Omni-2Bcurl http://localhost:1234/v1/systemone \ -H 'Content-Type: application/json' \ -d '{ "model": "default", "state": {"message": "Please cancel my subscription today."}, "questions": { "cancel": {"type": "noul", "instructions": "Does the customer want to cancel?"}, "intent": { "type": "choice", "instructions": "What is the customer requesting?", "criteria": {"cancel": "Cancel service", "billing": "Explain a payment", "other": null} } } }'Request fields
Section titled “Request fields”| Field | Description |
|---|---|
questions |
Required. Named questions; each name becomes a key in the response’s answers. See question types. |
state |
What the questions are about: a string or any JSON value. Use null when the questions are only about media. |
model |
Which loaded model to use. Defaults to the default model. |
images |
A list of image URL or data URL strings. |
media |
Images, audio, and video, in order. See media. |
set_size |
How many questions the model answers in one pass. Defaults to the model’s own setting. |
Any other field is rejected, including generation settings such as temperature, max_tokens, and stream.
mistral.rs splits the questions into groups of set_size and runs one pass per group, repeating the state and media in each. set_size doesn’t limit how many questions you can ask.
The response’s usage.input_tokens counts everything the model read, including media tokens and the state once per group. usage.output_tokens is always 0.
GGUF models take the same requests:
mistralrs serve -p 1234 -m LiquidAI/d1-Omni-2B-GGUF --quant 8The CLI downloads the matching projector file for you. The GGUF files alone answer noul, single-answer choice, and score questions. For the other types, and for choice with multiple: true, also pass the original model so mistral.rs can read its config and calibration:
mistralrs serve -p 1234 -m LiquidAI/d1-Omni-2B-GGUF --quant 8 --tok-model-id LiquidAI/d1-Omni-2BSee the d1 model notes for d1-2B and local loading.
Question types
Section titled “Question types”Every question has a type and instructions. Instructions are a string or, with d1-Omni, an audio content part.
| Type | Additional fields | Answer |
|---|---|---|
noul (yes/no) |
Optional criteria: {"true": "...", "false": "..."}, describing what counts as yes and no |
noul: probability of yes, from 0 to 1. |
choice |
criteria: option names mapped to descriptions, or to null. Optional multiple: true lets several options apply. |
One option: choice, confidence, and probabilities that sum to 1. With multiple: choices, and an independent probability per option. |
score |
criteria: 2 to 10 level descriptions, in order |
score: the expected level on a 0 to N-1 scale, plus confidence, probabilities, and legend. |
text |
None | text copied from the state, its span, found probability, and optional path. |
number |
Optional range: [min, max], step, and unit |
Without range: a number read from the state. With range: an estimated number and interval within it. |
date |
None | Normalized date, source text and span, found, and optional path. |
box |
None | One box with top, bottom, left, and right, plus found and confidence. |
boxes |
Optional max (1 to 16 for the shipped checkpoints) |
boxes, count, at_least, and a presence probability per slot. |
How to read extraction and box answers:
- Extracted values are copied from the state, never invented.
spanis a pair of Unicode character offsets into the state as rendered, with an exclusive end. JSON state is rendered with two-space indentation.- When the value comes from a single JSON string,
pathis a JSON Pointer to it. - Box questions need exactly one image. Coordinates run from 0 to 1, relative to the original image.
Media and spoken instructions
Section titled “Media and spoken instructions”images is a shortcut that takes a plain list of image URL or data URL strings. To mix images, audio, and video in a set order, use media with the same content parts as chat requests:
{ "state": null, "media": [ {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}, {"type": "audio_url", "audio_url": {"url": "https://example.com/question.wav"}}, {"type": "video_url", "video_url": {"url": "https://example.com/clip.mp4"}} ], "questions": { "visible": {"type": "noul", "instructions": "Is the described object visible?"} }}- WAV and MP3 clips can also be sent inline:
{"type":"input_audio","input_audio":{"data":"BASE64","format":"wav"}}. - To ask a question out loud, put an audio part where the
instructionsstring would go. imagescome beforemediain the input.- Over HTTP, media must be a remote URL or an inline data URL. The native SDKs also accept local files.
- Video frames are sampled using the model’s settings. The soundtrack isn’t extracted, so send audio separately.
Native SDKs
Section titled “Native SDKs”Rust uses typed DecisionRequest, DecisionQuestion, DecisionAnswer, and DecisionMedia values. Call Model::send_decision_request(request), or send_decision_request_with_model to pick a loaded model. See the Rust reference.
Python:
from mistralrs import DecisionQuestion, DecisionRequest, Runner, Which
runner = Runner(which=Which.MultimodalPlain(model_id="LiquidAI/d1-Omni-2B"))response = runner.send_decision_request(DecisionRequest( state={"message": "Please cancel my subscription."}, questions={"cancel": DecisionQuestion("noul", "Does the customer want to cancel?")},))print(response.answers["cancel"].noul)print(response.to_dict())In Python:
- Pass
model_id=to pick another loaded model. - Questions can also be dictionaries in the HTTP shape.
- An answer’s attributes depend on its
type. Optional attributes returnNonewhen absent. to_dict()returns the HTTP JSON shape, so fields the HTTP response leaves out are left out there too.imagesandmediaaccept URLs, data URLs, and localfile://URLs.
To load d1 from GGUF, pass the projector file with Rust’s GgufModelBuilder::with_mmproj_files or Python’s Which.GGUF(mmproj_filename=...). For every question type, also set with_tok_model_id or tok_model_id to the original model or a local folder with its config.json and calibration.json. Requests work the same. See GGUF SDK loading.