Skip to content

Run against a reasoning model served by mistralrs on port 1234

Run against a reasoning model served by mistralrs on port 1234.

"""Run against a reasoning model served by mistralrs on port 1234."""
from openai import OpenAI
client = OpenAI(api_key="local", base_url="http://localhost:1234/v1")
response = client.responses.create(
model="default",
input="What is 9 times 7? Explain your answer briefly.",
max_output_tokens=256,
extra_body={"reasoning_max_tokens": 64},
)
print(response.output_text)
print(response.usage)

Source: examples/server/thinking_budget.py