Fast, flexible LLM inference.
Run any modelText, vision, speech, image generation, and embeddings.
Serve an APIOpenAI- and Anthropic-compatible endpoints for your apps.
Use with coding agentsRun Claude Code and Codex on a local model.
Tune performanceQuantize, size the cache, and speed up generation.
Use PythonRun models in-process from Python.
Use RustEmbed inference in a Rust application.