Naga
LLM reasoning engine for studio use.
A from-scratch, high-performance LLM inference and reasoning runtime for Apple Silicon. OpenAI-compatible serving, agentic tool-calling and local RAG — built on MLX.
> cat ./overview.md
Naga is a from-scratch runtime for LLM inference, serving and agentic workloads on Apple Silicon. It implements the full inference stack on MLX tensor operators — no transformers, no vLLM — so studios can run reasoning models entirely on local hardware.
Hand-written Qwen2/2.5 and LLaVA-style vision models, INT4/INT8 quantization (~75 tok/s decode on M2 Max) and RadixAttention prefix caching (7.9× faster multi-turn) keep it fast; constrained decoding guarantees valid JSON-schema output for tool use.
An OpenAI-compatible API with streaming, a WebUI, local memory + RAG via embeddings, and a built-in MCP agent with tool-calling loops make Naga the reasoning core behind an MCP + Agentic + Orchestration stack — private by default, on your own machines.
> ls ./features
From-Scratch MLX Stack
Complete inference stack built directly on MLX operators — no transformers or vLLM dependency.
Fast Local Decode
INT4/INT8 quantization reaching ~75 tok/s on M2 Max, with RadixAttention prefix caching for 7.9× faster multi-turn.
Constrained Decoding
Guaranteed valid JSON-schema output — reliable structured results for agent tool-calling.
Local Memory + RAG
Semantic memory and document retrieval via embeddings, entirely on-device.
MCP Agent
Built-in MCP agent with tool-calling loops, ready to plug into an orchestration layer.
OpenAI-Compatible Serving
Streaming OpenAI-compatible API, WebUI and a live monitoring dashboard for studio integration.
> open ./screenshots
> cat ./stack.json
{
"name": "Naga",
"category": "Inference / Reasoning Engine",
"stack": [
"MLX",
"Apple Silicon",
"Python",
"FastAPI",
"MCP",
"RAG"
]
}