Inkling 2 Small: A 594B Open Model You Can Actually Run (If You Have 500 GB of VRAM)

Inkling 2 Small: A 594B Open Model You Can Actually Run (If You Have 500 GB of VRAM)

Published
AuthorDevansh

On November 5, 2026, Thinking Machines Lab released Inkling 2 Small, the first model of its second generation. A week later, on November 12, the full weights landed on Hugging Face under an Apache 2.0 license, as the lab had promised at launch after external safety testing. The model is a sparse mixture-of-experts design: 594 billion total parameters with only 17 billion active per token, routed to 8 of 384 experts, with text and image input, text output, and a context window of up to 1 million tokens. It is built for developers making agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation apps.

The VRAM math, before anything else

The model was trained natively in NVFP4, and that checkpoint is the one Thinking Machines recommends. The model card says the NVFP4 checkpoint needs a GPU cluster with at least 500 GB of aggregated VRAM, for example 2 NVIDIA B300 GPUs or 4 NVIDIA H200 GPUs. The alternative MXFP8 checkpoint needs at least 800 GB of aggregated VRAM, for example 4 B300s or 8 H200s, for what the lab calls a very marginal 1 to 2 percentage point gain on evals that it does not consider meaningful in practice. That is the whole story in one line: the weights are free and the license is permissive, but running them yourself is a multi-GPU server job, not a laptop job. If you do not have that hardware, hosted access starts around $0.30 per million input tokens and $1.00 per million output tokens on the hosted platforms, including Tinker and Ollama Cloud, and the model is also served by NVIDIA, Fireworks, Together, Baseten, Nebius, Modal, and Cloudflare.

How to run it

The model card gives recipes for local inference with SGLang, vLLM, TokenSpeed, and Unsloth, and post-training with Miles, SkyRL, and Snowflake Cortex Training. Start with the hosted API to learn the model's behaviour, then self-host only if you can justify the GPU bill: measure cost per completed task, not cost per token, because a model that needs retries is not cheap even when the weights are free. Our vLLM serving guide covers the measure-first discipline for the engine itself, and our model-free NIM explainer covers running any open model on Kubernetes with the same engines.

What the benchmarks say, and the catch

Thinking Machines' own model card reports 86.3% on SWE-bench Pro v2 (Hard), 72.1% on DeepSWE v1.1, 85.4% on MCP Atlas for tool use, and 81.6% on MMMU Pro for multimodal understanding, with 30.6 on the Artificial Analysis Intelligence Index. Every one of those figures is vendor-reported from the lab's own table, and none has been independently replicated outside those sources yet, so read them as positioning, not verdicts. The second catch is the license wording: Apache 2.0 covers the weights, but the training data and code are not released, so this is open weights rather than fully open source. Neither catch is a reason to skip the model, but both are reasons to test it on your own workload before committing a GPU cluster to it. Our local LLM guide covers the single-machine end of the same problem, where Ollama is the simpler first step. If you want the GPU sizing, the serving layer, and the monitoring around an AI workload checked by someone who runs infrastructure for a living, book a free infra audit with DevOpsBY.

Disclaimer: This post is educational only, not financial advice. Figures are as reported by the sources on 2026-11-17 and may change - always confirm version numbers, requirements, and prices in the official docs before acting on them.

Related Articles

DevOps & Infrastructure Services

Need hands-on help? We build and manage the infrastructure behind the concepts covered in this article.

DevOpsBy Assistant

How can we help?

Select a topic to get started: