Blog
Notes on running open models in production
GPU sizing, inference cost and operations, from what we learn building the platform.
November 16, 2026 · 6 min read
How much GPU memory does an LLM actually need?
Model weights are only part of the bill. A simple way to estimate weights, KV cache and overhead before you pick a GPU.
Read article →November 13, 2026 · 5 min read
When does self-hosting an open model beat a hosted API on cost?
Per-token pricing is simple until volume grows. How to compare an hourly GPU with a per-token bill, and where the break-even sits.
Read article →November 10, 2026 · 4 min read
One OpenAI-compatible endpoint for every open model
Why a single gateway with one set of keys makes it easier to test, swap and govern models across teams.
Read article →