A control plane for open models on NVIDIA GPUs, without handing over your data.

Sizing engine

Every catalog model carries its architecture: parameter count, layers, attention and KV heads, head dimension and maximum context.

AWSBee turns those into memory for weights, KV cache and runtime overhead, then checks every NVIDIA instance type with the right tensor-parallel degree. You see which ones fit, how many full-context requests each can hold, and the estimated cost per million output tokens.

One-click deployments

A deployment is a single CloudFormation stack in your account. It runs the Deep Learning Base GPU image with NVIDIA drivers and the container toolkit, and starts vLLM (pinned version) or an NVIDIA NIM container.

Stacks roll back automatically on failure and delete cleanly: no orphaned volumes, security groups or instances. Stop an instance to pause GPU billing and start it again later.

Gateway and keys

Applications call one endpoint with a AWSBee API key and the deployment name as the model. The gateway authenticates, routes to the right instance with that instance's own key, streams the response and records latency, time to first token and token counts.

Prompts and completions are streamed through, never stored.

Analytics

Requests, errors, p50/p95 latency, time to first token, output tokens per second and tokens over time, per deployment and per key. For vLLM deployments AWSBee also reads the server's own metrics: running and queued requests and KV-cache usage.