GPU calculator
How much GPU does your model need?
Pick a model, precision, context length and concurrency. AWSBee computes memory from each model's architecture and ranks every NVIDIA instance type by fit and cost.
GPU memory needed
48.0 GBof 57.6 GB usable on g4dn.12xlarge
- Weights18.5 GB
- KV cache22.5 GB
- Runtime overhead6.9 GB
- Usable memory (90% of VRAM)
Cheapest instance that fits
g4dn.12xlarge · 4× NVIDIA T4
~32 tok/s per stream · up to 11 full-context requests · ≈ $3.91/h
All NVIDIA GPU instances
Approximate on-demand list prices for us-east-1. Prices differ by region and change over time; check the official EC2 pricing page before launching. Throughput is a memory-bandwidth estimate, not a benchmark.
| Instance | GPUs | Usable | Max conc. | Tok/s (1 stream) | $/hour | $/1M out tok* |
|---|---|---|---|---|---|---|
g4dn.12xlarge | 4× T4 | 58 GB | 11 | 32 | 3.91 | 5.37 |
g6.12xlarge | 4× L4 | 86 GB | 21 | 30 | 4.60 | 6.74 |
g5.12xlarge | 4× A10G | 86 GB | 21 | 61 | 5.67 | 4.15 |
g6e.12xlarge | 4× L40S | 173 GB | 52 | 87 | 10.49 | 5.33 |
g6.48xlarge | 8× L4 | 173 GB | 50 | 61 | 13.35 | 9.77 |
g5.48xlarge | 8× A10G | 173 GB | 50 | 121 | 16.29 | 5.96 |
p4d.24xlarge | 8× A100 40GB | 288 GB | 91 | 315 | 21.96 | 3.10 |
g6e.48xlarge | 8× L40S | 346 GB | 111 | 175 | 30.13 | 7.66 |
p5.48xlarge | 8× H100 80GB | 576 GB | 193 | 678 | 55.04 | 3.61 |
g4dn.xlarge Needs 43.5 GB, 14.4 GB usable | 1× T4 | 14 GB | — | — | 0.53 | — |
g6.xlarge Needs 43.5 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.81 | — |
g6.2xlarge Needs 43.5 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.98 | — |
g5.xlarge Needs 43.5 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.01 | — |
g5.2xlarge Needs 43.5 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.21 | — |
g6e.xlarge Needs 43.5 GB, 43.2 GB usable | 1× L40S | 43 GB | — | — | 1.86 | — |
g6e.2xlarge Needs 43.5 GB, 43.2 GB usable | 1× L40S | 43 GB | — | — | 2.24 | — |
* At 8 concurrent streams with continuous batching, fully utilized. Tensor parallelism uses up to 8 GPUs for this model.
How the numbers are computed
- Weights = parameters × bytes per parameter (2 for BF16/FP16, 1 for FP8).
- KV cache = 2 × layers × KV heads × head dimension × 2 bytes, per token, times context × concurrent requests.
- Overhead = 1.5 GB per GPU for the CUDA context and graphs, plus 5% of weights for activations.
- Usable memory is 90% of VRAM, matching vLLM's default
--gpu-memory-utilization. - Throughput estimates assume decoding is memory-bandwidth bound: each token reads the active weights once, at ~55% of peak bandwidth.