GPU calculator
How much GPU does your model need?
Pick a model, precision, context length and concurrency. AWSBee computes memory from each model's architecture and ranks every NVIDIA instance type by fit and cost.
GPU memory needed
25.0 GBof 43.2 GB usable on g6e.xlarge
- Weights15.2 GB
- KV cache7.5 GB
- Runtime overhead2.3 GB
- Usable memory (90% of VRAM)
Cheapest instance that fits
g6e.xlarge · NVIDIA L40S
~31 tok/s per stream · up to 27 full-context requests · ≈ $1.86/h
All NVIDIA GPU instances
Approximate on-demand list prices for us-east-1. Prices differ by region and change over time; check the official EC2 pricing page before launching. Throughput is a memory-bandwidth estimate, not a benchmark.
| Instance | GPUs | Usable | Max conc. | Tok/s (1 stream) | $/hour | $/1M out tok* |
|---|---|---|---|---|---|---|
g6e.xlarge | 1× L40S | 43 GB | 27 | 31 | 1.86 | 2.65 |
g6e.2xlarge | 1× L40S | 43 GB | 27 | 31 | 2.24 | 3.20 |
g4dn.12xlarge | 4× T4 | 58 GB | 37 | 39 | 3.91 | 4.43 |
g6.12xlarge | 4× L4 | 86 GB | 68 | 37 | 4.60 | 5.56 |
g5.12xlarge | 4× A10G | 86 GB | 68 | 74 | 5.67 | 3.42 |
g6e.12xlarge | 4× L40S | 173 GB | 160 | 106 | 10.49 | 4.40 |
g6.48xlarge | 8× L4uses 4 (head count) | 86 GB | 68 | 37 | 13.35 | 16.12 |
g5.48xlarge | 8× A10Guses 4 (head count) | 86 GB | 68 | 74 | 16.29 | 9.83 |
p4d.24xlarge | 8× A100 40GBuses 4 (head count) | 144 GB | 129 | 191 | 21.96 | 5.12 |
g6e.48xlarge | 8× L40Suses 4 (head count) | 173 GB | 160 | 106 | 30.13 | 12.63 |
p5.48xlarge | 8× H100 80GBuses 4 (head count) | 288 GB | 283 | 411 | 55.04 | 5.95 |
g4dn.xlarge Needs 25.0 GB, 14.4 GB usable | 1× T4 | 14 GB | — | — | 0.53 | — |
g6.xlarge Needs 25.0 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.81 | — |
g6.2xlarge Needs 25.0 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.98 | — |
g5.xlarge Needs 25.0 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.01 | — |
g5.2xlarge Needs 25.0 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.21 | — |
* At 8 concurrent streams with continuous batching, fully utilized. Tensor parallelism uses up to 4 GPUs for this model.
How the numbers are computed
- Weights = parameters × bytes per parameter (2 for BF16/FP16, 1 for FP8).
- KV cache = 2 × layers × KV heads × head dimension × 2 bytes, per token, times context × concurrent requests.
- Overhead = 1.5 GB per GPU for the CUDA context and graphs, plus 5% of weights for activations.
- Usable memory is 90% of VRAM, matching vLLM's default
--gpu-memory-utilization. - Throughput estimates assume decoding is memory-bandwidth bound: each token reads the active weights once, at ~55% of peak bandwidth.