GPU calculator
How much GPU does your model need?
Pick a model, precision, context length and concurrency. AWSBee computes memory from each model's architecture and ranks every NVIDIA instance type by fit and cost.
GPU memory needed
35.5 GBof 43.2 GB usable on g6e.xlarge
- Weights16.1 GB
- KV cache17.2 GB
- Runtime overhead2.3 GB
- Usable memory (90% of VRAM)
Cheapest instance that fits
g6e.xlarge · NVIDIA L40S
~30 tok/s per stream · up to 11 full-context requests · ≈ $1.86/h
All NVIDIA GPU instances
Approximate on-demand list prices for us-east-1. Prices differ by region and change over time; check the official EC2 pricing page before launching. Throughput is a memory-bandwidth estimate, not a benchmark.
| Instance | GPUs | Usable | Max conc. | Tok/s (1 stream) | $/hour | $/1M out tok* |
|---|---|---|---|---|---|---|
g6e.xlarge | 1× L40S | 43 GB | 11 | 30 | 1.86 | 2.80 |
g6e.2xlarge | 1× L40S | 43 GB | 11 | 30 | 2.24 | 3.37 |
g4dn.12xlarge | 4× T4 | 58 GB | 16 | 37 | 3.91 | 4.67 |
g6.12xlarge | 4× L4 | 86 GB | 29 | 35 | 4.60 | 5.86 |
g5.12xlarge | 4× A10G | 86 GB | 29 | 70 | 5.67 | 3.61 |
g6e.12xlarge | 4× L40S | 173 GB | 69 | 101 | 10.49 | 4.64 |
g6.48xlarge | 8× L4 | 173 GB | 67 | 70 | 13.35 | 8.49 |
g5.48xlarge | 8× A10G | 173 GB | 67 | 140 | 16.29 | 5.18 |
p4d.24xlarge | 8× A100 40GB | 288 GB | 120 | 362 | 21.96 | 2.70 |
g6e.48xlarge | 8× L40S | 346 GB | 147 | 201 | 30.13 | 6.66 |
p5.48xlarge | 8× H100 80GB | 576 GB | 254 | 780 | 55.04 | 3.14 |
g4dn.xlarge Needs 35.5 GB, 14.4 GB usable | 1× T4 | 14 GB | — | — | 0.53 | — |
g6.xlarge Needs 35.5 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.81 | — |
g6.2xlarge Needs 35.5 GB, 21.6 GB usable | 1× L4 | 22 GB | — | — | 0.98 | — |
g5.xlarge Needs 35.5 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.01 | — |
g5.2xlarge Needs 35.5 GB, 21.6 GB usable | 1× A10G | 22 GB | — | — | 1.21 | — |
* At 8 concurrent streams with continuous batching, fully utilized. Tensor parallelism uses up to 8 GPUs for this model.
How the numbers are computed
- Weights = parameters × bytes per parameter (2 for BF16/FP16, 1 for FP8).
- KV cache = 2 × layers × KV heads × head dimension × 2 bytes, per token, times context × concurrent requests.
- Overhead = 1.5 GB per GPU for the CUDA context and graphs, plus 5% of weights for activations.
- Usable memory is 90% of VRAM, matching vLLM's default
--gpu-memory-utilization. - Throughput estimates assume decoding is memory-bandwidth bound: each token reads the active weights once, at ~55% of peak bandwidth.