LLM inference egress: per-token bandwidth math
For self-hosted LLM APIs, every response token leaves your cloud as egress. One token averages 4 bytes (UTF-8) plus SSE/JSON framing overhead (~2 bytes), so call it ~6 bytes per token. A 1000-token response is ~6 KB.
Per-million-token egress cost
| Volume | Bytes egressed | AWS direct | CloudFront NA/EU | Cloudflare R2 |
|---|---|---|---|---|
| 1M tokens | ~6 MB | $0.00 (within 100 GB free) | $0.00 | $0.00 |
| 1B tokens | ~6 GB | $0.00 (within 100 GB free) | $0.00 (within 1 TB free) | $0.00 |
| 100B tokens | ~600 GB | $45 | $0.00 (under 1 TB free) | $0.00 |
| 10T tokens | ~60 TB | ~$5,113 | ~$4,700 | $0.00 |
| 100T tokens | ~600 TB | ~$34,606 | ~$27,668 | $0.00 |
Excludes inbound prompt tokens (free as ingress on all clouds). Assumes typical OpenAI-style streaming JSON envelope ~6 bytes / token. Larger envelopes (function calling, tool use) can be 12-20 bytes / token. Dollar figures are computed from the standard US/EU tier tables with free allowances applied: AWS direct after the 100 GB free tier, CloudFront after the 1 TB free tier. See the AWS and CloudFront tier breakdowns.
Egress vs compute cost
For an H100 GPU serving Llama-3 70B at ~2,000 output tokens/sec on a single GPU, hourly compute is ~$3 (AWS p5.48xlarge fractional). At full throughput that is 7.2M tokens/hr = 5.2B tokens/month. That is only ~31 GB of token bytes, which sits inside the 100 GB free tier, so text-only inference egress is effectively $0 even before you optimise. Egress is <0.05 percent of inference cost at this scale.
For high-RPS API serving with multi-modal outputs (image generation, audio synthesis), egress becomes material:
- Image generation: 1 MP PNG at 200 KB. 1M requests / month = 200 GB egress = $18 AWS direct.
- Audio synthesis (TTS): 1 minute at 192kbps MP3 ~1.5 MB. 1M generations = 1.5 TB = $135 AWS direct.
- Video generation: 5-second clip at 720p ~10 MB. 1M generations = 10 TB = $900 AWS direct.
At those volumes, putting outputs on R2 with a signed-URL handoff cuts egress to zero and adds only R2 storage cost ($0.015/GB/month).
See also: Training data egress, Model & checkpoint transfer cost, Cloudflare R2 for media outputs.