GPU hourly pricing
Raw GPU instances billed by time, often with storage, network and idle-capacity costs outside the headline rate.
Pricing
GPU Pricing Guide
Understand GPU pricing models across cloud GPUs, marketplaces, serverless inference and self-hosted infrastructure.
Executive Summary
GPU pricing is difficult to compare because providers package accelerator capacity in different ways. A team may buy raw hourly instances, marketplace machines, serverless GPU runtime, dedicated endpoints, token APIs or infrastructure operated inside an existing cloud account. Each model shifts cost between the provider bill and the engineering work required to keep workloads reliable.
The right comparison starts with workload shape. Training jobs care about sustained throughput, checkpointing and data movement. Production inference cares about latency, concurrency, autoscaling and uptime. Experiments care about access and flexibility. Enterprise deployments care about governance, procurement and predictable controls.
Token-based inference pricing
Managed LLM APIs priced by input and output tokens. Cost depends on context length, traffic mix and model choice.
Serverless inference pricing
Usage-based runtime pricing that can reduce idle cost but may add cold-start, concurrency or platform constraints.
Dedicated GPU instances
Reserved or dedicated capacity for predictable workloads, usually with stronger planning and commitment requirements.
Self-hosted infrastructure
Hardware or cloud infrastructure operated by the team, including engineering, observability, security and maintenance costs.
Unit of billing
Hourly instances, serverless runtime, tokens, committed capacity and managed services are not directly interchangeable.
Utilization
A low hourly rate is only useful when the workload keeps the GPU busy or the platform can scale idle capacity down.
Data movement
Large datasets, checkpoints, embeddings and generated outputs can create storage and network costs outside the GPU line item.
Operations
Scheduling, observability, security, image management and incident response are real costs even when they do not appear on the invoice.
Pricing Model Comparison
| Model | Best fit | Main risk | Cost control tactic |
|---|---|---|---|
| Hourly GPU cloud | Development, training, custom inference | Idle capacity and operations overhead | Automated shutdown, queues and utilization tracking |
| GPU marketplace | Flexible experiments and batch work | Host variability and reliability burden | Checkpointing and host benchmarking |
| Serverless GPU | Bursty inference and jobs | Cold starts, platform constraints and concurrency limits | Measure real request patterns and warm capacity needs |
| Managed token API | Fast product integration | Token growth and model lock-in | Prompt optimization, caching and model routing |
| Reserved capacity | Predictable production workloads | Overcommitment if demand changes | Commit gradually and compare against utilization history |
Decision Framework
- Classify the workload as training, fine-tuning, batch inference, real-time inference or development.
- Estimate utilization, concurrency, context size, data movement and storage growth.
- Decide whether the team can operate infrastructure or needs managed serving.
- Compare pricing models using total cost, not only headline compute rates.
- Validate exact terms directly with providers before committing.
Practical Recommendations
- Track GPU utilization and idle time from the first experiment.
- Separate development, staging and production cost centers.
- Use smaller models or quantized models when quality allows.
- Revisit reserved capacity only after usage patterns are stable.
Related Guides
FAQ
Does this page show live GPU prices?
No. It explains pricing models and cost drivers because live GPU prices, capacity and regional availability change frequently.
What costs are easy to miss?
Storage, networking, idle time, failed jobs, engineering operations, observability, support and committed-capacity terms are commonly underestimated.
Is the cheapest GPU hour always best?
No. Reliability, utilization, data movement, support and engineering time can outweigh a lower headline GPU rate.
When should teams consider dedicated capacity?
Dedicated or reserved capacity can make sense when usage is predictable, service-level expectations are high or procurement values stability over flexibility.