Cloud GPU Pricing Explained

Every GPU cloud provider offers some combination of three pricing models:

On-demand — pay by the hour (or second/minute), no commitment. Start and stop anytime. This is the most common and most expensive option. An H100 on-demand ranges from $2.00-$4.00/GPU/hour depending on provider.

Spot / Preemptible — discounted instances that can be interrupted with little notice (typically 30 seconds to 2 minutes). Prices are 50-80% lower than on-demand. Great for training jobs that can checkpoint and resume.

Reserved — commit to 1 or 3 years, get 30-60% off on-demand pricing. Guaranteed capacity. Used by companies with predictable, steady GPU needs.

Providers bill at different granularities:

For short, iterative workloads (experimentation, hyperparameter tuning), per-second billing can save 20-40% compared to per-hour billing.

Raw instance prices are misleading. An AWS p5.48xlarge costs $98.32/hr, but that's for 8 GPUs. A Lambda H100 costs $2.49/hr for 1 GPU. You can't compare $98.32 to $2.49 — you need to normalize.

GIS solves this with the normalized section:

• cost_per_gpu_hour — the price per individual GPU per hour. AWS: $98.32 ÷ 8 = $12.29/GPU/hr. Lambda: $2.49/GPU/hr.
• cost_per_tflop_hour — price per TFLOP of compute. Accounts for different GPU performance levels.
• vram_per_dollar — GB of VRAM per dollar per hour. Useful for memory-bound workloads.

Always compare cost_per_gpu_hour as the baseline. Use cost_per_tflop_hour when comparing different GPU models (e.g., A100 vs H100).

{
  "pricing": {
    "currency": "USD",
    "billing_unit": "per-hour",
    "billing_granularity": "per-second",
    "on_demand": 2.49,
    "spot": null,
    "reserved_1yr": null,
    "reserved_3yr": null
  },
  "normalized": {
    "cost_per_gpu_hour": 2.49,
    "cost_per_tflop_hour": 0.00252,
    "vram_per_dollar": 32.13
  }
}

The pricing section captures the raw price structure. The normalized section provides computed comparison metrics. Together, they give you everything needed to compare any two GPU offerings on equal footing.