Compare GPU providers fairly: same card, same terms, same unit

Fix three things, then compare

To compare GPU providers fairly, hold three things the same on every row: the exact card including form factor and memory, the commitment (on-demand, reserved or interruptible), and the unit, which should be price per GPU per hour. Only then is a difference in rate a difference in price. After that, add the costs each rate leaves out, because two equal rates can produce unequal bills.

A comparison that lets one of the three vary can break without anyone noticing. The cheapest row in a mixed table can turn out to be a different product.

A fair comparison is also a dated one. Rates change, and a sheet that mixes a rate read this week with one read in spring is comparing moments in time, not sellers. Read every page within the same few days and write the date beside each rate.

The card: read past the family name

An H100 is not one product. The SXM part sits on a board with a fast link between cards; the PCIe part sits in an ordinary slot. They are different purchases at different prices. The H100 PCIe page shows a cheapest rate on record of $1.66 (sellers with a listed rate: 7); the H100 SXM page shows $1.68. Putting one provider's PCIe rate next to another's SXM rate compares different hardware, not different prices.

Memory size is the other half of the name. An A100 can be 40 GB or 80 GB. Where a provider's page leaves either detail out, write "unknown" in your table rather than guessing, and ask.

On this site, an A100 rate listed without a form factor sits on its own page rather than being merged with the SXM 80GB part. Apply the same caution in your own sheet: a rate you cannot place on an exact card does not go in the row for that card.

The terms and the unit

Label every rate with its commitment. An interruptible rate can be reclaimed mid-job and a reserved rate needs a signed term; neither belongs in a column of on-demand prices. A provider that publishes only reserved rates goes in a separate table.

Convert every rate to per GPU per hour. An instance price for a machine with several cards is divided by the card count. When the instance includes generous CPU, memory or disk, note it in a separate column instead of adjusting the rate by a guess.

Note the billing increment as well. For long runs it changes little; for many short tests, a provider that rounds each session up to the hour will cost more than its rate suggests.

Watch for minimums too. A provider that rents only for a minimum number of hours, or only whole machines, has a rate that is fair per hour but not available for the job you have. Put the minimum in its own column so it is seen before the rate is.

Using the site to do the first pass

The card pages hold the card fixed: one exact card per page, rates per GPU-hour, with each seller's source page and the date it was read. Check the terms on each row before setting it beside another. The GPU table lists every card and the compare page lets you put products side by side.

What those pages cannot do is the second pass. Storage, egress, region, support and availability are not in the rate and are not held here. Take the shortlist from the card page to each provider's own documentation for those columns.

Keep the shortlist short. Pick the sellers whose rate and card fit, then do the slow work of reading terms only for those. Reading every provider's terms for every card is how a comparison stalls before it produces an answer.

What the sheet should look like

One row per provider for one exact card. Columns, in order: card name as the provider writes it, card name as you have confirmed it, commitment, rate per GPU-hour, billing increment, region, smallest unit you can rent, what is bundled, and the date you read the page.

Leave the cells you cannot fill visibly empty. The habit of writing a guess into a cell is how a comparison drifts from fact to impression without anybody deciding it should. An empty cell is also the list of questions to send.

Keep cards on separate sheets. When choosing between an H100 and an H200, compare sellers of each card first, then compare the cheapest fit for each card against the other with your own job in mind. Mixing them in one sheet makes the cheaper card look like the cheaper provider.

The second pass: what the rate leaves out

Add columns for persistent storage per gigabyte per month, egress per gigabyte, whether stopped instances bill for disk, region, and how the provider handles a failed host. Fill each from the provider's own page and date it. Where a page says nothing, write "not stated".

Then work out a month for your own workload, not for an always-on machine unless that is your workload. The rental cost calculator gives the GPU part per seller. Add your storage and egress estimates to each row and sort by the total. The order can differ from the order by hourly rate.

An unchanged order means the hourly rate was a fair guide for your workload. Either way you now know which columns mattered, and the next comparison can start from them.

Keep the sheet. Rates change and a comparison you made in spring is not one to sign on in autumn.