On-demand, reserved or interruptible: which GPU rental fits

Three products that share one name

Rent on-demand when you do not know how many hours you will need, reserve when you can name the hours for months ahead, and choose interruptible only for jobs that checkpoint their state and restart on their own. The three are sold as the same card but are different products: on-demand is paid by the hour with no promise from you, reserved trades a commitment for a lower rate, and interruptible can be taken back by the provider while it runs.

Most first rentals should be on-demand. You learn how long your job really takes, how much memory it really uses and how much of the day the machine sits waiting for you. Those three facts are what the other two products ask you to guess.

On-demand is the price of not knowing yet

An on-demand rate is the one every comparison starts from, because it carries no conditions. For the H100 SXM 80GB, the cheapest on-demand rate per GPU-hour on record is $1.68 (sellers with a listed rate: 13), and every one of those rates is on the H100 SXM page with the page it came from and the date it was read.

You pay for that freedom in the rate, and you pay again for every hour the instance is up and doing nothing. On-demand suits a two-day experiment and costs most on a server that runs all year. Staying on it after usage has settled into a pattern you could commit to is where the money goes.

Check the billing increment before the first launch. A provider that bills by the second and one that rounds up to the hour charge very different amounts for the same short test, and a rate table may not say which is which.

Reserved capacity is a bet on your own forecast

A reservation lowers the hourly rate in exchange for paying for hours whether you use them or not. That turns the question from "what does this card cost" into "how sure am I that I will keep it busy". "Fairly sure, for most of the term" points to a reservation. A research project that might end next month points the other way, because the discount buys hours nobody will use.

Ask for the term, the payment schedule, whether unused hours roll over, and what happens if you need to change card or region halfway through. A reservation on last year's card is still a reservation on last year's card when the new one arrives. The break-even is a sum you can do yourself: divide the total you would commit by the on-demand rate for the same card, and that is how many on-demand hours the reservation is worth. Use more hours than that during the term and the reservation wins; use fewer and on-demand would have cost less.

Interruptible instances and the checkpoint rule

An interruptible instance can be reclaimed while your job is running. Some providers give a notice period first and some give none, and a marketplace may let another renter outbid you. The rate is lower because the provider keeps the right to stop you.

The rule is simple: if your job cannot write its state to storage that survives the instance, and pick up from that state on a new machine without you watching, do not run it on interruptible capacity. Training loops that checkpoint every so often, batch inference over a queue, and hyperparameter sweeps fit. A long training run that saves only at the end, or a live endpoint serving users, does not.

Count the cost of the restart, not only the rate. Every reclaim loses the work since the last checkpoint plus the time to start a new instance, pull the container and reload the weights. When that happens often, the cheaper rate can end up dearer than on-demand for the same finished job.

What to write down during the first month

Keep a plain log while you are on on-demand. Note when each instance started and stopped, what it ran, and whether the card was busy or waiting. A week of that tells you more about your real usage than any estimate made before the first launch, and it is the only honest input to a reservation decision later.

Note the peak memory as well. A job that never uses more than half the card may run on a smaller card at a lower rate. A job that runs out of memory shows it in the first hour, not on day twenty of a reserved term.

Finally, note what interrupted you. A failed host, a driver problem, a dataset that took a day to copy. These are the costs that do not show up in a rate, and they shape which of the three products suits the next phase of the work.

Matching the product to the work

Development and short experiments: on-demand, with a habit of stopping the instance when you stop working. Steady production serving with a predictable floor of traffic: reserve the floor and put the peaks on on-demand. Large batch jobs that checkpoint: interruptible, with the checkpoint interval set by how much lost work you can stand.

Mixed setups are normal. The rate you compare across providers must be the same product on both sides, though. An interruptible rate beside an on-demand rate makes the first provider look cheaper for a reason that has nothing to do with the provider. On the table of every GPU rate, check that the rates you set side by side carry the same terms.