Rent for a week or commit for a year

The break-even in one sentence

A reservation is worth it when the hours you will genuinely use over the term, multiplied by the on-demand rate, come to more than the total you commit to pay for the reservation. Put the other way: divide the reserved total by the on-demand rate and you get the number of on-demand hours it buys. Expect more GPU-hours than that during the term and the reservation wins; expect fewer and on-demand costs less.

Everything else in the decision is about how confident you are in that expected figure, and what the commitment ties you to while you find out whether you were right.

This post gives the method and quotes no discount. Reserved prices vary by provider, card, region and term, so a single figure would mislead, and the quote you get is the one to use.

Working it through without a discount figure

Take the on-demand rate for your exact card. Take the provider's reserved quote for the same card, same region, same term, and turn it into a total for the term. Divide the total by the on-demand rate. The answer is your break-even hours.

Compare that with the hours in the term. A year is 8,760 hours. A break-even close to that needs the card busy almost all the time to come out ahead. One well below it leaves room for a part-time workload to justify the commitment. The ratio of break-even hours to hours in the term is the utilisation you must beat.

The reserved quote has to come from the provider. The on-demand side is on the card pages, and the rental cost calculator gives the on-demand cost for any number of hours, which is the figure to set the quote against.

Estimating your own utilisation honestly

Use history, not hope. After a spell on demand, add up the hours the instances were actually running last month and the month before. Those hours are the most reliable forecast you have, and a first rental on demand is how you get them.

Separate the floor from the peaks. Serving traffic often has a level that never drops, plus busy hours. The floor is the part that suits a reservation; the peaks suit on-demand. Reserving for the peak means paying for the quiet hours too.

Discount for things ending. A research project, a contract with one customer or a model that may be replaced can each stop the work before the term does. Multiply your expected hours by how likely you think the work is to continue.

What a year commits you to besides money

The card. A reservation on the H100 is still on the H100 when a newer part arrives at the same provider. For the B200 SXM 192GB, for example, the B200 page shows sellers with a listed on-demand rate: 22; always-on month at the cheapest rate: $2,730.20. Whether a newer card will be cheaper per unit of your work in a year is something nobody can promise you.

The region and the provider. When your data, your users or your rules move, the reservation does not. Ask whether it can be moved, resold, exchanged for another card or ended early, and at what cost.

The payment schedule. Some terms are paid up front and some monthly. Up front ties up cash; monthly may carry a higher total. Compare the totals, not the monthly figures.

The terms of exit. Read what happens if the provider changes its hardware, retires the card or fails to deliver the capacity you reserved. A commitment runs both ways, and the page that describes your obligations should also describe theirs.

A worked example in words

Suppose a provider quotes a reservation for one card for a year. Turn the quote into a single total for the year. Divide it by the on-demand rate for the same card in the same region. Say that gives a figure somewhat below the 8,760 hours in a year: that figure is your break-even hours.

Now take your log of on-demand use and scale it to a year. Realistic yearly hours above the break-even mean the reservation saves money, as long as the work continues for the whole term. Below it, on-demand is cheaper even though its hourly rate is higher.

Then run the same sum against the cheapest on-demand seller of the card, not just the provider that quoted. A reservation is a discount on one provider's rate. Where another provider's on-demand rate is already close to the reserved one, the commitment buys very little.

A week, a month, a year

For a week of experiments, on-demand is the usual answer, since a reservation term runs far past the week. For a few months of steady work, ask whether the provider offers a short term, and run the same arithmetic on it. For a year, run the arithmetic carefully and reserve only the floor you are sure of.

Before signing, compare on-demand rates across sellers for the same card on the GPU table. A reserved quote from one provider is not always below another provider's on-demand rate, and a cheap on-demand seller with no lock-in is sometimes the better deal.

Whatever you decide, set a date to look again. Usage changes, rates change and new cards arrive. Redo the arithmetic when the figures move.