GPUs for serving a model

Enough memory to hold the weights, then throughput for what you pay.

16GB or more of memory, cheapest on-demand rate first.

Serving needs the weights resident and little else — no gradients, no optimiser state, no activations held for a backward pass — so the floor sits below the fine-tuning one rather than level with it. 16GB holds a 7B quantised. Renting a serving GPU is often the wrong call outright — a token API is cheaper below a few million tokens a month, and the calculator will tell you where the crossover is.

20 priced parts carry no memory figure on record and are excluded rather than assumed to qualify. Unknown is not a yes.

It orders by hourly rate, which is not the same as cost. A card at half the price that takes three times as long is dearer, and we hold no benchmark that would tell you which — so no page here claims a cost per unit of work. Nor does it know about FP64 throughput, interconnect bandwidth or encoder blocks. If those decide your choice, this page has narrowed the field and not made the decision.