A100 pricing: SXM, PCIe and the unlabelled rate

The A100 sits in more than one row

A100 rates are held in separate rows. The A100 SXM 80GB, named in full by its sellers, has a lowest listed on-demand rate of $0.41 and a highest of $2.79, across 9 sellers. The plain A100 row, for rates whose page names no form factor, has a lowest of $1.09 and a highest of $5.04, across 12 sellers. Each figure is one seller's cheapest on-demand list rate from its own page.

Both rows have a page: the A100 SXM 80GB page and the plain A100 page. The lowest rate in the named row comes to $299.30 over a 730-hour month, and the lowest in the plain row to $795.70.

One name, several products

The A100 is an Ampere card that comes with 40 GB or 80 GB of memory, as an SXM module or as a PCIe card. SXM modules sit on a board with a fast GPU-to-GPU link and PCIe cards sit in an ordinary slot, so the form factor changes what a buyer can do with several cards together.

A seller that writes only A100 has described any of those. The buyer cannot tell which, and neither can anyone building a table from the page. That is why the plain row exists: rather than guess a form factor or a memory size, we keep the rate under the name the seller used.

The result is a row whose rates may describe different products. Its low end and its high end need not be the same card, and that is a fact about the labels, not about the market.

Why the memory size changes the comparison

Weights need roughly the parameter count times the bytes per parameter. A 13 billion parameter model in 16-bit needs about 26 GB and fits on either memory size. A 30 billion parameter model in 16-bit needs about 60 GB, which fits on an 80 GB card and not on a 40 GB one. The key-value cache and activations come on top and vary with context length and batch size.

So a buyer reading an unlabelled A100 rate does not know whether the card holds the model they have in mind. A rate that cannot answer that question is hard to put beside one that can.

The same is true of the form factor. A buyer planning to run a model across several cards cares whether those cards share a fast link, and an unlabelled rate does not say. For a single-card job the form factor matters less, but the buyer still has to find out which one is on offer before trusting the rate.

How the A100 rows are filled

Each seller's own pricing page is read and one rate per card is kept, the cheapest on-demand list rate, per GPU per hour. The name on the page decides the row. A page that names the SXM 80GB lands in that row; a page that says A100 and nothing more lands in the plain one. Neither row is merged into the other.

A rate whose last good read is more than 21 days old is hidden. For a card that has been on sale as long as the A100, sellers sometimes leave old pages in place, and a page that stops being readable takes its rate out of whichever row it was in. The method page describes the reads.

For a seller in the plain row

For a rate in the plain A100 row, the quickest improvement costs no money: name the form factor and the memory on the page. A100 SXM 80GB, or A100 PCIe 40GB, in the product name rather than in a spec sheet further down.

Then look at where the named rows put you. The lowest named SXM 80GB rate is at Vast.ai, and the lowest in the plain row is at Civo. Once your page names the card, your rate is compared against the sellers who sell the same thing, which is the comparison a buyer is trying to make.

Moving rows can move a rate's position in its range. A rate that sat at the low end of the plain row may land mid-range in the named one, or the other way round.

A figure held for you in the wrong row can be corrected through a claimed listing. If you believe we read the page wrongly, a dispute with the page as evidence is the route.

What the two rows cannot settle

The plain row cannot say which A100 each rate describes, and the post does not guess. Comparing the plain row with the named row is therefore not a like-for-like comparison, and no conclusion about SXM against PCIe pricing should be drawn from it.

Neither row says anything about hours sold, availability or private terms, and neither weights a large seller above a small one. They are list rates, one per seller, read on the date shown beside each.

Nor does either row say why a seller set its A100 rate where it did. The A100 has been on sale longer than most cards in the table, and a list rate carries no reason with it.