Skip to content
InfratoApp
Layers / Inference / Serving / Inference engines

Inference engines

COMPARE TWO OF THESE →

2 of 47 inference engines listings publish a rate we can read. On z-ai/glm-5.3-flash, the one most of them price, rates run from $0.1500/1M in to $0.1500/1M in across 2 providers — a spread of 1.0×, median $0.1500/1M in. Each rate is read from the provider's own pricing page.

Software that actually executes a model and serves tokens: batching, KV cache, quantisation, streaming. Running it yourself is the point; paying somebody per token is an inference API.

47 in this category
2 PRICED · SORT AND VIEW BELOW

NARROW BY WHAT THIS CATEGORY ASKS

HARDWARE · 9 NOT READ

LICENCE · 12 NOT READ

OPEN SOURCE · 25 NOT READ

CHEAPEST · z-ai/glm-5.3-flash

$0.1500/1M in

MEDIAN

$0.1500/1M in

SPREAD

1.0×

PROVIDERS

2 of 47

Above: one card, z-ai/glm-5.3-flash, across the 2 providers selling it — the only like-for-like comparison here. Below: each provider’s own cheapest rate, on whatever card that is. They are different questions and the second one is not a ranking.

NO RATE ON RECORD · 45

We have not read a published rate for these. A dash in a price column reads as cheap; this reads as what it is. Some publish nothing, some quote on request, and some we have simply not reached.

Ollamallama.cppvllmLocalAIText Generation WebUIAnyscaleRay ServeSGLangllamafileMLC LLMopenvinokimi-k3-in-cLMDeploydynamovLLM-Omniturbo-fieldfarecactusshimmygpustackCTranslate2ExLlamaV2lightllmRapid-MLXSuperlinkedspiceaidistributed-llamachituaudio.cppopenlakeeaglemtplxSQLite AIinferencexxllmatomic-chatkvcachedrtp-llmgomodelvLLoraPegaInferLightSeek Foundationjetstreamhalogen-flash-serverGMI CloudNVIDIA NIM

ONE RATE PER PROVIDER — ITS CHEAPEST, WITH THE SKU IT IS FOR · A MEDIAN NEEDS EVERY OTHER SELLER’S PRICES, WHICH IS WHY NO VENDOR PUBLISHES ONE · EVERY RATE CARRIES THE PAGE IT WAS READ FROM AND THE DATE

SORTED BY ONE STATED COLUMN · USED BY = LISTED PRODUCTS DEPENDING ON IT, IN THIS DIRECTORY — NOT A POPULARITY FIGURE · NO OVERALL SCORE AND NO RANK COLUMN, BECAUSE BLENDING ADOPTION, STARS AND DOWNLOADS NEEDS WEIGHTS, AND WEIGHTS ARE AN OPINION PRESENTED AS ARITHMETIC

Cite this page

The count and its date sit inside the citation, because “Inference engines” without a date is a claim that quietly goes wrong as the category fills.

infratoapp, “Inference engines — Inference / Serving”, read 28 October 2026. 47 listed products. https://www.infratoapp.com/layers/inference/inference-engine

No overall score, and no rank column

Blending adoption, stars and downloads needs weights, and weights are an opinion presented as arithmetic — so this table sorts by one stated column at a time and shows which. This view holds no billing column and joins no billing table: a sponsor cannot appear higher, and the signals meter is a row’s own confidence, never a score you can sort on.

Frequently asked

What is in the Inference engines category?
47 products in the Inference / Serving layer of the AI stack. Software that actually executes a model and serves tokens: batching, KV cache, quantisation, streaming. Running it yourself is the point; paying somebody per token is an inference API. Listed members include Ollama, llama.cpp and vllm. Every fact on each listing carries the page it was read from and the date it was read.
How are Inference engines compared here?
By the 9 attributes this category declares, asked of every member and answered only where a source states it. A product with no answer on record shows a gap rather than a guess, and no blended score is published — weights would be an opinion presented as arithmetic.
Ollama or llama.cpp?
Both are listed under Inference engines. We publish no verdict between them — we hold no benchmark that would support one. What we do hold is the same attribute sheet for each, side by side with the source behind every value, at /compare/llama-cpp-vs-ollama.
What does a blank mean on a inference engines row?
That nobody has told us, not that the answer is no. A hollow ring is "no answer on record", which is a different state from a recorded no — and the two look different on purpose. A listing with a blank has not failed the question; it has not been read for it.
How often are inference engines figures re-read?
Every value carries the date it was read and a link to the page it came from. Prices move fastest and are re-read most often; an attribute read weeks ago says so rather than presenting itself as current. Anything past 21 days is hidden rather than shown stale.
Is this every inference engine there is?
No. This is 47 we have read and can cite. A directory that claimed completeness would be making a promise it cannot keep, so the page states its own count instead and says when a category is too thin to shortlist from.
Can a vendor pay to rank higher here?
No. Nothing on this page can be bought: a sponsor cannot enter a table, reorder a row or change a mark, and there is no ranked position for sale. How the site is paid for, and the line each source of money does not cross, is set out under money and independence at /our-data#sec5.