Skip to content
InfratoApp
Layers / Observability / Evals / Evaluation and benchmarking / LangSmith

LangSmith

UNCLAIMED

By LangChain

EVALUATION AND BENCHMARKINGCOST AND USAGE TRACKINGTRACING AND MONITORING

LangSmith — Client library to connect to the LangSmith Observability and Evaluation Platform. Listed under Evaluation and benchmarking in the Observability / Evals layer. 7 of 12 comparison attributes are recorded, each with the page it was read from.

Complete AI agent and LLM observability platform with tracing and real-time monitoring. Debug agents, find failures fast, and track costs and latency.

2 OF 4 SIGNALS HELD · MISSING LICENCE, REPOSITORY

WebsiteDocsPricing page

What it runs on, and what runs on it

18 cited edges
The differentiator

Computed from the stack graph, both directions. Every cited edge links the page it was read from; a vendor-added one links the vendor’s own source — this is the reason to be here rather than on the vendor’s own site.

RUNS ON / INTEGRATES WITH

USED BY

Open-source signals

LICENCE
not measured
STARS
not measured
DOWNLOADS 30D
24,619,866
LAST COMMIT
not measured

A HOLLOW RING IS UNMEASURED, NOT ZERO · WE DO NOT SUBSTITUTE A PARENT COMPANY’S REPOSITORY FOR A PRODUCT THAT HAS NONE

Price

We hold no price for this product.

The seller publishes a pricing page, linked above, but we read no rate from it. Either it quotes by sales call rather than in public, or the last reading passed 21 days and was hidden rather than shown stale. If that changes — or if you publish /.well-known/pricing.json — it appears here with a capture date.

How we read prices →

What its categories ask

7 answered · 5 unanswered

12 questions declared by Evaluation and benchmarking and Cost and usage tracking and Tracing and monitoring. Most rows across the directory read like these.

ATTRIBUTEANSWEREVIDENCE · SOURCE
AlertingYes
“LangSmith provides threshold-based alerting on the following metrics:”
2026-10-06
Cost attributionRun Type, Tag, Error
“For **Errors** and **Latency**, you can use the filter builder to stack conditions on fields such as **Status**, **Run Type**, **Tag**, and **Error**.”
2026-10-20
Data residencyUS control plane
“BYOC is generally available (GA) on AWS with a US control plane. Support for additional cloud providers is planned for the second half of 2026. See [Regions and cloud providers](/langsmith/byoc#regions-and-cloud-providers) for the supported AWS regions.”
2026-10-06
Deploymentcloud only2026-09-23
Human in the loopYes
“To review, edit, and approve tool calls in an agent or workflow, use LangGraph's [human-in-the-loop](/oss/python/langgraph/interrupts) features.”
2026-09-29
LLM-as-judgeYes
“View instructions for [configuring an LLM-as-a-judge evaluator in the Playground.](/langsmith/llm-as-judge?mode=ui)”
2026-10-06
OpenTelemetryYes
“Native OpenTelemetry child spans are no longer dropped when they arrive before an SDK-attributed parent span; they are buffered and correctly nested regardless of arrival order.”
2026-09-29
Budget limitsno answer on record
Dataset managementno answer on record
Framework integrationsno answer on record
Licenceno answer on record
Open sourceno answer on record

A ROW WITH NO ANSWER IS NOT A ROW WITH A “NO” · A VENDOR MAY ANSWER A BLANK ONE, MARKED VENDOR CLAIMED WITH ITS SOURCE · A DOCUMENTED VALUE IS DISPUTED, NOT OVERWRITTEN, AND STAYS VISIBLE WHILE WE RE-READ THE SOURCE

Practitioner reviews

Nobody has written one yet.

A review here is structured, not a star: hardware, scale band, what broke in production, and whether you would choose it again.

NO PERCENTAGE IS SHOWN UNDER FIVE REVIEWS · NO STAR RATINGS, EVER

More from LangChain

3 other products from LangChain. Listed under the same organisation record — see the company page.

LangChainBuilding applications with LLMs through composabilityOrchestration / AgentsLangGraphBuilding stateful, multi-actor applications with LLMsOrchestration / AgentsLLangMemLangChain's library for giving agents long-term memory across conversationsOrchestration / Agents

Also in this category

53 products in Evaluation and benchmarking. Comparison defaults to within the category — pairing this with a GPU cloud would produce a page that means nothing.

langfuseUSED BY 16COMPARE →
Arize PhoenixUSED BY 4COMPARE →
BraintrustUSED BY 3COMPARE →
RagasUSED BY 2COMPARE →
Weights & Biases WeaveUSED BY 2COMPARE →
opikUSED BY 1COMPARE →
PromptfooUSED BY 1COMPARE →
Traceloop OpenLLMetryUSED BY 1COMPARE →

USED BY = LISTED PRODUCTS DEPENDING ON IT, IN THIS DIRECTORY

Frequently asked

What does LangSmith run on?
Google DeepMind, OpenAI, OpenAI Agents SDK. Every one of those is an edge we recorded from documentation that states it, not an inference from what the two products have in common.
Does LangSmith offer alerting?
Yes. Read from their own page: “LangSmith provides threshold-based alerting on the following metrics:”.
Does LangSmith offer human in the loop?
Yes. Read from their own page: “To review, edit, and approve tool calls in an agent or workflow, use LangGraph's [human-in-the-loop](/oss/python/langgraph/interrupts) features.”.
What does a blank mean on the LangSmith page?
That nobody has told us, not that the answer is no. A hollow ring is "no answer on record", which is a different state from a recorded no — and the two look different on purpose. A blank is an absence of evidence, and this page shows how many of its questions are in that state rather than quietly leaving them out.
Where do the LangSmith figures come from?
Each one is read from a public page — usually the vendor's own documentation or pricing page — and carries a link to it with the date it was read. Nothing here is estimated, averaged across sources, or inferred from a similar product. If a figure cannot be traced to a page, it is not published.
Can LangSmith change what this page says?
They can claim the listing and supply or correct values, which are then labelled as theirs. They cannot remove the page, hide a comparison they are losing, or change a figure somebody else recorded with a source. Nothing on the page is for sale. How the site is paid for is set out under money and independence at /our-data#sec5.
Something here is wrong — how do I fix it?
Dispute the value. Every figure links to the page it was read from, so a dispute names the value and gives a source that contradicts it. That is the only lever anyone has over this page, and it is the same lever for the vendor, a competitor and a reader.
Is this listing run by LangChain?
No. It is built from public sources, and no value on it came from the company. Its website is on shared hosting, so an email address cannot prove who owns it and there is no claim by address. A correction with a source can be sent instead. A claim lets the company write the tagline, description and logo, and answer blank questions or add dependencies, each marked as theirs with a source. Prices, licence and OSS figures stay as the directory read them, category changes only through our review, and each can be disputed with a source.

Cite this page

A model that quotes us should quote us correctly. The reading and its date sit inside the citation rather than beside it, because a quotation carrying only a URL goes wrong the week the URL changes.

infratoapp, “LangSmith — Evaluation and benchmarking”, read 6 October 2026. Alerting: Yes as published at docs.langchain.com. https://www.infratoapp.com/products/langsmith

What LangSmith can and cannot change here

Claiming LangChain lets its admins and editors change the tagline, description, logo, docs link and deployment model of every product that org owns — live, labelled as theirs, and logged. Answers to blank questions and dependencies go live too, marked as the owner’s and sourced; a documented answer cannot be overwritten. Everything else on this page is a stated fact with a dispute route: price, licence and OSS figures. Those columns are ungranted at the database level, so they are not a feature we withheld — they are a write that cannot happen. Name, category, website, pricing page and package names can be changed by the owner only through our review. A dispute never deletes a value; it marks it and publishes the correction.

LangChain’s website is not recorded, so an email address cannot prove who owns it and there is no claim by address. If you maintain it, send a correction with a source.

Maintain LangChain? Ask us to verify you.