agent-observability-eval-pipeline
agent-observability-eval-pipeline is an agent skill published by datadog-labs, listed under deploy & operate. It lives in the datadog-labs/agent-skills repository, which has 178 GitHub stars and is licensed under MIT; we last read it on 2026-10-23.
This pipeline classifies production traces, roots-cause failures, and bootsrap evaluators, then optionally samples and publishes a dataset, generates and runs an experiment, and analyzes results. It connects to an instrumented ml_app and requires no specific credentials.
Install
skills is a generic installer for any repository of Claude agent skills -- it reads datadog-labs/agent-skills’s own structure rather than assuming one, the same way this page reads the skill’s own SKILL.md rather than assuming a format.
npx skills add datadog-labs/agent-skills
Maintenance
End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause failures, bootstrap evaluators, then (optionally) sample + publish a dataset, generate + run an experiment, and analyze results. Six narrated phases with a standardized banner and a "continue" checkpoint between each. Pure orchestration over the agent-observability sub-skills (`agent-observability-session-classify`, `agent-observability-trace-rca`, `agent-observability-eval-bootstrap`, `agent-observability-experiment-bootstrap`, `agent-observability-experiment-analyzer`). Use when user says "run the eval pipeline", "go from traces to evals", "bootstrap evals end to end", "classify then RCA then bootstrap", "build an eval set from scratch", "onboard me to datasets and experiments", "walk me through experiments", "I have an ml_app, now what", "Agent Observability onboarding", "guided experiment setup", "from traces to experiments", or wants a deterministic, narrated tour from production data through evaluators, datasets, and experiments. Stop early with `--stop-after <phase>` to short-circuit at evaluators or dataset, or resume mid-flow with `--start-at <phase>`.
agent-observability-eval-pipeline compared with similar skills
| Skill | Publisher | What it does | Repo stars | Last update |
|---|---|---|---|---|
| agent-observability-eval-pipeline (this page) | datadog-labs | End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause f… | 178 | 2026-10-22 |
| agent-observability-eval-bootstrap(same repository) | datadog-labs | Bootstrap evaluators from production traces — by default propose online LLM-judge evaluators and, after you c… | 178 | 2026-10-22 |
| agent-observability-experiment-analyzer(same repository) | datadog-labs | Analyze LLM experiment results. Handles single or comparative experiments, exploratory or Q&A modes. Use when… | 178 | 2026-10-22 |
| agent-observability-trace-rca(same repository) | datadog-labs | Root cause analysis on production LLM traces. Diagnoses why an LLM application is failing — works from eval j… | 178 | 2026-10-22 |
| agent-skills(same repository) | datadog-labs | Datadog skills for AI agents. Essential monitoring, logging, tracing and observability. | 178 | 2026-10-22 |
| dd-aws-integration(same repository) | datadog-labs | Set up the Datadog AWS integration with Terraform - creates the cross-account IAM role Datadog assumes (exter… | 178 | 2026-10-22 |
| dd-azure-integration(same repository) | datadog-labs | Set up the Datadog Azure integration with Terraform - creates an Entra ID app registration and service princi… | 178 | 2026-10-22 |
Skills from the same repository first, then others for this job by repository stars. Figures are from each repository when we last read it.
Common questions about agent-observability-eval-pipeline
- What does agent-observability-eval-pipeline do?
- This pipeline classifies production traces, roots-cause failures, and bootsrap evaluators, then optionally samples and publishes a dataset, generates and runs an experiment, and analyzes results. It connects to an instrumented ml_app and requires no specific credentials.
- How do I install agent-observability-eval-pipeline?
- Run npx skills add datadog-labs/agent-skills. The skills installer reads the repository’s own structure, so it works for any repository of agent skills.
- Who publishes agent-observability-eval-pipeline?
- datadog-labs.
- Is agent-observability-eval-pipeline maintained, and what licence does it use?
- Its repository has 178 GitHub stars and its state in our directory is listed. Its last commit we recorded was on 2026-10-22. The repository declares the MIT licence.
More agent skills for deploy & operate
- ai-content-pipelineby 101-skills
- explainer-video-guideby 101-skills
- ci-cd-and-automationby addyosmani
- observability-and-instrumentationby addyosmani
- internal-commsby anthropic
- xlsxby anthropic