observability-oss skills & plugins for progress-observability
Open source

Open-source tools for agent observability.

Plugins, skills, and getting-started templates that connect Claude Code and GitHub Copilot to the Progress Observability Platform — read traces, track cost, and build evals over MCP, from your coding agent.

Get started GitHub
Install
1
Set your MCP key (Progress Observability → API Keys)
export OBSERVABILITY_MCP_API_KEY="mcp_…"
2
Add the plugin and install
/plugin marketplace add \
  observability-oss/progress-observability-plugin
/plugin install progress-observability
3
Ask in plain language
"why did checkout-agent stall?"
Overview

The toolkit

One read-only MCP connection to the Progress Observability Platform, five skills on top of it — four that chain into a triage-to-eval loop, plus a setup check you run first. Shipped from a single source as a Claude Code plugin and a VS Code / Copilot setup, so the same skills run in either.

progress-observability Claude Code Copilot

Four read-only workflows over the Progress Observability MCP server — trace triage, cost and usage reports, eval coverage gaps, and LLM-as-a-Judge eval generation grounded in your real traces — plus health-check, a setup diagnostic you run first.

/trace-triage /cost-report /coverage-gaps /eval-from-trace /health-check

Run it three ways — a Claude Code plugin, a VS Code / Copilot setup, or a single lifted skill in any agent. Install steps below.

Repository Docs Install
Five skills

Each does one job well

Use them through the plugin, or lift a single skill into any agent. All read-only, all treating trace content as untrusted data.

/trace-triage

Root-cause a run

Walk a failed or slow run's span tree, find the fault, hand back a diagnosis and a fix.

 /cost-report

Watch the spend

Cost by model, app, and day; quota burn; and what drove a spike — decision-ready.

 /coverage-gaps

Find the blind spots

Which production behaviors have no eval measuring them, ranked by what to build first.

 /eval-from-trace

Build the judge

Research-grounded LLM-as-a-Judge evaluator prompts, grounded in your real traces.
/health-check

Check your setup

Read-only diagnostic — connection, key scope, whether traces are flowing, and how deep your instrumentation goes. The four above chain into a loop; this one you run first.

Setup · run first

Docs — how each skill works, what it reads, and how to run it → · Skill source on GitHub →

How it works

Triage a failure, find the gap, build the eval, track the cost

The four skills chain into one loop — each stage's output is the next stage's input.

01
trace-triage

A run fails. Find the failure signature in the spans.

02
coverage-gaps

Confirm nothing measures that behavior yet.

03
generate-eval

Build the judge that catches it going forward.

04
cost-report

Watch spend while you iterate on the fix.

Roadmap

Getting-started templates Coming soon

Starter repos pre-wired for observability — clone, add your keys, and you have a traced agent to build on. Listed here as they're published.

Using these tools, or want a say in what gets built next? Reach the maintainer on LinkedIn or Discord (Lyubomir Atanasov).

Quickstart

Running in three steps

You'll need a Progress Observability account and an MCP API key (paid plan). A Metadata-only key covers every skill except quoting raw trace text.

1 — set your key · Progress Observability → API Keys → MCP · both setups
export OBSERVABILITY_MCP_API_KEY="mcp_…"

Claude Code

2 — add the marketplace and install
/plugin marketplace add observability-oss/progress-observability-plugin
/plugin install progress-observability
3 — ask in plain language
"why did checkout-agent stall in the last hour?"
"what's driving my spend this week?"
"what should I evaluate next?"

VS Code · Copilot

2 — copy the Copilot setup from the repo into your project
cp -r copilot/.vscode copilot/.github your-repo/
# then reload VS Code and start the progress-observability MCP server
3 — run a prompt in Copilot Chat (agent mode)
/trace-triage   # or just ask in plain language