audit-activation
Audits OrchestKit sub-agent activation from real spawn telemetry — computes the generic-vs-specialist spawn split, flags dormant agents (never fired), and classifies each as fires/mis-triggered/niche. The agent-side analogue of audit-skills. Use when specialized agents feel under-used, before pruning the catalog, or after wiring new agent spawn paths.
pinned to #0e54dbcupdated 2 weeks ago
Ask your AI client: “install skills/audit-activation”.
Requires the metahub MCP server installed in your client. Set up MCP.
mh install skills/audit-activationmetahub onboarded this repo on the author's behalf.
If you own github.com/yonatangross/orchestkit on GitHub, claim the listing to take over publishing. Your claim preserves the existing eval history and badges; only the curator label is replaced with verified-publisher on your next publish.
Stars
201
Last commit
2 weeks ago
Latest release
published
- #agent-orchestration
- #agents
- #ai-agents
- #ai-development
- #anthropic
- #claude-code
- #claude-code-plugin
- #claude-plugin
- #developer-tools
- #fastapi
- #langgraph
- #llm
- #mcp
- #rag
- #react
- #security
- #testing
- #typescript
About this skill
Pulled from SKILL.md at publish time.
Reports whether OrchestKit's specialized sub-agents are actually being activated, from **real spawn telemetry** — not vibes. The agent-side analogue of `audit-skills` (which audits skill *quality*; this audits agent *activation*).
Evaluation report
WarningsAutomated checks the publisher passed at publish time — structure, docs, safety, and whether the artifact behaves as claimed.0e54dbc· 2 weeks ago
Kind-specific
31Skill: SKILL.md present
found at src/skills/audit-activation/SKILL.md · frontmatter source: SKILL.md
Skill: body content present
514 words · 4,168 chars · 7 sections · 2 code blocks
Skill: triggers declaredwarn
No `trigger` phrases in SKILL.md frontmatter
Add `trigger:` lines so Claude knows when to activate this skill — e.g. `when building MCP servers` or `for diagram creation`.
Skill: allowed-tools scope
no allowed-tools restriction (Claude may use anything)
Release history
1- releasecurrent0e54dbcwarn2 weeks ago
Contents
Reports whether OrchestKit's specialized sub-agents are actually being activated, from real spawn telemetry — not vibes. The agent-side analogue of audit-skills (which audits skill quality; this audits agent activation).
It answers: "Do my specialized agents get spawned, or does the model default to generic Explore/general-purpose?" Read-only — it never edits agents.
Why this exists: a 2026-06 audit found only ~14% of agent spawns hit the 37-agent catalog vs ~74% generic, with 17/37 agents dormant — and that agents fire ~1:1 with how often a high-traffic skill names them via
subagent_type=. Description rewrites ("use proactively") were A/B-tested and gave Δ0, so this skill measures wiring + usage, not description prose. Seedocs/feat--activation-audit/.
Quick Reference
| Category | File | Impact | When to Use |
|---|---|---|---|
| Activation Checks | ${CLAUDE_SKILL_DIR}/rules/activation-checks.md | HIGH | What to compute per agent |
| Classification | ${CLAUDE_SKILL_DIR}/rules/activation-status.md | HIGH | fires / mis-triggered / niche / dead buckets |
| Output Format | ${CLAUDE_SKILL_DIR}/references/output-format.md | MEDIUM | Report layout + the spawn-split summary |
CRITICAL: Task Management is MANDATORY (CC 2.1.16)
TaskCreate(subject="Audit activation: agent spawn analysis",
description="Computing generic-vs-specialist split + dormancy from spawn telemetry",
activeForm="Auditing agent activation")
TaskCreate(subject="Read spawn telemetry", activeForm="Reading subagent-spawns.jsonl")
TaskCreate(subject="Compute split + dormancy", activeForm="Computing split and dormant agents")
TaskCreate(subject="Classify & render", activeForm="Classifying agents and rendering report")
TaskUpdate(taskId="3", addBlockedBy=["2"])
TaskUpdate(taskId="4", addBlockedBy=["3"])
Workflow
- Inventory — Glob
src/agents/*.md(exclude README/INDEX/CONTRIBUTING) for the catalog. - Read telemetry — the FRESH stream is
.claude/logs/subagent-spawns.jsonl(writers:pretool/task/spawn-intent-logger+subagent-start/subagent-validator). The legacy~/.claude/analytics/agent-usage.jsonlis dead (orphaned in a refactor) — see edge cases. - Compute — run all checks from
Read("${CLAUDE_SKILL_DIR}/rules/activation-checks.md"): spawn split (generic / ork-catalog / other-plugin), per-agent fire counts, never-fired set, concentration (top-5 %). - Classify — apply
Read("${CLAUDE_SKILL_DIR}/rules/activation-status.md"): each dormant agent → mis-triggered (wired but no real spawn path) / niche (legit rare) / dead (no references anywhere). - Render — output using
Read("${CLAUDE_SKILL_DIR}/references/output-format.md").
Quick Start
bash "${CLAUDE_SKILL_DIR}/scripts/run-activation-audit.sh"
The script reads the spawn telemetry, joins it against src/agents/, and prints the split, top agents, never-fired list, and concentration. Then apply the classification rules to bucket the dormant agents (this step needs the model: it greps skills/agents for each dormant agent's real spawn path).
Key Decisions
| Decision | Recommendation |
|---|---|
| Telemetry source | Use .claude/logs/subagent-spawns.jsonl (fresh); ignore the dead agent-usage.jsonl |
| "never fired" caveat | Absence in the telemetry window is a strong signal, NOT proof of zero — state the window |
| Fix direction | Dormancy ⇒ wire a subagent_type= spawn from a high-traffic skill; do NOT rewrite descriptions (A/B Δ0) |
| Prune threshold | Only "dead" (no references in any skill/agent) is a prune candidate; niche stays |
Chain
After this audit, run the deeper experiment if you suspect descriptions: docs/feat--activation-audit/agent-routing-experiment.mjs (isolated A/B selection harness).
Related Skills
audit-skills— quality audit for skills (this skill's sibling, agent-side)telemetry-inspect— validates the telemetry data-plane this skill reads fromanalytics— raw usage queries across projectsdoctor— broader plugin health (manifests, hooks, memory budget)
Reviews
No reviews yet. Be the first.
Related
React Doctor
Your agent writes bad React. This catches it
orchestration-patterns
>
migration-patterns
>
mh install skills/audit-activation