Terminal
Terminal/Skill
Digs into specific Amplitude Agent Analytics sessions or failure patterns to find the root cause of AI agent failures and low quality.
By Amplitude
Works with
npx skills add amplitude/mcp-marketplace --skill investigate-ai-sessionYou investigate specific AI agent sessions or failure patterns to determine root causes. You operate at the session and span level — reading conversations, tracing execution, and connecting failures to their origins. This is the "why" skill that follows the "what" from /monitor-ai-quality.
The user will provide one of:
/monitor-ai-quality first, then come back with specific findingsCall Amplitude:get_amplitude_agent_analytics_info with view: "schema" to discover valid agent names, tool names, and evaluator fields. Then call it with view: "sessions" and supported filters:
agentNames: ["<agent>"], hasTaskFailure: truetoolNames: ["<tool>"], hasTaskFailure: truehasTechnicalFailure: truehasNegativeFeedback: true, or select evaluator sentiment scores at or below 0.4minCostUsd: <threshold>minDurationMs: <threshold>Use responseFormat: "concise", limit: 20, and sort by "-session_start" to get recent examples. Select the 3-5 most representative sessions for deep investigation.
Call Amplitude:get_amplitude_agent_analytics_info with view: "sessions" and the supported user identifier filter to find their sessions. If they reported a specific timeframe, add the date filters. Pick the session(s) that match the complaint.
For each session being investigated (max 3-5 sessions), run these in parallel per session:
Full session detail. Call Amplitude:get_amplitude_agent_analytics_info with view: "sessions", sessionIds: ["<id>"], and responseFormat: "detailed". This returns enrichment data: rubric scores, failure reasons, topic classifications, overall outcome, and quality flags.
Conversation transcript. Call Amplitude:get_amplitude_agent_analytics_info with view: "conversation" and the session ID. Read the full user-agent exchange to understand what was asked, how the agent responded, and where things broke down.
Execution trace. Call Amplitude:get_amplitude_agent_analytics_info with view: "spans" and the session ID. This shows every LLM call, tool call, and embedding operation — their latency, status, cost, and ordering. Look for:
status: "ERROR" — direct failuresWith conversation + trace + enrichment data, build the diagnosis:
Classify the failure type:
Determine scope: Is this a one-off or systemic?
Amplitude:get_amplitude_agent_analytics_info with view: "sessions" and group by a supported session-level field; aggregate topics locally from evaluator results.Amplitude:get_amplitude_agent_analytics_info with view: "sessions" and the same agent and time window to check if similar failures exist.Find the trigger: What changed?
Amplitude:get_amplitude_agent_analytics_info with view: "tool_reliability"If the root cause isn't clear from the session data alone:
Find related sessions. Call Amplitude:get_amplitude_agent_analytics_info with view: "sessions" and supported error filters, then compare evaluator topics locally to find other sessions with the same issue.
Check tool/model health. Call Amplitude:get_amplitude_agent_analytics_info with view: "tool_reliability" for tool health or view: "spans" grouped by model over the relevant time window. Look for tools with elevated error rates or latency that correlate with the failing sessions.
Structure the output as a root cause analysis.
Required sections:
Investigation summary (2-3 sentences): What was investigated, what was found, and the severity. Written as a headline for the team.
Sessions examined: A compact table of the sessions investigated:
| Session ID | Agent | Outcome | Quality | Sentiment | Failure Type |
|------------|-------|---------|---------|-----------|--------------|
| [id] | [name] | [outcome] | [score] | [score] | [type or —] |
Root cause (1 paragraph): The primary explanation for what went wrong. Be specific — name the tool, the error, the model behavior, or the orchestration issue. Include evidence from the conversation and trace.
Execution trace highlights (for the most illustrative session): Walk through the key spans showing the failure path:
Conversation excerpt (if revealing): Quote the 2-3 most relevant turns showing where the agent failed the user. Keep it brief.
Scope assessment: One-off vs. systemic. How many sessions are affected? Is it getting worse?
Recommended fixes (2-4 numbered items): Concrete actions. Examples:
Follow-on prompt: Offer next steps — "Want me to check if this tool timeout affects other agents, search for similar user complaints, or monitor this pattern over the next few days?"
User says: "What happened in session abc-123?"
Actions:
User says: "Why are Chart Agent sessions failing?"
Actions:
User says: "A customer said our AI gave them wrong data yesterday"
Actions:
The session may be from a different project, or outside the data retention window. Ask the user to confirm the project and check if the session ID is correct.
Span-level data requires OpenTelemetry-compatible tracing in the AI agent. Report what's available from the session and conversation level and note that span data would help narrow the root cause.
Don't try to investigate more than 5 sessions in detail. Instead, use get_amplitude_agent_analytics_info with view: "sessions" and group by the relevant dimension to find the common pattern, then deep-dive into 2-3 representative examples.