The most useful KQL queries for troubleshooting Copilot Studio agents
Twenty-four Kusto queries for Copilot Studio telemetry, split across the two scopes that do not share a schema, with the documented limitations that make some of the obvious ones quietly lie to you.
Someone reports that the agent gave a wrong answer yesterday afternoon, and the Copilot Studio analytics page shows a healthy green line. The instinct is to open Application Insights, filter requests for failures and check exceptions, which is how every other Azure workload gets debugged. Both tables come back empty, because Copilot Studio writes to neither.
That is the gap this is meant to close. It is a reference rather than a tutorial: each query says when to reach for it, what it returns, and the specific way it will mislead you. It assumes you already know Application Insights and KQL, and that you have an agent misbehaving somewhere you cannot attach a debugger.
Confirm which telemetry scope you are on
Everything else depends on this. Copilot Studio emits telemetry at two scopes with two different schemas, and running the right query against the wrong table returns nothing at all, which reads exactly like a broken pipeline.
| Agent-level | Environment-level (preview) | |
|---|---|---|
| Configured in | Agent Settings > Advanced > Application Insights | Power Platform admin center export package |
| Owner | Maker or dev team, per agent | Admin, platform team or CoE, once per environment |
| Telemetry model | Event-based | Trace and span based, OpenTelemetry GenAI conventions |
| Primary table | customEvents | dependencies |
| Topic events | Yes | No |
| Tool and sub-agent spans | No | Yes |
The practical consequence is that neither scope is a superset of the other. Topic funnels are only possible at agent level; tool and sub-agent spans only exist at environment level. On anything non-trivial you end up wanting both, in two resources.1
The first thirteen queries below target customEvents. The rest target dependencies.
Discover which events your agent actually emits
Run this first, always. The set of event names depends on which logging toggles are enabled (Enable logging, Log conversation details, Log sensitive Activity properties, Node execution events) and on whether the agent is classic or built in the new agent experience. Documented names include TopicStart, TopicEnd, BotMessageSend, BotMessageReceived, node events and GenerativeAnswers, but treat this query rather than the documentation as the source of truth for your instance.
customEvents
| where timestamp > ago(7d)
| summarize Events = count(),
Sessions = dcount(session_Id),
FirstSeen = min(timestamp),
LastSeen = max(timestamp)
by name
| order by Events descAn event name you expected and cannot see is nearly always a logging toggle that was never switched on, not a bug in the agent.
Discover every available customDimensions key
Almost all the useful Copilot Studio data lives inside the customDimensions bag, and the key set is dynamic: it changes as the product evolves and as logging settings change. This flattens every key, shows which events carry it, and gives a sample value, so you stop guessing at property names.
customEvents
| where timestamp > ago(7d)
| mv-expand Key = bag_keys(customDimensions) to typeof(string)
| extend Value = tostring(customDimensions[Key])
| summarize AppearsOnEvents = make_set(name, 20),
SampleValue = take_any(Value),
Occurrences = count()
by Key
| order by Key ascThis query and its environment-level twin are the two worth pinning. When a saved query goes quiet, run one of these before you start debugging the query itself.
Exclude test-canvas traffic from every other query
Every conversation in the Copilot Studio test pane is logged alongside real user traffic, which quietly poisons error rates, latency percentiles and user counts. Filter on the designMode dimension, and fold this into anything you are going to quote at somebody.
customEvents
| where timestamp > ago(7d)
| extend DesignMode = tostring(customDimensions['designMode'])
| extend DesignMode = iff(isempty(DesignMode),
tostring(customDimensions['DesignMode']),
DesignMode)
| where DesignMode !~ "True" // real traffic onlyReplay a single conversation end to end
The core triage query. When a user reports that the agent gave a weird answer, you need the exact ordered sequence of topics, nodes and messages for that one session.
let TargetSession = "<session_Id>";
customEvents
| where session_Id == TargetSession
| extend Topic = tostring(customDimensions['TopicName']),
Kind = tostring(customDimensions['Kind']),
Text = tostring(customDimensions['text']),
ActivityType = tostring(customDimensions['type']),
Channel = tostring(customDimensions['channelId']),
FromName = tostring(customDimensions['fromName'])
| project timestamp, name, Topic, Kind, ActivityType, Channel, FromName, Text
| order by timestamp ascRead it as a transcript with the machinery left in. The topic changes tell you what the agent thought was being asked, which is usually where the divergence starts.
Find the session from a user-reported conversation ID
Users and support tickets almost always carry a conversation ID, taken from an error message shown in chat or from typing /debug conversationid in the test pane, rather than a session_Id. This bridges the two so you can then replay the session.
let TargetConversation = "<conversation id>";
customEvents
| where timestamp > ago(30d)
| extend ConversationId = tostring(customDimensions['conversationId'])
| where ConversationId == TargetConversation
or tostring(customDimensions['text']) contains TargetConversation
| summarize Events = count(),
Start = min(timestamp),
End = max(timestamp)
by session_Id, ConversationIdThe contains arm is there because the ID is sometimes only present inside the error text shown to the user, rather than as its own dimension.
Rank the top error codes the agent is returning
Copilot Studio surfaces runtime failures as a message to the user containing Error code: and Conversation ID:. Aggregating those gives you a ranked list, so you attack the highest-volume failure first rather than the most recently reported one.
customEvents
| where timestamp > ago(7d)
| where name == "BotMessageSend"
| extend Text = tostring(customDimensions['text'])
| where Text contains "Error code:"
| extend ErrorCode = extract(@"Error code:\s*([^\s\.,;]+)", 1, Text),
ConversationId = extract(@"Conversation ID:\s*([^\s\.,;]+)", 1, Text)
| summarize Occurrences = count(),
Sessions = dcount(session_Id),
LastSeen = max(timestamp)
by ErrorCode
| order by Occurrences descDrill into individual error occurrences with topic context
Counting error codes tells you what is failing. This tells you where. Pairing each error with the topic that was executing is usually enough to identify the offending node, whether that is a connector call, a flow, an HTTP request or an authentication step.
customEvents
| where timestamp > ago(7d)
| where name == "BotMessageSend"
| extend Text = tostring(customDimensions['text']),
Topic = tostring(customDimensions['TopicName']),
Kind = tostring(customDimensions['Kind'])
| where Text contains "Error code:"
| extend ErrorCode = extract(@"Error code:\s*([^\s\.,;]+)", 1, Text)
| project timestamp, ErrorCode, Topic, Kind, session_Id, Text
| order by timestamp desc
| take 100Find topics that start but never finish
A topic logging many TopicStart events and far fewer TopicEnd events is one where users abandon, get redirected, or hit an unhandled error mid-flow. This is the fastest way to find broken conversation design without reading a single transcript.
customEvents
| where timestamp > ago(7d)
| where name in ("TopicStart", "TopicEnd")
| extend Topic = tostring(customDimensions['TopicName'])
| summarize Starts = countif(name == "TopicStart"),
Ends = countif(name == "TopicEnd"),
Sessions = dcount(session_Id)
by Topic
| extend CompletionRatePct = iff(Starts == 0, real(null),
round(100.0 * Ends / Starts, 1))
| order by Starts descA completion rate well under 100% is not automatically a fault. Topics that hand off to another topic by design will never log their own end. Compare a topic against its own history rather than against its neighbours.
Track fallback, escalation and sign-in topics
Spikes in these system topics are the clearest quantitative signal of agent quality problems. Fallback means the agent could not match intent, Escalate means it gave up to a human, and sign-in churn means an authentication misconfiguration. Trending them daily shows whether a publish made things better or worse.
customEvents
| where timestamp > ago(30d)
| where name == "TopicStart"
| extend Topic = tostring(customDimensions['TopicName'])
| where Topic has_any ("Fallback", "Escalate", "SignIn",
"ResetConversation", "ConversationalBoosting")
| summarize Triggers = count(), Sessions = dcount(session_Id)
by bin(timestamp, 1d), Topic
| render timechartFind the slowest steps in your topics
This measures elapsed time between consecutive events in the same session, which surfaces the nodes that make an agent feel slow: Power Automate flow calls, connector actions and generative answers.
customEvents
| where timestamp > ago(1d)
| order by session_Id asc, timestamp asc
| extend PrevTimestamp = iff(session_Id == prev(session_Id),
prev(timestamp), datetime(null))
| extend StepMs = iff(isnotnull(PrevTimestamp),
(timestamp - PrevTimestamp) / 1ms, real(null))
| where isnotnull(StepMs)
| extend Topic = tostring(customDimensions['TopicName']),
Kind = tostring(customDimensions['Kind'])
| where Kind !has "Question" // exclude nodes that wait on the user
| summarize Executions = count(),
P50Ms = percentile(StepMs, 50),
P95Ms = percentile(StepMs, 95),
MaxMs = max(StepMs)
by name, Kind, Topic
| order by P95Ms descAggregate generative answers outcomes
For agents with knowledge sources, the GenerativeAnswers event records whether the model actually produced a grounded answer. Aggregating Result exposes how often generative answers silently fail to find content, which is the single most common cause of the complaint that the agent knows nothing about your documents.
customEvents
| where timestamp > ago(7d)
| where name == "GenerativeAnswers"
| extend Result = tostring(customDimensions['Result']),
Topic = tostring(customDimensions['TopicName'])
| summarize Occurrences = count(), Sessions = dcount(session_Id)
by Result, Topic
| order by Occurrences descThe distinction worth holding on to is between finding nothing and finding something wrong. Only the first shows up here.
Inspect individual generative answer payloads
When the aggregate shows failures, this exposes the actual question, the summary returned and the serialised grounding data for individual calls. That is usually enough to separate retrieval failure, where no sources matched, from a permissions problem, where the source was inaccessible to that user.
customEvents
| where timestamp > ago(7d)
| where name == "GenerativeAnswers"
| extend cd = todynamic(customDimensions)
| extend ConversationId = tostring(cd.conversationId),
Topic = tostring(cd.TopicName),
Message = tostring(cd.Message),
Result = tostring(cd.Result),
Summary = tostring(cd.Summary),
SerializedData = tostring(cd.SerializedData)
| project timestamp, cloud_RoleInstance, ConversationId, Topic,
Message, Result, Summary, SerializedData
| order by timestamp desc
| take 50Compare behaviour across channels
That it works in the test canvas but breaks in Teams is one of the most common Copilot Studio support reports. Splitting volume and errors by channelId confirms or disproves it immediately, and often points straight at a channel-specific problem such as Adaptive Card rendering or authentication.
customEvents
| where timestamp > ago(7d)
| extend Channel = tostring(customDimensions['channelId']),
Locale = tostring(customDimensions['locale']),
Text = tostring(customDimensions['text']),
DesignMode = tostring(customDimensions['designMode'])
| summarize Events = count(),
Sessions = dcount(session_Id),
Errors = countif(Text contains "Error code:")
by Channel, Locale
| extend ErrorRatePct = round(100.0 * Errors / Events, 2)
| order by Events descVerify environment-level export is working
Run this immediately after enabling export, and again whenever a dashboard goes quiet. getschema confirms which native columns are available before you build anything on them.
dependencies
| getschema
| project ColumnName, ColumnType
| order by ColumnName ascDiscover the live gen_ai attribute set
The span attribute schema is in preview and still moving, so hard-coding key names out of documentation is fragile. This enumerates every key currently present, which events carry it, and a sample value.
dependencies
| where timestamp > ago(7d)
| mv-expand Key = bag_keys(customDimensions) to typeof(string)
| extend Value = tostring(customDimensions[Key])
| summarize Events = make_set(name), SampleValue = take_any(Value) by Key
| order by Key ascReturn the full trace for a known conversation ID
The environment-level equivalent of replaying a session, and the query you will reach for most. It returns every span for one conversation, ordered root-span-first within each turn so the execution tree reads top down.
let LatestConvo = "<Conversation ID>";
dependencies
| where tostring(customDimensions["gen_ai.conversation.id"]) == LatestConvo
| order by operation_Id asc, iff(name == "InvokeAgent", 0, 1) asc, timestamp asc
| project timestamp, name, id, operation_Id, operation_ParentId,
duration, target, type, cloud_RoleName, resultCode, customDimensionsEach operation_Id block is one turn. The InvokeAgent row opens it, each ExecuteTool row is a tool the agent chose, and OutputMessages is what the user saw.
Pull the most recent conversation for a named agent
When you have no conversation ID, because you just ran a test and want to see what happened, this finds the latest conversation for a given agent and returns its full trace in the same root-first order.
let Window = 7d;
let AgentName = "<Agent name>";
let LatestConvo = toscalar(
dependencies
| where timestamp > ago(Window)
| where tostring(customDimensions["gen_ai.agent.name"]) == AgentName
| where isnotempty(tostring(customDimensions["gen_ai.conversation.id"]))
| top 1 by timestamp desc
| project tostring(customDimensions["gen_ai.conversation.id"])
);
dependencies
| where timestamp > ago(Window)
| where tostring(customDimensions["gen_ai.conversation.id"]) == LatestConvo
| order by operation_Id asc, iff(name == "InvokeAgent", 0, 1) asc, timestamp asc
| project timestamp, name, id, operation_Id, operation_ParentId,
duration, target, type, cloud_RoleName, resultCode, customDimensionsFlatten a conversation into readable columns
Raw customDimensions blobs become unreadable after about three rows. This projects the GenAI attributes into named columns, giving a scannable turn-by-turn table of user input, agent output, tool name, arguments, result and model.
let Window = 7d;
let ConversationId = "<Conversation ID>";
dependencies
| where timestamp > ago(Window)
| where tostring(customDimensions["gen_ai.conversation.id"]) == ConversationId
| extend OperationName = tostring(customDimensions["gen_ai.operation.name"]),
AgentName = tostring(customDimensions["gen_ai.agent.name"]),
Model = tostring(customDimensions["gen_ai.request.model"]),
ToolName = tostring(customDimensions["gen_ai.tool.name"]),
ToolType = tostring(customDimensions["gen_ai.tool.type"]),
ToolCallId = tostring(customDimensions["gen_ai.tool.call.id"]),
ToolArguments = tostring(customDimensions["gen_ai.tool.call.arguments"]),
ToolResult = tostring(customDimensions["gen_ai.tool.call.result"]),
Channel = tostring(customDimensions["microsoft.channel.name"]),
UserEmail = tostring(customDimensions["user.email"])
| extend InputMessages = parse_json(tostring(customDimensions["gen_ai.input.messages"])),
OutputMessages = parse_json(tostring(customDimensions["gen_ai.output.messages"]))
| extend UserInput = tostring(InputMessages[0].parts[0].content),
AgentOutput = tostring(OutputMessages[0].parts[0].content)
| order by operation_Id asc, iff(name == "InvokeAgent", 0, 1) asc, timestamp asc
| project timestamp, name, OperationName, AgentName, Model, UserInput, AgentOutput,
ToolName, ToolType, ToolCallId, ToolArguments, ToolResult,
Channel, UserEmail, duration, resultCode, operation_Id, operation_ParentIdSurface every failing span in the last 24 hours
Your primary error sweep, and the one query here that has to be more complicated than it looks. It checks four failure signals at once because the obvious ones are not reliable on their own.
dependencies
| where timestamp > ago(24h)
| extend ErrorType = tostring(customDimensions["error.type"]),
StatusCode = tostring(customDimensions["Status.code"]),
StatusMessage = tostring(customDimensions["Status.message"])
| where resultCode =~ "ERROR" or success == false
or isnotempty(ErrorType) or StatusCode == "2" // 2 = ERROR per OTel
| extend AgentName = tostring(customDimensions["gen_ai.agent.name"]),
ToolName = tostring(customDimensions["gen_ai.tool.name"]),
ConversationId = tostring(customDimensions["gen_ai.conversation.id"]),
Channel = tostring(customDimensions["microsoft.channel.name"])
| project timestamp, name, AgentName, ToolName, ErrorType, StatusCode, StatusMessage,
Channel, ConversationId, operation_Id, id, operation_ParentId
| order by timestamp descRank tools by reliability
Tools are where most agent failures actually originate: connectors, MCP servers, flows and prompts. This ranks every one by volume, failures, failure rate and latency, which is what separates a flaky connector from a slow but working one.
dependencies
| where timestamp > ago(7d)
| where name == "ExecuteTool"
| extend ToolName = tostring(customDimensions["gen_ai.tool.name"]),
ToolType = tostring(customDimensions["gen_ai.tool.type"]),
ErrorType = tostring(customDimensions["error.type"]),
AgentName = tostring(customDimensions["gen_ai.agent.name"])
| summarize Calls = count(),
Failures = countif(resultCode =~ "ERROR" or success == false
or isnotempty(ErrorType)),
P50Ms = percentile(duration, 50),
P95Ms = percentile(duration, 95),
LastSeen = max(timestamp)
by AgentName, ToolName, ToolType
| extend FailureRatePct = round(100.0 * Failures / Calls, 2)
| order by Failures desc, Calls descSort by failure count rather than failure rate to start with. A tool with a 100% failure rate over three calls matters less than one failing a fifth of several thousand.
Inspect the arguments and results of a failing tool
Once the league table names the culprit, this shows the exact payload sent to the tool and returned from it. It is usually where you find a malformed argument, a null the agent invented, or an upstream error message the agent swallowed and paraphrased.
let ToolNameFilter = "<gen_ai.tool.name value>";
dependencies
| where timestamp > ago(24h)
| where name == "ExecuteTool"
| extend ToolName = tostring(customDimensions["gen_ai.tool.name"]),
ToolArguments = tostring(customDimensions["gen_ai.tool.call.arguments"]),
ToolResult = tostring(customDimensions["gen_ai.tool.call.result"]),
ErrorType = tostring(customDimensions["error.type"]),
StatusMessage = tostring(customDimensions["Status.message"]),
ConversationId = tostring(customDimensions["gen_ai.conversation.id"])
| where ToolName has ToolNameFilter
| project timestamp, ToolName, ErrorType, StatusMessage, ToolArguments, ToolResult,
ConversationId, operation_Id, duration, resultCode
| order by timestamp desc
| take 50Track turn latency percentiles per agent
InvokeAgent spans represent one complete turn, so their duration is the closest proxy you have for perceived responsiveness. Percentiles catch regressions after a publish and identify which agent is dragging a shared environment down.
dependencies
| where timestamp > ago(7d)
| where name == "InvokeAgent"
| extend AgentName = tostring(customDimensions["gen_ai.agent.name"]),
Channel = tostring(customDimensions["microsoft.channel.name"])
| summarize Turns = count(),
P50Ms = percentile(duration, 50),
P95Ms = percentile(duration, 95),
P99Ms = percentile(duration, 99)
by AgentName, Channel
| order by P95Ms descMap multi-agent and sub-agent invocations
When an agent calls another agent as a tool, the sub-agent inherits the parent’s conversation ID with a _<subConversationId> suffix. Splitting the conversation ID on _ reconstructs the orchestration tree, which is the only way to see which child agents ran, how deep the chain went, and where it stopped.
dependencies
| where timestamp > ago(7d)
| extend ConversationId = tostring(customDimensions["gen_ai.conversation.id"]),
AgentName = tostring(customDimensions["gen_ai.agent.name"]),
ToolName = tostring(customDimensions["gen_ai.tool.name"])
| where isnotempty(ConversationId)
| extend RootConversationId = tostring(split(ConversationId, "_")[0]),
Depth = countof(ConversationId, "_")
| summarize Spans = count(),
AgentsInvolved = make_set(AgentName, 20),
ToolsUsed = make_set(ToolName, 20),
MaxDepth = max(Depth),
Start = min(timestamp),
End = max(timestamp)
by RootConversationId
| where MaxDepth > 0 // only conversations with sub-agents
| order by Start descFind orphaned OutputMessages spans
OutputMessages spans can legitimately arrive with no parent InvokeAgent root, appearing as standalone single-node traces. Knowing which ones are expected stops you investigating a phantom broken trace, and an unusual volume of them is itself worth a look.
let Window = 24h;
let Roots = dependencies
| where timestamp > ago(Window)
| where name == "InvokeAgent"
| distinct operation_Id;
dependencies
| where timestamp > ago(Window)
| where name == "OutputMessages"
| where operation_Id !in (Roots)
| extend ConversationId = tostring(customDimensions["gen_ai.conversation.id"]),
AgentName = tostring(customDimensions["gen_ai.agent.name"])
| project timestamp, AgentName, ConversationId, operation_Id, id,
operation_ParentId, resultCode
| order by timestamp descWhat else will bite you
Four limitations do not fit the pattern above, because there is no query that works around them.
Environment-level export is not transactional, and small amounts of data loss can occur during transient service events. Counts from it are diagnostic, not billing-grade, and should never be the basis of a chargeback. The scope is also available for managed environments only and excludes declarative agents, which is worth confirming before you plan a rollout around it.
Trace and span IDs are emitted as GUIDs rather than the standard 32 and 16 character hex OpenTelemetry identifiers, so correlation with a third-party OTel backend needs a translation step you have to write yourself.
Finally, telemetry from agents authored in the new agent experience may differ from classic agents. After any migration, re-run the three discovery queries rather than assuming your saved queries survived.1
What I keep pinned
Four of these, and the rest get written when they are needed and thrown away: the two discovery queries, the conversation trace, and the tool reliability table. That set answers most of what anyone actually asks.
The habits that matter are duller than the queries. Start broad and filter down, because a query that returns nothing tells you almost nothing about why. Correlate on operation_Id within a turn and gen_ai.conversation.id across a conversation, and resist the urge to build a dashboard on native columns before you have checked they mean what they look like they mean. Promote a query to a workbook only once you have run it by hand enough times to trust it, and put an alert on it only once you know its false-positive rate.
The compromise worth naming is that saved queries and functions are a fork of a schema still in preview. One day they will keep returning the old shape without telling you. Keep them anyway, and when something looks wrong, re-run the discovery queries before you debug anything else.