AI Agent Analytics dashboard
Track the AI agent use and efficiency gain on your instance through the AI Agent Analytics dashboard. The dashboard can reveal trends in how AI agents are used to improve the time to resolution and the number of tasks closed.
Required ServiceNow AI Platform roles
To use the AI Agent Analytics dashboard, you must have either the sn_aia.viewer or the sn_aia.admin role.
If you want to change the dashboard, you must duplicate it and apply changes to the copy. You can redirect the Analytics page in AI Agent Studio to the new dashboard by replacing the dashboard sys_id in the system property sn_aia.analytics_dashboard_sysid.
Accessing the AI Agent Analytics dashboard
To open the dashboard, navigate to All > AI Agent Studio > Analytics.
You can also access the dashboard from the AI Agent Studio overview page. Go to the Recent agentic workflows and AI agents activity section and select the View analytics link.
Indicators
Most indicators are updated daily. The latency indicators are updated every 15 minutes.
Once you have installed Now Assist AI Agents, you can collect initial data by running the [Now Assist AI Agents] Historical Data Collection job. The other data collection jobs, [Now Assist AI Agents] Daily Data Collection and [Now Assist AI Agents] Periodic Data Collection, update the indicators.
| Name | Description |
|---|---|
| Agentic workflow latency | Time taken for an agentic workflow to complete. |
| AI agent execution plan P90 latency | Time taken for 90% of AI agent execution plans in a system to be processed. |
| AI agent execution plan P95 latency | Time taken for 95% of AI agent executions plans in a system to be processed. |
| AI agent execution plan P99 latency | Time taken for 99% of AI agent execution plans in a system to be processed. |
| AI agent latency | Time taken for an AI agent in a system to be processed. |
| All agentic workflows | Total number of agentic workflows created before today. |
| All AI agents | Total number of AI agents created before today. |
| All tools | Total number of tools created before today. |
| LLM P90 latency | Time taken for 90% of LLM requests in a system to be processed. |
| LLM P95 latency | Time taken for 95% of LLM requests in a system to be processed. |
| LLM P99 latency | Time taken for 99% of LLM requests in a system to be processed. |
| Number of closed tasks | Total number of tasks closed. |
| Number of closed tasks with AI agent assist | Total number of tasks closed today that were closed with agentic workflows or AI agents. |
| Summed duration of closed tasks | Total time taken to close tasks. |
| Summed duration of closed tasks with AI agent assist | Total time taken to close tasks today that were closed with agentic workflows or AI agents. |
| Tool execution P90 latency | Time taken for 90% of tool executions in a system to be processed. |
| Tool execution P95 latency | Time taken for 95% of tool executions in a system to be processed. |
| Tool execution P99 latency | Time taken for 99% of tool executions in a system to be processed. |
| Tool latency | Time taken for a tool in a system to be processed. |
| Total number of CS conversations | Total number of conversations in Now Assist panel or Virtual Agent. |
| Total number of execution plans | Total number of execution plans created by agentic workflows and AI agents today. |
| Total number of execution tasks | Total number of tasks executed by agentic workflows and AI agents today. |
| Total number of tool executions | Total number of tool executions in agentic workflows and AI agents today. |
| Use case execution plan P90 latency | Time taken for 90% of agentic workflows in a system to be processed. |
| Use case execution plan P95 latency | Time taken for 95% of agentic workflows in a system to be processed. |
| Use case execution plan P99 latency | Time taken for 99% of agentic workflows in a system to be processed. |
| Name | Description |
|---|---|
| % of conversations with an AI agent or agentic workflow defined | Number of conversations in Now Assist panel or Virtual Agent with a defined AI agentic or agentic workflow divided by the total number of conversations. |
| Average time to close a task | Average time taken for a task to go from creation to closure. |
| Average time to close a task with AI agent assist | Average time taken for a task to go from creation to closure with an agentic workflow or AI agent involved. |
| Efficiency gain | Percentage efficiency gain comparing average time taken to close a task with AI agent assist against a task closed without AI agent assist. |
| Percentage of tasks closed using AI agents | Number of tasks closed using AI agents and agentic workflows divided by the total number of tasks. |
Breakdowns
Different breakdowns enable you to divide the data differently to track specific aspects of the AI agent usage. Not all breakdowns are available for all indicators.
- Agentic workflow
- AI agent
- AI agent execution status
- Latency metric
- Tool
- Tool latency metric
Filters
You can filter data to review a subset of trends. The following table lists the available filters on the AI Agent Analytics dashboard.
| Page | Available filters |
|---|---|
| Status | Time created |
| Insights | - Time created - Agentic workflow - AI agent - AI Agent Type |
| Assist consumption | - Time created - Agentic workflow - AI agent - AI Agent Type |
| Performance | - Time created - Agentic workflow - AI agent - AI Agent Type - Tool |
| Troubleshooting | Time created |
Data visualizations
Visualizations are graphic elements that can be used to see data trends, percentages, and scores. The AI Agent Analytics dashboard includes the following visualizations.
| Title | Type | Description |
|---|---|---|
| Average inferred CSAT for AI agents in last 7 days | Score card | AI-generated score estimating what a customer would rate the AI response. 1 being the user is extremely dissatisfied and 5 being the user is extremely satisfied with the interaction. |
| Execution tasks in last 7 days | Pie chart | Execution tasks divided by status \(Success, Ongoing, Cancelled, Queued\). |
| AI Agent errors in last 90 days | Line graph | Number of errors seen by AI agents within the last 90 days broken down by week. |
| AI agent tool type | Pie chart | Tools divided by type. Example: Script, Flow action, Subflow, and so on. |
| AI agent tool execution mode | Pie chart | Tools divided by either autonomous or supervised execution mode. |
| Execution plans in last 7 days | Pie chart | Execution plans divided by status \(Completed, Terminated, Wrap Up, or In Progress\) |
| AI agent executions | List | List of AI agent executions. The list is broken down by AI agent and AI agent execution status. Columns span for multiple weeks, and there are change percentages and trend lines comparing the current against the previous week. |
| Number of agentic workflows | Trend line | Total number of agentic workflows. The agentic workflows created in the last day and a trend line are shown. |
| Number of AI agents | Trend line | Total number of AI agents. The AI agents created in the last day and a trend line are shown. |
| Number of tools | Trend line | Total number of tools. The tools created in the last day and a trend line are shown. |
| AI agent execution plans | List | List of AI agent execution plans. The list is broken down by agentic workflow. Columns span for multiple weeks, and there are change percentages and trend lines comparing the current against the previous week. |
| Title | Type | Description |
|---|---|---|
| AI agent execution plans | Score cards | Four score cards, one for each execution plan status \(Ready, In progress right now, Completed, and Terminated\). |
| AI agent executions | Score cards | Four score cards, one for each execution plan status \(In progress right now, Completed, Successful, and Errors/Cancelled\). |
| AI agent/tool executions | List | List of AI agent and tool executions. The list is broken down by AI agent and tool, and there are columns for each status \(Cancelled, Ongoing, Queued, and Success\). |
| Title | Type | Description |
|---|---|---|
| Tasks closed using AI agents | Trend line | Percentage of tasks closed using AI agents. Trend line is shown. |
| Task closure efficiency gain using AI agents | Trend line | Percentage efficiency gained with agentic workflows and AI agents. Trend line is shown. |
| Average close time assisted by AI agents | Trend line | Average time taken for a task to go from creation to closure with the involvement of an AI agent or agentic workflow. Trend line is shown. |
| Inferred CSAT of AI agents | Score card | Estimated CSAT score calculated using AI based on the subjective quality of the AI agents' responses. 1 being the user is extremely dissatisfied and 5 being the user is extremely satisfied with the interaction. |
| Inferred Session CSAT | Score card | Estimated CSAT score calculated using AI based on the overall session interactions of a user. 1 being the user is extremely dissatisfied and 5 being the user is extremely satisfied with the interaction. |
| Top user intents when using agentic solutions | Bar chart | Most common user intents as interpreted by the LLM. Intents are grouped by Group Action Framework. |
| Average AI agent CSAT by intent | Bar chart | Average CSAT score for each user intent interpreted by the LLM. Intents are grouped by Group Action Framework. |
| Transfers and escalations using AI agents | Bar chart | An estimated score using AI that indicates whether the user requested an escalation or there was a transfer to another AI agent or human. |
| User effort required with AI agents | Bar chart | Counts of AI-generated scores that measure the time and energy users had to put in during their interaction with an AI agent. Values are on a 3-point scale of low, medium, and high. |
| AI agent recommended next steps | Bar chart | Counts of AI-generated scores that indicate whether the AI agent recapped the outcome and provided clear instructions or guidance on what the customer should do next, if applicable. Values are on a 3-point scale of low, medium, and high. |
| AI agent resolved the users request | Score card | An AI-generated score that indicates whether the AI agent resolved the user's request. Values can be yes, no or unknown. |
| User frustration with AI agents | Bar chart | An AI-generated score that indicates whether the user expressed frustration at any point in the conversation. |
| AI agent empathy | Score card | An AI-generated score that indicates whether the AI agent was empathetic and friendly. Values are on a 3-point scale of low, medium, and high. |
| User or AI agent confusion | Bar chart | Counts of an AI-generated score that indicates whether the user or AI agent is confused at any point of the conversation. |
| Top 10 agentic workflows with highest session CSAT | Bar chart | Top 10 agentic workflows by highest CSAT score. |
| Bottom 10 agentic workflows with highest session CSAT | Bar chart | Bottom 10 agentic workflows by CSAT score. |
| Top 10 AI agents with highest session CSAT | Bar chart | Top 10 AI agents by highest CSAT score. |
| Bottom 10 AI agents with highest session CSAT | Bar chart | Bottom 10 AI agents by CSAT score. |
| AI agent conversations daily | Trend line | Total number of AI agent conversations in Now Assist panel or Virtual Agent. Trend line is shown. |
| % of conversations using AI agents | Trend line | Percentage of AI agent conversations in Now Assist panel or Virtual Agent that used agentic workflows or AI agents. Trend line is shown. |
| Efficiency gain supporting indicators | List | List of indicators. The list is broken down by indicator. Columns span across a time span. |
| Title | Type | Description |
|---|---|---|
| Sum of AI agent assists | Score card | Total number of assists used on the instance for both AI agents and agentic workflows. |
| Sum of AI agent assists per tier | Pie chart | Totals and percentages of assists divided by tier. An assist is marked as small until the number of tool executions exceeds 20. |
| Count of AI agent executions per assist tier. | Pie chart | Count of assist tiers for both AI agents and agentic workflows. An assist is marked as small until the number of tool executions exceeds 20. |
| Top 10 agentic workflows consuming assists | Pie chart | Percentages of assists consumed among the top 10 agentic workflows consuming assists. |
| Sum of AI agents assists | Bar chart | Total number of assists used by AI agents broken down by day. |
| Count of AI agent executions per assist tier | Bar chart | Count of assists used by AI agents broken down by assist tier and day. |
| Count of AI agent executions per agentic workflow | Bar chart | Count of assists used by AI agents broken down by agentic workflow. |
| Sum of AI agent assists per AI agent | Bar chart | Number of assists used by AI agents broken down by AI agent. |
| Average tool execution count per agentic workflow | List | Count of tools executed by an agentic workflow broken down by agentic workflow and day. |
| Average tool execution count per assist tier | List | Count of tools executed by an agentic workflow broken down by assist tier and day. |
| Title | Type | Description |
|---|---|---|
| P90 agentic workflow latency | Score card | Time in seconds of 90% of agentic workflows in a system to be processed. |
| P95 agentic workflow latency | Score card | Time in seconds of 95% of agentic workflows in a system to be processed. |
| P99 agentic workflow latency | Score card | Time in seconds of 99% of agentic workflows in a system to be processed. |
| Total agentic workflows executed per hour | Bar chart | Visualization of what time of day agentic workflows have been executed by hour. |
| Total agentic workflows errors per hour | Bar chart | Visualization of what time of day agentic workflows have been executed and experienced an error by hour. |
| Agentic workflow latency | Line chart | Agentic workflow latency over time. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| Agentic workflow latency \(weekly average\) | Line chart | Agentic workflow latency over time. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| Agentic workflow latency \(daily average\) | Table | Average agentic workflow latency today. Each row is an agentic workflow and each column is a different latency metric. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| P90 AI agent latency | Score card | Time in seconds of 90% of AI agent executions in a system to be processed. |
| P95 AI agent latency | Score card | Time in seconds of 95% of AI agent executions in a system to be processed. |
| P99 AI agent latency | Score card | Time in seconds of 99% of AI agent executions in a system to be processed. |
| Total AI agents executed per hour | Bar chart | Visualization of what time of day AI agent executions have been executed by hour. |
| Total AI agent errors per hour | Bar chart | Visualization of what time of day AI agent executions have been executed and experienced an error by hour. |
| AI agent latency | Line chart | AI agent latency within the time filter. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| AI agent latency \(weekly average\) | Line chart | AI agent latency over time by week. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| AI agent latency \(daily average\) | Table | Average AI agent latency today. Each row is an AI agent and each column is a different latency metric. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| P90 tool latency | Score card | Time in seconds of 90% of tool executions in a system to be processed. |
| P95 tool latency | Score card | Time in seconds of 95% of tool executions in a system to be processed. |
| P99 tool latency | Score card | Time in seconds of 99% of tool executions in a system to be processed. |
| Total tools executed per hour | Bar chart | Visualization of what time of day tool executions have been executed by hour. |
| Total tool errors per hour | Bar chart | Visualization of what time of day tool executions have been executed and experienced an error by hour. |
| Tool latency | Line chart | Tool latency within the time filter. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| Tool latency \(weekly average\) | Line chart | Tool latency over time by week. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| Tool latency \(daily average\) | Table | Average tool latency today. Each row is a tool and each column is a different latency metric. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| P90 LLM latency | Score card | Time in seconds of 90% of LLM requests in a system to be processed. |
| P95 LLM latency | Score card | Time in seconds of 95% of LLM requests in a system to be processed. |
| P99 LLM latency | Score card | Time in seconds of 99% of LLM requests in a system to be processed. |
| LLM latency | Line chart | Tool latency within the time filter. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| LLM latency \(weekly average\) | Line chart | Tool latency over time by week. Uses 3 different indicators: P90 tracks the fastest 90% of executions, P95 tracks the fastest 95% of executions, and P99 tracks the fastest 99% of executions. |
| LLM calls per hour | Bar chart | Visualization of what time of day LLM calls have been made by hour. |
| LLM errors per hour | Bar chart | Visualization of what time of day LLM calls have been made and experienced an error by hour. |
| Name | Type | Description |
|---|---|---|
| Agentic workflow errors | List | List of agentic workflow errors. |
For more information on security controls for agentic AI, see Security for AI agents.
| Name | Type | Description |
|---|---|---|
| Total blocked executions | Score card | Total number of agentic AI executions blocked because of security controls. |
| Total blocked executions | List | List of executions blocked. |
| Top agentic workflows with blocked executions | List | List of agentic workflows with the most blocked executions. |
| Top AI agents with blocked executions | List | List of AI agents with the most blocked executions. |
| Agentic Workflows w/o ACL defined | Score card | Total number of agentic workflows missing an ACL for user access. |
| Agentic Workflows ACLs | Pie chart | Pie chart displaying the percentage of agentic workflows with and without ACLs for user access. |
| Agentic Workflows w/o ACL defined | List | List of agentic workflows missing an ACL for user access. |
| AI agents with no ACLs defined | Score card | Total number of AI agents missing an ACL for user access. |
| AI agent ACLs | Pie chart | Pie chart displaying the percentage of AI agents with and without ACLs for user access. |
| AI agents w/o ACL defined | List | List of AI agents missing an ACL for user access. |
| Agentic workflows without role masking | Score card | Total number of agentic workflows without roles identified for role masking and data access. |
| Agentic workflows without role masking | List | List of agentic workflows without roles identified for role masking and data access. |
| AI agents without role masking | Score card | Total number of AI agents without roles identified for role masking and data access. |
| AI agents without role masking | List | List of AI agents without roles identified for role masking and data access. |
| Agentic workflows running as dynamic user | Score card | Total number of agentic workflows running as a dynamic user. |
| Agentic workflows running as dynamic user | List | List of agentic workflows running as a dynamic user. |
| AI agents running as dynamic user | Score card | Total number of AI agents running as a dynamic user. |
| AI agents running as dynamic user | List | List of AI agents running as a dynamic user. |
| Agentic workflows running as AI user | Score card | Total number of agentic workflows running as an AI user. |
| Agentic workflows running as AI user | List | List of agentic workflows running as an AI user. |
| AI agents running as AI user | Score card | Total number of AI agents running as an AI user. |
| AI agents running as AI user | List | List of AI agents running as an AI user. |