What it is
The AI App Usage Risk panel in the Insights page shows which users in a tenant have been active on AI-connected applications, alongside signals about the security posture of those sessions.
It is designed to surface where AI app access may warrant review β not to provide a definitive audit of every action an AI tool performed.
What the columns mean
User β the identity observed accessing one or more AI-connected applications.
AI categories β the type of AI tool accessed, for example General AI assistant, Code assistant, or Microsoft Copilot.
AI apps β the specific applications observed, such as ChatGPT, Claude, or Microsoft 365 Copilot.
Access telemetry β the total number of access events recorded for this user across the observed AI applications in the selected time window.
User/app/device days β the number of distinct days on which this user had observed activity. Useful for understanding whether this is a one-off or an ongoing pattern.
Single-factor events β sign-in events associated with this user where only a single authentication factor was used. AI app access over single-factor sessions carries higher risk.
Unmanaged device events β sign-in events where the device was not registered or managed by your organisation. Data accessed via unmanaged devices is outside your endpoint controls.
Non-compliant device events β sign-in events where the device was managed but did not meet your compliance policies at the time of access.
Last seen β the most recent timestamp of observed AI app activity for this user.
Review signals β a summary of the risk flags present for this user, combining single-factor sessions, unmanaged devices, and non-compliant devices into a single readable label.
Important caveats
Before acting on this data, keep the following in mind:
This view joins AI app activity to sign-in posture signals, not to individual actions within those apps. A high access count does not tell you what data was shared β only that the user was active in an AI application from a session with certain characteristics.
Unmanaged and non-compliant device signals come from sign-in telemetry, not from the AI app session itself. The posture signals reflect the state of the device at the time of sign-in, which may or may not be the same session as the AI app access.
AI app detection is based on known application names. Apps that do not match the known list will not appear here, even if they use AI features. The panel covers the most common AI tools but is not exhaustive.
Review signals indicate conditions that may require attention, not confirmed policy violations. Some users may have legitimate reasons for accessing AI tools from unmanaged devices, for example contractors or approved BYOD scenarios.
The time window defaults to the last 7 days. Activity outside this window is not reflected in the current view.
Use this panel as a starting point for conversations with your customers about AI tool governance and access policy β not as a standalone compliance report.
Turning this data into a customer conversation
The AI App Usage Risk panel is one of the most effective ways to open an AI readiness conversation with a customer. When a customer can see their own staff actively using ChatGPT, Claude, or Copilot β sometimes from unmanaged devices β it makes the question of AI governance real and immediate rather than theoretical.
The right question to ask is: "Where could AI help the business first, without putting sensitive client or company data at risk?"
This reframes the conversation away from "should we allow AI?" toward "how do we use it safely?" β which is a much more productive starting point for most customers.
From there, a natural conversation covers:
Which AI tools staff are already using, and whether those apps have access to business data
Whether the identity and device controls already in place are strong enough to support AI adoption safely
Which use cases make sense to start with β low-risk, high-value tasks like drafting, summarising, and routine administration
What governance steps are needed before broader rollout, such as acceptable use policies and Conditional Access rules scoped to approved AI apps
The good news for most customers is that the foundation is already in place. The identity controls, permissions structure, and data governance in Microsoft 365 that your security work has established give them a strong starting point for AI adoption.
For a ready-made conversation guide, email template, and AI readiness framework to use with customers, see the Overe MSP Growth Guide β the AI Readiness and Use Case Planning section walks through this conversation step by step.