The Overe Scanner is a powerful tool designed to provide MSPs and businesses with a comprehensive overview of potential security risks within their SaaS environments. This article will guide you through the features and information provided by the Overe Assess Scanner, helping you make the most of its capabilities.
1. Quick Scan Overview
The Security Controls Assessment (Quick Scan) feature allows you to rapidly assess the security posture of your SaaS environment. This scan identifies key security controls that your organization currently cannot fulfil with your existing Microsoft license and itemises a comprehensive list of all these controls, as well as details on how to enable these.
You can learn more about the Quick Scan here: https://intercom.help/overe/en/articles/9760900-what-is-the-purpose-of-the-free-overe-assess-service
2. Deep Scan (Beta)
The Deep Scan feature is currently in beta but offers a more detailed examination of your SaaS environment. This scan goes beyond the surface to identify risks associated with various aspects of your IT environment.
Key Risk Identification: The Deep Scan checks for vulnerabilities in:
Identities: Ensuring that all user accounts are secure, with no weak points such as inactive accounts or those without Multi-Factor Authentication (MFA).
Mailboxes: Identifying potential issues like external forwarding rules that could expose your organization to phishing or data loss risks.
Connected Apps: Assessing the security of third-party apps connected to your environment, looking for known vulnerabilities or apps that might pose a risk.
Devices: Evaluating devices connected to your network for potential threats or security vulnerabilities. (coming soon)
Scan Status and Results: After running a Deep Scan, the status of your request is updated in real-time.
Actionable Insights: The Deep Scan provides a quick overview of potential vulnerabilities, helping you take prompt action to enhance your security posture.
Note. For all sections of the scan to work, especially around User risks, "AAD Premium" is required on the MSFT SKU. Overe recommends, but does not require, Azure Active Directory Premium P1 licensing to access this data, which can be added as a standalone add-on or included in subscriptions like Business Premium.
3. Running a New Scan
If you need to run a new scan, simply click the "Run a new Scan" button. This will initiate a new assessment, providing you with up-to-date information about your SaaS environment's security status.
Note: Currently, scans are performed on-demand, and results are not yet automatically saved or sent. However, features for automatic saving and notifications are in development and will be available soon.
4. Risky MFA Settings
The Risky MFA Settings Scan Report identifies user accounts with MFA settings that pose an elevated security risk. This analysis focuses on active (not disabled) Member and Guest Admin accounts*.
We evaluate both MFA enrollment and enforcement settings. For MFA enforcement, we assess the following:
Security Defaults Status: Whether security defaults are enabled for the tenant.
Per-User MFA Enforcement: If MFA is enforced on a per-user basis.
Conditional Access Policies: The presence and configuration of conditional access policies. Only unconditional MFA access policies are considered sufficient for tagging accounts as "MFA enforced" (unless explicitly excluded by the policy).
Following a scan, you will receive a list of accounts with potential MFA risks, categorized as follows:
Privileged Admin (Priv. Admin):
A privileged administrator who can make critical changes to your tenant. We differentiate between admin types to provide better context on the level of risk associated with the account.MFA Enrolled:
Indicates whether Multi-Factor Authentication (MFA) is fully enabled and active for this account. This confirms whether the user has successfully completed MFA enrollment.MFA Enforced:
Shows if the account has been assigned an MFA policy. Note, this does not confirm whether the user has enrolled in MFA — refer to the "MFA Enrolled" status for that information.Risk Level:
The overall risk associated with the account is determined based on the following logic:
MFA Enforcement | MFA Enrollment Has the user fully enrolled into MFa and is it active/valid? | Risk |
Enforced | Enrolled | Low (this will not be visible in the report) |
Enforced | Not Enrolled | Medium |
Enforced | Unknown | Medium |
Not Enforced | Enrolled | High (if admin), Medium (if not admin) |
Not Enforced | Not Enrolled | High |
Not Enforced | Unknown | High |
(*) while in Beta, Scanner will evaluate MFA settings for ALL admin accounts, but only verify the first 100 non-admin accounts.
5. Understanding App risk
This section provides a detailed overview of the permissions granted to your connected applications. We categorize each permission according to its risk level—ranging from low to high—and provide the reasoning behind this categorization. It’s crucial to understand that these are risk categories, not indicators of specific malicious activity by the app. However, you should carefully review each app's permissions to ensure they align with the intended use and that the app isn't requesting more permissions than necessary.
This analysis covers both delegated permissions (OAuth2PermissionGrants) and application permissions (AppRoleAssignments), helping you identify potential risks and make informed decisions about the apps integrated into your environment.
Finally, the logic around the risk level is based on the hard work from both:
Tony Redmond: https://office365itpros.com/2024/02/05/Export-MsIdAppConsentGrantReport/
Merril Fernando: https://github.com/AzureAD/MSIdentityTools/tree/main
