Looks like MS has finally resolved the problems by releasing some fix. We will continue to monitor the services.

Issue type: Incident

Status

Service Degradation

Impacted services

Microsoft 365 suite, Exchange Online, SharePoint Online, Microsoft Teams, Microsoft Power Automate in Microsoft 365, Universal Print, Microsoft Purview

Details

Title: Users may be unable to access or use some Microsoft 365 services and features

User impact: Users may be unable to access or use some Microsoft 365 services and features.

More info: The affected scenarios are as follows:

Users may be unable to access Exchange Online using the following impacted connection methods:

- Outlook on the web

- Outlook desktop client

- Representational State Transfer (REST)

- Exchange ActiveSync (EAS)

Users may be unable to use the following features within Microsoft Teams:

- Users are unable to create or update Virtual Events, including webinars and Town Halls.

- Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings.

- Users are unable to create chat, add users and create or edited meetings.

- Users are unable to create or modify new teams and channels.

- Users may be unable to update presence.

- Users may be unable to use the search function.

- Users may not see updated list of files and links failing to load within the Chat shared tab.

Users may experience the following issues with Microsoft Purview:

- Users may be unable to access the Purview Portal, or Purview Solutions.

- Users may experience delays in policy stamping and with Adaptive Scope Evaluations.

Users may be unable to export content or set and view labels within Microsoft Fabric.

Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied.

Users may be unable to use the search feature within SharePoint Online.

Users may be unable to perform the following actions within Microsoft Defender for Office365.

- Users may be unable to create simulations, simulation payloads or end user notifications.

- Users may experience issues with delivery for end user notifications and simulation messages

- Users may experiences issues with viewing simulation reports, and content.

Users may also be unable to print via Universal Print.

Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop.

Current status: We’ve identified a recent change which we believe has resulted in impact. We’ve begun to revert the change and are investigating what additional actions are required to mitigate the issue.

Scope of impact: Impact is specific to users who are served through the affected infrastructure.

Start time: Sunday, November 24, 2024, at 1:20 AM UTC

Preliminary root cause: A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact.

Next update by: Monday, November 25, 2024, at 2:00 PM UTC

Microsoft is having issues with multiple O365 services including email worldwide.
​
From MS:
​

We have released the hotfix to correct the sorting error. We also decided to align sorting in all tabs with the same priority as we had in present/approve tab. Priority is as follows:
​
Priority 1: Shift status (if shift/period status different than new)
Priority 2: Matching percentage
​
On equal number of shift statuses between two candidates (ex two candidates accepted 2 shifts of 5 total), they are also sorted by priority 2; matching percentage.

We are testing the fix and will release it as a hotfix as soon as possible

After a recent technical upgrade, we are currently experiencing issues with sorting candidates based on shift status in "Approve/Present to customer" and "Register" tabs. We are working on resolving the problem ASAP.

All links seem to finally have been migrated to the new domain. Get in touch if you still have trouble with links in Globus.

We are patiently waiting and have reached out again to our contacts in Recman.

We are still waiting for information from Recman on when the links issue is resolved for all environments. We will continue to update this page.

Recman has acknowledged the issue with the links, and they have informed us that they will release a fix shortly. We will update the page when the problem is resolved.

We were recently notified that candidate and customer links from Globus to Recman were not working. We contacted Recman, who informed us that they had changed their URL from "app.recman.no" to "app.recman.io" for certain links. We made an update yesterday to resolve the issue and update all links in Globus to the new domain. But today we found that the links in Recman are not consistent across all Recman environments. Some still have the old domain ".no" while some have the new ".io". This is causing issues for users on the old domain. We are in contact with Recman to try to figure out why the links are not consistent for all. We will update this page when we have further updates.

The missing role issue is resolved. The fix is working and full ETL run is completed for all tenants. Recman have confirmed that this was a bug in their API after a release they did on 6. July.

UPDATE: The fix has just passed code review and has been merged to Production. We will run full ETL for all the tenants. It should be fully completed within approximately 1h 45mins.

UPDATE: Preliminary exploration suggest that this error is due to RecMan unexpectedly changing their API contract. We will keep investigating and fixing the issue.

We are experiencing a problem where candidates are missing roles on some tenants. We are working on correcting this error with highest priority.

The connection issues have been resolved. We are still closely monitoring the application. As part of the fix we found it was most likely related to some undetected change in Chromium, which is the foundation for both Chrome and Edge browsers. For now, we have resolved the issue with a temporary workaround as we found the problem was related to request cancellation, meaning when you click something in the application it requests the related data, if you click something else before the application finishes loading the data from the first click, we cancel the request. These cancellations caused connection errors in the browser. We could not replicate these issues outside of the browser so it seems it is at the network layer of the browser.
​
We will continue to work on this to ensure request cancellations are managed correctly and not cause further problems.

It seems like fixes released during the weekend did reduce the occurrence of the connection errors. User reports and logs show improved performance and reduced connection loss issues. Yesterday we released another fix at 13.30 CET related to request cancellations that seem to have resolved most of the connection issues. We will continue to monitor the performance and please get in contact with us if you are experiencing issues.

During the weekend we released several fixes to mitigate the connection issues in Globus. Unfortunately, these fixes did not resolve the connection issues. The biggest challenge has been consistently reproducing the error to find the root cause of the problems, as we see that the issues stem from the browser/network layer. We are working with the entire product team and are now working with two main hypotheses to resolve this. We will release multiple updates during the day.
​
We again apologize for the inconvenience and are very sorry for those affected by the interruption in our services!