[incident] 2026-07-29 10:55 AM - 12:54 PM - Cloud Portal and Connectivity Issues

Incident Summary

Date/Time : Jul 29, 202610:55 - 12:54 AM PST

Service Impacted: Cloud Authentication
Severity: High
Total Duration: 1 hr 59 min

Customer Impact:

Cloud Portal does not show the UI
Users cannot connect to their Sites
Already connected users were partially affected (disconnected from their Desktop / Mobile Clients).

Jul 29, 2026

10:55 AM - First occasional occurrence
11:06 AM - The performance severely degraded
11:08 AM - The faulty modules were automatically restarted and the functionality (potentially) restored
11:20 AM - New complaints received
11:21 AM - Investigation started
11:22 AM - Automatic restart did not help - severity raised
11:30 AM - Investigation in progress. Manual restart
11:45 AM - Services restart completed. Verifying the functionality
11:48 AM - Cloud Portal UI partially restored
11:50 AM - Cloud connectivity partially restored
11:51 AM - Confirming Cloud stability
11:57 AM - Cloud is still unstable. Investigation continues
12:03 PM - Issue is still occurring. Potential root cause identified.
12:20 PM - Root cause suspected to be a routine proactive security fix that rolled out to production unexpectedly. Preparing the rollback
12:35 PM - Partial Rollback is in progress
12:50 PM - Rollback is complete. Services are stabilizing. Testing is in progress
12:54 PM - Cloud Portal is restored, Cloud connectivity is restored, but takes longer than usual. Testing is still in progress
1:04 PM - All functionality confirmed to be working

The Post Mortem is published here.