What happened

On March 24th from 7:20PM to 8:20PM, Integrates became inaccessible due to a synchronous data migration of users and roles into a new database table.

What we’ve done

After we detected the error, we immediately reverted that change, deploying a new version of Integrates without the bug. The next day we deployed an asynchronous version of the original change successfully into production

What’s the impact

Approximately 0.8% of our users were affected within the mentioned hour.

What we are doing to help

We are always monitoring our deployments to production, and in this case, we were able to detect the problem and to recover from it in an hour.

Architectural changes are always challenging and even more when they are related to your system’s access control. However, you can have confidence that we are doing our best in doing it as seamlessly as possible