What happened

On 2020, from March 26th at 2:00PM to April 2 at 6:47PM (a total of 52 office hours), Integrates suffered of performance issues due to bad table design for the authorization model, enabled after three commits: b44cfa0, 79c48ba, and 6419323:

What we’ve done

We designed a more lightweight database adapter 2, and introduced a cached authorization model in b14adb0 1 and e5d8445 1. Finally, we introduced the best possible table design in a826210, recovering the expected performance levels:

What’s the impact

During 1 week, our users experienced slowness while navigating the application.

What we are doing to prevent this

We added 1200 mock users to the development database in order to early test the application performance.
Also, there is in course a big migration that puts our back-end to work in an asynchronous fashion which highly boosts those operations that have to do with simultaneous database access and retrieval.
We are also always monitoring production systems through a suite of different services, so we can redirect the development towards more performant solutions.