Investigating reports of connectivity issues across some accounts
Incident Report for monday.com
Postmortem

On August 23rd, our customers experienced platform disruption that affected customers reliant on our US servers and their ability to fully access the platform.

At 12:15 ET we investigated reports of connectivity issues affecting some users.

These issues were due to a problematic configuration that caused functionality issues in one of our clusters of servers. Our automatic failover system did not function as expected, resulting in this process needing to be done manually by our team.

By 12:34 ET the issue had been identified and our team remedied the connectivity issues by gradually releasing traffic to our other two clusters of servers.

At 13:15 ET the fix for the issue had been completed, and our team continued to monitor the issue as full connectivity was restored to all users.

There was no security risk and no data loss as a result of this incident.

As part of our efforts to prevent this issue from occurring again, we are conducting a full retrospective internally together with our cloud provider.

We have already identified the cause of the connectivity issues and we are reviewing our process including the way that we test configurations.

We know that our customers rely on the platform for their business workflows, and we sincerely apologize for any disruption caused by this incident.

We take such incidents very seriously and we'll implement the learnings from this experience to ensure we're providing the best service possible, with minimal disruption to your business.

Our dedicated support team is available 24/7 to assist on any issues or queries you have, and can be reached via our Help Center (https://monday.com/helpcenter/contact-support)

Posted Aug 24, 2023 - 10:33 UTC

Resolved
This incident has been resolved.
Posted Aug 23, 2023 - 17:42 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Aug 23, 2023 - 17:15 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Aug 23, 2023 - 16:34 UTC
Investigating
At the moment, we're investigating reports of connectivity issues affecting some users in the platform. Our developers are looking into this, and we'll get back to you shortly with an update on this.
Posted Aug 23, 2023 - 16:15 UTC
This incident affected: US (Platform).