Myphoner - Notice history

The incident was caused by an upgrade of the software version of our main database, which unexpectedly behaved slightly differently on planning queries. The solution was to remove an index that "confused" the query planner to select a more "expensive" query in the newer version of the database software. Once identified, the fix was easy, but we struggled to understand what was going on and it took a significant amount of research and experiments to make sure that we understood the problem correctly and that the fix was a proper one. We apologize to everyone affected by the degraded performance during this incident.

The implemented solution seems to have worked, and response times are back to normal. The incident was caused by an upgrade of the software version of our main database, which unexpectedly behaved slightly differently on planning queries. The solution was to remove an index that "confused" the query planner to select a more "expensive" query in the newer version of the database software. Once identified, the fix was easy, but we struggled to understand what was going on and it took a significant amount of research and experiments to make sure that we understood the problem correctly and that the fix was a proper one. We apologize to everyone affected by the degraded performance for the past 36 hours.

We have adjusted database indexes after having analyzed the situation and conducted experiments in a safe environment, and we are now monitoring the results.

We are performing emergency maintenance on the main database to repair an index that is not performing as expected. Expect long response times for 5-10 minutes. Some outages may occur.

This incident has been resolved.

On October 11 2023, Myphoner Voice experienced a downtime incident affecting call functionality for about 60 minutes due to an issue with some of our media switches across several regions. **Root Cause:** The problem originated from a central log service responsible for collecting network latency metrics from the media servers. The unresponsive state of this service led to the media server agents spawning additional instances, exhausting server memory resources, and necessitating a reboot. **Mitigation Measures:** We have conducted a thorough analysis and implemented necessary system adjustments to prevent such incidents in the future. Our actions are aimed at ensuring optimal media server operations and swift anomaly detection and rectification. We apologize to the customers affected and appreciate your understanding in situations like this. We are available to address any concerns or inquiries you may have regarding this incident. Thank you for your understanding. Best, Jeppe Liisberg Founder Myphoner

This incident has been resolved.

All systems are now operational and we are continuing to monitor the situation.

We are still seeing sporadic call failures and investigating.

All servers are now operational. We continue to monitor the situation.

We are still seeing some issues for our European and Australian customers. The root cause has been identified and we are working on restoring services now.

A fix has been implemented and we are monitoring the results.

Two of our servers processing VoIP have become unresponsive. We are rebooting and investigating the root cause.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.