Myphoner - Notice history

We are back online after rolling back to the last stable release. We identified a flaw in our deploy procedure that is being fixed.

A failed web instance resulted in routing timeouts for a few minutes. The failed web instance has been replaced and all systems are operational.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue was caused by an uncontrolled bouncing of bounces of emails sent to the myphoner dropbox. We saw a drastical rise in email activity (+50K emails in a few hours) and an automated fraud prevention mechanism kicked in and blocked all email processing. Aside from a temporary fix on our side, we have now identified and implemented a permanent fix to the myphoner dropbox processing algorithms. We are now awaiting the security team with our email service provider to verify and re-enable our account. This could take several hours still. We are truly sorry about the inconvenience and doing our very best to get the issues resolved asap. The reason the security measures here is so rigid is because it's very important to take possible email fraud seriously, as it could affect the service far out in the future if real fraud was going on undetected for a longer period of time. In other words, we're not taking any chances here.

We are still awaiting our email service provider to resolve an issue on their side for us. We are unfortunately anticipating this could take several hours.

We are waiting for our email service provider to resolve an issue on their side.

The issue has been identified and we are working with our email service vendor to resolve it.

All systems are operational again.

The application was down for a short period with extremely long response times. We have scaled the server capacity and the app has returned to normal. We are monitoring as well as investigating the root cause of this incident.

Background queue sizes are back to normal. We will be examining the import job that caused these delays and make sure a similar situation will be prevented in the future. We're sorry about the inconvenience.

Queues of delayed background jobs are being processed now and we are monitoring the situation. Everything should return to normal and imports should be quick again once all jobs has processed within the next 30 minutes or so.

An exceptionally large, empty import (more than 1 million rows) have cheated the "too large import" detection and put a halt to other imports. We have removed this particular import job from the system and are monitoring systems as delayed imports are being processed now.