All email notification systems are back up and operational for all customers.
Emails that were scheduled prior to the fix may have been lost as part of the service restart, however all emails and notifications moving forward appear to be functioning properly.
We will continue to monitor internally but are considering this incident resolved. If you continue to experience any issues with emails, please be sure to contact support@chilipiper.com for further assistance.
Posted Jan 31, 2023 - 21:46 UTC
Monitoring
A fix has been implemented for the majority of customers and we are already seeing improvement in services. We will continue to monitor to ensure this is resolved and are working to push the fix out to all customers now.
Until the fix is fully deployed, some customers may still experience issues with email notifications. We will provide an update here when all systems are operational.
Posted Jan 31, 2023 - 20:56 UTC
Identified
Our team has identified the probable cause for the email notification issue and are working to deploy a hotfix.
Once we have rolled out a fix, we will monitor to ensure proper resolution of the issue.
Posted Jan 31, 2023 - 19:15 UTC
Investigating
We are currently investigating an influx of reports that some emails are not being sent or received.
Not all customers are impacted, and even impacted customers appear to have most emails sent correctly.
The scope of the issue impacts reminder emails, booking confirmation emails, and other alerting emails for admins. These are both internal and public-facing. While we work to resolve this issue you may notice some emails are not received by yourself, your team, or your customers.
Posted Jan 31, 2023 - 18:38 UTC
This incident affected: Booking - expand for components (Email notifications & alerting).