All data that was delayed has been processed and full service has been restored.
Posted Sep 28, 2020 - 13:53 PDT
Monitoring
We have identified other delays that resulted from the earlier outage. Many features in the app were experiencing delays, such as cloning pages. Fixes for all delays and missing data have been implemented and full service will be restored shortly.
Posted Sep 28, 2020 - 12:21 PDT
Identified
Between Sep 28, 2:40 pm UTC – Sep 28, 4:30 pm UTC, both the Kajabi admin and customer facing sites experienced elevated error rates and approximately 0.3% of web requests failed during that time.
The outage was caused by a database migration that failed. The migration was immediately rolled back, but queries against the impacted tables continued to run and time out. This resulted in many different errors, from customers attempting to access their products to admins attempting to delete email broadcasts.
All errors related to the failed migration have been resolved. There will be a short delay before email delivery events are caught up, in the meantime, email broadcast reports, email sequence reports, and contact filtering based on email events will be out of date.
Posted Sep 28, 2020 - 10:03 PDT
Update
We have identified a possible cause. Many admin requests will continue having issues for a short time, but a fix is being worked on.
Posted Sep 28, 2020 - 09:25 PDT
Investigating
We are aware that some users are currently unable to complete certain actions across the application and we are actively investigating the issue. We will update this page as new information becomes available.