We have had to take Xero offline due to an issue within the application hosting infrastructure.
We have identified the issue and our team is working on resolving it. We’ll provide an update shortly on when we expect to have Xero back online.
Our apologies for any inconvenience this outage is causing.
(UPDATE) We are still working on fixing the issue, but expect to be back online within the next 30 minutes.
(UPDATE) We have encountered some further delays in bringing the system back online, which we expect will take around another 30 minutes to resolve. At this stage we now expect to be back online by 8:45pm (GMT) or 8:45am (NZT).
(UPDATE) The system is back up now and we have the team closely monitoring the system. Apologies for the inconvenience this outage caused.
(UPDATE) At 6:00am NZT (7:00pm GMT) one of our database servers failed and cut over automatically to its standby server. The standby server failed to start due to issues with one of the database files. The Xero operations team and Rackspace, our hosting provider, were forced to recreate the damaged file in order to recover the database server. There was no loss of data as a result of this issue however the application was offline for approximately two and one half hours. We have a number of levels of redundancy within our hosting environment that typically means our system is not impacted by infrastructure issues. Unfortunately in this situation there was no way for the automated recovery to resolve the issue. We are currently planning an upgrade to the latest version of the database platform that supports more advanced resilience, which should further reduce the risk of a similar outage.