Skip to content

Unexpected Outage (resolved)

We are currently offline due to an issue within our application hosting infrastructure. We have identified the cause of the issue and are working to resolve this.

We will provide any updates, including expected time for the service to be restored, as soon as we have more information.

UPDATE  (Wed 2nd Nov, 12:24pm GMT) – We have the service back online as of about about 15 minutes ago. Continuing to monitor everything closely, but not expecting any further issues. Will provide further information once our internal debrief is complete.

UPDATE  (Thurs 3rd Nov, 3:24am GMT) – A quick update with some further details on this outage. The initial cause was an issue within our application database server cluster, which triggered an automatic failover to the back-up database node. Due to a secondary issue this failover was unable to complete successfully, causing the outage. Resolution involved some storage disk configuration changes, which made up the majority of the time involved.  We are still completing our post incident review, but we have resolved the secondary issue with the automated database failover.

We apologise for the inconvenience caused to any customers by this outage. We continue to make a very significant investment in our hosting platform and resulting service availability level, which we have maintained at 99.99% since we launched Xero in 2007. It frustrates us if we have any unscheduled downtime, but we work hard to learn whatever we can from these incidents and use this to further improve and strengthen our platform going forward.

 

Read more about Company News

 

9 comments

Sarah Bradley
3 November 2011 #

As far as the timing of outages go – this one rocks! It coincided with Elevenses for me ;)

It is back up now – well done and THANK YOU!

Jo Wouters
3 November 2011 #

OK – Xero is okay for us now (13:20 CET – Brussels Time)

Barry
3 November 2011 #

Does Alastair have a time machine – predicting what will happen tomorrow!

Barry
3 November 2011 #

I see it is your automatic dating that is adrift

Michelle
3 November 2011 #

Still having problems here….12:54 GMT

Potential User
3 November 2011 #

How are we posting these comments in the future? Even the article above was resolved before it was posted! :-)

Howard Cheetham
3 November 2011 #

Thank you for your speedy response in solving the outage this AM.

Alastair Grigg
3 November 2011 #

@Barry @Potential User – I don’t, but time machine would be really handy especially in situations like this! The blog post is headlined with date/time of Xero’s Head Office, which being in Wellington, NZ, is currently GMT +13hrs. Hence the post appeared to be a day ahead for many of you. Conscious of this I gave the post update with GMT time noted.

Scott
3 November 2011 #

A positive spin here; what internal network boasts a 99.9% uptime,….the benefits of the cloud…

Add your comment





We welcome all feedback but prefer a real name and email address.