wbs.ac.uk & my.wbs.ac.uk are offline
Incident Report for Warwick Business School
Postmortem

Update on the outage to my.wbs and www.wbs on Wednesday 13th March.

What Happened?

On 13th March 2024, we experienced unexpected service interruptions affecting our main website (www.wbs.ac.uk) and the MyWBS platform. These disruptions were caused by unanticipated issues during essential upgrades to our IT infrastructure, specifically the replacement of a critical power supply unit (Uninterruptible Power Supply - UPS) that is designed to ensure our services remain online without interruption.

  • The main www.wbs website was unavailable from 16:09 to 16:30 (21 minutes).
  • The MyWBS platform was unavailable from 16:09 to 18:32 (2 hours and 23 minutes).

Why Did This Happen?

The service interruptions were part of a broader effort to enhance the resilience and reliability of our IT systems. Unfortunately, during the preparatory work for replacing a UPS, we encountered unforeseen technical issues that led to these temporary outages.

Impact on You

During the outage:

  • Visitors to the main website received an error message.
  • MyWBS users were redirected to the mini-my.wbs.ac.uk site, ensuring continued access to WBSLive sessions but without access to resources or functionality such as the assignment submission system.

What We've Done

We quickly identified and resolved the technical issues, restoring full access to both services as swiftly as possible. Our team worked late into the evening to ensure all systems were operational and secure.

Learning and Improvements

This experience has provided valuable lessons on enhancing our infrastructure's resilience and our response to unexpected challenges. We are taking several steps to prevent such interruptions in the future, including:

  • Upgrading our server room facilities to allow safer and quicker maintenance work
  • Updating our standards so that the lessons learned are applied to future server room designs
  • Refining our project planning and execution processes to reduce the risk of unforeseen issues
  • Improving communication strategies. In particular we noted two issues:

    • Without access to my.wbs our ability to communicate with users is harmed. For this reason we run an external site where you can see the status of our systems anytime - https://status.wbs.ac.uk . We encourage all users to visit that site at least once and to click the ‘Subscribe to Updates’ button towards the top right. Enter in your email address and you will then automatically receive emails as we post updates to any future incidents.

    • During outages to the www.wbs site visitors received an unhelpful error page. We will work to enhance this so that during such times visitors receive an improved experience.

Our Commitment to You

We understand the importance of uninterrupted access to our digital services for your academic and administrative needs. Please accept our sincerest apologies for the inconvenience these interruptions caused. We are committed to continuously improving our IT infrastructure to serve you better and to keeping you better informed, be that during maintenance work or other incidents..

For any concerns or further information, please do not hesitate to contact us via help@wbs.ac.uk

Posted Mar 22, 2024 - 16:50 GMT

Resolved
This incident has been resolved.
Posted Mar 13, 2024 - 21:22 GMT
Monitoring
We believe mywbs to be back now, will continue to monitor the datastores.
Posted Mar 13, 2024 - 18:35 GMT
Update
Storage migration is taking slightly longer than expected for mywbs
Posted Mar 13, 2024 - 16:55 GMT
Identified
During a migration of a section of our storage enviroment we encountered some issues and some services have been affected (mywbs / wbs.ac.uk & printing), we are currently migrating those systems again and they will be back online shortly.
Posted Mar 13, 2024 - 16:25 GMT
This incident affected: my.wbs (Site availability) and www.wbs : Site availability check.